Paper Insights - The Eternal Tussle: Exploring the Role of Centralization in IPFS

By

Paper Link

I'd like to delve into a technology I have significant research experience with: the InterPlanetary File System (IPFS). While the original IPFS white paper - IPFS - Content Addressed, Versioned, P2P File System - was influential, it is not peer reviewed. Therefore, I'll focus on a related paper presented at the 2024 Networked Systems Design and Implementation (NSDI) conference, a prestigious venue in distributed systems research. This paper was also discussed in Stanford's CS244b class.

For background, I recommend reading my "Paper Insights - Dynamo: Amazon's Highly Available Key-value Store", where I discuss consistent hashing and its techniques.

Now, let's explore some fundamental concepts of IPFS.

Decentralized Web

Traditional websites, akin to 1800 numbers, rely solely on the website owner to bear the costs of hosting and computation.

In contrast, decentralized web leverage a distributed network of nodes. This means that the computational burden, and therefore the associated costs, can be shared among numerous participants, regardless of the website's traffic load.

Decentralized web is popularly called Web3.

Decentralized File System

A decentralized file system represents each file block by its cryptographic hash. This hash uniquely identifies the block's content. Subsequently, these hashes are mapped to locations on a decentralized network of servers.

InterPlanetary File System (IPFS)

IPFS is a decentralized storage and distribution system that:

  1. Divides files into blocks.
  2. Generates a unique Content Identifier (CID) for each block using cryptographic hashing.
  3. Stores the mapping of CIDs to their locations on a Distributed Hash Table (DHT).

IPFS DHT Entries

Two key entries within the IPFS DHT are:

  • Provider Records: Map CIDs to the PeerIDs of nodes that possess the corresponding blocks.
  • Peer Records: Map PeerIDs to their respective Multiaddresses. Multiaddresses enable flexible p2p communication across various protocols (e.g., IPv4, IPv6, HTTP).

IPFS Lookup

To retrieve a file:

  1. Local Peer Search: IPFS first attempts to locate the desired CID among connected peers using the BitSwap protocol (an opportunistic p2p exchange protocol).
  2. DHT Lookup: If the CID is not found locally, the DHT is queried to find the PeerIDs associated with the CID.
  3. Peer Discovery: Using the DHT, the Multiaddresses of the identified peers are retrieved.
  4. Data Retrieval: Finally, the BitSwap protocol is employed to request the actual block data from the located peers.

Kademlia Hash

Kademlia is a distributed hash table for decentralized p2p computer networks designed by P Maymounkov and D Mazières in 2002. It was popularized by trackerless bittorrent, now it is used for DHT by IPFS.

Kademlia arranges the nodes into a binary tree where the leaf nodes store

Comments

Post a Comment

Popular posts from this blog

Comments in a Code v/s Code Readability

By
Recently, I earned Readability (a Google certification for language expertise) in Java and JavaScript. Over the past few months, I have been a Readability reviewer for most of the code that is typically checked in by my team to Google's codebase. I am an exhaustive code reviewer. I only accept code that adheres to the highest standards and best practices. (Okay, enough of blowing my own trumpet!) One of the most popular questions that I usually receive is, "Shall I add a comment here to make things clear?" Most of the time, my answer is a clear " No !" I am writing this small write-up to explain the reason. I truly believe that writing code is very similar to writing a very simple story without any twists. The variables are the characters. Expressions provide a characteristic feature to a character. Each function narrates a conversation between variables, much like an act in a play. While reading a novel, the reader's mind tries to guess what could be coming...
Read more

Paper Insights - Cassandra - A Decentralized Structured Storage System

By
Image
Paper Link This research paper, authored by Avinash Lakshman (co-inventor of Amazon Dynamo) and Prashant Malik , originates from Facebook and dates back to 2008. Paper Overview Cassandra, in its design, appears to be a synthesis of Amazon's Dynamo (2007) and Google's Bigtable (2006). It draws heavily upon the concepts of both systems. Notably, this paper was published during the rise of these influential databases. However, Cassandra also introduces novel ideas that warrant further investigation. Recommended Read: Paper Insights - Dynamo: Amazon's Highly Available Key-value Store Let's begin with some of fundamental concepts. SQL Databases SQL databases are a category of databases which are inherently consistency. This implies that data integrity is always upheld. For instance, in a banking database, the cumulative balance across all accounts must remain unchanged at any time regardless of the number of transfer transactions. To ensure this data consistency (the C i...
Read more

Architecture of High Performance Computing Server at BIT Mesra

By
Image
A High-Performance Computing (HPC) server was installed a few years back. It was a replacement for PARAM 10000, the supercomputer that is no longer available for use. Initially, the HPC was under the Department of Computer Science. The Department of Chemical Engineering and Biotechnology was the primary user of the HPC (mostly for simulation purposes), and so the administration decided to move it under the Central Instrumentation Facility (CIF). You need permission from the CIF to access the HPC. HPC is only available for research purposes, and you need to provide a good reason along with a proper recommendation from a professor to gain access to the HPC. The HPC is at least 20 times more powerful than the most powerful PC that anyone has on campus. Also, I recently checked the usage and realized that not even 10% of its power is being utilized. I hope this blog post will help you in understanding the core architecture of the HPC. Architecture The Architecture of High Performance Compu...
Read more