From fddc92d0c44f25d1d556a3d8d782295dfce9ddb7 Mon Sep 17 00:00:00 2001 From: bertrandfalguiere Date: Sat, 3 Aug 2019 23:10:38 +0200 Subject: [PATCH 01/16] Create DHT.md Initial content for Distributed Hash Tables --- content/guides/concepts/dht | 74 +++++++++++++++++++++++++++++++++++++ 1 file changed, 74 insertions(+) create mode 100644 content/guides/concepts/dht diff --git a/content/guides/concepts/dht b/content/guides/concepts/dht new file mode 100644 index 0000000..1f97e86 --- /dev/null +++ b/content/guides/concepts/dht @@ -0,0 +1,74 @@ +--- +title: "Distributed Hash Table (DHT)" +menu: + guides: + parent: concepts +--- + +## DHT + +Distributed Hash Tables (DHT) are a distributed key-value store where key are cryptographic hashes. +DHT provides additionnal advantages compared to a basic Key-Value store: +- *scalability* as each node only need to manage a fraction of the Key-value pairs. +- *fault tolerance* via redunduncy, so that lookup are possible even if nodes unexpectedly leave or join the DHT. + +In IPFS, we use a DHT and the key can be: +- Content IDs of the data added to IPFS. A lookup of this value will give the peerIDs of the peers having this content. +- PeerIDs of IPFS (libp2p?) nodes. A lookup will give all the multiaddresses to reach the peer. +Consequently, IPFS's DHT is use for content routing (1st lookup) and for peer routing (2nd lookup). + +DHTs are usually too big to be stored entirely by each peer, so they are distributed: each peer is responsible for a fraction of the DHT (redundantly). +These fractions are called 'buckets', and map to some prefix of the hash. For exemple, a peer can be responsible for lookups for hash beginning by Qme8T33g... +More precisely, a peer maintains the DHT for hashes which share their prefix with its own PeerID (which is also a hash). + +# How it works +When a peer A sends a lookup request to a peer B, B will check if the requested hash falls into its bucket. If it does, it serves the request. If it doesn't, it will forward it to the peer it knows having the 'closer' peerID to the requested hash. +For exemple, if peer A request the hash *QmAAAA*... and peer B's ID is *QmBBBB*..., he migth forward the request to its peer C which Id is *QmAA*CC. +This peer will also either answer the request or forward it. + + + + +Hashes are functions that take some arbitrary input and return a fixed-length value. The particular value depends on the given hash algorithm in use, such as [SHA-1](https://en.wikipedia.org/wiki/SHA-1) (used by Git), [SHA-256](https://en.wikipedia.org/wiki/SHA-2), or [BLAKE2](https://en.wikipedia.org/wiki/BLAKE_(hash_function)#BLAKE2), but a given hash algorithm always returns the same value for a given input. Have a look at the [full list of hash functions](https://en.wikipedia.org/wiki/List_of_hash_functions) for more. + +As an example, the input: + +``` +Hello world +``` + +would be represented by **SHA-1** as: + +``` +0x7B502C3A1F48C8609AE212CDFB639DEE39673F5E +``` + +However, the exact same input generates the following output using **SHA-256**: + +``` +0x64EC88CA00B268E5BA1A35678A1B5316D212F4F366B2477232534A8AECA37F3C +``` + +Notice that the second hash is longer than the first one. This is because SHA-1 creates a 160 bit hash, while SHA-256 creates a 256 bit hash. Also, the prepended `0x` is just an indicator that tells us that the following hash is represented as a base 16 (or hexadecimal) number. + +Hashes can be represented in different bases (`base2`, `base16`, `base32`, etc.). In fact, IPFS makes use of that as part of its [Content Identifiers]({{< relref "cid.md" >}}) and supports mulitiple base representations at the same time, using the [Multibase](https://github.com/multiformats/multibase) protocol. + +For example, the SHA-256 hash of "Hello World" from above can be represented as base 32 as: + +``` +mtwirsqawjuoloq2gvtyug2tc3jbf5htm2zeo4rsknfiv3fdp46a +``` + +## Characteristics of cryptographic hashes + +Cryptographic hashes come with a couple of very important characteristics: + +- **deterministic** - the same input message always returns exactly the same output hash +- **uncorrelated** - a small change in the message should generate a completely different hash +- **unique** - it's infeasible to generate the same hash from two different messages +- **one-way** - it's infeasible to guess or calculate the input message from its hash + +It turns out these features also mean we can use a cryptographic hash to identify any piece of data: the hash is unique to the data we calculated it from and it’s not too long (a hash is a fixed length, so the SHA-256 hash of a 1 Gigabyte video file is still only 32 bytes), so sending it around the network doesn't take up a lot of resources. + +That's critical for a distributed system like IPFS, where we want to be able to store and retrieve data from many places. A computer running IPFS can ask all the peers it's connected to whether they have a file with a particular hash and, if one of them does, they send back the whole file. Without a short, unique identifier like a cryptographic hash, that wouldn't be possible. This technique is called “content addressing” — because the content itself is used to form an address, rather than information about the computer and disk location it's stored at. + From 35cabc6d685d33a11b440019e28bc146aca5e219 Mon Sep 17 00:00:00 2001 From: bertrandfalguiere Date: Mon, 5 Aug 2019 13:57:58 +0200 Subject: [PATCH 02/16] Develop DHT definition --- content/guides/concepts/dht | 50 +++++++++++++++++++++++++++++++------ 1 file changed, 42 insertions(+), 8 deletions(-) diff --git a/content/guides/concepts/dht b/content/guides/concepts/dht index 1f97e86..3109c54 100644 --- a/content/guides/concepts/dht +++ b/content/guides/concepts/dht @@ -8,22 +8,56 @@ menu: ## DHT Distributed Hash Tables (DHT) are a distributed key-value store where key are cryptographic hashes. -DHT provides additionnal advantages compared to a basic Key-Value store: -- *scalability* as each node only need to manage a fraction of the Key-value pairs. + +DHTs are usually too big to be stored entirely by each peer, so they are distributed: each peer is responsible for a fraction of the DHT (redundantly). +These subsets are called 'buckets', and map to some prefix of the hash. For exemple, a peer can be responsible for lookups for hash beginning by Qme8T33g... +More precisely, a peer maintains the DHT for hashes which share their prefix with its own PeerID (which is also a hash). + +For example, the peer with PeerID "ABCDEF12345" can be responsible to maintain mapping for all hashes starting with the prefix "ABCD". +Some hashes that would fall into this bucket would be ABCD38E56, ABCD09CBA or ABCD17ABB. + +The size of the buckets are directly related to the size of the prefix: the longer the prefix, the less hashes each nodes has to manage, the more nodes is needed. + +In most DHTs' implementations, including Kademlia which is used by IPFS, the size of the buckets, or the size of the prefix, is dynamic. +Buckets size growth and prefix size shortens when many nodes leaves the DHT, and vice versa. (or does it depends of the number of records and not the number of nodes? Is it at a bucket level or DHT level?) + +Peers also keep connection to other peers so that they can forward requests if the requested hash is not in their own bucket. +If hashes are of lentgh n, they will keep n lists of peers: +- the first list contains peers which ID have a different 1st bit. +- the second list contains peer which have their first bits indentical to its own, but a different second bit +- ... +- the m-th list contains peer which have their first m-1 bits identical, but a differnt m-th bit +- ... + +The higher m, the harder it is to find peers which have the same ID up to m bits. The lists of "closest" peers typically remains empty. +"Close" here is defined as the XOR distance, so the longer the prefix they share, the closer they are. + +List also have a maximum of entries k, otherwise the first lists would contain half the network, then a fourth, etc. + + + + + +DHT provides additionnal advantages compared to a classic Key-Value store: +- *scalability* as each node only needs to manage a fraction of the Key-Value pairs. - *fault tolerance* via redunduncy, so that lookup are possible even if nodes unexpectedly leave or join the DHT. +- *load balancing* as requests are made to different nodes and no unique nodes process all the requests + +## The DHT of IPFS -In IPFS, we use a DHT and the key can be: +# Use of DHT in IPFS + +In IPFS, keys are not hashes but multihashes: a generalisation of the cryptographic hashes containing information about which hashing function was used, and the length of the hash. + +We use a DHT to lookup two types of objects (both represented by a multihash): - Content IDs of the data added to IPFS. A lookup of this value will give the peerIDs of the peers having this content. -- PeerIDs of IPFS (libp2p?) nodes. A lookup will give all the multiaddresses to reach the peer. +- PeerIDs of IPFS (libp2p?) nodes. A lookup will give all the multiaddresses to reach the peer(s) actually having the content. Consequently, IPFS's DHT is use for content routing (1st lookup) and for peer routing (2nd lookup). -DHTs are usually too big to be stored entirely by each peer, so they are distributed: each peer is responsible for a fraction of the DHT (redundantly). -These fractions are called 'buckets', and map to some prefix of the hash. For exemple, a peer can be responsible for lookups for hash beginning by Qme8T33g... -More precisely, a peer maintains the DHT for hashes which share their prefix with its own PeerID (which is also a hash). # How it works When a peer A sends a lookup request to a peer B, B will check if the requested hash falls into its bucket. If it does, it serves the request. If it doesn't, it will forward it to the peer it knows having the 'closer' peerID to the requested hash. -For exemple, if peer A request the hash *QmAAAA*... and peer B's ID is *QmBBBB*..., he migth forward the request to its peer C which Id is *QmAA*CC. +For exemple, if peer A request the hash *QmAAAA*... and peer B's ID is *QmBBBB*..., it might forward the request to its peer C which Id is *QmAA*CC. This peer will also either answer the request or forward it. From 97e79cdd95927d66308370f9d726e662fc533f6a Mon Sep 17 00:00:00 2001 From: bertrandfalguiere Date: Mon, 5 Aug 2019 14:15:07 +0200 Subject: [PATCH 03/16] Update and rename dht to dht.md --- content/guides/concepts/dht | 108 --------------------------------- content/guides/concepts/dht.md | 53 ++++++++++++++++ 2 files changed, 53 insertions(+), 108 deletions(-) delete mode 100644 content/guides/concepts/dht create mode 100644 content/guides/concepts/dht.md diff --git a/content/guides/concepts/dht b/content/guides/concepts/dht deleted file mode 100644 index 3109c54..0000000 --- a/content/guides/concepts/dht +++ /dev/null @@ -1,108 +0,0 @@ ---- -title: "Distributed Hash Table (DHT)" -menu: - guides: - parent: concepts ---- - -## DHT - -Distributed Hash Tables (DHT) are a distributed key-value store where key are cryptographic hashes. - -DHTs are usually too big to be stored entirely by each peer, so they are distributed: each peer is responsible for a fraction of the DHT (redundantly). -These subsets are called 'buckets', and map to some prefix of the hash. For exemple, a peer can be responsible for lookups for hash beginning by Qme8T33g... -More precisely, a peer maintains the DHT for hashes which share their prefix with its own PeerID (which is also a hash). - -For example, the peer with PeerID "ABCDEF12345" can be responsible to maintain mapping for all hashes starting with the prefix "ABCD". -Some hashes that would fall into this bucket would be ABCD38E56, ABCD09CBA or ABCD17ABB. - -The size of the buckets are directly related to the size of the prefix: the longer the prefix, the less hashes each nodes has to manage, the more nodes is needed. - -In most DHTs' implementations, including Kademlia which is used by IPFS, the size of the buckets, or the size of the prefix, is dynamic. -Buckets size growth and prefix size shortens when many nodes leaves the DHT, and vice versa. (or does it depends of the number of records and not the number of nodes? Is it at a bucket level or DHT level?) - -Peers also keep connection to other peers so that they can forward requests if the requested hash is not in their own bucket. -If hashes are of lentgh n, they will keep n lists of peers: -- the first list contains peers which ID have a different 1st bit. -- the second list contains peer which have their first bits indentical to its own, but a different second bit -- ... -- the m-th list contains peer which have their first m-1 bits identical, but a differnt m-th bit -- ... - -The higher m, the harder it is to find peers which have the same ID up to m bits. The lists of "closest" peers typically remains empty. -"Close" here is defined as the XOR distance, so the longer the prefix they share, the closer they are. - -List also have a maximum of entries k, otherwise the first lists would contain half the network, then a fourth, etc. - - - - - -DHT provides additionnal advantages compared to a classic Key-Value store: -- *scalability* as each node only needs to manage a fraction of the Key-Value pairs. -- *fault tolerance* via redunduncy, so that lookup are possible even if nodes unexpectedly leave or join the DHT. -- *load balancing* as requests are made to different nodes and no unique nodes process all the requests - -## The DHT of IPFS - -# Use of DHT in IPFS - -In IPFS, keys are not hashes but multihashes: a generalisation of the cryptographic hashes containing information about which hashing function was used, and the length of the hash. - -We use a DHT to lookup two types of objects (both represented by a multihash): -- Content IDs of the data added to IPFS. A lookup of this value will give the peerIDs of the peers having this content. -- PeerIDs of IPFS (libp2p?) nodes. A lookup will give all the multiaddresses to reach the peer(s) actually having the content. -Consequently, IPFS's DHT is use for content routing (1st lookup) and for peer routing (2nd lookup). - - -# How it works -When a peer A sends a lookup request to a peer B, B will check if the requested hash falls into its bucket. If it does, it serves the request. If it doesn't, it will forward it to the peer it knows having the 'closer' peerID to the requested hash. -For exemple, if peer A request the hash *QmAAAA*... and peer B's ID is *QmBBBB*..., it might forward the request to its peer C which Id is *QmAA*CC. -This peer will also either answer the request or forward it. - - - - -Hashes are functions that take some arbitrary input and return a fixed-length value. The particular value depends on the given hash algorithm in use, such as [SHA-1](https://en.wikipedia.org/wiki/SHA-1) (used by Git), [SHA-256](https://en.wikipedia.org/wiki/SHA-2), or [BLAKE2](https://en.wikipedia.org/wiki/BLAKE_(hash_function)#BLAKE2), but a given hash algorithm always returns the same value for a given input. Have a look at the [full list of hash functions](https://en.wikipedia.org/wiki/List_of_hash_functions) for more. - -As an example, the input: - -``` -Hello world -``` - -would be represented by **SHA-1** as: - -``` -0x7B502C3A1F48C8609AE212CDFB639DEE39673F5E -``` - -However, the exact same input generates the following output using **SHA-256**: - -``` -0x64EC88CA00B268E5BA1A35678A1B5316D212F4F366B2477232534A8AECA37F3C -``` - -Notice that the second hash is longer than the first one. This is because SHA-1 creates a 160 bit hash, while SHA-256 creates a 256 bit hash. Also, the prepended `0x` is just an indicator that tells us that the following hash is represented as a base 16 (or hexadecimal) number. - -Hashes can be represented in different bases (`base2`, `base16`, `base32`, etc.). In fact, IPFS makes use of that as part of its [Content Identifiers]({{< relref "cid.md" >}}) and supports mulitiple base representations at the same time, using the [Multibase](https://github.com/multiformats/multibase) protocol. - -For example, the SHA-256 hash of "Hello World" from above can be represented as base 32 as: - -``` -mtwirsqawjuoloq2gvtyug2tc3jbf5htm2zeo4rsknfiv3fdp46a -``` - -## Characteristics of cryptographic hashes - -Cryptographic hashes come with a couple of very important characteristics: - -- **deterministic** - the same input message always returns exactly the same output hash -- **uncorrelated** - a small change in the message should generate a completely different hash -- **unique** - it's infeasible to generate the same hash from two different messages -- **one-way** - it's infeasible to guess or calculate the input message from its hash - -It turns out these features also mean we can use a cryptographic hash to identify any piece of data: the hash is unique to the data we calculated it from and it’s not too long (a hash is a fixed length, so the SHA-256 hash of a 1 Gigabyte video file is still only 32 bytes), so sending it around the network doesn't take up a lot of resources. - -That's critical for a distributed system like IPFS, where we want to be able to store and retrieve data from many places. A computer running IPFS can ask all the peers it's connected to whether they have a file with a particular hash and, if one of them does, they send back the whole file. Without a short, unique identifier like a cryptographic hash, that wouldn't be possible. This technique is called “content addressing” — because the content itself is used to form an address, rather than information about the computer and disk location it's stored at. - diff --git a/content/guides/concepts/dht.md b/content/guides/concepts/dht.md new file mode 100644 index 0000000..b329403 --- /dev/null +++ b/content/guides/concepts/dht.md @@ -0,0 +1,53 @@ +--- +title: "Distributed Hash Table (DHT)" +menu: + guides: + parent: concepts +--- + +## DHT + +Distributed Hash Tables (DHT) are a distributed key-value store where key are cryptographic hashes. + +DHTs are usually too big to be stored entirely by each peer, so they are distributed: each "peer" (or "node") is responsible for a fraction of the DHT (redundantly). +These subsets are called 'buckets', and map to some prefix of the hash. For exemple, a peer can be responsible for lookups for hash beginning by Qme8T33g... +More precisely, a peer maintains the DHT for hashes which share their prefix with its own PeerID (which is also a hash). + +For example, the peer with PeerID "ABCDEF12345" can be responsible to maintain mapping for all hashes starting with the prefix "ABCD". +Some hashes that would fall into this bucket would be ABCD38E56, ABCD09CBA or ABCD17ABB. + +The size of the buckets are directly related to the size of the prefix: the longer the prefix, the less hashes each nodes has to manage, the more nodes is needed. +Several nodes can be in charge of the same bucket if they have the same prefix. + +In most DHTs' implementations, including Kademlia which is used by IPFS, the size of the buckets, or the size of the prefix, is dynamic. +Buckets size growth and prefix size shortens when many nodes leaves the DHT, and vice versa. (or does it depends of the number of records and not the number of nodes? Is it at a bucket level or DHT level?) + +Peers also keep connection to other peers so that they can forward requests if the requested hash is not in their own bucket. +If hashes are of lentgh n, they will keep n lists of peers: +- the first list contains peers which ID have a different 1st bit. +- the second list contains peer which have their first bits indentical to its own, but a different second bit +- ... +- the m-th list contains peer which have their first m-1 bits identical, but a differnt m-th bit +- ... + +The higher m, the harder it is to find peers which have the same ID up to m bits. The lists of "closest" peers typically remains empty. +"Close" here is defined as the XOR distance, so the longer the prefix they share, the closer they are. +List also have a maximum of entries k, otherwise the first lists would contain half the network, then a fourth, etc. + +When a peer receives a lookup request, it will either answer with a value if it falls into its own bucket, or answer with the address of peer ID the closest to the requested hash. +If peer lists are well-filled, requests are forward to peers closer and closer to the requested hash, until a peer is finally able to answer it. +A request on a hash of length n will take log2(n) steps as a maximum. + +DHT's decentralisation provides additionnal advantages compared to a classic Key-Value store: +- *scalability* as each node only needs to maintain mapping for a fraction of the Key-Value pairs. +- *fault tolerance* via redunduncy, so that lookup are possible even if peers unexpectedly leave or join the DHT. +- *load balancing* as requests are made to different nodes and no unique peers process all the requests. Additionaly, any request can be addressed to any peer. + +## The DHT of IPFS + +In IPFS Kademlia's DHT, keys are not hashes but multihashes: a generalisation of the cryptographic hashes containing also information about which hashing function was used, and the length of the hash. + +We use a DHT to lookup two types of objects (both represented by a multihash): +- Content IDs of the data added to IPFS. A lookup of this value will give the peerIDs of the peers having this content. +- PeerIDs of IPFS (libp2p?) nodes. A lookup will give all the multiaddresses to reach the peer(s) actually having the content. +Consequently, IPFS's DHT is use for content routing (1st lookup) and for peer routing (2nd lookup). From a5bd799642946dd4b5fe624956e31363006273fb Mon Sep 17 00:00:00 2001 From: bertrandfalguiere Date: Mon, 5 Aug 2019 14:50:46 +0200 Subject: [PATCH 04/16] Gramar and simplification of dht.md --- content/guides/concepts/dht.md | 39 +++++++++++++++++++--------------- 1 file changed, 22 insertions(+), 17 deletions(-) diff --git a/content/guides/concepts/dht.md b/content/guides/concepts/dht.md index b329403..73f230e 100644 --- a/content/guides/concepts/dht.md +++ b/content/guides/concepts/dht.md @@ -7,23 +7,26 @@ menu: ## DHT -Distributed Hash Tables (DHT) are a distributed key-value store where key are cryptographic hashes. +# What is a DHT? -DHTs are usually too big to be stored entirely by each peer, so they are distributed: each "peer" (or "node") is responsible for a fraction of the DHT (redundantly). -These subsets are called 'buckets', and map to some prefix of the hash. For exemple, a peer can be responsible for lookups for hash beginning by Qme8T33g... -More precisely, a peer maintains the DHT for hashes which share their prefix with its own PeerID (which is also a hash). +Distributed Hash Tables (DHT) are a distributed key-value store where keys are cryptographic hashes. -For example, the peer with PeerID "ABCDEF12345" can be responsible to maintain mapping for all hashes starting with the prefix "ABCD". -Some hashes that would fall into this bucket would be ABCD38E56, ABCD09CBA or ABCD17ABB. +DHTs are distributed. Each "peer" (or "node") is responsible for a subset of the DHT. +These subsets are called 'buckets', and map to some prefix of the hash. +More precisely, a peer maintains the DHT for hashes sharing their prefix with its own PeerID (which is also a hash) up to a length of m bits. If n is the length of the hashes in bits, each bucket maps for 2^(n-m) hashes. -The size of the buckets are directly related to the size of the prefix: the longer the prefix, the less hashes each nodes has to manage, the more nodes is needed. -Several nodes can be in charge of the same bucket if they have the same prefix. +For example, if m = 2^4 and an hexadecimal encoding, the peer with PeerID "ABCDEF12345" can maintain mapping for hashes starting with "ABCD". +Some hashes falling into this bucket would be *ABCD*38E56, *ABCD*09CBA or *ABCD*17ABB ... -In most DHTs' implementations, including Kademlia which is used by IPFS, the size of the buckets, or the size of the prefix, is dynamic. -Buckets size growth and prefix size shortens when many nodes leaves the DHT, and vice versa. (or does it depends of the number of records and not the number of nodes? Is it at a bucket level or DHT level?) +The size of the buckets are related to the size of the prefix. The longer the prefix, the less hashes each peer has to manage, the more peers are needed. +Several peers can be in charge of the same bucket if they have the same prefix. -Peers also keep connection to other peers so that they can forward requests if the requested hash is not in their own bucket. -If hashes are of lentgh n, they will keep n lists of peers: +In most DHTs, including IPFS's Kademlia implementation, the size of the buckets (and the size of the prefix), are dynamic. +Buckets size growths and prefix size shortens when many nodes leaves the DHT, and vice versa. (/!\ or does it depends on the number of records and not the number of nodes? Is it at bucket level or DHT level?) + +Peers also keep connection to other peers to forward requests if the requested hash is not in their own bucket. + +If hashes are of lentgh n, a peer will keep n-1 lists of peers: - the first list contains peers which ID have a different 1st bit. - the second list contains peer which have their first bits indentical to its own, but a different second bit - ... @@ -32,13 +35,13 @@ If hashes are of lentgh n, they will keep n lists of peers: The higher m, the harder it is to find peers which have the same ID up to m bits. The lists of "closest" peers typically remains empty. "Close" here is defined as the XOR distance, so the longer the prefix they share, the closer they are. -List also have a maximum of entries k, otherwise the first lists would contain half the network, then a fourth, etc. +Lists also have a maximum of entries k, otherwise the first lists would contain half the network, then a fourth, etc. -When a peer receives a lookup request, it will either answer with a value if it falls into its own bucket, or answer with the address of peer ID the closest to the requested hash. -If peer lists are well-filled, requests are forward to peers closer and closer to the requested hash, until a peer is finally able to answer it. -A request on a hash of length n will take log2(n) steps as a maximum. +When a peer receives a lookup request, it will either answer with a value if it falls into its own bucket, or forward it to the closest peer it knows from the requested hash. +the process goes on until a peer is able to answer it. +A request for a hash of length n will take at maximum log2(n) steps. -DHT's decentralisation provides additionnal advantages compared to a classic Key-Value store: +DHT's decentralisation provides advantages compared to a classic Key-Value store: - *scalability* as each node only needs to maintain mapping for a fraction of the Key-Value pairs. - *fault tolerance* via redunduncy, so that lookup are possible even if peers unexpectedly leave or join the DHT. - *load balancing* as requests are made to different nodes and no unique peers process all the requests. Additionaly, any request can be addressed to any peer. @@ -51,3 +54,5 @@ We use a DHT to lookup two types of objects (both represented by a multihash): - Content IDs of the data added to IPFS. A lookup of this value will give the peerIDs of the peers having this content. - PeerIDs of IPFS (libp2p?) nodes. A lookup will give all the multiaddresses to reach the peer(s) actually having the content. Consequently, IPFS's DHT is use for content routing (1st lookup) and for peer routing (2nd lookup). + +(what is m and k for IPFS? Does it depends on specs? implementation? ) From 8fe5257bb25b5bb61a25b63213564463e6851427 Mon Sep 17 00:00:00 2001 From: bertrandfalguiere Date: Mon, 5 Aug 2019 15:19:13 +0200 Subject: [PATCH 05/16] Adding structure in dht.md --- content/guides/concepts/dht.md | 45 +++++++++++++++++++++------------- 1 file changed, 28 insertions(+), 17 deletions(-) diff --git a/content/guides/concepts/dht.md b/content/guides/concepts/dht.md index 73f230e..0be6487 100644 --- a/content/guides/concepts/dht.md +++ b/content/guides/concepts/dht.md @@ -5,17 +5,30 @@ menu: parent: concepts --- -## DHT +# DHTs -# What is a DHT? +## What is a DHT? Distributed Hash Tables (DHT) are a distributed key-value store where keys are cryptographic hashes. DHTs are distributed. Each "peer" (or "node") is responsible for a subset of the DHT. -These subsets are called 'buckets', and map to some prefix of the hash. -More precisely, a peer maintains the DHT for hashes sharing their prefix with its own PeerID (which is also a hash) up to a length of m bits. If n is the length of the hashes in bits, each bucket maps for 2^(n-m) hashes. +A receiving peer either answers the request, or forward it until another peer can answer it. -For example, if m = 2^4 and an hexadecimal encoding, the peer with PeerID "ABCDEF12345" can maintain mapping for hashes starting with "ABCD". +DHT's decentralization provides advantages compared to a classic Key-Value store: +- *scalability* as a request for a hash of length *n* takes at most *log2(n)* steps to resolve. +- *fault tolerance* via redundancy, so that lookups are possible even if peers unexpectedly leave or join the DHT. Additionally, requests can be addressed to any peer if one is slow or unavailable. +- *load balancing* as requests are made to different nodes and no unique peers process all the requests. + +## How do DHTs work? + +### Peer IDs +Each peer have a peerID which is a hash with the same length *n* as the DHT keys. + +### Buckets +A subset of the DHT maintained by a peer is called a 'bucket'. +A bucket maps to hashes with the same prefix as the PeerID up to *m* bits. There are 2^m buckets. Each bucket maps for 2^(n-m) hashes. + +- For example, if m = 2^16 and with an hexadecimal encoding (4 bits per displayed character), the peer with PeerID "ABCDEF12345" maintains mapping for hashes starting with "ABCD". Some hashes falling into this bucket would be *ABCD*38E56, *ABCD*09CBA or *ABCD*17ABB ... The size of the buckets are related to the size of the prefix. The longer the prefix, the less hashes each peer has to manage, the more peers are needed. @@ -24,31 +37,29 @@ Several peers can be in charge of the same bucket if they have the same prefix. In most DHTs, including IPFS's Kademlia implementation, the size of the buckets (and the size of the prefix), are dynamic. Buckets size growths and prefix size shortens when many nodes leaves the DHT, and vice versa. (/!\ or does it depends on the number of records and not the number of nodes? Is it at bucket level or DHT level?) +### Peer lists + Peers also keep connection to other peers to forward requests if the requested hash is not in their own bucket. -If hashes are of lentgh n, a peer will keep n-1 lists of peers: +If hashes are of length n, a peer will keep n-1 lists of peers: - the first list contains peers which ID have a different 1st bit. -- the second list contains peer which have their first bits indentical to its own, but a different second bit +- the second list contains peer which have their first bits identical to its own, but a different second bit - ... -- the m-th list contains peer which have their first m-1 bits identical, but a differnt m-th bit +- the m-th list contains peer which have their first m-1 bits identical, but a different m-th bit - ... The higher m, the harder it is to find peers which have the same ID up to m bits. The lists of "closest" peers typically remains empty. "Close" here is defined as the XOR distance, so the longer the prefix they share, the closer they are. Lists also have a maximum of entries k, otherwise the first lists would contain half the network, then a fourth, etc. -When a peer receives a lookup request, it will either answer with a value if it falls into its own bucket, or forward it to the closest peer it knows from the requested hash. -the process goes on until a peer is able to answer it. -A request for a hash of length n will take at maximum log2(n) steps. +### Using the DHT -DHT's decentralisation provides advantages compared to a classic Key-Value store: -- *scalability* as each node only needs to maintain mapping for a fraction of the Key-Value pairs. -- *fault tolerance* via redunduncy, so that lookup are possible even if peers unexpectedly leave or join the DHT. -- *load balancing* as requests are made to different nodes and no unique peers process all the requests. Additionaly, any request can be addressed to any peer. +When a peer receives a lookup request, it will either answer with a value if it falls into its own bucket, or forward it to the closest peer it knows from the requested hash. The process goes on until a peer is able to answer it. (Does is it answer directly to requesting peer? Or does the answer takes the same path as the request? ) +A request for a hash of length n will take at maximum log2(n) steps. -## The DHT of IPFS +# The DHT of IPFS -In IPFS Kademlia's DHT, keys are not hashes but multihashes: a generalisation of the cryptographic hashes containing also information about which hashing function was used, and the length of the hash. +In IPFS Kademlia's DHT, keys are not hashes but multihashes: a generalization of the cryptographic hashes containing also information about which hashing function was used, and the length of the hash in bits. We use a DHT to lookup two types of objects (both represented by a multihash): - Content IDs of the data added to IPFS. A lookup of this value will give the peerIDs of the peers having this content. From 6e508cfbbfa454e0261b7b59a564ce009e123578 Mon Sep 17 00:00:00 2001 From: bertrandfalguiere Date: Mon, 5 Aug 2019 15:38:12 +0200 Subject: [PATCH 06/16] Adding links --- content/guides/concepts/dht.md | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/content/guides/concepts/dht.md b/content/guides/concepts/dht.md index 0be6487..780dc36 100644 --- a/content/guides/concepts/dht.md +++ b/content/guides/concepts/dht.md @@ -9,7 +9,7 @@ menu: ## What is a DHT? -Distributed Hash Tables (DHT) are a distributed key-value store where keys are cryptographic hashes. +[Distributed Hash Tables](https://en.wikipedia.org/wiki/Distributed_hash_table) (DHT) are a distributed key-value store where keys are [cryptographic hashes](https://docs.ipfs.io/guides/concepts/hashes/). DHTs are distributed. Each "peer" (or "node") is responsible for a subset of the DHT. A receiving peer either answers the request, or forward it until another peer can answer it. @@ -35,7 +35,7 @@ The size of the buckets are related to the size of the prefix. The longer the pr Several peers can be in charge of the same bucket if they have the same prefix. In most DHTs, including IPFS's Kademlia implementation, the size of the buckets (and the size of the prefix), are dynamic. -Buckets size growths and prefix size shortens when many nodes leaves the DHT, and vice versa. (/!\ or does it depends on the number of records and not the number of nodes? Is it at bucket level or DHT level?) +Buckets size growths and prefix size shortens when many nodes leaves the DHT, and vice versa. **(/!\ or does it depends on the number of records and not the number of nodes? Is it at bucket level or DHT level?)** ### Peer lists @@ -54,16 +54,17 @@ Lists also have a maximum of entries k, otherwise the first lists would contain ### Using the DHT -When a peer receives a lookup request, it will either answer with a value if it falls into its own bucket, or forward it to the closest peer it knows from the requested hash. The process goes on until a peer is able to answer it. (Does is it answer directly to requesting peer? Or does the answer takes the same path as the request? ) +When a peer receives a lookup request, it will either answer with a value if it falls into its own bucket, or forward it to the closest peer it knows from the requested hash. The process goes on until a peer is able to answer it. **(Does is it answer directly to requesting peer? Or does the answer takes the same path as the request?)** A request for a hash of length n will take at maximum log2(n) steps. # The DHT of IPFS -In IPFS Kademlia's DHT, keys are not hashes but multihashes: a generalization of the cryptographic hashes containing also information about which hashing function was used, and the length of the hash in bits. +In IPFS Kademlia's DHT, keys are not hashes but [multihashes](https://multiformats.io/multihash/): a generalization of the cryptographic hashes containing also information about which hashing function was used, and the length of the hash in bits. +[PeerIDs](https://docs.libp2p.io/concepts/peer-id/) are those of [libp2p]([PeerIDs](https://docs.libp2p.io/concepts/peer-id/), the networking library used by IPFS. We use a DHT to lookup two types of objects (both represented by a multihash): -- Content IDs of the data added to IPFS. A lookup of this value will give the peerIDs of the peers having this content. -- PeerIDs of IPFS (libp2p?) nodes. A lookup will give all the multiaddresses to reach the peer(s) actually having the content. -Consequently, IPFS's DHT is use for content routing (1st lookup) and for peer routing (2nd lookup). +- [Content IDs](https://docs.ipfs.io/guides/concepts/cid/) of the data added to IPFS. A lookup of this value will give the peerIDs of the peers having this content. +- PeerIDs. A lookup will give all the [multiaddresses](https://multiformats.io/multiaddr/) to reach the peer(s) actually having the content. +Consequently, IPFS's DHT is used for content routing (1st lookup) and for peer routing (2nd lookup). -(what is m and k for IPFS? Does it depends on specs? implementation? ) +**(what is m and k for IPFS? Does it depends on specs? implementation? )** From 698e391a47ad733aa97309b2a1ca7673223cc2fd Mon Sep 17 00:00:00 2001 From: bertrandfalguiere Date: Mon, 5 Aug 2019 15:40:24 +0200 Subject: [PATCH 07/16] Add a question: multiaddress or multiaddrs? --- content/guides/concepts/dht.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/guides/concepts/dht.md b/content/guides/concepts/dht.md index 780dc36..5f9c5dc 100644 --- a/content/guides/concepts/dht.md +++ b/content/guides/concepts/dht.md @@ -64,7 +64,7 @@ In IPFS Kademlia's DHT, keys are not hashes but [multihashes](https://multiforma We use a DHT to lookup two types of objects (both represented by a multihash): - [Content IDs](https://docs.ipfs.io/guides/concepts/cid/) of the data added to IPFS. A lookup of this value will give the peerIDs of the peers having this content. -- PeerIDs. A lookup will give all the [multiaddresses](https://multiformats.io/multiaddr/) to reach the peer(s) actually having the content. +- PeerIDs. A lookup will give all the [multiaddresses](https://multiformats.io/multiaddr/) **(or "multiaddrs"?)** to reach the peer(s) actually having the content. Consequently, IPFS's DHT is used for content routing (1st lookup) and for peer routing (2nd lookup). **(what is m and k for IPFS? Does it depends on specs? implementation? )** From 7f922bba0815679097b5f7f57afe337af8550d4c Mon Sep 17 00:00:00 2001 From: bertrandfalguiere Date: Mon, 5 Aug 2019 16:52:20 +0200 Subject: [PATCH 08/16] Add link to libp2p.io/implementation/peer-routing --- content/guides/concepts/dht.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/content/guides/concepts/dht.md b/content/guides/concepts/dht.md index 5f9c5dc..09f2ddb 100644 --- a/content/guides/concepts/dht.md +++ b/content/guides/concepts/dht.md @@ -67,4 +67,6 @@ We use a DHT to lookup two types of objects (both represented by a multihash): - PeerIDs. A lookup will give all the [multiaddresses](https://multiformats.io/multiaddr/) **(or "multiaddrs"?)** to reach the peer(s) actually having the content. Consequently, IPFS's DHT is used for content routing (1st lookup) and for peer routing (2nd lookup). +Implementation status can be checked [here](https://libp2p.io/implementations/#peer-routing). + **(what is m and k for IPFS? Does it depends on specs? implementation? )** From dc3fafc3e9dc18227e3057bf7bb977eccb415443 Mon Sep 17 00:00:00 2001 From: bertrandfalguiere Date: Tue, 6 Aug 2019 11:19:30 +0200 Subject: [PATCH 09/16] mutiaddresses are multiaddrs. Add forward strats --- content/guides/concepts/dht.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/content/guides/concepts/dht.md b/content/guides/concepts/dht.md index 09f2ddb..9a59c2e 100644 --- a/content/guides/concepts/dht.md +++ b/content/guides/concepts/dht.md @@ -12,7 +12,11 @@ menu: [Distributed Hash Tables](https://en.wikipedia.org/wiki/Distributed_hash_table) (DHT) are a distributed key-value store where keys are [cryptographic hashes](https://docs.ipfs.io/guides/concepts/hashes/). DHTs are distributed. Each "peer" (or "node") is responsible for a subset of the DHT. -A receiving peer either answers the request, or forward it until another peer can answer it. +A receiving peer either answers the request, or the request is passed to another peer until a peer can answer it. +Depending on implementations, a request not answered by the first contacted node can be : +- forwarded from peer to peer and the last peer contact the requesting peer +- forwarded from peer to peer and the answered is forwarded following the same path +- answered with contacting information of a node having better chances to be able to answer (IPFS uses this strategy) DHT's decentralization provides advantages compared to a classic Key-Value store: - *scalability* as a request for a hash of length *n* takes at most *log2(n)* steps to resolve. @@ -64,7 +68,7 @@ In IPFS Kademlia's DHT, keys are not hashes but [multihashes](https://multiforma We use a DHT to lookup two types of objects (both represented by a multihash): - [Content IDs](https://docs.ipfs.io/guides/concepts/cid/) of the data added to IPFS. A lookup of this value will give the peerIDs of the peers having this content. -- PeerIDs. A lookup will give all the [multiaddresses](https://multiformats.io/multiaddr/) **(or "multiaddrs"?)** to reach the peer(s) actually having the content. +- PeerIDs. A lookup will give all the [multiadds](https://multiformats.io/multiaddr/) to reach the peer(s) actually having the content. Consequently, IPFS's DHT is used for content routing (1st lookup) and for peer routing (2nd lookup). Implementation status can be checked [here](https://libp2p.io/implementations/#peer-routing). From 0ba0567c868d0cc599bc13d290ac053594613635 Mon Sep 17 00:00:00 2001 From: bertrandfalguiere Date: Tue, 6 Aug 2019 11:36:01 +0200 Subject: [PATCH 10/16] Correct forwarding info & IPFS DHT content DHT doesn't forward requests: it return a peer ID of a closer peer. DHT doesn't store mapping of peerID to multiaddrs. It doesn't do peer routing. but it stores IPNS records. --- content/guides/concepts/dht.md | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/content/guides/concepts/dht.md b/content/guides/concepts/dht.md index 9a59c2e..3d70179 100644 --- a/content/guides/concepts/dht.md +++ b/content/guides/concepts/dht.md @@ -58,8 +58,8 @@ Lists also have a maximum of entries k, otherwise the first lists would contain ### Using the DHT -When a peer receives a lookup request, it will either answer with a value if it falls into its own bucket, or forward it to the closest peer it knows from the requested hash. The process goes on until a peer is able to answer it. **(Does is it answer directly to requesting peer? Or does the answer takes the same path as the request?)** -A request for a hash of length n will take at maximum log2(n) steps. +When a peer receives a lookup request, it will either answer with a value if it falls into its own bucket, or answer with the contacting information (IP + port, peerID, etc) of a closer peer. The requesting peer can then send its request to this closer peer. The process goes on until a peer is able to answer it. +A request for a hash of length n will take at maximum log2(n) steps, or even log2m(n). # The DHT of IPFS @@ -67,10 +67,9 @@ In IPFS Kademlia's DHT, keys are not hashes but [multihashes](https://multiforma [PeerIDs](https://docs.libp2p.io/concepts/peer-id/) are those of [libp2p]([PeerIDs](https://docs.libp2p.io/concepts/peer-id/), the networking library used by IPFS. We use a DHT to lookup two types of objects (both represented by a multihash): -- [Content IDs](https://docs.ipfs.io/guides/concepts/cid/) of the data added to IPFS. A lookup of this value will give the peerIDs of the peers having this content. -- PeerIDs. A lookup will give all the [multiadds](https://multiformats.io/multiaddr/) to reach the peer(s) actually having the content. -Consequently, IPFS's DHT is used for content routing (1st lookup) and for peer routing (2nd lookup). +- [Content IDs](https://docs.ipfs.io/guides/concepts/cid/) of the data added to IPFS. A lookup of this value will give the peerIDs of the peers having this immutable content +- [IPNS records](https://docs.ipfs.io/guides/concepts/ipns/). A lookup will give the last Content ID associated with this IPNS address, enabling routing mutable content -Implementation status can be checked [here](https://libp2p.io/implementations/#peer-routing). +Consequently, IPFS's DHT is one of the way of mutable and immutable [Content Routing]. It's currently the only one [implemented](https://libp2p.io/implementations/#peer-routing). **(what is m and k for IPFS? Does it depends on specs? implementation? )** From 43ba4086c0ec6e4cc1ee254aafa707c86a76bcd4 Mon Sep 17 00:00:00 2001 From: bertrandfalguiere Date: Tue, 6 Aug 2019 11:51:09 +0200 Subject: [PATCH 11/16] fix a broken link --- content/guides/concepts/dht.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/guides/concepts/dht.md b/content/guides/concepts/dht.md index 3d70179..c6a8dfa 100644 --- a/content/guides/concepts/dht.md +++ b/content/guides/concepts/dht.md @@ -64,7 +64,7 @@ A request for a hash of length n will take at maximum log2(n) steps, or even log # The DHT of IPFS In IPFS Kademlia's DHT, keys are not hashes but [multihashes](https://multiformats.io/multihash/): a generalization of the cryptographic hashes containing also information about which hashing function was used, and the length of the hash in bits. -[PeerIDs](https://docs.libp2p.io/concepts/peer-id/) are those of [libp2p]([PeerIDs](https://docs.libp2p.io/concepts/peer-id/), the networking library used by IPFS. +[PeerIDs](https://docs.libp2p.io/concepts/peer-id/) are those of [libp2p](https://docs.libp2p.io/concepts/peer-id/), the networking library used by IPFS. We use a DHT to lookup two types of objects (both represented by a multihash): - [Content IDs](https://docs.ipfs.io/guides/concepts/cid/) of the data added to IPFS. A lookup of this value will give the peerIDs of the peers having this immutable content From d4e5643cfc3c2f0d3dd2b5081755a0079c82b7bc Mon Sep 17 00:00:00 2001 From: bertrandfalguiere Date: Tue, 6 Aug 2019 11:56:44 +0200 Subject: [PATCH 12/16] fix links --- content/guides/concepts/dht.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/guides/concepts/dht.md b/content/guides/concepts/dht.md index c6a8dfa..7266fbd 100644 --- a/content/guides/concepts/dht.md +++ b/content/guides/concepts/dht.md @@ -64,12 +64,12 @@ A request for a hash of length n will take at maximum log2(n) steps, or even log # The DHT of IPFS In IPFS Kademlia's DHT, keys are not hashes but [multihashes](https://multiformats.io/multihash/): a generalization of the cryptographic hashes containing also information about which hashing function was used, and the length of the hash in bits. -[PeerIDs](https://docs.libp2p.io/concepts/peer-id/) are those of [libp2p](https://docs.libp2p.io/concepts/peer-id/), the networking library used by IPFS. +[PeerIDs](https://docs.libp2p.io/concepts/peer-id/) are those of [libp2p](https://libp2p.io/), the networking library used by IPFS. We use a DHT to lookup two types of objects (both represented by a multihash): - [Content IDs](https://docs.ipfs.io/guides/concepts/cid/) of the data added to IPFS. A lookup of this value will give the peerIDs of the peers having this immutable content - [IPNS records](https://docs.ipfs.io/guides/concepts/ipns/). A lookup will give the last Content ID associated with this IPNS address, enabling routing mutable content -Consequently, IPFS's DHT is one of the way of mutable and immutable [Content Routing]. It's currently the only one [implemented](https://libp2p.io/implementations/#peer-routing). +Consequently, IPFS's DHT is one of the way of mutable and immutable [Content Routing](https://docs.libp2p.io/concepts/content-routing/). It's currently the only one [implemented](https://libp2p.io/implementations/#peer-routing). **(what is m and k for IPFS? Does it depends on specs? implementation? )** From 0a59cce077221d08bde64003211e7dd65dfa862a Mon Sep 17 00:00:00 2001 From: bertrandfalguiere Date: Tue, 6 Aug 2019 16:48:16 +0200 Subject: [PATCH 13/16] Entries are SHA256, not multihash Or are they? --- content/guides/concepts/dht.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/content/guides/concepts/dht.md b/content/guides/concepts/dht.md index 7266fbd..613e38f 100644 --- a/content/guides/concepts/dht.md +++ b/content/guides/concepts/dht.md @@ -63,13 +63,14 @@ A request for a hash of length n will take at maximum log2(n) steps, or even log # The DHT of IPFS -In IPFS Kademlia's DHT, keys are not hashes but [multihashes](https://multiformats.io/multihash/): a generalization of the cryptographic hashes containing also information about which hashing function was used, and the length of the hash in bits. +In IPFS Kademlia's DHT, keys are SHA256 hashes. [PeerIDs](https://docs.libp2p.io/concepts/peer-id/) are those of [libp2p](https://libp2p.io/), the networking library used by IPFS. -We use a DHT to lookup two types of objects (both represented by a multihash): +We use a DHT to lookup two types of objects (both represented by a SHA256): - [Content IDs](https://docs.ipfs.io/guides/concepts/cid/) of the data added to IPFS. A lookup of this value will give the peerIDs of the peers having this immutable content - [IPNS records](https://docs.ipfs.io/guides/concepts/ipns/). A lookup will give the last Content ID associated with this IPNS address, enabling routing mutable content Consequently, IPFS's DHT is one of the way of mutable and immutable [Content Routing](https://docs.libp2p.io/concepts/content-routing/). It's currently the only one [implemented](https://libp2p.io/implementations/#peer-routing). +Per specification, the default bucket size k is 20 : each of the 255 lists of peers contain at most 20 peers. **(what is m and k for IPFS? Does it depends on specs? implementation? )** From 07588f549fcb260a18fac22e3b351c14e169575f Mon Sep 17 00:00:00 2001 From: bertrandfalguiere Date: Sun, 11 Aug 2019 21:34:24 +0200 Subject: [PATCH 14/16] Add adding an entry to DHT --- content/guides/concepts/dht.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/content/guides/concepts/dht.md b/content/guides/concepts/dht.md index 613e38f..f0dc333 100644 --- a/content/guides/concepts/dht.md +++ b/content/guides/concepts/dht.md @@ -73,4 +73,15 @@ We use a DHT to lookup two types of objects (both represented by a SHA256): Consequently, IPFS's DHT is one of the way of mutable and immutable [Content Routing](https://docs.libp2p.io/concepts/content-routing/). It's currently the only one [implemented](https://libp2p.io/implementations/#peer-routing). Per specification, the default bucket size k is 20 : each of the 255 lists of peers contain at most 20 peers. + +## Usage + +### Adding an entry + +Adding a blob of data to IPFS is just advertizing that you have it. Since DHT is the only content routing implemented, you can just use +` ipfs add myData` +IPFS will automatically chunk your data and add a mapping on the DHT between the Content ID and your PeerID. Note that there can be other Peer IDs already mapped to that value, so you will be added to the list. Also note that if the provided data is bigger than 124kb, it will be chunked and both chunks and overall blob of data will be mapped. + + + **(what is m and k for IPFS? Does it depends on specs? implementation? )** From 672204b30ccf8ac7c4f915b8891c4c012b9e23ac Mon Sep 17 00:00:00 2001 From: bertrandfalguiere Date: Sun, 11 Aug 2019 21:39:38 +0200 Subject: [PATCH 15/16] rename chunks ->blocks --- content/guides/concepts/dht.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/guides/concepts/dht.md b/content/guides/concepts/dht.md index f0dc333..941dc57 100644 --- a/content/guides/concepts/dht.md +++ b/content/guides/concepts/dht.md @@ -80,7 +80,7 @@ Per specification, the default bucket size k is 20 : each of the 255 lists of pe Adding a blob of data to IPFS is just advertizing that you have it. Since DHT is the only content routing implemented, you can just use ` ipfs add myData` -IPFS will automatically chunk your data and add a mapping on the DHT between the Content ID and your PeerID. Note that there can be other Peer IDs already mapped to that value, so you will be added to the list. Also note that if the provided data is bigger than 124kb, it will be chunked and both chunks and overall blob of data will be mapped. +IPFS will automatically chunk your data and add a mapping on the DHT between the Content ID and your PeerID. Note that there can be other Peer IDs already mapped to that value, so you will be added to the list. Also note that if the provided data is bigger than 124kb, it will be chunked in "blocks" and both blocks and overall data will be mapped. From d6d20b64c2cf64a935c1f7270b5b53e2776715db Mon Sep 17 00:00:00 2001 From: bertrandfalguiere Date: Sun, 11 Aug 2019 21:47:35 +0200 Subject: [PATCH 16/16] link to IPNS --- content/guides/concepts/dht.md | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/content/guides/concepts/dht.md b/content/guides/concepts/dht.md index 941dc57..90f0370 100644 --- a/content/guides/concepts/dht.md +++ b/content/guides/concepts/dht.md @@ -72,7 +72,7 @@ We use a DHT to lookup two types of objects (both represented by a SHA256): Consequently, IPFS's DHT is one of the way of mutable and immutable [Content Routing](https://docs.libp2p.io/concepts/content-routing/). It's currently the only one [implemented](https://libp2p.io/implementations/#peer-routing). -Per specification, the default bucket size k is 20 : each of the 255 lists of peers contain at most 20 peers. +Per specification, the default bucket size k is 20 meaning each of the 255 lists of peers contain at most 20 peers. ## Usage @@ -82,6 +82,4 @@ Adding a blob of data to IPFS is just advertizing that you have it. Since DHT is ` ipfs add myData` IPFS will automatically chunk your data and add a mapping on the DHT between the Content ID and your PeerID. Note that there can be other Peer IDs already mapped to that value, so you will be added to the list. Also note that if the provided data is bigger than 124kb, it will be chunked in "blocks" and both blocks and overall data will be mapped. - - -**(what is m and k for IPFS? Does it depends on specs? implementation? )** +You can publish an IPNS record using [`ipfs.name.publish`](https://docs.ipfs.io/guides/concepts/ipns/).