Skip to content

Commit 4494ad0

Browse files
johnnymatthewsjohnnymatthews
authored
Merge pull request #122 from edgo914/master
Updates on Bitswap and libp2p
2 parents 0161a18 + 589dddf commit 4494ad0

File tree

2 files changed

+111
-10
lines changed

2 files changed

+111
-10
lines changed

‎docs/concepts/bitswap.md

Lines changed: 47 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,6 @@
11
---
22
title: Bitswap
33
sidebarDepth: 0
4-
issueUrl: https://github.com/ipfs/docs/issues/85
54
description: Learn about Bitswap and how it plays into the overall architecture of IPFS, the InterPlanetary File System.
65
related:
76
'Article: Swapping bits and distributing hashes on the decentralized web (Textile)': https://medium.com/textileio/swapping-bits-and-distributing-hashes-on-the-decentralized-web-5da98a3507
@@ -11,4 +10,50 @@ related:
1110

1211
# Bitswap
1312

14-
<ContentStatus />
13+
Bitswap is a core module of IPFS for exchanging blocks of data. It directs the requesting and sending of blocks to and from other peers in the network. Bitswap is a _message-based protocol_ where all messages contain [want-lists](#want-list) or blocks. Bitswap has a [Go implementation](https://github.com/ipfs/go-bitswap) and a [JavaScript implementation](https://github.com/ipfs/js-ipfs-bitswap).
14+
15+
Bitswap has two main jobs:
16+
- Acquire blocks requested by the client from the network.
17+
- Send blocks in its possession to other peers who want them.
18+
19+
### Want-list
20+
21+
A _want-list_ is a data structure that an IPFS node sends to the network to tell peers which blocks it wants. It's a list of data that the IPFS node wants. The want-list contains [CIDs](/concepts/content-addressing/) (content-addressed identifiers that refer to particular blocks desired) along with priority and cancellation information.
22+
23+
Here is a simplified example of a want-list:
24+
25+
```sh
26+
Want-list {
27+
QmZtmD2qt6fJot32nabSP3CUjicnypEBz7bHVDhPQt9aAy, WANT,
28+
QmTudJSaoKxtbEnTddJ9vh8hbN84ZLVvD5pNpUaSbxwGoa, WANT,
29+
...
30+
QmPvaEQFVvuiaYzkSVUp23iHTQeEUpDaJnP8U7C3PqE57w, CANCEL
31+
}
32+
```
33+
34+
Take a look at the [Go source code](https://github.com/ipfs/go-bitswap/blob/master/wantlist/wantlist.go) for a deeper dive into how want-lists are structured..
35+
36+
### Message
37+
38+
A single Bitswap message may contain any of the following content:
39+
40+
- **The sender's want-list:** This want-list may either be the sender's complete want-list or just the changes to the sender's want-list that the receiver needs to know.
41+
- **Data blocks:** These are blocks that have been requested (i.e., blocks in the sender's want-list, as far as the receiver is aware at the time of message receipt).
42+
43+
### Session
44+
45+
IPFS content blocks are often connected through a [Merkle DAG](/concepts/merkle-dag/). If a sender's block requests are related, a mechanism called a _Bitswap session_ in the Go implementation can optimize block requests sent to other peers. This mechanism can increase transfer speed and reduce the number of duplicate blocks on the network. Active peers are favored, as they are tracked relative to them having the requested blocks and how quickly they respond.
46+
47+
If a remote peer has one block then it is likely that they have other blocks from the same DAG. So any new block requests are sent to that peer directly. This skips the need to send losts of requsts to multiple servers, saving the user time and power.
48+
49+
### Provider system
50+
51+
The provider system is responsible for announcing and reannouncing to the IPFS network that a node has content. If a node cannot find content with connected peers, it will query a content routing system (usually a form of [DHT](/concepts/dht/)) to locate a peer with the content.
52+
53+
### Additional references
54+
55+
- [February 2020: New improvements to IPFS Bitswap](https://blog.ipfs.io/2020-02-14-improved-bitswap-for-container-distribution/)
56+
- [Technical overview of the Go implementation of Bitswap](https://docs.google.com/presentation/d/1mbFFGIIKNvboHyLn-k26egOSWkt9nXjlNbxpmCEQfqQ/edit#slide=id.p)
57+
- [Article: Swapping bits and distributing hashes on the decentralized web (Textile)](https://medium.com/textileio/swapping-bits-and-distributing-hashes-on-the-decentralized-web-5da98a3507)
58+
- "About Bitswap" Go implementation poster from the IPFS developer summit in Berlin in July 2018:
59+
!['About Bitswap' poster](https://user-images.githubusercontent.com/74178/43230914-f818dab2-901e-11e8-876b-73ba6a084f76.jpg "Bitswap-Poster_Berlin-July-2018")

‎docs/concepts/libp2p.md

Lines changed: 64 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,71 @@
11
---
2-
title: libp2p
2+
title: Libp2p
33
sidebarDepth: 0
44
issueUrl: https://github.com/ipfs/docs/issues/388
5-
description: Learn about the libp2p protocol and why it's an important ingredient in how IPFS works.
5+
description: Learn about the Libp2p protocol and why it's an important ingredient in how IPFS works.
66
related:
7-
'What is libp2p?': https://docs.libp2p.io/introduction/what-is-libp2p/
8-
'Foundational libp2p concepts': https://docs.libp2p.io/concepts/
9-
'Getting started with libp2p': https://docs.libp2p.io/tutorials/getting-started/
10-
'Examples of libp2p key features': https://docs.libp2p.io/examples/
7+
'What is Libp2p?': https://docs.libp2p.io/introduction/what-is-libp2p/
8+
'Foundational Libp2p concepts': https://docs.libp2p.io/concepts/
9+
'Getting started with Libp2p': https://docs.libp2p.io/tutorials/getting-started/
10+
'Examples of Libp2p key features': https://docs.libp2p.io/examples/
1111
---
1212

13-
# The libp2p protocol
13+
# Libp2p
1414

15-
<ContentStatus />
15+
[Libp2p](https://libp2p.io/) is a modular system of _protocols_, _specifications_, and _libraries_ that enable the development of peer-to-peer network applications. Libp2p began as part of the IPFS project and is still an essential component of IPFS. As the network layer for IPFS, Libp2p provides flexible solutions for essential peer-to-peer elements like transport, security, peer routing, and content discovery. Libp2p has implementations in [Go](https://github.com/libp2p/go-libp2p), [JavaScript](https://github.com/libp2p/js-libp2p), [Rust](https://github.com/libp2p/rust-libp2p), [Python](https://github.com/libp2p/py-libp2p), and [C++](https://github.com/soramitsu/libp2p).
16+
17+
## Peer-to-peer network applications
18+
19+
A [peer-to-peer network](https://docs.libp2p.io/reference/glossary/#peer-to-peer-p2p) is one in which the players, known as _"peers"_, communicate with each other directly as equal participants. This is in direct contrast to the traditional [client-server model](https://docs.libp2p.io/reference/glossary/#client-server), where a privileged central server may provide services to many client programs on the network. These client programs usually do not communicate with each other; they communicate only with the central server.
20+
21+
Someone using Libp2p for the network layer of their peer-to-peer application is instantly freed up to focus on their own unique tasks, knowing that Libp2p handles a lot of [tasks in a decentralized system](https://hub.packtpub.com/libp2p-the-modular-p2p-network-stack-by-ipfs-for-better-decentralized-computing/). At the same time, they can customize Libp2p regarding key elements like transport, identity, and security. Some applications using Libp2p are [Filecoin](https://filecoin.io/), [Parity](https://www.parity.io/why-libp2p/), and [OpenBazaar](https://www.openbazaar.org/).
22+
23+
### Features of Libp2p
24+
25+
#### [Addressing](https://docs.libp2p.io/concepts/addressing/)
26+
27+
Libp2p works with a lot of different addressing schemes in a consistent way. A multiaddress (abbreviated [multiaddr](https://github.com/multiformats/multiaddr)) encodes multiple layers of addressing information into a single "future-proof" path structure. For example, `/ipv4/171.113.242.172/udp/162` indicates the use of the IPv4 protocol with the address 171.113.242.172, along with sending UDP packets to port 162.
28+
29+
#### [Transport](https://docs.libp2p.io/concepts/transport/)
30+
31+
The technology used to move your data from one machine to another. Transports are defined in terms of two core operations, _listening_ and _dialing_. Listening means that you can accept incoming connections from other peers. _Dialing_ is the process of opening an outgoing connection to a listening peer. One of Libp2p's core requirements is to be _transport agnostic_, meaning the decision of what transport protocol to use is up to an application's developer (who may decide to support many different _transports_ at the same time).
32+
33+
#### [Security](https://docs.libp2p.io/introduction/what-is-libp2p/#security)
34+
35+
Libp2p supports upgrading a transport connection into a securely encrypted channel. You can then trust the identity of the peer you're communicating with and that no third-party can read the conversation or alter it in-flight. The current default is [secio](https://docs.libp2p.io/concepts/secure-comms/), with support for [TLS 1.3](https://www.ietf.org/blog/tls13/) under development.
36+
37+
#### [Peer identity](https://docs.libp2p.io/concepts/peer-id/)
38+
39+
A Peer Identity ([often written _PeerId_](https://docs.libp2p.io/reference/glossary/#peerid)) is a unique reference to a specific peer on the peer-to-peer network. Each Libp2p peer has a private key which it keeps secret from all other peers and a corresponding public key, which is shared with other peers. The PeerId is a [cryptographic hash](https://en.wikipedia.org/wiki/Cryptographic_hash_function) of a peer's public key. PeerIds are encoded using the [multihash](https://docs.libp2p.io/reference/glossary/#multihash) format.
40+
41+
#### [Peer routing](https://docs.libp2p.io/introduction/what-is-libp2p/#peer-routing)
42+
43+
Peer Routing is the process of discovering peer addresses by using the knowledge of other peers. In a peer routing system, a peer can either give us the address we need if they have it or else send our inquiry to another peer who's more likely to have the answer. Peer Routing in Libp2p uses a [distributed hash table](https://docs.libp2p.io/reference/glossary/#dht) to iteratively route requests closer to the desired PeerId using the [Kademlia](https://en.wikipedia.org/wiki/Kademlia) routing algorithm.
44+
45+
#### [Content discovery](https://docs.libp2p.io/introduction/what-is-libp2p/#content-discovery)
46+
47+
In Content discovery, you ask for some specific piece of data, but you don't care who sends it since you're able to verify its integrity. Libp2p provides a [content routing interface](https://github.com/libp2p/interface-content-routing) for this purpose, with the primary stable implementation using the same [Kademlia](https://en.wikipedia.org/wiki/Kademlia)-based DHT as used in peer routing.
48+
49+
#### [NAT traversal](https://docs.libp2p.io/concepts/nat/)
50+
51+
Network Address Translation (NAT) allows you to move traffic seamlessly between network boundaries. NAT maps an address from one address space to another. While NAT is usually transparent for outgoing connections, listening for incoming connections requires some configuration. Libp2p has the following main approaches to NAT traversal available: [Automatic router configuration](https://docs.libp2p.io/concepts/nat/#automatic-router-configuration), [Hole punching (STUN)](https://docs.libp2p.io/concepts/nat/#hole-punching-stun), [AutoNAT](https://docs.libp2p.io/concepts/nat/#autonat), and [Circuit Relay (TURN)](https://docs.libp2p.io/concepts/nat/#circuit-relay-turn).
52+
53+
#### [Protocol](https://docs.libp2p.io/concepts/protocols/)
54+
55+
These are the protocols built with Libp2p itself, using core Libp2p abstractions like [transport](https://docs.libp2p.io/concepts/transport/), [peer identity](https://docs.libp2p.io/concepts/peer-id/), and [addressing](https://docs.libp2p.io/concepts/addressing/). Each Libp2p protocol has a unique string identifier used in the [protocol negotiation](https://docs.libp2p.io/concepts/protocols/#protocol-negotiation) process when connections are first opened. The core Libp2p protocols are [Ping](https://docs.libp2p.io/concepts/protocols/#ping), [Identify](https://docs.libp2p.io/concepts/protocols/#identify), [secio](https://docs.libp2p.io/concepts/protocols/#secio), [kad-dht](https://docs.libp2p.io/concepts/protocols/#kad-dht), and [Circuit Relay](https://docs.libp2p.io/concepts/protocols/#circuit-relay).
56+
57+
#### [Stream multiplexing](https://docs.libp2p.io/concepts/stream-multiplexing/)
58+
59+
Often abbreviated as _stream muxing_, this allows multiple independent logical streams to all share a common underlying transport medium. Libp2p's stream multiplexer sits "above" the transport stack and allows many streams to flow over a single TCP port or other raw transport connection. The current stream multiplexing implementations are [mplex](https://docs.libp2p.io/concepts/stream-multiplexing/#mplex), [yamux](https://docs.libp2p.io/concepts/stream-multiplexing/#yamux), [quic](https://docs.libp2p.io/concepts/stream-multiplexing/#quic), [spdy](https://docs.libp2p.io/concepts/stream-multiplexing/#spdy), and [muxado](https://docs.libp2p.io/concepts/stream-multiplexing/#muxado).
60+
61+
#### [Publish and subscribe](https://docs.libp2p.io/concepts/publish-subscribe/)
62+
63+
Often abbreviated as _pub-sub_, this is a system where peers congregate around topics they are interested in. Peers interested in a topic are said to be subscribed to that topic. Peers send messages to topics, which get delivered to all peers subscribed to the topic. Example uses of pub-sub are chat rooms and file sharing. For more detail and a discussion of other pub-sub designs, see the [gossipsub specification](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md).
64+
65+
## Additional resources
66+
67+
- [What is Libp2p?](https://docs.libp2p.io/introduction/what-is-libp2p/)
68+
- [Introduction to Libp2p](https://www.youtube.com/embed/CRe_oDtfRLw)
69+
- [Getting started with Libp2p](https://docs.libp2p.io/tutorials/getting-started/)
70+
- [The Libp2p glossary](https://docs.libp2p.io/reference/glossary/)
71+
- [The Libp2p specification](https://github.com/libp2p/specs)

0 commit comments

Comments
 (0)