From a1fe116d3e7f6069145dc16de8b1bef8aa356c45 Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Wed, 31 Aug 2022 00:59:14 -0400 Subject: [PATCH 01/30] init guides section --- content/{examples => guides}/_index.md | 4 +- content/guides/examples/_index.md | 12 + content/guides/examples/getting-started/go.md | 440 ++++++++++++++++++ .../examples/getting-started/javascript.md | 279 +++++++++++ .../guides/examples/getting-started/rust.md | 8 + content/{ => guides}/tutorials/_index.md | 0 .../tutorials/getting-started/_index.md | 0 .../tutorials/getting-started/go.md | 0 .../tutorials/getting-started/javascript.md | 0 .../tutorials/getting-started/rust.md | 0 10 files changed, 741 insertions(+), 2 deletions(-) rename content/{examples => guides}/_index.md (95%) create mode 100644 content/guides/examples/_index.md create mode 100644 content/guides/examples/getting-started/go.md create mode 100644 content/guides/examples/getting-started/javascript.md create mode 100644 content/guides/examples/getting-started/rust.md rename content/{ => guides}/tutorials/_index.md (100%) rename content/{ => guides}/tutorials/getting-started/_index.md (100%) rename content/{ => guides}/tutorials/getting-started/go.md (100%) rename content/{ => guides}/tutorials/getting-started/javascript.md (100%) rename content/{ => guides}/tutorials/getting-started/rust.md (100%) diff --git a/content/examples/_index.md b/content/guides/_index.md similarity index 95% rename from content/examples/_index.md rename to content/guides/_index.md index 388f869d..45c85d6f 100644 --- a/content/examples/_index.md +++ b/content/guides/_index.md @@ -1,6 +1,6 @@ --- -title: Examples -weight: 3 +title: Guides +weight: 4 pre: ' ' --- diff --git a/content/guides/examples/_index.md b/content/guides/examples/_index.md new file mode 100644 index 00000000..5906d11d --- /dev/null +++ b/content/guides/examples/_index.md @@ -0,0 +1,12 @@ +--- +title: Examples +weight: 2 +pre: ' ' +--- + +This tutorial covers setting up your development environment, getting familiar with some libp2p basics, and implementing a super simple node that can send and receive "ping" messages. + + + +{{% children description="true" %}} +----- diff --git a/content/guides/examples/getting-started/go.md b/content/guides/examples/getting-started/go.md new file mode 100644 index 00000000..9ded04fe --- /dev/null +++ b/content/guides/examples/getting-started/go.md @@ -0,0 +1,440 @@ +--- +title: "Getting Started with go-libp2p" +menuTitle: go-libp2p +--- + +This is the first in a series of tutorials on working with libp2p’s Go implementation, +[go-libp2p](https://github.com/libp2p/go-libp2p). + +We’ll cover installing Go, setting up a new Go module, starting libp2p nodes, and sending ping +messages between them. + + + +## Install Go + +go-libp2p recommends the use of a Go version that includes the [modules feature](https://github.com/golang/go/wiki/Modules), +which means you need to have a version of Go that is at least 1.11. + +You can install a recent version of Go by following the [official installation instructions](https://golang.org/doc/install). + +Once installed, you should be able to run `go version` and see a version >= 1.11, for example: + +```sh +$ go version +go version go1.12 darwin/amd64 +``` + +## Create a Go module + +We're going to create a Go module that can be run from the command line. + +Let's create a new directory and use `go mod` to initialize it as a module. We'll create it in +`/tmp`, but you can equally create it anywhere on your filesystem (however it's advisable _not_ to +create this directory under your `GOPATH`). We'll also initialize it with the module name +`github.com/user/go-libp2p-tutorial`, but you may want to replace this with a name that corresponds +to a repository name you have the rights to push to if you want to publish your version of the code. + +```sh +$ mkdir -p /tmp/go-libp2p-tutorial + +$ cd /tmp/go-libp2p-tutorial + +$ go mod init github.com/user/go-libp2p-tutorial +``` + +You should now have a `go.mod` file in the current directory containing the name of the module you +initialized and the version of Go you're using, for example: + +```sh +$ cat go.mod +module github.com/user/go-libp2p-tutorial + +go 1.12 +``` + +## Start a libp2p node + +We'll now add some code to our module to start a libp2p node. + +Let's start by creating a `main.go` file that simply starts a libp2p node with default settings, +prints the node's listening addresses, then shuts the node down: + +```go +package main + +import ( + "fmt" + "github.com/libp2p/go-libp2p" +) + +func main() { + // start a libp2p node with default settings + node, err := libp2p.New() + if err != nil { + panic(err) + } + + // print the node's listening addresses + fmt.Println("Listen addresses:", node.Addrs()) + + // shut the node down + if err := node.Close(); err != nil { + panic(err) + } +} +``` + +Import the `libp2p/go-libp2p` module: + +```shell +$ go get github.com/libp2p/go-libp2p +``` + +We can now compile this into an executable using `go build` and run it from the command line: + +```sh +$ go build -o libp2p-node + +$ ./libp2p-node +Listen addresses: [/ip6/::1/tcp/57666 /ip4/127.0.0.1/tcp/57665 /ip4/192.168.1.56/tcp/57665] +``` + +The listening addresses are formatted using the [multiaddr](https://github.com/multiformats/multiaddr) +format, and there is typically more than one printed because go-libp2p will listen on all available +IPv4 and IPv6 network interfaces by default. + +### Configure the node + +A node's default settings can be overridden by passing extra arguments to `libp2p.New`. Let's use +`libp2p.ListenAddrStrings` to configure the node to listen on TCP port 2000 on the IPv4 loopback +interface: + +```go +func main() { + ... + + // start a libp2p node that listens on TCP port 2000 on the IPv4 + // loopback interface + node, err := libp2p.New( + libp2p.ListenAddrStrings("/ip4/127.0.0.1/tcp/2000"), + ) + if err != nil { + panic(err) + } + + ... +} +``` + +Re-building and running the executable again now prints the explicit listen address we've configured: + +```sh +$ go build -o libp2p-node + +$ ./libp2p-node +Listening addresses: [/ip4/127.0.0.1/tcp/2000] +``` + +`libp2p.New` accepts a variety of arguments to configure most aspects of the node. See +[options.go](https://github.com/libp2p/go-libp2p/blob/master/options.go) for a full list of those +options. + +### Wait for a signal + +A node that immediately exits is not all that useful. Let's add the following towards the end of the +`main` function that blocks waiting for an OS signal before shutting down the node: + +```go +func main() { + ... + + // wait for a SIGINT or SIGTERM signal + ch := make(chan os.Signal, 1) + signal.Notify(ch, syscall.SIGINT, syscall.SIGTERM) + <-ch + fmt.Println("Received signal, shutting down...") + + // shut the node down + if err := node.Close(); err != nil { + panic(err) + } +} +``` + +We also need to update the list of imports at the top of the file to include the `os`, `os/signal` +and `syscall` packages we're now using: + +```go +import ( + "fmt" + "os" + "os/signal" + "syscall" + + "github.com/libp2p/go-libp2p" +) +``` + +Running the node now waits until it receives a SIGINT (i.e. a `ctrl-c` key press) or a SIGTERM signal +before shutting down: + +```sh +$ ./libp2p-node +Listening addresses: [/ip4/127.0.0.1/tcp/2000] +^CReceived signal, shutting down... +``` + +## Run the ping protocol + +Now that we have the ability to configure and start libp2p nodes, we can start communicating! + +### Set a stream handler + +A node started with go-libp2p will run its own ping protocol by default, but let's disable it and +set it up manually to demonstrate the process of running protocols by registering stream handlers. + +The object returned from `libp2p.New` implements the [Host interface](https://pkg.go.dev/github.com/libp2p/go-libp2p-core/host#Host), +and we'll use the `SetStreamHandler` method to set a handler for our ping protocol. + +First, let's add the `github.com/libp2p/go-libp2p/p2p/protocol/ping` package to our list of +imported packages: + +```go +import ( + ... + + "github.com/libp2p/go-libp2p" + "github.com/libp2p/go-libp2p/p2p/protocol/ping" +) +``` + +Now we'll pass an argument to `libp2p.New` to disable the built-in ping protocol, and then use the +`PingService` type from the ping package to set a stream handler manually (note that we're also +configuring the node to listen on a random local TCP port rather than a hard coded one, which means +we'll be able to run multiple nodes on the same machine without them trying to listen on the same +port): + +```go +func main() { + ... + + // start a libp2p node that listens on a random local TCP port, + // but without running the built-in ping protocol + node, err := libp2p.New( + libp2p.ListenAddrStrings("/ip4/127.0.0.1/tcp/0"), + libp2p.Ping(false), + ) + if err != nil { + panic(err) + } + + // configure our own ping protocol + pingService := &ping.PingService{Host: node} + node.SetStreamHandler(ping.ID, pingService.PingHandler) + + ... +} +``` + +### Connect to a peer + +With the ping protocol configured, we need a way to instruct the node to connect to another node and +send it ping messages. + +We'll first expand the log message that we've been printing after starting the node to include +its `PeerId` value, as we'll need that to instruct other nodes to connect to it. Let's import the +`github.com/libp2p/go-libp2p-core/peer` package and use it to replace the "Listen addresses" log +message with something that prints both the listen address and the `PeerId` as a multiaddr string: + +```go +import ( + ... + + "github.com/libp2p/go-libp2p" + peerstore "github.com/libp2p/go-libp2p-core/peer" + "github.com/libp2p/go-libp2p/p2p/protocol/ping" +) + +func main() { + ... + + // print the node's PeerInfo in multiaddr format + peerInfo := peerstore.AddrInfo{ + ID: node.ID(), + Addrs: node.Addrs(), + } + addrs, err := peerstore.AddrInfoToP2pAddrs(&peerInfo) + fmt.Println("libp2p node address:", addrs[0]) + + ... +} +``` + +Running the node now prints the node's address that can be used to connect to it: + +```sh +$ ./libp2p-node +libp2p node address: /ip4/127.0.0.1/tcp/62268/ipfs/QmfQzWnLu4UX1cW7upgyuFLyuBXqze7nrPB4qWYqQiTHwt +``` + +Let's also accept a command line argument that is the address of a peer to send ping messages to, +allowing us to either just run a listening node that waits for a signal, or run a node that connects +to another node and pings it a few times before shutting down (we'll use the `github.com/multiformats/go-multiaddr` +package to parse the peer's address from the command line argument): + +```go +import ( + ... + + "github.com/libp2p/go-libp2p" + peerstore "github.com/libp2p/go-libp2p-core/peer" + "github.com/libp2p/go-libp2p/p2p/protocol/ping" + multiaddr "github.com/multiformats/go-multiaddr" +) + +func main() { + ... + fmt.Println("libp2p node address:", addrs[0]) + + // if a remote peer has been passed on the command line, connect to it + // and send it 5 ping messages, otherwise wait for a signal to stop + if len(os.Args) > 1 { + addr, err := multiaddr.NewMultiaddr(os.Args[1]) + if err != nil { + panic(err) + } + peer, err := peerstore.AddrInfoFromP2pAddr(addr) + if err != nil { + panic(err) + } + if err := node.Connect(context.Background(), *peer); err != nil { + panic(err) + } + fmt.Println("sending 5 ping messages to", addr) + ch := pingService.Ping(context.Background(), peer.ID) + for i := 0; i < 5; i++ { + res := <-ch + fmt.Println("got ping response!", "RTT:", res.RTT) + } + } else { + // wait for a SIGINT or SIGTERM signal + ch := make(chan os.Signal, 1) + signal.Notify(ch, syscall.SIGINT, syscall.SIGTERM) + <-ch + fmt.Println("Received signal, shutting down...") + } + + // shut the node down + if err := node.Close(); err != nil { + panic(err) + } +} +``` + +## Let's play ping pong! + +We are finally in a position to run two libp2p nodes, have one connect to the other and for +them to run a protocol! + +To recap, here is the full program we have written: + +```go +package main + +import ( + "context" + "fmt" + "os" + "os/signal" + "syscall" + + "github.com/libp2p/go-libp2p" + peerstore "github.com/libp2p/go-libp2p-core/peer" + "github.com/libp2p/go-libp2p/p2p/protocol/ping" + multiaddr "github.com/multiformats/go-multiaddr" +) + +func main() { + // start a libp2p node that listens on a random local TCP port, + // but without running the built-in ping protocol + node, err := libp2p.New( + libp2p.ListenAddrStrings("/ip4/127.0.0.1/tcp/0"), + libp2p.Ping(false), + ) + if err != nil { + panic(err) + } + + // configure our own ping protocol + pingService := &ping.PingService{Host: node} + node.SetStreamHandler(ping.ID, pingService.PingHandler) + + // print the node's PeerInfo in multiaddr format + peerInfo := peerstore.AddrInfo{ + ID: node.ID(), + Addrs: node.Addrs(), + } + addrs, err := peerstore.AddrInfoToP2pAddrs(&peerInfo) + if err != nil { + panic(err) + } + fmt.Println("libp2p node address:", addrs[0]) + + // if a remote peer has been passed on the command line, connect to it + // and send it 5 ping messages, otherwise wait for a signal to stop + if len(os.Args) > 1 { + addr, err := multiaddr.NewMultiaddr(os.Args[1]) + if err != nil { + panic(err) + } + peer, err := peerstore.AddrInfoFromP2pAddr(addr) + if err != nil { + panic(err) + } + if err := node.Connect(context.Background(), *peer); err != nil { + panic(err) + } + fmt.Println("sending 5 ping messages to", addr) + ch := pingService.Ping(context.Background(), peer.ID) + for i := 0; i < 5; i++ { + res := <-ch + fmt.Println("pinged", addr, "in", res.RTT) + } + } else { + // wait for a SIGINT or SIGTERM signal + ch := make(chan os.Signal, 1) + signal.Notify(ch, syscall.SIGINT, syscall.SIGTERM) + <-ch + fmt.Println("Received signal, shutting down...") + } + + // shut the node down + if err := node.Close(); err != nil { + panic(err) + } +} +``` + +In one terminal window, let's start a listening node (i.e. don't pass any command line arguments): + +```sh +$ ./libp2p-node +libp2p node address: /ip4/127.0.0.1/tcp/61790/ipfs/QmZKjsGJ6ukXVRXVEcExx9GhiyWoJC97onYpzBwCHPWqpL +``` + +In another terminal window, let's run a second node but pass the address of the first node, and we +should see some ping responses logged: + +```sh +$ ./libp2p-node /ip4/127.0.0.1/tcp/61790/ipfs/QmZKjsGJ6ukXVRXVEcExx9GhiyWoJC97onYpzBwCHPWqpL +libp2p node address: /ip4/127.0.0.1/tcp/61846/ipfs/QmVyKLTLswap3VYbpBATsgNpi6JdwSwsZALPxEnEbEndup +sending 5 ping messages to /ip4/127.0.0.1/tcp/61790/ipfs/QmZKjsGJ6ukXVRXVEcExx9GhiyWoJC97onYpzBwCHPWqpL +pinged /ip4/127.0.0.1/tcp/61790/ipfs/QmZKjsGJ6ukXVRXVEcExx9GhiyWoJC97onYpzBwCHPWqpL in 431.231µs +pinged /ip4/127.0.0.1/tcp/61790/ipfs/QmZKjsGJ6ukXVRXVEcExx9GhiyWoJC97onYpzBwCHPWqpL in 164.94µs +pinged /ip4/127.0.0.1/tcp/61790/ipfs/QmZKjsGJ6ukXVRXVEcExx9GhiyWoJC97onYpzBwCHPWqpL in 220.544µs +pinged /ip4/127.0.0.1/tcp/61790/ipfs/QmZKjsGJ6ukXVRXVEcExx9GhiyWoJC97onYpzBwCHPWqpL in 208.761µs +pinged /ip4/127.0.0.1/tcp/61790/ipfs/QmZKjsGJ6ukXVRXVEcExx9GhiyWoJC97onYpzBwCHPWqpL in 201.37µs +``` + +Success! Our two peers are now communicating using go-libp2p! Sure, they can only say “ping”, but it’s a start! diff --git a/content/guides/examples/getting-started/javascript.md b/content/guides/examples/getting-started/javascript.md new file mode 100644 index 00000000..1ac7d746 --- /dev/null +++ b/content/guides/examples/getting-started/javascript.md @@ -0,0 +1,279 @@ +--- +title: "Getting Started with js-libp2p" +menuTitle: js-libp2p +weight: 2 +--- + +This is the first in a series of tutorials on working with libp2p's javascript implementation, [js-libp2p](https://github.com/libp2p/js-libp2p). + +We will walk you through setting up a fully functional libp2p node with some basic functionality, and finally we'll send ping messages back and forth between two peers. + + + + + +## Install node.js + +Working with js-libp2p requires [node.js](https://nodejs.org) >= v16 for development. If you haven't already, install node using whatever package manager you prefer or [using the official installer](https://nodejs.org/en/download/). + +We recommend using the latest stable version of node, but anything fairly recent should work fine. If you want to see how low you can go, the current version requirements can always be found at the [js-libp2p project page](https://github.com/libp2p/js-libp2p). + +## Create an empty project + +We need a place to put our work, so open a terminal to make a new directory for your project somewhere and set it up as an npm project: + +```shell +# create a directory for the project and `cd` into it +> mkdir hello-libp2p +> mkdir hello-libp2p/src +> cd hello-libp2p + +# make it a git repository +> git init . + +# make it an npm project. fill in the prompts with info for your project +# when asked for your project's entry point, enter "src/index.js" +> npm init +``` + +Side note: throughout this tutorial, we use the `> ` character to indicate your terminal's shell prompt. When following along, don't type the `>` character, or you'll get some weird errors. + +## Configure libp2p + +libp2p is a very modular framework, which allows javascript devs to target different runtime environments and opt-in to various features by including a custom selection of modules. + +Because every application is different, we recommend configuring your libp2p node with just the modules you need. You can even make more than one configuration, if you want to target multiple javascript runtimes with different features. + +{{% notice note %}} +In a production application, it may make sense to create a separate npm module for your libp2p node, which will give you one place to manage the libp2p dependencies for all your javascript projects. In that case, you should not depend on `libp2p` directly in your application. Instead you'd depend on your libp2p configuration module, which would in turn depend on `libp2p` and whatever modules (transports, etc) you might need. +{{% /notice %}} + +If you're new to libp2p, we recommend configuring your node in stages, as this can make troubleshooting configuration issues much easier. In this tutorial, we'll do just that. If you're more experienced with libp2p, you may wish to jump to the [Configuration readme](https://github.com/libp2p/js-libp2p/blob/master/doc/CONFIGURATION.md). + +As an initial step, you should install libp2p module. + +```shell +npm install libp2p +``` + +### Basic setup + +Now that we have libp2p installed, let's configure the minimum needed to get your node running. The only modules libp2p requires are a **Transport** and **Crypto** module. However, we recommend that a basic setup should also have a **Stream Multiplexer** configured, which we will explain shortly. Let's start by setting up a Transport. + +#### Transports + +Libp2p uses Transports to establish connections between peers over the network. You can configure any number of Transports, but you only need 1 to start with. + +You should select Transports according to the runtime of your application; Node.js or the browser. You can see a list of some of the available Transports in the [configuration readme](https://github.com/libp2p/js-libp2p/blob/master/doc/CONFIGURATION.md#transport). For this guide let's install `@libp2p/tcp`. + +```sh +npm install @libp2p/tcp +``` + +Now that we have the module installed, let's configure libp2p to use the Transport. We'll use the `createLibp2pNode` method, which takes a single configuration object as its only parameter. We can add the Transport by passing it into the `transports` array. Create a `src/index.js` file and have the following code in it: + +```js +import { createLibp2p } from 'libp2p' +import { TCP } from '@libp2p/tcp' + +const node = await createLibp2p({ + transports: [new TCP()] +}) + +``` + +You can add as many transports as you like to `transports` in order to establish connections with as many peers as possible. + +#### Connection Encryption + +Every connection must be encrypted to help ensure security for everyone. As such, Connection Encryption (Crypto) is a required component of libp2p. + +There are a growing number of Crypto modules being developed for libp2p. As those are released they will be tracked in the [Connection Encryption section of the configuration readme](https://github.com/libp2p/js-libp2p/blob/master/doc/CONFIGURATION.md#connection-encryption). For now, we are going to configure our node to use the `@chainsafe/libp2p-noise` module. + +```sh +npm install @chainsafe/libp2p-noise +``` + +```js +import { createLibp2p } from 'libp2p' +import { TCP } from '@libp2p/tcp' +import { Noise } from '@chainsafe/libp2p-noise' + +const node = await createLibp2p({ + transports: [new TCP()], + connectionEncryption: [new Noise()] +}) + +``` + +#### Multiplexing + +While multiplexers are not strictly required, they are highly recommended as they improve the effectiveness and efficiency of connections for the various protocols libp2p runs. + +Looking at the [available stream multiplexing](https://github.com/libp2p/js-libp2p/blob/master/doc/CONFIGURATION.md#stream-multiplexing) modules, js-libp2p currently only supports `@libp2p/mplex`, so we will use that here. You can install `@libp2p/mplex` and add it to your libp2p node as follows in the next example. + +```sh +npm install @libp2p/mplex +``` + +```js +import { createLibp2p } from 'libp2p' +import { TCP } from '@libp2p/tcp' +import { Noise } from '@chainsafe/libp2p-noise' +import { Mplex } from '@libp2p/mplex' + +const node = await createLibp2p({ + transports: [new TCP()], + connectionEncryption: [new Noise()], + streamMuxers: [new Mplex()] +}) + +``` + +#### Running Libp2p + +Now that you have configured a **Transport**, **Crypto** and **Stream Multiplexer** module, you can start your libp2p node. We can start and stop libp2p using the [`libp2p.start()`](https://github.com/libp2p/js-libp2p/blob/master/doc/API.md#start) and [`libp2p.stop()`](https://github.com/libp2p/js-libp2p/blob/master/doc/API.md#stop) methods. + + +```js +import { createLibp2p } from 'libp2p' +import { TCP } from '@libp2p/tcp' +import { Noise } from '@chainsafe/libp2p-noise' +import { Mplex } from '@libp2p/mplex' + +const main = async () => { + const node = await createLibp2p({ + addresses: { + // add a listen address (localhost) to accept TCP connections on a random port + listen: ['/ip4/127.0.0.1/tcp/0'] + }, + transports: [new TCP()], + connectionEncryption: [new Noise()], + streamMuxers: [new Mplex()] + }) + + // start libp2p + await node.start() + console.log('libp2p has started') + + // print out listening addresses + console.log('listening on addresses:') + node.getMultiaddrs().forEach((addr) => { + console.log(addr.toString()) + }) + + // stop libp2p + await node.stop() + console.log('libp2p has stopped') +} + +main().then().catch(console.error) + +``` + +Try running the code with `node src/index.js`. You should see something like: + +``` +libp2p has started +listening on addresses: +/ip4/127.0.0.1/tcp/50626/p2p/QmYoqzFj5rhzFy7thCPPGbDkDkLMbQzanxCNwefZd3qTkz +libp2p has stopped +``` + +## Lets play ping pong! + +Now that we have the basic building blocks of transport, multiplexing, and security in place, we can start communicating! + +We can use [`libp2p.ping()`](https://github.com/libp2p/js-libp2p/blob/master/doc/API.md#ping) to dial and send ping messages to another peer. That peer will send back a "pong" message, so that we know that it is still alive. This also enables us to measure the latency between peers. + +We can have our application accepting a peer multiaddress via command line argument and try to ping it. To do so, we'll need to add a couple things. First, require the `process` module so that we can get the command line arguments. Then we'll need to parse the multiaddress from the command line and try to ping it: + +```sh +npm install multiaddr +``` + +```javascript +import process from 'node:process' +import { createLibp2p } from 'libp2p' +import { TCP } from '@libp2p/tcp' +import { Noise } from '@chainsafe/libp2p-noise' +import { Mplex } from '@libp2p/mplex' +import { multiaddr } from 'multiaddr' + +const node = await createLibp2p({ + addresses: { + // add a listen address (localhost) to accept TCP connections on a random port + listen: ['/ip4/127.0.0.1/tcp/0'] + }, + transports: [new TCP()], + connectionEncryption: [new Noise()], + streamMuxers: [new Mplex()] +}) + +// start libp2p +await node.start() +console.log('libp2p has started') + +// print out listening addresses +console.log('listening on addresses:') +node.getMultiaddrs().forEach((addr) => { + console.log(addr.toString()) +}) + +// ping peer if received multiaddr +if (process.argv.length >= 3) { + const ma = multiaddr(process.argv[2]) + console.log(`pinging remote peer at ${process.argv[2]}`) + const latency = await node.ping(ma) + console.log(`pinged ${process.argv[2]} in ${latency}ms`) +} else { + console.log('no remote peer address given, skipping ping') +} + +const stop = async () => { + // stop libp2p + await node.stop() + console.log('libp2p has stopped') + process.exit(0) +} + +process.on('SIGTERM', stop) +process.on('SIGINT', stop) + +``` + +Now we can start one instance with no arguments: + +```shell +> node src/index.js +libp2p has started +listening on addresses: +/ip4/127.0.0.1/tcp/50775/p2p/QmcafwJSsCsnjMo2fyf1doMjin8nrMawfwZiPftBDpahzN +no remote peer address given, skipping ping +``` + +Grab the `/ip4/...` address printed above and use it as an argument to another instance. In a new terminal: + +```shell +> node src/index.js /ip4/127.0.0.1/tcp/50775/p2p/QmcafwJSsCsnjMo2fyf1doMjin8nrMawfwZiPftBDpahzN +libp2p has started +listening on addresses: +/ip4/127.0.0.1/tcp/50777/p2p/QmYZirEPREz9vSRFznxhQbWNya2LXPz5VCahRCT7caTLGm +pinging remote peer at /ip4/127.0.0.1/tcp/50775/p2p/QmcafwJSsCsnjMo2fyf1doMjin8nrMawfwZiPftBDpahzN +pinged /ip4/127.0.0.1/tcp/50775/p2p/QmcafwJSsCsnjMo2fyf1doMjin8nrMawfwZiPftBDpahzN in 3ms +libp2p has stopped +``` + +Success! Our two peers are now communicating over a multiplexed, secure channel. Sure, they can only say "ping", but it's a start! + +## What's next? + +After finishing this tutorial, you should have a look into the [js-libp2p getting started](https://github.com/libp2p/js-libp2p/blob/master/doc/GETTING_STARTED.md) document, which goes from a base configuration like this one, to more custom ones. + +You also have a panoply of examples on [js-libp2p repo](https://github.com/libp2p/js-libp2p/tree/master/examples) that you can leverage to learn how to use `js-libp2p` for several different use cases and runtimes. + +[definition_multiaddress]: /reference/glossary/#multiaddr +[definition_multiplexer]: /reference/glossary/#multiplexer +[definition_peerid]: /reference/glossary/#peerid diff --git a/content/guides/examples/getting-started/rust.md b/content/guides/examples/getting-started/rust.md new file mode 100644 index 00000000..269a1cec --- /dev/null +++ b/content/guides/examples/getting-started/rust.md @@ -0,0 +1,8 @@ +--- +title: "Getting Started with rust-libp2p" +menuTitle: rust-libp2p +weight: 1 +--- + +Check out [tutorials of the Rust libp2p +implementation](https://docs.rs/libp2p/newest/libp2p/tutorials/index.html). diff --git a/content/tutorials/_index.md b/content/guides/tutorials/_index.md similarity index 100% rename from content/tutorials/_index.md rename to content/guides/tutorials/_index.md diff --git a/content/tutorials/getting-started/_index.md b/content/guides/tutorials/getting-started/_index.md similarity index 100% rename from content/tutorials/getting-started/_index.md rename to content/guides/tutorials/getting-started/_index.md diff --git a/content/tutorials/getting-started/go.md b/content/guides/tutorials/getting-started/go.md similarity index 100% rename from content/tutorials/getting-started/go.md rename to content/guides/tutorials/getting-started/go.md diff --git a/content/tutorials/getting-started/javascript.md b/content/guides/tutorials/getting-started/javascript.md similarity index 100% rename from content/tutorials/getting-started/javascript.md rename to content/guides/tutorials/getting-started/javascript.md diff --git a/content/tutorials/getting-started/rust.md b/content/guides/tutorials/getting-started/rust.md similarity index 100% rename from content/tutorials/getting-started/rust.md rename to content/guides/tutorials/getting-started/rust.md From 57d8d1f2cbe6d87bd582cb49691fb7762f9a3a32 Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Thu, 1 Sep 2022 06:01:30 -0400 Subject: [PATCH 02/30] edit section --- content/guides/_index.md | 7 +- content/guides/examples/_index.md | 7 +- content/guides/examples/getting-started/go.md | 440 ------------------ .../examples/getting-started/javascript.md | 279 ----------- .../guides/examples/getting-started/rust.md | 8 - content/guides/tutorials/_index.md | 5 +- .../tutorials/getting-started/_index.md | 11 - .../tutorials/{getting-started => }/go.md | 2 +- .../{getting-started => }/javascript.md | 0 .../tutorials/{getting-started => }/rust.md | 2 +- 10 files changed, 13 insertions(+), 748 deletions(-) delete mode 100644 content/guides/examples/getting-started/go.md delete mode 100644 content/guides/examples/getting-started/javascript.md delete mode 100644 content/guides/examples/getting-started/rust.md delete mode 100644 content/guides/tutorials/getting-started/_index.md rename content/guides/tutorials/{getting-started => }/go.md (99%) rename content/guides/tutorials/{getting-started => }/javascript.md (100%) rename content/guides/tutorials/{getting-started => }/rust.md (94%) diff --git a/content/guides/_index.md b/content/guides/_index.md index 45c85d6f..cb382c6a 100644 --- a/content/guides/_index.md +++ b/content/guides/_index.md @@ -4,8 +4,7 @@ weight: 4 pre: ' ' --- -Here's where to find working examples illustrating some of libp2p's key features for each of its main implementations: +The guides section of the libp2p documentation hub is a developer portal +containing how-to guides, technical tutorials, and examples for each libp2p implementation. -* For go, see the [go-libp2p-examples repo](https://github.com/libp2p/go-libp2p/tree/master/examples). -* For javascript, see the [/examples directory of the js-libp2p repo](https://github.com/libp2p/js-libp2p/tree/master/examples). -* For rust, see [/examples directory of the rust-libp2p repo](https://github.com/libp2p/rust-libp2p/tree/master/examples). +**This section is being updated, stay tuned!** diff --git a/content/guides/examples/_index.md b/content/guides/examples/_index.md index 5906d11d..f3eddb56 100644 --- a/content/guides/examples/_index.md +++ b/content/guides/examples/_index.md @@ -4,9 +4,12 @@ weight: 2 pre: ' ' --- -This tutorial covers setting up your development environment, getting familiar with some libp2p basics, and implementing a super simple node that can send and receive "ping" messages. +Here’s where to find working examples illustrating some of libp2p’s key features +for each of its main implementations: - +* For go, see the [go-libp2p-examples repo](https://github.com/libp2p/go-libp2p/tree/master/examples). +* For javascript, see the [/examples directory of the js-libp2p repo](https://github.com/libp2p/js-libp2p/tree/master/examples). +* For rust, see [/examples directory of the rust-libp2p repo](https://github.com/libp2p/rust-libp2p/tree/master/examples). {{% children description="true" %}} ----- diff --git a/content/guides/examples/getting-started/go.md b/content/guides/examples/getting-started/go.md deleted file mode 100644 index 9ded04fe..00000000 --- a/content/guides/examples/getting-started/go.md +++ /dev/null @@ -1,440 +0,0 @@ ---- -title: "Getting Started with go-libp2p" -menuTitle: go-libp2p ---- - -This is the first in a series of tutorials on working with libp2p’s Go implementation, -[go-libp2p](https://github.com/libp2p/go-libp2p). - -We’ll cover installing Go, setting up a new Go module, starting libp2p nodes, and sending ping -messages between them. - - - -## Install Go - -go-libp2p recommends the use of a Go version that includes the [modules feature](https://github.com/golang/go/wiki/Modules), -which means you need to have a version of Go that is at least 1.11. - -You can install a recent version of Go by following the [official installation instructions](https://golang.org/doc/install). - -Once installed, you should be able to run `go version` and see a version >= 1.11, for example: - -```sh -$ go version -go version go1.12 darwin/amd64 -``` - -## Create a Go module - -We're going to create a Go module that can be run from the command line. - -Let's create a new directory and use `go mod` to initialize it as a module. We'll create it in -`/tmp`, but you can equally create it anywhere on your filesystem (however it's advisable _not_ to -create this directory under your `GOPATH`). We'll also initialize it with the module name -`github.com/user/go-libp2p-tutorial`, but you may want to replace this with a name that corresponds -to a repository name you have the rights to push to if you want to publish your version of the code. - -```sh -$ mkdir -p /tmp/go-libp2p-tutorial - -$ cd /tmp/go-libp2p-tutorial - -$ go mod init github.com/user/go-libp2p-tutorial -``` - -You should now have a `go.mod` file in the current directory containing the name of the module you -initialized and the version of Go you're using, for example: - -```sh -$ cat go.mod -module github.com/user/go-libp2p-tutorial - -go 1.12 -``` - -## Start a libp2p node - -We'll now add some code to our module to start a libp2p node. - -Let's start by creating a `main.go` file that simply starts a libp2p node with default settings, -prints the node's listening addresses, then shuts the node down: - -```go -package main - -import ( - "fmt" - "github.com/libp2p/go-libp2p" -) - -func main() { - // start a libp2p node with default settings - node, err := libp2p.New() - if err != nil { - panic(err) - } - - // print the node's listening addresses - fmt.Println("Listen addresses:", node.Addrs()) - - // shut the node down - if err := node.Close(); err != nil { - panic(err) - } -} -``` - -Import the `libp2p/go-libp2p` module: - -```shell -$ go get github.com/libp2p/go-libp2p -``` - -We can now compile this into an executable using `go build` and run it from the command line: - -```sh -$ go build -o libp2p-node - -$ ./libp2p-node -Listen addresses: [/ip6/::1/tcp/57666 /ip4/127.0.0.1/tcp/57665 /ip4/192.168.1.56/tcp/57665] -``` - -The listening addresses are formatted using the [multiaddr](https://github.com/multiformats/multiaddr) -format, and there is typically more than one printed because go-libp2p will listen on all available -IPv4 and IPv6 network interfaces by default. - -### Configure the node - -A node's default settings can be overridden by passing extra arguments to `libp2p.New`. Let's use -`libp2p.ListenAddrStrings` to configure the node to listen on TCP port 2000 on the IPv4 loopback -interface: - -```go -func main() { - ... - - // start a libp2p node that listens on TCP port 2000 on the IPv4 - // loopback interface - node, err := libp2p.New( - libp2p.ListenAddrStrings("/ip4/127.0.0.1/tcp/2000"), - ) - if err != nil { - panic(err) - } - - ... -} -``` - -Re-building and running the executable again now prints the explicit listen address we've configured: - -```sh -$ go build -o libp2p-node - -$ ./libp2p-node -Listening addresses: [/ip4/127.0.0.1/tcp/2000] -``` - -`libp2p.New` accepts a variety of arguments to configure most aspects of the node. See -[options.go](https://github.com/libp2p/go-libp2p/blob/master/options.go) for a full list of those -options. - -### Wait for a signal - -A node that immediately exits is not all that useful. Let's add the following towards the end of the -`main` function that blocks waiting for an OS signal before shutting down the node: - -```go -func main() { - ... - - // wait for a SIGINT or SIGTERM signal - ch := make(chan os.Signal, 1) - signal.Notify(ch, syscall.SIGINT, syscall.SIGTERM) - <-ch - fmt.Println("Received signal, shutting down...") - - // shut the node down - if err := node.Close(); err != nil { - panic(err) - } -} -``` - -We also need to update the list of imports at the top of the file to include the `os`, `os/signal` -and `syscall` packages we're now using: - -```go -import ( - "fmt" - "os" - "os/signal" - "syscall" - - "github.com/libp2p/go-libp2p" -) -``` - -Running the node now waits until it receives a SIGINT (i.e. a `ctrl-c` key press) or a SIGTERM signal -before shutting down: - -```sh -$ ./libp2p-node -Listening addresses: [/ip4/127.0.0.1/tcp/2000] -^CReceived signal, shutting down... -``` - -## Run the ping protocol - -Now that we have the ability to configure and start libp2p nodes, we can start communicating! - -### Set a stream handler - -A node started with go-libp2p will run its own ping protocol by default, but let's disable it and -set it up manually to demonstrate the process of running protocols by registering stream handlers. - -The object returned from `libp2p.New` implements the [Host interface](https://pkg.go.dev/github.com/libp2p/go-libp2p-core/host#Host), -and we'll use the `SetStreamHandler` method to set a handler for our ping protocol. - -First, let's add the `github.com/libp2p/go-libp2p/p2p/protocol/ping` package to our list of -imported packages: - -```go -import ( - ... - - "github.com/libp2p/go-libp2p" - "github.com/libp2p/go-libp2p/p2p/protocol/ping" -) -``` - -Now we'll pass an argument to `libp2p.New` to disable the built-in ping protocol, and then use the -`PingService` type from the ping package to set a stream handler manually (note that we're also -configuring the node to listen on a random local TCP port rather than a hard coded one, which means -we'll be able to run multiple nodes on the same machine without them trying to listen on the same -port): - -```go -func main() { - ... - - // start a libp2p node that listens on a random local TCP port, - // but without running the built-in ping protocol - node, err := libp2p.New( - libp2p.ListenAddrStrings("/ip4/127.0.0.1/tcp/0"), - libp2p.Ping(false), - ) - if err != nil { - panic(err) - } - - // configure our own ping protocol - pingService := &ping.PingService{Host: node} - node.SetStreamHandler(ping.ID, pingService.PingHandler) - - ... -} -``` - -### Connect to a peer - -With the ping protocol configured, we need a way to instruct the node to connect to another node and -send it ping messages. - -We'll first expand the log message that we've been printing after starting the node to include -its `PeerId` value, as we'll need that to instruct other nodes to connect to it. Let's import the -`github.com/libp2p/go-libp2p-core/peer` package and use it to replace the "Listen addresses" log -message with something that prints both the listen address and the `PeerId` as a multiaddr string: - -```go -import ( - ... - - "github.com/libp2p/go-libp2p" - peerstore "github.com/libp2p/go-libp2p-core/peer" - "github.com/libp2p/go-libp2p/p2p/protocol/ping" -) - -func main() { - ... - - // print the node's PeerInfo in multiaddr format - peerInfo := peerstore.AddrInfo{ - ID: node.ID(), - Addrs: node.Addrs(), - } - addrs, err := peerstore.AddrInfoToP2pAddrs(&peerInfo) - fmt.Println("libp2p node address:", addrs[0]) - - ... -} -``` - -Running the node now prints the node's address that can be used to connect to it: - -```sh -$ ./libp2p-node -libp2p node address: /ip4/127.0.0.1/tcp/62268/ipfs/QmfQzWnLu4UX1cW7upgyuFLyuBXqze7nrPB4qWYqQiTHwt -``` - -Let's also accept a command line argument that is the address of a peer to send ping messages to, -allowing us to either just run a listening node that waits for a signal, or run a node that connects -to another node and pings it a few times before shutting down (we'll use the `github.com/multiformats/go-multiaddr` -package to parse the peer's address from the command line argument): - -```go -import ( - ... - - "github.com/libp2p/go-libp2p" - peerstore "github.com/libp2p/go-libp2p-core/peer" - "github.com/libp2p/go-libp2p/p2p/protocol/ping" - multiaddr "github.com/multiformats/go-multiaddr" -) - -func main() { - ... - fmt.Println("libp2p node address:", addrs[0]) - - // if a remote peer has been passed on the command line, connect to it - // and send it 5 ping messages, otherwise wait for a signal to stop - if len(os.Args) > 1 { - addr, err := multiaddr.NewMultiaddr(os.Args[1]) - if err != nil { - panic(err) - } - peer, err := peerstore.AddrInfoFromP2pAddr(addr) - if err != nil { - panic(err) - } - if err := node.Connect(context.Background(), *peer); err != nil { - panic(err) - } - fmt.Println("sending 5 ping messages to", addr) - ch := pingService.Ping(context.Background(), peer.ID) - for i := 0; i < 5; i++ { - res := <-ch - fmt.Println("got ping response!", "RTT:", res.RTT) - } - } else { - // wait for a SIGINT or SIGTERM signal - ch := make(chan os.Signal, 1) - signal.Notify(ch, syscall.SIGINT, syscall.SIGTERM) - <-ch - fmt.Println("Received signal, shutting down...") - } - - // shut the node down - if err := node.Close(); err != nil { - panic(err) - } -} -``` - -## Let's play ping pong! - -We are finally in a position to run two libp2p nodes, have one connect to the other and for -them to run a protocol! - -To recap, here is the full program we have written: - -```go -package main - -import ( - "context" - "fmt" - "os" - "os/signal" - "syscall" - - "github.com/libp2p/go-libp2p" - peerstore "github.com/libp2p/go-libp2p-core/peer" - "github.com/libp2p/go-libp2p/p2p/protocol/ping" - multiaddr "github.com/multiformats/go-multiaddr" -) - -func main() { - // start a libp2p node that listens on a random local TCP port, - // but without running the built-in ping protocol - node, err := libp2p.New( - libp2p.ListenAddrStrings("/ip4/127.0.0.1/tcp/0"), - libp2p.Ping(false), - ) - if err != nil { - panic(err) - } - - // configure our own ping protocol - pingService := &ping.PingService{Host: node} - node.SetStreamHandler(ping.ID, pingService.PingHandler) - - // print the node's PeerInfo in multiaddr format - peerInfo := peerstore.AddrInfo{ - ID: node.ID(), - Addrs: node.Addrs(), - } - addrs, err := peerstore.AddrInfoToP2pAddrs(&peerInfo) - if err != nil { - panic(err) - } - fmt.Println("libp2p node address:", addrs[0]) - - // if a remote peer has been passed on the command line, connect to it - // and send it 5 ping messages, otherwise wait for a signal to stop - if len(os.Args) > 1 { - addr, err := multiaddr.NewMultiaddr(os.Args[1]) - if err != nil { - panic(err) - } - peer, err := peerstore.AddrInfoFromP2pAddr(addr) - if err != nil { - panic(err) - } - if err := node.Connect(context.Background(), *peer); err != nil { - panic(err) - } - fmt.Println("sending 5 ping messages to", addr) - ch := pingService.Ping(context.Background(), peer.ID) - for i := 0; i < 5; i++ { - res := <-ch - fmt.Println("pinged", addr, "in", res.RTT) - } - } else { - // wait for a SIGINT or SIGTERM signal - ch := make(chan os.Signal, 1) - signal.Notify(ch, syscall.SIGINT, syscall.SIGTERM) - <-ch - fmt.Println("Received signal, shutting down...") - } - - // shut the node down - if err := node.Close(); err != nil { - panic(err) - } -} -``` - -In one terminal window, let's start a listening node (i.e. don't pass any command line arguments): - -```sh -$ ./libp2p-node -libp2p node address: /ip4/127.0.0.1/tcp/61790/ipfs/QmZKjsGJ6ukXVRXVEcExx9GhiyWoJC97onYpzBwCHPWqpL -``` - -In another terminal window, let's run a second node but pass the address of the first node, and we -should see some ping responses logged: - -```sh -$ ./libp2p-node /ip4/127.0.0.1/tcp/61790/ipfs/QmZKjsGJ6ukXVRXVEcExx9GhiyWoJC97onYpzBwCHPWqpL -libp2p node address: /ip4/127.0.0.1/tcp/61846/ipfs/QmVyKLTLswap3VYbpBATsgNpi6JdwSwsZALPxEnEbEndup -sending 5 ping messages to /ip4/127.0.0.1/tcp/61790/ipfs/QmZKjsGJ6ukXVRXVEcExx9GhiyWoJC97onYpzBwCHPWqpL -pinged /ip4/127.0.0.1/tcp/61790/ipfs/QmZKjsGJ6ukXVRXVEcExx9GhiyWoJC97onYpzBwCHPWqpL in 431.231µs -pinged /ip4/127.0.0.1/tcp/61790/ipfs/QmZKjsGJ6ukXVRXVEcExx9GhiyWoJC97onYpzBwCHPWqpL in 164.94µs -pinged /ip4/127.0.0.1/tcp/61790/ipfs/QmZKjsGJ6ukXVRXVEcExx9GhiyWoJC97onYpzBwCHPWqpL in 220.544µs -pinged /ip4/127.0.0.1/tcp/61790/ipfs/QmZKjsGJ6ukXVRXVEcExx9GhiyWoJC97onYpzBwCHPWqpL in 208.761µs -pinged /ip4/127.0.0.1/tcp/61790/ipfs/QmZKjsGJ6ukXVRXVEcExx9GhiyWoJC97onYpzBwCHPWqpL in 201.37µs -``` - -Success! Our two peers are now communicating using go-libp2p! Sure, they can only say “ping”, but it’s a start! diff --git a/content/guides/examples/getting-started/javascript.md b/content/guides/examples/getting-started/javascript.md deleted file mode 100644 index 1ac7d746..00000000 --- a/content/guides/examples/getting-started/javascript.md +++ /dev/null @@ -1,279 +0,0 @@ ---- -title: "Getting Started with js-libp2p" -menuTitle: js-libp2p -weight: 2 ---- - -This is the first in a series of tutorials on working with libp2p's javascript implementation, [js-libp2p](https://github.com/libp2p/js-libp2p). - -We will walk you through setting up a fully functional libp2p node with some basic functionality, and finally we'll send ping messages back and forth between two peers. - - - - - -## Install node.js - -Working with js-libp2p requires [node.js](https://nodejs.org) >= v16 for development. If you haven't already, install node using whatever package manager you prefer or [using the official installer](https://nodejs.org/en/download/). - -We recommend using the latest stable version of node, but anything fairly recent should work fine. If you want to see how low you can go, the current version requirements can always be found at the [js-libp2p project page](https://github.com/libp2p/js-libp2p). - -## Create an empty project - -We need a place to put our work, so open a terminal to make a new directory for your project somewhere and set it up as an npm project: - -```shell -# create a directory for the project and `cd` into it -> mkdir hello-libp2p -> mkdir hello-libp2p/src -> cd hello-libp2p - -# make it a git repository -> git init . - -# make it an npm project. fill in the prompts with info for your project -# when asked for your project's entry point, enter "src/index.js" -> npm init -``` - -Side note: throughout this tutorial, we use the `> ` character to indicate your terminal's shell prompt. When following along, don't type the `>` character, or you'll get some weird errors. - -## Configure libp2p - -libp2p is a very modular framework, which allows javascript devs to target different runtime environments and opt-in to various features by including a custom selection of modules. - -Because every application is different, we recommend configuring your libp2p node with just the modules you need. You can even make more than one configuration, if you want to target multiple javascript runtimes with different features. - -{{% notice note %}} -In a production application, it may make sense to create a separate npm module for your libp2p node, which will give you one place to manage the libp2p dependencies for all your javascript projects. In that case, you should not depend on `libp2p` directly in your application. Instead you'd depend on your libp2p configuration module, which would in turn depend on `libp2p` and whatever modules (transports, etc) you might need. -{{% /notice %}} - -If you're new to libp2p, we recommend configuring your node in stages, as this can make troubleshooting configuration issues much easier. In this tutorial, we'll do just that. If you're more experienced with libp2p, you may wish to jump to the [Configuration readme](https://github.com/libp2p/js-libp2p/blob/master/doc/CONFIGURATION.md). - -As an initial step, you should install libp2p module. - -```shell -npm install libp2p -``` - -### Basic setup - -Now that we have libp2p installed, let's configure the minimum needed to get your node running. The only modules libp2p requires are a **Transport** and **Crypto** module. However, we recommend that a basic setup should also have a **Stream Multiplexer** configured, which we will explain shortly. Let's start by setting up a Transport. - -#### Transports - -Libp2p uses Transports to establish connections between peers over the network. You can configure any number of Transports, but you only need 1 to start with. - -You should select Transports according to the runtime of your application; Node.js or the browser. You can see a list of some of the available Transports in the [configuration readme](https://github.com/libp2p/js-libp2p/blob/master/doc/CONFIGURATION.md#transport). For this guide let's install `@libp2p/tcp`. - -```sh -npm install @libp2p/tcp -``` - -Now that we have the module installed, let's configure libp2p to use the Transport. We'll use the `createLibp2pNode` method, which takes a single configuration object as its only parameter. We can add the Transport by passing it into the `transports` array. Create a `src/index.js` file and have the following code in it: - -```js -import { createLibp2p } from 'libp2p' -import { TCP } from '@libp2p/tcp' - -const node = await createLibp2p({ - transports: [new TCP()] -}) - -``` - -You can add as many transports as you like to `transports` in order to establish connections with as many peers as possible. - -#### Connection Encryption - -Every connection must be encrypted to help ensure security for everyone. As such, Connection Encryption (Crypto) is a required component of libp2p. - -There are a growing number of Crypto modules being developed for libp2p. As those are released they will be tracked in the [Connection Encryption section of the configuration readme](https://github.com/libp2p/js-libp2p/blob/master/doc/CONFIGURATION.md#connection-encryption). For now, we are going to configure our node to use the `@chainsafe/libp2p-noise` module. - -```sh -npm install @chainsafe/libp2p-noise -``` - -```js -import { createLibp2p } from 'libp2p' -import { TCP } from '@libp2p/tcp' -import { Noise } from '@chainsafe/libp2p-noise' - -const node = await createLibp2p({ - transports: [new TCP()], - connectionEncryption: [new Noise()] -}) - -``` - -#### Multiplexing - -While multiplexers are not strictly required, they are highly recommended as they improve the effectiveness and efficiency of connections for the various protocols libp2p runs. - -Looking at the [available stream multiplexing](https://github.com/libp2p/js-libp2p/blob/master/doc/CONFIGURATION.md#stream-multiplexing) modules, js-libp2p currently only supports `@libp2p/mplex`, so we will use that here. You can install `@libp2p/mplex` and add it to your libp2p node as follows in the next example. - -```sh -npm install @libp2p/mplex -``` - -```js -import { createLibp2p } from 'libp2p' -import { TCP } from '@libp2p/tcp' -import { Noise } from '@chainsafe/libp2p-noise' -import { Mplex } from '@libp2p/mplex' - -const node = await createLibp2p({ - transports: [new TCP()], - connectionEncryption: [new Noise()], - streamMuxers: [new Mplex()] -}) - -``` - -#### Running Libp2p - -Now that you have configured a **Transport**, **Crypto** and **Stream Multiplexer** module, you can start your libp2p node. We can start and stop libp2p using the [`libp2p.start()`](https://github.com/libp2p/js-libp2p/blob/master/doc/API.md#start) and [`libp2p.stop()`](https://github.com/libp2p/js-libp2p/blob/master/doc/API.md#stop) methods. - - -```js -import { createLibp2p } from 'libp2p' -import { TCP } from '@libp2p/tcp' -import { Noise } from '@chainsafe/libp2p-noise' -import { Mplex } from '@libp2p/mplex' - -const main = async () => { - const node = await createLibp2p({ - addresses: { - // add a listen address (localhost) to accept TCP connections on a random port - listen: ['/ip4/127.0.0.1/tcp/0'] - }, - transports: [new TCP()], - connectionEncryption: [new Noise()], - streamMuxers: [new Mplex()] - }) - - // start libp2p - await node.start() - console.log('libp2p has started') - - // print out listening addresses - console.log('listening on addresses:') - node.getMultiaddrs().forEach((addr) => { - console.log(addr.toString()) - }) - - // stop libp2p - await node.stop() - console.log('libp2p has stopped') -} - -main().then().catch(console.error) - -``` - -Try running the code with `node src/index.js`. You should see something like: - -``` -libp2p has started -listening on addresses: -/ip4/127.0.0.1/tcp/50626/p2p/QmYoqzFj5rhzFy7thCPPGbDkDkLMbQzanxCNwefZd3qTkz -libp2p has stopped -``` - -## Lets play ping pong! - -Now that we have the basic building blocks of transport, multiplexing, and security in place, we can start communicating! - -We can use [`libp2p.ping()`](https://github.com/libp2p/js-libp2p/blob/master/doc/API.md#ping) to dial and send ping messages to another peer. That peer will send back a "pong" message, so that we know that it is still alive. This also enables us to measure the latency between peers. - -We can have our application accepting a peer multiaddress via command line argument and try to ping it. To do so, we'll need to add a couple things. First, require the `process` module so that we can get the command line arguments. Then we'll need to parse the multiaddress from the command line and try to ping it: - -```sh -npm install multiaddr -``` - -```javascript -import process from 'node:process' -import { createLibp2p } from 'libp2p' -import { TCP } from '@libp2p/tcp' -import { Noise } from '@chainsafe/libp2p-noise' -import { Mplex } from '@libp2p/mplex' -import { multiaddr } from 'multiaddr' - -const node = await createLibp2p({ - addresses: { - // add a listen address (localhost) to accept TCP connections on a random port - listen: ['/ip4/127.0.0.1/tcp/0'] - }, - transports: [new TCP()], - connectionEncryption: [new Noise()], - streamMuxers: [new Mplex()] -}) - -// start libp2p -await node.start() -console.log('libp2p has started') - -// print out listening addresses -console.log('listening on addresses:') -node.getMultiaddrs().forEach((addr) => { - console.log(addr.toString()) -}) - -// ping peer if received multiaddr -if (process.argv.length >= 3) { - const ma = multiaddr(process.argv[2]) - console.log(`pinging remote peer at ${process.argv[2]}`) - const latency = await node.ping(ma) - console.log(`pinged ${process.argv[2]} in ${latency}ms`) -} else { - console.log('no remote peer address given, skipping ping') -} - -const stop = async () => { - // stop libp2p - await node.stop() - console.log('libp2p has stopped') - process.exit(0) -} - -process.on('SIGTERM', stop) -process.on('SIGINT', stop) - -``` - -Now we can start one instance with no arguments: - -```shell -> node src/index.js -libp2p has started -listening on addresses: -/ip4/127.0.0.1/tcp/50775/p2p/QmcafwJSsCsnjMo2fyf1doMjin8nrMawfwZiPftBDpahzN -no remote peer address given, skipping ping -``` - -Grab the `/ip4/...` address printed above and use it as an argument to another instance. In a new terminal: - -```shell -> node src/index.js /ip4/127.0.0.1/tcp/50775/p2p/QmcafwJSsCsnjMo2fyf1doMjin8nrMawfwZiPftBDpahzN -libp2p has started -listening on addresses: -/ip4/127.0.0.1/tcp/50777/p2p/QmYZirEPREz9vSRFznxhQbWNya2LXPz5VCahRCT7caTLGm -pinging remote peer at /ip4/127.0.0.1/tcp/50775/p2p/QmcafwJSsCsnjMo2fyf1doMjin8nrMawfwZiPftBDpahzN -pinged /ip4/127.0.0.1/tcp/50775/p2p/QmcafwJSsCsnjMo2fyf1doMjin8nrMawfwZiPftBDpahzN in 3ms -libp2p has stopped -``` - -Success! Our two peers are now communicating over a multiplexed, secure channel. Sure, they can only say "ping", but it's a start! - -## What's next? - -After finishing this tutorial, you should have a look into the [js-libp2p getting started](https://github.com/libp2p/js-libp2p/blob/master/doc/GETTING_STARTED.md) document, which goes from a base configuration like this one, to more custom ones. - -You also have a panoply of examples on [js-libp2p repo](https://github.com/libp2p/js-libp2p/tree/master/examples) that you can leverage to learn how to use `js-libp2p` for several different use cases and runtimes. - -[definition_multiaddress]: /reference/glossary/#multiaddr -[definition_multiplexer]: /reference/glossary/#multiplexer -[definition_peerid]: /reference/glossary/#peerid diff --git a/content/guides/examples/getting-started/rust.md b/content/guides/examples/getting-started/rust.md deleted file mode 100644 index 269a1cec..00000000 --- a/content/guides/examples/getting-started/rust.md +++ /dev/null @@ -1,8 +0,0 @@ ---- -title: "Getting Started with rust-libp2p" -menuTitle: rust-libp2p -weight: 1 ---- - -Check out [tutorials of the Rust libp2p -implementation](https://docs.rs/libp2p/newest/libp2p/tutorials/index.html). diff --git a/content/guides/tutorials/_index.md b/content/guides/tutorials/_index.md index e621185b..e89566dc 100644 --- a/content/guides/tutorials/_index.md +++ b/content/guides/tutorials/_index.md @@ -4,7 +4,8 @@ weight: 2 pre: ' ' --- - -Here you'll find step-by-step tutorials for using libp2p in your favorite languages. +Here, you'll find step-by-step tutorials about setting up your development environment, +getting familiar with some libp2p basics, and implementing a super simple node that can +send and receive "ping" messages. {{% children description="true" %}} diff --git a/content/guides/tutorials/getting-started/_index.md b/content/guides/tutorials/getting-started/_index.md deleted file mode 100644 index a8aeb59e..00000000 --- a/content/guides/tutorials/getting-started/_index.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -title: "Getting Started" -weight: 1 ---- - -This tutorial covers setting up your development environment, getting familiar with some libp2p basics, and implementing a super simple node that can send and receive "ping" messages. - - - -{{% children description="true" %}} ------ diff --git a/content/guides/tutorials/getting-started/go.md b/content/guides/tutorials/go.md similarity index 99% rename from content/guides/tutorials/getting-started/go.md rename to content/guides/tutorials/go.md index 34cca073..27da6c4d 100644 --- a/content/guides/tutorials/getting-started/go.md +++ b/content/guides/tutorials/go.md @@ -1,6 +1,6 @@ --- title: "Getting Started with go-libp2p" -menuTitle: Go +menuTitle: Golang weight: 1 --- diff --git a/content/guides/tutorials/getting-started/javascript.md b/content/guides/tutorials/javascript.md similarity index 100% rename from content/guides/tutorials/getting-started/javascript.md rename to content/guides/tutorials/javascript.md diff --git a/content/guides/tutorials/getting-started/rust.md b/content/guides/tutorials/rust.md similarity index 94% rename from content/guides/tutorials/getting-started/rust.md rename to content/guides/tutorials/rust.md index 5296fdae..e0c3898c 100644 --- a/content/guides/tutorials/getting-started/rust.md +++ b/content/guides/tutorials/rust.md @@ -1,7 +1,7 @@ --- title: "Getting Started with rust-libp2p" menuTitle: Rust -weight: 1 +weight: 3 --- Check out [tutorials of the Rust libp2p From 6a9bc77aa3848276ccbd5ed8150c1b65705d6ade Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Wed, 7 Sep 2022 04:28:58 -0400 Subject: [PATCH 03/30] init --- content/_index.md | 59 ++------ content/concepts/_index.md | 9 +- content/introduction/_index.md | 186 +----------------------- content/introduction/getting-started.md | 182 +++++++++++++++++++++++ content/introduction/network-basics.md | 7 + 5 files changed, 212 insertions(+), 231 deletions(-) create mode 100644 content/introduction/getting-started.md create mode 100644 content/introduction/network-basics.md diff --git a/content/_index.md b/content/_index.md index da480862..1766f132 100644 --- a/content/_index.md +++ b/content/_index.md @@ -1,59 +1,20 @@ --- -title: libp2p +title: Welcome the libp2p weight: 1 +pre: "1. " +chapter: false --- -Welcome to the libp2p documentation portal! Whether you’re just learning how to build peer-to-peer systems with libp2p, want to dive into peer-to-peer concepts and solutions, or are looking for detailed reference information, this is the place to start. +# Welcome the libp2p Documentation Portal -## Overview +Welcome to the official libp2p documentation hub! -Head over to [What is libp2p?](/introduction/what-is-libp2p/) for an introduction to the basics of libp2p and an overview of the problems it addresses. +### Chapter 1 +### Chapter 2 -## Tutorials +### Chapter 3 -If you want to dive in, check out our collection of [tutorials](/tutorials/), which will help guide you through your explorations of libp2p. +### Chapter 4 -## Examples - -If you want to get a feel for what's possible with libp2p, or just want to see some idiomatic usage, check out the [examples](/examples/). Each libp2p implementation maintains a set of working example projects that can illustrate key concepts and use cases. - -## Reference - -### Specifications & Planning - -While libp2p has several implementations, it is fundamentally a set of protocols for peer identity, discover, routing, transport and more. - -See the [specifications section](/reference/specs/) for details. - -### Implementations - -At the core of libp2p is a set of [specifications](/reference/specs/), which together form the definition for what libp2p is in the abstract and what makes a "correct" libp2p implementation. Today, implementations of libp2p exist in several languages, with varying degrees of completeness. The most complete implementations are in [Go](/reference/go/) and [JavaScript](/reference/js/), with [rust](https://github.com/libp2p/rust-libp2p) maturing rapidly. - -In addition to the implementations above, the libp2p community is actively working on implementations in [python](https://github.com/libp2p/py-libp2p) and [the JVM via Kotlin](https://github.com/web3j/libp2p). Please check the project pages for each implementation to see its status and current state of completeness. - - -## Community - -Get in touch with other members of the libp2p community who are building tools and applications with libp2p! You can ask questions, discuss new ideas, or get support for problems at [libp2p discussion forums](https://discuss.libp2p.io), but you can also [hop on IRC](irc://irc.freenode.org/%23libp2p) for a quick chat. - -See the other links in the community section for more information about meetings, events, apps people are building, and more. - -Information about contributing to libp2p and about other software projects in the community are also hosted here. - - -### Get Involved - -libp2p is an open-source community project. While [Protocol Labs](https://protocol.ai) is able to sponsor some of the work around it, much of the design, code, and effort is contributed by volunteers and community members like you. If you’re interested in helping improve libp2p, check the [contributing](/contributing/) guide to get started. - -If you are diving in to contribute new code, make sure you check both the [contribution guidelines](https://github.com/libp2p/community/blob/master/CONTRIBUTE.md) and the style guide for your language ([Go](https://github.com/ipfs/community/blob/master/CONTRIBUTING_GO.md), [JavaScript](https://github.com/ipfs/community/blob/master/CONTRIBUTING_JS.md)). - - -### Related Projects - -libp2p began as part of the [IPFS](https://ipfs.io) project, and is still an essential component of IPFS. As such, libp2p composes well with the abstractions and tools provided by other projects in the IPFS "family". Check their individual sites for specific information and references: - -- [IPFS](https://ipfs.io) is the InterPlanetary File System, which uses libp2p as its networking layer. -- [Multiformats](https://multiformats.io) is a variety of *self-describing* data formats. -- [IPLD](https://ipld.io) is a set of tools for describing links between content-addressed data, like IPFS files, Git commits, or Ethereum blocks. -- [The Permissive License Stack](https://protocol.ai/blog/announcing-the-permissive-license-stack) is a licensing strategy for software development that embraces open-source values. +### Chapter 5 diff --git a/content/concepts/_index.md b/content/concepts/_index.md index de391551..cd9cf676 100644 --- a/content/concepts/_index.md +++ b/content/concepts/_index.md @@ -1,9 +1,14 @@ --- -title: "Core Components" +title: "Explore libp2p" weight: 2 -pre: ' ' +pre: "2. " +chapter: true --- +### Chapter 2 + +# Explore libp2p + libp2p covers a lot of ground, and may involve unfamiliar terminology and concepts. This section goes over the foundational concepts involved in libp2p. diff --git a/content/introduction/_index.md b/content/introduction/_index.md index d81f99c5..a3954419 100644 --- a/content/introduction/_index.md +++ b/content/introduction/_index.md @@ -1,189 +1,15 @@ --- -title: "Introduction" +title: Quick Start weight: 1 -pre: ' ' +pre: "1. " +chapter: true --- -Welcome to the official libp2p documentation hub! +### Chapter 1 + +# Get Started with libp2p Whether you’re just starting to dive into peer-to-peer concepts and solutions, learning how to build peer-to-peer systems with libp2p, or are looking for detailed reference information, this is the place to start. - -## What is libp2p? - -libp2p is a modular system of *protocols*, *specifications* and *libraries* -that enable the development of peer-to-peer network applications. - -**A p2p networking stack** - -Because of libp2p's peer-to-peer and distributed architecture, most of the -needs and considerations that the current web was built on no longer apply. -The internet, such as it is, with firewalls and NATs, was designed to [securely] -provide data by relying on trust assumptions. There are many distributed -peer-to-peer network models with different challenges and tradeoffs that try -to improve on the way we network. Libp2p aims to be a modular, general-purpose -toolkit for any peer-to-peer application. - -## Peer-to-peer basics - -Let's start with what a peer-to-peer network application is: - -A [peer-to-peer network][definition_p2p] is one in which the participants -(referred to as [peers][definition_peer]) communicate directly with one another -on a relative "equal footing". This does not mean that all peers are identical -as some may have different roles in the overall network. However, one of the -defining characteristics of a peer-to-peer network is that the network does not -require a privileged set of "servers" which behave completely differently from -their "clients", as is the case in the predominant -[client / server model][definition_client_server]. - -Because the definition of peer-to-peer networking is quite broad, many different -kinds of systems have been built that all fall under the umbrella of "peer-to-peer". -The most culturally prominent examples are likely file-sharing networks like BitTorrent, -and, more recently, the proliferation of blockchain networks that communicate in a -peer-to-peer fashion. - -### What problems can libp2p solve? - -While peer-to-peer networks have many advantages over the client-server model, -there are unique challenges that require careful thought and practice to overcome. - -libp2p lets all users preserve their network identity, overcome network censorship, -and communicate over different transport protocols. - -In overcoming these challenges while building [IPFS](https://ipfs.io), -we took care to build our solutions in a modular, composable way into what is -now libp2p. Although libp2p grew out of IPFS, it is not dependent on IPFS, and -today, [many projects][built_with_libp2p] use libp2p as their networking layer. - -Together, we can leverage our collective experience and solve these foundational -problems in a way that benefits an entire ecosystem of developers and a world of users. - -Here, we'll briefly outline the main problem areas that libp2p attempts to address. -This is an ever-growing space, so don't be surprised if things change over time. -If you notice something missing or have other ideas for improving this documentation, -please [reach out to let us know][help_improve_docs]. - -### Data transmission - -The transport layer is at the foundation of libp2p, which is responsible for -transmitting and receiving bytes between two peers. There are many -ways to send data across networks in use today, including TCP, QUIC, WebSocket, -WebTransport and WebRTC, with some still in development and others still yet -to be designed. - -libp2p also provides a list of specifications [specifcations](https://github.com/libp2p/specs) -that can be adapted to support existing and future protocols, allowing libp2p applications -to operate in many different runtime and networking environments. - -### Peer identity - -Knowing who you're talking to is key to secure and reliable communication in a world -with billions of networked devices. libp2p uses -[public key cryptography](https://en.wikipedia.org/wiki/Public-key_cryptography) -as the basis of peer identity, which serves two complementary purposes. - -1. It gives each peer a globally unique "name", in the form of a - [PeerId][definition_peerid]. -2. The `PeerId` allows anyone to retrieve the public key for the identified - peer, which enables secure communication between peers. - -### Secure communication - -There needs to be a method to securely send and receive data between peers, -where peers are able to trust the [identity](#peer-identity) of the peer they're -communicating with while ensuring that no external entity can access or tamper with -the communication. - -All libp2p connections are encrypted and authenticated. Some [transport protocol](#transport) -protocols are encrypted at the transport layer (e.g. QUIC). For other protocols, libp2p runs -a cryptographic handshake on top of an unencrypted connection (e.g. TCP). - -For secure communication channels, libp2p currently supports -[TLS 1.3](https://www.ietf.org/blog/tls13/) and [Noise](https://noiseprotocol.org/), -though not every language implementation of libp2p supports both of these. - -> (Older versions of libp2p may support a -> [deprecated](https://blog.ipfs.io/2020-08-07-deprecating-secio/) protocol called SECIO; -> all projects should switch to TLS 1.3 or Noise instead.) - -### Peer routing - -When you want to send a message to another peer, you need two key pieces -of information: their [PeerId][definition_peerid], and a way to locate them -on the network to open a connection. - -There are many cases where we only have the `PeerId` for the peer we want to -contact, and we need a way to discover their network address. Peer routing is -the process of discovering peer addresses by leveraging the knowledge of other -peers. - -In a peer routing system, a peer can either give us the address we need if they -have it, or else send our inquiry to another peer who's more likely to have the -answer. As we contact more and more peers, we not only increase our chances of -finding the peer we're looking for, we build a more complete view of the network -in our own routing tables, which enables us to answer routing queries from others. - -The current stable implementation of peer routing in libp2p uses a -[distributed hash table][definition_dht] to iteratively route requests closer -to the desired `PeerId` using the [Kademlia][wiki_kademlia] routing algorithm. - -### Content discovery - -In some systems, we care less about who we're speaking with than what they can offer us. -For example, we may want some specific piece of data, but we don't care who we get it from -since we can verify its integrity. - -libp2p provides a [content routing specification][spec_content_routing] for this -purpose, with the primary stable implementation using the same -[Kademlia][wiki_kademlia]-based DHT as used in peer routing. - -### Peer messaging - -Sending messages to other peers is at the heart of most peer-to-peer systems, -and pubsub (short for publish/subscribe) is an instrumental pattern for sending -a message to groups of interested receivers. - -libp2p defines a [pubsub specification][spec_pubsub] for sending messages to all -peers subscribed to a given "topic". The specification currently has two stable -implementations; `floodsub` uses a very simple but inefficient "network flooding" -strategy, and [gossipsub](https://github.com/libp2p/specs/tree/master/pubsub/gossipsub) -defines an extensible gossip protocol. There is also active development in progress on -[episub](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/episub.md), an -extended `gossipsub` that is optimized for single source multicast and scenarios with a -few fixed sources broadcasting to a large number of clients in a topic. - -Head over to [What is libp2p?](/introduction/what-is-libp2p/) for an introduction to -the basics of libp2p and an overview of the problems it addresses. - -## Related projects - -libp2p began as part of the [IPFS](https://ipfs.io) project and is still an -essential component of IPFS. As such, libp2p composes well with the abstractions -and tools provided by other projects in the IPFS "family". Check their sites for -specific information and references: - -- [IPFS](https://libp2p.io) is the InterPlanetary File System, which uses libp2p as - its networking layer. -- [Multiformats](https://multiformats.io) is a variety of *self-describing* data formats. -- [IPLD](https://ipld.io) is a set of tools for describing links between content-addressed - data, like IPFS files, Git commits, or Ethereum blocks. -- [The Permissive License Stack](https://protocol.ai/blog/announcing-the-permissive-license-stack) - is a licensing strategy for software development that embraces open-source values. - -[glossary]: {{< ref "/reference/glossary.md" >}} -[definition_dht]: {{< ref "/reference/glossary.md#dht" >}} -[definition_p2p]: {{< ref "/reference/glossary.md#p2p" >}} -[definition_peer]: {{< ref "/reference/glossary.md#peer" >}} -[definition_peerid]: {{< ref "/reference/glossary.md#peerid" >}} -[definition_secio]: {{< ref "/reference/glossary.md#secio" >}} -[definition_muiltiaddress]: {{< ref "/reference/glossary.md#multiaddr" >}} -[definition_client_server]: {{< ref "/reference/glossary.md#client-server" >}} - -[spec_content_routing]: https://github.com/libp2p/specs/blob/master/kad-dht/README.md -[spec_pubsub]: https://github.com/libp2p/specs/blob/master/pubsub/README.md -[built_with_libp2p]: https://discuss.libp2p.io/c/ecosystem-community -[help_improve_docs]: https://github.com/libp2p/docs/issues -[wiki_kademlia]: https://en.wikipedia.org/wiki/Kademlia diff --git a/content/introduction/getting-started.md b/content/introduction/getting-started.md new file mode 100644 index 00000000..d25b4eeb --- /dev/null +++ b/content/introduction/getting-started.md @@ -0,0 +1,182 @@ +--- +title: "Introduction" +weight: 2 +pre: ' ' +--- + +## What is libp2p? + +libp2p is a modular system of *protocols*, *specifications* and *libraries* +that enable the development of peer-to-peer network applications. + +**A p2p networking stack** + +Because of libp2p's peer-to-peer and distributed architecture, most of the +needs and considerations that the current web was built on no longer apply. +The internet, such as it is, with firewalls and NATs, was designed to [securely] +provide data by relying on trust assumptions. There are many distributed +peer-to-peer network models with different challenges and tradeoffs that try +to improve on the way we network. Libp2p aims to be a modular, general-purpose +toolkit for any peer-to-peer application. + +## Peer-to-peer basics + +Let's start with what a peer-to-peer network application is: + +A [peer-to-peer network][definition_p2p] is one in which the participants +(referred to as [peers][definition_peer]) communicate directly with one another +on a relative "equal footing". This does not mean that all peers are identical +as some may have different roles in the overall network. However, one of the +defining characteristics of a peer-to-peer network is that the network does not +require a privileged set of "servers" which behave completely differently from +their "clients", as is the case in the predominant +[client / server model][definition_client_server]. + +Because the definition of peer-to-peer networking is quite broad, many different +kinds of systems have been built that all fall under the umbrella of "peer-to-peer". +The most culturally prominent examples are likely file-sharing networks like BitTorrent, +and, more recently, the proliferation of blockchain networks that communicate in a +peer-to-peer fashion. + +### What problems can libp2p solve? + +While peer-to-peer networks have many advantages over the client-server model, +there are unique challenges that require careful thought and practice to overcome. + +libp2p lets all users preserve their network identity, overcome network censorship, +and communicate over different transport protocols. + +In overcoming these challenges while building [IPFS](https://ipfs.io), +we took care to build our solutions in a modular, composable way into what is +now libp2p. Although libp2p grew out of IPFS, it is not dependent on IPFS, and +today, [many projects][built_with_libp2p] use libp2p as their networking layer. + +Together, we can leverage our collective experience and solve these foundational +problems in a way that benefits an entire ecosystem of developers and a world of users. + +Here, we'll briefly outline the main problem areas that libp2p attempts to address. +This is an ever-growing space, so don't be surprised if things change over time. +If you notice something missing or have other ideas for improving this documentation, +please [reach out to let us know][help_improve_docs]. + +### Data transmission + +The transport layer is at the foundation of libp2p, which is responsible for +transmitting and receiving bytes between two peers. There are many +ways to send data across networks in use today, including TCP, QUIC, WebSocket, +WebTransport and WebRTC, with some still in development and others still yet +to be designed. + +libp2p also provides a list of specifications [specifcations](https://github.com/libp2p/specs) +that can be adapted to support existing and future protocols, allowing libp2p applications +to operate in many different runtime and networking environments. + +### Peer identity + +Knowing who you're talking to is key to secure and reliable communication in a world +with billions of networked devices. libp2p uses +[public key cryptography](https://en.wikipedia.org/wiki/Public-key_cryptography) +as the basis of peer identity, which serves two complementary purposes. + +1. It gives each peer a globally unique "name", in the form of a + [PeerId][definition_peerid]. +2. The `PeerId` allows anyone to retrieve the public key for the identified + peer, which enables secure communication between peers. + +### Secure communication + +There needs to be a method to securely send and receive data between peers, +where peers are able to trust the [identity](#peer-identity) of the peer they're +communicating with while ensuring that no external entity can access or tamper with +the communication. + +All libp2p connections are encrypted and authenticated. Some [transport protocol](#transport) +protocols are encrypted at the transport layer (e.g. QUIC). For other protocols, libp2p runs +a cryptographic handshake on top of an unencrypted connection (e.g. TCP). + +For secure communication channels, libp2p currently supports +[TLS 1.3](https://www.ietf.org/blog/tls13/) and [Noise](https://noiseprotocol.org/), +though not every language implementation of libp2p supports both of these. + +> (Older versions of libp2p may support a +> [deprecated](https://blog.ipfs.io/2020-08-07-deprecating-secio/) protocol called SECIO; +> all projects should switch to TLS 1.3 or Noise instead.) + +### Peer routing + +When you want to send a message to another peer, you need two key pieces +of information: their [PeerId][definition_peerid], and a way to locate them +on the network to open a connection. + +There are many cases where we only have the `PeerId` for the peer we want to +contact, and we need a way to discover their network address. Peer routing is +the process of discovering peer addresses by leveraging the knowledge of other +peers. + +In a peer routing system, a peer can either give us the address we need if they +have it, or else send our inquiry to another peer who's more likely to have the +answer. As we contact more and more peers, we not only increase our chances of +finding the peer we're looking for, we build a more complete view of the network +in our own routing tables, which enables us to answer routing queries from others. + +The current stable implementation of peer routing in libp2p uses a +[distributed hash table][definition_dht] to iteratively route requests closer +to the desired `PeerId` using the [Kademlia][wiki_kademlia] routing algorithm. + +### Content discovery + +In some systems, we care less about who we're speaking with than what they can offer us. +For example, we may want some specific piece of data, but we don't care who we get it from +since we can verify its integrity. + +libp2p provides a [content routing specification][spec_content_routing] for this +purpose, with the primary stable implementation using the same +[Kademlia][wiki_kademlia]-based DHT as used in peer routing. + +### Peer messaging + +Sending messages to other peers is at the heart of most peer-to-peer systems, +and pubsub (short for publish/subscribe) is an instrumental pattern for sending +a message to groups of interested receivers. + +libp2p defines a [pubsub specification][spec_pubsub] for sending messages to all +peers subscribed to a given "topic". The specification currently has two stable +implementations; `floodsub` uses a very simple but inefficient "network flooding" +strategy, and [gossipsub](https://github.com/libp2p/specs/tree/master/pubsub/gossipsub) +defines an extensible gossip protocol. There is also active development in progress on +[episub](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/episub.md), an +extended `gossipsub` that is optimized for single source multicast and scenarios with a +few fixed sources broadcasting to a large number of clients in a topic. + +Head over to [What is libp2p?](/introduction/what-is-libp2p/) for an introduction to +the basics of libp2p and an overview of the problems it addresses. + +## Related projects + +libp2p began as part of the [IPFS](https://ipfs.io) project and is still an +essential component of IPFS. As such, libp2p composes well with the abstractions +and tools provided by other projects in the IPFS "family". Check their sites for +specific information and references: + +- [IPFS](https://libp2p.io) is the InterPlanetary File System, which uses libp2p as + its networking layer. +- [Multiformats](https://multiformats.io) is a variety of *self-describing* data formats. +- [IPLD](https://ipld.io) is a set of tools for describing links between content-addressed + data, like IPFS files, Git commits, or Ethereum blocks. +- [The Permissive License Stack](https://protocol.ai/blog/announcing-the-permissive-license-stack) + is a licensing strategy for software development that embraces open-source values. + +[glossary]: {{< ref "/reference/glossary.md" >}} +[definition_dht]: {{< ref "/reference/glossary.md#dht" >}} +[definition_p2p]: {{< ref "/reference/glossary.md#p2p" >}} +[definition_peer]: {{< ref "/reference/glossary.md#peer" >}} +[definition_peerid]: {{< ref "/reference/glossary.md#peerid" >}} +[definition_secio]: {{< ref "/reference/glossary.md#secio" >}} +[definition_muiltiaddress]: {{< ref "/reference/glossary.md#multiaddr" >}} +[definition_client_server]: {{< ref "/reference/glossary.md#client-server" >}} + +[spec_content_routing]: https://github.com/libp2p/specs/blob/master/kad-dht/README.md +[spec_pubsub]: https://github.com/libp2p/specs/blob/master/pubsub/README.md +[built_with_libp2p]: https://discuss.libp2p.io/c/ecosystem-community +[help_improve_docs]: https://github.com/libp2p/docs/issues +[wiki_kademlia]: https://en.wikipedia.org/wiki/Kademlia diff --git a/content/introduction/network-basics.md b/content/introduction/network-basics.md new file mode 100644 index 00000000..4c519dfd --- /dev/null +++ b/content/introduction/network-basics.md @@ -0,0 +1,7 @@ +--- +title: "Network Basics" +weight: 1 +pre: ' ' +--- + +This section is coming soon! From f8704e2916dd65105afeecaa450eb884861a2c8c Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Wed, 7 Sep 2022 05:28:15 -0400 Subject: [PATCH 04/30] add placeholder to examples sec --- content/guides/examples/_index.md | 3 --- content/guides/examples/network.md | 7 +++++++ content/guides/tutorials/go.md | 4 ++-- content/guides/tutorials/rust.md | 2 +- 4 files changed, 10 insertions(+), 6 deletions(-) create mode 100644 content/guides/examples/network.md diff --git a/content/guides/examples/_index.md b/content/guides/examples/_index.md index f3eddb56..d9d47ab3 100644 --- a/content/guides/examples/_index.md +++ b/content/guides/examples/_index.md @@ -10,6 +10,3 @@ for each of its main implementations: * For go, see the [go-libp2p-examples repo](https://github.com/libp2p/go-libp2p/tree/master/examples). * For javascript, see the [/examples directory of the js-libp2p repo](https://github.com/libp2p/js-libp2p/tree/master/examples). * For rust, see [/examples directory of the rust-libp2p repo](https://github.com/libp2p/rust-libp2p/tree/master/examples). - -{{% children description="true" %}} ------ diff --git a/content/guides/examples/network.md b/content/guides/examples/network.md new file mode 100644 index 00000000..c484e65d --- /dev/null +++ b/content/guides/examples/network.md @@ -0,0 +1,7 @@ +--- +title: "Getting Started with libp2p" +menuTitle: Start Networking +weight: 1 +--- + +Stay tuned for libp2p examples! diff --git a/content/guides/tutorials/go.md b/content/guides/tutorials/go.md index 3d8dafd6..4d1e69a4 100644 --- a/content/guides/tutorials/go.md +++ b/content/guides/tutorials/go.md @@ -1,7 +1,7 @@ --- title: "Getting Started with go-libp2p" -menuTitle: Golang -weight: 1 +menuTitle: Go +weight: 2 --- This is the first in a series of tutorials on working with libp2p’s Go implementation, diff --git a/content/guides/tutorials/rust.md b/content/guides/tutorials/rust.md index e0c3898c..a5ed7ad0 100644 --- a/content/guides/tutorials/rust.md +++ b/content/guides/tutorials/rust.md @@ -1,7 +1,7 @@ --- title: "Getting Started with rust-libp2p" menuTitle: Rust -weight: 3 +weight: 2 --- Check out [tutorials of the Rust libp2p From 6501faeb2faf99555827feb74550ba68e10a1ca4 Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Wed, 7 Sep 2022 23:22:57 -0400 Subject: [PATCH 05/30] new struct for intro and core comp --- content/_index.md | 3 +++ content/concepts/_index.md | 2 +- content/concepts/content-routing.md | 8 -------- content/concepts/data-exchange/_index.md | 18 ++++++++++++++++++ content/concepts/data-exchange/bitswap.md | 11 +++++++++++ content/concepts/data-exchange/overview.md | 5 +++++ content/concepts/discovery/_index.md | 18 ++++++++++++++++++ content/concepts/discovery/mdns.md | 11 +++++++++++ content/concepts/discovery/overview.md | 11 +++++++++++ content/concepts/discovery/random-walk.md | 11 +++++++++++ content/concepts/discovery/rendezvous.md | 11 +++++++++++ content/concepts/introduction/_index.md | 18 ++++++++++++++++++ .../concepts/{ => introduction}/addressing.md | 3 ++- content/concepts/introduction/overview.md | 11 +++++++++++ .../concepts/{ => introduction}/protocols.md | 3 ++- content/concepts/messaging/_index.md | 18 ++++++++++++++++++ content/concepts/messaging/overview.md | 11 +++++++++++ .../{ => messaging}/publish-subscribe.md | 2 ++ .../publish-subscribe/fanout_forget.png | Bin .../publish-subscribe/fanout_forget.svg | 0 .../fanout_grafting_preference.png | Bin .../fanout_grafting_preference.svg | 0 .../publish-subscribe/fanout_initial_pick.png | Bin .../publish-subscribe/fanout_initial_pick.svg | 0 .../publish-subscribe/fanout_message_send.png | Bin .../publish-subscribe/fanout_message_send.svg | 0 .../full_message_forward.png | Bin .../full_message_forward.svg | 0 .../full_message_network.png | Bin .../full_message_network.svg | 0 .../publish-subscribe/full_message_send.png | Bin .../publish-subscribe/full_message_send.svg | 0 .../publish-subscribe/gossip_deliver.png | Bin .../publish-subscribe/gossip_deliver.svg | 0 .../publish-subscribe/graft_prune.png | Bin .../publish-subscribe/graft_prune.svg | 0 .../publish-subscribe/maintain_graft.png | Bin .../publish-subscribe/maintain_graft.svg | 0 .../publish-subscribe/maintain_prune.png | Bin .../publish-subscribe/maintain_prune.svg | 0 .../message_delivered_to_all.png | Bin .../message_delivered_to_all.svg | 0 .../metadata_only_network.png | Bin .../metadata_only_network.svg | 0 .../network_packet_structure.png | Bin .../network_packet_structure.svg | 0 .../request_gossiped_message.png | Bin .../request_gossiped_message.svg | 0 .../publish-subscribe/state.png | Bin .../publish-subscribe/state.svg | 0 .../publish-subscribe/subscribe_graft.png | Bin .../publish-subscribe/subscribe_graft.svg | 0 .../publish-subscribe/subscribed_peers.png | Bin .../publish-subscribe/subscribed_peers.svg | 0 .../subscription_list_change.png | Bin .../subscription_list_change.svg | 0 .../subscription_list_first_connect.png | Bin .../subscription_list_first_connect.svg | 0 .../subscriptions_local_view.png | Bin .../subscriptions_local_view.svg | 0 .../publish-subscribe/types_of_peering.png | Bin .../publish-subscribe/types_of_peering.svg | 0 .../publish-subscribe/unsubscribe_prune.png | Bin .../publish-subscribe/unsubscribe_prune.svg | 0 content/concepts/multiplex/_index.md | 18 ++++++++++++++++++ content/concepts/multiplex/mplex.md | 11 +++++++++++ .../{ => multiplex}/stream-multiplexing.md | 4 +++- content/concepts/multiplex/yamux.md | 11 +++++++++++ content/concepts/nat/_index.md | 18 ++++++++++++++++++ content/concepts/nat/autonat.md | 11 +++++++++++ content/concepts/{ => nat}/circuit-relay.md | 3 ++- content/concepts/nat/dcutr.md | 11 +++++++++++ content/concepts/nat/hole-punching.md | 11 +++++++++++ content/concepts/{ => nat}/nat.md | 5 +++-- content/concepts/peer-routing.md | 8 -------- content/concepts/routing/_index.md | 18 ++++++++++++++++++ .../routing/content-routing/_index.md | 11 +++++++++++ .../routing/content-routing/kaddht.md | 11 +++++++++++ content/concepts/routing/overview.md | 5 +++++ content/concepts/{ => routing}/peer-id.md | 5 +++-- .../concepts/routing/peer-routing/_index.md | 11 +++++++++++ .../concepts/routing/peer-routing/kaddht.md | 11 +++++++++++ content/concepts/secure-comm/_index.md | 18 ++++++++++++++++++ content/concepts/secure-comm/noise.md | 11 +++++++++++ content/concepts/secure-comm/secure-comms.md | 11 +++++++++++ content/concepts/secure-comm/tls.md | 11 +++++++++++ content/concepts/secure-comms.md | 8 -------- content/concepts/security/_index.md | 18 ++++++++++++++++++ .../{ => security}/security-considerations.md | 5 +++-- content/concepts/transports/_index.md | 18 ++++++++++++++++++ content/concepts/transports/quic.md | 11 +++++++++++ content/concepts/transports/tcp.md | 11 +++++++++++ .../concepts/{ => transports}/transport.md | 3 ++- content/concepts/transports/webrtc.md | 11 +++++++++++ content/concepts/transports/websocket.md | 11 +++++++++++ content/concepts/transports/webtransport.md | 11 +++++++++++ content/introduction/_index.md | 2 +- content/introduction/libp2p/_index.md | 14 ++++++++++++++ .../{ => libp2p}/getting-started.md | 6 ++---- content/introduction/network-basics.md | 7 ------- content/introduction/network-basics/_index.md | 13 +++++++++++++ .../network-basics/computer-networks.md | 11 +++++++++++ .../network-basics/data-structures.md | 11 +++++++++++ .../network-basics/distributed-computing.md | 11 +++++++++++ .../network-basics/internet-web.md | 11 +++++++++++ 105 files changed, 554 insertions(+), 48 deletions(-) delete mode 100644 content/concepts/content-routing.md create mode 100644 content/concepts/data-exchange/_index.md create mode 100644 content/concepts/data-exchange/bitswap.md create mode 100644 content/concepts/data-exchange/overview.md create mode 100644 content/concepts/discovery/_index.md create mode 100644 content/concepts/discovery/mdns.md create mode 100644 content/concepts/discovery/overview.md create mode 100644 content/concepts/discovery/random-walk.md create mode 100644 content/concepts/discovery/rendezvous.md create mode 100644 content/concepts/introduction/_index.md rename content/concepts/{ => introduction}/addressing.md (97%) create mode 100644 content/concepts/introduction/overview.md rename content/concepts/{ => introduction}/protocols.md (99%) create mode 100644 content/concepts/messaging/_index.md create mode 100644 content/concepts/messaging/overview.md rename content/concepts/{ => messaging}/publish-subscribe.md (99%) rename content/concepts/{ => messaging}/publish-subscribe/fanout_forget.png (100%) rename content/concepts/{ => messaging}/publish-subscribe/fanout_forget.svg (100%) rename content/concepts/{ => messaging}/publish-subscribe/fanout_grafting_preference.png (100%) rename content/concepts/{ => messaging}/publish-subscribe/fanout_grafting_preference.svg (100%) rename content/concepts/{ => messaging}/publish-subscribe/fanout_initial_pick.png (100%) rename content/concepts/{ => messaging}/publish-subscribe/fanout_initial_pick.svg (100%) rename content/concepts/{ => messaging}/publish-subscribe/fanout_message_send.png (100%) rename content/concepts/{ => messaging}/publish-subscribe/fanout_message_send.svg (100%) rename content/concepts/{ => messaging}/publish-subscribe/full_message_forward.png (100%) rename content/concepts/{ => messaging}/publish-subscribe/full_message_forward.svg (100%) rename content/concepts/{ => messaging}/publish-subscribe/full_message_network.png (100%) rename content/concepts/{ => messaging}/publish-subscribe/full_message_network.svg (100%) rename content/concepts/{ => messaging}/publish-subscribe/full_message_send.png (100%) rename content/concepts/{ => messaging}/publish-subscribe/full_message_send.svg (100%) rename content/concepts/{ => messaging}/publish-subscribe/gossip_deliver.png (100%) rename content/concepts/{ => messaging}/publish-subscribe/gossip_deliver.svg (100%) rename content/concepts/{ => messaging}/publish-subscribe/graft_prune.png (100%) rename content/concepts/{ => messaging}/publish-subscribe/graft_prune.svg (100%) rename content/concepts/{ => messaging}/publish-subscribe/maintain_graft.png (100%) rename content/concepts/{ => messaging}/publish-subscribe/maintain_graft.svg (100%) rename content/concepts/{ => messaging}/publish-subscribe/maintain_prune.png (100%) rename content/concepts/{ => messaging}/publish-subscribe/maintain_prune.svg (100%) rename content/concepts/{ => messaging}/publish-subscribe/message_delivered_to_all.png (100%) rename content/concepts/{ => messaging}/publish-subscribe/message_delivered_to_all.svg (100%) rename content/concepts/{ => messaging}/publish-subscribe/metadata_only_network.png (100%) rename content/concepts/{ => messaging}/publish-subscribe/metadata_only_network.svg (100%) rename content/concepts/{ => messaging}/publish-subscribe/network_packet_structure.png (100%) rename content/concepts/{ => messaging}/publish-subscribe/network_packet_structure.svg (100%) rename content/concepts/{ => messaging}/publish-subscribe/request_gossiped_message.png (100%) rename content/concepts/{ => messaging}/publish-subscribe/request_gossiped_message.svg (100%) rename content/concepts/{ => messaging}/publish-subscribe/state.png (100%) rename content/concepts/{ => messaging}/publish-subscribe/state.svg (100%) rename content/concepts/{ => messaging}/publish-subscribe/subscribe_graft.png (100%) rename content/concepts/{ => messaging}/publish-subscribe/subscribe_graft.svg (100%) rename content/concepts/{ => messaging}/publish-subscribe/subscribed_peers.png (100%) rename content/concepts/{ => messaging}/publish-subscribe/subscribed_peers.svg (100%) rename content/concepts/{ => messaging}/publish-subscribe/subscription_list_change.png (100%) rename content/concepts/{ => messaging}/publish-subscribe/subscription_list_change.svg (100%) rename content/concepts/{ => messaging}/publish-subscribe/subscription_list_first_connect.png (100%) rename content/concepts/{ => messaging}/publish-subscribe/subscription_list_first_connect.svg (100%) rename content/concepts/{ => messaging}/publish-subscribe/subscriptions_local_view.png (100%) rename content/concepts/{ => messaging}/publish-subscribe/subscriptions_local_view.svg (100%) rename content/concepts/{ => messaging}/publish-subscribe/types_of_peering.png (100%) rename content/concepts/{ => messaging}/publish-subscribe/types_of_peering.svg (100%) rename content/concepts/{ => messaging}/publish-subscribe/unsubscribe_prune.png (100%) rename content/concepts/{ => messaging}/publish-subscribe/unsubscribe_prune.svg (100%) create mode 100644 content/concepts/multiplex/_index.md create mode 100644 content/concepts/multiplex/mplex.md rename content/concepts/{ => multiplex}/stream-multiplexing.md (98%) create mode 100644 content/concepts/multiplex/yamux.md create mode 100644 content/concepts/nat/_index.md create mode 100644 content/concepts/nat/autonat.md rename content/concepts/{ => nat}/circuit-relay.md (98%) create mode 100644 content/concepts/nat/dcutr.md create mode 100644 content/concepts/nat/hole-punching.md rename content/concepts/{ => nat}/nat.md (98%) delete mode 100644 content/concepts/peer-routing.md create mode 100644 content/concepts/routing/_index.md create mode 100644 content/concepts/routing/content-routing/_index.md create mode 100644 content/concepts/routing/content-routing/kaddht.md create mode 100644 content/concepts/routing/overview.md rename content/concepts/{ => routing}/peer-id.md (97%) create mode 100644 content/concepts/routing/peer-routing/_index.md create mode 100644 content/concepts/routing/peer-routing/kaddht.md create mode 100644 content/concepts/secure-comm/_index.md create mode 100644 content/concepts/secure-comm/noise.md create mode 100644 content/concepts/secure-comm/secure-comms.md create mode 100644 content/concepts/secure-comm/tls.md delete mode 100644 content/concepts/secure-comms.md create mode 100644 content/concepts/security/_index.md rename content/concepts/{ => security}/security-considerations.md (99%) create mode 100644 content/concepts/transports/_index.md create mode 100644 content/concepts/transports/quic.md create mode 100644 content/concepts/transports/tcp.md rename content/concepts/{ => transports}/transport.md (98%) create mode 100644 content/concepts/transports/webrtc.md create mode 100644 content/concepts/transports/websocket.md create mode 100644 content/concepts/transports/webtransport.md create mode 100644 content/introduction/libp2p/_index.md rename content/introduction/{ => libp2p}/getting-started.md (98%) delete mode 100644 content/introduction/network-basics.md create mode 100644 content/introduction/network-basics/_index.md create mode 100644 content/introduction/network-basics/computer-networks.md create mode 100644 content/introduction/network-basics/data-structures.md create mode 100644 content/introduction/network-basics/distributed-computing.md create mode 100644 content/introduction/network-basics/internet-web.md diff --git a/content/_index.md b/content/_index.md index 1766f132..53d06a36 100644 --- a/content/_index.md +++ b/content/_index.md @@ -11,6 +11,9 @@ Welcome to the official libp2p documentation hub! ### Chapter 1 +For new comers and those interested to understand key computer network concepts, start +here. + ### Chapter 2 ### Chapter 3 diff --git a/content/concepts/_index.md b/content/concepts/_index.md index cd9cf676..3d9e818d 100644 --- a/content/concepts/_index.md +++ b/content/concepts/_index.md @@ -1,5 +1,5 @@ --- -title: "Explore libp2p" +title: "Core Components" weight: 2 pre: "2. " chapter: true diff --git a/content/concepts/content-routing.md b/content/concepts/content-routing.md deleted file mode 100644 index 8d212cbc..00000000 --- a/content/concepts/content-routing.md +++ /dev/null @@ -1,8 +0,0 @@ ---- -title: Content Routing -weight: 5 ---- - -This article is coming soon! - -Please [refer to this issue](https://github.com/libp2p/docs/issues/23) to track the progress and make suggestions. diff --git a/content/concepts/data-exchange/_index.md b/content/concepts/data-exchange/_index.md new file mode 100644 index 00000000..df9ebfe9 --- /dev/null +++ b/content/concepts/data-exchange/_index.md @@ -0,0 +1,18 @@ +--- +title: "Data Exchange" +weight: 9 +pre: ' 2.8. ' +chapter: true +--- + +### Chapter 2.8 + +# Exchange Data + +{{% children %}} + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/concepts/data-exchange/bitswap.md b/content/concepts/data-exchange/bitswap.md new file mode 100644 index 00000000..2b20888c --- /dev/null +++ b/content/concepts/data-exchange/bitswap.md @@ -0,0 +1,11 @@ +--- +title: "Bitswap" +weight: 8 +pre: ' 2.8.1. ' +--- + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/concepts/data-exchange/overview.md b/content/concepts/data-exchange/overview.md new file mode 100644 index 00000000..ac9cc626 --- /dev/null +++ b/content/concepts/data-exchange/overview.md @@ -0,0 +1,5 @@ +--- +title: 'Overview' +weight: 1 +pre: ' 2.9.1. ' +--- diff --git a/content/concepts/discovery/_index.md b/content/concepts/discovery/_index.md new file mode 100644 index 00000000..1efeba4c --- /dev/null +++ b/content/concepts/discovery/_index.md @@ -0,0 +1,18 @@ +--- +title: "Discovery" +weight: 6 +pre: ' 2.5. ' +chapter: true +--- + +### Chapter 2.5 + +# Discover Peers + +{{% children %}} + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/concepts/discovery/mdns.md b/content/concepts/discovery/mdns.md new file mode 100644 index 00000000..8f87076a --- /dev/null +++ b/content/concepts/discovery/mdns.md @@ -0,0 +1,11 @@ +--- +title: mDNS +weight: 2 +pre: ' 2.5.2. ' +--- + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/concepts/discovery/overview.md b/content/concepts/discovery/overview.md new file mode 100644 index 00000000..b636f264 --- /dev/null +++ b/content/concepts/discovery/overview.md @@ -0,0 +1,11 @@ +--- +title: "Overview" +weight: 1 +pre: ' 2.5.1. ' +--- + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/concepts/discovery/random-walk.md b/content/concepts/discovery/random-walk.md new file mode 100644 index 00000000..4bea2bcb --- /dev/null +++ b/content/concepts/discovery/random-walk.md @@ -0,0 +1,11 @@ +--- +title: Random Walk +weight: 3 +pre: ' 2.5.3. ' +--- + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/concepts/discovery/rendezvous.md b/content/concepts/discovery/rendezvous.md new file mode 100644 index 00000000..ac57cd01 --- /dev/null +++ b/content/concepts/discovery/rendezvous.md @@ -0,0 +1,11 @@ +--- +title: Rendezvous +weight: 4 +pre: ' 2.5.4. ' +--- + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/concepts/introduction/_index.md b/content/concepts/introduction/_index.md new file mode 100644 index 00000000..2c53c1e5 --- /dev/null +++ b/content/concepts/introduction/_index.md @@ -0,0 +1,18 @@ +--- +title: "Introduction" +weight: 1 +pre: ' 2.0. ' +chapter: true +--- + +### Chapter 2.0 + +# Key Components of libp2p + +{{% children %}} + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/concepts/addressing.md b/content/concepts/introduction/addressing.md similarity index 97% rename from content/concepts/addressing.md rename to content/concepts/introduction/addressing.md index 77526533..ff93f377 100644 --- a/content/concepts/addressing.md +++ b/content/concepts/introduction/addressing.md @@ -1,6 +1,7 @@ --- title: Addressing -weight: 6 +weight: 1 +pre: ' 2.0.2. ' --- Flexible networks need flexible addressing systems. Since libp2p is designed to work across a wide variety of networks, we need a way to work with a lot of different addressing schemes in a consistent way. diff --git a/content/concepts/introduction/overview.md b/content/concepts/introduction/overview.md new file mode 100644 index 00000000..8792d150 --- /dev/null +++ b/content/concepts/introduction/overview.md @@ -0,0 +1,11 @@ +--- +title: "Overview" +weight: 2 +pre: ' 2.0.1 ' +--- + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/concepts/protocols.md b/content/concepts/introduction/protocols.md similarity index 99% rename from content/concepts/protocols.md rename to content/concepts/introduction/protocols.md index 62e647f7..c6ebfe77 100644 --- a/content/concepts/protocols.md +++ b/content/concepts/introduction/protocols.md @@ -1,6 +1,7 @@ --- title: Protocols -weight: 3 +weight: 2 +pre: ' 2.0.3. ' --- There are protocols everywhere you look when you're writing network applications, and libp2p is diff --git a/content/concepts/messaging/_index.md b/content/concepts/messaging/_index.md new file mode 100644 index 00000000..47a1e789 --- /dev/null +++ b/content/concepts/messaging/_index.md @@ -0,0 +1,18 @@ +--- +title: "Messaging" +weight: 8 +pre: ' 2.7. ' +chapter: true +--- + +### Chapter 2.7 + +# Transmit Data + +{{% children %}} + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/concepts/messaging/overview.md b/content/concepts/messaging/overview.md new file mode 100644 index 00000000..eb6b7898 --- /dev/null +++ b/content/concepts/messaging/overview.md @@ -0,0 +1,11 @@ +--- +title: Overview +weight: 1 +pre: ' 2.7.1. ' +--- + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/concepts/publish-subscribe.md b/content/concepts/messaging/publish-subscribe.md similarity index 99% rename from content/concepts/publish-subscribe.md rename to content/concepts/messaging/publish-subscribe.md index 376ef448..c3c9cb78 100644 --- a/content/concepts/publish-subscribe.md +++ b/content/concepts/messaging/publish-subscribe.md @@ -1,5 +1,7 @@ --- title: Publish/Subscribe +weight: 2 +pre: ' 2.7.2 ' --- Publish/Subscribe is a system where peers congregate around topics they are diff --git a/content/concepts/publish-subscribe/fanout_forget.png b/content/concepts/messaging/publish-subscribe/fanout_forget.png similarity index 100% rename from content/concepts/publish-subscribe/fanout_forget.png rename to content/concepts/messaging/publish-subscribe/fanout_forget.png diff --git a/content/concepts/publish-subscribe/fanout_forget.svg b/content/concepts/messaging/publish-subscribe/fanout_forget.svg similarity index 100% rename from content/concepts/publish-subscribe/fanout_forget.svg rename to content/concepts/messaging/publish-subscribe/fanout_forget.svg diff --git a/content/concepts/publish-subscribe/fanout_grafting_preference.png b/content/concepts/messaging/publish-subscribe/fanout_grafting_preference.png similarity index 100% rename from content/concepts/publish-subscribe/fanout_grafting_preference.png rename to content/concepts/messaging/publish-subscribe/fanout_grafting_preference.png diff --git a/content/concepts/publish-subscribe/fanout_grafting_preference.svg b/content/concepts/messaging/publish-subscribe/fanout_grafting_preference.svg similarity index 100% rename from content/concepts/publish-subscribe/fanout_grafting_preference.svg rename to content/concepts/messaging/publish-subscribe/fanout_grafting_preference.svg diff --git a/content/concepts/publish-subscribe/fanout_initial_pick.png b/content/concepts/messaging/publish-subscribe/fanout_initial_pick.png similarity index 100% rename from content/concepts/publish-subscribe/fanout_initial_pick.png rename to content/concepts/messaging/publish-subscribe/fanout_initial_pick.png diff --git a/content/concepts/publish-subscribe/fanout_initial_pick.svg b/content/concepts/messaging/publish-subscribe/fanout_initial_pick.svg similarity index 100% rename from content/concepts/publish-subscribe/fanout_initial_pick.svg rename to content/concepts/messaging/publish-subscribe/fanout_initial_pick.svg diff --git a/content/concepts/publish-subscribe/fanout_message_send.png b/content/concepts/messaging/publish-subscribe/fanout_message_send.png similarity index 100% rename from content/concepts/publish-subscribe/fanout_message_send.png rename to content/concepts/messaging/publish-subscribe/fanout_message_send.png diff --git a/content/concepts/publish-subscribe/fanout_message_send.svg b/content/concepts/messaging/publish-subscribe/fanout_message_send.svg similarity index 100% rename from content/concepts/publish-subscribe/fanout_message_send.svg rename to content/concepts/messaging/publish-subscribe/fanout_message_send.svg diff --git a/content/concepts/publish-subscribe/full_message_forward.png b/content/concepts/messaging/publish-subscribe/full_message_forward.png similarity index 100% rename from content/concepts/publish-subscribe/full_message_forward.png rename to content/concepts/messaging/publish-subscribe/full_message_forward.png diff --git a/content/concepts/publish-subscribe/full_message_forward.svg b/content/concepts/messaging/publish-subscribe/full_message_forward.svg similarity index 100% rename from content/concepts/publish-subscribe/full_message_forward.svg rename to content/concepts/messaging/publish-subscribe/full_message_forward.svg diff --git a/content/concepts/publish-subscribe/full_message_network.png b/content/concepts/messaging/publish-subscribe/full_message_network.png similarity index 100% rename from content/concepts/publish-subscribe/full_message_network.png rename to content/concepts/messaging/publish-subscribe/full_message_network.png diff --git a/content/concepts/publish-subscribe/full_message_network.svg b/content/concepts/messaging/publish-subscribe/full_message_network.svg similarity index 100% rename from content/concepts/publish-subscribe/full_message_network.svg rename to content/concepts/messaging/publish-subscribe/full_message_network.svg diff --git a/content/concepts/publish-subscribe/full_message_send.png b/content/concepts/messaging/publish-subscribe/full_message_send.png similarity index 100% rename from content/concepts/publish-subscribe/full_message_send.png rename to content/concepts/messaging/publish-subscribe/full_message_send.png diff --git a/content/concepts/publish-subscribe/full_message_send.svg b/content/concepts/messaging/publish-subscribe/full_message_send.svg similarity index 100% rename from content/concepts/publish-subscribe/full_message_send.svg rename to content/concepts/messaging/publish-subscribe/full_message_send.svg diff --git a/content/concepts/publish-subscribe/gossip_deliver.png b/content/concepts/messaging/publish-subscribe/gossip_deliver.png similarity index 100% rename from content/concepts/publish-subscribe/gossip_deliver.png rename to content/concepts/messaging/publish-subscribe/gossip_deliver.png diff --git a/content/concepts/publish-subscribe/gossip_deliver.svg b/content/concepts/messaging/publish-subscribe/gossip_deliver.svg similarity index 100% rename from content/concepts/publish-subscribe/gossip_deliver.svg rename to content/concepts/messaging/publish-subscribe/gossip_deliver.svg diff --git a/content/concepts/publish-subscribe/graft_prune.png b/content/concepts/messaging/publish-subscribe/graft_prune.png similarity index 100% rename from content/concepts/publish-subscribe/graft_prune.png rename to content/concepts/messaging/publish-subscribe/graft_prune.png diff --git a/content/concepts/publish-subscribe/graft_prune.svg b/content/concepts/messaging/publish-subscribe/graft_prune.svg similarity index 100% rename from content/concepts/publish-subscribe/graft_prune.svg rename to content/concepts/messaging/publish-subscribe/graft_prune.svg diff --git a/content/concepts/publish-subscribe/maintain_graft.png b/content/concepts/messaging/publish-subscribe/maintain_graft.png similarity index 100% rename from content/concepts/publish-subscribe/maintain_graft.png rename to content/concepts/messaging/publish-subscribe/maintain_graft.png diff --git a/content/concepts/publish-subscribe/maintain_graft.svg b/content/concepts/messaging/publish-subscribe/maintain_graft.svg similarity index 100% rename from content/concepts/publish-subscribe/maintain_graft.svg rename to content/concepts/messaging/publish-subscribe/maintain_graft.svg diff --git a/content/concepts/publish-subscribe/maintain_prune.png b/content/concepts/messaging/publish-subscribe/maintain_prune.png similarity index 100% rename from content/concepts/publish-subscribe/maintain_prune.png rename to content/concepts/messaging/publish-subscribe/maintain_prune.png diff --git a/content/concepts/publish-subscribe/maintain_prune.svg b/content/concepts/messaging/publish-subscribe/maintain_prune.svg similarity index 100% rename from content/concepts/publish-subscribe/maintain_prune.svg rename to content/concepts/messaging/publish-subscribe/maintain_prune.svg diff --git a/content/concepts/publish-subscribe/message_delivered_to_all.png b/content/concepts/messaging/publish-subscribe/message_delivered_to_all.png similarity index 100% rename from content/concepts/publish-subscribe/message_delivered_to_all.png rename to content/concepts/messaging/publish-subscribe/message_delivered_to_all.png diff --git a/content/concepts/publish-subscribe/message_delivered_to_all.svg b/content/concepts/messaging/publish-subscribe/message_delivered_to_all.svg similarity index 100% rename from content/concepts/publish-subscribe/message_delivered_to_all.svg rename to content/concepts/messaging/publish-subscribe/message_delivered_to_all.svg diff --git a/content/concepts/publish-subscribe/metadata_only_network.png b/content/concepts/messaging/publish-subscribe/metadata_only_network.png similarity index 100% rename from content/concepts/publish-subscribe/metadata_only_network.png rename to content/concepts/messaging/publish-subscribe/metadata_only_network.png diff --git a/content/concepts/publish-subscribe/metadata_only_network.svg b/content/concepts/messaging/publish-subscribe/metadata_only_network.svg similarity index 100% rename from content/concepts/publish-subscribe/metadata_only_network.svg rename to content/concepts/messaging/publish-subscribe/metadata_only_network.svg diff --git a/content/concepts/publish-subscribe/network_packet_structure.png b/content/concepts/messaging/publish-subscribe/network_packet_structure.png similarity index 100% rename from content/concepts/publish-subscribe/network_packet_structure.png rename to content/concepts/messaging/publish-subscribe/network_packet_structure.png diff --git a/content/concepts/publish-subscribe/network_packet_structure.svg b/content/concepts/messaging/publish-subscribe/network_packet_structure.svg similarity index 100% rename from content/concepts/publish-subscribe/network_packet_structure.svg rename to content/concepts/messaging/publish-subscribe/network_packet_structure.svg diff --git a/content/concepts/publish-subscribe/request_gossiped_message.png b/content/concepts/messaging/publish-subscribe/request_gossiped_message.png similarity index 100% rename from content/concepts/publish-subscribe/request_gossiped_message.png rename to content/concepts/messaging/publish-subscribe/request_gossiped_message.png diff --git a/content/concepts/publish-subscribe/request_gossiped_message.svg b/content/concepts/messaging/publish-subscribe/request_gossiped_message.svg similarity index 100% rename from content/concepts/publish-subscribe/request_gossiped_message.svg rename to content/concepts/messaging/publish-subscribe/request_gossiped_message.svg diff --git a/content/concepts/publish-subscribe/state.png b/content/concepts/messaging/publish-subscribe/state.png similarity index 100% rename from content/concepts/publish-subscribe/state.png rename to content/concepts/messaging/publish-subscribe/state.png diff --git a/content/concepts/publish-subscribe/state.svg b/content/concepts/messaging/publish-subscribe/state.svg similarity index 100% rename from content/concepts/publish-subscribe/state.svg rename to content/concepts/messaging/publish-subscribe/state.svg diff --git a/content/concepts/publish-subscribe/subscribe_graft.png b/content/concepts/messaging/publish-subscribe/subscribe_graft.png similarity index 100% rename from content/concepts/publish-subscribe/subscribe_graft.png rename to content/concepts/messaging/publish-subscribe/subscribe_graft.png diff --git a/content/concepts/publish-subscribe/subscribe_graft.svg b/content/concepts/messaging/publish-subscribe/subscribe_graft.svg similarity index 100% rename from content/concepts/publish-subscribe/subscribe_graft.svg rename to content/concepts/messaging/publish-subscribe/subscribe_graft.svg diff --git a/content/concepts/publish-subscribe/subscribed_peers.png b/content/concepts/messaging/publish-subscribe/subscribed_peers.png similarity index 100% rename from content/concepts/publish-subscribe/subscribed_peers.png rename to content/concepts/messaging/publish-subscribe/subscribed_peers.png diff --git a/content/concepts/publish-subscribe/subscribed_peers.svg b/content/concepts/messaging/publish-subscribe/subscribed_peers.svg similarity index 100% rename from content/concepts/publish-subscribe/subscribed_peers.svg rename to content/concepts/messaging/publish-subscribe/subscribed_peers.svg diff --git a/content/concepts/publish-subscribe/subscription_list_change.png b/content/concepts/messaging/publish-subscribe/subscription_list_change.png similarity index 100% rename from content/concepts/publish-subscribe/subscription_list_change.png rename to content/concepts/messaging/publish-subscribe/subscription_list_change.png diff --git a/content/concepts/publish-subscribe/subscription_list_change.svg b/content/concepts/messaging/publish-subscribe/subscription_list_change.svg similarity index 100% rename from content/concepts/publish-subscribe/subscription_list_change.svg rename to content/concepts/messaging/publish-subscribe/subscription_list_change.svg diff --git a/content/concepts/publish-subscribe/subscription_list_first_connect.png b/content/concepts/messaging/publish-subscribe/subscription_list_first_connect.png similarity index 100% rename from content/concepts/publish-subscribe/subscription_list_first_connect.png rename to content/concepts/messaging/publish-subscribe/subscription_list_first_connect.png diff --git a/content/concepts/publish-subscribe/subscription_list_first_connect.svg b/content/concepts/messaging/publish-subscribe/subscription_list_first_connect.svg similarity index 100% rename from content/concepts/publish-subscribe/subscription_list_first_connect.svg rename to content/concepts/messaging/publish-subscribe/subscription_list_first_connect.svg diff --git a/content/concepts/publish-subscribe/subscriptions_local_view.png b/content/concepts/messaging/publish-subscribe/subscriptions_local_view.png similarity index 100% rename from content/concepts/publish-subscribe/subscriptions_local_view.png rename to content/concepts/messaging/publish-subscribe/subscriptions_local_view.png diff --git a/content/concepts/publish-subscribe/subscriptions_local_view.svg b/content/concepts/messaging/publish-subscribe/subscriptions_local_view.svg similarity index 100% rename from content/concepts/publish-subscribe/subscriptions_local_view.svg rename to content/concepts/messaging/publish-subscribe/subscriptions_local_view.svg diff --git a/content/concepts/publish-subscribe/types_of_peering.png b/content/concepts/messaging/publish-subscribe/types_of_peering.png similarity index 100% rename from content/concepts/publish-subscribe/types_of_peering.png rename to content/concepts/messaging/publish-subscribe/types_of_peering.png diff --git a/content/concepts/publish-subscribe/types_of_peering.svg b/content/concepts/messaging/publish-subscribe/types_of_peering.svg similarity index 100% rename from content/concepts/publish-subscribe/types_of_peering.svg rename to content/concepts/messaging/publish-subscribe/types_of_peering.svg diff --git a/content/concepts/publish-subscribe/unsubscribe_prune.png b/content/concepts/messaging/publish-subscribe/unsubscribe_prune.png similarity index 100% rename from content/concepts/publish-subscribe/unsubscribe_prune.png rename to content/concepts/messaging/publish-subscribe/unsubscribe_prune.png diff --git a/content/concepts/publish-subscribe/unsubscribe_prune.svg b/content/concepts/messaging/publish-subscribe/unsubscribe_prune.svg similarity index 100% rename from content/concepts/publish-subscribe/unsubscribe_prune.svg rename to content/concepts/messaging/publish-subscribe/unsubscribe_prune.svg diff --git a/content/concepts/multiplex/_index.md b/content/concepts/multiplex/_index.md new file mode 100644 index 00000000..81489c26 --- /dev/null +++ b/content/concepts/multiplex/_index.md @@ -0,0 +1,18 @@ +--- +title: "Stream Multiplexing" +weight: 4 +pre: ' 2.3. ' +chapter: true +--- + +### Chapter 2.3 + +# Stream bytes + +{{% children %}} + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/concepts/multiplex/mplex.md b/content/concepts/multiplex/mplex.md new file mode 100644 index 00000000..c67332e8 --- /dev/null +++ b/content/concepts/multiplex/mplex.md @@ -0,0 +1,11 @@ +--- +title: "mplex" +weight: 2 +pre: ' 2.3.2. ' +--- + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/concepts/stream-multiplexing.md b/content/concepts/multiplex/stream-multiplexing.md similarity index 98% rename from content/concepts/stream-multiplexing.md rename to content/concepts/multiplex/stream-multiplexing.md index d467286d..2381a132 100644 --- a/content/concepts/stream-multiplexing.md +++ b/content/concepts/multiplex/stream-multiplexing.md @@ -1,5 +1,7 @@ --- -title: Stream Multiplexing +title: "Overview" +weight: 1 +pre: ' 2.3.1. ' --- Stream Multiplexing (_stream muxing_) is a way of sending multiple streams of data over one communication link. It combines multiple signals into one unified signal so it can be transported 'over the wires', then it is demulitiplexed (_demuxed_) so it can be output and used by separate applications. diff --git a/content/concepts/multiplex/yamux.md b/content/concepts/multiplex/yamux.md new file mode 100644 index 00000000..80615cd6 --- /dev/null +++ b/content/concepts/multiplex/yamux.md @@ -0,0 +1,11 @@ +--- +title: "yamux" +weight: 2 +pre: ' 2.3.3. ' +--- + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/concepts/nat/_index.md b/content/concepts/nat/_index.md new file mode 100644 index 00000000..29ad09d5 --- /dev/null +++ b/content/concepts/nat/_index.md @@ -0,0 +1,18 @@ +--- +title: "NAT Traversal" +weight: 5 +pre: ' 2.4. ' +chapter: true +--- + +### Chapter 2.4 + +# Traverse Bytes + +{{% children %}} + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/concepts/nat/autonat.md b/content/concepts/nat/autonat.md new file mode 100644 index 00000000..188f9688 --- /dev/null +++ b/content/concepts/nat/autonat.md @@ -0,0 +1,11 @@ +--- +title: AutoNAT +weight: 3 +pre: ' 2.4.3. ' +--- + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/concepts/circuit-relay.md b/content/concepts/nat/circuit-relay.md similarity index 98% rename from content/concepts/circuit-relay.md rename to content/concepts/nat/circuit-relay.md index 9284e73b..e368eeb1 100644 --- a/content/concepts/circuit-relay.md +++ b/content/concepts/nat/circuit-relay.md @@ -1,6 +1,7 @@ --- title: Circuit Relay -weight: 3 +weight: 2 +pre: ' 2.4.2. ' --- Circuit relay is a [transport protocol](/concepts/transport/) that routes traffic between two peers over a third-party "relay" peer. diff --git a/content/concepts/nat/dcutr.md b/content/concepts/nat/dcutr.md new file mode 100644 index 00000000..1a7dc013 --- /dev/null +++ b/content/concepts/nat/dcutr.md @@ -0,0 +1,11 @@ +--- +title: DCUtR +weight: 4 +pre: ' 2.4.4. ' +--- + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/concepts/nat/hole-punching.md b/content/concepts/nat/hole-punching.md new file mode 100644 index 00000000..6dd314fb --- /dev/null +++ b/content/concepts/nat/hole-punching.md @@ -0,0 +1,11 @@ +--- +title: Hole Punching +weight: 2 +pre: ' 2.4.5. ' +--- + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/concepts/nat.md b/content/concepts/nat/nat.md similarity index 98% rename from content/concepts/nat.md rename to content/concepts/nat/nat.md index 6d2ad6f8..28a893af 100644 --- a/content/concepts/nat.md +++ b/content/concepts/nat/nat.md @@ -1,6 +1,7 @@ --- -title: NAT Traversal -weight: 2 +title: Overview +weight: 1 +pre: ' 2.4.1. ' --- The internet is composed of countless networks, bound together into shared address spaces by foundational [transport protocols](/concepts/transport/). diff --git a/content/concepts/peer-routing.md b/content/concepts/peer-routing.md deleted file mode 100644 index 4a406b46..00000000 --- a/content/concepts/peer-routing.md +++ /dev/null @@ -1,8 +0,0 @@ ---- -title: Peer Routing -weight: 5 ---- - -This article is coming soon! - -Please [refer to this issue](https://github.com/libp2p/docs/issues/13) to track the progress and make suggestions. diff --git a/content/concepts/routing/_index.md b/content/concepts/routing/_index.md new file mode 100644 index 00000000..9a9d77da --- /dev/null +++ b/content/concepts/routing/_index.md @@ -0,0 +1,18 @@ +--- +title: "Routing" +weight: 7 +pre: ' 2.6. ' +chapter: true +--- + +### Chapter 2.6 + +# Route with Peers + +{{% children %}} + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/concepts/routing/content-routing/_index.md b/content/concepts/routing/content-routing/_index.md new file mode 100644 index 00000000..bab33726 --- /dev/null +++ b/content/concepts/routing/content-routing/_index.md @@ -0,0 +1,11 @@ +--- +title: "Content Routing" +weight: 4 +pre: ' 2.6.4. ' +--- + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/concepts/routing/content-routing/kaddht.md b/content/concepts/routing/content-routing/kaddht.md new file mode 100644 index 00000000..135a2f0a --- /dev/null +++ b/content/concepts/routing/content-routing/kaddht.md @@ -0,0 +1,11 @@ +--- +title: KadDHT-based Content Routing +weight: 4 +pre: ' 2.6.4.1. ' +--- + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/concepts/routing/overview.md b/content/concepts/routing/overview.md new file mode 100644 index 00000000..ed65621e --- /dev/null +++ b/content/concepts/routing/overview.md @@ -0,0 +1,5 @@ +--- +title: "Overview" +weight: 1 +pre: ' 2.6.1. ' +--- diff --git a/content/concepts/peer-id.md b/content/concepts/routing/peer-id.md similarity index 97% rename from content/concepts/peer-id.md rename to content/concepts/routing/peer-id.md index 3f21c76e..7dbd1777 100644 --- a/content/concepts/peer-id.md +++ b/content/concepts/routing/peer-id.md @@ -1,6 +1,7 @@ --- -title: Peer Identity -weight: 4 +title: "Peer Identity" +weight: 2 +pre: ' 2.6.2. ' --- A Peer Identity (often written `PeerId`) is a unique reference to a specific diff --git a/content/concepts/routing/peer-routing/_index.md b/content/concepts/routing/peer-routing/_index.md new file mode 100644 index 00000000..22095a7c --- /dev/null +++ b/content/concepts/routing/peer-routing/_index.md @@ -0,0 +1,11 @@ +--- +title: "Peer Routing" +weight: 3 +pre: ' 2.6.3. ' +--- + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/concepts/routing/peer-routing/kaddht.md b/content/concepts/routing/peer-routing/kaddht.md new file mode 100644 index 00000000..b74b575b --- /dev/null +++ b/content/concepts/routing/peer-routing/kaddht.md @@ -0,0 +1,11 @@ +--- +title: KadDHT-based Peer Routing +weight: 3 +pre: ' 2.6.3.1 ' +--- + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/concepts/secure-comm/_index.md b/content/concepts/secure-comm/_index.md new file mode 100644 index 00000000..a1af748e --- /dev/null +++ b/content/concepts/secure-comm/_index.md @@ -0,0 +1,18 @@ +--- +title: "Secure Communication" +weight: 3 +pre: ' 2.2. ' +chapter: true +--- + +### Chapter 2.2 + +# Secure bytes + +{{% children %}} + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/concepts/secure-comm/noise.md b/content/concepts/secure-comm/noise.md new file mode 100644 index 00000000..f5ccd051 --- /dev/null +++ b/content/concepts/secure-comm/noise.md @@ -0,0 +1,11 @@ +--- +title: Noise +weight: 2 +pre: ' 2.2.2. ' +--- + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/concepts/secure-comm/secure-comms.md b/content/concepts/secure-comm/secure-comms.md new file mode 100644 index 00000000..7eadcb06 --- /dev/null +++ b/content/concepts/secure-comm/secure-comms.md @@ -0,0 +1,11 @@ +--- +title: Overview +weight: 1 +pre: ' 2.2.1. ' +--- + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/concepts/secure-comm/tls.md b/content/concepts/secure-comm/tls.md new file mode 100644 index 00000000..5ebdafe4 --- /dev/null +++ b/content/concepts/secure-comm/tls.md @@ -0,0 +1,11 @@ +--- +title: TLS +weight: 3 +pre: ' 2.2.3. ' +--- + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/concepts/secure-comms.md b/content/concepts/secure-comms.md deleted file mode 100644 index c6512a46..00000000 --- a/content/concepts/secure-comms.md +++ /dev/null @@ -1,8 +0,0 @@ ---- -title: Secure Communication -weight: 2 ---- - -This article is coming soon! - -Please [refer to this issue](https://github.com/libp2p/docs/issues/19) to track the progress and make suggestions. diff --git a/content/concepts/security/_index.md b/content/concepts/security/_index.md new file mode 100644 index 00000000..89be8250 --- /dev/null +++ b/content/concepts/security/_index.md @@ -0,0 +1,18 @@ +--- +title: "Security Considerations" +weight: 10 +pre: ' 2.9. ' +chapter: true +--- + +### Chapter 2.9 + +# Secure Data + +{{% children %}} + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/concepts/security-considerations.md b/content/concepts/security/security-considerations.md similarity index 99% rename from content/concepts/security-considerations.md rename to content/concepts/security/security-considerations.md index 2a53b606..1f72ffed 100644 --- a/content/concepts/security-considerations.md +++ b/content/concepts/security/security-considerations.md @@ -1,6 +1,7 @@ --- -title: 'Security Considerations' -weight: 12 +title: 'Overview' +weight: 1 +pre: ' 2.9.1. ' --- libp2p makes it simple to establish [encrypted, authenticated communication diff --git a/content/concepts/transports/_index.md b/content/concepts/transports/_index.md new file mode 100644 index 00000000..5df8f6a7 --- /dev/null +++ b/content/concepts/transports/_index.md @@ -0,0 +1,18 @@ +--- +title: "Transports" +weight: 2 +pre: ' 2.1. ' +chapter: true +--- + +### Chapter 2.1 + +# Exchange Bytes + +{{% children %}} + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/concepts/transports/quic.md b/content/concepts/transports/quic.md new file mode 100644 index 00000000..590d86c2 --- /dev/null +++ b/content/concepts/transports/quic.md @@ -0,0 +1,11 @@ +--- +title: QUIC +weight: 3 +pre: ' 2.1.3. ' +--- + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/concepts/transports/tcp.md b/content/concepts/transports/tcp.md new file mode 100644 index 00000000..06ca31d1 --- /dev/null +++ b/content/concepts/transports/tcp.md @@ -0,0 +1,11 @@ +--- +title: TCP +weight: 2 +pre: ' 2.1.2. ' +--- + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/concepts/transport.md b/content/concepts/transports/transport.md similarity index 98% rename from content/concepts/transport.md rename to content/concepts/transports/transport.md index b54039e2..5b64e934 100644 --- a/content/concepts/transport.md +++ b/content/concepts/transports/transport.md @@ -1,6 +1,7 @@ --- -title: Transport +title: Overview weight: 1 +pre: ' 2.1.1. ' --- When you make a connection from your computer to a machine on the internet, diff --git a/content/concepts/transports/webrtc.md b/content/concepts/transports/webrtc.md new file mode 100644 index 00000000..721916f0 --- /dev/null +++ b/content/concepts/transports/webrtc.md @@ -0,0 +1,11 @@ +--- +title: WebRTC +weight: 5 +pre: ' 2.1.5. ' +--- + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/concepts/transports/websocket.md b/content/concepts/transports/websocket.md new file mode 100644 index 00000000..72cbed31 --- /dev/null +++ b/content/concepts/transports/websocket.md @@ -0,0 +1,11 @@ +--- +title: WebSocket +weight: 4 +pre: ' 2.1.4. ' +--- + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/concepts/transports/webtransport.md b/content/concepts/transports/webtransport.md new file mode 100644 index 00000000..935c686f --- /dev/null +++ b/content/concepts/transports/webtransport.md @@ -0,0 +1,11 @@ +--- +title: WebTransport +weight: 6 +pre: ' 2.1.6. ' +--- + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/introduction/_index.md b/content/introduction/_index.md index a3954419..876c1e15 100644 --- a/content/introduction/_index.md +++ b/content/introduction/_index.md @@ -1,5 +1,5 @@ --- -title: Quick Start +title: Introduction weight: 1 pre: "1. " chapter: true diff --git a/content/introduction/libp2p/_index.md b/content/introduction/libp2p/_index.md new file mode 100644 index 00000000..9c79507b --- /dev/null +++ b/content/introduction/libp2p/_index.md @@ -0,0 +1,14 @@ +--- +title: LIBP2P +weight: 2 +pre: ' 1.2. ' +--- + +### Chapter 1 + +# Get Started with libp2p + +Whether you’re just starting to dive into peer-to-peer concepts and +solutions, learning how to build peer-to-peer systems with libp2p, or +are looking for detailed reference information, this is the place to +start. diff --git a/content/introduction/getting-started.md b/content/introduction/libp2p/getting-started.md similarity index 98% rename from content/introduction/getting-started.md rename to content/introduction/libp2p/getting-started.md index d25b4eeb..47bcdf59 100644 --- a/content/introduction/getting-started.md +++ b/content/introduction/libp2p/getting-started.md @@ -1,7 +1,7 @@ --- -title: "Introduction" +title: A Modular P2P Networking Framework weight: 2 -pre: ' ' +pre: ' ' --- ## What is libp2p? @@ -9,8 +9,6 @@ pre: ' ' libp2p is a modular system of *protocols*, *specifications* and *libraries* that enable the development of peer-to-peer network applications. -**A p2p networking stack** - Because of libp2p's peer-to-peer and distributed architecture, most of the needs and considerations that the current web was built on no longer apply. The internet, such as it is, with firewalls and NATs, was designed to [securely] diff --git a/content/introduction/network-basics.md b/content/introduction/network-basics.md deleted file mode 100644 index 4c519dfd..00000000 --- a/content/introduction/network-basics.md +++ /dev/null @@ -1,7 +0,0 @@ ---- -title: "Network Basics" -weight: 1 -pre: ' ' ---- - -This section is coming soon! diff --git a/content/introduction/network-basics/_index.md b/content/introduction/network-basics/_index.md new file mode 100644 index 00000000..14d96c37 --- /dev/null +++ b/content/introduction/network-basics/_index.md @@ -0,0 +1,13 @@ +--- +title: Network Basics +weight: 1 +pre: ' 1.1. ' +chapter: false +--- + +### Chapter 1.1 + +Whether you’re just starting to dive into peer-to-peer concepts and +solutions, learning how to build peer-to-peer systems with libp2p, or +are looking for detailed reference information, this is the place to +start. diff --git a/content/introduction/network-basics/computer-networks.md b/content/introduction/network-basics/computer-networks.md new file mode 100644 index 00000000..3c5e1c08 --- /dev/null +++ b/content/introduction/network-basics/computer-networks.md @@ -0,0 +1,11 @@ +--- +title: Introduction to Computer Networks +weight: 2 +pre: ' 1.1.2. ' +--- + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/introduction/network-basics/data-structures.md b/content/introduction/network-basics/data-structures.md new file mode 100644 index 00000000..2bf94c6d --- /dev/null +++ b/content/introduction/network-basics/data-structures.md @@ -0,0 +1,11 @@ +--- +title: "Introduction to Data Structures" +weight: 4 +pre: ' 1.1.4. ' +--- + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/introduction/network-basics/distributed-computing.md b/content/introduction/network-basics/distributed-computing.md new file mode 100644 index 00000000..e97fdd99 --- /dev/null +++ b/content/introduction/network-basics/distributed-computing.md @@ -0,0 +1,11 @@ +--- +title: "Introduction to Distributed Computing" +weight: 3 +pre: ' 1.1.3. ' +--- + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/introduction/network-basics/internet-web.md b/content/introduction/network-basics/internet-web.md new file mode 100644 index 00000000..f4d53f70 --- /dev/null +++ b/content/introduction/network-basics/internet-web.md @@ -0,0 +1,11 @@ +--- +title: "The Internet and the Permanent Web" +weight: 1 +pre: ' 1.1.1 ' +--- + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} From 3758724d1d191c8be5dd694dbe71ff1927fec7b4 Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Thu, 8 Sep 2022 00:50:30 -0400 Subject: [PATCH 06/30] new struct for guide, ref, and involve sections --- content/_index.md | 18 +++++-------- content/community/_index.md | 11 -------- content/concepts/miscellaneous/_index.md | 18 +++++++++++++ .../miscellaneous}/glossary.md | 1 + content/get-involved/_index.md | 13 +++++++++ .../_index.md => get-involved/community.md} | 6 ++--- .../_index.md => get-involved/contribute.md} | 6 ++--- content/guides/_index.md | 17 +++++++++--- content/guides/examples/_index.md | 15 ++++++----- content/guides/examples/chat-app.md | 13 +++++++++ content/guides/examples/network.md | 7 ----- content/guides/examples/overview.md | 14 ++++++++++ content/guides/get-started/_index.md | 18 +++++++++++++ .../guides/{tutorials => get-started}/go.md | 0 content/guides/get-started/install.md | 27 +++++++++++++++++++ .../{tutorials => get-started}/javascript.md | 0 content/guides/get-started/quick-start.md | 12 +++++++++ .../guides/{tutorials => get-started}/rust.md | 0 content/guides/protocol-basics/_index.md | 20 ++++++++++++++ .../guides/protocol-basics/content-routing.md | 12 +++++++++ .../guides/protocol-basics/data-exchange.md | 12 +++++++++ content/guides/protocol-basics/discovery.md | 12 +++++++++ content/guides/protocol-basics/messaging.md | 12 +++++++++ content/guides/protocol-basics/muxers.md | 12 +++++++++ content/guides/protocol-basics/nat.md | 12 +++++++++ content/guides/protocol-basics/overview.md | 11 ++++++++ .../guides/protocol-basics/peer-routing.md | 12 +++++++++ .../protocol-basics/secure-communication.md | 12 +++++++++ content/guides/protocol-basics/transports.md | 12 +++++++++ content/guides/tutorials/_index.md | 11 -------- .../introduction/libp2p/getting-started.md | 16 +++++------ content/reference/_index.md | 11 +++++--- content/reference/dos-mitigation.md | 1 + content/reference/monitoring.md | 9 ++++++- content/reference/overview.md | 11 ++++++++ content/reference/specs.md | 7 ++++- 36 files changed, 332 insertions(+), 69 deletions(-) delete mode 100644 content/community/_index.md create mode 100644 content/concepts/miscellaneous/_index.md rename content/{reference => concepts/miscellaneous}/glossary.md (99%) create mode 100644 content/get-involved/_index.md rename content/{discuss/_index.md => get-involved/community.md} (88%) rename content/{contributing/_index.md => get-involved/contribute.md} (98%) create mode 100644 content/guides/examples/chat-app.md delete mode 100644 content/guides/examples/network.md create mode 100644 content/guides/examples/overview.md create mode 100644 content/guides/get-started/_index.md rename content/guides/{tutorials => get-started}/go.md (100%) create mode 100644 content/guides/get-started/install.md rename content/guides/{tutorials => get-started}/javascript.md (100%) create mode 100644 content/guides/get-started/quick-start.md rename content/guides/{tutorials => get-started}/rust.md (100%) create mode 100644 content/guides/protocol-basics/_index.md create mode 100644 content/guides/protocol-basics/content-routing.md create mode 100644 content/guides/protocol-basics/data-exchange.md create mode 100644 content/guides/protocol-basics/discovery.md create mode 100644 content/guides/protocol-basics/messaging.md create mode 100644 content/guides/protocol-basics/muxers.md create mode 100644 content/guides/protocol-basics/nat.md create mode 100644 content/guides/protocol-basics/overview.md create mode 100644 content/guides/protocol-basics/peer-routing.md create mode 100644 content/guides/protocol-basics/secure-communication.md create mode 100644 content/guides/protocol-basics/transports.md delete mode 100644 content/guides/tutorials/_index.md create mode 100644 content/reference/overview.md diff --git a/content/_index.md b/content/_index.md index 53d06a36..5a8a2597 100644 --- a/content/_index.md +++ b/content/_index.md @@ -1,5 +1,6 @@ --- -title: Welcome the libp2p +title: Welcome to the libp2p documentation portal +menuTitle: libp2p weight: 1 pre: "1. " chapter: false @@ -7,17 +8,12 @@ chapter: false # Welcome the libp2p Documentation Portal -Welcome to the official libp2p documentation hub! +### Chapter 1: Introduction -### Chapter 1 +### Chapter 2: Core Components -For new comers and those interested to understand key computer network concepts, start -here. +### Chapter 3: Guides -### Chapter 2 +### Chapter 4: Reference Documentation -### Chapter 3 - -### Chapter 4 - -### Chapter 5 +### Chapter 5: Get Involved diff --git a/content/community/_index.md b/content/community/_index.md deleted file mode 100644 index 33ec44d6..00000000 --- a/content/community/_index.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -title: "Community" -weight: 10 -pre: ' ' ---- - -Get in touch with other members of the libp2p community who are building tools and applications with libp2p! You can ask questions, discuss new ideas, or get support for problems at [libp2p discussion forums](https://discuss.libp2p.io/), but you can also [hop on IRC](/community/irc/) for a quick chat. - -See the other links in the community section for more information about meetings, events, apps people are building, and more. - -Information about contributing to libp2p and about other software projects in the community are also hosted here. diff --git a/content/concepts/miscellaneous/_index.md b/content/concepts/miscellaneous/_index.md new file mode 100644 index 00000000..5fc08904 --- /dev/null +++ b/content/concepts/miscellaneous/_index.md @@ -0,0 +1,18 @@ +--- +title: "Miscellaneous" +weight: 11 +pre: ' 2.10. ' +chapter: true +--- + +### Chapter 2.10 + +# Learn more + +{{% children %}} + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/reference/glossary.md b/content/concepts/miscellaneous/glossary.md similarity index 99% rename from content/reference/glossary.md rename to content/concepts/miscellaneous/glossary.md index b70443d9..304f881a 100644 --- a/content/reference/glossary.md +++ b/content/concepts/miscellaneous/glossary.md @@ -1,6 +1,7 @@ --- title: "Glossary" weight: 1 +pre: ' 2.10.1 ' --- ### Boot Node diff --git a/content/get-involved/_index.md b/content/get-involved/_index.md new file mode 100644 index 00000000..32d0ee4d --- /dev/null +++ b/content/get-involved/_index.md @@ -0,0 +1,13 @@ +--- +title: "Get Involved" +weight: 5 +pre: "5. " +--- + +{{% children %}} + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/discuss/_index.md b/content/get-involved/community.md similarity index 88% rename from content/discuss/_index.md rename to content/get-involved/community.md index 6cd8c534..61b53387 100644 --- a/content/discuss/_index.md +++ b/content/get-involved/community.md @@ -1,7 +1,7 @@ --- -title: "Help & Discussion" -weight: 100 -pre: " " +title: "Community" +weight: 2 +pre: ' 5.2 ' --- We love talking about libp2p, and we'd be happy to have you in the mix. diff --git a/content/contributing/_index.md b/content/get-involved/contribute.md similarity index 98% rename from content/contributing/_index.md rename to content/get-involved/contribute.md index 80be193d..3df69cae 100644 --- a/content/contributing/_index.md +++ b/content/get-involved/contribute.md @@ -1,7 +1,7 @@ --- -title: "Get Involved" -weight: 10 -pre: ' ' +title: "How to Contribute" +weight: 1 +pre: ' 5.1 ' --- So you want to contribute to libp2p and the peer-to-peer ecosystem? Here is a quick listing diff --git a/content/guides/_index.md b/content/guides/_index.md index cb382c6a..17325ef1 100644 --- a/content/guides/_index.md +++ b/content/guides/_index.md @@ -1,10 +1,21 @@ --- title: Guides -weight: 4 -pre: ' ' +weight: 3 +pre: "3. " +chapter: true --- +### Chapter 3 + +# Guides + The guides section of the libp2p documentation hub is a developer portal containing how-to guides, technical tutorials, and examples for each libp2p implementation. -**This section is being updated, stay tuned!** +{{% children %}} + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/guides/examples/_index.md b/content/guides/examples/_index.md index d9d47ab3..d893604f 100644 --- a/content/guides/examples/_index.md +++ b/content/guides/examples/_index.md @@ -1,12 +1,13 @@ --- title: Examples -weight: 2 -pre: ' ' +weight: 4 +pre: ' 3.3. ' --- -Here’s where to find working examples illustrating some of libp2p’s key features -for each of its main implementations: +Stay tuned for libp2p examples! -* For go, see the [go-libp2p-examples repo](https://github.com/libp2p/go-libp2p/tree/master/examples). -* For javascript, see the [/examples directory of the js-libp2p repo](https://github.com/libp2p/js-libp2p/tree/master/examples). -* For rust, see [/examples directory of the rust-libp2p repo](https://github.com/libp2p/rust-libp2p/tree/master/examples). +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/guides/examples/chat-app.md b/content/guides/examples/chat-app.md new file mode 100644 index 00000000..8f4b2d67 --- /dev/null +++ b/content/guides/examples/chat-app.md @@ -0,0 +1,13 @@ +--- +title: Chat Application +weight: 2 +pre: ' 3.3.2 ' +--- + +Stay tuned for libp2p examples! + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/guides/examples/network.md b/content/guides/examples/network.md deleted file mode 100644 index c484e65d..00000000 --- a/content/guides/examples/network.md +++ /dev/null @@ -1,7 +0,0 @@ ---- -title: "Getting Started with libp2p" -menuTitle: Start Networking -weight: 1 ---- - -Stay tuned for libp2p examples! diff --git a/content/guides/examples/overview.md b/content/guides/examples/overview.md new file mode 100644 index 00000000..d27cf654 --- /dev/null +++ b/content/guides/examples/overview.md @@ -0,0 +1,14 @@ +--- +title: "libp2p Examples" +menuTitle: Overview +weight: 1 +pre: ' 3.3.1 ' +--- + +Stay tuned for libp2p examples! + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/guides/get-started/_index.md b/content/guides/get-started/_index.md new file mode 100644 index 00000000..ddac504b --- /dev/null +++ b/content/guides/get-started/_index.md @@ -0,0 +1,18 @@ +--- +title: Get Started +weight: 1 +pre: ' 3.1. ' +--- + +Here, you'll find step-by-step tutorials about setting up your development environment, +getting familiar with some libp2p basics, and implementing a super simple node that can +send and receive "ping" messages. + +{{% children description="true" %}} + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} + diff --git a/content/guides/tutorials/go.md b/content/guides/get-started/go.md similarity index 100% rename from content/guides/tutorials/go.md rename to content/guides/get-started/go.md diff --git a/content/guides/get-started/install.md b/content/guides/get-started/install.md new file mode 100644 index 00000000..37832328 --- /dev/null +++ b/content/guides/get-started/install.md @@ -0,0 +1,27 @@ +--- +title: Install libp2p +weight: 1 +pre: ' 3.1.1. ' +--- + +{{< tabs >}} +{{% tab name="Go" %}} +```go +``` +{{% /tab %}} +{{% tab name="Rust" %}} +```rust +``` +{{% /tab %}} +{{% tab name="JavaScript" %}} +```js +``` +{{% /tab %}} +{{< /tabs >}} + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} + diff --git a/content/guides/tutorials/javascript.md b/content/guides/get-started/javascript.md similarity index 100% rename from content/guides/tutorials/javascript.md rename to content/guides/get-started/javascript.md diff --git a/content/guides/get-started/quick-start.md b/content/guides/get-started/quick-start.md new file mode 100644 index 00000000..5eeff7fe --- /dev/null +++ b/content/guides/get-started/quick-start.md @@ -0,0 +1,12 @@ +--- +title: Build your First Application with libp2p +menuTitle: Quick Start +weight: 2 +pre: ' 3.1.2. ' +--- + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/guides/tutorials/rust.md b/content/guides/get-started/rust.md similarity index 100% rename from content/guides/tutorials/rust.md rename to content/guides/get-started/rust.md diff --git a/content/guides/protocol-basics/_index.md b/content/guides/protocol-basics/_index.md new file mode 100644 index 00000000..ea58a2b8 --- /dev/null +++ b/content/guides/protocol-basics/_index.md @@ -0,0 +1,20 @@ +--- +title: Protocol Basics +weight: 3 +pre: ' 3.2. ' +--- + +Here’s where to find working examples illustrating some of libp2p’s key features +for each of its main implementations: + +* For go, see the [go-libp2p-examples repo](https://github.com/libp2p/go-libp2p/tree/master/examples). +* For javascript, see the [/examples directory of the js-libp2p repo](https://github.com/libp2p/js-libp2p/tree/master/examples). +* For rust, see [/examples directory of the rust-libp2p repo](https://github.com/libp2p/rust-libp2p/tree/master/examples). + +{{% children %}} + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/guides/protocol-basics/content-routing.md b/content/guides/protocol-basics/content-routing.md new file mode 100644 index 00000000..422ec9ce --- /dev/null +++ b/content/guides/protocol-basics/content-routing.md @@ -0,0 +1,12 @@ +--- +title: Get Started with Content Routing +menuTitle: Content Routing +weight: 8 +pre: ' 3.2.8. ' +--- + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/guides/protocol-basics/data-exchange.md b/content/guides/protocol-basics/data-exchange.md new file mode 100644 index 00000000..6603ec7f --- /dev/null +++ b/content/guides/protocol-basics/data-exchange.md @@ -0,0 +1,12 @@ +--- +title: Get Started with Data Exchange +menuTitle: Data Exchange +weight: 10 +pre: ' 3.2.10. ' +--- + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/guides/protocol-basics/discovery.md b/content/guides/protocol-basics/discovery.md new file mode 100644 index 00000000..431e07e3 --- /dev/null +++ b/content/guides/protocol-basics/discovery.md @@ -0,0 +1,12 @@ +--- +title: Get Started with Discovery +menuTitle: Peer Discovery +weight: 5 +pre: ' 3.2.6. ' +--- + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/guides/protocol-basics/messaging.md b/content/guides/protocol-basics/messaging.md new file mode 100644 index 00000000..2e5bd9b7 --- /dev/null +++ b/content/guides/protocol-basics/messaging.md @@ -0,0 +1,12 @@ +--- +title: Get Started with Messaging +menuTitle: P2P Messaging +weight: 9 +pre: ' 3.2.9. ' +--- + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/guides/protocol-basics/muxers.md b/content/guides/protocol-basics/muxers.md new file mode 100644 index 00000000..b9ca3530 --- /dev/null +++ b/content/guides/protocol-basics/muxers.md @@ -0,0 +1,12 @@ +--- +title: Get Started with Steam Muxers +menuTitle: Steam Muxers +weight: 4 +pre: ' 3.2.4. ' +--- + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/guides/protocol-basics/nat.md b/content/guides/protocol-basics/nat.md new file mode 100644 index 00000000..4e8a5451 --- /dev/null +++ b/content/guides/protocol-basics/nat.md @@ -0,0 +1,12 @@ +--- +title: Get Started with NATs +menuTitle: NATs +weight: 5 +pre: ' 3.2.5. ' +--- + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/guides/protocol-basics/overview.md b/content/guides/protocol-basics/overview.md new file mode 100644 index 00000000..d33e39d3 --- /dev/null +++ b/content/guides/protocol-basics/overview.md @@ -0,0 +1,11 @@ +--- +title: Prerequisites +weight: 1 +pre: ' 3.2.1. ' +--- + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/guides/protocol-basics/peer-routing.md b/content/guides/protocol-basics/peer-routing.md new file mode 100644 index 00000000..66c6f944 --- /dev/null +++ b/content/guides/protocol-basics/peer-routing.md @@ -0,0 +1,12 @@ +--- +title: Get Started with Peer Routing +menuTitle: Peer Routing +weight: 7 +pre: ' 3.2.7. ' +--- + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/guides/protocol-basics/secure-communication.md b/content/guides/protocol-basics/secure-communication.md new file mode 100644 index 00000000..ac37a05e --- /dev/null +++ b/content/guides/protocol-basics/secure-communication.md @@ -0,0 +1,12 @@ +--- +title: Get Started with Secure Communication +menuTitle: Secure Channels +weight: 3 +pre: ' 3.2.3. ' +--- + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/guides/protocol-basics/transports.md b/content/guides/protocol-basics/transports.md new file mode 100644 index 00000000..75a23350 --- /dev/null +++ b/content/guides/protocol-basics/transports.md @@ -0,0 +1,12 @@ +--- +title: Get Started with Transports +menuTitle: Transports +weight: 2 +pre: ' 3.2.2. ' +--- + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/guides/tutorials/_index.md b/content/guides/tutorials/_index.md deleted file mode 100644 index e89566dc..00000000 --- a/content/guides/tutorials/_index.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -title: Tutorials -weight: 2 -pre: ' ' ---- - -Here, you'll find step-by-step tutorials about setting up your development environment, -getting familiar with some libp2p basics, and implementing a super simple node that can -send and receive "ping" messages. - -{{% children description="true" %}} diff --git a/content/introduction/libp2p/getting-started.md b/content/introduction/libp2p/getting-started.md index 47bcdf59..e7cada1e 100644 --- a/content/introduction/libp2p/getting-started.md +++ b/content/introduction/libp2p/getting-started.md @@ -164,14 +164,14 @@ specific information and references: - [The Permissive License Stack](https://protocol.ai/blog/announcing-the-permissive-license-stack) is a licensing strategy for software development that embraces open-source values. -[glossary]: {{< ref "/reference/glossary.md" >}} -[definition_dht]: {{< ref "/reference/glossary.md#dht" >}} -[definition_p2p]: {{< ref "/reference/glossary.md#p2p" >}} -[definition_peer]: {{< ref "/reference/glossary.md#peer" >}} -[definition_peerid]: {{< ref "/reference/glossary.md#peerid" >}} -[definition_secio]: {{< ref "/reference/glossary.md#secio" >}} -[definition_muiltiaddress]: {{< ref "/reference/glossary.md#multiaddr" >}} -[definition_client_server]: {{< ref "/reference/glossary.md#client-server" >}} +[glossary]: {{< ref "concepts/miscellaneous/glossary.md" >}} +[definition_dht]: {{< ref "concepts/miscellaneous/glossary.md#dht" >}} +[definition_p2p]: {{< ref "concepts/miscellaneous/glossary.md#p2p" >}} +[definition_peer]: {{< ref "concepts/miscellaneous/glossary.md#peer" >}} +[definition_peerid]: {{< ref "concepts/miscellaneous/glossary.md#peerid" >}} +[definition_secio]: {{< ref "concepts/miscellaneous/glossary.md#secio" >}} +[definition_muiltiaddress]: {{< ref "concepts/miscellaneous/glossary.md#multiaddr" >}} +[definition_client_server]: {{< ref "concepts/miscellaneous/glossary.md#client-server" >}} [spec_content_routing]: https://github.com/libp2p/specs/blob/master/kad-dht/README.md [spec_pubsub]: https://github.com/libp2p/specs/blob/master/pubsub/README.md diff --git a/content/reference/_index.md b/content/reference/_index.md index 6143f038..2118067b 100644 --- a/content/reference/_index.md +++ b/content/reference/_index.md @@ -1,9 +1,14 @@ --- -title: Reference -weight: 5 -pre: ' ' +title: "Reference" +weight: 4 +pre: "4. " +chapter: true --- +### Chapter 4 + +# Reference Documentation + This section contains in-depth reference material, including a [glossary](/reference/glossary/) with definitions of libp2p terminology. ### Specifications & Planning diff --git a/content/reference/dos-mitigation.md b/content/reference/dos-mitigation.md index 9151c251..af50aae1 100644 --- a/content/reference/dos-mitigation.md +++ b/content/reference/dos-mitigation.md @@ -1,6 +1,7 @@ --- title: "DOS Mitigation" weight: 3 +pre: ' 4.3. ' --- DOS mitigation is an essential part of any P2P application. We need to design diff --git a/content/reference/monitoring.md b/content/reference/monitoring.md index 31fceaea..d4778f29 100644 --- a/content/reference/monitoring.md +++ b/content/reference/monitoring.md @@ -1,7 +1,14 @@ --- title: "Monitoring and Observability" weight: 4 +pre: ' 4.4. ' --- Reference the [Monitoring your application](../dos-mitigation/#monitoring-your-application) section in [DOS -Mitigation](../dos-mitigation/). \ No newline at end of file +Mitigation](../dos-mitigation/). + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/reference/overview.md b/content/reference/overview.md new file mode 100644 index 00000000..757cec3e --- /dev/null +++ b/content/reference/overview.md @@ -0,0 +1,11 @@ +--- +title: Overview +weight: 1 +pre: ' 4.1. ' +--- + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/reference/specs.md b/content/reference/specs.md index d40f4181..cff9cdd7 100644 --- a/content/reference/specs.md +++ b/content/reference/specs.md @@ -1,8 +1,13 @@ --- title: Specifications weight: 2 +pre: ' 4.2. ' --- Check out the [specs repo](https://github.com/libp2p/specs) for details about the libp2p architecture. - +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} From 410641f623b88a486469899b9b20cc8598f9d597 Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Mon, 12 Sep 2022 23:43:47 -0400 Subject: [PATCH 07/30] edits edits --- config.toml | 2 +- content/_index.md | 23 +++++++++++++++++++-- content/concepts/_index.md | 2 +- content/concepts/data-exchange/_index.md | 2 +- content/concepts/data-exchange/bitswap.md | 4 ++-- content/concepts/data-exchange/overview.md | 2 +- content/concepts/discovery/_index.md | 2 +- content/concepts/introduction/_index.md | 2 +- content/concepts/introduction/addressing.md | 2 +- content/concepts/introduction/overview.md | 2 +- content/concepts/introduction/protocols.md | 4 ++-- content/concepts/messaging/_index.md | 2 +- content/concepts/miscellaneous/_index.md | 2 +- content/concepts/multiplex/_index.md | 2 +- content/concepts/nat/_index.md | 2 +- content/concepts/nat/autonat.md | 4 ++-- content/concepts/nat/dcutr.md | 4 ++-- content/concepts/nat/hole-punching.md | 4 ++-- content/concepts/routing/_index.md | 2 +- content/concepts/secure-comm/_index.md | 2 +- content/concepts/security/_index.md | 2 +- content/concepts/transports/_index.md | 2 +- content/concepts/transports/webrtc.md | 4 ++-- content/concepts/transports/webtransport.md | 4 ++-- content/get-involved/_index.md | 2 +- content/guides/get-started/install.md | 1 - content/guides/protocol-basics/discovery.md | 2 +- content/introduction/libp2p/_index.md | 2 -- 28 files changed, 53 insertions(+), 37 deletions(-) diff --git a/config.toml b/config.toml index 7144de4d..95d50e77 100644 --- a/config.toml +++ b/config.toml @@ -3,7 +3,7 @@ relativeURLs = true languageCode = "en-us" title = "libp2p Documentation" -theme = "hugo-theme-learn" +theme = ["hugo-theme-learn"] [params] themeVariant = "libp2p" diff --git a/content/_index.md b/content/_index.md index 5a8a2597..25bb12f5 100644 --- a/content/_index.md +++ b/content/_index.md @@ -1,19 +1,38 @@ --- -title: Welcome to the libp2p documentation portal +title: libp2p book menuTitle: libp2p weight: 1 pre: "1. " chapter: false --- -# Welcome the libp2p Documentation Portal +# Welcome to the libp2p Book + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} ### Chapter 1: Introduction +Get started with computer networking and science concepts, distributed systems, and +how libp2p fits into the internet as we know it. + ### Chapter 2: Core Components +Learn about the core components that make up a full libp2p network stack +including network components, + ### Chapter 3: Guides +Start interacting with libp2p, spawn a node, transmit data, and learn how to +use libp2p protocols in your application. + ### Chapter 4: Reference Documentation +Find all the available libp2p specifications and different types of reference documentation. + ### Chapter 5: Get Involved + +Learn how to get involved in the libp2p community. diff --git a/content/concepts/_index.md b/content/concepts/_index.md index 3d9e818d..4e76a409 100644 --- a/content/concepts/_index.md +++ b/content/concepts/_index.md @@ -12,7 +12,7 @@ chapter: true libp2p covers a lot of ground, and may involve unfamiliar terminology and concepts. This section goes over the foundational concepts involved in libp2p. -{{% children %}} +{{% children description="true" %}} {{% notice "note" %}} This section is incomplete, and many of the articles are stubs. To help fill in diff --git a/content/concepts/data-exchange/_index.md b/content/concepts/data-exchange/_index.md index df9ebfe9..47d586f5 100644 --- a/content/concepts/data-exchange/_index.md +++ b/content/concepts/data-exchange/_index.md @@ -9,7 +9,7 @@ chapter: true # Exchange Data -{{% children %}} +{{% children description="true" %}} {{% notice "note" %}} This section is incomplete, and many of the articles are stubs. To help fill in diff --git a/content/concepts/data-exchange/bitswap.md b/content/concepts/data-exchange/bitswap.md index 2b20888c..fa8cb63c 100644 --- a/content/concepts/data-exchange/bitswap.md +++ b/content/concepts/data-exchange/bitswap.md @@ -1,7 +1,7 @@ --- title: "Bitswap" -weight: 8 -pre: ' 2.8.1. ' +weight: 2 +pre: ' 2.8.2. ' --- {{% notice "note" %}} diff --git a/content/concepts/data-exchange/overview.md b/content/concepts/data-exchange/overview.md index ac9cc626..30df3442 100644 --- a/content/concepts/data-exchange/overview.md +++ b/content/concepts/data-exchange/overview.md @@ -1,5 +1,5 @@ --- title: 'Overview' weight: 1 -pre: ' 2.9.1. ' +pre: ' 2.8.1. ' --- diff --git a/content/concepts/discovery/_index.md b/content/concepts/discovery/_index.md index 1efeba4c..bf0f7716 100644 --- a/content/concepts/discovery/_index.md +++ b/content/concepts/discovery/_index.md @@ -9,7 +9,7 @@ chapter: true # Discover Peers -{{% children %}} +{{% children description="true" %}} {{% notice "note" %}} This section is incomplete, and many of the articles are stubs. To help fill in diff --git a/content/concepts/introduction/_index.md b/content/concepts/introduction/_index.md index 2c53c1e5..fb038e08 100644 --- a/content/concepts/introduction/_index.md +++ b/content/concepts/introduction/_index.md @@ -9,7 +9,7 @@ chapter: true # Key Components of libp2p -{{% children %}} +{{% children description="true" %}} {{% notice "note" %}} This section is incomplete, and many of the articles are stubs. To help fill in diff --git a/content/concepts/introduction/addressing.md b/content/concepts/introduction/addressing.md index ff93f377..e4435a6b 100644 --- a/content/concepts/introduction/addressing.md +++ b/content/concepts/introduction/addressing.md @@ -1,6 +1,6 @@ --- title: Addressing -weight: 1 +weight: 2 pre: ' 2.0.2. ' --- diff --git a/content/concepts/introduction/overview.md b/content/concepts/introduction/overview.md index 8792d150..2d05be02 100644 --- a/content/concepts/introduction/overview.md +++ b/content/concepts/introduction/overview.md @@ -1,6 +1,6 @@ --- title: "Overview" -weight: 2 +weight: 1 pre: ' 2.0.1 ' --- diff --git a/content/concepts/introduction/protocols.md b/content/concepts/introduction/protocols.md index c6ebfe77..ad8677e6 100644 --- a/content/concepts/introduction/protocols.md +++ b/content/concepts/introduction/protocols.md @@ -1,7 +1,7 @@ --- title: Protocols -weight: 2 -pre: ' 2.0.3. ' +weight: 3 +pre: ' 2.0.2. ' --- There are protocols everywhere you look when you're writing network applications, and libp2p is diff --git a/content/concepts/messaging/_index.md b/content/concepts/messaging/_index.md index 47a1e789..f9641852 100644 --- a/content/concepts/messaging/_index.md +++ b/content/concepts/messaging/_index.md @@ -9,7 +9,7 @@ chapter: true # Transmit Data -{{% children %}} +{{% children description="true" %}} {{% notice "note" %}} This section is incomplete, and many of the articles are stubs. To help fill in diff --git a/content/concepts/miscellaneous/_index.md b/content/concepts/miscellaneous/_index.md index 5fc08904..b6bd19c0 100644 --- a/content/concepts/miscellaneous/_index.md +++ b/content/concepts/miscellaneous/_index.md @@ -9,7 +9,7 @@ chapter: true # Learn more -{{% children %}} +{{% children description="true" %}} {{% notice "note" %}} This section is incomplete, and many of the articles are stubs. To help fill in diff --git a/content/concepts/multiplex/_index.md b/content/concepts/multiplex/_index.md index 81489c26..223d6c47 100644 --- a/content/concepts/multiplex/_index.md +++ b/content/concepts/multiplex/_index.md @@ -9,7 +9,7 @@ chapter: true # Stream bytes -{{% children %}} +{{% children description="true" %}} {{% notice "note" %}} This section is incomplete, and many of the articles are stubs. To help fill in diff --git a/content/concepts/nat/_index.md b/content/concepts/nat/_index.md index 29ad09d5..31538017 100644 --- a/content/concepts/nat/_index.md +++ b/content/concepts/nat/_index.md @@ -9,7 +9,7 @@ chapter: true # Traverse Bytes -{{% children %}} +{{% children description="true" %}} {{% notice "note" %}} This section is incomplete, and many of the articles are stubs. To help fill in diff --git a/content/concepts/nat/autonat.md b/content/concepts/nat/autonat.md index 188f9688..68ff7ba5 100644 --- a/content/concepts/nat/autonat.md +++ b/content/concepts/nat/autonat.md @@ -1,7 +1,7 @@ --- title: AutoNAT -weight: 3 -pre: ' 2.4.3. ' +weight: 4 +pre: ' 2.4.4. ' --- {{% notice "note" %}} diff --git a/content/concepts/nat/dcutr.md b/content/concepts/nat/dcutr.md index 1a7dc013..3fa0797b 100644 --- a/content/concepts/nat/dcutr.md +++ b/content/concepts/nat/dcutr.md @@ -1,7 +1,7 @@ --- title: DCUtR -weight: 4 -pre: ' 2.4.4. ' +weight: 5 +pre: ' 2.4.5. ' --- {{% notice "note" %}} diff --git a/content/concepts/nat/hole-punching.md b/content/concepts/nat/hole-punching.md index 6dd314fb..5287c326 100644 --- a/content/concepts/nat/hole-punching.md +++ b/content/concepts/nat/hole-punching.md @@ -1,7 +1,7 @@ --- title: Hole Punching -weight: 2 -pre: ' 2.4.5. ' +weight: 3 +pre: ' 2.4.3. ' --- {{% notice "note" %}} diff --git a/content/concepts/routing/_index.md b/content/concepts/routing/_index.md index 9a9d77da..c92523d6 100644 --- a/content/concepts/routing/_index.md +++ b/content/concepts/routing/_index.md @@ -9,7 +9,7 @@ chapter: true # Route with Peers -{{% children %}} +{{% children description="true" %}} {{% notice "note" %}} This section is incomplete, and many of the articles are stubs. To help fill in diff --git a/content/concepts/secure-comm/_index.md b/content/concepts/secure-comm/_index.md index a1af748e..de489886 100644 --- a/content/concepts/secure-comm/_index.md +++ b/content/concepts/secure-comm/_index.md @@ -9,7 +9,7 @@ chapter: true # Secure bytes -{{% children %}} +{{% children description="true" %}} {{% notice "note" %}} This section is incomplete, and many of the articles are stubs. To help fill in diff --git a/content/concepts/security/_index.md b/content/concepts/security/_index.md index 89be8250..8b13b163 100644 --- a/content/concepts/security/_index.md +++ b/content/concepts/security/_index.md @@ -9,7 +9,7 @@ chapter: true # Secure Data -{{% children %}} +{{% children description="true" %}} {{% notice "note" %}} This section is incomplete, and many of the articles are stubs. To help fill in diff --git a/content/concepts/transports/_index.md b/content/concepts/transports/_index.md index 5df8f6a7..64664d7c 100644 --- a/content/concepts/transports/_index.md +++ b/content/concepts/transports/_index.md @@ -9,7 +9,7 @@ chapter: true # Exchange Bytes -{{% children %}} +{{% children description="true" %}} {{% notice "note" %}} This section is incomplete, and many of the articles are stubs. To help fill in diff --git a/content/concepts/transports/webrtc.md b/content/concepts/transports/webrtc.md index 721916f0..9e9ddcb5 100644 --- a/content/concepts/transports/webrtc.md +++ b/content/concepts/transports/webrtc.md @@ -1,7 +1,7 @@ --- title: WebRTC -weight: 5 -pre: ' 2.1.5. ' +weight: 6 +pre: ' 2.1.6. ' --- {{% notice "note" %}} diff --git a/content/concepts/transports/webtransport.md b/content/concepts/transports/webtransport.md index 935c686f..52ab168a 100644 --- a/content/concepts/transports/webtransport.md +++ b/content/concepts/transports/webtransport.md @@ -1,7 +1,7 @@ --- title: WebTransport -weight: 6 -pre: ' 2.1.6. ' +weight: 5 +pre: ' 2.1.5. ' --- {{% notice "note" %}} diff --git a/content/get-involved/_index.md b/content/get-involved/_index.md index 32d0ee4d..b8c71ee8 100644 --- a/content/get-involved/_index.md +++ b/content/get-involved/_index.md @@ -4,7 +4,7 @@ weight: 5 pre: "5. " --- -{{% children %}} +{{% children description="true" %}} {{% notice "note" %}} This section is incomplete, and many of the articles are stubs. To help fill in diff --git a/content/guides/get-started/install.md b/content/guides/get-started/install.md index 37832328..cc750060 100644 --- a/content/guides/get-started/install.md +++ b/content/guides/get-started/install.md @@ -24,4 +24,3 @@ This section is incomplete, and many of the articles are stubs. To help fill in the gaps, please see the issues linked in each article to add your input and help us prioritize the outstanding work. {{% /notice %}} - diff --git a/content/guides/protocol-basics/discovery.md b/content/guides/protocol-basics/discovery.md index 431e07e3..f0399fe4 100644 --- a/content/guides/protocol-basics/discovery.md +++ b/content/guides/protocol-basics/discovery.md @@ -1,7 +1,7 @@ --- title: Get Started with Discovery menuTitle: Peer Discovery -weight: 5 +weight: 6 pre: ' 3.2.6. ' --- diff --git a/content/introduction/libp2p/_index.md b/content/introduction/libp2p/_index.md index 9c79507b..31fe5f11 100644 --- a/content/introduction/libp2p/_index.md +++ b/content/introduction/libp2p/_index.md @@ -4,8 +4,6 @@ weight: 2 pre: ' 1.2. ' --- -### Chapter 1 - # Get Started with libp2p Whether you’re just starting to dive into peer-to-peer concepts and From 4c2b7fc6cf4249e0093ed3e982afbc4e9588103b Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Tue, 13 Sep 2022 00:00:37 -0400 Subject: [PATCH 08/30] more toc, edits --- content/get-involved/_index.md | 5 +++++ content/guides/_index.md | 2 +- content/guides/examples/_index.md | 2 ++ content/guides/protocol-basics/_index.md | 2 +- content/introduction/_index.md | 14 +++++++----- content/reference/_index.md | 27 ++++++------------------ content/reference/overview.md | 23 ++++++++++++++++++++ 7 files changed, 47 insertions(+), 28 deletions(-) diff --git a/content/get-involved/_index.md b/content/get-involved/_index.md index b8c71ee8..a91cfd1d 100644 --- a/content/get-involved/_index.md +++ b/content/get-involved/_index.md @@ -2,8 +2,13 @@ title: "Get Involved" weight: 5 pre: "5. " +chapter: true --- +### Chapter 5 + +# Get Involved + {{% children description="true" %}} {{% notice "note" %}} diff --git a/content/guides/_index.md b/content/guides/_index.md index 17325ef1..a57bb2de 100644 --- a/content/guides/_index.md +++ b/content/guides/_index.md @@ -12,7 +12,7 @@ chapter: true The guides section of the libp2p documentation hub is a developer portal containing how-to guides, technical tutorials, and examples for each libp2p implementation. -{{% children %}} +{{% children description="true" %}} {{% notice "note" %}} This section is incomplete, and many of the articles are stubs. To help fill in diff --git a/content/guides/examples/_index.md b/content/guides/examples/_index.md index d893604f..a4c67503 100644 --- a/content/guides/examples/_index.md +++ b/content/guides/examples/_index.md @@ -6,6 +6,8 @@ pre: ' 3.3. ' Stay tuned for libp2p examples! +{{% children description="true" %}} + {{% notice "note" %}} This section is incomplete, and many of the articles are stubs. To help fill in the gaps, please see the issues linked in each article to add your input and diff --git a/content/guides/protocol-basics/_index.md b/content/guides/protocol-basics/_index.md index ea58a2b8..2b02a3e5 100644 --- a/content/guides/protocol-basics/_index.md +++ b/content/guides/protocol-basics/_index.md @@ -11,7 +11,7 @@ for each of its main implementations: * For javascript, see the [/examples directory of the js-libp2p repo](https://github.com/libp2p/js-libp2p/tree/master/examples). * For rust, see [/examples directory of the rust-libp2p repo](https://github.com/libp2p/rust-libp2p/tree/master/examples). -{{% children %}} +{{% children description="true" %}} {{% notice "note" %}} This section is incomplete, and many of the articles are stubs. To help fill in diff --git a/content/introduction/_index.md b/content/introduction/_index.md index 876c1e15..9125918a 100644 --- a/content/introduction/_index.md +++ b/content/introduction/_index.md @@ -7,9 +7,13 @@ chapter: true ### Chapter 1 -# Get Started with libp2p +# Get Started with Networking + +{{% children description="true" %}} + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} -Whether you’re just starting to dive into peer-to-peer concepts and -solutions, learning how to build peer-to-peer systems with libp2p, or -are looking for detailed reference information, this is the place to -start. diff --git a/content/reference/_index.md b/content/reference/_index.md index 2118067b..ed2548d1 100644 --- a/content/reference/_index.md +++ b/content/reference/_index.md @@ -9,25 +9,10 @@ chapter: true # Reference Documentation -This section contains in-depth reference material, including a [glossary](/reference/glossary/) with definitions of libp2p terminology. +{{% children description="true" %}} -### Specifications & Planning - -While libp2p has several implementations, it is fundamentally a set of protocols for -peer identity, discover, routing, transport and more. - -See the [specifications section](/reference/specs/) for details. - -### Implementations - -At the core of libp2p is a set of [specifications](/reference/specs/), which together -form the definition for what libp2p is in the abstract and what makes a "correct" libp2p -implementation. Today, implementations of libp2p exist in several languages, with varying degrees -of completeness. The most complete implementations are in [Go](/reference/go/), -[JavaScript](/reference/js/), and [Rust](https://github.com/libp2p/rust-libp2p). - -In addition to the implementations above, the libp2p community is actively working on -implementations in [python](https://github.com/libp2p/py-libp2p) and [the JVM via Kotlin](https://github.com/web3j/libp2p). Please check the project pages for each implementation to see its status and current -state of completeness. - -Please help make this section more complete by [filing an issue](https://github.com/libp2p/docs/issues/new). +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/reference/overview.md b/content/reference/overview.md index 757cec3e..d295e0e1 100644 --- a/content/reference/overview.md +++ b/content/reference/overview.md @@ -4,6 +4,29 @@ weight: 1 pre: ' 4.1. ' --- +This section contains in-depth reference material, including a [glossary](/reference/glossary/) with definitions of libp2p terminology. + +### Specifications & Planning + +While libp2p has several implementations, it is fundamentally a set of protocols for +peer identity, discover, routing, transport and more. + +See the [specifications section](/reference/specs/) for details. + +### Implementations + +At the core of libp2p is a set of [specifications](/reference/specs/), which together +form the definition for what libp2p is in the abstract and what makes a "correct" libp2p +implementation. Today, implementations of libp2p exist in several languages, with varying degrees +of completeness. The most complete implementations are in [Go](/reference/go/), +[JavaScript](/reference/js/), and [Rust](https://github.com/libp2p/rust-libp2p). + +In addition to the implementations above, the libp2p community is actively working on +implementations in [python](https://github.com/libp2p/py-libp2p) and [the JVM via Kotlin](https://github.com/web3j/libp2p). Please check the project pages for each implementation to see its status and current +state of completeness. + +Please help make this section more complete by [filing an issue](https://github.com/libp2p/docs/issues/new). + {{% notice "note" %}} This section is incomplete, and many of the articles are stubs. To help fill in the gaps, please see the issues linked in each article to add your input and From 3f182d8816e4c8111107c59869e647b01544931e Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Thu, 22 Sep 2022 01:56:21 -0400 Subject: [PATCH 09/30] rmv most placeholder docs + edits --- content/concepts/data-exchange/bitswap.md | 11 -------- content/concepts/discovery/mdns.md | 11 -------- content/concepts/discovery/random-walk.md | 11 -------- content/concepts/discovery/rendezvous.md | 11 -------- content/concepts/introduction/overview.md | 11 -------- content/concepts/messaging/overview.md | 11 -------- content/concepts/multiplex/mplex.md | 11 -------- content/concepts/multiplex/yamux.md | 11 -------- content/concepts/nat/autonat.md | 11 -------- content/concepts/nat/dcutr.md | 11 -------- content/concepts/nat/hole-punching.md | 11 -------- .../routing/content-routing/kaddht.md | 11 -------- content/concepts/routing/overview.md | 5 ---- .../concepts/routing/peer-routing/kaddht.md | 11 -------- content/concepts/secure-comm/noise.md | 11 -------- .../{secure-comms.md => overview.md} | 0 content/concepts/secure-comm/tls.md | 11 -------- content/concepts/transports/quic.md | 11 -------- content/concepts/transports/tcp.md | 11 -------- content/concepts/transports/webrtc.md | 11 -------- content/concepts/transports/websocket.md | 11 -------- content/concepts/transports/webtransport.md | 11 -------- content/guides/examples/chat-app.md | 13 ---------- content/guides/examples/overview.md | 2 -- content/guides/get-started/go.md | 3 ++- content/guides/get-started/install.md | 26 ------------------- content/guides/get-started/javascript.md | 3 ++- content/guides/get-started/quick-start.md | 12 --------- content/guides/get-started/rust.md | 3 ++- .../guides/protocol-basics/content-routing.md | 12 --------- .../guides/protocol-basics/data-exchange.md | 12 --------- content/guides/protocol-basics/discovery.md | 12 --------- content/guides/protocol-basics/messaging.md | 12 --------- content/guides/protocol-basics/muxers.md | 12 --------- content/guides/protocol-basics/nat.md | 12 --------- .../guides/protocol-basics/peer-routing.md | 12 --------- .../protocol-basics/secure-communication.md | 12 --------- content/guides/protocol-basics/transports.md | 12 --------- content/introduction/network-basics/_index.md | 7 +++++ .../network-basics/computer-networks.md | 11 -------- .../network-basics/data-structures.md | 11 -------- .../network-basics/distributed-computing.md | 11 -------- .../network-basics/internet-web.md | 11 -------- static/css/theme.css | 8 +++--- 44 files changed, 18 insertions(+), 436 deletions(-) delete mode 100644 content/concepts/data-exchange/bitswap.md delete mode 100644 content/concepts/discovery/mdns.md delete mode 100644 content/concepts/discovery/random-walk.md delete mode 100644 content/concepts/discovery/rendezvous.md delete mode 100644 content/concepts/introduction/overview.md delete mode 100644 content/concepts/messaging/overview.md delete mode 100644 content/concepts/multiplex/mplex.md delete mode 100644 content/concepts/multiplex/yamux.md delete mode 100644 content/concepts/nat/autonat.md delete mode 100644 content/concepts/nat/dcutr.md delete mode 100644 content/concepts/nat/hole-punching.md delete mode 100644 content/concepts/routing/content-routing/kaddht.md delete mode 100644 content/concepts/routing/overview.md delete mode 100644 content/concepts/routing/peer-routing/kaddht.md delete mode 100644 content/concepts/secure-comm/noise.md rename content/concepts/secure-comm/{secure-comms.md => overview.md} (100%) delete mode 100644 content/concepts/secure-comm/tls.md delete mode 100644 content/concepts/transports/quic.md delete mode 100644 content/concepts/transports/tcp.md delete mode 100644 content/concepts/transports/webrtc.md delete mode 100644 content/concepts/transports/websocket.md delete mode 100644 content/concepts/transports/webtransport.md delete mode 100644 content/guides/examples/chat-app.md delete mode 100644 content/guides/get-started/install.md delete mode 100644 content/guides/get-started/quick-start.md delete mode 100644 content/guides/protocol-basics/content-routing.md delete mode 100644 content/guides/protocol-basics/data-exchange.md delete mode 100644 content/guides/protocol-basics/discovery.md delete mode 100644 content/guides/protocol-basics/messaging.md delete mode 100644 content/guides/protocol-basics/muxers.md delete mode 100644 content/guides/protocol-basics/nat.md delete mode 100644 content/guides/protocol-basics/peer-routing.md delete mode 100644 content/guides/protocol-basics/secure-communication.md delete mode 100644 content/guides/protocol-basics/transports.md delete mode 100644 content/introduction/network-basics/computer-networks.md delete mode 100644 content/introduction/network-basics/data-structures.md delete mode 100644 content/introduction/network-basics/distributed-computing.md delete mode 100644 content/introduction/network-basics/internet-web.md diff --git a/content/concepts/data-exchange/bitswap.md b/content/concepts/data-exchange/bitswap.md deleted file mode 100644 index fa8cb63c..00000000 --- a/content/concepts/data-exchange/bitswap.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -title: "Bitswap" -weight: 2 -pre: ' 2.8.2. ' ---- - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} diff --git a/content/concepts/discovery/mdns.md b/content/concepts/discovery/mdns.md deleted file mode 100644 index 8f87076a..00000000 --- a/content/concepts/discovery/mdns.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -title: mDNS -weight: 2 -pre: ' 2.5.2. ' ---- - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} diff --git a/content/concepts/discovery/random-walk.md b/content/concepts/discovery/random-walk.md deleted file mode 100644 index 4bea2bcb..00000000 --- a/content/concepts/discovery/random-walk.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -title: Random Walk -weight: 3 -pre: ' 2.5.3. ' ---- - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} diff --git a/content/concepts/discovery/rendezvous.md b/content/concepts/discovery/rendezvous.md deleted file mode 100644 index ac57cd01..00000000 --- a/content/concepts/discovery/rendezvous.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -title: Rendezvous -weight: 4 -pre: ' 2.5.4. ' ---- - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} diff --git a/content/concepts/introduction/overview.md b/content/concepts/introduction/overview.md deleted file mode 100644 index 2d05be02..00000000 --- a/content/concepts/introduction/overview.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -title: "Overview" -weight: 1 -pre: ' 2.0.1 ' ---- - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} diff --git a/content/concepts/messaging/overview.md b/content/concepts/messaging/overview.md deleted file mode 100644 index eb6b7898..00000000 --- a/content/concepts/messaging/overview.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -title: Overview -weight: 1 -pre: ' 2.7.1. ' ---- - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} diff --git a/content/concepts/multiplex/mplex.md b/content/concepts/multiplex/mplex.md deleted file mode 100644 index c67332e8..00000000 --- a/content/concepts/multiplex/mplex.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -title: "mplex" -weight: 2 -pre: ' 2.3.2. ' ---- - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} diff --git a/content/concepts/multiplex/yamux.md b/content/concepts/multiplex/yamux.md deleted file mode 100644 index 80615cd6..00000000 --- a/content/concepts/multiplex/yamux.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -title: "yamux" -weight: 2 -pre: ' 2.3.3. ' ---- - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} diff --git a/content/concepts/nat/autonat.md b/content/concepts/nat/autonat.md deleted file mode 100644 index 68ff7ba5..00000000 --- a/content/concepts/nat/autonat.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -title: AutoNAT -weight: 4 -pre: ' 2.4.4. ' ---- - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} diff --git a/content/concepts/nat/dcutr.md b/content/concepts/nat/dcutr.md deleted file mode 100644 index 3fa0797b..00000000 --- a/content/concepts/nat/dcutr.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -title: DCUtR -weight: 5 -pre: ' 2.4.5. ' ---- - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} diff --git a/content/concepts/nat/hole-punching.md b/content/concepts/nat/hole-punching.md deleted file mode 100644 index 5287c326..00000000 --- a/content/concepts/nat/hole-punching.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -title: Hole Punching -weight: 3 -pre: ' 2.4.3. ' ---- - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} diff --git a/content/concepts/routing/content-routing/kaddht.md b/content/concepts/routing/content-routing/kaddht.md deleted file mode 100644 index 135a2f0a..00000000 --- a/content/concepts/routing/content-routing/kaddht.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -title: KadDHT-based Content Routing -weight: 4 -pre: ' 2.6.4.1. ' ---- - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} diff --git a/content/concepts/routing/overview.md b/content/concepts/routing/overview.md deleted file mode 100644 index ed65621e..00000000 --- a/content/concepts/routing/overview.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -title: "Overview" -weight: 1 -pre: ' 2.6.1. ' ---- diff --git a/content/concepts/routing/peer-routing/kaddht.md b/content/concepts/routing/peer-routing/kaddht.md deleted file mode 100644 index b74b575b..00000000 --- a/content/concepts/routing/peer-routing/kaddht.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -title: KadDHT-based Peer Routing -weight: 3 -pre: ' 2.6.3.1 ' ---- - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} diff --git a/content/concepts/secure-comm/noise.md b/content/concepts/secure-comm/noise.md deleted file mode 100644 index f5ccd051..00000000 --- a/content/concepts/secure-comm/noise.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -title: Noise -weight: 2 -pre: ' 2.2.2. ' ---- - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} diff --git a/content/concepts/secure-comm/secure-comms.md b/content/concepts/secure-comm/overview.md similarity index 100% rename from content/concepts/secure-comm/secure-comms.md rename to content/concepts/secure-comm/overview.md diff --git a/content/concepts/secure-comm/tls.md b/content/concepts/secure-comm/tls.md deleted file mode 100644 index 5ebdafe4..00000000 --- a/content/concepts/secure-comm/tls.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -title: TLS -weight: 3 -pre: ' 2.2.3. ' ---- - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} diff --git a/content/concepts/transports/quic.md b/content/concepts/transports/quic.md deleted file mode 100644 index 590d86c2..00000000 --- a/content/concepts/transports/quic.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -title: QUIC -weight: 3 -pre: ' 2.1.3. ' ---- - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} diff --git a/content/concepts/transports/tcp.md b/content/concepts/transports/tcp.md deleted file mode 100644 index 06ca31d1..00000000 --- a/content/concepts/transports/tcp.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -title: TCP -weight: 2 -pre: ' 2.1.2. ' ---- - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} diff --git a/content/concepts/transports/webrtc.md b/content/concepts/transports/webrtc.md deleted file mode 100644 index 9e9ddcb5..00000000 --- a/content/concepts/transports/webrtc.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -title: WebRTC -weight: 6 -pre: ' 2.1.6. ' ---- - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} diff --git a/content/concepts/transports/websocket.md b/content/concepts/transports/websocket.md deleted file mode 100644 index 72cbed31..00000000 --- a/content/concepts/transports/websocket.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -title: WebSocket -weight: 4 -pre: ' 2.1.4. ' ---- - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} diff --git a/content/concepts/transports/webtransport.md b/content/concepts/transports/webtransport.md deleted file mode 100644 index 52ab168a..00000000 --- a/content/concepts/transports/webtransport.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -title: WebTransport -weight: 5 -pre: ' 2.1.5. ' ---- - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} diff --git a/content/guides/examples/chat-app.md b/content/guides/examples/chat-app.md deleted file mode 100644 index 8f4b2d67..00000000 --- a/content/guides/examples/chat-app.md +++ /dev/null @@ -1,13 +0,0 @@ ---- -title: Chat Application -weight: 2 -pre: ' 3.3.2 ' ---- - -Stay tuned for libp2p examples! - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} diff --git a/content/guides/examples/overview.md b/content/guides/examples/overview.md index d27cf654..00feac79 100644 --- a/content/guides/examples/overview.md +++ b/content/guides/examples/overview.md @@ -5,8 +5,6 @@ weight: 1 pre: ' 3.3.1 ' --- -Stay tuned for libp2p examples! - {{% notice "note" %}} This section is incomplete, and many of the articles are stubs. To help fill in the gaps, please see the issues linked in each article to add your input and diff --git a/content/guides/get-started/go.md b/content/guides/get-started/go.md index 4d1e69a4..7e359e10 100644 --- a/content/guides/get-started/go.md +++ b/content/guides/get-started/go.md @@ -1,7 +1,8 @@ --- title: "Getting Started with go-libp2p" -menuTitle: Go +menuTitle: go-libp2p weight: 2 +pre: ' 3.1.1. ' --- This is the first in a series of tutorials on working with libp2p’s Go implementation, diff --git a/content/guides/get-started/install.md b/content/guides/get-started/install.md deleted file mode 100644 index cc750060..00000000 --- a/content/guides/get-started/install.md +++ /dev/null @@ -1,26 +0,0 @@ ---- -title: Install libp2p -weight: 1 -pre: ' 3.1.1. ' ---- - -{{< tabs >}} -{{% tab name="Go" %}} -```go -``` -{{% /tab %}} -{{% tab name="Rust" %}} -```rust -``` -{{% /tab %}} -{{% tab name="JavaScript" %}} -```js -``` -{{% /tab %}} -{{< /tabs >}} - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} diff --git a/content/guides/get-started/javascript.md b/content/guides/get-started/javascript.md index 213aba1f..cd65aac5 100644 --- a/content/guides/get-started/javascript.md +++ b/content/guides/get-started/javascript.md @@ -1,7 +1,8 @@ --- title: "Getting Started with js-libp2p" -menuTitle: "Javascript" +menuTitle: js-libp2p weight: 2 +pre: ' 3.1.2. ' --- This is the first in a series of tutorials on working with libp2p's javascript implementation, [js-libp2p](https://github.com/libp2p/js-libp2p). diff --git a/content/guides/get-started/quick-start.md b/content/guides/get-started/quick-start.md deleted file mode 100644 index 5eeff7fe..00000000 --- a/content/guides/get-started/quick-start.md +++ /dev/null @@ -1,12 +0,0 @@ ---- -title: Build your First Application with libp2p -menuTitle: Quick Start -weight: 2 -pre: ' 3.1.2. ' ---- - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} diff --git a/content/guides/get-started/rust.md b/content/guides/get-started/rust.md index a5ed7ad0..06aa36bc 100644 --- a/content/guides/get-started/rust.md +++ b/content/guides/get-started/rust.md @@ -1,7 +1,8 @@ --- title: "Getting Started with rust-libp2p" -menuTitle: Rust +menuTitle: rust-libp2p weight: 2 +pre: ' 3.1.3. ' --- Check out [tutorials of the Rust libp2p diff --git a/content/guides/protocol-basics/content-routing.md b/content/guides/protocol-basics/content-routing.md deleted file mode 100644 index 422ec9ce..00000000 --- a/content/guides/protocol-basics/content-routing.md +++ /dev/null @@ -1,12 +0,0 @@ ---- -title: Get Started with Content Routing -menuTitle: Content Routing -weight: 8 -pre: ' 3.2.8. ' ---- - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} diff --git a/content/guides/protocol-basics/data-exchange.md b/content/guides/protocol-basics/data-exchange.md deleted file mode 100644 index 6603ec7f..00000000 --- a/content/guides/protocol-basics/data-exchange.md +++ /dev/null @@ -1,12 +0,0 @@ ---- -title: Get Started with Data Exchange -menuTitle: Data Exchange -weight: 10 -pre: ' 3.2.10. ' ---- - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} diff --git a/content/guides/protocol-basics/discovery.md b/content/guides/protocol-basics/discovery.md deleted file mode 100644 index f0399fe4..00000000 --- a/content/guides/protocol-basics/discovery.md +++ /dev/null @@ -1,12 +0,0 @@ ---- -title: Get Started with Discovery -menuTitle: Peer Discovery -weight: 6 -pre: ' 3.2.6. ' ---- - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} diff --git a/content/guides/protocol-basics/messaging.md b/content/guides/protocol-basics/messaging.md deleted file mode 100644 index 2e5bd9b7..00000000 --- a/content/guides/protocol-basics/messaging.md +++ /dev/null @@ -1,12 +0,0 @@ ---- -title: Get Started with Messaging -menuTitle: P2P Messaging -weight: 9 -pre: ' 3.2.9. ' ---- - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} diff --git a/content/guides/protocol-basics/muxers.md b/content/guides/protocol-basics/muxers.md deleted file mode 100644 index b9ca3530..00000000 --- a/content/guides/protocol-basics/muxers.md +++ /dev/null @@ -1,12 +0,0 @@ ---- -title: Get Started with Steam Muxers -menuTitle: Steam Muxers -weight: 4 -pre: ' 3.2.4. ' ---- - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} diff --git a/content/guides/protocol-basics/nat.md b/content/guides/protocol-basics/nat.md deleted file mode 100644 index 4e8a5451..00000000 --- a/content/guides/protocol-basics/nat.md +++ /dev/null @@ -1,12 +0,0 @@ ---- -title: Get Started with NATs -menuTitle: NATs -weight: 5 -pre: ' 3.2.5. ' ---- - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} diff --git a/content/guides/protocol-basics/peer-routing.md b/content/guides/protocol-basics/peer-routing.md deleted file mode 100644 index 66c6f944..00000000 --- a/content/guides/protocol-basics/peer-routing.md +++ /dev/null @@ -1,12 +0,0 @@ ---- -title: Get Started with Peer Routing -menuTitle: Peer Routing -weight: 7 -pre: ' 3.2.7. ' ---- - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} diff --git a/content/guides/protocol-basics/secure-communication.md b/content/guides/protocol-basics/secure-communication.md deleted file mode 100644 index ac37a05e..00000000 --- a/content/guides/protocol-basics/secure-communication.md +++ /dev/null @@ -1,12 +0,0 @@ ---- -title: Get Started with Secure Communication -menuTitle: Secure Channels -weight: 3 -pre: ' 3.2.3. ' ---- - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} diff --git a/content/guides/protocol-basics/transports.md b/content/guides/protocol-basics/transports.md deleted file mode 100644 index 75a23350..00000000 --- a/content/guides/protocol-basics/transports.md +++ /dev/null @@ -1,12 +0,0 @@ ---- -title: Get Started with Transports -menuTitle: Transports -weight: 2 -pre: ' 3.2.2. ' ---- - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} diff --git a/content/introduction/network-basics/_index.md b/content/introduction/network-basics/_index.md index 14d96c37..ba8439bc 100644 --- a/content/introduction/network-basics/_index.md +++ b/content/introduction/network-basics/_index.md @@ -11,3 +11,10 @@ Whether you’re just starting to dive into peer-to-peer concepts and solutions, learning how to build peer-to-peer systems with libp2p, or are looking for detailed reference information, this is the place to start. + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} + diff --git a/content/introduction/network-basics/computer-networks.md b/content/introduction/network-basics/computer-networks.md deleted file mode 100644 index 3c5e1c08..00000000 --- a/content/introduction/network-basics/computer-networks.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -title: Introduction to Computer Networks -weight: 2 -pre: ' 1.1.2. ' ---- - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} diff --git a/content/introduction/network-basics/data-structures.md b/content/introduction/network-basics/data-structures.md deleted file mode 100644 index 2bf94c6d..00000000 --- a/content/introduction/network-basics/data-structures.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -title: "Introduction to Data Structures" -weight: 4 -pre: ' 1.1.4. ' ---- - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} diff --git a/content/introduction/network-basics/distributed-computing.md b/content/introduction/network-basics/distributed-computing.md deleted file mode 100644 index e97fdd99..00000000 --- a/content/introduction/network-basics/distributed-computing.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -title: "Introduction to Distributed Computing" -weight: 3 -pre: ' 1.1.3. ' ---- - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} diff --git a/content/introduction/network-basics/internet-web.md b/content/introduction/network-basics/internet-web.md deleted file mode 100644 index f4d53f70..00000000 --- a/content/introduction/network-basics/internet-web.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -title: "The Internet and the Permanent Web" -weight: 1 -pre: ' 1.1.1 ' ---- - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} diff --git a/static/css/theme.css b/static/css/theme.css index 7fbcce0d..39461e2f 100644 --- a/static/css/theme.css +++ b/static/css/theme.css @@ -417,6 +417,7 @@ textarea:focus, input[type="email"]:focus, input[type="number"]:focus, input[typ justify-content: center; height: 100%; padding: 2rem 0; + font-size: 1.5rem; } #chapter #body-inner { padding-bottom: 3rem; @@ -428,12 +429,13 @@ textarea:focus, input[type="email"]:focus, input[type="number"]:focus, input[typ text-align: center; } #chapter h1 { - font-size: 5rem; + font-size: 10rem; border-bottom: 4px solid #F0F2F4; } #chapter p { - text-align: center; - font-size: 1.2rem; + text-align: left; + font-size: 1.3rem; + padding: 2rem 1rem; } #footer { padding: 3rem 1rem; From 881ddaeebc57253f2c1922b6620688099622074f Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Thu, 22 Sep 2022 11:06:35 -0400 Subject: [PATCH 10/30] edits + lp index --- content/_index.md | 31 +-- content/concepts/_index.md | 2 +- content/introduction/libp2p/_index.md | 182 +++++++++++++++++- .../introduction/libp2p/getting-started.md | 180 ----------------- static/css/theme.css | 5 +- 5 files changed, 184 insertions(+), 216 deletions(-) delete mode 100644 content/introduction/libp2p/getting-started.md diff --git a/content/_index.md b/content/_index.md index 25bb12f5..61d06cce 100644 --- a/content/_index.md +++ b/content/_index.md @@ -8,31 +8,8 @@ chapter: false # Welcome to the libp2p Book -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} +Welcome to the libp2p book! Whether you’re just learning how to build peer-to-peer systems with +libp2p, want to dive into peer-to-peer concepts and solutions, or are looking for detailed reference +information, this is the place to start. -### Chapter 1: Introduction - -Get started with computer networking and science concepts, distributed systems, and -how libp2p fits into the internet as we know it. - -### Chapter 2: Core Components - -Learn about the core components that make up a full libp2p network stack -including network components, - -### Chapter 3: Guides - -Start interacting with libp2p, spawn a node, transmit data, and learn how to -use libp2p protocols in your application. - -### Chapter 4: Reference Documentation - -Find all the available libp2p specifications and different types of reference documentation. - -### Chapter 5: Get Involved - -Learn how to get involved in the libp2p community. +{{% children depth="10" showhidden="true" %}} diff --git a/content/concepts/_index.md b/content/concepts/_index.md index 4e76a409..286080fe 100644 --- a/content/concepts/_index.md +++ b/content/concepts/_index.md @@ -1,5 +1,5 @@ --- -title: "Core Components" +title: Core Components weight: 2 pre: "2. " chapter: true diff --git a/content/introduction/libp2p/_index.md b/content/introduction/libp2p/_index.md index 31fe5f11..8b046b4d 100644 --- a/content/introduction/libp2p/_index.md +++ b/content/introduction/libp2p/_index.md @@ -1,12 +1,182 @@ --- -title: LIBP2P +title: Get Started with libp2p weight: 2 pre: ' 1.2. ' --- -# Get Started with libp2p +### Chapter 1.2 -Whether you’re just starting to dive into peer-to-peer concepts and -solutions, learning how to build peer-to-peer systems with libp2p, or -are looking for detailed reference information, this is the place to -start. +## What is libp2p? + +libp2p is a modular system of *protocols*, *specifications* and *libraries* +that enable the development of peer-to-peer network applications. + +Because of libp2p's peer-to-peer and distributed architecture, most of the +needs and considerations that the current web was built on no longer apply. +The internet, such as it is, with firewalls and NATs, was designed to [securely] +provide data by relying on trust assumptions. There are many distributed +peer-to-peer network models with different challenges and tradeoffs that try +to improve on the way we network. Libp2p aims to be a modular, general-purpose +toolkit for any peer-to-peer application. + +## Peer-to-peer basics + +Let's start with what a peer-to-peer network application is: + +A [peer-to-peer network][definition_p2p] is one in which the participants +(referred to as [peers][definition_peer]) communicate directly with one another +on a relative "equal footing". This does not mean that all peers are identical +as some may have different roles in the overall network. However, one of the +defining characteristics of a peer-to-peer network is that the network does not +require a privileged set of "servers" which behave completely differently from +their "clients", as is the case in the predominant +[client / server model][definition_client_server]. + +Because the definition of peer-to-peer networking is quite broad, many different +kinds of systems have been built that all fall under the umbrella of "peer-to-peer". +The most culturally prominent examples are likely file-sharing networks like BitTorrent, +and, more recently, the proliferation of blockchain networks that communicate in a +peer-to-peer fashion. + +### What problems can libp2p solve? + +While peer-to-peer networks have many advantages over the client-server model, +there are unique challenges that require careful thought and practice to overcome. + +libp2p lets all users preserve their network identity, overcome network censorship, +and communicate over different transport protocols. + +In overcoming these challenges while building [IPFS](https://ipfs.io), +we took care to build our solutions in a modular, composable way into what is +now libp2p. Although libp2p grew out of IPFS, it is not dependent on IPFS, and +today, [many projects][built_with_libp2p] use libp2p as their networking layer. + +Together, we can leverage our collective experience and solve these foundational +problems in a way that benefits an entire ecosystem of developers and a world of users. + +Here, we'll briefly outline the main problem areas that libp2p attempts to address. +This is an ever-growing space, so don't be surprised if things change over time. +If you notice something missing or have other ideas for improving this documentation, +please [reach out to let us know][help_improve_docs]. + +### Data transmission + +The transport layer is at the foundation of libp2p, which is responsible for +transmitting and receiving bytes between two peers. There are many +ways to send data across networks in use today, including TCP, QUIC, WebSocket, +WebTransport and WebRTC, with some still in development and others still yet +to be designed. + +libp2p also provides a list of specifications [specifcations](https://github.com/libp2p/specs) +that can be adapted to support existing and future protocols, allowing libp2p applications +to operate in many different runtime and networking environments. + +### Peer identity + +Knowing who you're talking to is key to secure and reliable communication in a world +with billions of networked devices. libp2p uses +[public key cryptography](https://en.wikipedia.org/wiki/Public-key_cryptography) +as the basis of peer identity, which serves two complementary purposes. + +1. It gives each peer a globally unique "name", in the form of a + [PeerId][definition_peerid]. +2. The `PeerId` allows anyone to retrieve the public key for the identified + peer, which enables secure communication between peers. + +### Secure communication + +There needs to be a method to securely send and receive data between peers, +where peers are able to trust the [identity](#peer-identity) of the peer they're +communicating with while ensuring that no external entity can access or tamper with +the communication. + +All libp2p connections are encrypted and authenticated. Some [transport protocol](#transport) +protocols are encrypted at the transport layer (e.g. QUIC). For other protocols, libp2p runs +a cryptographic handshake on top of an unencrypted connection (e.g. TCP). + +For secure communication channels, libp2p currently supports +[TLS 1.3](https://www.ietf.org/blog/tls13/) and [Noise](https://noiseprotocol.org/), +though not every language implementation of libp2p supports both of these. + +> (Older versions of libp2p may support a +> [deprecated](https://blog.ipfs.io/2020-08-07-deprecating-secio/) protocol called SECIO; +> all projects should switch to TLS 1.3 or Noise instead.) + +### Peer routing + +When you want to send a message to another peer, you need two key pieces +of information: their [PeerId][definition_peerid], and a way to locate them +on the network to open a connection. + +There are many cases where we only have the `PeerId` for the peer we want to +contact, and we need a way to discover their network address. Peer routing is +the process of discovering peer addresses by leveraging the knowledge of other +peers. + +In a peer routing system, a peer can either give us the address we need if they +have it, or else send our inquiry to another peer who's more likely to have the +answer. As we contact more and more peers, we not only increase our chances of +finding the peer we're looking for, we build a more complete view of the network +in our own routing tables, which enables us to answer routing queries from others. + +The current stable implementation of peer routing in libp2p uses a +[distributed hash table][definition_dht] to iteratively route requests closer +to the desired `PeerId` using the [Kademlia][wiki_kademlia] routing algorithm. + +### Content discovery + +In some systems, we care less about who we're speaking with than what they can offer us. +For example, we may want some specific piece of data, but we don't care who we get it from +since we can verify its integrity. + +libp2p provides a [content routing specification][spec_content_routing] for this +purpose, with the primary stable implementation using the same +[Kademlia][wiki_kademlia]-based DHT as used in peer routing. + +### Peer messaging + +Sending messages to other peers is at the heart of most peer-to-peer systems, +and pubsub (short for publish/subscribe) is an instrumental pattern for sending +a message to groups of interested receivers. + +libp2p defines a [pubsub specification][spec_pubsub] for sending messages to all +peers subscribed to a given "topic". The specification currently has two stable +implementations; `floodsub` uses a very simple but inefficient "network flooding" +strategy, and [gossipsub](https://github.com/libp2p/specs/tree/master/pubsub/gossipsub) +defines an extensible gossip protocol. There is also active development in progress on +[episub](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/episub.md), an +extended `gossipsub` that is optimized for single source multicast and scenarios with a +few fixed sources broadcasting to a large number of clients in a topic. + +Head over to [What is libp2p?](/introduction/what-is-libp2p/) for an introduction to +the basics of libp2p and an overview of the problems it addresses. + +## Related projects + +libp2p began as part of the [IPFS](https://ipfs.io) project and is still an +essential component of IPFS. As such, libp2p composes well with the abstractions +and tools provided by other projects in the IPFS "family". Check their sites for +specific information and references: + +- [IPFS](https://libp2p.io) is the InterPlanetary File System, which uses libp2p as + its networking layer. +- [Multiformats](https://multiformats.io) is a variety of *self-describing* data formats. +- [IPLD](https://ipld.io) is a set of tools for describing links between content-addressed + data, like IPFS files, Git commits, or Ethereum blocks. +- [The Permissive License Stack](https://protocol.ai/blog/announcing-the-permissive-license-stack) + is a licensing strategy for software development that embraces open-source values. + +[glossary]: {{< ref "concepts/miscellaneous/glossary.md" >}} +[definition_dht]: {{< ref "concepts/miscellaneous/glossary.md#dht" >}} +[definition_p2p]: {{< ref "concepts/miscellaneous/glossary.md#p2p" >}} +[definition_peer]: {{< ref "concepts/miscellaneous/glossary.md#peer" >}} +[definition_peerid]: {{< ref "concepts/miscellaneous/glossary.md#peerid" >}} +[definition_secio]: {{< ref "concepts/miscellaneous/glossary.md#secio" >}} +[definition_muiltiaddress]: {{< ref "concepts/miscellaneous/glossary.md#multiaddr" >}} +[definition_client_server]: {{< ref "concepts/miscellaneous/glossary.md#client-server" >}} + +[spec_content_routing]: https://github.com/libp2p/specs/blob/master/kad-dht/README.md +[spec_pubsub]: https://github.com/libp2p/specs/blob/master/pubsub/README.md +[built_with_libp2p]: https://discuss.libp2p.io/c/ecosystem-community +[help_improve_docs]: https://github.com/libp2p/docs/issues +[wiki_kademlia]: https://en.wikipedia.org/wiki/Kademlia diff --git a/content/introduction/libp2p/getting-started.md b/content/introduction/libp2p/getting-started.md deleted file mode 100644 index e7cada1e..00000000 --- a/content/introduction/libp2p/getting-started.md +++ /dev/null @@ -1,180 +0,0 @@ ---- -title: A Modular P2P Networking Framework -weight: 2 -pre: ' ' ---- - -## What is libp2p? - -libp2p is a modular system of *protocols*, *specifications* and *libraries* -that enable the development of peer-to-peer network applications. - -Because of libp2p's peer-to-peer and distributed architecture, most of the -needs and considerations that the current web was built on no longer apply. -The internet, such as it is, with firewalls and NATs, was designed to [securely] -provide data by relying on trust assumptions. There are many distributed -peer-to-peer network models with different challenges and tradeoffs that try -to improve on the way we network. Libp2p aims to be a modular, general-purpose -toolkit for any peer-to-peer application. - -## Peer-to-peer basics - -Let's start with what a peer-to-peer network application is: - -A [peer-to-peer network][definition_p2p] is one in which the participants -(referred to as [peers][definition_peer]) communicate directly with one another -on a relative "equal footing". This does not mean that all peers are identical -as some may have different roles in the overall network. However, one of the -defining characteristics of a peer-to-peer network is that the network does not -require a privileged set of "servers" which behave completely differently from -their "clients", as is the case in the predominant -[client / server model][definition_client_server]. - -Because the definition of peer-to-peer networking is quite broad, many different -kinds of systems have been built that all fall under the umbrella of "peer-to-peer". -The most culturally prominent examples are likely file-sharing networks like BitTorrent, -and, more recently, the proliferation of blockchain networks that communicate in a -peer-to-peer fashion. - -### What problems can libp2p solve? - -While peer-to-peer networks have many advantages over the client-server model, -there are unique challenges that require careful thought and practice to overcome. - -libp2p lets all users preserve their network identity, overcome network censorship, -and communicate over different transport protocols. - -In overcoming these challenges while building [IPFS](https://ipfs.io), -we took care to build our solutions in a modular, composable way into what is -now libp2p. Although libp2p grew out of IPFS, it is not dependent on IPFS, and -today, [many projects][built_with_libp2p] use libp2p as their networking layer. - -Together, we can leverage our collective experience and solve these foundational -problems in a way that benefits an entire ecosystem of developers and a world of users. - -Here, we'll briefly outline the main problem areas that libp2p attempts to address. -This is an ever-growing space, so don't be surprised if things change over time. -If you notice something missing or have other ideas for improving this documentation, -please [reach out to let us know][help_improve_docs]. - -### Data transmission - -The transport layer is at the foundation of libp2p, which is responsible for -transmitting and receiving bytes between two peers. There are many -ways to send data across networks in use today, including TCP, QUIC, WebSocket, -WebTransport and WebRTC, with some still in development and others still yet -to be designed. - -libp2p also provides a list of specifications [specifcations](https://github.com/libp2p/specs) -that can be adapted to support existing and future protocols, allowing libp2p applications -to operate in many different runtime and networking environments. - -### Peer identity - -Knowing who you're talking to is key to secure and reliable communication in a world -with billions of networked devices. libp2p uses -[public key cryptography](https://en.wikipedia.org/wiki/Public-key_cryptography) -as the basis of peer identity, which serves two complementary purposes. - -1. It gives each peer a globally unique "name", in the form of a - [PeerId][definition_peerid]. -2. The `PeerId` allows anyone to retrieve the public key for the identified - peer, which enables secure communication between peers. - -### Secure communication - -There needs to be a method to securely send and receive data between peers, -where peers are able to trust the [identity](#peer-identity) of the peer they're -communicating with while ensuring that no external entity can access or tamper with -the communication. - -All libp2p connections are encrypted and authenticated. Some [transport protocol](#transport) -protocols are encrypted at the transport layer (e.g. QUIC). For other protocols, libp2p runs -a cryptographic handshake on top of an unencrypted connection (e.g. TCP). - -For secure communication channels, libp2p currently supports -[TLS 1.3](https://www.ietf.org/blog/tls13/) and [Noise](https://noiseprotocol.org/), -though not every language implementation of libp2p supports both of these. - -> (Older versions of libp2p may support a -> [deprecated](https://blog.ipfs.io/2020-08-07-deprecating-secio/) protocol called SECIO; -> all projects should switch to TLS 1.3 or Noise instead.) - -### Peer routing - -When you want to send a message to another peer, you need two key pieces -of information: their [PeerId][definition_peerid], and a way to locate them -on the network to open a connection. - -There are many cases where we only have the `PeerId` for the peer we want to -contact, and we need a way to discover their network address. Peer routing is -the process of discovering peer addresses by leveraging the knowledge of other -peers. - -In a peer routing system, a peer can either give us the address we need if they -have it, or else send our inquiry to another peer who's more likely to have the -answer. As we contact more and more peers, we not only increase our chances of -finding the peer we're looking for, we build a more complete view of the network -in our own routing tables, which enables us to answer routing queries from others. - -The current stable implementation of peer routing in libp2p uses a -[distributed hash table][definition_dht] to iteratively route requests closer -to the desired `PeerId` using the [Kademlia][wiki_kademlia] routing algorithm. - -### Content discovery - -In some systems, we care less about who we're speaking with than what they can offer us. -For example, we may want some specific piece of data, but we don't care who we get it from -since we can verify its integrity. - -libp2p provides a [content routing specification][spec_content_routing] for this -purpose, with the primary stable implementation using the same -[Kademlia][wiki_kademlia]-based DHT as used in peer routing. - -### Peer messaging - -Sending messages to other peers is at the heart of most peer-to-peer systems, -and pubsub (short for publish/subscribe) is an instrumental pattern for sending -a message to groups of interested receivers. - -libp2p defines a [pubsub specification][spec_pubsub] for sending messages to all -peers subscribed to a given "topic". The specification currently has two stable -implementations; `floodsub` uses a very simple but inefficient "network flooding" -strategy, and [gossipsub](https://github.com/libp2p/specs/tree/master/pubsub/gossipsub) -defines an extensible gossip protocol. There is also active development in progress on -[episub](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/episub.md), an -extended `gossipsub` that is optimized for single source multicast and scenarios with a -few fixed sources broadcasting to a large number of clients in a topic. - -Head over to [What is libp2p?](/introduction/what-is-libp2p/) for an introduction to -the basics of libp2p and an overview of the problems it addresses. - -## Related projects - -libp2p began as part of the [IPFS](https://ipfs.io) project and is still an -essential component of IPFS. As such, libp2p composes well with the abstractions -and tools provided by other projects in the IPFS "family". Check their sites for -specific information and references: - -- [IPFS](https://libp2p.io) is the InterPlanetary File System, which uses libp2p as - its networking layer. -- [Multiformats](https://multiformats.io) is a variety of *self-describing* data formats. -- [IPLD](https://ipld.io) is a set of tools for describing links between content-addressed - data, like IPFS files, Git commits, or Ethereum blocks. -- [The Permissive License Stack](https://protocol.ai/blog/announcing-the-permissive-license-stack) - is a licensing strategy for software development that embraces open-source values. - -[glossary]: {{< ref "concepts/miscellaneous/glossary.md" >}} -[definition_dht]: {{< ref "concepts/miscellaneous/glossary.md#dht" >}} -[definition_p2p]: {{< ref "concepts/miscellaneous/glossary.md#p2p" >}} -[definition_peer]: {{< ref "concepts/miscellaneous/glossary.md#peer" >}} -[definition_peerid]: {{< ref "concepts/miscellaneous/glossary.md#peerid" >}} -[definition_secio]: {{< ref "concepts/miscellaneous/glossary.md#secio" >}} -[definition_muiltiaddress]: {{< ref "concepts/miscellaneous/glossary.md#multiaddr" >}} -[definition_client_server]: {{< ref "concepts/miscellaneous/glossary.md#client-server" >}} - -[spec_content_routing]: https://github.com/libp2p/specs/blob/master/kad-dht/README.md -[spec_pubsub]: https://github.com/libp2p/specs/blob/master/pubsub/README.md -[built_with_libp2p]: https://discuss.libp2p.io/c/ecosystem-community -[help_improve_docs]: https://github.com/libp2p/docs/issues -[wiki_kademlia]: https://en.wikipedia.org/wiki/Kademlia diff --git a/static/css/theme.css b/static/css/theme.css index 39461e2f..282f56f8 100644 --- a/static/css/theme.css +++ b/static/css/theme.css @@ -605,8 +605,9 @@ section.attachments.grey .attachments-files { padding-bottom: 0px; } .children-li p { - font-size: small; - font-style: italic; + font-size: medium; + font-style: normal; + padding-bottom: 3px; } .children-h2 p, .children-h3 p { From dc28bf62c5c4ebb8704c2162b6cc3580388ec962 Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Thu, 13 Oct 2022 17:54:58 -0400 Subject: [PATCH 11/30] chapterize subchapters --- config.toml | 1 - content/concepts/_index.md | 4 +- content/concepts/data-exchange/_index.md | 2 +- content/concepts/data-exchange/overview.md | 11 +++ content/concepts/discovery/_index.md | 2 +- content/concepts/discovery/overview.md | 5 ++ content/concepts/introduction/_index.md | 2 +- content/concepts/introduction/addressing.md | 9 +- content/concepts/introduction/peers.md | 5 ++ content/concepts/introduction/protocols.md | 8 +- content/concepts/messaging/_index.md | 4 +- .../concepts/messaging/publish-subscribe.md | 5 ++ content/concepts/miscellaneous/_index.md | 2 +- content/concepts/miscellaneous/glossary.md | 5 ++ content/concepts/multiplex/_index.md | 2 +- .../concepts/multiplex/stream-multiplexing.md | 5 ++ content/concepts/nat/_index.md | 2 +- content/concepts/nat/circuit-relay.md | 7 +- content/concepts/nat/nat.md | 7 +- content/concepts/routing/_index.md | 2 +- .../routing/content-routing/_index.md | 5 ++ .../concepts/routing/peer-routing/_index.md | 5 ++ content/concepts/secure-comm/_index.md | 2 +- content/concepts/secure-comm/overview.md | 5 ++ content/concepts/security/_index.md | 2 +- .../security/security-considerations.md | 7 +- content/concepts/transports/_index.md | 2 +- content/concepts/transports/transport.md | 7 +- content/get-involved/_index.md | 4 +- content/introduction/_index.md | 7 +- content/introduction/libp2p/_index.md | 5 +- content/introduction/network-basics/_index.md | 6 +- content/reference/_index.md | 4 +- content/reference/dos-mitigation.md | 90 ++++++++++--------- content/reference/monitoring.md | 5 ++ content/reference/overview.md | 7 +- content/reference/specs.md | 7 +- static/css/theme.css | 2 +- 38 files changed, 186 insertions(+), 76 deletions(-) diff --git a/config.toml b/config.toml index 95d50e77..d6e6072b 100644 --- a/config.toml +++ b/config.toml @@ -26,4 +26,3 @@ weight = 2 name = " Discussion Forums" url = "https://discuss.libp2p.io" weight = 3 - diff --git a/content/concepts/_index.md b/content/concepts/_index.md index 286080fe..f54cd4c3 100644 --- a/content/concepts/_index.md +++ b/content/concepts/_index.md @@ -10,9 +10,9 @@ chapter: true # Explore libp2p libp2p covers a lot of ground, and may involve unfamiliar terminology and concepts. -This section goes over the foundational concepts involved in libp2p. +This chapter goes over the foundational concepts involved in libp2p. -{{% children description="true" %}} +{{% children %}} {{% notice "note" %}} This section is incomplete, and many of the articles are stubs. To help fill in diff --git a/content/concepts/data-exchange/_index.md b/content/concepts/data-exchange/_index.md index 47d586f5..df9ebfe9 100644 --- a/content/concepts/data-exchange/_index.md +++ b/content/concepts/data-exchange/_index.md @@ -9,7 +9,7 @@ chapter: true # Exchange Data -{{% children description="true" %}} +{{% children %}} {{% notice "note" %}} This section is incomplete, and many of the articles are stubs. To help fill in diff --git a/content/concepts/data-exchange/overview.md b/content/concepts/data-exchange/overview.md index 30df3442..8fb54af7 100644 --- a/content/concepts/data-exchange/overview.md +++ b/content/concepts/data-exchange/overview.md @@ -2,4 +2,15 @@ title: 'Overview' weight: 1 pre: ' 2.8.1. ' +chapter: true --- + +### Chapter 2.8.1 + +# Overview + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/concepts/discovery/_index.md b/content/concepts/discovery/_index.md index bf0f7716..1efeba4c 100644 --- a/content/concepts/discovery/_index.md +++ b/content/concepts/discovery/_index.md @@ -9,7 +9,7 @@ chapter: true # Discover Peers -{{% children description="true" %}} +{{% children %}} {{% notice "note" %}} This section is incomplete, and many of the articles are stubs. To help fill in diff --git a/content/concepts/discovery/overview.md b/content/concepts/discovery/overview.md index b636f264..86c046ae 100644 --- a/content/concepts/discovery/overview.md +++ b/content/concepts/discovery/overview.md @@ -2,8 +2,13 @@ title: "Overview" weight: 1 pre: ' 2.5.1. ' +chapter: true --- +### Chapter 2.5.1 + +# Overview + {{% notice "note" %}} This section is incomplete, and many of the articles are stubs. To help fill in the gaps, please see the issues linked in each article to add your input and diff --git a/content/concepts/introduction/_index.md b/content/concepts/introduction/_index.md index fb038e08..2c53c1e5 100644 --- a/content/concepts/introduction/_index.md +++ b/content/concepts/introduction/_index.md @@ -9,7 +9,7 @@ chapter: true # Key Components of libp2p -{{% children description="true" %}} +{{% children %}} {{% notice "note" %}} This section is incomplete, and many of the articles are stubs. To help fill in diff --git a/content/concepts/introduction/addressing.md b/content/concepts/introduction/addressing.md index c8c92079..f223ab98 100644 --- a/content/concepts/introduction/addressing.md +++ b/content/concepts/introduction/addressing.md @@ -1,9 +1,14 @@ --- -title: Addressing +title: "Addressing" weight: 1 -pre: ' 2.0.2. ' +pre: ' 2.0.1. ' +chapter: true --- +### Chapter 2.0.1 + +# Addressing + Flexible networks need flexible addressing systems. Since libp2p is designed to work across a wide variety of networks, we need a way to work with a lot of different addressing schemes in a consistent way. A `multiaddress` (often abbreviated `multiaddr`), is a convention for encoding multiple layers of addressing information into a single "future-proof" path structure. It [defines][spec_multiaddr] human-readable and machine-optimized encodings of common transport and overlay protocols and allows many layers of addressing to be combined and used together. diff --git a/content/concepts/introduction/peers.md b/content/concepts/introduction/peers.md index 30656e00..bcafc560 100644 --- a/content/concepts/introduction/peers.md +++ b/content/concepts/introduction/peers.md @@ -2,8 +2,13 @@ title: "Peers" weight: 3 pre: ' 2.0.3. ' +chapter: true --- +### Chapter 2.0.3 + +# Peers + Peers are what make up a libp2p network. We'll go over their key properties in this guide. diff --git a/content/concepts/introduction/protocols.md b/content/concepts/introduction/protocols.md index 4111ccb2..0a067fab 100644 --- a/content/concepts/introduction/protocols.md +++ b/content/concepts/introduction/protocols.md @@ -1,9 +1,15 @@ --- -title: Protocols +title: "Protocols" weight: 2 pre: ' 2.0.2. ' +chapter: true --- +### Chapter 2.0.2 + +# Protocols + + There are protocols everywhere you look when you're writing network applications, and libp2p is especially thick with them. diff --git a/content/concepts/messaging/_index.md b/content/concepts/messaging/_index.md index f9641852..962b45a3 100644 --- a/content/concepts/messaging/_index.md +++ b/content/concepts/messaging/_index.md @@ -7,9 +7,9 @@ chapter: true ### Chapter 2.7 -# Transmit Data +# Message Peers -{{% children description="true" %}} +{{% children %}} {{% notice "note" %}} This section is incomplete, and many of the articles are stubs. To help fill in diff --git a/content/concepts/messaging/publish-subscribe.md b/content/concepts/messaging/publish-subscribe.md index 4dee30d2..b0beaf60 100644 --- a/content/concepts/messaging/publish-subscribe.md +++ b/content/concepts/messaging/publish-subscribe.md @@ -2,8 +2,13 @@ title: Publish/Subscribe weight: 2 pre: ' 2.7.2 ' +chapter: true --- +### Chapter 2.7.2 + +# Publish/Subscribe + Publish/Subscribe is a system where peers congregate around topics they are interested in. Peers interested in a topic are said to be subscribed to that topic: diff --git a/content/concepts/miscellaneous/_index.md b/content/concepts/miscellaneous/_index.md index b6bd19c0..5fc08904 100644 --- a/content/concepts/miscellaneous/_index.md +++ b/content/concepts/miscellaneous/_index.md @@ -9,7 +9,7 @@ chapter: true # Learn more -{{% children description="true" %}} +{{% children %}} {{% notice "note" %}} This section is incomplete, and many of the articles are stubs. To help fill in diff --git a/content/concepts/miscellaneous/glossary.md b/content/concepts/miscellaneous/glossary.md index 304f881a..617ec6ad 100644 --- a/content/concepts/miscellaneous/glossary.md +++ b/content/concepts/miscellaneous/glossary.md @@ -2,8 +2,13 @@ title: "Glossary" weight: 1 pre: ' 2.10.1 ' +chapter: true --- +### Chapter 2.10.1 + +# Glossary + ### Boot Node New nodes in a p2p network often make their initial connection to the p2p network through a set of nodes known as boot nodes. Information (e.g. addresses) about these boot nodes is e.g. embedded in an application binary or provided as a configuration option. diff --git a/content/concepts/multiplex/_index.md b/content/concepts/multiplex/_index.md index 223d6c47..81489c26 100644 --- a/content/concepts/multiplex/_index.md +++ b/content/concepts/multiplex/_index.md @@ -9,7 +9,7 @@ chapter: true # Stream bytes -{{% children description="true" %}} +{{% children %}} {{% notice "note" %}} This section is incomplete, and many of the articles are stubs. To help fill in diff --git a/content/concepts/multiplex/stream-multiplexing.md b/content/concepts/multiplex/stream-multiplexing.md index 2381a132..24005fe1 100644 --- a/content/concepts/multiplex/stream-multiplexing.md +++ b/content/concepts/multiplex/stream-multiplexing.md @@ -2,8 +2,13 @@ title: "Overview" weight: 1 pre: ' 2.3.1. ' +chapter: true --- +### Chapter 2.3.1 + +# Overview + Stream Multiplexing (_stream muxing_) is a way of sending multiple streams of data over one communication link. It combines multiple signals into one unified signal so it can be transported 'over the wires', then it is demulitiplexed (_demuxed_) so it can be output and used by separate applications. ## Multiplexing diff --git a/content/concepts/nat/_index.md b/content/concepts/nat/_index.md index 31538017..29ad09d5 100644 --- a/content/concepts/nat/_index.md +++ b/content/concepts/nat/_index.md @@ -9,7 +9,7 @@ chapter: true # Traverse Bytes -{{% children description="true" %}} +{{% children %}} {{% notice "note" %}} This section is incomplete, and many of the articles are stubs. To help fill in diff --git a/content/concepts/nat/circuit-relay.md b/content/concepts/nat/circuit-relay.md index e05d457d..00c3fc18 100644 --- a/content/concepts/nat/circuit-relay.md +++ b/content/concepts/nat/circuit-relay.md @@ -1,9 +1,14 @@ --- -title: Circuit Relay +title: "Circuit Relay" weight: 2 pre: ' 2.4.2. ' +chapter: true --- +### Chapter 2.4.2 + +# Circuit Relay + Circuit relay is a [transport protocol](/concepts/transport/) that routes traffic between two peers over a third-party "relay" peer. In many cases, peers will be unable to [traverse their NAT and/or firewall](/concepts/nat/) in a way that makes them publicly accessible. Or they may not share common [transport protocols](/concepts/transport/) that would allow them to communicate directly. diff --git a/content/concepts/nat/nat.md b/content/concepts/nat/nat.md index 2183d724..5decc48e 100644 --- a/content/concepts/nat/nat.md +++ b/content/concepts/nat/nat.md @@ -1,9 +1,14 @@ --- -title: Overview +title: "Overview" weight: 1 pre: ' 2.4.1. ' +chapter: true --- +### Chapter 2.4.1 + +# Overview + The internet is composed of countless networks, bound together into shared address spaces by foundational [transport protocols](/concepts/transport/). As traffic moves between network boundaries, it's very common for a process called Network Address Translation to occur. Network Address Translation (NAT) maps an address from one address space to another. diff --git a/content/concepts/routing/_index.md b/content/concepts/routing/_index.md index c92523d6..9a9d77da 100644 --- a/content/concepts/routing/_index.md +++ b/content/concepts/routing/_index.md @@ -9,7 +9,7 @@ chapter: true # Route with Peers -{{% children description="true" %}} +{{% children %}} {{% notice "note" %}} This section is incomplete, and many of the articles are stubs. To help fill in diff --git a/content/concepts/routing/content-routing/_index.md b/content/concepts/routing/content-routing/_index.md index bab33726..ed36de84 100644 --- a/content/concepts/routing/content-routing/_index.md +++ b/content/concepts/routing/content-routing/_index.md @@ -2,8 +2,13 @@ title: "Content Routing" weight: 4 pre: ' 2.6.4. ' +chapter: true --- +### Chapter 2.6.4 + +# Content Routing + {{% notice "note" %}} This section is incomplete, and many of the articles are stubs. To help fill in the gaps, please see the issues linked in each article to add your input and diff --git a/content/concepts/routing/peer-routing/_index.md b/content/concepts/routing/peer-routing/_index.md index 22095a7c..ff448c6d 100644 --- a/content/concepts/routing/peer-routing/_index.md +++ b/content/concepts/routing/peer-routing/_index.md @@ -2,8 +2,13 @@ title: "Peer Routing" weight: 3 pre: ' 2.6.3. ' +chapter: true --- +### Chapter 2.6.3 + +# Peer Routing + {{% notice "note" %}} This section is incomplete, and many of the articles are stubs. To help fill in the gaps, please see the issues linked in each article to add your input and diff --git a/content/concepts/secure-comm/_index.md b/content/concepts/secure-comm/_index.md index de489886..a1af748e 100644 --- a/content/concepts/secure-comm/_index.md +++ b/content/concepts/secure-comm/_index.md @@ -9,7 +9,7 @@ chapter: true # Secure bytes -{{% children description="true" %}} +{{% children %}} {{% notice "note" %}} This section is incomplete, and many of the articles are stubs. To help fill in diff --git a/content/concepts/secure-comm/overview.md b/content/concepts/secure-comm/overview.md index 7eadcb06..86a521ed 100644 --- a/content/concepts/secure-comm/overview.md +++ b/content/concepts/secure-comm/overview.md @@ -2,8 +2,13 @@ title: Overview weight: 1 pre: ' 2.2.1. ' +chapter: true --- +### Chapter 2.2.1 + +# Overview + {{% notice "note" %}} This section is incomplete, and many of the articles are stubs. To help fill in the gaps, please see the issues linked in each article to add your input and diff --git a/content/concepts/security/_index.md b/content/concepts/security/_index.md index 8b13b163..89be8250 100644 --- a/content/concepts/security/_index.md +++ b/content/concepts/security/_index.md @@ -9,7 +9,7 @@ chapter: true # Secure Data -{{% children description="true" %}} +{{% children %}} {{% notice "note" %}} This section is incomplete, and many of the articles are stubs. To help fill in diff --git a/content/concepts/security/security-considerations.md b/content/concepts/security/security-considerations.md index 0fa675d1..0ecc428e 100644 --- a/content/concepts/security/security-considerations.md +++ b/content/concepts/security/security-considerations.md @@ -1,9 +1,14 @@ --- -title: 'Overview' +title: "Overview" weight: 1 pre: ' 2.9.1. ' +chapter: true --- +### Chapter 2.9.1 + +# Overview + libp2p makes it simple to establish [encrypted, authenticated communication channels](../secure-comms/) between two peers, but there are other important security issues to consider when building robust peer-to-peer systems. diff --git a/content/concepts/transports/_index.md b/content/concepts/transports/_index.md index 64664d7c..5df8f6a7 100644 --- a/content/concepts/transports/_index.md +++ b/content/concepts/transports/_index.md @@ -9,7 +9,7 @@ chapter: true # Exchange Bytes -{{% children description="true" %}} +{{% children %}} {{% notice "note" %}} This section is incomplete, and many of the articles are stubs. To help fill in diff --git a/content/concepts/transports/transport.md b/content/concepts/transports/transport.md index 8b7471dc..c81b1ba9 100644 --- a/content/concepts/transports/transport.md +++ b/content/concepts/transports/transport.md @@ -1,9 +1,14 @@ --- -title: Overview +title: "Overview" weight: 1 pre: ' 2.1.1. ' +chapter: true --- +### Chapter 2.1.1 + +# Overview + When you make a connection from your computer to a machine on the internet, chances are pretty good you're sending your bits and bytes using TCP/IP, the wildly successful combination of the Internet Protocol, which handles addressing diff --git a/content/get-involved/_index.md b/content/get-involved/_index.md index a91cfd1d..e214af6c 100644 --- a/content/get-involved/_index.md +++ b/content/get-involved/_index.md @@ -7,9 +7,9 @@ chapter: true ### Chapter 5 -# Get Involved +# Get Involved with libp2p -{{% children description="true" %}} +{{% children %}} {{% notice "note" %}} This section is incomplete, and many of the articles are stubs. To help fill in diff --git a/content/introduction/_index.md b/content/introduction/_index.md index 449e5a9b..626b21fa 100644 --- a/content/introduction/_index.md +++ b/content/introduction/_index.md @@ -9,10 +9,15 @@ chapter: true # Get Started with Networking -{{% children description="true" %}} +Learn about the basics of computer networks, peer-to-peer communication, +data structures, and more and how all these components come together to create libp2p +in this chapter. + +{{% children %}} {{% notice "note" %}} This section is incomplete, and many of the articles are stubs. To help fill in the gaps, please see the issues linked in each article to add your input and help us prioritize the outstanding work. {{% /notice %}} + diff --git a/content/introduction/libp2p/_index.md b/content/introduction/libp2p/_index.md index 8b046b4d..0a4c7c7e 100644 --- a/content/introduction/libp2p/_index.md +++ b/content/introduction/libp2p/_index.md @@ -1,11 +1,14 @@ --- -title: Get Started with libp2p +title: "Get Started with libp2p" weight: 2 pre: ' 1.2. ' +chapter: true --- ### Chapter 1.2 +# Get Started with libp2p + ## What is libp2p? libp2p is a modular system of *protocols*, *specifications* and *libraries* diff --git a/content/introduction/network-basics/_index.md b/content/introduction/network-basics/_index.md index ba8439bc..6114378f 100644 --- a/content/introduction/network-basics/_index.md +++ b/content/introduction/network-basics/_index.md @@ -1,12 +1,14 @@ --- -title: Network Basics +title: "Network Basics" weight: 1 pre: ' 1.1. ' -chapter: false +chapter: true --- ### Chapter 1.1 +# Network Basics + Whether you’re just starting to dive into peer-to-peer concepts and solutions, learning how to build peer-to-peer systems with libp2p, or are looking for detailed reference information, this is the place to diff --git a/content/reference/_index.md b/content/reference/_index.md index ed2548d1..b4f2141f 100644 --- a/content/reference/_index.md +++ b/content/reference/_index.md @@ -9,7 +9,9 @@ chapter: true # Reference Documentation -{{% children description="true" %}} +This chapter covers the reference documentation available for libp2p. + +{{% children depth="10" showhidden="true" %}} {{% notice "note" %}} This section is incomplete, and many of the articles are stubs. To help fill in diff --git a/content/reference/dos-mitigation.md b/content/reference/dos-mitigation.md index af50aae1..0233ebf6 100644 --- a/content/reference/dos-mitigation.md +++ b/content/reference/dos-mitigation.md @@ -2,8 +2,13 @@ title: "DOS Mitigation" weight: 3 pre: ' 4.3. ' +chapter: true --- +### Chapter 4.3 + +# DOS Mitigation + DOS mitigation is an essential part of any P2P application. We need to design our protocols to be resilient to malicious peers. We need to monitor our application for signs of suspicious activity or an attack. And we need to be @@ -11,28 +16,29 @@ able to respond to an attack. Here we'll cover how we can use libp2p to achieve the above goals. -# Table of contents - -- [What we mean by a DOS attack](#what-we-mean-by-a-dos-attack) -- [Incorporating DOS mitigation from the start](#incorporating-dos-mitigation-from-the-start) - - [Limit the number of connections your application needs](#limit-the-number-of-connections-your-application-needs) - - [Transient Connections](#transient-connections) - - [Limit the number of concurrent streams per connection your protocol needs](#limit-the-number-of-concurrent-streams-per-connection-your-protocol-needs) - - [Reduce blast radius](#reduce-blast-radius) - - [Fail2ban](#fail2ban) - - [Leverage the resource manager to limit resource usage (go-libp2p only)](#leverage-the-resource-manager-to-limit-resource-usage-go-libp2p-only) - - [Rate limiting incoming connections (go-libp2p only)](#rate-limiting-incoming-connections-go-libp2p-only) -- [Monitoring your application](#monitoring-your-application) -- [Responding to an attack](#responding-to-an-attack) - - [Who’s misbehaving?](#whos-misbehaving) - - [How to block a misbehaving peer](#how-to-block-a-misbehaving-peer) - - [How to automate blocking with fail2ban](#how-to-automate-blocking-with-fail2ban) - - [Example screen recording of fail2ban in action](#example-screen-recording-of-fail2ban-in-action) - - [Setting Up fail2ban](#setting-up-fail2ban) - - [Leverage Resource Manager and a set of trusted peers to form an allow list (go-libp2p only)](#leverage-resource-manager-and-a-set-of-trusted-peers-to-form-an-allow-list-go-libp2p-only) -- [Summary](#summary) - -# What we mean by a DOS attack +## Table of contents + +- [DOS Mitigation](#dos-mitigation) + - [What we mean by a DOS attack](#what-we-mean-by-a-dos-attack) + - [Incorporating DOS mitigation from the start](#incorporating-dos-mitigation-from-the-start) + - [Limit the number of connections your application needs](#limit-the-number-of-connections-your-application-needs) + - [Transient Connections](#transient-connections) + - [Limit the number of concurrent streams per connection your protocol needs](#limit-the-number-of-concurrent-streams-per-connection-your-protocol-needs) + - [Reduce blast radius](#reduce-blast-radius) + - [Fail2ban](#fail2ban) + - [Leverage the resource manager to limit resource usage (go-libp2p only)](#leverage-the-resource-manager-to-limit-resource-usage-go-libp2p-only) + - [Rate limiting incoming connections (go-libp2p only)](#rate-limiting-incoming-connections-go-libp2p-only) + - [Monitoring your application](#monitoring-your-application) + - [Responding to an attack](#responding-to-an-attack) + - [Who’s misbehaving?](#whos-misbehaving) + - [How to block a misbehaving peer](#how-to-block-a-misbehaving-peer) + - [How to automate blocking with fail2ban](#how-to-automate-blocking-with-fail2ban) + - [Example screen recording of fail2ban in action](#example-screen-recording-of-fail2ban-in-action) + - [Setting Up fail2ban](#setting-up-fail2ban) + - [Leverage Resource Manager and a set of trusted peers to form an allow list (go-libp2p only)](#leverage-resource-manager-and-a-set-of-trusted-peers-to-form-an-allow-list-go-libp2p-only) + - [Summary](#summary) + +## What we mean by a DOS attack A DOS attack is any attack that can cause your application to crash, stall, or otherwise fail to respond normally. An attack is considered viable if it takes @@ -64,7 +70,7 @@ stay up and healthy. In the next section we'll cover some design strategies you should incorporate into your protocol to make sure your application stays up and healthy. -# Incorporating DOS mitigation from the start +## Incorporating DOS mitigation from the start The general strategy is to use the minimum amount of resources as possible and make sure that there's no untrusted amplification mechanism (e.g. an untrusted @@ -75,7 +81,7 @@ below). Below are some more specific recommendations -## Limit the number of connections your application needs +### Limit the number of connections your application needs Each connection has a resource cost associated with it. A connection will usually represent a peer and a set of protocols with each their own resource @@ -113,7 +119,7 @@ In rust-libp2p this is done by using and passing it to the [`SwarmBuilder`](https://docs.rs/libp2p/latest/libp2p/swarm/struct.SwarmBuilder.html#method.connection_limits). -## Transient Connections +### Transient Connections When a connection is first established to libp2p but before that connection has been tied to a specific peer (before security and muxer have been negotiated), @@ -132,7 +138,7 @@ scope](https://github.com/libp2p/go-libp2p/blob/v0.22.0/p2p/host/resource-manage In rust-libp2p you can tune this with `ConnectionLimits` as explained above. -## Limit the number of concurrent streams per connection your protocol needs +### Limit the number of concurrent streams per connection your protocol needs Each stream has some resource cost associated with it. Depending on the transport and multiplexer, this can be bigger or smaller. Design your protocol @@ -172,7 +178,7 @@ If we add a limit in this protocol of say 10 streams, then method 1 will mean we can only have 10 concurrent RPC calls, while method 2 would let us have a much larger number of concurrent RPC calls. -## Reduce blast radius +### Reduce blast radius If you can split up your libp2p application into multiple separate processes you can increase the resiliency of your overall system. For example, your node may @@ -180,15 +186,14 @@ have to help achieve consensus and respond to user queries. By splitting this up into two processes you now rely on the OS’s guarantee that the user query process won’t take down the consensus process. -## Fail2ban +### Fail2ban If you can log when a peer is misbehaving or is malicious, you can then hook up those logs to fail2ban and have fail2ban manage your firewall to automatically block misbehaving nodes. go-libp2p includes some built-in support for this use case. More details below. - -## Leverage the resource manager to limit resource usage (go-libp2p only) +### Leverage the resource manager to limit resource usage (go-libp2p only) go-libp2p includes a powerful [resource manager](https://github.com/libp2p/go-libp2p/tree/master/p2p/host/resource-manager) that keeps track @@ -197,7 +202,7 @@ within your protocol implementation to make sure you don't allocate more than some predetermined amount of memory per connection. It's basically a resource accounting abstraction that you can make use of in your own application. -## Rate limiting incoming connections (go-libp2p only) +### Rate limiting incoming connections (go-libp2p only) Depending on your use case, it can help to limit the number of inbound connections. You can use go-libp2p's @@ -206,17 +211,15 @@ and `InterceptAccept` for this. For a concrete example, take a look at how Prysm implements their [Connection Gater](https://github.com/prysmaticlabs/prysm/blob/63a8690140c00ba6e3e4054cac3f38a5107b7fb2/beacon-chain/p2p/connection_gater.go#L43). -# Monitoring your application +## Monitoring your application Once we've designed our protocols to be resilient to DOS attacks and deployed them, we then need to monitor our application both to verify our mitigation works and to be alerted if a new attack vector is exploited. - Monitoring is implementation specific, so consult the links below to see how your implementation does it. - For rust-libp2p look at the [libp2p-metrics crate](https://github.com/libp2p/rust-libp2p/tree/master/misc/metrics). For go-libp2p resource usage take a look at the OpenCensus metrics exposed by the resource @@ -226,12 +229,12 @@ In general, go-libp2p wants to add more metrics across the stack. This work is being tracked in issue [go-libp2p#1356](https://github.com/libp2p/go-libp2p/issues/1356). -# Responding to an attack +## Responding to an attack When you see that your node is being attacked (e.g. crashing, stalling, high cpu usage), then the next step is responding to the attack. -## Who’s misbehaving? +### Who’s misbehaving? To answer the question of which peer is misbehaving and harming you, go-libp2p exposes a [canonical log @@ -253,7 +256,7 @@ or by setting the environment variable `GOLOG_LOG_LEVEL="canonical-log=info"`. In rust-libp2p you can do something similar yourself by logging a sample of connection events from [SwarmEvent](https://docs.rs/libp2p/latest/libp2p/swarm/enum.SwarmEvent.html). -## How to block a misbehaving peer +### How to block a misbehaving peer Once you’ve identified the misbehaving peer, you can block them with `iptables` or `ufw`. Here we’ll outline how to block the peer with `ufw`. You can get the @@ -264,7 +267,7 @@ ip address of the peer from the sudo ufw deny from 1.2.3.4 ``` -## How to automate blocking with fail2ban +### How to automate blocking with fail2ban You can hook up [fail2ban](https://www.fail2ban.org) to automatically block connections from these misbehaving peers if they emit this @@ -327,14 +330,13 @@ RestartSec=1min User=ipfs ``` -### Example screen recording of fail2ban in action +#### Example screen recording of fail2ban in action - [fail2ban+go-libp2p screen recording](/images/fail2bango-libp2p.mp4) -### Setting Up fail2ban +#### Setting Up fail2ban We’ll focus on the specifics around fail2ban and go-libp2p here. The steps to take are: @@ -362,7 +364,7 @@ Status for the jail: go-libp2p-weird-behavior-iptables Then you’re good to go! You’ve successfully set up a go-libp2p jail. -## Leverage Resource Manager and a set of trusted peers to form an allow list (go-libp2p only) +### Leverage Resource Manager and a set of trusted peers to form an allow list (go-libp2p only) The [resource manager](https://github.com/libp2p/go-libp2p/tree/master/p2p/host/resource-manager) can accept a list of trusted multiaddrs and can use a different set of limits in @@ -372,11 +374,11 @@ higher limits for trusted peers. See the [allowlist section](https://github.com/libp2p/go-libp2p/tree/master/p2p/host/resource-manager#allowlisting-multiaddrs-to-mitigate-eclipse-attacks) for more details. -# Summary +## Summary Mitigating DOS attacks is hard because an attacker needs only one flaw, while a protocol developer needs to cover _all_ their bases. Libp2p provides some tools to design better protocols, but developers should still monitor their applications to protect against novel attacks. Finally, developers should leverage existing tools like `fail2ban` to automate blocking misbehaving nodes -by logging when peers behave maliciously. \ No newline at end of file +by logging when peers behave maliciously. diff --git a/content/reference/monitoring.md b/content/reference/monitoring.md index d4778f29..3791403c 100644 --- a/content/reference/monitoring.md +++ b/content/reference/monitoring.md @@ -2,8 +2,13 @@ title: "Monitoring and Observability" weight: 4 pre: ' 4.4. ' +chapter: true --- +### Chapter 4.4 + +# Monitoring and Observability + Reference the [Monitoring your application](../dos-mitigation/#monitoring-your-application) section in [DOS Mitigation](../dos-mitigation/). diff --git a/content/reference/overview.md b/content/reference/overview.md index d295e0e1..bd6f1a14 100644 --- a/content/reference/overview.md +++ b/content/reference/overview.md @@ -1,9 +1,14 @@ --- -title: Overview +title: "Overview" weight: 1 pre: ' 4.1. ' +chapter: true --- +### Chapter 4.1 + +# Overview + This section contains in-depth reference material, including a [glossary](/reference/glossary/) with definitions of libp2p terminology. ### Specifications & Planning diff --git a/content/reference/specs.md b/content/reference/specs.md index cff9cdd7..b7ea1a1e 100644 --- a/content/reference/specs.md +++ b/content/reference/specs.md @@ -1,9 +1,14 @@ --- -title: Specifications +title: "Specifications" weight: 2 pre: ' 4.2. ' +chapter: true --- +### Chapter 4.2 + +# Specifications + Check out the [specs repo](https://github.com/libp2p/specs) for details about the libp2p architecture. {{% notice "note" %}} diff --git a/static/css/theme.css b/static/css/theme.css index 996f3841..2d27f4eb 100644 --- a/static/css/theme.css +++ b/static/css/theme.css @@ -614,7 +614,7 @@ section.attachments.grey .attachments-files { font-size: large; margin-top: 0px; padding-top: 0px; - margin-bottom: 0px; + margin-bottom: 0px; padding-bottom: 0px; } .children h3,.children h2 { From 25971830465b4b732565ed3b5618ee0d601237a4 Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Thu, 13 Oct 2022 20:35:09 -0400 Subject: [PATCH 12/30] edits --- content/concepts/transports/{transport.md => overview.md} | 0 content/get-involved/_index.md | 2 ++ content/reference/_index.md | 4 ++-- 3 files changed, 4 insertions(+), 2 deletions(-) rename content/concepts/transports/{transport.md => overview.md} (100%) diff --git a/content/concepts/transports/transport.md b/content/concepts/transports/overview.md similarity index 100% rename from content/concepts/transports/transport.md rename to content/concepts/transports/overview.md diff --git a/content/get-involved/_index.md b/content/get-involved/_index.md index e214af6c..9f02646e 100644 --- a/content/get-involved/_index.md +++ b/content/get-involved/_index.md @@ -9,6 +9,8 @@ chapter: true # Get Involved with libp2p +Find out how to get involved with libp2p and join the community. + {{% children %}} {{% notice "note" %}} diff --git a/content/reference/_index.md b/content/reference/_index.md index b4f2141f..915f6642 100644 --- a/content/reference/_index.md +++ b/content/reference/_index.md @@ -9,9 +9,9 @@ chapter: true # Reference Documentation -This chapter covers the reference documentation available for libp2p. +This chapter includes the reference documentation available for libp2p. -{{% children depth="10" showhidden="true" %}} +{{% children %}} {{% notice "note" %}} This section is incomplete, and many of the articles are stubs. To help fill in From 5a1fa2ac1b4bb02b985fbde45b3ea72d8beed7df Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Thu, 13 Oct 2022 20:42:23 -0400 Subject: [PATCH 13/30] mv dos mitigation to core components --- content/concepts/dos-mitigation/_index.md | 18 ++++++++++++++++++ .../dos-mitigation/overview.md} | 12 ++++++------ content/concepts/miscellaneous/_index.md | 6 +++--- content/reference/monitoring.md | 4 ++-- 4 files changed, 29 insertions(+), 11 deletions(-) create mode 100644 content/concepts/dos-mitigation/_index.md rename content/{reference/dos-mitigation.md => concepts/dos-mitigation/overview.md} (99%) diff --git a/content/concepts/dos-mitigation/_index.md b/content/concepts/dos-mitigation/_index.md new file mode 100644 index 00000000..e1fff4c9 --- /dev/null +++ b/content/concepts/dos-mitigation/_index.md @@ -0,0 +1,18 @@ +--- +title: "DOS Mitigation" +weight: 11 +pre: ' 2.10. ' +chapter: true +--- + +### Chapter 2.10 + +# DOS Mitigation + +{{% children %}} + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/reference/dos-mitigation.md b/content/concepts/dos-mitigation/overview.md similarity index 99% rename from content/reference/dos-mitigation.md rename to content/concepts/dos-mitigation/overview.md index 0233ebf6..90fba561 100644 --- a/content/reference/dos-mitigation.md +++ b/content/concepts/dos-mitigation/overview.md @@ -1,13 +1,13 @@ --- -title: "DOS Mitigation" -weight: 3 -pre: ' 4.3. ' +title: "Overview" +weight: 1 +pre: ' 2.10.1 ' chapter: true --- -### Chapter 4.3 +### Chapter 2.10.1 -# DOS Mitigation +# Overview DOS mitigation is an essential part of any P2P application. We need to design our protocols to be resilient to malicious peers. We need to monitor our @@ -18,7 +18,7 @@ Here we'll cover how we can use libp2p to achieve the above goals. ## Table of contents -- [DOS Mitigation](#dos-mitigation) +- [Overview](#overview) - [What we mean by a DOS attack](#what-we-mean-by-a-dos-attack) - [Incorporating DOS mitigation from the start](#incorporating-dos-mitigation-from-the-start) - [Limit the number of connections your application needs](#limit-the-number-of-connections-your-application-needs) diff --git a/content/concepts/miscellaneous/_index.md b/content/concepts/miscellaneous/_index.md index 5fc08904..0e818d48 100644 --- a/content/concepts/miscellaneous/_index.md +++ b/content/concepts/miscellaneous/_index.md @@ -1,11 +1,11 @@ --- title: "Miscellaneous" -weight: 11 -pre: ' 2.10. ' +weight: 12 +pre: ' 2.11. ' chapter: true --- -### Chapter 2.10 +### Chapter 2.11 # Learn more diff --git a/content/reference/monitoring.md b/content/reference/monitoring.md index 3791403c..5141fea8 100644 --- a/content/reference/monitoring.md +++ b/content/reference/monitoring.md @@ -1,11 +1,11 @@ --- title: "Monitoring and Observability" weight: 4 -pre: ' 4.4. ' +pre: ' 4.3. ' chapter: true --- -### Chapter 4.4 +### Chapter 4.3 # Monitoring and Observability From 57bb28d821375c6df5b2837dd679fb4db4d6f4f5 Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Thu, 13 Oct 2022 20:45:52 -0400 Subject: [PATCH 14/30] fix subchapters out of order --- content/concepts/messaging/publish-subscribe.md | 6 +++--- content/concepts/routing/content-routing/_index.md | 6 +++--- content/concepts/routing/peer-routing/_index.md | 6 +++--- 3 files changed, 9 insertions(+), 9 deletions(-) diff --git a/content/concepts/messaging/publish-subscribe.md b/content/concepts/messaging/publish-subscribe.md index b0beaf60..a878fc28 100644 --- a/content/concepts/messaging/publish-subscribe.md +++ b/content/concepts/messaging/publish-subscribe.md @@ -1,11 +1,11 @@ --- title: Publish/Subscribe -weight: 2 -pre: ' 2.7.2 ' +weight: 1 +pre: ' 2.7.1 ' chapter: true --- -### Chapter 2.7.2 +### Chapter 2.7.1 # Publish/Subscribe diff --git a/content/concepts/routing/content-routing/_index.md b/content/concepts/routing/content-routing/_index.md index ed36de84..bb002027 100644 --- a/content/concepts/routing/content-routing/_index.md +++ b/content/concepts/routing/content-routing/_index.md @@ -1,11 +1,11 @@ --- title: "Content Routing" -weight: 4 -pre: ' 2.6.4. ' +weight: 2 +pre: ' 2.6.2. ' chapter: true --- -### Chapter 2.6.4 +### Chapter 2.6.2 # Content Routing diff --git a/content/concepts/routing/peer-routing/_index.md b/content/concepts/routing/peer-routing/_index.md index ff448c6d..3e0cf136 100644 --- a/content/concepts/routing/peer-routing/_index.md +++ b/content/concepts/routing/peer-routing/_index.md @@ -1,11 +1,11 @@ --- title: "Peer Routing" -weight: 3 -pre: ' 2.6.3. ' +weight: 1 +pre: ' 2.6.1. ' chapter: true --- -### Chapter 2.6.3 +### Chapter 2.6.1 # Peer Routing From f72842c391b0f9f5c3b5b905ff4af592228e1ea9 Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Thu, 13 Oct 2022 21:07:06 -0400 Subject: [PATCH 15/30] add required redirects --- content/_index.md | 2 +- content/concepts/dos-mitigation/_index.md | 1 + content/concepts/introduction/addressing.md | 1 + content/concepts/introduction/peers.md | 3 +++ content/concepts/introduction/protocols.md | 1 + content/concepts/miscellaneous/glossary.md | 4 ++-- 6 files changed, 9 insertions(+), 3 deletions(-) diff --git a/content/_index.md b/content/_index.md index 61d06cce..aa9471f3 100644 --- a/content/_index.md +++ b/content/_index.md @@ -12,4 +12,4 @@ Welcome to the libp2p book! Whether you’re just learning how to build peer-to- libp2p, want to dive into peer-to-peer concepts and solutions, or are looking for detailed reference information, this is the place to start. -{{% children depth="10" showhidden="true" %}} +{{% children depth="8" showhidden="true" %}} diff --git a/content/concepts/dos-mitigation/_index.md b/content/concepts/dos-mitigation/_index.md index e1fff4c9..b3a1fb5e 100644 --- a/content/concepts/dos-mitigation/_index.md +++ b/content/concepts/dos-mitigation/_index.md @@ -3,6 +3,7 @@ title: "DOS Mitigation" weight: 11 pre: ' 2.10. ' chapter: true +aliases: /reference/dos-mitigation/ --- ### Chapter 2.10 diff --git a/content/concepts/introduction/addressing.md b/content/concepts/introduction/addressing.md index f223ab98..b7e2692f 100644 --- a/content/concepts/introduction/addressing.md +++ b/content/concepts/introduction/addressing.md @@ -3,6 +3,7 @@ title: "Addressing" weight: 1 pre: ' 2.0.1. ' chapter: true +aliases: /concepts/addressing/ --- ### Chapter 2.0.1 diff --git a/content/concepts/introduction/peers.md b/content/concepts/introduction/peers.md index bcafc560..aa7ab312 100644 --- a/content/concepts/introduction/peers.md +++ b/content/concepts/introduction/peers.md @@ -3,6 +3,9 @@ title: "Peers" weight: 3 pre: ' 2.0.3. ' chapter: true +aliases: + - /concepts/peer-id/ + - /concepts/peers/ --- ### Chapter 2.0.3 diff --git a/content/concepts/introduction/protocols.md b/content/concepts/introduction/protocols.md index 0a067fab..f76a64d0 100644 --- a/content/concepts/introduction/protocols.md +++ b/content/concepts/introduction/protocols.md @@ -3,6 +3,7 @@ title: "Protocols" weight: 2 pre: ' 2.0.2. ' chapter: true +aliases: /concepts/protocols/ --- ### Chapter 2.0.2 diff --git a/content/concepts/miscellaneous/glossary.md b/content/concepts/miscellaneous/glossary.md index 617ec6ad..95895131 100644 --- a/content/concepts/miscellaneous/glossary.md +++ b/content/concepts/miscellaneous/glossary.md @@ -1,11 +1,11 @@ --- title: "Glossary" weight: 1 -pre: ' 2.10.1 ' +pre: ' 2.11.1 ' chapter: true --- -### Chapter 2.10.1 +### Chapter 2.11.1 # Glossary From 16943a6e36225b1a705176c63a2fc045daaeb5ed Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Fri, 14 Oct 2022 05:57:51 -0400 Subject: [PATCH 16/30] rmv temp dir for network basics until docs exist --- content/introduction/_index.md | 185 ++++++++++++++++-- content/introduction/libp2p/_index.md | 185 ------------------ content/introduction/network-basics/_index.md | 22 --- 3 files changed, 174 insertions(+), 218 deletions(-) delete mode 100644 content/introduction/libp2p/_index.md delete mode 100644 content/introduction/network-basics/_index.md diff --git a/content/introduction/_index.md b/content/introduction/_index.md index 626b21fa..b5752804 100644 --- a/content/introduction/_index.md +++ b/content/introduction/_index.md @@ -1,5 +1,6 @@ --- -title: Introduction +title: "Get Started with libp2p" +menuTitle: "Introduction" weight: 1 pre: "1. " chapter: true @@ -7,17 +8,179 @@ chapter: true ### Chapter 1 -# Get Started with Networking +# Get Started with libp2p -Learn about the basics of computer networks, peer-to-peer communication, -data structures, and more and how all these components come together to create libp2p -in this chapter. +## What is libp2p? -{{% children %}} +libp2p is a modular system of *protocols*, *specifications* and *libraries* +that enable the development of peer-to-peer network applications. -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} +Because of libp2p's peer-to-peer and distributed architecture, most of the +needs and considerations that the current web was built on no longer apply. +The internet, such as it is, with firewalls and NATs, was designed to [securely] +provide data by relying on trust assumptions. There are many distributed +peer-to-peer network models with different challenges and tradeoffs that try +to improve on the way we network. Libp2p aims to be a modular, general-purpose +toolkit for any peer-to-peer application. +## Peer-to-peer basics + +Let's start with what a peer-to-peer network application is: + +A [peer-to-peer network][definition_p2p] is one in which the participants +(referred to as [peers][definition_peer]) communicate directly with one another +on a relative "equal footing". This does not mean that all peers are identical +as some may have different roles in the overall network. However, one of the +defining characteristics of a peer-to-peer network is that the network does not +require a privileged set of "servers" which behave completely differently from +their "clients", as is the case in the predominant +[client / server model][definition_client_server]. + +Because the definition of peer-to-peer networking is quite broad, many different +kinds of systems have been built that all fall under the umbrella of "peer-to-peer". +The most culturally prominent examples are likely file-sharing networks like BitTorrent, +and, more recently, the proliferation of blockchain networks that communicate in a +peer-to-peer fashion. + +### What problems can libp2p solve? + +While peer-to-peer networks have many advantages over the client-server model, +there are unique challenges that require careful thought and practice to overcome. + +libp2p lets all users preserve their network identity, overcome network censorship, +and communicate over different transport protocols. + +In overcoming these challenges while building [IPFS](https://ipfs.io), +we took care to build our solutions in a modular, composable way into what is +now libp2p. Although libp2p grew out of IPFS, it is not dependent on IPFS, and +today, [many projects][built_with_libp2p] use libp2p as their networking layer. + +Together, we can leverage our collective experience and solve these foundational +problems in a way that benefits an entire ecosystem of developers and a world of users. + +Here, we'll briefly outline the main problem areas that libp2p attempts to address. +This is an ever-growing space, so don't be surprised if things change over time. +If you notice something missing or have other ideas for improving this documentation, +please [reach out to let us know][help_improve_docs]. + +### Data transmission + +The transport layer is at the foundation of libp2p, which is responsible for +transmitting and receiving bytes between two peers. There are many +ways to send data across networks in use today, including TCP, QUIC, WebSocket, +WebTransport and WebRTC, with some still in development and others still yet +to be designed. + +libp2p also provides a list of specifications [specifcations](https://github.com/libp2p/specs) +that can be adapted to support existing and future protocols, allowing libp2p applications +to operate in many different runtime and networking environments. + +### Peer identity + +Knowing who you're talking to is key to secure and reliable communication in a world +with billions of networked devices. libp2p uses +[public key cryptography](https://en.wikipedia.org/wiki/Public-key_cryptography) +as the basis of peer identity, which serves two complementary purposes. + +1. It gives each peer a globally unique "name", in the form of a + [PeerId][definition_peerid]. +2. The `PeerId` allows anyone to retrieve the public key for the identified + peer, which enables secure communication between peers. + +### Secure communication + +There needs to be a method to securely send and receive data between peers, +where peers are able to trust the [identity](#peer-identity) of the peer they're +communicating with while ensuring that no external entity can access or tamper with +the communication. + +All libp2p connections are encrypted and authenticated. Some [transport protocol](#transport) +protocols are encrypted at the transport layer (e.g. QUIC). For other protocols, libp2p runs +a cryptographic handshake on top of an unencrypted connection (e.g. TCP). + +For secure communication channels, libp2p currently supports +[TLS 1.3](https://www.ietf.org/blog/tls13/) and [Noise](https://noiseprotocol.org/), +though not every language implementation of libp2p supports both of these. + +> (Older versions of libp2p may support a +> [deprecated](https://blog.ipfs.io/2020-08-07-deprecating-secio/) protocol called SECIO; +> all projects should switch to TLS 1.3 or Noise instead.) + +### Peer routing + +When you want to send a message to another peer, you need two key pieces +of information: their [PeerId][definition_peerid], and a way to locate them +on the network to open a connection. + +There are many cases where we only have the `PeerId` for the peer we want to +contact, and we need a way to discover their network address. Peer routing is +the process of discovering peer addresses by leveraging the knowledge of other +peers. + +In a peer routing system, a peer can either give us the address we need if they +have it, or else send our inquiry to another peer who's more likely to have the +answer. As we contact more and more peers, we not only increase our chances of +finding the peer we're looking for, we build a more complete view of the network +in our own routing tables, which enables us to answer routing queries from others. + +The current stable implementation of peer routing in libp2p uses a +[distributed hash table][definition_dht] to iteratively route requests closer +to the desired `PeerId` using the [Kademlia][wiki_kademlia] routing algorithm. + +### Content discovery + +In some systems, we care less about who we're speaking with than what they can offer us. +For example, we may want some specific piece of data, but we don't care who we get it from +since we can verify its integrity. + +libp2p provides a [content routing specification][spec_content_routing] for this +purpose, with the primary stable implementation using the same +[Kademlia][wiki_kademlia]-based DHT as used in peer routing. + +### Peer messaging + +Sending messages to other peers is at the heart of most peer-to-peer systems, +and pubsub (short for publish/subscribe) is an instrumental pattern for sending +a message to groups of interested receivers. + +libp2p defines a [pubsub specification][spec_pubsub] for sending messages to all +peers subscribed to a given "topic". The specification currently has two stable +implementations; `floodsub` uses a very simple but inefficient "network flooding" +strategy, and [gossipsub](https://github.com/libp2p/specs/tree/master/pubsub/gossipsub) +defines an extensible gossip protocol. There is also active development in progress on +[episub](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/episub.md), an +extended `gossipsub` that is optimized for single source multicast and scenarios with a +few fixed sources broadcasting to a large number of clients in a topic. + +Head over to [What is libp2p?](/introduction/what-is-libp2p/) for an introduction to +the basics of libp2p and an overview of the problems it addresses. + +## Related projects + +libp2p began as part of the [IPFS](https://ipfs.io) project and is still an +essential component of IPFS. As such, libp2p composes well with the abstractions +and tools provided by other projects in the IPFS "family". Check their sites for +specific information and references: + +- [IPFS](https://libp2p.io) is the InterPlanetary File System, which uses libp2p as + its networking layer. +- [Multiformats](https://multiformats.io) is a variety of *self-describing* data formats. +- [IPLD](https://ipld.io) is a set of tools for describing links between content-addressed + data, like IPFS files, Git commits, or Ethereum blocks. +- [The Permissive License Stack](https://protocol.ai/blog/announcing-the-permissive-license-stack) + is a licensing strategy for software development that embraces open-source values. + +[glossary]: {{< ref "concepts/miscellaneous/glossary.md" >}} +[definition_dht]: {{< ref "concepts/miscellaneous/glossary.md#dht" >}} +[definition_p2p]: {{< ref "concepts/miscellaneous/glossary.md#p2p" >}} +[definition_peer]: {{< ref "concepts/miscellaneous/glossary.md#peer" >}} +[definition_peerid]: {{< ref "concepts/miscellaneous/glossary.md#peerid" >}} +[definition_secio]: {{< ref "concepts/miscellaneous/glossary.md#secio" >}} +[definition_muiltiaddress]: {{< ref "concepts/miscellaneous/glossary.md#multiaddr" >}} +[definition_client_server]: {{< ref "concepts/miscellaneous/glossary.md#client-server" >}} + +[spec_content_routing]: https://github.com/libp2p/specs/blob/master/kad-dht/README.md +[spec_pubsub]: https://github.com/libp2p/specs/blob/master/pubsub/README.md +[built_with_libp2p]: https://discuss.libp2p.io/c/ecosystem-community +[help_improve_docs]: https://github.com/libp2p/docs/issues +[wiki_kademlia]: https://en.wikipedia.org/wiki/Kademlia diff --git a/content/introduction/libp2p/_index.md b/content/introduction/libp2p/_index.md deleted file mode 100644 index 0a4c7c7e..00000000 --- a/content/introduction/libp2p/_index.md +++ /dev/null @@ -1,185 +0,0 @@ ---- -title: "Get Started with libp2p" -weight: 2 -pre: ' 1.2. ' -chapter: true ---- - -### Chapter 1.2 - -# Get Started with libp2p - -## What is libp2p? - -libp2p is a modular system of *protocols*, *specifications* and *libraries* -that enable the development of peer-to-peer network applications. - -Because of libp2p's peer-to-peer and distributed architecture, most of the -needs and considerations that the current web was built on no longer apply. -The internet, such as it is, with firewalls and NATs, was designed to [securely] -provide data by relying on trust assumptions. There are many distributed -peer-to-peer network models with different challenges and tradeoffs that try -to improve on the way we network. Libp2p aims to be a modular, general-purpose -toolkit for any peer-to-peer application. - -## Peer-to-peer basics - -Let's start with what a peer-to-peer network application is: - -A [peer-to-peer network][definition_p2p] is one in which the participants -(referred to as [peers][definition_peer]) communicate directly with one another -on a relative "equal footing". This does not mean that all peers are identical -as some may have different roles in the overall network. However, one of the -defining characteristics of a peer-to-peer network is that the network does not -require a privileged set of "servers" which behave completely differently from -their "clients", as is the case in the predominant -[client / server model][definition_client_server]. - -Because the definition of peer-to-peer networking is quite broad, many different -kinds of systems have been built that all fall under the umbrella of "peer-to-peer". -The most culturally prominent examples are likely file-sharing networks like BitTorrent, -and, more recently, the proliferation of blockchain networks that communicate in a -peer-to-peer fashion. - -### What problems can libp2p solve? - -While peer-to-peer networks have many advantages over the client-server model, -there are unique challenges that require careful thought and practice to overcome. - -libp2p lets all users preserve their network identity, overcome network censorship, -and communicate over different transport protocols. - -In overcoming these challenges while building [IPFS](https://ipfs.io), -we took care to build our solutions in a modular, composable way into what is -now libp2p. Although libp2p grew out of IPFS, it is not dependent on IPFS, and -today, [many projects][built_with_libp2p] use libp2p as their networking layer. - -Together, we can leverage our collective experience and solve these foundational -problems in a way that benefits an entire ecosystem of developers and a world of users. - -Here, we'll briefly outline the main problem areas that libp2p attempts to address. -This is an ever-growing space, so don't be surprised if things change over time. -If you notice something missing or have other ideas for improving this documentation, -please [reach out to let us know][help_improve_docs]. - -### Data transmission - -The transport layer is at the foundation of libp2p, which is responsible for -transmitting and receiving bytes between two peers. There are many -ways to send data across networks in use today, including TCP, QUIC, WebSocket, -WebTransport and WebRTC, with some still in development and others still yet -to be designed. - -libp2p also provides a list of specifications [specifcations](https://github.com/libp2p/specs) -that can be adapted to support existing and future protocols, allowing libp2p applications -to operate in many different runtime and networking environments. - -### Peer identity - -Knowing who you're talking to is key to secure and reliable communication in a world -with billions of networked devices. libp2p uses -[public key cryptography](https://en.wikipedia.org/wiki/Public-key_cryptography) -as the basis of peer identity, which serves two complementary purposes. - -1. It gives each peer a globally unique "name", in the form of a - [PeerId][definition_peerid]. -2. The `PeerId` allows anyone to retrieve the public key for the identified - peer, which enables secure communication between peers. - -### Secure communication - -There needs to be a method to securely send and receive data between peers, -where peers are able to trust the [identity](#peer-identity) of the peer they're -communicating with while ensuring that no external entity can access or tamper with -the communication. - -All libp2p connections are encrypted and authenticated. Some [transport protocol](#transport) -protocols are encrypted at the transport layer (e.g. QUIC). For other protocols, libp2p runs -a cryptographic handshake on top of an unencrypted connection (e.g. TCP). - -For secure communication channels, libp2p currently supports -[TLS 1.3](https://www.ietf.org/blog/tls13/) and [Noise](https://noiseprotocol.org/), -though not every language implementation of libp2p supports both of these. - -> (Older versions of libp2p may support a -> [deprecated](https://blog.ipfs.io/2020-08-07-deprecating-secio/) protocol called SECIO; -> all projects should switch to TLS 1.3 or Noise instead.) - -### Peer routing - -When you want to send a message to another peer, you need two key pieces -of information: their [PeerId][definition_peerid], and a way to locate them -on the network to open a connection. - -There are many cases where we only have the `PeerId` for the peer we want to -contact, and we need a way to discover their network address. Peer routing is -the process of discovering peer addresses by leveraging the knowledge of other -peers. - -In a peer routing system, a peer can either give us the address we need if they -have it, or else send our inquiry to another peer who's more likely to have the -answer. As we contact more and more peers, we not only increase our chances of -finding the peer we're looking for, we build a more complete view of the network -in our own routing tables, which enables us to answer routing queries from others. - -The current stable implementation of peer routing in libp2p uses a -[distributed hash table][definition_dht] to iteratively route requests closer -to the desired `PeerId` using the [Kademlia][wiki_kademlia] routing algorithm. - -### Content discovery - -In some systems, we care less about who we're speaking with than what they can offer us. -For example, we may want some specific piece of data, but we don't care who we get it from -since we can verify its integrity. - -libp2p provides a [content routing specification][spec_content_routing] for this -purpose, with the primary stable implementation using the same -[Kademlia][wiki_kademlia]-based DHT as used in peer routing. - -### Peer messaging - -Sending messages to other peers is at the heart of most peer-to-peer systems, -and pubsub (short for publish/subscribe) is an instrumental pattern for sending -a message to groups of interested receivers. - -libp2p defines a [pubsub specification][spec_pubsub] for sending messages to all -peers subscribed to a given "topic". The specification currently has two stable -implementations; `floodsub` uses a very simple but inefficient "network flooding" -strategy, and [gossipsub](https://github.com/libp2p/specs/tree/master/pubsub/gossipsub) -defines an extensible gossip protocol. There is also active development in progress on -[episub](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/episub.md), an -extended `gossipsub` that is optimized for single source multicast and scenarios with a -few fixed sources broadcasting to a large number of clients in a topic. - -Head over to [What is libp2p?](/introduction/what-is-libp2p/) for an introduction to -the basics of libp2p and an overview of the problems it addresses. - -## Related projects - -libp2p began as part of the [IPFS](https://ipfs.io) project and is still an -essential component of IPFS. As such, libp2p composes well with the abstractions -and tools provided by other projects in the IPFS "family". Check their sites for -specific information and references: - -- [IPFS](https://libp2p.io) is the InterPlanetary File System, which uses libp2p as - its networking layer. -- [Multiformats](https://multiformats.io) is a variety of *self-describing* data formats. -- [IPLD](https://ipld.io) is a set of tools for describing links between content-addressed - data, like IPFS files, Git commits, or Ethereum blocks. -- [The Permissive License Stack](https://protocol.ai/blog/announcing-the-permissive-license-stack) - is a licensing strategy for software development that embraces open-source values. - -[glossary]: {{< ref "concepts/miscellaneous/glossary.md" >}} -[definition_dht]: {{< ref "concepts/miscellaneous/glossary.md#dht" >}} -[definition_p2p]: {{< ref "concepts/miscellaneous/glossary.md#p2p" >}} -[definition_peer]: {{< ref "concepts/miscellaneous/glossary.md#peer" >}} -[definition_peerid]: {{< ref "concepts/miscellaneous/glossary.md#peerid" >}} -[definition_secio]: {{< ref "concepts/miscellaneous/glossary.md#secio" >}} -[definition_muiltiaddress]: {{< ref "concepts/miscellaneous/glossary.md#multiaddr" >}} -[definition_client_server]: {{< ref "concepts/miscellaneous/glossary.md#client-server" >}} - -[spec_content_routing]: https://github.com/libp2p/specs/blob/master/kad-dht/README.md -[spec_pubsub]: https://github.com/libp2p/specs/blob/master/pubsub/README.md -[built_with_libp2p]: https://discuss.libp2p.io/c/ecosystem-community -[help_improve_docs]: https://github.com/libp2p/docs/issues -[wiki_kademlia]: https://en.wikipedia.org/wiki/Kademlia diff --git a/content/introduction/network-basics/_index.md b/content/introduction/network-basics/_index.md deleted file mode 100644 index 6114378f..00000000 --- a/content/introduction/network-basics/_index.md +++ /dev/null @@ -1,22 +0,0 @@ ---- -title: "Network Basics" -weight: 1 -pre: ' 1.1. ' -chapter: true ---- - -### Chapter 1.1 - -# Network Basics - -Whether you’re just starting to dive into peer-to-peer concepts and -solutions, learning how to build peer-to-peer systems with libp2p, or -are looking for detailed reference information, this is the place to -start. - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} - From cefd58108de97e0e8629ce062535cf8751ce46b7 Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Fri, 14 Oct 2022 06:25:17 -0400 Subject: [PATCH 17/30] simplify current chapters and notices, re-add descriptions --- config.toml | 1 + content/concepts/_index.md | 8 +------- content/concepts/data-exchange/_index.md | 6 +----- content/concepts/data-exchange/overview.md | 4 +--- content/concepts/discovery/_index.md | 6 +----- content/concepts/discovery/overview.md | 4 +--- content/concepts/dos-mitigation/_index.md | 8 +------- content/concepts/introduction/_index.md | 8 +------- content/concepts/messaging/_index.md | 8 +------- content/concepts/miscellaneous/_index.md | 8 +------- content/concepts/multiplex/_index.md | 9 ++------- content/concepts/nat/_index.md | 8 +------- content/concepts/routing/_index.md | 6 +----- content/concepts/routing/content-routing/_index.md | 4 +--- content/concepts/routing/peer-routing/_index.md | 4 +--- content/concepts/secure-comm/_index.md | 6 +----- content/concepts/secure-comm/overview.md | 4 +--- content/concepts/security/_index.md | 8 +------- content/concepts/transports/_index.md | 8 +------- 19 files changed, 20 insertions(+), 98 deletions(-) diff --git a/config.toml b/config.toml index d6e6072b..477c5978 100644 --- a/config.toml +++ b/config.toml @@ -2,6 +2,7 @@ baseURL = "https://docs.libp2p.io" relativeURLs = true languageCode = "en-us" title = "libp2p Documentation" +summaryLength = 20 theme = ["hugo-theme-learn"] diff --git a/content/concepts/_index.md b/content/concepts/_index.md index f54cd4c3..5ae84a4f 100644 --- a/content/concepts/_index.md +++ b/content/concepts/_index.md @@ -12,10 +12,4 @@ chapter: true libp2p covers a lot of ground, and may involve unfamiliar terminology and concepts. This chapter goes over the foundational concepts involved in libp2p. -{{% children %}} - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} +{{% children description="true"%}} diff --git a/content/concepts/data-exchange/_index.md b/content/concepts/data-exchange/_index.md index df9ebfe9..d679c19e 100644 --- a/content/concepts/data-exchange/_index.md +++ b/content/concepts/data-exchange/_index.md @@ -9,10 +9,6 @@ chapter: true # Exchange Data -{{% children %}} - {{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. +Coming soon! {{% /notice %}} diff --git a/content/concepts/data-exchange/overview.md b/content/concepts/data-exchange/overview.md index 8fb54af7..8bfd4d81 100644 --- a/content/concepts/data-exchange/overview.md +++ b/content/concepts/data-exchange/overview.md @@ -10,7 +10,5 @@ chapter: true # Overview {{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. +Coming soon! {{% /notice %}} diff --git a/content/concepts/discovery/_index.md b/content/concepts/discovery/_index.md index 1efeba4c..f8449e27 100644 --- a/content/concepts/discovery/_index.md +++ b/content/concepts/discovery/_index.md @@ -9,10 +9,6 @@ chapter: true # Discover Peers -{{% children %}} - {{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. +Coming soon! {{% /notice %}} diff --git a/content/concepts/discovery/overview.md b/content/concepts/discovery/overview.md index 86c046ae..a9b1c722 100644 --- a/content/concepts/discovery/overview.md +++ b/content/concepts/discovery/overview.md @@ -10,7 +10,5 @@ chapter: true # Overview {{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. +Coming soon! {{% /notice %}} diff --git a/content/concepts/dos-mitigation/_index.md b/content/concepts/dos-mitigation/_index.md index b3a1fb5e..1e4dbabe 100644 --- a/content/concepts/dos-mitigation/_index.md +++ b/content/concepts/dos-mitigation/_index.md @@ -10,10 +10,4 @@ aliases: /reference/dos-mitigation/ # DOS Mitigation -{{% children %}} - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} +{{% children description="true"%}} diff --git a/content/concepts/introduction/_index.md b/content/concepts/introduction/_index.md index 2c53c1e5..dd353e89 100644 --- a/content/concepts/introduction/_index.md +++ b/content/concepts/introduction/_index.md @@ -9,10 +9,4 @@ chapter: true # Key Components of libp2p -{{% children %}} - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} +{{% children description="true"%}} diff --git a/content/concepts/messaging/_index.md b/content/concepts/messaging/_index.md index 962b45a3..ed08e46f 100644 --- a/content/concepts/messaging/_index.md +++ b/content/concepts/messaging/_index.md @@ -9,10 +9,4 @@ chapter: true # Message Peers -{{% children %}} - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} +{{% children description="true"%}} diff --git a/content/concepts/miscellaneous/_index.md b/content/concepts/miscellaneous/_index.md index 0e818d48..1eef79a7 100644 --- a/content/concepts/miscellaneous/_index.md +++ b/content/concepts/miscellaneous/_index.md @@ -9,10 +9,4 @@ chapter: true # Learn more -{{% children %}} - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} +{{% children description="true"%}} diff --git a/content/concepts/multiplex/_index.md b/content/concepts/multiplex/_index.md index 81489c26..eb6a66ed 100644 --- a/content/concepts/multiplex/_index.md +++ b/content/concepts/multiplex/_index.md @@ -3,16 +3,11 @@ title: "Stream Multiplexing" weight: 4 pre: ' 2.3. ' chapter: true +aliases: /concepts/stream-multiplexing/ --- ### Chapter 2.3 # Stream bytes -{{% children %}} - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} +{{% children description="true"%}} diff --git a/content/concepts/nat/_index.md b/content/concepts/nat/_index.md index 29ad09d5..52f84b4a 100644 --- a/content/concepts/nat/_index.md +++ b/content/concepts/nat/_index.md @@ -9,10 +9,4 @@ chapter: true # Traverse Bytes -{{% children %}} - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} +{{% children description="true"%}} diff --git a/content/concepts/routing/_index.md b/content/concepts/routing/_index.md index 9a9d77da..645a2cbb 100644 --- a/content/concepts/routing/_index.md +++ b/content/concepts/routing/_index.md @@ -9,10 +9,6 @@ chapter: true # Route with Peers -{{% children %}} - {{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. +Coming soon! {{% /notice %}} diff --git a/content/concepts/routing/content-routing/_index.md b/content/concepts/routing/content-routing/_index.md index bb002027..967f1fc1 100644 --- a/content/concepts/routing/content-routing/_index.md +++ b/content/concepts/routing/content-routing/_index.md @@ -10,7 +10,5 @@ chapter: true # Content Routing {{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. +Coming soon! {{% /notice %}} diff --git a/content/concepts/routing/peer-routing/_index.md b/content/concepts/routing/peer-routing/_index.md index 3e0cf136..b51d50b3 100644 --- a/content/concepts/routing/peer-routing/_index.md +++ b/content/concepts/routing/peer-routing/_index.md @@ -10,7 +10,5 @@ chapter: true # Peer Routing {{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. +Coming soon! {{% /notice %}} diff --git a/content/concepts/secure-comm/_index.md b/content/concepts/secure-comm/_index.md index a1af748e..6cb3ca00 100644 --- a/content/concepts/secure-comm/_index.md +++ b/content/concepts/secure-comm/_index.md @@ -9,10 +9,6 @@ chapter: true # Secure bytes -{{% children %}} - {{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. +Coming soon! {{% /notice %}} diff --git a/content/concepts/secure-comm/overview.md b/content/concepts/secure-comm/overview.md index 86a521ed..211e1c7d 100644 --- a/content/concepts/secure-comm/overview.md +++ b/content/concepts/secure-comm/overview.md @@ -10,7 +10,5 @@ chapter: true # Overview {{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. +Coming soon! {{% /notice %}} diff --git a/content/concepts/security/_index.md b/content/concepts/security/_index.md index 89be8250..ec729779 100644 --- a/content/concepts/security/_index.md +++ b/content/concepts/security/_index.md @@ -9,10 +9,4 @@ chapter: true # Secure Data -{{% children %}} - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} +{{% children description="true"%}} diff --git a/content/concepts/transports/_index.md b/content/concepts/transports/_index.md index 5df8f6a7..e9a31bc3 100644 --- a/content/concepts/transports/_index.md +++ b/content/concepts/transports/_index.md @@ -9,10 +9,4 @@ chapter: true # Exchange Bytes -{{% children %}} - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} +{{% children description="true"%}} From 7611874e133659dbd35b1d064c1bfba2d1ee04a0 Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Fri, 14 Oct 2022 06:43:15 -0400 Subject: [PATCH 18/30] define summary in frontmatter, rmv redundancy in ref docs and edits --- content/concepts/_index.md | 2 +- content/concepts/dos-mitigation/overview.md | 1 + content/concepts/introduction/addressing.md | 1 + content/concepts/introduction/peers.md | 1 + content/concepts/introduction/protocols.md | 2 +- .../concepts/messaging/publish-subscribe.md | 1 + content/concepts/miscellaneous/glossary.md | 1 + .../concepts/multiplex/stream-multiplexing.md | 2 ++ content/concepts/nat/circuit-relay.md | 1 + content/concepts/nat/nat.md | 1 + .../security/security-considerations.md | 1 + content/concepts/transports/overview.md | 2 ++ content/reference/monitoring.md | 19 ------------------- content/reference/specs.md | 7 +------ 14 files changed, 15 insertions(+), 27 deletions(-) delete mode 100644 content/reference/monitoring.md diff --git a/content/concepts/_index.md b/content/concepts/_index.md index 5ae84a4f..25e39a0b 100644 --- a/content/concepts/_index.md +++ b/content/concepts/_index.md @@ -12,4 +12,4 @@ chapter: true libp2p covers a lot of ground, and may involve unfamiliar terminology and concepts. This chapter goes over the foundational concepts involved in libp2p. -{{% children description="true"%}} +{{% children %}} diff --git a/content/concepts/dos-mitigation/overview.md b/content/concepts/dos-mitigation/overview.md index 90fba561..ffe5faca 100644 --- a/content/concepts/dos-mitigation/overview.md +++ b/content/concepts/dos-mitigation/overview.md @@ -3,6 +3,7 @@ title: "Overview" weight: 1 pre: ' 2.10.1 ' chapter: true +summary: DOS mitigation is an essential part of any peer-to-peer application. We need to design our protocols to be resilient to malicious peers. We need to monitor our application for signs of suspicious activity or an attack. And we need to be able to respond to an attack. --- ### Chapter 2.10.1 diff --git a/content/concepts/introduction/addressing.md b/content/concepts/introduction/addressing.md index b7e2692f..b830e29a 100644 --- a/content/concepts/introduction/addressing.md +++ b/content/concepts/introduction/addressing.md @@ -4,6 +4,7 @@ weight: 1 pre: ' 2.0.1. ' chapter: true aliases: /concepts/addressing/ +summary: Flexible networks need flexible addressing systems. Since libp2p is designed to work across a wide variety of networks, we need a way to work with a lot of different addressing schemes in a consistent way. --- ### Chapter 2.0.1 diff --git a/content/concepts/introduction/peers.md b/content/concepts/introduction/peers.md index aa7ab312..3900ec16 100644 --- a/content/concepts/introduction/peers.md +++ b/content/concepts/introduction/peers.md @@ -6,6 +6,7 @@ chapter: true aliases: - /concepts/peer-id/ - /concepts/peers/ +summary: Peers are what make up a libp2p network. They include unique identifiers and storage methods to facilitate peer-to-peer connections in libp2p. --- ### Chapter 2.0.3 diff --git a/content/concepts/introduction/protocols.md b/content/concepts/introduction/protocols.md index f76a64d0..b1c163b4 100644 --- a/content/concepts/introduction/protocols.md +++ b/content/concepts/introduction/protocols.md @@ -4,13 +4,13 @@ weight: 2 pre: ' 2.0.2. ' chapter: true aliases: /concepts/protocols/ +summary: There are protocols everywhere you look when you're writing network applications, and libp2p is especially thick with them. --- ### Chapter 2.0.2 # Protocols - There are protocols everywhere you look when you're writing network applications, and libp2p is especially thick with them. diff --git a/content/concepts/messaging/publish-subscribe.md b/content/concepts/messaging/publish-subscribe.md index a878fc28..496b34e8 100644 --- a/content/concepts/messaging/publish-subscribe.md +++ b/content/concepts/messaging/publish-subscribe.md @@ -3,6 +3,7 @@ title: Publish/Subscribe weight: 1 pre: ' 2.7.1 ' chapter: true +summary: Publish/Subscribe is a system where peers congregate around topics they are interested in. Peers interested in a topic are said to be subscribed to that topic. --- ### Chapter 2.7.1 diff --git a/content/concepts/miscellaneous/glossary.md b/content/concepts/miscellaneous/glossary.md index 95895131..75c8ace6 100644 --- a/content/concepts/miscellaneous/glossary.md +++ b/content/concepts/miscellaneous/glossary.md @@ -3,6 +3,7 @@ title: "Glossary" weight: 1 pre: ' 2.11.1 ' chapter: true +summary: A list of words, phrases, and abbreviations related to libp2p. --- ### Chapter 2.11.1 diff --git a/content/concepts/multiplex/stream-multiplexing.md b/content/concepts/multiplex/stream-multiplexing.md index 24005fe1..f35f9c74 100644 --- a/content/concepts/multiplex/stream-multiplexing.md +++ b/content/concepts/multiplex/stream-multiplexing.md @@ -3,6 +3,8 @@ title: "Overview" weight: 1 pre: ' 2.3.1. ' chapter: true +summary: Stream Multiplexing is a way of sending multiple streams of data over one communication link. It combines multiple signals into one unified signal so it can be transported 'over the wires', then it is demulitiplexed so it can be output and used by separate applications. + --- ### Chapter 2.3.1 diff --git a/content/concepts/nat/circuit-relay.md b/content/concepts/nat/circuit-relay.md index 00c3fc18..cb347a12 100644 --- a/content/concepts/nat/circuit-relay.md +++ b/content/concepts/nat/circuit-relay.md @@ -3,6 +3,7 @@ title: "Circuit Relay" weight: 2 pre: ' 2.4.2. ' chapter: true +summary: Circuit relay is a transport protocol that routes traffic between two peers over a third-party "relay" peer. In many cases, peers will be unable to traverse their NAT and/or firewall in a way that makes them publicly accessible. Or, they may not share common transport protocols that would allow them to communicate directly. --- ### Chapter 2.4.2 diff --git a/content/concepts/nat/nat.md b/content/concepts/nat/nat.md index 5decc48e..37fe51a4 100644 --- a/content/concepts/nat/nat.md +++ b/content/concepts/nat/nat.md @@ -3,6 +3,7 @@ title: "Overview" weight: 1 pre: ' 2.4.1. ' chapter: true +summary: The internet is composed of countless networks, bound together into shared address spaces by foundational transport protocols. As traffic moves between network boundaries, it's very common for a process called Network Address Translation to occur. Network Address Translation (NAT) maps an address from one address space to another. --- ### Chapter 2.4.1 diff --git a/content/concepts/security/security-considerations.md b/content/concepts/security/security-considerations.md index 0ecc428e..c5398f1c 100644 --- a/content/concepts/security/security-considerations.md +++ b/content/concepts/security/security-considerations.md @@ -3,6 +3,7 @@ title: "Overview" weight: 1 pre: ' 2.9.1. ' chapter: true +summary: libp2p makes it simple to establish encrypted, authenticated communication channels between two peers, but there are other important security issues to consider when building robust peer-to-peer systems. --- ### Chapter 2.9.1 diff --git a/content/concepts/transports/overview.md b/content/concepts/transports/overview.md index c81b1ba9..e7daa254 100644 --- a/content/concepts/transports/overview.md +++ b/content/concepts/transports/overview.md @@ -3,6 +3,7 @@ title: "Overview" weight: 1 pre: ' 2.1.1. ' chapter: true +summary: The foundational protocols that move bits around are called transports, and one of libp2p's core requirements is to be transport agnostic. This means that the decision of what transport protocol to use is up to the developer, and an application can support many different transports at the same time. --- ### Chapter 2.1.1 @@ -33,6 +34,7 @@ to use is up to the developer, and in fact one application can support many different transports at the same time. ## Listening and Dialing + Transports are defined in terms of two core operations, **listening** and **dialing**. diff --git a/content/reference/monitoring.md b/content/reference/monitoring.md deleted file mode 100644 index 5141fea8..00000000 --- a/content/reference/monitoring.md +++ /dev/null @@ -1,19 +0,0 @@ ---- -title: "Monitoring and Observability" -weight: 4 -pre: ' 4.3. ' -chapter: true ---- - -### Chapter 4.3 - -# Monitoring and Observability - -Reference the [Monitoring your application](../dos-mitigation/#monitoring-your-application) section in [DOS -Mitigation](../dos-mitigation/). - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} diff --git a/content/reference/specs.md b/content/reference/specs.md index b7ea1a1e..f2cec5bf 100644 --- a/content/reference/specs.md +++ b/content/reference/specs.md @@ -3,6 +3,7 @@ title: "Specifications" weight: 2 pre: ' 4.2. ' chapter: true +summary: Learn about the official libp2p specifications. --- ### Chapter 4.2 @@ -10,9 +11,3 @@ chapter: true # Specifications Check out the [specs repo](https://github.com/libp2p/specs) for details about the libp2p architecture. - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} From ebe71c95bcd7976094004b69919855ec021201fc Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Fri, 14 Oct 2022 07:02:11 -0400 Subject: [PATCH 19/30] incorporate PR feedback --- content/_index.md | 6 +++--- content/introduction/_index.md | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/content/_index.md b/content/_index.md index aa9471f3..ef8dbbf0 100644 --- a/content/_index.md +++ b/content/_index.md @@ -1,14 +1,14 @@ --- -title: libp2p book +title: libp2p Documentation Portal menuTitle: libp2p weight: 1 pre: "1. " chapter: false --- -# Welcome to the libp2p Book +# Welcome to the Documentation Portal -Welcome to the libp2p book! Whether you’re just learning how to build peer-to-peer systems with +Welcome to the libp2p Documentation Portal! Whether you’re just learning how to build peer-to-peer systems with libp2p, want to dive into peer-to-peer concepts and solutions, or are looking for detailed reference information, this is the place to start. diff --git a/content/introduction/_index.md b/content/introduction/_index.md index b5752804..5de8f639 100644 --- a/content/introduction/_index.md +++ b/content/introduction/_index.md @@ -1,6 +1,6 @@ --- title: "Get Started with libp2p" -menuTitle: "Introduction" +menuTitle: "Basics" weight: 1 pre: "1. " chapter: true From 6ded166adf7593e3d5f9c53ff8d17cf622301278 Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Fri, 14 Oct 2022 07:17:05 -0400 Subject: [PATCH 20/30] rmv reference section, chapterize get involved and apply further redirects --- content/concepts/_index.md | 15 +++++++++ content/concepts/miscellaneous/glossary.md | 1 + content/get-involved/_index.md | 6 ++-- content/get-involved/community.md | 7 +++- content/get-involved/contribute.md | 30 ++++++++++------- content/guides/_index.md | 2 ++ content/reference/_index.md | 20 ----------- content/reference/overview.md | 39 ---------------------- content/reference/specs.md | 13 -------- 9 files changed, 45 insertions(+), 88 deletions(-) delete mode 100644 content/reference/_index.md delete mode 100644 content/reference/overview.md delete mode 100644 content/reference/specs.md diff --git a/content/concepts/_index.md b/content/concepts/_index.md index 25e39a0b..24b81a4b 100644 --- a/content/concepts/_index.md +++ b/content/concepts/_index.md @@ -3,6 +3,7 @@ title: Core Components weight: 2 pre: "2. " chapter: true +aliases: /reference/specs/ --- ### Chapter 2 @@ -13,3 +14,17 @@ libp2p covers a lot of ground, and may involve unfamiliar terminology and concep This chapter goes over the foundational concepts involved in libp2p. {{% children %}} + +### Specifications & Planning + +While libp2p has several implementations, it is fundamentally a set of protocols for +peer identity, discover, routing, transport, and more. + +See the [specifications repository](https://github.com/libp2p/specs) for the offical libp2p specifications. + +### Implementations + +At the core of libp2p is a set of specifications, which together +form the definition for what libp2p is in the abstract and what makes a "correct" libp2p +implementation. Keep up with the latest implementations on the +[libp2p implementations page](https://libp2p.io/implementations/). diff --git a/content/concepts/miscellaneous/glossary.md b/content/concepts/miscellaneous/glossary.md index 75c8ace6..2b9aa25f 100644 --- a/content/concepts/miscellaneous/glossary.md +++ b/content/concepts/miscellaneous/glossary.md @@ -4,6 +4,7 @@ weight: 1 pre: ' 2.11.1 ' chapter: true summary: A list of words, phrases, and abbreviations related to libp2p. +aliases: /reference/glossary/ --- ### Chapter 2.11.1 diff --git a/content/get-involved/_index.md b/content/get-involved/_index.md index 9f02646e..797db15f 100644 --- a/content/get-involved/_index.md +++ b/content/get-involved/_index.md @@ -1,11 +1,11 @@ --- title: "Get Involved" -weight: 5 -pre: "5. " +weight: 4 +pre: "4. " chapter: true --- -### Chapter 5 +### Chapter 4 # Get Involved with libp2p diff --git a/content/get-involved/community.md b/content/get-involved/community.md index 61b53387..53d848ed 100644 --- a/content/get-involved/community.md +++ b/content/get-involved/community.md @@ -1,9 +1,14 @@ --- title: "Community" weight: 2 -pre: ' 5.2 ' +pre: ' 4.2 ' +chapter: true --- +### Chapter 4.2 + +# Community + We love talking about libp2p, and we'd be happy to have you in the mix. Visit our [discussion forums](https://discuss.libp2p.io) to ask questions about [using libp2p](https://discuss.libp2p.io/c/users), discuss exciting [research](https://discuss.libp2p.io/c/research), and [stay up to date with important announcments](https://discuss.libp2p.io/c/news). diff --git a/content/get-involved/contribute.md b/content/get-involved/contribute.md index 3df69cae..5cf7f8a0 100644 --- a/content/get-involved/contribute.md +++ b/content/get-involved/contribute.md @@ -1,26 +1,32 @@ --- title: "How to Contribute" weight: 1 -pre: ' 5.1 ' +pre: ' 4.1 ' +chapter: true --- +### Chapter 4.1 + +# How to Contribute + So you want to contribute to libp2p and the peer-to-peer ecosystem? Here is a quick listing of things we need help with and how you can get started. Even if what you want to do is not listed here, we probably accept contributions for it! If you're unsure, please open a issue. ## Areas of contribution -- [Areas of contribution](#areas-of-contribution) - - [Code](#code) - - [Documentation](#documentation) - - [Support](#support) - - [Testing](#testing) - - [Design](#design) - - [Issues / Triaging](#issues--triaging) - - [Community](#community) - - [Applications](#applications) - - [Protocol Design](#protocol-design) - - [Research](#research) +- [How to Contribute](#how-to-contribute) + - [Areas of contribution](#areas-of-contribution) + - [Code](#code) + - [Documentation](#documentation) + - [Support](#support) + - [Testing](#testing) + - [Design](#design) + - [Issues / Triaging](#issues--triaging) + - [Community](#community) + - [Applications](#applications) + - [Protocol Design](#protocol-design) + - [Research](#research) ### Code diff --git a/content/guides/_index.md b/content/guides/_index.md index 39b648b8..2e85a591 100644 --- a/content/guides/_index.md +++ b/content/guides/_index.md @@ -5,6 +5,8 @@ pre: "3. " chapter: true --- +### Chapter 3 + # Get Started with libp2p ## Examples using libp2p diff --git a/content/reference/_index.md b/content/reference/_index.md deleted file mode 100644 index 915f6642..00000000 --- a/content/reference/_index.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -title: "Reference" -weight: 4 -pre: "4. " -chapter: true ---- - -### Chapter 4 - -# Reference Documentation - -This chapter includes the reference documentation available for libp2p. - -{{% children %}} - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} diff --git a/content/reference/overview.md b/content/reference/overview.md deleted file mode 100644 index bd6f1a14..00000000 --- a/content/reference/overview.md +++ /dev/null @@ -1,39 +0,0 @@ ---- -title: "Overview" -weight: 1 -pre: ' 4.1. ' -chapter: true ---- - -### Chapter 4.1 - -# Overview - -This section contains in-depth reference material, including a [glossary](/reference/glossary/) with definitions of libp2p terminology. - -### Specifications & Planning - -While libp2p has several implementations, it is fundamentally a set of protocols for -peer identity, discover, routing, transport and more. - -See the [specifications section](/reference/specs/) for details. - -### Implementations - -At the core of libp2p is a set of [specifications](/reference/specs/), which together -form the definition for what libp2p is in the abstract and what makes a "correct" libp2p -implementation. Today, implementations of libp2p exist in several languages, with varying degrees -of completeness. The most complete implementations are in [Go](/reference/go/), -[JavaScript](/reference/js/), and [Rust](https://github.com/libp2p/rust-libp2p). - -In addition to the implementations above, the libp2p community is actively working on -implementations in [python](https://github.com/libp2p/py-libp2p) and [the JVM via Kotlin](https://github.com/web3j/libp2p). Please check the project pages for each implementation to see its status and current -state of completeness. - -Please help make this section more complete by [filing an issue](https://github.com/libp2p/docs/issues/new). - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} diff --git a/content/reference/specs.md b/content/reference/specs.md deleted file mode 100644 index f2cec5bf..00000000 --- a/content/reference/specs.md +++ /dev/null @@ -1,13 +0,0 @@ ---- -title: "Specifications" -weight: 2 -pre: ' 4.2. ' -chapter: true -summary: Learn about the official libp2p specifications. ---- - -### Chapter 4.2 - -# Specifications - -Check out the [specs repo](https://github.com/libp2p/specs) for details about the libp2p architecture. From e01a6459a6e2199e17356268408efd6e21c56431 Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Fri, 14 Oct 2022 07:29:19 -0400 Subject: [PATCH 21/30] edits --- config.toml | 2 +- content/concepts/introduction/_index.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/config.toml b/config.toml index 477c5978..c56802fb 100644 --- a/config.toml +++ b/config.toml @@ -19,7 +19,7 @@ url = "https://libp2p.io" weight = 1 [[menu.shortcuts]] -name = " libp2p Github" +name = " libp2p GitHub" url = "https://github.com/libp2p/libp2p" weight = 2 diff --git a/content/concepts/introduction/_index.md b/content/concepts/introduction/_index.md index dd353e89..a181cfd7 100644 --- a/content/concepts/introduction/_index.md +++ b/content/concepts/introduction/_index.md @@ -1,5 +1,5 @@ --- -title: "Introduction" +title: "Fundamentals" weight: 1 pre: ' 2.0. ' chapter: true From 212f8485b69d2d28f04ba3d37621fd47e14da94c Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Fri, 14 Oct 2022 14:05:28 -0400 Subject: [PATCH 22/30] completely simplify, rmv chapters, segment overviews, enhance presentation, re-add reference --- .DS_Store | Bin 0 -> 6148 bytes content/.DS_Store | Bin 0 -> 6148 bytes content/_index.md | 6 +- content/concepts/_index.md | 4 +- .../publish-subscribe/fanout_forget.png | Bin .../publish-subscribe/fanout_forget.svg | 0 .../fanout_grafting_preference.png | Bin .../fanout_grafting_preference.svg | 0 .../publish-subscribe/fanout_initial_pick.png | Bin .../publish-subscribe/fanout_initial_pick.svg | 0 .../publish-subscribe/fanout_message_send.png | Bin .../publish-subscribe/fanout_message_send.svg | 0 .../full_message_forward.png | Bin .../full_message_forward.svg | 0 .../full_message_network.png | Bin .../full_message_network.svg | 0 .../publish-subscribe/full_message_send.png | Bin .../publish-subscribe/full_message_send.svg | 0 .../publish-subscribe/gossip_deliver.png | Bin .../publish-subscribe/gossip_deliver.svg | 0 .../publish-subscribe/graft_prune.png | Bin .../publish-subscribe/graft_prune.svg | 0 .../publish-subscribe/maintain_graft.png | Bin .../publish-subscribe/maintain_graft.svg | 0 .../publish-subscribe/maintain_prune.png | Bin .../publish-subscribe/maintain_prune.svg | 0 .../message_delivered_to_all.png | Bin .../message_delivered_to_all.svg | 0 .../metadata_only_network.png | Bin .../metadata_only_network.svg | 0 .../network_packet_structure.png | Bin .../network_packet_structure.svg | 0 .../request_gossiped_message.png | Bin .../request_gossiped_message.svg | 0 .../publish-subscribe/state.png | Bin .../publish-subscribe/state.svg | 0 .../publish-subscribe/subscribe_graft.png | Bin .../publish-subscribe/subscribe_graft.svg | 0 .../publish-subscribe/subscribed_peers.png | Bin .../publish-subscribe/subscribed_peers.svg | 0 .../subscription_list_change.png | Bin .../subscription_list_change.svg | 0 .../subscription_list_first_connect.png | Bin .../subscription_list_first_connect.svg | 0 .../subscriptions_local_view.png | Bin .../subscriptions_local_view.svg | 0 .../publish-subscribe/types_of_peering.png | Bin .../publish-subscribe/types_of_peering.svg | 0 .../publish-subscribe/unsubscribe_prune.png | Bin .../publish-subscribe/unsubscribe_prune.svg | 0 content/concepts/data-exchange/_index.md | 14 - content/concepts/data-exchange/overview.md | 14 - content/concepts/discovery/_index.md | 14 - content/concepts/discovery/overview.md | 14 - content/concepts/dos-mitigation/_index.md | 13 - .../concepts/{miscellaneous => }/glossary.md | 11 +- content/concepts/introduction/_index.md | 4 +- content/concepts/introduction/addressing.md | 4 +- content/concepts/introduction/peers.md | 4 +- content/concepts/introduction/protocols.md | 4 +- content/concepts/messaging/_index.md | 12 - content/concepts/miscellaneous/_index.md | 12 - content/concepts/multiplex/_index.md | 26 +- content/concepts/multiplex/mplex.md | 13 + content/concepts/multiplex/quic.md | 13 + .../concepts/multiplex/stream-multiplexing.md | 77 ----- content/concepts/multiplex/switch.md | 15 + content/concepts/multiplex/yamux.md | 13 + content/concepts/nat/_index.md | 35 +- content/concepts/nat/autonat.md | 21 ++ content/concepts/nat/circuit-relay.md | 6 +- content/concepts/nat/hole-punching.md | 25 ++ content/concepts/nat/nat.md | 82 ----- .../{messaging => }/publish-subscribe.md | 11 +- content/concepts/routing/_index.md | 14 - .../routing/content-routing/_index.md | 14 - .../concepts/routing/peer-routing/_index.md | 14 - content/concepts/secure-comm/_index.md | 19 +- content/concepts/secure-comm/noise.md | 13 + content/concepts/secure-comm/tls.md | 13 + content/concepts/security/_index.md | 12 - .../security/security-considerations.md | 11 +- content/concepts/transports/_index.md | 27 +- .../concepts/transports/listen-and-dial.md | 98 ++++++ content/concepts/transports/overview.md | 315 ------------------ content/concepts/transports/quic.md | 113 +++++++ content/concepts/transports/webtransport.md | 91 +++++ content/get-involved/_index.md | 6 +- content/get-involved/community.md | 4 +- content/get-involved/contribute.md | 4 +- content/guides/_index.md | 4 +- content/introduction/_index.md | 25 +- content/reference/_index.md | 18 + .../dos-mitigation.md} | 13 +- content/reference/specs.md | 12 + static/css/theme.css | 2 +- 96 files changed, 593 insertions(+), 716 deletions(-) create mode 100644 .DS_Store create mode 100644 content/.DS_Store rename content/concepts/{messaging => assets}/publish-subscribe/fanout_forget.png (100%) rename content/concepts/{messaging => assets}/publish-subscribe/fanout_forget.svg (100%) rename content/concepts/{messaging => assets}/publish-subscribe/fanout_grafting_preference.png (100%) rename content/concepts/{messaging => assets}/publish-subscribe/fanout_grafting_preference.svg (100%) rename content/concepts/{messaging => assets}/publish-subscribe/fanout_initial_pick.png (100%) rename content/concepts/{messaging => assets}/publish-subscribe/fanout_initial_pick.svg (100%) rename content/concepts/{messaging => assets}/publish-subscribe/fanout_message_send.png (100%) rename content/concepts/{messaging => assets}/publish-subscribe/fanout_message_send.svg (100%) rename content/concepts/{messaging => assets}/publish-subscribe/full_message_forward.png (100%) rename content/concepts/{messaging => assets}/publish-subscribe/full_message_forward.svg (100%) rename content/concepts/{messaging => assets}/publish-subscribe/full_message_network.png (100%) rename content/concepts/{messaging => assets}/publish-subscribe/full_message_network.svg (100%) rename content/concepts/{messaging => assets}/publish-subscribe/full_message_send.png (100%) rename content/concepts/{messaging => assets}/publish-subscribe/full_message_send.svg (100%) rename content/concepts/{messaging => assets}/publish-subscribe/gossip_deliver.png (100%) rename content/concepts/{messaging => assets}/publish-subscribe/gossip_deliver.svg (100%) rename content/concepts/{messaging => assets}/publish-subscribe/graft_prune.png (100%) rename content/concepts/{messaging => assets}/publish-subscribe/graft_prune.svg (100%) rename content/concepts/{messaging => assets}/publish-subscribe/maintain_graft.png (100%) rename content/concepts/{messaging => assets}/publish-subscribe/maintain_graft.svg (100%) rename content/concepts/{messaging => assets}/publish-subscribe/maintain_prune.png (100%) rename content/concepts/{messaging => assets}/publish-subscribe/maintain_prune.svg (100%) rename content/concepts/{messaging => assets}/publish-subscribe/message_delivered_to_all.png (100%) rename content/concepts/{messaging => assets}/publish-subscribe/message_delivered_to_all.svg (100%) rename content/concepts/{messaging => assets}/publish-subscribe/metadata_only_network.png (100%) rename content/concepts/{messaging => assets}/publish-subscribe/metadata_only_network.svg (100%) rename content/concepts/{messaging => assets}/publish-subscribe/network_packet_structure.png (100%) rename content/concepts/{messaging => assets}/publish-subscribe/network_packet_structure.svg (100%) rename content/concepts/{messaging => assets}/publish-subscribe/request_gossiped_message.png (100%) rename content/concepts/{messaging => assets}/publish-subscribe/request_gossiped_message.svg (100%) rename content/concepts/{messaging => assets}/publish-subscribe/state.png (100%) rename content/concepts/{messaging => assets}/publish-subscribe/state.svg (100%) rename content/concepts/{messaging => assets}/publish-subscribe/subscribe_graft.png (100%) rename content/concepts/{messaging => assets}/publish-subscribe/subscribe_graft.svg (100%) rename content/concepts/{messaging => assets}/publish-subscribe/subscribed_peers.png (100%) rename content/concepts/{messaging => assets}/publish-subscribe/subscribed_peers.svg (100%) rename content/concepts/{messaging => assets}/publish-subscribe/subscription_list_change.png (100%) rename content/concepts/{messaging => assets}/publish-subscribe/subscription_list_change.svg (100%) rename content/concepts/{messaging => assets}/publish-subscribe/subscription_list_first_connect.png (100%) rename content/concepts/{messaging => assets}/publish-subscribe/subscription_list_first_connect.svg (100%) rename content/concepts/{messaging => assets}/publish-subscribe/subscriptions_local_view.png (100%) rename content/concepts/{messaging => assets}/publish-subscribe/subscriptions_local_view.svg (100%) rename content/concepts/{messaging => assets}/publish-subscribe/types_of_peering.png (100%) rename content/concepts/{messaging => assets}/publish-subscribe/types_of_peering.svg (100%) rename content/concepts/{messaging => assets}/publish-subscribe/unsubscribe_prune.png (100%) rename content/concepts/{messaging => assets}/publish-subscribe/unsubscribe_prune.svg (100%) delete mode 100644 content/concepts/data-exchange/_index.md delete mode 100644 content/concepts/data-exchange/overview.md delete mode 100644 content/concepts/discovery/_index.md delete mode 100644 content/concepts/discovery/overview.md delete mode 100644 content/concepts/dos-mitigation/_index.md rename content/concepts/{miscellaneous => }/glossary.md (99%) delete mode 100644 content/concepts/messaging/_index.md delete mode 100644 content/concepts/miscellaneous/_index.md create mode 100644 content/concepts/multiplex/mplex.md create mode 100644 content/concepts/multiplex/quic.md delete mode 100644 content/concepts/multiplex/stream-multiplexing.md create mode 100644 content/concepts/multiplex/switch.md create mode 100644 content/concepts/multiplex/yamux.md create mode 100644 content/concepts/nat/autonat.md create mode 100644 content/concepts/nat/hole-punching.md delete mode 100644 content/concepts/nat/nat.md rename content/concepts/{messaging => }/publish-subscribe.md (99%) delete mode 100644 content/concepts/routing/_index.md delete mode 100644 content/concepts/routing/content-routing/_index.md delete mode 100644 content/concepts/routing/peer-routing/_index.md create mode 100644 content/concepts/secure-comm/noise.md create mode 100644 content/concepts/secure-comm/tls.md delete mode 100644 content/concepts/security/_index.md create mode 100644 content/concepts/transports/listen-and-dial.md delete mode 100644 content/concepts/transports/overview.md create mode 100644 content/concepts/transports/quic.md create mode 100644 content/concepts/transports/webtransport.md create mode 100644 content/reference/_index.md rename content/{concepts/dos-mitigation/overview.md => reference/dos-mitigation.md} (97%) create mode 100644 content/reference/specs.md diff --git a/.DS_Store b/.DS_Store new file mode 100644 index 0000000000000000000000000000000000000000..77fc116c8688e264cd0241fc54e0c2b277deb329 GIT binary patch literal 6148 zcmeHKF>V4u473xZAkk1#?iccd6(TR-15tnu<%%TzRe4vQmKobc&_M?U(OB}%uGh1x zo#K2pGhcu0pUswLHir}Ko5R$&PoLOBMH~pn83%i}&;7^l_%=+k&j-kTvYicN{^bux zZpoy86p#W^Knh5KD=Sb1cD}gsxjIe?NP+uTfZvA(C-%ZIF+Lp_q6GlX5DvpUdI?}- z0N4x1L_}boRA5rQS`1G*;;r&};h30o^J+f4ZuaU>JZ{JNEz-?-s>Lo_KV_ZRqsRTRE}A3!8fMI0i~U&n7Vegq}Dq@_S($)4L;dsjEb z`3%5TU)y_N31FZ*;@iXAeBXU$SH(D5obiHpd}6=dZ+5fn^Muz9*kHsP<5B(dIPQ2T zZty|+mF^fxe*9tk{HbE4fE17dQa}nwflCT_d!-GxiHcG{3P^!p1^oNa=#IT`N{mkj zLyQ2#1?ez8$1Fi?P7r(Hlt_kVSxU@Os}aMpoP4XiUN|LYIV^7GId!wugko_!`4;K0 zo~S4Vq`;{H*STDJ|9_xgnEy|Uw2}f+;9n_Vi}mAr#aF7{I(s?qwT=EnKNxc(ogrE= jF1. " -chapter: false +pre: " " +chapter: true --- # Welcome to the Documentation Portal @@ -12,4 +12,4 @@ Welcome to the libp2p Documentation Portal! Whether you’re just learning how t libp2p, want to dive into peer-to-peer concepts and solutions, or are looking for detailed reference information, this is the place to start. -{{% children depth="8" showhidden="true" %}} +{{% children description="true" %}} diff --git a/content/concepts/_index.md b/content/concepts/_index.md index 24b81a4b..2595fb11 100644 --- a/content/concepts/_index.md +++ b/content/concepts/_index.md @@ -1,13 +1,11 @@ --- title: Core Components weight: 2 -pre: "2. " +pre: " " chapter: true aliases: /reference/specs/ --- -### Chapter 2 - # Explore libp2p libp2p covers a lot of ground, and may involve unfamiliar terminology and concepts. diff --git a/content/concepts/messaging/publish-subscribe/fanout_forget.png b/content/concepts/assets/publish-subscribe/fanout_forget.png similarity index 100% rename from content/concepts/messaging/publish-subscribe/fanout_forget.png rename to content/concepts/assets/publish-subscribe/fanout_forget.png diff --git a/content/concepts/messaging/publish-subscribe/fanout_forget.svg b/content/concepts/assets/publish-subscribe/fanout_forget.svg similarity index 100% rename from content/concepts/messaging/publish-subscribe/fanout_forget.svg rename to content/concepts/assets/publish-subscribe/fanout_forget.svg diff --git a/content/concepts/messaging/publish-subscribe/fanout_grafting_preference.png b/content/concepts/assets/publish-subscribe/fanout_grafting_preference.png similarity index 100% rename from content/concepts/messaging/publish-subscribe/fanout_grafting_preference.png rename to content/concepts/assets/publish-subscribe/fanout_grafting_preference.png diff --git a/content/concepts/messaging/publish-subscribe/fanout_grafting_preference.svg b/content/concepts/assets/publish-subscribe/fanout_grafting_preference.svg similarity index 100% rename from content/concepts/messaging/publish-subscribe/fanout_grafting_preference.svg rename to content/concepts/assets/publish-subscribe/fanout_grafting_preference.svg diff --git a/content/concepts/messaging/publish-subscribe/fanout_initial_pick.png b/content/concepts/assets/publish-subscribe/fanout_initial_pick.png similarity index 100% rename from content/concepts/messaging/publish-subscribe/fanout_initial_pick.png rename to content/concepts/assets/publish-subscribe/fanout_initial_pick.png diff --git a/content/concepts/messaging/publish-subscribe/fanout_initial_pick.svg b/content/concepts/assets/publish-subscribe/fanout_initial_pick.svg similarity index 100% rename from content/concepts/messaging/publish-subscribe/fanout_initial_pick.svg rename to content/concepts/assets/publish-subscribe/fanout_initial_pick.svg diff --git a/content/concepts/messaging/publish-subscribe/fanout_message_send.png b/content/concepts/assets/publish-subscribe/fanout_message_send.png similarity index 100% rename from content/concepts/messaging/publish-subscribe/fanout_message_send.png rename to content/concepts/assets/publish-subscribe/fanout_message_send.png diff --git a/content/concepts/messaging/publish-subscribe/fanout_message_send.svg b/content/concepts/assets/publish-subscribe/fanout_message_send.svg similarity index 100% rename from content/concepts/messaging/publish-subscribe/fanout_message_send.svg rename to content/concepts/assets/publish-subscribe/fanout_message_send.svg diff --git a/content/concepts/messaging/publish-subscribe/full_message_forward.png b/content/concepts/assets/publish-subscribe/full_message_forward.png similarity index 100% rename from content/concepts/messaging/publish-subscribe/full_message_forward.png rename to content/concepts/assets/publish-subscribe/full_message_forward.png diff --git a/content/concepts/messaging/publish-subscribe/full_message_forward.svg b/content/concepts/assets/publish-subscribe/full_message_forward.svg similarity index 100% rename from content/concepts/messaging/publish-subscribe/full_message_forward.svg rename to content/concepts/assets/publish-subscribe/full_message_forward.svg diff --git a/content/concepts/messaging/publish-subscribe/full_message_network.png b/content/concepts/assets/publish-subscribe/full_message_network.png similarity index 100% rename from content/concepts/messaging/publish-subscribe/full_message_network.png rename to content/concepts/assets/publish-subscribe/full_message_network.png diff --git a/content/concepts/messaging/publish-subscribe/full_message_network.svg b/content/concepts/assets/publish-subscribe/full_message_network.svg similarity index 100% rename from content/concepts/messaging/publish-subscribe/full_message_network.svg rename to content/concepts/assets/publish-subscribe/full_message_network.svg diff --git a/content/concepts/messaging/publish-subscribe/full_message_send.png b/content/concepts/assets/publish-subscribe/full_message_send.png similarity index 100% rename from content/concepts/messaging/publish-subscribe/full_message_send.png rename to content/concepts/assets/publish-subscribe/full_message_send.png diff --git a/content/concepts/messaging/publish-subscribe/full_message_send.svg b/content/concepts/assets/publish-subscribe/full_message_send.svg similarity index 100% rename from content/concepts/messaging/publish-subscribe/full_message_send.svg rename to content/concepts/assets/publish-subscribe/full_message_send.svg diff --git a/content/concepts/messaging/publish-subscribe/gossip_deliver.png b/content/concepts/assets/publish-subscribe/gossip_deliver.png similarity index 100% rename from content/concepts/messaging/publish-subscribe/gossip_deliver.png rename to content/concepts/assets/publish-subscribe/gossip_deliver.png diff --git a/content/concepts/messaging/publish-subscribe/gossip_deliver.svg b/content/concepts/assets/publish-subscribe/gossip_deliver.svg similarity index 100% rename from content/concepts/messaging/publish-subscribe/gossip_deliver.svg rename to content/concepts/assets/publish-subscribe/gossip_deliver.svg diff --git a/content/concepts/messaging/publish-subscribe/graft_prune.png b/content/concepts/assets/publish-subscribe/graft_prune.png similarity index 100% rename from content/concepts/messaging/publish-subscribe/graft_prune.png rename to content/concepts/assets/publish-subscribe/graft_prune.png diff --git a/content/concepts/messaging/publish-subscribe/graft_prune.svg b/content/concepts/assets/publish-subscribe/graft_prune.svg similarity index 100% rename from content/concepts/messaging/publish-subscribe/graft_prune.svg rename to content/concepts/assets/publish-subscribe/graft_prune.svg diff --git a/content/concepts/messaging/publish-subscribe/maintain_graft.png b/content/concepts/assets/publish-subscribe/maintain_graft.png similarity index 100% rename from content/concepts/messaging/publish-subscribe/maintain_graft.png rename to content/concepts/assets/publish-subscribe/maintain_graft.png diff --git a/content/concepts/messaging/publish-subscribe/maintain_graft.svg b/content/concepts/assets/publish-subscribe/maintain_graft.svg similarity index 100% rename from content/concepts/messaging/publish-subscribe/maintain_graft.svg rename to content/concepts/assets/publish-subscribe/maintain_graft.svg diff --git a/content/concepts/messaging/publish-subscribe/maintain_prune.png b/content/concepts/assets/publish-subscribe/maintain_prune.png similarity index 100% rename from content/concepts/messaging/publish-subscribe/maintain_prune.png rename to content/concepts/assets/publish-subscribe/maintain_prune.png diff --git a/content/concepts/messaging/publish-subscribe/maintain_prune.svg b/content/concepts/assets/publish-subscribe/maintain_prune.svg similarity index 100% rename from content/concepts/messaging/publish-subscribe/maintain_prune.svg rename to content/concepts/assets/publish-subscribe/maintain_prune.svg diff --git a/content/concepts/messaging/publish-subscribe/message_delivered_to_all.png b/content/concepts/assets/publish-subscribe/message_delivered_to_all.png similarity index 100% rename from content/concepts/messaging/publish-subscribe/message_delivered_to_all.png rename to content/concepts/assets/publish-subscribe/message_delivered_to_all.png diff --git a/content/concepts/messaging/publish-subscribe/message_delivered_to_all.svg b/content/concepts/assets/publish-subscribe/message_delivered_to_all.svg similarity index 100% rename from content/concepts/messaging/publish-subscribe/message_delivered_to_all.svg rename to content/concepts/assets/publish-subscribe/message_delivered_to_all.svg diff --git a/content/concepts/messaging/publish-subscribe/metadata_only_network.png b/content/concepts/assets/publish-subscribe/metadata_only_network.png similarity index 100% rename from content/concepts/messaging/publish-subscribe/metadata_only_network.png rename to content/concepts/assets/publish-subscribe/metadata_only_network.png diff --git a/content/concepts/messaging/publish-subscribe/metadata_only_network.svg b/content/concepts/assets/publish-subscribe/metadata_only_network.svg similarity index 100% rename from content/concepts/messaging/publish-subscribe/metadata_only_network.svg rename to content/concepts/assets/publish-subscribe/metadata_only_network.svg diff --git a/content/concepts/messaging/publish-subscribe/network_packet_structure.png b/content/concepts/assets/publish-subscribe/network_packet_structure.png similarity index 100% rename from content/concepts/messaging/publish-subscribe/network_packet_structure.png rename to content/concepts/assets/publish-subscribe/network_packet_structure.png diff --git a/content/concepts/messaging/publish-subscribe/network_packet_structure.svg b/content/concepts/assets/publish-subscribe/network_packet_structure.svg similarity index 100% rename from content/concepts/messaging/publish-subscribe/network_packet_structure.svg rename to content/concepts/assets/publish-subscribe/network_packet_structure.svg diff --git a/content/concepts/messaging/publish-subscribe/request_gossiped_message.png b/content/concepts/assets/publish-subscribe/request_gossiped_message.png similarity index 100% rename from content/concepts/messaging/publish-subscribe/request_gossiped_message.png rename to content/concepts/assets/publish-subscribe/request_gossiped_message.png diff --git a/content/concepts/messaging/publish-subscribe/request_gossiped_message.svg b/content/concepts/assets/publish-subscribe/request_gossiped_message.svg similarity index 100% rename from content/concepts/messaging/publish-subscribe/request_gossiped_message.svg rename to content/concepts/assets/publish-subscribe/request_gossiped_message.svg diff --git a/content/concepts/messaging/publish-subscribe/state.png b/content/concepts/assets/publish-subscribe/state.png similarity index 100% rename from content/concepts/messaging/publish-subscribe/state.png rename to content/concepts/assets/publish-subscribe/state.png diff --git a/content/concepts/messaging/publish-subscribe/state.svg b/content/concepts/assets/publish-subscribe/state.svg similarity index 100% rename from content/concepts/messaging/publish-subscribe/state.svg rename to content/concepts/assets/publish-subscribe/state.svg diff --git a/content/concepts/messaging/publish-subscribe/subscribe_graft.png b/content/concepts/assets/publish-subscribe/subscribe_graft.png similarity index 100% rename from content/concepts/messaging/publish-subscribe/subscribe_graft.png rename to content/concepts/assets/publish-subscribe/subscribe_graft.png diff --git a/content/concepts/messaging/publish-subscribe/subscribe_graft.svg b/content/concepts/assets/publish-subscribe/subscribe_graft.svg similarity index 100% rename from content/concepts/messaging/publish-subscribe/subscribe_graft.svg rename to content/concepts/assets/publish-subscribe/subscribe_graft.svg diff --git a/content/concepts/messaging/publish-subscribe/subscribed_peers.png b/content/concepts/assets/publish-subscribe/subscribed_peers.png similarity index 100% rename from content/concepts/messaging/publish-subscribe/subscribed_peers.png rename to content/concepts/assets/publish-subscribe/subscribed_peers.png diff --git a/content/concepts/messaging/publish-subscribe/subscribed_peers.svg b/content/concepts/assets/publish-subscribe/subscribed_peers.svg similarity index 100% rename from content/concepts/messaging/publish-subscribe/subscribed_peers.svg rename to content/concepts/assets/publish-subscribe/subscribed_peers.svg diff --git a/content/concepts/messaging/publish-subscribe/subscription_list_change.png b/content/concepts/assets/publish-subscribe/subscription_list_change.png similarity index 100% rename from content/concepts/messaging/publish-subscribe/subscription_list_change.png rename to content/concepts/assets/publish-subscribe/subscription_list_change.png diff --git a/content/concepts/messaging/publish-subscribe/subscription_list_change.svg b/content/concepts/assets/publish-subscribe/subscription_list_change.svg similarity index 100% rename from content/concepts/messaging/publish-subscribe/subscription_list_change.svg rename to content/concepts/assets/publish-subscribe/subscription_list_change.svg diff --git a/content/concepts/messaging/publish-subscribe/subscription_list_first_connect.png b/content/concepts/assets/publish-subscribe/subscription_list_first_connect.png similarity index 100% rename from content/concepts/messaging/publish-subscribe/subscription_list_first_connect.png rename to content/concepts/assets/publish-subscribe/subscription_list_first_connect.png diff --git a/content/concepts/messaging/publish-subscribe/subscription_list_first_connect.svg b/content/concepts/assets/publish-subscribe/subscription_list_first_connect.svg similarity index 100% rename from content/concepts/messaging/publish-subscribe/subscription_list_first_connect.svg rename to content/concepts/assets/publish-subscribe/subscription_list_first_connect.svg diff --git a/content/concepts/messaging/publish-subscribe/subscriptions_local_view.png b/content/concepts/assets/publish-subscribe/subscriptions_local_view.png similarity index 100% rename from content/concepts/messaging/publish-subscribe/subscriptions_local_view.png rename to content/concepts/assets/publish-subscribe/subscriptions_local_view.png diff --git a/content/concepts/messaging/publish-subscribe/subscriptions_local_view.svg b/content/concepts/assets/publish-subscribe/subscriptions_local_view.svg similarity index 100% rename from content/concepts/messaging/publish-subscribe/subscriptions_local_view.svg rename to content/concepts/assets/publish-subscribe/subscriptions_local_view.svg diff --git a/content/concepts/messaging/publish-subscribe/types_of_peering.png b/content/concepts/assets/publish-subscribe/types_of_peering.png similarity index 100% rename from content/concepts/messaging/publish-subscribe/types_of_peering.png rename to content/concepts/assets/publish-subscribe/types_of_peering.png diff --git a/content/concepts/messaging/publish-subscribe/types_of_peering.svg b/content/concepts/assets/publish-subscribe/types_of_peering.svg similarity index 100% rename from content/concepts/messaging/publish-subscribe/types_of_peering.svg rename to content/concepts/assets/publish-subscribe/types_of_peering.svg diff --git a/content/concepts/messaging/publish-subscribe/unsubscribe_prune.png b/content/concepts/assets/publish-subscribe/unsubscribe_prune.png similarity index 100% rename from content/concepts/messaging/publish-subscribe/unsubscribe_prune.png rename to content/concepts/assets/publish-subscribe/unsubscribe_prune.png diff --git a/content/concepts/messaging/publish-subscribe/unsubscribe_prune.svg b/content/concepts/assets/publish-subscribe/unsubscribe_prune.svg similarity index 100% rename from content/concepts/messaging/publish-subscribe/unsubscribe_prune.svg rename to content/concepts/assets/publish-subscribe/unsubscribe_prune.svg diff --git a/content/concepts/data-exchange/_index.md b/content/concepts/data-exchange/_index.md deleted file mode 100644 index d679c19e..00000000 --- a/content/concepts/data-exchange/_index.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -title: "Data Exchange" -weight: 9 -pre: ' 2.8. ' -chapter: true ---- - -### Chapter 2.8 - -# Exchange Data - -{{% notice "note" %}} -Coming soon! -{{% /notice %}} diff --git a/content/concepts/data-exchange/overview.md b/content/concepts/data-exchange/overview.md deleted file mode 100644 index 8bfd4d81..00000000 --- a/content/concepts/data-exchange/overview.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -title: 'Overview' -weight: 1 -pre: ' 2.8.1. ' -chapter: true ---- - -### Chapter 2.8.1 - -# Overview - -{{% notice "note" %}} -Coming soon! -{{% /notice %}} diff --git a/content/concepts/discovery/_index.md b/content/concepts/discovery/_index.md deleted file mode 100644 index f8449e27..00000000 --- a/content/concepts/discovery/_index.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -title: "Discovery" -weight: 6 -pre: ' 2.5. ' -chapter: true ---- - -### Chapter 2.5 - -# Discover Peers - -{{% notice "note" %}} -Coming soon! -{{% /notice %}} diff --git a/content/concepts/discovery/overview.md b/content/concepts/discovery/overview.md deleted file mode 100644 index a9b1c722..00000000 --- a/content/concepts/discovery/overview.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -title: "Overview" -weight: 1 -pre: ' 2.5.1. ' -chapter: true ---- - -### Chapter 2.5.1 - -# Overview - -{{% notice "note" %}} -Coming soon! -{{% /notice %}} diff --git a/content/concepts/dos-mitigation/_index.md b/content/concepts/dos-mitigation/_index.md deleted file mode 100644 index 1e4dbabe..00000000 --- a/content/concepts/dos-mitigation/_index.md +++ /dev/null @@ -1,13 +0,0 @@ ---- -title: "DOS Mitigation" -weight: 11 -pre: ' 2.10. ' -chapter: true -aliases: /reference/dos-mitigation/ ---- - -### Chapter 2.10 - -# DOS Mitigation - -{{% children description="true"%}} diff --git a/content/concepts/miscellaneous/glossary.md b/content/concepts/glossary.md similarity index 99% rename from content/concepts/miscellaneous/glossary.md rename to content/concepts/glossary.md index 2b9aa25f..f2d062f3 100644 --- a/content/concepts/miscellaneous/glossary.md +++ b/content/concepts/glossary.md @@ -1,14 +1,10 @@ --- title: "Glossary" -weight: 1 -pre: ' 2.11.1 ' +weight: 12 +pre: ' ' chapter: true -summary: A list of words, phrases, and abbreviations related to libp2p. -aliases: /reference/glossary/ --- -### Chapter 2.11.1 - # Glossary ### Boot Node @@ -116,7 +112,6 @@ In the peer-to-peer model, accepting connections from other peers is often just ### NAT Traversal - NAT traversal refers to the process of establishing connections with other machines across a [NAT](#nat) boundary. When crossing the boundary between IP networks (e.g. from a local network to the global internet), a [Network Address Translation](#nat) process occurs which maps addresses from one space to another. For example, my home network has an internal range of IP addresses (10.0.1.x), which is part of a range of addresses that are reserved for private networks. If I start a program on my computer that listens for connections on its internal address, a user from the public internet has no way of reaching me, even if they know my public IP address. This is because I haven't made my router aware of my program yet. When a connection comes in from the internet to my public IP address, the router needs to figure out which internal IP to route the request to, and to which port. @@ -125,7 +120,6 @@ There are many ways to inform one's router about services you want to expose. Fo In some cases, automatic NAT traversal is impossible, often because multiple layers of NAT are involved. In such cases, we still want to be able to communicate, and we especially want to be reachable and allow other peers to [dial in](#dial) and use our services. This is the one of the motivations for [Circuit Relay](#circuit-relay), which is a protocol involving a "relay" peer that is publicly reachable and can route traffic on behalf of others. Once a relay circuit is established, a peer behind an especially intractable NAT can advertise the relay circuit's [multiaddr](#multiaddr), and the relay will accept incoming connections on our behalf and send us traffic via the relay. - ### Node The word "node" is quite overloaded in general programming contexts, and this is especially the case in peer-to-peer networking circles. @@ -259,5 +253,4 @@ In libp2p, `transport` refers to the technology that lets us move bits from one Note that in some environments such as javascript running in the browser, not all transports will be available. In such cases, it may be possible to establish a [Circuit Relay](#circuit-relay) with the help of a peer that can support many common transports. Such a relay can act as a "transport adapter" of sorts, allowing peers that can't communicate with each other directly to interact. For example, a peer in the browser that can only make websocket connections could relay through a peer able to make TCP connections, which would enable communication with a wider variety of peers. - [js-docs-home]: /reference/ diff --git a/content/concepts/introduction/_index.md b/content/concepts/introduction/_index.md index a181cfd7..1301ae90 100644 --- a/content/concepts/introduction/_index.md +++ b/content/concepts/introduction/_index.md @@ -1,12 +1,10 @@ --- title: "Fundamentals" weight: 1 -pre: ' 2.0. ' +pre: ' ' chapter: true --- -### Chapter 2.0 - # Key Components of libp2p {{% children description="true"%}} diff --git a/content/concepts/introduction/addressing.md b/content/concepts/introduction/addressing.md index b830e29a..aa7c258e 100644 --- a/content/concepts/introduction/addressing.md +++ b/content/concepts/introduction/addressing.md @@ -1,14 +1,12 @@ --- title: "Addressing" weight: 1 -pre: ' 2.0.1. ' +pre: ' ' chapter: true aliases: /concepts/addressing/ summary: Flexible networks need flexible addressing systems. Since libp2p is designed to work across a wide variety of networks, we need a way to work with a lot of different addressing schemes in a consistent way. --- -### Chapter 2.0.1 - # Addressing Flexible networks need flexible addressing systems. Since libp2p is designed to work across a wide variety of networks, we need a way to work with a lot of different addressing schemes in a consistent way. diff --git a/content/concepts/introduction/peers.md b/content/concepts/introduction/peers.md index 3900ec16..5be83c8d 100644 --- a/content/concepts/introduction/peers.md +++ b/content/concepts/introduction/peers.md @@ -1,7 +1,7 @@ --- title: "Peers" weight: 3 -pre: ' 2.0.3. ' +pre: ' ' chapter: true aliases: - /concepts/peer-id/ @@ -9,8 +9,6 @@ aliases: summary: Peers are what make up a libp2p network. They include unique identifiers and storage methods to facilitate peer-to-peer connections in libp2p. --- -### Chapter 2.0.3 - # Peers Peers are what make up a libp2p network. We'll go over their key diff --git a/content/concepts/introduction/protocols.md b/content/concepts/introduction/protocols.md index b1c163b4..4079d38e 100644 --- a/content/concepts/introduction/protocols.md +++ b/content/concepts/introduction/protocols.md @@ -1,14 +1,12 @@ --- title: "Protocols" weight: 2 -pre: ' 2.0.2. ' +pre: ' ' chapter: true aliases: /concepts/protocols/ summary: There are protocols everywhere you look when you're writing network applications, and libp2p is especially thick with them. --- -### Chapter 2.0.2 - # Protocols There are protocols everywhere you look when you're writing network applications, and libp2p is diff --git a/content/concepts/messaging/_index.md b/content/concepts/messaging/_index.md deleted file mode 100644 index ed08e46f..00000000 --- a/content/concepts/messaging/_index.md +++ /dev/null @@ -1,12 +0,0 @@ ---- -title: "Messaging" -weight: 8 -pre: ' 2.7. ' -chapter: true ---- - -### Chapter 2.7 - -# Message Peers - -{{% children description="true"%}} diff --git a/content/concepts/miscellaneous/_index.md b/content/concepts/miscellaneous/_index.md deleted file mode 100644 index 1eef79a7..00000000 --- a/content/concepts/miscellaneous/_index.md +++ /dev/null @@ -1,12 +0,0 @@ ---- -title: "Miscellaneous" -weight: 12 -pre: ' 2.11. ' -chapter: true ---- - -### Chapter 2.11 - -# Learn more - -{{% children description="true"%}} diff --git a/content/concepts/multiplex/_index.md b/content/concepts/multiplex/_index.md index eb6a66ed..8e49f1dd 100644 --- a/content/concepts/multiplex/_index.md +++ b/content/concepts/multiplex/_index.md @@ -1,13 +1,33 @@ --- title: "Stream Multiplexing" weight: 4 -pre: ' 2.3. ' +pre: ' ' chapter: true aliases: /concepts/stream-multiplexing/ --- -### Chapter 2.3 - # Stream bytes +Stream Multiplexing (_stream muxing_) is a way of sending multiple streams of data over one +communication link. It combines multiple signals into one unified signal so it can be transported +'over the wires', then it is demulitiplexed (_demuxed_) so it can be output and used by separate +applications. This is done to share a single TCP connection using unique port numbers to distinguish +streams, between the multiple proceeses (such as kademlia and gossipsub) used by applications (such as IPFS) +to make connection and transmission more efficient. + +With muxing, libp2p applications may have many separate streams of communication between peers, as well as +have multiple concurrent streams open at the same time with a peer. Stream multiplexing allows us to initialize +and use the same [transport](/concepts/transport/) connection across the lifetime of our interaction with a peer. +With muxing, we also only need to deal with [NAT traversal](nat) once to be able to open as many +streams as we need, since they will all share the same underlying transport connection. Applications can enable +support for multiple multiplexers, which will allow you to fall back to a widely-supported multiplexer if a preferred +choice is not supported by a remote peer. + +{{% notice "info" %}} +libp2p's multiplexing happens at the application layer, meaning it's not provided by the +operating system's network stack. However, developers writing libp2p applications rarely need to i +nteract with stream multiplexers directly, except during initial configuration to control which +modules are enabled. +{{% /notice %}} + {{% children description="true"%}} diff --git a/content/concepts/multiplex/mplex.md b/content/concepts/multiplex/mplex.md new file mode 100644 index 00000000..c4df8e94 --- /dev/null +++ b/content/concepts/multiplex/mplex.md @@ -0,0 +1,13 @@ +--- +title: "mplex" +weight: 2 +pre: ' ' +chapter: true +summary: Stream Multiplexing is a way of sending multiple streams of data over one communication link. It combines multiple signals into one unified signal so it can be transported 'over the wires', then it is demulitiplexed so it can be output and used by separate applications. +--- + +# mplex + +mplex is a protocol developed for libp2p. The [spec](https://github.com/libp2p/specs/tree/master/mplex) defines a simple protocol for multiplexing that is widely supported across libp2p language implementations: + + diff --git a/content/concepts/multiplex/quic.md b/content/concepts/multiplex/quic.md new file mode 100644 index 00000000..844c8b91 --- /dev/null +++ b/content/concepts/multiplex/quic.md @@ -0,0 +1,13 @@ +--- +title: "QUIC" +weight: 4 +pre: ' ' +chapter: true +summary: Stream Multiplexing is a way of sending multiple streams of data over one communication link. It combines multiple signals into one unified signal so it can be transported 'over the wires', then it is demulitiplexed so it can be output and used by separate applications. +--- + +# QUIC + +QUIC is a [transport](/concepts/transport/) protocol that contains a "native" stream multiplexer. +libp2p will automatically use the native multiplexer for streams using a QUIC transport. View the [QUIC +section](/concepts/transport/quic/) to learn about QUIC. diff --git a/content/concepts/multiplex/stream-multiplexing.md b/content/concepts/multiplex/stream-multiplexing.md deleted file mode 100644 index f35f9c74..00000000 --- a/content/concepts/multiplex/stream-multiplexing.md +++ /dev/null @@ -1,77 +0,0 @@ ---- -title: "Overview" -weight: 1 -pre: ' 2.3.1. ' -chapter: true -summary: Stream Multiplexing is a way of sending multiple streams of data over one communication link. It combines multiple signals into one unified signal so it can be transported 'over the wires', then it is demulitiplexed so it can be output and used by separate applications. - ---- - -### Chapter 2.3.1 - -# Overview - -Stream Multiplexing (_stream muxing_) is a way of sending multiple streams of data over one communication link. It combines multiple signals into one unified signal so it can be transported 'over the wires', then it is demulitiplexed (_demuxed_) so it can be output and used by separate applications. - -## Multiplexing - -Multiplexing is by no means unique to libp2p. Most communication networks involve some kind of multiplexing, as the transport medium is generally scarce and needs to be shared by many participants. - -This is done to share a single TCP connection using unique port numbers to distinguish streams, between the multiple proceeses (such as kademlia and gossipsub) used by applications (such as ipfs) to make connection and transmission more efficient. With muxing, libp2p applications may have many separate streams of communication between peers, as well as have multiple concurrent streams open at the same time with a peer. - -Stream multiplexing allows us to initialize and use the same [transport](/concepts/transport/) connection across the lifetime of our interaction with a peer. With muxing, we also only need to deal with [NAT traversal](nat) once to be able to open as many streams as we need, since they will all share the same underlying transport connection. - -libp2p provides a common [interface](#interface) for stream multiplexers with several [implementations](#implementations) available. Applications can enable support for multiple multiplexers, which will allow you to fall back to a widely-supported multiplexer if a preferred choice is not supported by a remote peer. - -## Where it fits in the libp2p stack - -libp2p's multiplexing happens at the "application layer", meaning it's not provided by the operating system's network stack. However, developers writing libp2p applications rarely need to interact with stream multiplexers directly, except during initial configuration to control which modules are enabled. - -### Switch / swarm - -libp2p maintains some state about known peers and existing connections in a component known as the switch (or "swarm", depending on the implementation). The switch provides a dialing and listening interface that abstracts the details of which stream multiplexer is used for a given connection. - -When configuring libp2p, applications enable stream muxing modules, which the switch will use when dialing peers and listening for connections. If the remote peers support any of the same stream muxing implementations, the switch will select and use it when establishing the connection. If you dial a peer that the switch already has an open connection to, the new stream will automatically be multiplexed over the existing connection. - -Reaching agreement on which stream multiplexer to use happens early in the connection establishment process. Peers use [protocol negotiation](/concepts/protocols/#protocol-negotiation/) to agree on a commonly supported multiplexer, which upgrades a "raw" transport connection into a muxed connection capable of opening new streams. - -## Interface and Implementations - -### Interface -The [stream multiplexing interface][interface-stream-muxing] defines how a stream muxing module can be applied to a connection and what operations are supported by a multiplexed connection. - -### Implementations - -There are several stream multiplexing modules available in libp2p. Please note that not all stream muxers are supported by every libp2p language implementation. - -#### mplex - -mplex is a protocol developed for libp2p. The [spec](https://github.com/libp2p/specs/tree/master/mplex) defines a simple protocol for multiplexing that is widely supported across libp2p language implementations: - -- Go: [go-mplex](https://github.com/libp2p/go-mplex) -- Javascript: [js-mplex](https://github.com/libp2p/js-libp2p-mplex) -- Rust: [rust-libp2p mplex module](https://github.com/libp2p/rust-libp2p/tree/master/muxers/mplex) - -#### yamux - -[yamux](https://github.com/hashicorp/yamux) is a multiplexing protocol designed by [Hashicorp](https://www.hashicorp.com/). - -yamux offers more sophisticated flow control than mplex, and can scale to thousands of multiplexed streams over a single connection. - -yamux is currently supported in go and rust: - -- Go: [go-smux-yamux](https://github.com/whyrusleeping/go-smux-yamux) -- Rust: [rust-libp2p yamux module](https://github.com/libp2p/rust-libp2p/tree/master/muxers/yamux). - -#### quic - -[QUIC][wiki-quic] is a [transport](/concepts/transport/) protocol that contains a "native" stream multiplexer. libp2p will automatically use the native multiplexer for streams using a quic transport. - -quic is currently supported in go via [go-libp2p-quic-transport](https://github.com/libp2p/go-libp2p/tree/master/p2p/transport/quic). - - -[interface-stream-muxing]: https://github.com/libp2p/js-libp2p-interfaces/tree/master/packages/interface-stream-muxer - -[repo-multistream-select]: https://github.com/multiformats/multistream-select - -[wiki-quic]: https://en.wikipedia.org/wiki/QUIC diff --git a/content/concepts/multiplex/switch.md b/content/concepts/multiplex/switch.md new file mode 100644 index 00000000..09457315 --- /dev/null +++ b/content/concepts/multiplex/switch.md @@ -0,0 +1,15 @@ +--- +title: "Switch" +weight: 1 +pre: ' ' +chapter: true +summary: Stream Multiplexing is a way of sending multiple streams of data over one communication link. It combines multiple signals into one unified signal so it can be transported 'over the wires', then it is demulitiplexed so it can be output and used by separate applications. +--- + +# Switch + +libp2p maintains some state about known peers and existing connections in a component known as the switch (or "swarm", depending on the implementation). The switch provides a dialing and listening interface that abstracts the details of which stream multiplexer is used for a given connection. + +When configuring libp2p, applications enable stream muxing modules, which the switch will use when dialing peers and listening for connections. If the remote peers support any of the same stream muxing implementations, the switch will select and use it when establishing the connection. If you dial a peer that the switch already has an open connection to, the new stream will automatically be multiplexed over the existing connection. + +Reaching agreement on which stream multiplexer to use happens early in the connection establishment process. Peers use [protocol negotiation](/concepts/fundamentals/protocols/#protocol-negotiation/) to agree on a commonly supported multiplexer, which upgrades a "raw" transport connection into a muxed connection capable of opening new streams. diff --git a/content/concepts/multiplex/yamux.md b/content/concepts/multiplex/yamux.md new file mode 100644 index 00000000..57c23c85 --- /dev/null +++ b/content/concepts/multiplex/yamux.md @@ -0,0 +1,13 @@ +--- +title: "yamux" +weight: 3 +pre: ' ' +chapter: true +summary: Stream Multiplexing is a way of sending multiple streams of data over one communication link. It combines multiple signals into one unified signal so it can be transported 'over the wires', then it is demulitiplexed so it can be output and used by separate applications. +--- + +# yamux + +[yamux](https://github.com/hashicorp/yamux) is a multiplexing protocol designed by [Hashicorp](https://www.hashicorp.com/). + +yamux offers more sophisticated flow control than mplex, and can scale to thousands of multiplexed streams over a single connection. diff --git a/content/concepts/nat/_index.md b/content/concepts/nat/_index.md index 52f84b4a..23b06317 100644 --- a/content/concepts/nat/_index.md +++ b/content/concepts/nat/_index.md @@ -1,12 +1,41 @@ --- title: "NAT Traversal" weight: 5 -pre: ' 2.4. ' +pre: ' ' chapter: true --- -### Chapter 2.4 - # Traverse Bytes +The internet is composed of countless networks, bound together into shared address spaces by foundational [transport protocols](/concepts/transport/). + +As traffic moves between network boundaries, it's very common for a process called Network Address Translation to occur. Network Address Translation (NAT) maps an address from one address space to another. + +NAT allows many machines to share a single public address, and it is essential for the continued functioning of the IPv4 protocol, which would otherwise be unable to serve the needs of the modern networked population with its 32-bit address space. + +For example, when I connect to my home wifi, my computer gets an IPv4 address of `10.0.1.15`. This is part of a range of IP addresses reserved for internal use by private networks. When I make an outgoing connection to a public IP address, the router replaces my internal IP with its own public IP address. When data comes back from the other side, the router will translate back to the internal address. + +While NAT is usually transparent for outgoing connections, listening for incoming connections requires some configuration. The router listens on a single public IP address, but any number of machines on the internal network could handle the request. To serve requests, your router must be configured to send certain traffic to a specific machine, usually by mapping one or more TCP or UDP ports from the public IP to an internal one. + +While it's usually possible to manually configure routers, not everyone that wants to run a peer-to-peer application or other network service will have the ability to do so. + +We want libp2p applications to run everywhere, not just in data centers or on machines with stable public IP addresses. To enable this, here are the main approaches to NAT traversal available in libp2p today. + +### Automatic router configuration + +Many routers support automatic configuration protocols for port forwarding, most commonly [UPnP][wiki_upnp] or [nat-pmp.][wiki_nat-pmp] + +If your router supports one of those protocols, libp2p will attempt to automatically configure a port mapping that will allow it to listen for incoming traffic. This is usually the simplest option if supported by the network and libp2p implementation. + +{{% notice "info" %}} +Support for automatic NAT configuration varies by libp2p implementation. +Check the [current implementation status](https://libp2p.io/implementations/#nat-traversal) for details. +{{% /notice %}} + {{% children description="true"%}} + +[wiki_upnp]: https://en.wikipedia.org/wiki/Universal_Plug_and_Play +[wiki_nat-pmp]: https://en.wikipedia.org/wiki/NAT_Port_Mapping_Protocol +[wiki_stun]: https://en.wikipedia.org/wiki/STUN +[rfc_stun]: https://tools.ietf.org/html/rfc3489 +[lwn_reuseport]: https://lwn.net/Articles/542629/ diff --git a/content/concepts/nat/autonat.md b/content/concepts/nat/autonat.md new file mode 100644 index 00000000..9c3ad175 --- /dev/null +++ b/content/concepts/nat/autonat.md @@ -0,0 +1,21 @@ +--- +title: "AutoNAT" +weight: 2 +pre: ' ' +chapter: true +summary: The internet is composed of countless networks, bound together into shared address spaces by foundational transport protocols. As traffic moves between network boundaries, it's very common for a process called Network Address Translation to occur. Network Address Translation (NAT) maps an address from one address space to another. +--- + +# AutoNAT + +While the [identify protocol][spec_identify] described above lets peers inform each other about their observed network addresses, not all networks will allow incoming connections on the same port used for dialing out. + +Once again, other peers can help us observe our situation, this time by attempting to dial us at our observed addresses. If this succeeds, we can rely on other peers being able to dial us as well and we can start advertising our listen address. + +A libp2p protocol called AutoNAT lets peers request dial-backs from peers providing the AutoNAT service. + +{{% notice "info" %}} +AutoNAT is currently implemented in go-libp2p via [go-libp2p-autonat](https://github.com/libp2p/go-libp2p/tree/master/p2p/host/autonat). +{{% /notice %}} + +[spec_identify]: https://github.com/libp2p/specs/tree/master/identify diff --git a/content/concepts/nat/circuit-relay.md b/content/concepts/nat/circuit-relay.md index cb347a12..1ce0b8e7 100644 --- a/content/concepts/nat/circuit-relay.md +++ b/content/concepts/nat/circuit-relay.md @@ -1,13 +1,11 @@ --- title: "Circuit Relay" -weight: 2 -pre: ' 2.4.2. ' +weight: 3 +pre: ' ' chapter: true summary: Circuit relay is a transport protocol that routes traffic between two peers over a third-party "relay" peer. In many cases, peers will be unable to traverse their NAT and/or firewall in a way that makes them publicly accessible. Or, they may not share common transport protocols that would allow them to communicate directly. --- -### Chapter 2.4.2 - # Circuit Relay Circuit relay is a [transport protocol](/concepts/transport/) that routes traffic between two peers over a third-party "relay" peer. diff --git a/content/concepts/nat/hole-punching.md b/content/concepts/nat/hole-punching.md new file mode 100644 index 00000000..c7a7d2a3 --- /dev/null +++ b/content/concepts/nat/hole-punching.md @@ -0,0 +1,25 @@ +--- +title: "Hole Punching" +weight: 1 +pre: ' ' +chapter: true +summary: The internet is composed of countless networks, bound together into shared address spaces by foundational transport protocols. As traffic moves between network boundaries, it's very common for a process called Network Address Translation to occur. Network Address Translation (NAT) maps an address from one address space to another. +--- + +# Hole Punching + +When an internal machine "dials out" and makes a connection to a public address, the router will map a public port to the internal IP address to use for the connection. In some cases, the router will also accept *incoming* connections on that port and route them to the same internal IP. + +libp2p will try to take advantage of this behavior when using IP-backed transports by using the same port for both dialing and listening, using a socket option called [`SO_REUSEPORT`](https://lwn.net/Articles/542629/). + +If our peer is in a favorable network environment, they will be able to make an outgoing connection and get a publicly-reachable listening port "for free," but they might never know it. Unfortunately, there's no way for the dialing program to discover what port was assigned to the connection on its own. + +However, an external peer can tell us what address they observed us on. We can then take that address and advertise it to other peers in our [peer routing network](/concepts/peer-routing/) to let them know where to find us. + +This basic premise of peers informing each other of their observed addresses is the foundation of [STUN][wiki_stun] (Session Traversal Utilities for NAT), which [describes][rfc_stun] a client / server protocol for discovering publicly reachable IP address and port combinations. + +One of libp2p's core protocols is the [identify protocol][spec_identify], which allows one peer to ask another for some identifying information. When sending over their [public key](/concepts/peers/) and some other useful information, the peer being identified includes the set of addresses that it has observed for the peer asking the question. + +This external discovery mechanism serves the same role as STUN, but without the need for a set of "STUN servers". + +The identify protocol allows some peers to communicate across NATs that would otherwise be impenetrable. \ No newline at end of file diff --git a/content/concepts/nat/nat.md b/content/concepts/nat/nat.md deleted file mode 100644 index 37fe51a4..00000000 --- a/content/concepts/nat/nat.md +++ /dev/null @@ -1,82 +0,0 @@ ---- -title: "Overview" -weight: 1 -pre: ' 2.4.1. ' -chapter: true -summary: The internet is composed of countless networks, bound together into shared address spaces by foundational transport protocols. As traffic moves between network boundaries, it's very common for a process called Network Address Translation to occur. Network Address Translation (NAT) maps an address from one address space to another. ---- - -### Chapter 2.4.1 - -# Overview - -The internet is composed of countless networks, bound together into shared address spaces by foundational [transport protocols](/concepts/transport/). - -As traffic moves between network boundaries, it's very common for a process called Network Address Translation to occur. Network Address Translation (NAT) maps an address from one address space to another. - -NAT allows many machines to share a single public address, and it is essential for the continued functioning of the IPv4 protocol, which would otherwise be unable to serve the needs of the modern networked population with its 32-bit address space. - -For example, when I connect to my home wifi, my computer gets an IPv4 address of `10.0.1.15`. This is part of a range of IP addresses reserved for internal use by private networks. When I make an outgoing connection to a public IP address, the router replaces my internal IP with its own public IP address. When data comes back from the other side, the router will translate back to the internal address. - -While NAT is usually transparent for outgoing connections, listening for incoming connections requires some configuration. The router listens on a single public IP address, but any number of machines on the internal network could handle the request. To serve requests, your router must be configured to send certain traffic to a specific machine, usually by mapping one or more TCP or UDP ports from the public IP to an internal one. - -While it's usually possible to manually configure routers, not everyone that wants to run a peer-to-peer application or other network service will have the ability to do so. - -We want libp2p applications to run everywhere, not just in data centers or on machines with stable public IP addresses. To enable this, here are the main approaches to NAT traversal available in libp2p today. - -### Automatic router configuration - -Many routers support automatic configuration protocols for port forwarding, most commonly [UPnP][wiki_upnp] or [nat-pmp.][wiki_nat-pmp] - -If your router supports one of those protocols, libp2p will attempt to automatically configure a port mapping that will allow it to listen for incoming traffic. This is usually the simplest option if supported by the network and libp2p implementation. - -{{% notice "info" %}} -Support for automatic NAT configuration varies by libp2p implementation. -Check the [current implementation status](https://libp2p.io/implementations/#nat-traversal) for details. -{{% /notice %}} - -### Hole-punching (STUN) - -When an internal machine "dials out" and makes a connection to a public address, the router will map a public port to the internal IP address to use for the connection. In some cases, the router will also accept *incoming* connections on that port and route them to the same internal IP. - -libp2p will try to take advantage of this behavior when using IP-backed transports by using the same port for both dialing and listening, using a socket option called [`SO_REUSEPORT`](https://lwn.net/Articles/542629/). - -If our peer is in a favorable network environment, they will be able to make an outgoing connection and get a publicly-reachable listening port "for free," but they might never know it. Unfortunately, there's no way for the dialing program to discover what port was assigned to the connection on its own. - -However, an external peer can tell us what address they observed us on. We can then take that address and advertise it to other peers in our [peer routing network](/concepts/peer-routing/) to let them know where to find us. - -This basic premise of peers informing each other of their observed addresses is the foundation of [STUN][wiki_stun] (Session Traversal Utilities for NAT), which [describes][rfc_stun] a client / server protocol for discovering publicly reachable IP address and port combinations. - -One of libp2p's core protocols is the [identify protocol][spec_identify], which allows one peer to ask another for some identifying information. When sending over their [public key](/concepts/peers/) and some other useful information, the peer being identified includes the set of addresses that it has observed for the peer asking the question. - -This external discovery mechanism serves the same role as STUN, but without the need for a set of "STUN servers". - -The identify protocol allows some peers to communicate across NATs that would otherwise be impenetrable. - -### AutoNAT - -While the [identify protocol][spec_identify] described above lets peers inform each other about their observed network addresses, not all networks will allow incoming connections on the same port used for dialing out. - -Once again, other peers can help us observe our situation, this time by attempting to dial us at our observed addresses. If this succeeds, we can rely on other peers being able to dial us as well and we can start advertising our listen address. - -A libp2p protocol called AutoNAT lets peers request dial-backs from peers providing the AutoNAT service. - -{{% notice "info" %}} -AutoNAT is currently implemented in go-libp2p via [go-libp2p-autonat](https://github.com/libp2p/go-libp2p/tree/master/p2p/host/autonat). -{{% /notice %}} - - -### Circuit Relay (TURN) - -In some cases, peers will be unable to traverse their NAT in a way that makes them publicly accessible. - -libp2p provides a [Circuit Relay protocol](/concepts/circuit-relay/) that allows peers to communicate indirectly via a helpful intermediary peer. - -This serves a similar function to the [TURN protocol](https://tools.ietf.org/html/rfc5766) in other systems. - -[wiki_upnp]: https://en.wikipedia.org/wiki/Universal_Plug_and_Play -[wiki_nat-pmp]: https://en.wikipedia.org/wiki/NAT_Port_Mapping_Protocol -[wiki_stun]: https://en.wikipedia.org/wiki/STUN -[rfc_stun]: https://tools.ietf.org/html/rfc3489 -[lwn_reuseport]: https://lwn.net/Articles/542629/ -[spec_identify]: https://github.com/libp2p/specs/tree/master/identify diff --git a/content/concepts/messaging/publish-subscribe.md b/content/concepts/publish-subscribe.md similarity index 99% rename from content/concepts/messaging/publish-subscribe.md rename to content/concepts/publish-subscribe.md index 496b34e8..99ae7774 100644 --- a/content/concepts/messaging/publish-subscribe.md +++ b/content/concepts/publish-subscribe.md @@ -1,14 +1,11 @@ --- -title: Publish/Subscribe -weight: 1 -pre: ' 2.7.1 ' +title: "Publish/Subscribe" +weight: 8 +pre: ' ' chapter: true -summary: Publish/Subscribe is a system where peers congregate around topics they are interested in. Peers interested in a topic are said to be subscribed to that topic. --- -### Chapter 2.7.1 - -# Publish/Subscribe +# Message Data Publish/Subscribe is a system where peers congregate around topics they are interested in. Peers interested in a topic are said to be subscribed to that diff --git a/content/concepts/routing/_index.md b/content/concepts/routing/_index.md deleted file mode 100644 index 645a2cbb..00000000 --- a/content/concepts/routing/_index.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -title: "Routing" -weight: 7 -pre: ' 2.6. ' -chapter: true ---- - -### Chapter 2.6 - -# Route with Peers - -{{% notice "note" %}} -Coming soon! -{{% /notice %}} diff --git a/content/concepts/routing/content-routing/_index.md b/content/concepts/routing/content-routing/_index.md deleted file mode 100644 index 967f1fc1..00000000 --- a/content/concepts/routing/content-routing/_index.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -title: "Content Routing" -weight: 2 -pre: ' 2.6.2. ' -chapter: true ---- - -### Chapter 2.6.2 - -# Content Routing - -{{% notice "note" %}} -Coming soon! -{{% /notice %}} diff --git a/content/concepts/routing/peer-routing/_index.md b/content/concepts/routing/peer-routing/_index.md deleted file mode 100644 index b51d50b3..00000000 --- a/content/concepts/routing/peer-routing/_index.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -title: "Peer Routing" -weight: 1 -pre: ' 2.6.1. ' -chapter: true ---- - -### Chapter 2.6.1 - -# Peer Routing - -{{% notice "note" %}} -Coming soon! -{{% /notice %}} diff --git a/content/concepts/secure-comm/_index.md b/content/concepts/secure-comm/_index.md index 6cb3ca00..776b5950 100644 --- a/content/concepts/secure-comm/_index.md +++ b/content/concepts/secure-comm/_index.md @@ -1,14 +1,23 @@ --- title: "Secure Communication" weight: 3 -pre: ' 2.2. ' +pre: ' ' chapter: true --- -### Chapter 2.2 - # Secure bytes -{{% notice "note" %}} -Coming soon! +Before two peers can transmit data, the communication channel they established +with a transport protocol should be secure. Each peer must also be able to open +multiple independent communication streams over a single channel. A transport +protocol like QUIC provides these guarantees out-of-the-box, but other transports +in libp2p do not provide the logic to secure their channel. +This requires an upgrade to the transport using an upgrader. +Security is always established first over the raw connection. + +{{% notice "info" %}} +Several security protocols are supported in libp2p for encryption, the two primary +ones being Noise and TLS 1.3. {{% /notice %}} + +{{% children description="true"%}} diff --git a/content/concepts/secure-comm/noise.md b/content/concepts/secure-comm/noise.md new file mode 100644 index 00000000..ab126a6f --- /dev/null +++ b/content/concepts/secure-comm/noise.md @@ -0,0 +1,13 @@ +--- +title: "Noise" +weight: 1 +pre: ' ' +chapter: true +summary: Learn about Noise in libp2p. +--- + +# Noise + +{{% notice "note" %}} +Coming soon! +{{% /notice %}} diff --git a/content/concepts/secure-comm/tls.md b/content/concepts/secure-comm/tls.md new file mode 100644 index 00000000..5aa05ff7 --- /dev/null +++ b/content/concepts/secure-comm/tls.md @@ -0,0 +1,13 @@ +--- +title: "TLS" +weight: 2 +pre: ' ' +chapter: true +summary: Learn about TLS 1.3 in libp2p. +--- + +# TLS + +{{% notice "note" %}} +Coming soon! +{{% /notice %}} diff --git a/content/concepts/security/_index.md b/content/concepts/security/_index.md deleted file mode 100644 index ec729779..00000000 --- a/content/concepts/security/_index.md +++ /dev/null @@ -1,12 +0,0 @@ ---- -title: "Security Considerations" -weight: 10 -pre: ' 2.9. ' -chapter: true ---- - -### Chapter 2.9 - -# Secure Data - -{{% children description="true"%}} diff --git a/content/concepts/security/security-considerations.md b/content/concepts/security/security-considerations.md index c5398f1c..0579fcba 100644 --- a/content/concepts/security/security-considerations.md +++ b/content/concepts/security/security-considerations.md @@ -1,14 +1,11 @@ --- -title: "Overview" -weight: 1 -pre: ' 2.9.1. ' +title: "Security Considerations" +weight: 10 +pre: ' ' chapter: true -summary: libp2p makes it simple to establish encrypted, authenticated communication channels between two peers, but there are other important security issues to consider when building robust peer-to-peer systems. --- -### Chapter 2.9.1 - -# Overview +# Secure Data libp2p makes it simple to establish [encrypted, authenticated communication channels](../secure-comms/) between two peers, but there are other important diff --git a/content/concepts/transports/_index.md b/content/concepts/transports/_index.md index e9a31bc3..66a0fec9 100644 --- a/content/concepts/transports/_index.md +++ b/content/concepts/transports/_index.md @@ -1,12 +1,33 @@ --- title: "Transports" weight: 2 -pre: ' 2.1. ' +pre: ' ' chapter: true --- -### Chapter 2.1 - # Exchange Bytes +When you make a connection from your computer to a machine on the internet, +chances are pretty good you're sending your bits and bytes using TCP/IP, the +wildly successful combination of the Internet Protocol, which handles addressing +and delivery of data packets, and the Transmission Control Protocol, which +ensures that the data that gets sent over the wire is received completely and in +the right order. + +Because TCP/IP is so ubiquitous and well-supported, it's often the default +choice for networked applications. In some cases, TCP adds too much overhead, +so applications might use [UDP](https://en.wikipedia.org/wiki/User_Datagram_Protocol), +a much simpler protocol with no guarantees about reliability or ordering. + +While TCP and UDP (together with IP) are the most common protocols in use today, +they are by no means the only options. Alternatives exist at lower levels +(e.g. sending raw ethernet packets or bluetooth frames), and higher levels +(e.g. QUIC, which is layered over UDP). + +The foundational protocols that move bits around are called transports, and one of +libp2p's core requirements is to be transport agnostic. This means that the decision +of what transport protocol to use is up to the developer, and an application can support +many different transports at the same time. Learn about the fundamentals of transport +protocols in libp2p. + {{% children description="true"%}} diff --git a/content/concepts/transports/listen-and-dial.md b/content/concepts/transports/listen-and-dial.md new file mode 100644 index 00000000..b82ea4e2 --- /dev/null +++ b/content/concepts/transports/listen-and-dial.md @@ -0,0 +1,98 @@ +--- +title: "Listening and Dialing" +weight: 1 +pre: ' ' +chapter: true +summary: Transports are defined in terms of two core operations, listening and dialing. Learn about how peers use both interfaces in libp2p. +--- + +# Listening and Dialing + +Transports are defined in terms of two core operations, **listening** and +**dialing**. + +Listening means that you can accept incoming connections from other peers, +using whatever facility is provided by the +transport implementation. For example, a TCP transport on a unix platform could +use the `bind` and `listen` system calls to have the operating system route +traffic on a given TCP port to the application. + +Dialing is the process of opening an outgoing connection to a listening peer. +Like listening, the specifics are determined by the implementation, but every +transport in a libp2p implementation will share the same programmatic interface. + +## Addresses + +Before you can dial up a peer and open a connection, you need to know how to +reach them. Because each transport will likely require its own address scheme, +libp2p uses a convention called a "multiaddress" or `multiaddr` to encode +many different addressing schemes. + +The [addressing doc](/concepts/addressing/) goes into more detail, but an overview of +how multiaddresses work is helpful for understanding the dial and listen +interfaces. + +Here's an example of a multiaddr for a TCP/IP transport: + +``` +/ip4/7.7.7.7/tcp/6543 +``` + +This is equivalent to the more familiar `7.7.7.7:6543` construction, but it +has the advantage of being explicit about the protocols that are being +described. With the multiaddr, you can see at a glance that the `7.7.7.7` +address belongs to the IPv4 protocol, and the `6543` belongs to TCP. + +For more complex examples, see [Addressing](/concepts/addressing/). + +Both dial and listen deal with multiaddresses. When listening, you give the +transport the address you'd like to listen on, and when dialing you provide the +address to dial to. + +When dialing a remote peer, the multiaddress should include the +[PeerId](/concepts/peers/) of the peer you're trying to reach. +This lets libp2p establish a [secure communication channel](/concepts/secure-comms/) +and prevents impersonation. + +An example multiaddress that includes a `PeerId`: + +``` +/ip4/1.2.3.4/tcp/4321/p2p/QmcEPrat8ShnCph8WjkREzt5CPXF2RwhYxYBALDcLC1iV6 +``` + +The `/p2p/QmcEPrat8ShnCph8WjkREzt5CPXF2RwhYxYBALDcLC1iV6` component uniquely +identifies the remote peer using the hash of its public key. +For more, see [Peer Identity](/concepts/peers/). + +{{% notice "tip" %}} + +When [peer routing](/concepts/peer-routing/) is enabled, you can dial peers +using just their PeerId, without needing to know their transport addresses +before hand. + +{{% /notice %}} + +## Supporting multiple transports + +libp2p applications often need to support multiple transports at once. For +example, you might want your services to be usable from long-running daemon +processes via TCP, while also accepting websocket connections from peers running +in a web browser. + +The libp2p component responsible for managing the transports is called the +[switch][definition_switch], which also coordinates +[protocol negotiation](/concepts/protocols/#protocol-negotiation), +[stream multiplexing](/concepts/stream-multiplexing), +[establishing secure communication](/concepts/secure-comms/) and other forms of +"connection upgrading". + +The switch provides a single "entry point" for dialing and listening, and frees +up your application code from having to worry about the specific transports +and other pieces of the "connection stack" that are used under the hood. + +{{% notice "note" %}} +The term "swarm" was previously used to refer to what is now called the "switch", +and some places in the codebase still use the "swarm" terminology. +{{% /notice %}} + +[definition_switch]: /reference/glossary/#switch diff --git a/content/concepts/transports/overview.md b/content/concepts/transports/overview.md deleted file mode 100644 index e7daa254..00000000 --- a/content/concepts/transports/overview.md +++ /dev/null @@ -1,315 +0,0 @@ ---- -title: "Overview" -weight: 1 -pre: ' 2.1.1. ' -chapter: true -summary: The foundational protocols that move bits around are called transports, and one of libp2p's core requirements is to be transport agnostic. This means that the decision of what transport protocol to use is up to the developer, and an application can support many different transports at the same time. ---- - -### Chapter 2.1.1 - -# Overview - -When you make a connection from your computer to a machine on the internet, -chances are pretty good you're sending your bits and bytes using TCP/IP, the -wildly successful combination of the Internet Protocol, which handles addressing -and delivery of data packets, and the Transmission Control Protocol, which -ensures that the data that gets sent over the wire is received completely and in -the right order. - -Because TCP/IP is so ubiquitous and well-supported, it's often the default -choice for networked applications. In some cases, TCP adds too much overhead, -so applications might use [UDP](https://en.wikipedia.org/wiki/User_Datagram_Protocol), -a much simpler protocol with no guarantees about reliability or ordering. - -While TCP and UDP (together with IP) are the most common protocols in use today, -they are by no means the only options. Alternatives exist at lower levels -(e.g. sending raw ethernet packets or bluetooth frames), and higher levels -(e.g. QUIC, which is layered over UDP). - -In libp2p, we call these foundational protocols that move bits around -**transports**, and one of libp2p's core requirements is to be -*transport agnostic*. This means that the decision of what transport protocol -to use is up to the developer, and in fact one application can support many -different transports at the same time. - -## Listening and Dialing - -Transports are defined in terms of two core operations, **listening** and -**dialing**. - -Listening means that you can accept incoming connections from other peers, -using whatever facility is provided by the -transport implementation. For example, a TCP transport on a unix platform could -use the `bind` and `listen` system calls to have the operating system route -traffic on a given TCP port to the application. - -Dialing is the process of opening an outgoing connection to a listening peer. -Like listening, the specifics are determined by the implementation, but every -transport in a libp2p implementation will share the same programmatic interface. - -## Addresses - -Before you can dial up a peer and open a connection, you need to know how to -reach them. Because each transport will likely require its own address scheme, -libp2p uses a convention called a "multiaddress" or `multiaddr` to encode -many different addressing schemes. - -The [addressing doc](/concepts/addressing/) goes into more detail, but an overview of -how multiaddresses work is helpful for understanding the dial and listen -interfaces. - -Here's an example of a multiaddr for a TCP/IP transport: - -``` -/ip4/7.7.7.7/tcp/6543 -``` - -This is equivalent to the more familiar `7.7.7.7:6543` construction, but it -has the advantage of being explicit about the protocols that are being -described. With the multiaddr, you can see at a glance that the `7.7.7.7` -address belongs to the IPv4 protocol, and the `6543` belongs to TCP. - -For more complex examples, see [Addressing](/concepts/addressing/). - -Both dial and listen deal with multiaddresses. When listening, you give the -transport the address you'd like to listen on, and when dialing you provide the -address to dial to. - -When dialing a remote peer, the multiaddress should include the -[PeerId](/concepts/peers/) of the peer you're trying to reach. -This lets libp2p establish a [secure communication channel](/concepts/secure-comms/) -and prevents impersonation. - -An example multiaddress that includes a `PeerId`: - -``` -/ip4/1.2.3.4/tcp/4321/p2p/QmcEPrat8ShnCph8WjkREzt5CPXF2RwhYxYBALDcLC1iV6 -``` - -The `/p2p/QmcEPrat8ShnCph8WjkREzt5CPXF2RwhYxYBALDcLC1iV6` component uniquely -identifies the remote peer using the hash of its public key. -For more, see [Peer Identity](/concepts/peers/). - -{{% notice "tip" %}} - -When [peer routing](/concepts/peer-routing/) is enabled, you can dial peers -using just their PeerId, without needing to know their transport addresses -before hand. - -{{% /notice %}} - -## Supporting multiple transports - -libp2p applications often need to support multiple transports at once. For -example, you might want your services to be usable from long-running daemon -processes via TCP, while also accepting websocket connections from peers running -in a web browser. - -The libp2p component responsible for managing the transports is called the -[switch][definition_switch], which also coordinates -[protocol negotiation](/concepts/protocols/#protocol-negotiation), -[stream multiplexing](/concepts/stream-multiplexing), -[establishing secure communication](/concepts/secure-comms/) and other forms of -"connection upgrading". - -The switch provides a single "entry point" for dialing and listening, and frees -up your application code from having to worry about the specific transports -and other pieces of the "connection stack" that are used under the hood. - -{{% notice "note" %}} -The term "swarm" was previously used to refer to what is now called the "switch", -and some places in the codebase still use the "swarm" terminology. -{{% /notice %}} - -[definition_switch]: /reference/glossary/#switch - -## QUIC - -QUIC is a new transport protocol that provides an always-encrypted, stream-multiplexed -connection built on top of UDP. It started as an experiment by Google between Google -services and Chrome in 2014, and was later standardized by the IETF in -[RFC 9000](https://datatracker.ietf.org/doc/html/rfc9000), -[RFC 9001](https://datatracker.ietf.org/doc/html/rfc9001), and -[RFC 9002](https://datatracker.ietf.org/doc/html/rfc9002). - -### Key challenges with TCP - -1. Head-of-line blocking (HoL blocking): TCP is a single byte stream exposed by the - kernel, so streams layered on top of TCP experience head-of-line (HoL) blocking. - - {{% notice "info" %}} - In TCP, head-of-line blocking occurs when a single packet is lost, and packets delivered - after that need to wait in the kernel buffer until a retransmission for the lost packet - is received. - {{% /notice %}} - -2. Ossification: Because the header of TCP packet is not encrypted, middleboxes can - inspect and modify TCP header fields and may break unexpectedly when they encounter - anything they don’t understand. This makes it practically impossible to deploy any - changes to the TCP protocol that change the wire format. - -3. Handshake inefficiency: TCP spends one network round-trip (RTT) on verifying the - client's address. Only after this can TLS start the cryptographic handshake, consuming - another RTT. Setting up an encrypted connection therefore always takes 2 RTTs. - -QUIC was designed with the following goals in mind: - -- Making the transport layer aware of streams, so that packet loss doesn't cause HoL blocking - between streams. -- Reducing the latency of connection establishment to a single RTT for new connections, and to - allow sending of 0 RTT application data for resumed connections. -- Encrypting as much as possible. This eliminates the ossification risk, as middleboxes aren't - able to read any encrypted fields. This allows future evolution of the protocol. - -### Comparing HTTP/2 and HTTP/3 - -In addition to defining the QUIC transport, the IETF also standardized a new version of HTTP that runs on top of QUIC: HTTP/3 ( -[RFC 9114](https://datatracker.ietf.org/doc/html/rfc9114)). HTTP/3 combines the advantages -of the existing transfer protocols HTTP/2 and HTTP over QUIC in one standard for faster and -more stable data transmission. - -The following diagram illustrates the OSI model for HTTP/2 and HTTP/3 [1]: - -![HTTP/2 & HTTP/3 OSI model](https://cloudspoint.xyz/wp-content/uploads/2022/03/http3.png) - -A web browser connection typically entails the following **(TCP+TLS+HTTP/2)**: - -1. Transport layer: TCP runs on top of the IP layer to provide a reliable - byte stream. - - TCP provides a reliable, bidirectional connection between two end systems. -2. Security layer: A TLS handshake runs on top of TCP to - establish an encrypted and authenticated connection. - - Standard TLS over TCP requires 3 RTT. A typical TLS 1.3 handshake takes 1 RTT. -3. Application layer: HTTP runs on a secure transport connection to transfer - information and applies a stream muxer to serve multiple requests. - - Application data starts to flow. - -In contrast, HTTP/3 runs over [QUIC](##what-is-quic), where QUIC is similar to -TCP+TLS+HTTP/2 and runs over UDP. Building on UDP allows HTTP/3 to bypass the challenges -found in TCP and use all the advantages of HTTP/2 and HTTP over QUIC. - -### What is QUIC? - -QUIC combines the functionality of these layers. Instead of TCP, it builds on UDP. -When a UDP datagram is lost, it is not automatically retransmitted by the kernel. -QUIC therefore takes responsibility for loss detection and repair itself. By using -encryption, QUIC avoids ossified middleboxes. The TLS 1.3 handshake is performed in -the first flight, removing the cost of verifying the client’s address and saving an -RTT. QUIC also exposes multiple streams (and not just a single byte stream), so -no stream multiplexer is needed at the application layer. Part of the application -layer is also built directly into QUIC. - -In addition, a client can make use of QUIC's 0 RTT feature for subsequent connections -when it has already communicated with a certain server. The client can then send -(encrypted) application data even before the QUIC handshake has finished. - -### QUIC in libp2p - -libp2p only supports bidirectional streams and uses TLS 1.3 by default. -Since QUIC already provides an encrypted, stream-multiplexed connection, -libp2p directly uses QUIC streams, without any additional framing. - -To authenticate each others' peer IDs, peers encode their peer ID into a self-signed -certificate, which they sign using their host's private key. This is the same way peer -IDs are authenticated in the -[libp2p TLS handshake](https://github.com/libp2p/specs/blob/master/tls/tls.md). - -{{% notice "note" %}} - -To be clear, there is no additional security handshake and stream muxer needed as QUIC -provides all of this by default. This also means that establishing a libp2p connection between -two nodes using QUIC only takes a single RTT. - -{{% /notice %}} - -Following the multiaddress format described earlier, a standard QUIC connection will -look like: `/ip4/127.0.0.1/udp/65432/quic/`. - -## WebTransport - -While browsers perform HTTP request using HTTP/3, so far they don't offer an API to allow applications -to gain access to the underlying QUIC stream. -WebTransport is a new specification that uses QUIC to offer an alternative to -WebSocket. Conceptually, it can be considered WebSocket over QUIC. -It allows browsers to establish a stream-multiplexed and bidirectional connection -to servers, and use streams to send and receive application data. - -While WebSocket provides a single bidirectional, full-duplex communication between a -browser and a server over a TCP connection, WebTransport exposes allows the endpoints to use multiple -streams in parallel. - -When connecting to a WebSocket server, browsers require the server to present a -TLS certificate signed by a trusted CA (certificate authority). Few nodes have such -a certificate, which is the reason that WebSocket never saw widespread adoption in the -libp2p network. libp2p WebTransport offers a browser API that includes a way to -accept the server's certificate by checking the (SHA-256) hash of the certificate -(using the -[`serverCertificateHashes` option](https://www.w3.org/TR/webtransport/#dom-webtransportoptions-servercertificatehashes)), -even if the certificate is "just" a self-signed certificate. This allows us to connect -any browser node to any server node, as long as the browser knows the certificate hash in -advance (see [WebTransport in libp2p](#webtransport-in-libp2p) for how WebTransport addresses -achieve this). - -Therefore, WebTransport exhibits all the advantages of QUIC over TCP, that being -faster handshakes, no HoL blocking, and being future-proof. - -{{% notice "caution" %}} - -There is an experimental WebTransport transport in go-libp2p that is part -of the [v0.23 release](https://github.com/libp2p/go-libp2p/releases/tag/v0.23.0). -The implementation should be used experimentally and is not recommended for production -environments. - -js-libp2p also plans to release -[WebTransport support](https://github.com/libp2p/js-libp2p-webtransport) very soon. - -There are currently no concrete plans to support WebTransport beyond the Go and JS -implementations. - -{{% /notice %}} - -For network stacks like libp2p, WebTransport is a pluggable -protocol that fits well with a modular network design. - -For a standard WebSocket connection, the roundtrips required are as follows: - -- 1 RTT for TCP handshake -- 1 RTT for TLS 1.3 handshake -- 1 RTT for WebSocket upgrade -- 1 RTT for multistream security negotiation (Noise or TLS 1.3) -- 1 RTT for security handshake (Noise or TLS 1.3) -- 1 RTT for multistream muxer negotiation (mplex or yamux) - -In total, 6 RTTs. - -WebTransport running over QUIC only requires 3 RTTs, as: - -- 1 RTT for QUIC handshake -- 1 RTT for WebTransport handshake -- 1 RTT for libp2p handshake; one for multistream and one for authentication - (with a Noise handshake) - -In principle, the WebTransport protocol would even allow running the WebTransport -handshake and the Noise handshake in parallel. However, this is currently not -possible since the [browser API doesn't allow that yet](https://github.com/w3c/webtransport/issues/409). - -### WebTransport in libp2p - -WebTransport multiaddresses are composed of a QUIC multiaddr, followed -by `/webtransport` and a list of multihashes of the node certificates that the server uses. - -For instance, for multiaddress `/ip4/127.0.0.1/udp/123/quic/webtransport/certhash/`, -a standard local QUIC connection is defined up until and including `/quic.` -Then, `/webtransport/` runs over QUIC. The self-signed certificate hash that the -server will use to verify the connection. - -The WebTransport CONNECT request is sent to an HTTPS endpoint. libp2p WebTransport server use -`/.well-known/libp2p-webtransport`. For instance, the WebTransport URL of a WebTransport -server advertising `/ip4/1.2.3.4/udp/1234/quic/webtransport/` would be -`https://1.2.3.4:1234/.well-known/libp2p-webtransport?type=noise` -(the ?type=noise refers to the authentication scheme using Noise). - -## References - -[1] [What is HTTP/3 by Cloudspoint](https://cloudspoint.xyz/what-is-http3/) diff --git a/content/concepts/transports/quic.md b/content/concepts/transports/quic.md new file mode 100644 index 00000000..2efa59f7 --- /dev/null +++ b/content/concepts/transports/quic.md @@ -0,0 +1,113 @@ +--- +title: "QUIC" +weight: 1 +pre: ' ' +chapter: true +summary: QUIC is a new transport protocol that provides an always-encrypted, stream-multiplexed connection built on top of UDP. Learn about QUIC and how it is used in libp2p. +--- + +# QUIC + +QUIC is a new transport protocol that provides an always-encrypted, stream-multiplexed +connection built on top of UDP. It started as an experiment by Google between Google +services and Chrome in 2014, and was later standardized by the IETF in +[RFC 9000](https://datatracker.ietf.org/doc/html/rfc9000), +[RFC 9001](https://datatracker.ietf.org/doc/html/rfc9001), and +[RFC 9002](https://datatracker.ietf.org/doc/html/rfc9002). + +### Key challenges with TCP + +1. Head-of-line blocking (HoL blocking): TCP is a single byte stream exposed by the + kernel, so streams layered on top of TCP experience head-of-line (HoL) blocking. + + {{% notice "info" %}} + In TCP, head-of-line blocking occurs when a single packet is lost, and packets delivered + after that need to wait in the kernel buffer until a retransmission for the lost packet + is received. + {{% /notice %}} + +2. Ossification: Because the header of TCP packet is not encrypted, middleboxes can + inspect and modify TCP header fields and may break unexpectedly when they encounter + anything they don’t understand. This makes it practically impossible to deploy any + changes to the TCP protocol that change the wire format. + +3. Handshake inefficiency: TCP spends one network round-trip (RTT) on verifying the + client's address. Only after this can TLS start the cryptographic handshake, consuming + another RTT. Setting up an encrypted connection therefore always takes 2 RTTs. + +QUIC was designed with the following goals in mind: + +- Making the transport layer aware of streams, so that packet loss doesn't cause HoL blocking + between streams. +- Reducing the latency of connection establishment to a single RTT for new connections, and to + allow sending of 0 RTT application data for resumed connections. +- Encrypting as much as possible. This eliminates the ossification risk, as middleboxes aren't + able to read any encrypted fields. This allows future evolution of the protocol. + +### Comparing HTTP/2 and HTTP/3 + +In addition to defining the QUIC transport, the IETF also standardized a new version of HTTP that runs on top of QUIC: HTTP/3 ( +[RFC 9114](https://datatracker.ietf.org/doc/html/rfc9114)). HTTP/3 combines the advantages +of the existing transfer protocols HTTP/2 and HTTP over QUIC in one standard for faster and +more stable data transmission. + +The following diagram illustrates the OSI model for HTTP/2 and HTTP/3 [1]: + +![HTTP/2 & HTTP/3 OSI model](https://cloudspoint.xyz/wp-content/uploads/2022/03/http3.png) + +A web browser connection typically entails the following **(TCP+TLS+HTTP/2)**: + +1. Transport layer: TCP runs on top of the IP layer to provide a reliable + byte stream. + - TCP provides a reliable, bidirectional connection between two end systems. +2. Security layer: A TLS handshake runs on top of TCP to + establish an encrypted and authenticated connection. + - Standard TLS over TCP requires 3 RTT. A typical TLS 1.3 handshake takes 1 RTT. +3. Application layer: HTTP runs on a secure transport connection to transfer + information and applies a stream muxer to serve multiple requests. + - Application data starts to flow. + +In contrast, HTTP/3 runs over [QUIC](##what-is-quic), where QUIC is similar to +TCP+TLS+HTTP/2 and runs over UDP. Building on UDP allows HTTP/3 to bypass the challenges +found in TCP and use all the advantages of HTTP/2 and HTTP over QUIC. + +### What is QUIC? + +QUIC combines the functionality of these layers. Instead of TCP, it builds on UDP. +When a UDP datagram is lost, it is not automatically retransmitted by the kernel. +QUIC therefore takes responsibility for loss detection and repair itself. By using +encryption, QUIC avoids ossified middleboxes. The TLS 1.3 handshake is performed in +the first flight, removing the cost of verifying the client’s address and saving an +RTT. QUIC also exposes multiple streams (and not just a single byte stream), so +no stream multiplexer is needed at the application layer. Part of the application +layer is also built directly into QUIC. + +In addition, a client can make use of QUIC's 0 RTT feature for subsequent connections +when it has already communicated with a certain server. The client can then send +(encrypted) application data even before the QUIC handshake has finished. + +### QUIC in libp2p + +libp2p only supports bidirectional streams and uses TLS 1.3 by default. +Since QUIC already provides an encrypted, stream-multiplexed connection, +libp2p directly uses QUIC streams, without any additional framing. + +To authenticate each others' peer IDs, peers encode their peer ID into a self-signed +certificate, which they sign using their host's private key. This is the same way peer +IDs are authenticated in the +[libp2p TLS handshake](https://github.com/libp2p/specs/blob/master/tls/tls.md). + +{{% notice "note" %}} + +To be clear, there is no additional security handshake and stream muxer needed as QUIC +provides all of this by default. This also means that establishing a libp2p connection between +two nodes using QUIC only takes a single RTT. + +{{% /notice %}} + +Following the multiaddress format described earlier, a standard QUIC connection will +look like: `/ip4/127.0.0.1/udp/65432/quic/`. + +## References + +[1] [What is HTTP/3 by Cloudspoint](https://cloudspoint.xyz/what-is-http3/) diff --git a/content/concepts/transports/webtransport.md b/content/concepts/transports/webtransport.md new file mode 100644 index 00000000..d78bbfb4 --- /dev/null +++ b/content/concepts/transports/webtransport.md @@ -0,0 +1,91 @@ +--- +title: "WebTransport" +weight: 3 +pre: ' ' +chapter: true +summary: WebTransport is a new specification that uses QUIC to offer an alternative to WebSocket. Conceptually, it can be considered WebSocket over QUIC.Learn about WebTransport and how it is used in libp2p. +--- + +# WebTransport + +While browsers perform HTTP request using HTTP/3, so far they don't offer an API to allow +applications to gain access to the underlying QUIC stream. +WebTransport is a new specification that uses QUIC to offer an alternative to +WebSocket. Conceptually, it can be considered WebSocket over QUIC. +It allows browsers to establish a stream-multiplexed and bidirectional connection +to servers, and use streams to send and receive application data. + +While WebSocket provides a single bidirectional, full-duplex communication between a +browser and a server over a TCP connection, WebTransport exposes allows the endpoints to use multiple +streams in parallel. + +When connecting to a WebSocket server, browsers require the server to present a +TLS certificate signed by a trusted CA (certificate authority). Few nodes have such +a certificate, which is the reason that WebSocket never saw widespread adoption in the +libp2p network. libp2p WebTransport offers a browser API that includes a way to +accept the server's certificate by checking the (SHA-256) hash of the certificate +(using the +[`serverCertificateHashes` option](https://www.w3.org/TR/webtransport/#dom-webtransportoptions-servercertificatehashes)), +even if the certificate is "just" a self-signed certificate. This allows us to connect +any browser node to any server node, as long as the browser knows the certificate hash in +advance (see [WebTransport in libp2p](#webtransport-in-libp2p) for how WebTransport addresses +achieve this). + +Therefore, WebTransport exhibits all the advantages of QUIC over TCP, that being +faster handshakes, no HoL blocking, and being future-proof. + +{{% notice "caution" %}} + +There is an experimental WebTransport transport in go-libp2p that is part +of the [v0.23 release](https://github.com/libp2p/go-libp2p/releases/tag/v0.23.0). +The implementation should be used experimentally and is not recommended for production +environments. + +js-libp2p also plans to release +[WebTransport support](https://github.com/libp2p/js-libp2p-webtransport) very soon. + +There are currently no concrete plans to support WebTransport beyond the Go and JS +implementations. + +{{% /notice %}} + +For network stacks like libp2p, WebTransport is a pluggable +protocol that fits well with a modular network design. + +For a standard WebSocket connection, the roundtrips required are as follows: + +- 1 RTT for TCP handshake +- 1 RTT for TLS 1.3 handshake +- 1 RTT for WebSocket upgrade +- 1 RTT for multistream security negotiation (Noise or TLS 1.3) +- 1 RTT for security handshake (Noise or TLS 1.3) +- 1 RTT for multistream muxer negotiation (mplex or yamux) + +In total, 6 RTTs. + +WebTransport running over QUIC only requires 3 RTTs, as: + +- 1 RTT for QUIC handshake +- 1 RTT for WebTransport handshake +- 1 RTT for libp2p handshake; one for multistream and one for authentication + (with a Noise handshake) + +In principle, the WebTransport protocol would even allow running the WebTransport +handshake and the Noise handshake in parallel. However, this is currently not +possible since the [browser API doesn't allow that yet](https://github.com/w3c/webtransport/issues/409). + +### WebTransport in libp2p + +WebTransport multiaddresses are composed of a QUIC multiaddr, followed +by `/webtransport` and a list of multihashes of the node certificates that the server uses. + +For instance, for multiaddress `/ip4/127.0.0.1/udp/123/quic/webtransport/certhash/`, +a standard local QUIC connection is defined up until and including `/quic.` +Then, `/webtransport/` runs over QUIC. The self-signed certificate hash that the +server will use to verify the connection. + +The WebTransport CONNECT request is sent to an HTTPS endpoint. libp2p WebTransport server use +`/.well-known/libp2p-webtransport`. For instance, the WebTransport URL of a WebTransport +server advertising `/ip4/1.2.3.4/udp/1234/quic/webtransport/` would be +`https://1.2.3.4:1234/.well-known/libp2p-webtransport?type=noise` +(the ?type=noise refers to the authentication scheme using Noise). diff --git a/content/get-involved/_index.md b/content/get-involved/_index.md index 797db15f..421658d9 100644 --- a/content/get-involved/_index.md +++ b/content/get-involved/_index.md @@ -1,12 +1,10 @@ --- title: "Get Involved" -weight: 4 -pre: "4. " +weight: 5 +pre: " " chapter: true --- -### Chapter 4 - # Get Involved with libp2p Find out how to get involved with libp2p and join the community. diff --git a/content/get-involved/community.md b/content/get-involved/community.md index 53d848ed..51c0454d 100644 --- a/content/get-involved/community.md +++ b/content/get-involved/community.md @@ -1,12 +1,10 @@ --- title: "Community" weight: 2 -pre: ' 4.2 ' +pre: ' ' chapter: true --- -### Chapter 4.2 - # Community We love talking about libp2p, and we'd be happy to have you in the mix. diff --git a/content/get-involved/contribute.md b/content/get-involved/contribute.md index 5cf7f8a0..b45e49c3 100644 --- a/content/get-involved/contribute.md +++ b/content/get-involved/contribute.md @@ -1,12 +1,10 @@ --- title: "How to Contribute" weight: 1 -pre: ' 4.1 ' +pre: ' ' chapter: true --- -### Chapter 4.1 - # How to Contribute So you want to contribute to libp2p and the peer-to-peer ecosystem? Here is a quick listing diff --git a/content/guides/_index.md b/content/guides/_index.md index 2e85a591..3780c9bd 100644 --- a/content/guides/_index.md +++ b/content/guides/_index.md @@ -1,12 +1,10 @@ --- title: "Guides" weight: 3 -pre: "3. " +pre: " " chapter: true --- -### Chapter 3 - # Get Started with libp2p ## Examples using libp2p diff --git a/content/introduction/_index.md b/content/introduction/_index.md index 5de8f639..ab4460c2 100644 --- a/content/introduction/_index.md +++ b/content/introduction/_index.md @@ -1,14 +1,11 @@ --- -title: "Get Started with libp2p" -menuTitle: "Basics" +title: "Introduction" weight: 1 -pre: "1. " +pre: " " chapter: true --- -### Chapter 1 - -# Get Started with libp2p +# An Introduction to libp2p ## What is libp2p? @@ -170,14 +167,14 @@ specific information and references: - [The Permissive License Stack](https://protocol.ai/blog/announcing-the-permissive-license-stack) is a licensing strategy for software development that embraces open-source values. -[glossary]: {{< ref "concepts/miscellaneous/glossary.md" >}} -[definition_dht]: {{< ref "concepts/miscellaneous/glossary.md#dht" >}} -[definition_p2p]: {{< ref "concepts/miscellaneous/glossary.md#p2p" >}} -[definition_peer]: {{< ref "concepts/miscellaneous/glossary.md#peer" >}} -[definition_peerid]: {{< ref "concepts/miscellaneous/glossary.md#peerid" >}} -[definition_secio]: {{< ref "concepts/miscellaneous/glossary.md#secio" >}} -[definition_muiltiaddress]: {{< ref "concepts/miscellaneous/glossary.md#multiaddr" >}} -[definition_client_server]: {{< ref "concepts/miscellaneous/glossary.md#client-server" >}} +[glossary]: {{< ref "concepts/glossary.md" >}} +[definition_dht]: {{< ref "concepts/glossary.md#dht" >}} +[definition_p2p]: {{< ref "concepts/glossary.md#p2p" >}} +[definition_peer]: {{< ref "concepts/glossary.md#peer" >}} +[definition_peerid]: {{< ref "concepts/glossary.md#peerid" >}} +[definition_secio]: {{< ref "concepts/glossary.md#secio" >}} +[definition_muiltiaddress]: {{< ref "concepts/glossary.md#multiaddr" >}} +[definition_client_server]: {{< ref "concepts/glossary.md#client-server" >}} [spec_content_routing]: https://github.com/libp2p/specs/blob/master/kad-dht/README.md [spec_pubsub]: https://github.com/libp2p/specs/blob/master/pubsub/README.md diff --git a/content/reference/_index.md b/content/reference/_index.md new file mode 100644 index 00000000..ca280559 --- /dev/null +++ b/content/reference/_index.md @@ -0,0 +1,18 @@ +--- +title: "Reference" +weight: 4 +pre: " " +chapter: true +--- + +# Reference Documentation + +Find out how to get involved with libp2p and join the community. + +{{% children description="true" %}} + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/concepts/dos-mitigation/overview.md b/content/reference/dos-mitigation.md similarity index 97% rename from content/concepts/dos-mitigation/overview.md rename to content/reference/dos-mitigation.md index ffe5faca..7421ca45 100644 --- a/content/concepts/dos-mitigation/overview.md +++ b/content/reference/dos-mitigation.md @@ -1,14 +1,11 @@ --- -title: "Overview" -weight: 1 -pre: ' 2.10.1 ' +title: "DoS Mitigation" +weight: 2 +pre: ' ' chapter: true -summary: DOS mitigation is an essential part of any peer-to-peer application. We need to design our protocols to be resilient to malicious peers. We need to monitor our application for signs of suspicious activity or an attack. And we need to be able to respond to an attack. --- -### Chapter 2.10.1 - -# Overview +# DoS Mitigation DOS mitigation is an essential part of any P2P application. We need to design our protocols to be resilient to malicious peers. We need to monitor our @@ -19,7 +16,7 @@ Here we'll cover how we can use libp2p to achieve the above goals. ## Table of contents -- [Overview](#overview) +- [DoS Mitigation](#dos-mitigation) - [What we mean by a DOS attack](#what-we-mean-by-a-dos-attack) - [Incorporating DOS mitigation from the start](#incorporating-dos-mitigation-from-the-start) - [Limit the number of connections your application needs](#limit-the-number-of-connections-your-application-needs) diff --git a/content/reference/specs.md b/content/reference/specs.md new file mode 100644 index 00000000..488609a8 --- /dev/null +++ b/content/reference/specs.md @@ -0,0 +1,12 @@ +--- +title: "Specifications" +weight: 1 +pre: ' ' +chapter: true +--- + +# Technical Specifications + +Check out the [specs repo](https://github.com/libp2p/specs) for details about the libp2p architecture. + + diff --git a/static/css/theme.css b/static/css/theme.css index 2d27f4eb..75445fe7 100644 --- a/static/css/theme.css +++ b/static/css/theme.css @@ -435,7 +435,7 @@ textarea:focus, input[type="email"]:focus, input[type="number"]:focus, input[typ #chapter p { text-align: left; font-size: 1.3rem; - padding: 2rem 1rem; + padding: 0.5rem 1rem; } #footer { padding: 3rem 1rem; From 2d37360a1345bff5ed65b569bd5cabfbc84dcfd9 Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Fri, 14 Oct 2022 17:51:15 -0400 Subject: [PATCH 23/30] various edits, update summaries, fix assets, and more --- config.toml | 10 +++- content/_index.md | 1 + content/community/_index.md | 12 +++++ .../{get-involved => community}/community.md | 7 +-- .../{get-involved => community}/contribute.md | 3 +- content/concepts/_index.md | 20 ++------ .../{introduction => fundamentals}/_index.md | 1 + .../addressing.md | 0 .../{introduction => fundamentals}/peers.md | 0 .../protocols.md | 0 content/concepts/glossary.md | 1 + content/concepts/multiplex/_index.md | 1 + content/concepts/multiplex/mplex.md | 2 +- content/concepts/multiplex/quic.md | 2 +- content/concepts/multiplex/switch.md | 2 +- content/concepts/multiplex/yamux.md | 2 +- content/concepts/nat/_index.md | 1 + content/concepts/nat/autonat.md | 2 +- content/concepts/nat/circuit-relay.md | 2 +- content/concepts/nat/hole-punching.md | 6 +-- content/concepts/publish-subscribe.md | 47 ++++++++++--------- content/concepts/secure-comm/_index.md | 9 ++-- content/concepts/secure-comm/overview.md | 14 ------ .../security/security-considerations.md | 1 + content/concepts/transports/_index.md | 1 + content/get-involved/_index.md | 18 ------- content/guides/_index.md | 2 +- content/introduction/_index.md | 2 +- content/reference/_index.md | 9 +--- content/reference/dos-mitigation.md | 3 +- content/reference/specs.md | 3 +- layouts/partials/renderMarkdown.html | 27 +++++++++++ 32 files changed, 108 insertions(+), 103 deletions(-) create mode 100644 content/community/_index.md rename content/{get-involved => community}/community.md (87%) rename content/{get-involved => community}/contribute.md (98%) rename content/concepts/{introduction => fundamentals}/_index.md (65%) rename content/concepts/{introduction => fundamentals}/addressing.md (100%) rename content/concepts/{introduction => fundamentals}/peers.md (100%) rename content/concepts/{introduction => fundamentals}/protocols.md (100%) delete mode 100644 content/concepts/secure-comm/overview.md delete mode 100644 content/get-involved/_index.md create mode 100644 layouts/partials/renderMarkdown.html diff --git a/config.toml b/config.toml index c56802fb..add10022 100644 --- a/config.toml +++ b/config.toml @@ -23,7 +23,15 @@ name = " libp2p GitHub" url = "https://github.com/libp2p/libp2p" weight = 2 +[[menu.shortcuts]] +name = " libp2p Specs" +url = "https://github.com/libp2p/specs" +weight = 3 + [[menu.shortcuts]] name = " Discussion Forums" url = "https://discuss.libp2p.io" -weight = 3 +weight = 4 + +[markup.goldmark.renderer] + unsafe = true # Allow HTML in md files diff --git a/content/_index.md b/content/_index.md index ccd1071e..129c8774 100644 --- a/content/_index.md +++ b/content/_index.md @@ -13,3 +13,4 @@ libp2p, want to dive into peer-to-peer concepts and solutions, or are looking fo information, this is the place to start. {{% children description="true" %}} + diff --git a/content/community/_index.md b/content/community/_index.md new file mode 100644 index 00000000..3d6890de --- /dev/null +++ b/content/community/_index.md @@ -0,0 +1,12 @@ +--- +title: "Community" +weight: 5 +pre: ' ' +chapter: true +--- + +# Get Involved with libp2p + +Find out how to get involved with libp2p and join the community. + +{{% children description="true"%}} diff --git a/content/get-involved/community.md b/content/community/community.md similarity index 87% rename from content/get-involved/community.md rename to content/community/community.md index 51c0454d..3e29577f 100644 --- a/content/get-involved/community.md +++ b/content/community/community.md @@ -1,11 +1,12 @@ --- -title: "Community" -weight: 2 +title: "Get Involved" +weight: 1 pre: ' ' chapter: true +summary: Get involved and participate in the libp2p community. --- -# Community +# Get Involved We love talking about libp2p, and we'd be happy to have you in the mix. diff --git a/content/get-involved/contribute.md b/content/community/contribute.md similarity index 98% rename from content/get-involved/contribute.md rename to content/community/contribute.md index b45e49c3..ed828a86 100644 --- a/content/get-involved/contribute.md +++ b/content/community/contribute.md @@ -1,8 +1,9 @@ --- title: "How to Contribute" -weight: 1 +weight: 2 pre: ' ' chapter: true +summary: Learn how you can contribute to libp2p. --- # How to Contribute diff --git a/content/concepts/_index.md b/content/concepts/_index.md index 2595fb11..89fc8cca 100644 --- a/content/concepts/_index.md +++ b/content/concepts/_index.md @@ -1,28 +1,14 @@ --- title: Core Components weight: 2 -pre: " " +pre: ' ' chapter: true aliases: /reference/specs/ --- -# Explore libp2p +# Learn about libp2p libp2p covers a lot of ground, and may involve unfamiliar terminology and concepts. This chapter goes over the foundational concepts involved in libp2p. -{{% children %}} - -### Specifications & Planning - -While libp2p has several implementations, it is fundamentally a set of protocols for -peer identity, discover, routing, transport, and more. - -See the [specifications repository](https://github.com/libp2p/specs) for the offical libp2p specifications. - -### Implementations - -At the core of libp2p is a set of specifications, which together -form the definition for what libp2p is in the abstract and what makes a "correct" libp2p -implementation. Keep up with the latest implementations on the -[libp2p implementations page](https://libp2p.io/implementations/). +{{% children description="true"%}} diff --git a/content/concepts/introduction/_index.md b/content/concepts/fundamentals/_index.md similarity index 65% rename from content/concepts/introduction/_index.md rename to content/concepts/fundamentals/_index.md index 1301ae90..6dc80b43 100644 --- a/content/concepts/introduction/_index.md +++ b/content/concepts/fundamentals/_index.md @@ -3,6 +3,7 @@ title: "Fundamentals" weight: 1 pre: ' ' chapter: true +summary: Learn about the core pillars that compose each libp2p peer and a libp2p network. --- # Key Components of libp2p diff --git a/content/concepts/introduction/addressing.md b/content/concepts/fundamentals/addressing.md similarity index 100% rename from content/concepts/introduction/addressing.md rename to content/concepts/fundamentals/addressing.md diff --git a/content/concepts/introduction/peers.md b/content/concepts/fundamentals/peers.md similarity index 100% rename from content/concepts/introduction/peers.md rename to content/concepts/fundamentals/peers.md diff --git a/content/concepts/introduction/protocols.md b/content/concepts/fundamentals/protocols.md similarity index 100% rename from content/concepts/introduction/protocols.md rename to content/concepts/fundamentals/protocols.md diff --git a/content/concepts/glossary.md b/content/concepts/glossary.md index f2d062f3..858555d8 100644 --- a/content/concepts/glossary.md +++ b/content/concepts/glossary.md @@ -3,6 +3,7 @@ title: "Glossary" weight: 12 pre: ' ' chapter: true +summary: A compiled list of words, phrases, and abbreviations for libp2p. --- # Glossary diff --git a/content/concepts/multiplex/_index.md b/content/concepts/multiplex/_index.md index 8e49f1dd..cc4f1635 100644 --- a/content/concepts/multiplex/_index.md +++ b/content/concepts/multiplex/_index.md @@ -4,6 +4,7 @@ weight: 4 pre: ' ' chapter: true aliases: /concepts/stream-multiplexing/ +summary: Stream Multiplexing is a way of sending multiple streams of data over one communication link. It combines multiple signals into one unified signal so it can be transported 'over the wires', then it is demulitiplexed so it can be output and used by separate applications. --- # Stream bytes diff --git a/content/concepts/multiplex/mplex.md b/content/concepts/multiplex/mplex.md index c4df8e94..3b61864d 100644 --- a/content/concepts/multiplex/mplex.md +++ b/content/concepts/multiplex/mplex.md @@ -3,7 +3,7 @@ title: "mplex" weight: 2 pre: ' ' chapter: true -summary: Stream Multiplexing is a way of sending multiple streams of data over one communication link. It combines multiple signals into one unified signal so it can be transported 'over the wires', then it is demulitiplexed so it can be output and used by separate applications. +summary: mplex is a protocol developed for libp2p. --- # mplex diff --git a/content/concepts/multiplex/quic.md b/content/concepts/multiplex/quic.md index 844c8b91..4aaaedc2 100644 --- a/content/concepts/multiplex/quic.md +++ b/content/concepts/multiplex/quic.md @@ -3,7 +3,7 @@ title: "QUIC" weight: 4 pre: ' ' chapter: true -summary: Stream Multiplexing is a way of sending multiple streams of data over one communication link. It combines multiple signals into one unified signal so it can be transported 'over the wires', then it is demulitiplexed so it can be output and used by separate applications. +summary: QUIC is a transport protocol that contains a native stream multiplexer. --- # QUIC diff --git a/content/concepts/multiplex/switch.md b/content/concepts/multiplex/switch.md index 09457315..e43ff067 100644 --- a/content/concepts/multiplex/switch.md +++ b/content/concepts/multiplex/switch.md @@ -3,7 +3,7 @@ title: "Switch" weight: 1 pre: ' ' chapter: true -summary: Stream Multiplexing is a way of sending multiple streams of data over one communication link. It combines multiple signals into one unified signal so it can be transported 'over the wires', then it is demulitiplexed so it can be output and used by separate applications. +summary: libp2p maintains some state about known peers and existing connections in a component known as the switch --- # Switch diff --git a/content/concepts/multiplex/yamux.md b/content/concepts/multiplex/yamux.md index 57c23c85..c2bef13c 100644 --- a/content/concepts/multiplex/yamux.md +++ b/content/concepts/multiplex/yamux.md @@ -3,7 +3,7 @@ title: "yamux" weight: 3 pre: ' ' chapter: true -summary: Stream Multiplexing is a way of sending multiple streams of data over one communication link. It combines multiple signals into one unified signal so it can be transported 'over the wires', then it is demulitiplexed so it can be output and used by separate applications. +summary: yamux is a multiplexing protocol designed by Hashicorp. --- # yamux diff --git a/content/concepts/nat/_index.md b/content/concepts/nat/_index.md index 23b06317..261634a4 100644 --- a/content/concepts/nat/_index.md +++ b/content/concepts/nat/_index.md @@ -3,6 +3,7 @@ title: "NAT Traversal" weight: 5 pre: ' ' chapter: true +summary: We want libp2p applications to run everywhere, not just in data centers or on machines with stable public IP addresses. Learn about the main approaches to NAT traversal available in libp2p. --- # Traverse Bytes diff --git a/content/concepts/nat/autonat.md b/content/concepts/nat/autonat.md index 9c3ad175..3696f95f 100644 --- a/content/concepts/nat/autonat.md +++ b/content/concepts/nat/autonat.md @@ -3,7 +3,7 @@ title: "AutoNAT" weight: 2 pre: ' ' chapter: true -summary: The internet is composed of countless networks, bound together into shared address spaces by foundational transport protocols. As traffic moves between network boundaries, it's very common for a process called Network Address Translation to occur. Network Address Translation (NAT) maps an address from one address space to another. +summary: AutoNAT lets peers request dial-backs from peers providing the AutoNAT service. --- # AutoNAT diff --git a/content/concepts/nat/circuit-relay.md b/content/concepts/nat/circuit-relay.md index 1ce0b8e7..a5683c2e 100644 --- a/content/concepts/nat/circuit-relay.md +++ b/content/concepts/nat/circuit-relay.md @@ -3,7 +3,7 @@ title: "Circuit Relay" weight: 3 pre: ' ' chapter: true -summary: Circuit relay is a transport protocol that routes traffic between two peers over a third-party "relay" peer. In many cases, peers will be unable to traverse their NAT and/or firewall in a way that makes them publicly accessible. Or, they may not share common transport protocols that would allow them to communicate directly. +summary: Circuit relay is a transport protocol that routes traffic between two peers over a third-party "relay" peer. --- # Circuit Relay diff --git a/content/concepts/nat/hole-punching.md b/content/concepts/nat/hole-punching.md index c7a7d2a3..fdc49c20 100644 --- a/content/concepts/nat/hole-punching.md +++ b/content/concepts/nat/hole-punching.md @@ -8,12 +8,13 @@ summary: The internet is composed of countless networks, bound together into sha # Hole Punching + + When an internal machine "dials out" and makes a connection to a public address, the router will map a public port to the internal IP address to use for the connection. In some cases, the router will also accept *incoming* connections on that port and route them to the same internal IP. libp2p will try to take advantage of this behavior when using IP-backed transports by using the same port for both dialing and listening, using a socket option called [`SO_REUSEPORT`](https://lwn.net/Articles/542629/). If our peer is in a favorable network environment, they will be able to make an outgoing connection and get a publicly-reachable listening port "for free," but they might never know it. Unfortunately, there's no way for the dialing program to discover what port was assigned to the connection on its own. - However, an external peer can tell us what address they observed us on. We can then take that address and advertise it to other peers in our [peer routing network](/concepts/peer-routing/) to let them know where to find us. This basic premise of peers informing each other of their observed addresses is the foundation of [STUN][wiki_stun] (Session Traversal Utilities for NAT), which [describes][rfc_stun] a client / server protocol for discovering publicly reachable IP address and port combinations. @@ -21,5 +22,4 @@ This basic premise of peers informing each other of their observed addresses is One of libp2p's core protocols is the [identify protocol][spec_identify], which allows one peer to ask another for some identifying information. When sending over their [public key](/concepts/peers/) and some other useful information, the peer being identified includes the set of addresses that it has observed for the peer asking the question. This external discovery mechanism serves the same role as STUN, but without the need for a set of "STUN servers". - -The identify protocol allows some peers to communicate across NATs that would otherwise be impenetrable. \ No newline at end of file +The identify protocol allows some peers to communicate across NATs that would otherwise be impenetrable. diff --git a/content/concepts/publish-subscribe.md b/content/concepts/publish-subscribe.md index 99ae7774..897039bf 100644 --- a/content/concepts/publish-subscribe.md +++ b/content/concepts/publish-subscribe.md @@ -3,6 +3,7 @@ title: "Publish/Subscribe" weight: 8 pre: ' ' chapter: true +summary: Publish/Subscribe is a system where peers congregate around topics they are interested in. Peers interested in a topic are said to be subscribed to that topic. Learn about how peers can message data in libp2p. --- # Message Data @@ -13,7 +14,7 @@ topic: ![Diagram showing a shaded area with curved outline representing a topic. Scattered within the shaded area are twelve dots representing peers. A label -points to the dots which reads “Peers subscribed to topic”.](subscribed_peers.png) +points to the dots which reads “Peers subscribed to topic”.](../assets/publish-subscribe/subscribed_peers.png) Peers can send messages to topics. Each message gets delivered to all peers subscribed to the topic: @@ -23,7 +24,7 @@ panel are scattered dots within a shaded area representing peers subscribed to a topic. From one of the dots comes a speech bubble labeled with “Message”. In the second panel all dots now have a copy of the speech bubble above them representing that the message has been transmitted to all peers subscribed to -the topic.](message_delivered_to_all.png) +the topic.](../assets/publish-subscribe/message_delivered_to_all.png) Example uses of pub/sub: @@ -89,7 +90,7 @@ labelled “Full-message peering” connect all the dots in a loose mesh, formin many triangles and polygons. Between these lines runs a dense mesh of thinner, lighter lines labelled “Metadata-only peering”. These lines run from each dot to almost every other dot around it, criss-crossing over each other -frequently.](types_of_peering.png) +frequently.]../assets/publish-subscribe/types_of_peering.png) ### Full-message @@ -115,7 +116,7 @@ different dot has only two lines running from it and is labelled “Peer reached lower bound”. Beneath the diagram is a legend reading “Network peering degree = 3; Upper bound = 4; Lower bound = 2“ accompanied with small symbols showing dots with three, four and two lines running from them -respectively.](full_message_network.png) +respectively.](../assets/publish-subscribe/full_message_network.png)

Throughout this guide, numbers highlighted in purple can be configured @@ -147,7 +148,7 @@ performs functions to help maintain the network of full-message peerings. ![Diagram showing a large shaded area with scattered dots inside connected by many thin, light lines representing metadata-only peerings between peers. The lines between the dots are labelled “Each peering is a network connection -between two peers”.](metadata_only_network.png) +between two peers”.](../assets/publish-subscribe/metadata_only_network.png) ## Grafting and pruning @@ -168,7 +169,7 @@ on the right. In the following step, the line becomes thick, dark and is now labelled “Full-message peering”. The second process is the reverse of the first process; two dots are connected by a thick, dark line which becomes a thin, light line labelled “Metadata-only peering”. The speech bubble reads “I’m -pruning our connection back to a metadata-only peering.”](graft_prune.png) +pruning our connection back to a metadata-only peering.”](../assets/publish-subscribe/graft_prune.png) When a peer has too few full-message peerings it will randomly graft some of its metadata-only peerings to become full-message peerings: @@ -187,7 +188,7 @@ now highlighted lines. Four circles in the series are also highlighted green, up to the circle labelled “Ideal”. The panel is titled “Select more peers to graft to get to the ideal number”. In the final panel the highlighted green lines and dots have become dark to indicate they have become full-content peerings. The -title reads “Grafting complete”.](maintain_graft.png) +title reads “Grafting complete”.](../assets/publish-subscribe/maintain_graft.png) Conversely, when a peer has too many full-message peerings it will randomly prune some of them back to metadata-only: @@ -208,7 +209,7 @@ series are also highlighted pink, from the end down to the circle labelled “Ideal”. The panel is titled “Select peers to prune to get the ideal number”. In the final panel the highlighted pink lines and dots have become light to indicate they have become metadata-only peerings. The title reads “Pruning -complete”.](maintain_prune.png) +complete”.](../assets/publish-subscribe/maintain_prune.png) In libp2p’s implementation, each peer performs a series of checks every 1 second. These checks are called the @@ -229,7 +230,7 @@ from “Topic 1” to “Topic 5”. The shaded areas fade out with distance fro large, central dot indicating that the peer’s perspective has limited range. One of the smaller dots that does not share a shaded area with the large dot is labelled “Peers keep track of other’s subscriptions even if not subscribed to -the same topics as them”.](subscriptions_local_view.png) +the same topics as them”.](../assets/publish-subscribe/subscriptions_local_view.png) Keeping track of subscriptions happens by sending **subscribe** and **unsubscribe** messages. When a new connection is established between two peers @@ -239,7 +240,7 @@ they start by sending each other the list of topics they are subscribed to: speech bubble emanating from it with an arrow showing it moving along the connecting line towards the other dot. The left dot’s speech bubble says “I am subscribed to: Topic 1, Topic 2, Topic 3”. The right dot’s speech bubble says “I -am subscribed to: Topic 1, Topic 5”.](subscription_list_first_connect.png) +am subscribed to: Topic 1, Topic 5”.](../assets/publish-subscribe/subscription_list_first_connect.png) Then over time, whenever a peer subscribes or unsubscribes from a topic, it will send each of its peers a subscribe or unsubscribe message. These messages are @@ -249,7 +250,7 @@ subscribed to the topic in question: ![Diagram showing two dots connected by a thin, light line. The left dot has a speech bubble emanating from it with an arrow showing it moving along the connecting line towards the right dot. The speech bubble says “I am -subscribed to: Topic 5. I am unsubscribed from: Topic 2, Topic 3.”](subscription_list_change.png) +subscribed to: Topic 5. I am unsubscribed from: Topic 2, Topic 3.”](../assets/publish-subscribe/subscription_list_change.png) Subscribe and unsubscribe messages go hand-in-hand with graft and prune messages. When a peer subscribes to a topic it will pick some peers that will @@ -266,7 +267,7 @@ subscribed to Topic 3. Also, I’m grafting our connection into a full-message peering.” The next panel shows the same three dots, however the central dot is now inside the shaded area labelled “Topic 3” and the line connecting the central and right dots has become thick and dark indicating a full-message -peering.](subscribe_graft.png) +peering.](../assets/publish-subscribe/subscribe_graft.png) When a peer unsubscribes from a topic it will notify its full-message peers that their connection has been pruned at the same time as sending their unsubscribe @@ -285,7 +286,7 @@ pruning our connection back to a metadata-only peering.” The next panel shows the same three dots, however the central dot is no longer inside the area labelled “Topic 3” and the line connecting the central and right dots has become thin and light like the left line to indicate a metadata-only -peering.](unsubscribe_prune.png) +peering.](../assets/publish-subscribe/unsubscribe_prune.png) ## Sending messages @@ -297,7 +298,7 @@ panel is titled “Peer creates new message of its own”. A small, unlabelled speech bubble emanates from a dot. The dot has four thick, dark lines radiating outward from it. The second panel is titled “Message sent to all other full-message peers”. It shows four copies of the speech bubble now moving away -from the dot along each of the four lines.](full_message_send.png) +from the dot along each of the four lines.](../assets/publish-subscribe/full_message_send.png) Similarly, when a peer receives a new message from another peer, it stores the message and forwards a copy to all other full-message peers it is connected to: @@ -310,7 +311,7 @@ thick, dark lines radiating outward from the dot. The second panel is titled outward. The speech bubble is now centred above the dot. The final panel is titled “Message forwarded to all other full-message peers”. It shows three copies of the speech bubble now moving away from the dot along the three lines -that the speech bubble has not appeared on yet.](full_message_forward.png) +that the speech bubble has not appeared on yet.](../assets/publish-subscribe/full_message_forward.png) In the [gossipsub specification](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md#controlling-the-flood), peers are also known as *routers* because of this function they have in routing @@ -345,7 +346,7 @@ from the central dot. The six speech bubbles are identical and read “I have seen:” followed by three small speech bubble symbols inside the larger speech bubble, in different shades of purple to indicate three different messages. One of the small purple speech bubbles is labelled “Seen messages specify the -sender and sequence number, but not the full message contents”.](gossip_deliver.png) +sender and sequence number, but not the full message contents”.](../assets/publish-subscribe/gossip_deliver.png) Gossiping gives peers a chance to notice in case they missed a message on the full-message network. If a peer notices it is repeatedly missing messages then @@ -382,7 +383,7 @@ now a purple speech bubble travelling along the line connecting the two dots from left to right. The final panel is titled “Newly received message is broadcast to full-content peers”. There are now three copies of the purple speech bubble travelling outwards from the right dot along the three thick, dark -lines connected to it.](request_gossiped_message.png) +lines connected to it.](../assets/publish-subscribe/request_gossiped_message.png) In the [gossipsub specification](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md#control-messages), gossip announcing recently seen messages are called *IHAVE* messages and @@ -412,7 +413,7 @@ connecting the outside dot to dots inside the shaded area have become blue and now have arrowheads at the end pointing towards the peer inside the shaded area they are connected to. These lines represent fan-out peerings. A dice symbol is present indicating random selection of which metadata-only peerings became -fan-out peerings.](fanout_initial_pick.png) +fan-out peerings.](../assets/publish-subscribe/fanout_initial_pick.png) Unlike the other types of peering, fan-out peerings are unidirectional; they always point from the peer outside the topic to a peer subscribed to the topic. @@ -432,7 +433,7 @@ peers subscribed to the topic. The second panel is titled “Once inside the topic, the message is forwarded to all other subscribers as usual”. The purple speech bubbles have moved from outside the shaded area to inside out. Now there is a copy of the speech bubble above every dot in the shaded -area.](fanout_message_send.png) +area.](../assets/publish-subscribe/fanout_message_send.png) If the sender goes to send a message but notices some of their fan-out peers went away since last time, they will randomly select additional fan-out peers @@ -455,7 +456,7 @@ peer that was previously outside the shaded area is now inside it, representing that it is now subscribed to the topic. The three previously-blue arrows have become thick, dark lines representing former fan-out peerings becoming full-message peerings. The other lines from the dot are still thin and light as -before.](fanout_grafting_preference.png) +before.](../assets/publish-subscribe/fanout_grafting_preference.png) After 2 minutes of not sending any messages to a topic, all the fan-out peers for that topic are forgotten: @@ -468,7 +469,7 @@ minutes…” There is a stopwatch above the single dot indicating the passage o time. The second panel is titled “All fan-out peerings revert to metadata-only peerings”. In this panel the three previously blue, arrowed lines connecting the single dot to dots inside the shaded area have become thin and light, -representing the peer’s fan-out peerings becoming metadata-only peerings.](fanout_forget.png) +representing the peer’s fan-out peerings becoming metadata-only peerings.](../assets/publish-subscribe/fanout_forget.png) ## Network packets @@ -512,7 +513,7 @@ columns, similar to the previous table. The first column is titled “For this topic…”, however the second column is titled “You have been…” with the choice of grafted or pruned. There are three rows in this table. Two of the topics have been grafted and one has been pruned (no particular connection to the -previous table).](network_packet_structure.png) +previous table).](../assets/publish-subscribe/network_packet_structure.png) See the [specification](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md#protobuf) for the exact [Protocol Buffers](https://developers.google.com/protocol-buffers) schema used to encode network packets. @@ -565,7 +566,7 @@ the full message column is empty. These rows only have the sender (Peer ID), sequence number and time first seen columns populated. The table rows are listed in order of time first seen, from 1 second ago in the top row to 90 seconds ago in the bottom row. Some of the sequence numbers are shared between -messages, but only where the sender is different.](state.png) +messages, but only where the sender is different.](../assets/publish-subscribe/state.png) ## More information diff --git a/content/concepts/secure-comm/_index.md b/content/concepts/secure-comm/_index.md index 776b5950..f2ede80c 100644 --- a/content/concepts/secure-comm/_index.md +++ b/content/concepts/secure-comm/_index.md @@ -3,16 +3,15 @@ title: "Secure Communication" weight: 3 pre: ' ' chapter: true +summary: Before two peers can transmit data, the communication channel they established with a transport protocol should be secure. Learn about secure channels in libp2p. --- # Secure bytes Before two peers can transmit data, the communication channel they established -with a transport protocol should be secure. Each peer must also be able to open -multiple independent communication streams over a single channel. A transport -protocol like QUIC provides these guarantees out-of-the-box, but other transports -in libp2p do not provide the logic to secure their channel. -This requires an upgrade to the transport using an upgrader. +with a transport protocol should be secure. A transport protocol like QUIC provides +security guarantees out-of-the-box, but other transports in libp2p do not provide the +logic to secure their channel. This requires an upgrade to the transport using an upgrader. Security is always established first over the raw connection. {{% notice "info" %}} diff --git a/content/concepts/secure-comm/overview.md b/content/concepts/secure-comm/overview.md deleted file mode 100644 index 211e1c7d..00000000 --- a/content/concepts/secure-comm/overview.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -title: Overview -weight: 1 -pre: ' 2.2.1. ' -chapter: true ---- - -### Chapter 2.2.1 - -# Overview - -{{% notice "note" %}} -Coming soon! -{{% /notice %}} diff --git a/content/concepts/security/security-considerations.md b/content/concepts/security/security-considerations.md index 0579fcba..1564f614 100644 --- a/content/concepts/security/security-considerations.md +++ b/content/concepts/security/security-considerations.md @@ -3,6 +3,7 @@ title: "Security Considerations" weight: 10 pre: ' ' chapter: true +summary: libp2p makes it simple to establish encrypted, authenticated communication channels between two peers, but there are other important security issues to consider when building robust peer-to-peer systems. --- # Secure Data diff --git a/content/concepts/transports/_index.md b/content/concepts/transports/_index.md index 66a0fec9..400096f4 100644 --- a/content/concepts/transports/_index.md +++ b/content/concepts/transports/_index.md @@ -3,6 +3,7 @@ title: "Transports" weight: 2 pre: ' ' chapter: true +summary: The foundational protocols that move bits around are called transports, and one of libp2p's core requirements is to be transport agnostic. Learn about the transport protocols in libp2p. --- # Exchange Bytes diff --git a/content/get-involved/_index.md b/content/get-involved/_index.md deleted file mode 100644 index 421658d9..00000000 --- a/content/get-involved/_index.md +++ /dev/null @@ -1,18 +0,0 @@ ---- -title: "Get Involved" -weight: 5 -pre: " " -chapter: true ---- - -# Get Involved with libp2p - -Find out how to get involved with libp2p and join the community. - -{{% children %}} - -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} diff --git a/content/guides/_index.md b/content/guides/_index.md index 3780c9bd..ae3caf98 100644 --- a/content/guides/_index.md +++ b/content/guides/_index.md @@ -1,7 +1,7 @@ --- title: "Guides" weight: 3 -pre: " " +pre: ' ' chapter: true --- diff --git a/content/introduction/_index.md b/content/introduction/_index.md index ab4460c2..57c9f233 100644 --- a/content/introduction/_index.md +++ b/content/introduction/_index.md @@ -1,7 +1,7 @@ --- title: "Introduction" weight: 1 -pre: " " +pre: ' ' chapter: true --- diff --git a/content/reference/_index.md b/content/reference/_index.md index ca280559..a473cbad 100644 --- a/content/reference/_index.md +++ b/content/reference/_index.md @@ -1,18 +1,13 @@ --- title: "Reference" weight: 4 -pre: " " +pre: ' ' chapter: true --- # Reference Documentation -Find out how to get involved with libp2p and join the community. +This section contains reference material for libp2p. {{% children description="true" %}} -{{% notice "note" %}} -This section is incomplete, and many of the articles are stubs. To help fill in -the gaps, please see the issues linked in each article to add your input and -help us prioritize the outstanding work. -{{% /notice %}} diff --git a/content/reference/dos-mitigation.md b/content/reference/dos-mitigation.md index 7421ca45..fb6a40a1 100644 --- a/content/reference/dos-mitigation.md +++ b/content/reference/dos-mitigation.md @@ -3,11 +3,12 @@ title: "DoS Mitigation" weight: 2 pre: ' ' chapter: true +summary: DoS mitigation is an essential part of any peer-to-peer application. Learn how to design protocols to be resilient to malicious peers. --- # DoS Mitigation -DOS mitigation is an essential part of any P2P application. We need to design +DoS mitigation is an essential part of any peer-to-peer application. We need to design our protocols to be resilient to malicious peers. We need to monitor our application for signs of suspicious activity or an attack. And we need to be able to respond to an attack. diff --git a/content/reference/specs.md b/content/reference/specs.md index 488609a8..c14c42d4 100644 --- a/content/reference/specs.md +++ b/content/reference/specs.md @@ -3,10 +3,9 @@ title: "Specifications" weight: 1 pre: ' ' chapter: true +summary: Reference the official specifications for libp2p. --- # Technical Specifications Check out the [specs repo](https://github.com/libp2p/specs) for details about the libp2p architecture. - - diff --git a/layouts/partials/renderMarkdown.html b/layouts/partials/renderMarkdown.html new file mode 100644 index 00000000..249d4554 --- /dev/null +++ b/layouts/partials/renderMarkdown.html @@ -0,0 +1,27 @@ + +

+ From f342a2fd708e40a7740b887f644bf7d34be9f322 Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Fri, 14 Oct 2022 19:13:41 -0400 Subject: [PATCH 24/30] update descriptions for primary indexes --- content/community/_index.md | 1 + content/concepts/_index.md | 1 + content/guides/_index.md | 1 + content/introduction/_index.md | 1 + content/reference/_index.md | 1 + 5 files changed, 5 insertions(+) diff --git a/content/community/_index.md b/content/community/_index.md index 3d6890de..f7ab5b1a 100644 --- a/content/community/_index.md +++ b/content/community/_index.md @@ -3,6 +3,7 @@ title: "Community" weight: 5 pre: ' ' chapter: true +summary: Get in touch with other members working on or with libp2p, learn how to contribute, and become an active participant in the libp2p community. --- # Get Involved with libp2p diff --git a/content/concepts/_index.md b/content/concepts/_index.md index 89fc8cca..99a216a5 100644 --- a/content/concepts/_index.md +++ b/content/concepts/_index.md @@ -4,6 +4,7 @@ weight: 2 pre: ' ' chapter: true aliases: /reference/specs/ +summary: libp2p covers a lot of ground, and may involve unfamiliar terminology and concepts. This section goes over the foundational concepts involved in libp2p. --- # Learn about libp2p diff --git a/content/guides/_index.md b/content/guides/_index.md index ae3caf98..ac505c10 100644 --- a/content/guides/_index.md +++ b/content/guides/_index.md @@ -3,6 +3,7 @@ title: "Guides" weight: 3 pre: ' ' chapter: true +summary: Get Started with libp2p examples and learn how to spin up a libp2p node. --- # Get Started with libp2p diff --git a/content/introduction/_index.md b/content/introduction/_index.md index 57c9f233..2f93abc6 100644 --- a/content/introduction/_index.md +++ b/content/introduction/_index.md @@ -3,6 +3,7 @@ title: "Introduction" weight: 1 pre: ' ' chapter: true +summary: libp2p is a modular system of protocols, specifications and libraries that enable the development of peer-to-peer network applications. --- # An Introduction to libp2p diff --git a/content/reference/_index.md b/content/reference/_index.md index a473cbad..9d376237 100644 --- a/content/reference/_index.md +++ b/content/reference/_index.md @@ -3,6 +3,7 @@ title: "Reference" weight: 4 pre: ' ' chapter: true +summary: This section contains in-depth reference material for libp2p. --- # Reference Documentation From 12fa4faa06830125611f71b5b6b653c69b9c1373 Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Fri, 14 Oct 2022 19:47:57 -0400 Subject: [PATCH 25/30] stub notice muxer content --- content/concepts/multiplex/mplex.md | 5 +++++ content/concepts/multiplex/quic.md | 6 ++++++ content/concepts/multiplex/yamux.md | 6 ++++++ 3 files changed, 17 insertions(+) diff --git a/content/concepts/multiplex/mplex.md b/content/concepts/multiplex/mplex.md index 3b61864d..e0d819ad 100644 --- a/content/concepts/multiplex/mplex.md +++ b/content/concepts/multiplex/mplex.md @@ -10,4 +10,9 @@ summary: mplex is a protocol developed for libp2p. mplex is a protocol developed for libp2p. The [spec](https://github.com/libp2p/specs/tree/master/mplex) defines a simple protocol for multiplexing that is widely supported across libp2p language implementations: +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/concepts/multiplex/quic.md b/content/concepts/multiplex/quic.md index 4aaaedc2..f84fa697 100644 --- a/content/concepts/multiplex/quic.md +++ b/content/concepts/multiplex/quic.md @@ -11,3 +11,9 @@ summary: QUIC is a transport protocol that contains a native stream multiplexer. QUIC is a [transport](/concepts/transport/) protocol that contains a "native" stream multiplexer. libp2p will automatically use the native multiplexer for streams using a QUIC transport. View the [QUIC section](/concepts/transport/quic/) to learn about QUIC. + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} diff --git a/content/concepts/multiplex/yamux.md b/content/concepts/multiplex/yamux.md index c2bef13c..dda48e9f 100644 --- a/content/concepts/multiplex/yamux.md +++ b/content/concepts/multiplex/yamux.md @@ -11,3 +11,9 @@ summary: yamux is a multiplexing protocol designed by Hashicorp. [yamux](https://github.com/hashicorp/yamux) is a multiplexing protocol designed by [Hashicorp](https://www.hashicorp.com/). yamux offers more sophisticated flow control than mplex, and can scale to thousands of multiplexed streams over a single connection. + +{{% notice "note" %}} +This section is incomplete, and many of the articles are stubs. To help fill in +the gaps, please see the issues linked in each article to add your input and +help us prioritize the outstanding work. +{{% /notice %}} From cab81a6e7de3ca3a0140a06b02d3c01de7137bb4 Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Mon, 17 Oct 2022 16:19:24 -0400 Subject: [PATCH 26/30] mv dos doc to components + edits --- content/{reference => concepts}/dos-mitigation.md | 3 ++- content/concepts/multiplex/yamux.md | 4 ++-- content/concepts/secure-comm/_index.md | 7 +------ 3 files changed, 5 insertions(+), 9 deletions(-) rename content/{reference => concepts}/dos-mitigation.md (99%) diff --git a/content/reference/dos-mitigation.md b/content/concepts/dos-mitigation.md similarity index 99% rename from content/reference/dos-mitigation.md rename to content/concepts/dos-mitigation.md index fb6a40a1..2dcfc268 100644 --- a/content/reference/dos-mitigation.md +++ b/content/concepts/dos-mitigation.md @@ -1,8 +1,9 @@ --- title: "DoS Mitigation" -weight: 2 +weight: 11 pre: ' ' chapter: true +aliases: /reference/dos-mitigation/ summary: DoS mitigation is an essential part of any peer-to-peer application. Learn how to design protocols to be resilient to malicious peers. --- diff --git a/content/concepts/multiplex/yamux.md b/content/concepts/multiplex/yamux.md index dda48e9f..e695ec3d 100644 --- a/content/concepts/multiplex/yamux.md +++ b/content/concepts/multiplex/yamux.md @@ -9,8 +9,8 @@ summary: yamux is a multiplexing protocol designed by Hashicorp. # yamux [yamux](https://github.com/hashicorp/yamux) is a multiplexing protocol designed by [Hashicorp](https://www.hashicorp.com/). - -yamux offers more sophisticated flow control than mplex, and can scale to thousands of multiplexed streams over a single connection. +yamux offers more sophisticated flow control than mplex, and can scale to thousands of multiplexed streams over a single +connection. {{% notice "note" %}} This section is incomplete, and many of the articles are stubs. To help fill in diff --git a/content/concepts/secure-comm/_index.md b/content/concepts/secure-comm/_index.md index f2ede80c..c9a1d105 100644 --- a/content/concepts/secure-comm/_index.md +++ b/content/concepts/secure-comm/_index.md @@ -12,11 +12,6 @@ Before two peers can transmit data, the communication channel they established with a transport protocol should be secure. A transport protocol like QUIC provides security guarantees out-of-the-box, but other transports in libp2p do not provide the logic to secure their channel. This requires an upgrade to the transport using an upgrader. -Security is always established first over the raw connection. - -{{% notice "info" %}} -Several security protocols are supported in libp2p for encryption, the two primary -ones being Noise and TLS 1.3. -{{% /notice %}} +Security is always established first over the raw connection. {{% children description="true"%}} From a2d2bf86f5f75e879f20ae16e30eb32880f73fc3 Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Mon, 17 Oct 2022 21:30:46 -0400 Subject: [PATCH 27/30] add missing redirect to intro --- content/introduction/_index.md | 1 + 1 file changed, 1 insertion(+) diff --git a/content/introduction/_index.md b/content/introduction/_index.md index 2f93abc6..7353d291 100644 --- a/content/introduction/_index.md +++ b/content/introduction/_index.md @@ -3,6 +3,7 @@ title: "Introduction" weight: 1 pre: ' ' chapter: true +aliases: /introduction/what-is-libp2p/ summary: libp2p is a modular system of protocols, specifications and libraries that enable the development of peer-to-peer network applications. --- From 4ef04c5fe1c05fd2726b8511475014f2b22284ba Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Mon, 17 Oct 2022 21:36:40 -0400 Subject: [PATCH 28/30] add missing examples redirect --- content/guides/_index.md | 1 + 1 file changed, 1 insertion(+) diff --git a/content/guides/_index.md b/content/guides/_index.md index ac505c10..33fd9c6a 100644 --- a/content/guides/_index.md +++ b/content/guides/_index.md @@ -3,6 +3,7 @@ title: "Guides" weight: 3 pre: ' ' chapter: true +aliases: /examples/ summary: Get Started with libp2p examples and learn how to spin up a libp2p node. --- From 8681f36050f45ee2085bff48a4384cb6aead8034 Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Wed, 26 Oct 2022 17:05:52 +0100 Subject: [PATCH 29/30] update frontmatter for hole punching + edit --- content/concepts/hole-punching.md | 13 +++++++------ .../concepts/security/security-considerations.md | 4 ++-- 2 files changed, 9 insertions(+), 8 deletions(-) diff --git a/content/concepts/hole-punching.md b/content/concepts/hole-punching.md index 39dfb54d..4d62c84f 100644 --- a/content/concepts/hole-punching.md +++ b/content/concepts/hole-punching.md @@ -1,11 +1,12 @@ --- -title: Hole Punching -weight: 3 +title: "Hole Punching" +weight: 8 +pre: ' ' +chapter: true +summary: In most configurations, both public and non-public nodes can dial connections to other public nodes. However, it's not possible to establish a connection from the public internet to a non-public node. --- -## Background - -### Types of nodes +# Hole Punching Nodes on a peer-to-peer network can be categorized into two groups: public and non-public. Public nodes are those nodes that have unobstructed @@ -15,7 +16,7 @@ as well as mobile phones. In most configurations, both public and non-public nodes can dial connections to other public nodes. However, it's not possible to establish a connection from the public internet to a non-public node. -### How can a node become dialable despite being behind a firewall and/or NAT? +## Dialing a non-public node Here are a few methods that nodes can use to dial a non-public node: diff --git a/content/concepts/security/security-considerations.md b/content/concepts/security/security-considerations.md index 1564f614..5b2944a1 100644 --- a/content/concepts/security/security-considerations.md +++ b/content/concepts/security/security-considerations.md @@ -8,8 +8,8 @@ summary: libp2p makes it simple to establish encrypted, authenticated communicat # Secure Data -libp2p makes it simple to establish [encrypted, authenticated communication -channels](../secure-comms/) between two peers, but there are other important +libp2p makes it simple to establish encrypted, authenticated communication +channels between two peers, but there are other important security issues to consider when building robust peer-to-peer systems. Many of the issues described here have no known "perfect solution," and the From be8f293d4052fa47fe8750c6726e0e1753b61b2b Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Wed, 26 Oct 2022 17:10:43 +0100 Subject: [PATCH 30/30] one more edit --- content/concepts/dos-mitigation.md | 41 ++++++++++++++++-------------- 1 file changed, 22 insertions(+), 19 deletions(-) diff --git a/content/concepts/dos-mitigation.md b/content/concepts/dos-mitigation.md index 9c5883f8..521e9fac 100644 --- a/content/concepts/dos-mitigation.md +++ b/content/concepts/dos-mitigation.md @@ -7,6 +7,8 @@ aliases: /reference/dos-mitigation/ summary: DoS mitigation is an essential part of any peer-to-peer application. Learn how to design protocols to be resilient to malicious peers. --- +# DoS Mitigation + DoS mitigation is an essential part of any P2P application. We need to design our protocols to be resilient to malicious peers. We need to monitor our application for signs of suspicious activity or an attack. And we need to be @@ -16,25 +18,26 @@ Here we'll cover how we can use libp2p to achieve the above goals. ## Table of contents -- [Table of contents](#table-of-contents) -- [What we mean by a DOS attack](#what-we-mean-by-a-dos-attack) -- [Incorporating DOS mitigation from the start](#incorporating-dos-mitigation-from-the-start) - - [Limit the number of connections your application needs](#limit-the-number-of-connections-your-application-needs) - - [Transient Connections](#transient-connections) - - [Limit the number of concurrent streams per connection your protocol needs](#limit-the-number-of-concurrent-streams-per-connection-your-protocol-needs) - - [Reduce blast radius](#reduce-blast-radius) - - [Fail2ban](#fail2ban) - - [Leverage the resource manager to limit resource usage (go-libp2p only)](#leverage-the-resource-manager-to-limit-resource-usage-go-libp2p-only) - - [Rate limiting incoming connections](#rate-limiting-incoming-connections) -- [Monitoring your application](#monitoring-your-application) -- [Responding to an attack](#responding-to-an-attack) - - [Who’s misbehaving?](#whos-misbehaving) - - [How to block a misbehaving peer](#how-to-block-a-misbehaving-peer) - - [How to automate blocking with fail2ban](#how-to-automate-blocking-with-fail2ban) - - [Example screen recording of fail2ban in action](#example-screen-recording-of-fail2ban-in-action) - - [Setting Up fail2ban](#setting-up-fail2ban) - - [Deny specific peers or create an allow list of trusted peers](#deny-specific-peers-or-create-an-allow-list-of-trusted-peers) -- [Summary](#summary) +- [DoS Mitigation](#dos-mitigation) + - [Table of contents](#table-of-contents) + - [What we mean by a DOS attack](#what-we-mean-by-a-dos-attack) + - [Incorporating DOS mitigation from the start](#incorporating-dos-mitigation-from-the-start) + - [Limit the number of connections your application needs](#limit-the-number-of-connections-your-application-needs) + - [Transient Connections](#transient-connections) + - [Limit the number of concurrent streams per connection your protocol needs](#limit-the-number-of-concurrent-streams-per-connection-your-protocol-needs) + - [Reduce blast radius](#reduce-blast-radius) + - [Fail2ban](#fail2ban) + - [Leverage the resource manager to limit resource usage (go-libp2p only)](#leverage-the-resource-manager-to-limit-resource-usage-go-libp2p-only) + - [Rate limiting incoming connections](#rate-limiting-incoming-connections) + - [Monitoring your application](#monitoring-your-application) + - [Responding to an attack](#responding-to-an-attack) + - [Who’s misbehaving?](#whos-misbehaving) + - [How to block a misbehaving peer](#how-to-block-a-misbehaving-peer) + - [How to automate blocking with fail2ban](#how-to-automate-blocking-with-fail2ban) + - [Example screen recording of fail2ban in action](#example-screen-recording-of-fail2ban-in-action) + - [Setting Up fail2ban](#setting-up-fail2ban) + - [Deny specific peers or create an allow list of trusted peers](#deny-specific-peers-or-create-an-allow-list-of-trusted-peers) + - [Summary](#summary) ## What we mean by a DOS attack