Sprint: May 16 → May 30th

[RichardLitt]

Sprint Discussions

Schedule

Please take notes in a separate pad, if you can, and link it here.

Endeavour	Lead	Time (PDT - UTC/Z - CEST)	Pad
sync	@RichardLitt	9:00PDT 16:00Z 18:00CEST	IRC: #ipfs on Freenode
apps on ipfs	@dignifiedquire	10:30PDT 17:30Z 19:30CEST	https://public.etherpad-mozilla.org/p/ipfs-may-16-apps-on-ipfs
infrastructure	@lgierth	11:00PDT 18:00Z 20:00CEST	https://public.etherpad-mozilla.org/p/ipfs-may-16-infrastructure
js-ipfs task force mega sprint	@diasdavid	11:30PDT 18:30Z 20:30CEST	https://public.etherpad-mozilla.org/p/ipfs-may-16-js-ipfs-task-force
go-ipfs	@whyrusleeping	12:30PDT 19:30Z 21:30CEST	https://public.etherpad-mozilla.org/p/ipfs-may-16-go-ipfs

Please add the Agenda to the Pad before the endeavour sprint starts.

Sprint Deliverables

• Add your tasks below in a comment, that way we only have people listed who are really in the sprint
• Add links to issues down here. Only add things you can finish this sprint.

Timezones are off, fixed in #110

[RichardLitt]

Thank you.

[dignifiedquire]

Apps on IPFS

Lead: @dignifiedquire

Participants

• @haadcode
• @whyrusleeping
• @diasdavid
• @dignifiedquire
• @Kubuxu
• @RichardLitt

Notes

Orbit Update

• write improvements (100 ops/sec)
• Akasha wants to use orbit-db
• Added user auto complete
• Emoji auto complete + preview coming soon
• Next Level: orbit-db in the browser, orbit in the browser, everything in the browser :)

https://ipfs.io/ipns/bin.ipfs.ovh

• Kubuxu if the work on orbit-db would help adding write access to his pastebin style project
• Answer, mostly no but js-ipfs will help by having your own node in the browser connecting to other (potentially browser) nodes and
  sharing the content

ipfs-twitter-resolver

• Storing "ipns records" in a tweet

Demo of gx from whyrusleeping

• deps tree
• pinning site for gx packages

Infrastructure

Lead: @lgierth

Participants

Agenda

• SSL
  • HSTS preloading
  • http=>https redirect
• metrics stability
  • mtail bug
  • flakey cjdns routing
• pages deploys
  • it's in progress, but not yet working :)

Notes

• /version commit needs to be fixed
• whyrusleeping wants to be able to use provsn
  • provsn needs secrets handling
  • provsn needs host grouping
  • provsn needs parallelism (it's sooooo sloooow)
• lgierth badly wants to not be a bottleneck for others (pages && provsn)

@lgierth todo

• js-ipfs crunch
  • deploy the things needed
    • Running orbit infra#165
    • make others happy by having them deploy their own stuff :)
  • p2p_peers_total metric by transport metrics: add transport label to p2p_peers_total kubo#2728
    • create graph http://metrics.ipfs.team/dashboard/db/0verview
  • catch go-ipfs crashes on gateways/bootstrappers
  • changelog for go-ipfs 0.4.2 Add changelog for v0.4.2 kubo#2714
  • update dist.ipfs.io and update.ipfs.io records
  • point install doc on website to 0.4.2
• infrastructure
  • stop being a bottleneck
    • provsn secrets handling
    • provsn parallelism
    • make ssh <page>@pages.ipfs.team publish <hash> work
  • fix /api/v0/version#CurrentCommit
  • fix cjdns routing ipfs/infra@fc651c7
  • figure out http/https redirect logic
  • fix workaround for mtail double-metric bug
• go-ipfs
  • revert ipfs init regression in 0.4.2 Revert "Implements repository initialization with default config" kubo#2749
  • build: specify a dist.ipfs.io hash build: specify a dist.ipfs.io hash kubo#2711
  • add ipns-pub to dist
  • add netbsd build to dist Builds of go-ipfs for NetBSD distributions#64
  • figure out CORS problems
• various
  • deploy weekly Sprint I #10
  • deploy newsletter
  • newsletter content
  • recover fc00 spec drafts from old laptop cryptoauth: draft 2 fc00/spec#2 switch: draft 1 fc00/spec#3
  • move domains to protocol labs account
    • prepare drill list

[dignifiedquire]

@dignifiedquire Todo

• Workshop preparation
• "Introduction to IPFS" Talk preparation
• Help out @diasdavid and @haadcode where I can, with demos and all the things js-ipfs
  • First steps:
    • https://github.com/haadcode/orbit-db/pull/5
    • https://github.com/haadcode/ipfs-log/pull/5

[daviddias]

JS IPFS TASK FORCE MEGA SPRINT

This is a joint hangout to sync in the latest details to get js-ipfs out the door and some really cool examples working on it.

Lead: @diasdavid (David Dias)

Participants

• David
• Richard
• Friedel
• Kuba
• Samuli
• Nathan

Agenda

• Status Update
• What is left
• Assign Tasks and try to come up with ETA
• Make sure we are not missing anything

Notes

last mile

• interface-ipfs-core complete (@noffle) First iteration of interface-ipfs-core ipfs-inactive/interface-js-ipfs-core#18
• WebRTC Transports (@diasdavid)
  • full mesh
  • packet switching
• Migrate websockets multiaddrs to become ws
  • update the spec https://github.com/jbenet/multiaddr/blob/master/protocols.csv
  • update js-multiaddr rename websockets to ws multiformats/js-multiaddr#20
  • update all the modules that us js-multiaddr
• Upgrade js-multistream to the latest multistream spec (@diasdavid) new multistream api multiformats/js-multistream-select#13
• add, cat and get 100% complete
  • core api spec complete + tests WIP Files ipfs-inactive/interface-js-ipfs-core#8
  • add complete (offline + online) + interop with go
    • Adds "files add" http route. js-ipfs#235
  • cat complete (offline + online) + interop with go
  • get complete (offline + online) + interop with go
    • js-ipfs-api get files.get ipfs-inactive/js-ipfs-http-client#271
    • migrate unixfs-engine exporter to use duplex object stream
    • Create an issue with the constructed http-response from get so that we can infer what is wrong (@ngginever)
• js-ipfs interop (bitswap and swarm) with go-ipfs jsipfs compatibility blockers kubo#2738
• js-libp2p and go-libp2p interop tests (@diasdavid)
  • swarm
  • identify
  • update the examples Does this work still? whyrusleeping/libp2p-examples#1
• Orbit on js-ipfs
  • orbit demo 1 - Two orbit-electron nodes connect together through tcp/ws, then a third, browser node, joins and connects to them.
  • orbit demo 2 - Similar to the first one, but with WebRTC enabled, connection as many nodes as you need. If orbit demo 2 is ready, the first one gets kind of reduntant
  • orbit
    • on js-ipfs
    • on js-ipfs browser connected to a js-ipfs Node.js node
    • on js-ipfs browser nodes connected through WebRTC
  • orbit-db
    • get core-ipfs api spec for ipfs cat
  • ipfs-log
    • Working (+ tests) with js-ipfs
• go-ipfs
  • Migrate it to use the right multicodecs defined by spec (@whyrusleeping)
    • Upgrade Protocol Handles to Follow the SPEC kubo#1549
    • update protocol IDs to use versioned strings kubo#2704
  • Upgrade go-multistream to the latest multistream spec (@whyrusleeping) Update ls to be compliant with spec  multiformats/go-multistream#5
  • Websockets support (@whyrusleeping)
    • make multiaddr-net more pluggable multiformats/go-multiaddr-net#15
    • update go-multiaddr add a number to websockets multiformats/go-multiaddr#20 (comment)
• How to use js-ipfs Demos - Create easy to reproduce demos with good visualizations of what is going on:
  • Load an Image from js-ipfs (through blocks)
  • Load a Video from js-ipfs (through blocks) using the MediaSourceAPI Video player with js-ipfs js-ipfs#128
  • Load a modular app (load pieces of the app by clicking "Load this piece" example: Use js-ipfs to load the entire assets of a WebSite/WebApplication js-ipfs#127
• Infrastructure
  • Some bootstrap nodes with Websockets and with disabled encryption (disjoint network) (@lgierth)
  • metrics for peers count by transport (/tcp, /udp/utp, /ws) @lgierth

[Kubuxu]

@Kubuxu todo:

go-ipfs:

• Fix minor bugs:
  • Make ipfs config edit run on client kubo#2718
• Quality of Life changes:
  • Add --format option to mfs stat command kubo#2706
• Features:
  • Offline mode: Offline daemon mode kubo#2696
• Take over AUR go-ipfs-git
  • fix it
  • add ipfs.service user service file
  • bump version

general:

• specs for Unix Socket based IPFS API: IPFS API via Unix sockets notes#129 (I need your input).
• discussion on bloom filter based garbage collection. Bloom frequency table based garbage collector notes#130

[RichardLitt]

go-ipfs

Lead: @whyrusleeping

Participants

• whyrusleeping
• kubuxu
• jbenet
• RichardLitt
• daviddias
• dignifiedquire
• noffle

Agenda

• Unite the Files API 🗂 specs#98
• yamux bug
• multistream name compat
• tests greener?
• issue triage
• bitswap stream reuse

what we need for the demos

tag these with "dev", "solarnet", and/or "release" based on what we NEED and WANT by May 29

(ordered by priority)

• yamux bug fixed (NEED: dev + solarnet, WANT: release)
• websockets in go-ipfs (NEED: dev solarnet, WANT: release)
• multstream name fixes (NEED: dev + solarnet, WANT: release)
• relay in go-ipfs (NEED: dev + solarnet, WANT: release)
• metrics change to monitor peers per transport (NEED: solarnet)
• CORS issue api

You have 30 minutes for this agenda, 5 minutes before the meeting ends, consider moving the remaining items to a github discussion thread so that all the other sprint meetings can start in time.

Notes

After sprint meeting is finished, create the respective action items on the Github sprint issue

[dignifiedquire]

Mid Sprint hangout will be happening now, at 12:10noon EDT, 9:10am PDT, 6:10pm CEST, 4:10pm UTC.

[Kubuxu]

@Kubuxu sprint summary

During this week I worked on resolving some issues, and introducing few QoL changes. I also kept the go-ipfs issues and PRs up to date as @whyrusleeping was on a trip. Most notably the offline mode of go-ipfs (no p2p connections, still provides local content and API) works but might still require few changes. I also moved some go-ipfs dependencies from Godeps to gx, it seems simple at first but is quite a bit of work. I am also thinking about new design of API and API transport that would be focused on low lever applications and high performance.

go-ipfs

• Offline mode
  • fixed issue that showed up last time and added tests
  • review - Offline daemon mode kubo#2696
• Quality of Life changes:
  • Add --format option to mfs stat command - Add --format option to mfs stat command kubo#2706
  • Allow to specify where to put default options in option description - Allow to specify where to put default options in option desciption kubo#2744
• Bugfixes:
  • Make ipfs config edit run on client - Make ipfs config edit run on client kubo#2718
  • Require diag cmds command to run on daemon - Require diag cmds command to run on daemon kubo#2761
  • Make RO API use Gateway's CORS headers - got stuck - https://github.com/ipfs/go-ipfs/tree/feature/GatewayCORS
• Docs:
  • Remove note about Go1.6 not being supported - Remove note about Go1.6 not being supported kubo#2724
  • Use one verbose flag description - Use one verbose flag description kubo#2746
  • Remove conflict of names in the debugging guide - Remove conflict of names in the debugging guide kubo#2752
• gx migrations - Migrating away from Godeps kubo#2722:
  • Wave one - Bunch of migrations from Godeps to gx kubo#2733
  • Wave two - Migrate go.uuid to gx from Godeps kubo#2736
  • Wave three - WIP - Second wave of gx migrations kubo#2757
• Tests:
  • Add test for directly adding hidden file - Add test for directly adding hidden file kubo#2756
• Take over go-ipfs-git on AUR and improve it - https://aur.archlinux.org/cgit/aur.git/log/?h=go-ipfs-git

General

• Discuss UNIX socket based API - IPFS API via Unix sockets notes#129
• Propose bloom frequency table based garbage collector - Bloom frequency table based garbage collector notes#130
• Fix gx-go not working on symlinked workspaces - Add support for symlinks in dvcs-deps whyrusleeping/gx-go#8

[dignifiedquire]

@dignifiedquire sprint summary

The past two weeks were fast paced but exciting. I prepared and gave my first talk on IPFS at my local meetup. In addition to that I nearly finished my implementation of secio in JavaScript. There is just one racing condition fix and some integration work to be done there. And all the rest of my time was spent helping @diasdavid and @haadcode to get js-ipfs in the best shape possible, including an important fix to js-bitswap to ensure blocks are actually distributed to everyone.

---

• "Introduction to IPFS" Talk
  • Prepare talk
  • Give talk
• Help out @diasdavid and @haadcode where I can, with demos and all the things js-ipfs
  • https://github.com/haadcode/orbit-db/pull/5
  • https://github.com/haadcode/ipfs-log/pull/5
  • Fix issues in bitswap: Stronger faster better js-ipfs-bitswap#17
• Secio
  • libp2p-crypto
  • libp2p-secio
  • [~] Integrate into libp2p-swarm: [WIP] Add secio & move to pull-streams libp2p/js-libp2p-switch#67

[daviddias]

David Dias Check In

Summary: This last sprint (2 weeks) has been super intense, having a clear, common goal (shipping orbit on js-ipfs) + getting js-ipfs milestone one ready, has given the team an immense focus on a set of things that were crucial to achieve those goals, making everyone working very closely together and that feeling has been amazing :).

---

Here is a gist of some of the things I've been doing:

• Debug and fix things for go & js IPFS interop with @whyrusleeping. There are still some missing parts, but we know what to do (mostly, move js-ipfs Identify to use length-prefixed-messages as well)
• Made libp2p-webrtc-star, a WebRTC Transport for libp2p, enables Orbit peers to use IPFS just from a browser tab
• Lot's of fixes on spdy, tcp, websockets, identify, duplex streams that were not being wrapped properly, fix identify that was not working on the browser (cause of the missing keys).
• libp2p-ipfs got a face wash and now it is less fragmented between the internals of js-ipfs and libp2p itself
• Lot's of more test coverage
• Havoc tests (blow up processes so that the others stay intact)
• Worked with @ngginever and @noffle on interface-ipfs-core
• Community
  • IPFS talk at require('lx')
  • Prepare talk for WebRebels

[haadcode]

@haadcode sprint end report - 16 May - 29 May

Summary

I've worked on getting Orbit to run in the browser with js-ipfs (https://github.com/haadcode/orbit/tree/js-ipfs). We've been making very good progress and as of couple of days ago, we got it all (Orbit) working fully, with IPFS, in the browser! \o/ It's still brittle so the next steps are to make it more stable and performant. Nevertheless, a huge kudos to the JS team, you've made awesome work so far! Also helped the Akasha project to integrate orbit-db and they wrote a really nice blog post describing what they achieved with it (http://blog.akasha.world/2016/05/26/update-highlights-and-interesting-developments/).