Provide clear, reliable, up-to-date roadmaps and information about statuses of features

[Mr0grog]

Goal: Help developers understand the status of concepts/technologies/approaches in IPFS. Let them know which things they should be using, what's happening, what's moving forward, what's a placholder for future work, etc. -- less emphasis on "is it done yet?", more emphasis on what you should use in which ways going forward.

The most consistent (and insistent) issue I heard from folks developing or attempting to develop software on top of IPFS was that they often can’t figure the status of various ideas, technologies, or approaches used (or going to be used) within the IPFS ecosystem. They are worried they shouldn’t be building on top of a particular API or protocol because it’s getting replaced or that they should be using it in a particular way (which way, they don't know) in order to best prepare for upcoming changes. Should they just build now and not worry about it, or hold off for something that's coming soon?

It's worth noting that they all said this at the same time as saying that better API documentation would be nice, but they've managed to figure a lot out by reading source code and puzzling around things. But understanding the direction of various parts of IPFS is something much more important that they can’t figure out that way.

Getting a handle on implementation status (as discussed in #35) is only a small part of this issue. The questions here are more like:

• What’s the deal and status with IPNS being slow? Should I even use it now? I know there’s some discussion about re-implementing it on top of libp2p pubsub — when is that happening? (IS that really happening?) How should I make use of things differently (or not) so my code functions optimally in future where that's how IPNS works?

• What's the deal with IPNS vs. IPRS? Is the spec final or still subject to change? The IPRS spec was written a year ago, but it doesn’t seem like things are using it? Is it still a thing or is it left behind? When will it be a concrete thing?

• What’s the deal with IPLD? Is it being used for the all (or any and which?) of the commands in go-ipfs or js-ipfs? If no, when? What’s solid about it and what’s not, and when (specifically or even just roughly) do we think it will be all solid? Should I be using it today?

• Similar questions for CIDs, for files vs. objects vs. dag, pretty much everything in ipfs/specs and ipld/specs. (Less so for libp2p/specs, which folks have said seems largely more stable and clear, and multiformats/specs, which, even though that repo is really uninformative, Multiformats’ other constituent are quite clear.)

• Hypothetical confused developer: "I'm so confused about these things that I don't know which one I should be using. Which ones do the maintainers really want me to use? Which ones do the maintainers specifically discourage using? Which things should I watch for in the coming months, etc"

Some Approaches

So what are the right approaches to solving this issue and how do we manage that work?

• A clear and updated Roadmap that focuses on these sorts of technology/usage issues rather than implementation, like the Roadmap to 1.0.0 does.

• Clear indicators in every spec and major repo (maybe on each major API section?) that accurately cover the current, up-to-date status. Many have no status indicators, others feel inaccurate — it might say something is a solid draft, but in reality, there are big discussions in the issues for it that indicate it’s not so solid at all, e.g. IPLD. These should be updated if major discussion on revising the spec begins somwhere (and, ideally, point to that discussion).

• What else?

[Mr0grog]

This is somewhat related to #12.

[Mr0grog]

Another way to articulate this: “where’s the effort, and what’s real, what’s not-yet-real (how soon?), and what’s been dropped?”

@mikeal I want to make sure you’ve seen this issue. General improvements in our docs and making sure we are on top of fixing #54 (being clear about deprecation/repo status) and that we stay on top of it will make the need to this kind of roadmap or status doc less acute.

• Do you have good thoughts about how to address this otherwise?
• Is it just not something we have the bandwidth to address directly and keep up-to-date? (i.e. should we close this and just focus on all the other docs & community improvements?)

[mikeal]

I actually have mixed feelings on roadmaps. They are good guides for users to know where the project is heading, and are certainly useful in the aligning our own resources to prioritize the problems we're tackling.

But for casual contributors and people who are considering contributing outside of PL, they're going to have their own thoughts on what they think is important and will want to work towards those goals. We don't really get to tell them what is important and we need to make sure we're open to contributions outside of our roadmap if we want to continue to encourage new contributors to be involved in the project.

To summarize, I think that the roadmap is most useful in helping align PL resources. But because we have OKR's and a fair amount of communication it seems like we're aligned well enough even as the roadmaps fall out of date.

For users, a roadmap would be nice, but I'd put its importance well below the other documentation work you've surfaced.

[Mr0grog]

But for casual contributors and people who are considering contributing outside of PL, they're going to have their own thoughts on what they think is important and will want to work towards those goals. We don't really get to tell them what is important and we need to make sure we're open to contributions outside of our roadmap

Oh, totally! This wasn’t about that at all. Sorry if I made it seem that way.

This is about communicating from the core team out as to what the core team is thinking and focusing on — and not about setting direction. For an external user, this is answering questions about what parts they should or shouldn’t be building on top of or waiting on. For a contributor, this helps explain that something might be going away before they dive in to fix some aspect of it, thinking that will be valuable. That information problem (whatever the right solution is) was one of the clear needs expressed in the research. People would look at the implementation status and roadmap docs for this info, but not find it, or find that reality didn’t match it.

I think your project lifecycle work in ipfs/community#332 gets at a lot of the same issues this is about, but in a more per-project way rather than a big-picture roadmap. It might actually solve the problem just as well. If so, maybe we should close this in favor of that. (This issue predates your presence here by several months!)

[Mr0grog]

Maybe this also helps explain why I focused a lot on delivering explicit clarity about more things in your lifecycle PR ;)