Support content "sections"

[lucperkins]

One of the things that I really like about Hugo is its sections feature, which enables you to separate nested directories of content into well-defined groups. So instead of a kind of "flat" array of files you end up getting a nested structure like this:

Documentation
\_ Getting started
  \_ Installation
  \_ Platforms
\_ Administration
  \_ Kubernetes
    \_ Setup
    \_ Configuration
\_ etc

As expected, sections can be infinitely nested and each can have its own metadata. I don't necessarily endorse the way that Hugo does this (you need to sprinkle _index.md files throughout your tree) but I do think that having this kind of rich structure available to you is great for creating tables of content, sidebar navs, and all of that good stuff. Personally, it's the thing that I'm missing the most as I cross the chasm from mostly doing things in Hugo to doing things in JS world.

I could envision being able to do something like this in Contentlayer:

export const Doc = defineDocumentType(() => ({
  name: 'Doc',
  filePathPattern: `**/*.md`,
  fields: {
    title: {
      type: 'string',
      description: 'The title of the doc',
      required: true,
    },
  },
  computedFields: {
    url: {
      type: 'string',
      resolve: (post) => `/docs/${post._raw.flattenedPath}`,
    },
  },
}))

export default makeSource({
  contentDirPath: 'docs',
  documentTypes: [Doc],
  // tiny little change
  sections: true,
})

This could export an allDocsSections object of type Sections<Doc> that would enable you to recurse through the tree.

<ul>
  {allDocs.sections.map(section => (
  <li>
    <p>{section.title}</p>
    
    <ul>
      {section.docs.map(doc => (
        <li>
          {doc.title}
        </li>
      ))}
      
      {section.sections.map(section => ( /* nest */ ))}
    </ul>
  </li>)}
</ul>

This is very off the cuff but hopefully provides some sense of what I'm suggesting. I'd be quite happy to help out with this if others find the idea compelling. Just wanted to register it here in this forum.

Note: Docusaurus also supports this using a _category_.{json|yml} file that creates a new "category" (basically an analogue to the sections concept) wherever it's found in the directory tree.

[seancdavis]

Interesting proposal. This sparked a couple thoughts.

Equivalent Work with Contentlayer

I'm curious what this would look like to build manually, given Contentlayer's current abilities? Would you build it all as a single model and use server-side JS to manipulate it all into a tree of docs? How messy is that?

It could be helpful to see the equivalent code. This could provide insight into what exactly a feature like this should contain and what the shape of the API should be.

Plugin System / Community Plugins

After also commenting on #210, this has me thinking about what a plugin system might look like.

Suppose we didn't build this feature. One way to help others achieve it is with an example. Another is to enable processing plugins that could build objects/relationships with content. Then the community could build self-encapsulating plugins to share. CL then doesn't have to add opinion to its base, but the community could streamline common patterns.

@lucperkins What do you think of this?

@schickling Have you thought about community plugins? Maybe we should open a separate issue to discuss?

[lucperkins]

@seancdavis There may be a way to do this in CL at the moment, but it's not really coming to me. The issue is that it requires not just pulling in documents à la **/*.md but also scanning the directories for config files that define the section logic. So essentially you'd need to use CL to pull in documents, inspect the target directory for config files, and then tie all that information together to recursively create section objects. I think the resulting logic would be sufficiently hairy that you may be better off just not using CL at all. Thus, this proposal I think is much more of a "this isn't really possible now" proposal than #210.

[lucperkins]

@seancdavis To make this more concrete, I threw together an example repo:

https://github.com/lucperkins/contentlayer-sections-example

It's pretty basic but I think that it shows the kind of input/output that I, personally, have relied on pretty heavily when using Hugo across many projects. There maaaaaay be a way to do something like this in CL currently, and I'm all ears if so, but I'm currently not quite seeing it.

This might be a bit inscrutable because of the Go templating, but here's an example of a sidebar that I made (using Hugo, Tailwind, and AlpineJS) that relies fundamentally on Hugo's nested sections feature: https://github.com/vectordotdev/vector/blob/master/website/layouts/partials/docs/sidebar.html.

[seancdavis]

Okay, @lucperkins — I see what you're looking to do here and why it's a hairy process. It's almost like we could open up some sort of lifecycle hooks so that you could use values of some content (i.e. section.yml files) to apply to other documents within the same directory.

There are meta values generated by CL that could help, but yes, definitely a cumbersome task right now.