DocFx PostProcessor: Adding manifest items results in html files after 2nd run only.

[Onnno]

Operation System: 'Windows 10'
DocFX Version Used: 2.42.3

Template used: default + 'myCustomTemplate'

Steps to Reproduce:

1. implement Manifest IPostProcessor.Process(Manifest manifest, string outputFolder)
2. in Process(...), create some valid .yml file inside same folder DocFx generates .yml files
3. Also add a ManifestItem for each of those files to the manifest.
4. Execute docfx (rebuild in Visual Studio) -> .yml files are created correctly (also those created by my post processor), but html files are not created for .yml files created by my post processor.
5. Execute docfx 2nd time -> html files are created.

It's also strange that html files even exist before calling my post processor!

Expected Behavior:

• html files are generated when running docfx 1st time.

Actual Behavior:

• html files are generated when running docfx 2nd time.

I tried almost anything I can imagine of. I don't really have any idea left.
I was very happy to get some help to get nested namespaces running (which I could push to github then :-))

[superyyrrzz]

I'd like to first introduce the flow of docfx one run, which may help you understand the behavior:

1. metadata command => output YAML files
2. build command
   1. build YAML files into HTML files
   2. run post processors

You can find that it is not a good idea to generate YAML files in post processor, as it is after the step to generate HTML files. This can also explain why your second run can see these HTML files.

Could you try this workflow?

1. docfx metadata: to generate YAML files
2. run your tool (formally the post processor): to generate your YAML files
3. docfx build: build YAML files altogether into HTML files.

[Onnno]

Hmm, I don't know how to call 'docfx metadata' and 'docfx build' separately since I use docfx as a nuget package. It does not allow for this sequence - maybe if I change the .csproj files manually, it does.'

However, I finished my own solution today: I found that IPostprocessor.prepareMetadata() hook is just the right place in build sequence where yaml files have been generated but html files have not.
So I was able to add my own yml files here successfully. I had to checkout and modify docfx sources as well to fix a bug (changes to metadata in prepareMetadata()) will be overridden by the next postprocessor hook) and to add some code that will check for references to the yml files I added.

Alltogether, I now have nested namespaces :-). I'd like to share this, however, its a pretty hacky solution
(though it works fine) and I had to prepare one or the other thing before releasing it here.
I'd really appreciate to have this more simple...

[superyyrrzz]

Glad you finally make it work. I am curious where you transform the newly generated YAML files into HTML files? In your custom postprocessor? Currently, such transformation always happens before postprocessor in docfx.

I think this line must be the bug you mentioned... PR is welcomed.

docfx/src/Microsoft.DocAsCode.Build.Engine/PostProcessors/PostProcessorsManager.cs

Line 52 in b3d848a

updatedMetadata = postProcessor.Processor.PrepareMetadata(metadata);

[Onnno]

1. Right, that's the bug I mentioned :-)

2. I do not transform the newly generated YAML files into HTML. DocFx does it for me.
   Therefore, I had to add the new .yml files in Postprocessor.prepareMetadata(manifest) which ist called by docfx before transformation to html.
   Additionally, I had to put links to these new files into the manifest parameter of prepareMetadata - and I had to modify docfx sources to read these links and add them to the container holding references to yml files that will be converted to html by docfx in the next step.
   After that, I copy the resulting dll's to the docfx nuget folder (overwriting the old dlls) as a post build step of my postprocessor, finally executing the docfx nuget package by building the docfx main project.
   Pretty hacky, but works nice :-) :-O.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in a week if no further activity occurs.

[Onnno]

Glad you finally make it work. I am curious where you transform the newly generated YAML files into HTML files? In your custom postprocessor? Currently, such transformation always happens before postprocessor in docfx.

I think this line must be the bug you mentioned... PR is welcomed.

docfx/src/Microsoft.DocAsCode.Build.Engine/PostProcessors/PostProcessorsManager.cs

Line 52 in b3d848a

updatedMetadata = postProcessor.Processor.PrepareMetadata(metadata);

Yes, that's it. I just made a PR on it.

[Onnno]

Hello everybody,
I posted the codes here - please read readme!
https://github.com/billbeeio/docfx-namespace-tree
Please make a watch on that project. You can use it when I release it.

It needs some stuff to release it as a nuget package - in the meanwhile you can try to get it run as I described there. If you let me know your problem's doing so, I'll help you.
However, I strongly recommend you to wait for this as a nuget package.
I'm currently waiting for a pull request to docfx necessary to get my plugin running.
Reviewer is pretty active so it shouldn't take too long.