Is there an option or a way to preserve the blank lines at the end?

[felixfeng33]

According to the specification, the blank lines at the end of the document will be removed. However, due to some special use cases (related to AI streaming), I want to preserve it as much as possible.I looked through the related issues but couldn't find an answer. Thanks in advance, and I really appreciate you maintaining such a useful library!

https://spec.commonmark.org/0.29/#example-197

[felixfeng33]

[ChristianMurphy]

Welcome @felixfeng33! 👋
What library in the remark/unified are you using?
Are you trying to output Markdown, HTML, or something else?

related to AI streaming

There is no streaming markdown, incomplete markdown, or invalid markdown.
All inputs are considered a complete and valid document.

I want to preserve it as much as possible.

What are you aiming to achieve by preserving it?
Are you doing formatting markdown to be shown to a user as raw text? Are you rendering it to HTML or another presentation format? Something else?

[felixfeng33]

Thank you for your quick reply.

What library in the remark/unified are you using?

I am using remark-parse to convert a string into an MDAST.

I convert it into the structure of the Slate editor.
The content returned by AI in the stream transmission is indeed incomplete.

Currently, for example, the content returned by ChatGPT is not a complete markdown chunk by chunk.
For example, chunk1 = 'chunk1\n\n' and chunk2 = 'chunk2'. If the \n\n is ignored, I would get chunk1chunk2.

What I expect is:
chunk1
chunk2

Same for chunk1 = 'chun1 ', chunk2='chunk2'
expected ouput 'chunk1 chunk2'
but I get 'chunk1chunk2'
The space after chunk1 has been deleted.

Why I want to execute string => md every time AI returns a response.?
I can't wait for the entire document to be completed before displaying it. I build the syntax tree on each chunk to avoid showing the user markdown marks like **bold in the complete document.

[felixfeng33]

All inputs are considered a complete and valid document.

But I found that as long as I stop removing the newline and spaces at the end of paragraphs, it can basically ensure the correctness during streaming.

if this use case might be too specific to my needs.
Can you point me in the right direction to help me fork the repository and then fixes it?

[remcohaszing]

You can’t treat chunk1\n\n as markdown, process it, and then treat chunk2 as markdown. Markdown doesn’t support streaming. You need to track the markdown document, append every new chunk, and process the whole markdown document every time you want to render it.

[felixfeng33]

Thank you for your suggestion.