Different behaviors when generating Markdown links for headings containing punctuations and other symbols

[federico-ntr]

Description

When you use double quotes in a heading, Gitea removes them from the reference link.

Example:

If you use a header with double quotes like:

## Header with "double quotes"

Gitea generated link to that header is <URL>#header-with-double-quotes instead of <URL>#header-with-"double-quotes".
This behaviour leads to inconsistency between renderers, even between those which follow CommonMark specs. For example VSCode's Markdown preview (which follows CommonMark specs) keeps the double quotes in the links.

I reproduced the issue here

Gitea Version

1.16.8

Can you reproduce the bug on the Gitea demo site?

Yes

Log Gist

No response

Screenshots

No response

Git Version

Irrelevant

Operating System

Ubuntu 20.04

How are you running Gitea?

The instance runs on a virtual machine without containerization. I don't know which DB is used since I didn't installed it myself. By the way I don't think it's relevant here.

Database

No response

[silverwind]

Can you push that file to GitHub as well to see how they do it?

[federico-ntr]

Here's the repo. The behaviour seems to be the same.
I guess it's a matter of parser then. VSCode uses markdown-it, while Gitea uses goldmark. Also, I didn't find any reference to this behaviour among CommonMark's specs.

EDIT: I imported the repo from try.gitea.io, I didn't create one from scratch. I don't know if it makes any difference.

[wxiaoguang]

I would say it's more like a feature but not a bug, because Markdown is not a strict system, and there seems no unique standard.

There are various characters would be removed&replaced during URL generation. For example, the single quote ' in your demo file, too.

https://github.com/federico-ntr/gitea-double-quotes-test#placeholder-to-force-scrolling-on-links-click
https://try.gitea.io/federico-ntr/double-quotes-test#placeholder-to-force-scrolling-on-link-s-click

Since there is no standard, so there is no right or wrong, as long as it works.

Maybe the answer to the question could be: if there is a definition in CommonMark, then make upstream goldmark use CommonMark standard.

[federico-ntr]

I tried to search on CommonMark and I didn't find anything related to this matter. I think it's a free different choice made by goldmark and markdown-it.

I'll format my links according to what goldmark does, since it's more important for me that they are working on Gitea rather than on VSCode md preview.

[wxiaoguang]

Thank you for your confirmation. Since there is no standard, I think we can close this issue at the moment, until there are some standards in the future (I believe that on that day, there will be new issues for it)