MDXv2 broken with remark-heading-id

[Eyas]

Initial checklist

• I read the support docs
• I read the contributing guide
• I agree to follow the code of conduct
• I searched issues and couldn’t find anything (or linked relevant results below)

Affected packages and versions

2.0.0

Link to runnable example

No response

Steps to reproduce

• Install & import remark-heading-id
• in MDX remarkPlugins, add remark-heading-id
• Try to parse MDX: ## foo {#heading}

Expected behavior

The plugin handles the {#...} extended markup syntax, and MDX parses the document normally

Actual behavior

Get error: Could not parse expression with acorn: Unexpected character '#'

It makes sense that MDX thinks that {...} syntax is inline JS, but I would have thought plugins can be used to support heading ID Markdown extended syntax.

Runtime

Node v16

Package manager

npm v8

OS

Windows

Build and bundle tools

No response

[ChristianMurphy]

https://github.com/imcuttle/remark-heading-id hasn't been updated in several years, and doesn't support the latest version of remark/micromark (imcuttle/remark-heading-id#1) which by extension means it doesn't work with the latest version of mdx.
This can be resolved by opening a PR to remark-heading-id upgrading it to support the new and improved micromark parser.

[Eyas]

Hm. I have a local version of remark-heading-id that uses "unist-util-visit": "^4.1.0" instead (which should use micromark afaict?) and it doesn't seem to work still. Am I missing something?

[ChristianMurphy]

There's a deeper upgrade needed to make it work with MDX.
The current approach, using unist-util-visit works on the AST, after the document has already parsed, which means JSX parsing will always take precedence and cause an issue before the plugin can do anything.
If you want to add a new syntax feature, it needs to be implemented as a part of the parser.
See https://github.com/micromark/micromark#extending-markdown and https://github.com/micromark/micromark#creating-a-micromark-extension for more information.

[Eyas]

Thanks, that helps! One more question: I'm looking at @mdx-js/mdx's CompileOptions -- where would a micromark syntax extension be registered? I assume it's not a regular remark Plugin in this case, right?

[ChristianMurphy]

It would be registered as a plugin, similar to how MDX itself registers itself and extends remark and micromark

mdx/packages/remark-mdx/index.js

Lines 10 to 39 in e527ac7

	import {mdxjs} from 'micromark-extension-mdxjs'
	import {mdxFromMarkdown, mdxToMarkdown} from 'mdast-util-mdx'
	/**
	* Plugin to support MDX (import/exports: `export {x} from 'y'`; expressions:
	* `{1 + 1}`; and JSX: `<Video id={123} />`).
	*
	* @type {import('unified').Plugin<[Options?]|Array<void>, Root>}
	*/
	export default function remarkMdx(options = {}) {
	const data = this.data()
	add('micromarkExtensions', mdxjs(options))
	add('fromMarkdownExtensions', mdxFromMarkdown())
	add('toMarkdownExtensions', mdxToMarkdown(options))
	/**
	* @param {string} field
	* @param {unknown} value
	*/
	function add(field, value) {
	const list = /** @type {Array<unknown>} */ (
	// Other extensions
	/* c8 ignore next 2 */
	data[field] ? data[field] : (data[field] = [])
	)
	list.push(value)
	}
	}

[Eyas]

Thanks for all the pointers! ended up making a separate package: remark-custom-heading-id. The micromark code is substantially different than the unist-util-visit-based approach, and the old extension is still good-enough for non-MDX cases.

[rikusen0335]

I'd say thank you so much @Eyas that repo helped me a lot.

[Danii2020]

Hello, hope you can still help me, I want to use the remark-custom-heading-id to parse custom heading IDs, I'm using the mdxToMarkdown to allow MDX expressions but that function doesn't allow me to use the custom heading id syntax {#custom-id}. I'm using the following function to convert to MDX:

function mdxStringify(this: Processor) {
  const compiler = (tree: Node): string => {
    const markdown = toMarkdown(tree, {
      extensions: [gfmToMarkdown(), mdxToMarkdown()],
      bullet: "-",
      emphasis: "_",
      fences: true,
      resourceLink: true,
    });
    return markdown;
  };

  Object.assign(this, { Compiler: compiler });
}

If I remove the mdxToMarkdown function it handles custom heading ids properly, but I need that function, so is there a way to add a the remark-custom-heading-id as a plugin in that function? Thanks in advance.

[ChristianMurphy]

@Danii2020 welcome! 👋
You don't appear to be using MDX directly?
Also you asked about parsing, but shared your compiler code 🤔
And your question appears to be more related to https://github.com/Eyas/md-heading-id?

Some suggestions on how to proceed:

• Your question appears to go beyond this issue, consider opening a question at https://github.com/Eyas/md-heading-id for how to integrate it into your workflow
• If you do want to ask the broader MDX community, considering opening a discussion https://github.com/orgs/mdx-js/discussions
• Either way, take some additional time framing the question https://mdxjs.com/community/support/ with a focus on what you want to achieve and why
• And consider sharing a runnable example in a codesandbox or stackblitz sandbox.