Can pandoc normalise HTML links when converting Markdown to ePub3?

[mattgemmell]

Hello there. I'm using pandoc to convert Markdown (as CommonMark) to ePub3. Because the ePub3 format creates one embedded xhtml file per section of the collated original Markdown document, cross-section links to headings must be rewritten by pandoc in order to remain functional in the resulting ePub file.

Pandoc indeed does this rewriting for links in Markdown format, but doesn't do so for links in HTML format within a Markdown document. Is there any way to cause pandoc to do such rewriting in this circumstance?

A minimal example follows. Consider a Markdown file named test.md with these contents:

# First section

[This link will work in ePubs](#second-section)

<a href="#second-section">This link won't work in ePubs</a>

# Second section

This pandoc invocation will generate an ePub file:

pandoc test.md --from=commonmark_x+smart --to=epub3 --output=test.epub

In the resulting ePub file, the first link (in Markdown format) will work as expected. The second link (in HTML format) will not; in order to work, pandoc would need to rewrite it to target ch002.xhtml#second-section instead. I'm wondering if there's some option/configuration/etc which would cause it to do so.

Thank you for reading.

[rnwst]

Hi Matt,
Thanks for explaining the problem you are facing so clearly. You can see how your example Markdown document is parsed into pandoc's AST by converting to the native format:

$ pandoc test.md -t native
[ Header
    1
    ( "first-section" , [] , [] )
    [ Str "First" , Space , Str "section" ]
, Para
    [ Link
        ( "" , [] , [] )
        [ Str "This"
        , Space
        , Str "link"
        , Space
        , Str "will"
        , Space
        , Str "work"
        , Space
        , Str "in"
        , Space
        , Str "ePubs"
        ]
        ( "#second-section" , "" )
    ]
, Para
    [ RawInline (Format "html") "<a href=\"#second-section\">"
    , Str "This"
    , Space
    , Str "link"
    , Space
    , Str "won\8217t"
    , Space
    , Str "work"
    , Space
    , Str "in"
    , Space
    , Str "ePubs"
    , RawInline (Format "html") "</a>"
    ]
, Header
    1
    ( "second-section" , [] , [] )
    [ Str "Second" , Space , Str "section" ]
]

As you can see, pandoc parses the second link as a RawInline element. The corresponding HTML is then included in the EPUB output verbatim, and no attempt is made to parse or modify it.
If you have no control over the format of the link in the Markdown input, you could try to use a Lua filter to parse the RawInline elements into Links. Let me know if you need help with that.

[rnwst]

You can take inspiration from this filter, which does a similar thing.

[mattgemmell]

Excellent, thank you.

[mattgemmell]

My first time ever writing Lua, but I adapted your filter and this seems to do the trick. Thanks again.

function string.starts(String,Start)
	return string.sub(String,1,string.len(Start))==Start
end

---Parse raw HTML A-tag links into Link AST elements. Modifies Inlines in place.
---@param inlines			  Inlines
local function parse_links(inlines)
	-- Based on https://github.com/rnwst/pandoc-discussions/blob/master/10840/filter.lua
	-- Go back to front to avoid problems with changing indices.
	for i = #inlines, 1, -1 do
		if inlines[i].tag == 'RawInline' and string.starts(inlines[i].text, '<a ') then
			local a_start, a_end, delim, href = string.find(inlines[i].text, '<a%s+[^>]*href=(["\'])([^%1]+)%1[^>]->')
			if a_start then
				for j = i + 1, #inlines, 1 do
					if inlines[j].tag == 'RawInline' and inlines[j].text == '</a>' then
						inlines[i] = pandoc.Link( { table.unpack(inlines, i + 1, j - 1)}, href)
						for _ = i + 1, j, 1 do
							inlines:remove(i + 1)
						end
						break
					end
				end
			end
		end
	end
end

---@param inlines Inlines
---@return Inlines | nil
function Inlines(inlines)
	parse_links(inlines)
	return inlines
end

[rnwst]

Nice work!!

[bpj]

A pattern match inlines[i].text:match('^%<a%s') is probably more efficient than that helper function.