Support for multiple bibliographies

[Mellich]

Description

For my PhD thesis I would like to have two separate bibliographies in my book/website: One for my own articles and one for other literature. This structure is used quite often in my area. Therefore, I have prepared two .bib files ad I would like to use the usual cite syntax in my .qmd files and generate the two bibliographies at the end of the book. In Latex, this is usually achieved using the multibib package.

I tried to use different pandoc extensions that seem to implement the desired features such as:

• https://github.com/pandoc-ext/multibib
• https://github.com/bcdavasconcelos/citetools

Building the document with each of these extensions leads to unsatsifying results: References not showing up on the website version, citeproc warnings for every reference, overlapping numbers when using numerial citation styles, ...

Does anyone know how such a multibib setup (working for websites and PDF) could be implemented within Quarto? Is there an extension or feature I have overseen?

Best Regards!

[mcanouil]

Did you look at https://github.com/pandoc-ext/section-bibliographies?

[Mellich]

Thank you for the answer.

Yes, I did. It creates bibliographies for each chapter as far as I understand, which is not what I want to do.
I would like to have both bibliographies at the end of the document but in separate listings. Something like:

# Section 1

Cite [1] [4]...

# Section 2

Cite [2] [5]...

# My References

[1] ...
[2] ...
[3] ...

# Other References

[4] ...
[5] ...

[mcanouil]

Then https://github.com/pandoc-ext/multibib is the extension you need.
If you encounter something "not satisfying", you should report that on the extension repository.

Also because there is no small reproducible example, there is not much that can de done to help.
And no information of what Quarto version you are using which could have some conflicts with the extension.

For further help, please provide a reproducible example and in your case, preferably a Git repository we can clone.

[Mellich]

Sure, I put something together here: https://github.com/Mellich/quarto_multibib_minimal_example using Quarto 1.3.450.
Quarto crashes at the end of the document generation with the following traceback:

ERROR: TypeError: bibliography.map is not a function

TypeError: bibliography.map is not a function
    at bookBibliography (file:///Applications/quarto/bin/quarto.js:98657:48)
    at async bookBibliographyPostRender (file:///Applications/quarto/bin/quarto.js:98689:55)
    at async Object.bookPostRender [as postRender] (file:///Applications/quarto/bin/quarto.js:99065:9)
    at async renderProject (file:///Applications/quarto/bin/quarto.js:86998:13)
    at async Command.fn (file:///Applications/quarto/bin/quarto.js:90856:32)
    at async Command.execute (file:///Applications/quarto/bin/quarto.js:8437:13)
    at async quarto (file:///Applications/quarto/bin/quarto.js:127545:5)
    at async file:///Applications/quarto/bin/quarto.js:127563:9

It seems to be related to the book project type. This is why I think it is a Quarto issue. The problem only occurs for the html output.

[cderv]

I don't think the Extension is compatible with Books and / or with Quarto syntax

In your _quarto.yml you get this warning by Quarto linter

bibliography in Quarto is not supposed to be a named list, it is supposed to be a string or list of string. (https://quarto.org/docs/reference/formats/html.html#references)

You deactivate on purpose the validation by setting validate-yaml: false. If you would have let it the default. This is the error you would have get

> quarto preview
ERROR: Project _quarto.yml validation failed.

In file C:\Users\chris\AppData\Local\Temp\quarto\quarto_multibib_minimal_example\_quarto.yaml
(line 24, column 5 through line 28, column 29) Field "bibliography" has value

main: ref1.bib
  own: ref2.bib
...
# citeproc: false
# suppress-bibliography: true

The value must instead be a string.
23: bibliography:
24:     main: ref1.bib
       ~~~~~~~~~~~~~~~
25:     own: ref2.bib
   ~~~~~~~~~~~~~~~~~~
...
27: # citeproc: false
   ~~~~~~~~~~~~~~~~~~
28: # suppress-bibliography: true
   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
x The value

main: ref1.bib
  own: ref2.bib
...
# citeproc: false
# suppress-bibliography: true

is of type object.
i The error happened in location bibliography.

So this is an error because multibib is using a field from Pandoc / Quarto, and using it in a different way that expected. This should be solved by using a dedicated field for the extensions.

It was discussed in

• Strict YAML validation collides with existing lua filters for Pandoc #4139
• Render failed due to invalid YAML (1.2.313) pandoc-ext/multibib#1

Just curious:

@dragonstyle do we / should we have a way to check for correct expected fields content (string or list of strings) before doing some processing ?

quarto-cli/src/project/types/book/book-bibliography.ts

Lines 56 to 81 in 0dfc909

	if (!bibliography) {
	return {
	bibliographyPaths: [],
	};
	}
	// We need to be sure we're properly resolving the bibliography
	// path from the metadata using the path of the file that provided the
	// metadata (the same goes for the CSL file, if present)
	// The relative path to the output file
	const fileRelativePath = relative(
	projectFormatOutputDir(file.format, context, projType),
	file.file,
	);
	// The path to the input file
	const inputfile = await inputFileForOutputFile(context, fileRelativePath);
	const bibliographyPaths: string[] = [];
	let csl = file.format.metadata[kCsl] as string;
	if (inputfile) {
	// Use the dirname from the input file to resolve the bibliography paths
	const firstFileDir = dirname(inputfile.file);
	bibliographyPaths.push(
	...bibliography.map((file) =>
	isAbsolute(file) ? file : join(firstFileDir, file)
	),
	);

Or do we rely on YAML validation for this even for our Typescript code ?

This would allow to have a better error.

[Mellich]

Okay, thank you for the clarification. I fixed the issue above by using dedicated fields for multibib.
However, the html output does not show the references (the divs are empty). PDF output works now as expected.
Any idea what could be the reason for this?

I updated the minimal example accordingly: https://github.com/Mellich/quarto_multibib_minimal_example
You will need to use my patched version of multibib using quarto add mellich/pandoc-multibib.

[cderv]

However, the html output does not show the references (the divs are empty). PDF output works now as expected.
Any idea what could be the reason for this?

I do think this could come from the fact that PDF is rendered a single document, so you adding blocks and refs div yourself from Lua work ok.

However, I don't think Books are built this way. So adding two chapters from Lua for your book is probably not working as you think it would. I don't if you logic for Books allow such interaction. (In HTML results we can see a main chapters in TOC, but no Own chapter which shows something is not right probably).

We would need to experiment to see how this could be done.

@cscheid do you think we have what is necessary to develop such Lua Extensions ?

PDF output works now as expected.

I see

[WARNING] Citeproc: citation c not found
[WARNING] Citeproc: citation a not found
[WARNING] Citeproc: citation b not found

but maybe they come from Quarto trying to run citeproc with no bib really. So maybe running citeproc should be deactivated as you do it yourself. This other filter that plays with bib does deactivate it (https://github.com/pandoc-ext/section-bibliographies#quarto)

[andrewheiss]

Here's an example of how to use multibib with a single document (i.e. not with books) with HTML, PDF, and docx, using Quarto 1.4.x: https://www.andrewheiss.com/blog/2023/12/11/separate-bibliographies-quarto/

The only downside is that it generates warnings for each citation, like @cderv notices at #7642 (reply in thread)

[WARNING] Citeproc: citation Stan-2-26-1 not found
[WARNING] Citeproc: citation rproject-4-3-1 not found
[WARNING] Citeproc: citation Lovelace1842 not found
[WARNING] Citeproc: citation Turing1936 not found

I'm pretty sure that's because of where the filter is getting run within the filter order stack, but moving the location to after-quarto or other places doesn't change anything 🤷‍♂️. But at least it's working!

[cderv]

With Quarto 1.4, there are now more precise way to target where the filter should apply:
https://quarto.org/docs/prerelease/1.4/lua_changes.html#more-precise-targeting-of-ast-processing-phases

Maybe it can help run filter in the right place ?

[cscheid]

The citeproc filter itself is currently run externally to the Lua filter chain, so I don't think that will work (look at the TODO at the end of main.lua :))

[andrewheiss]

Yeah, no matter which location I use (pre/post ast, quarto, render), it still happens

[cscheid]

Tracking for 1.5: #7888