Intersphinx support in toctrees

[xmo-odoo]

Currently, toctree links can be either internal documents or arbitrary external URLs.

For Sphinx projects which serve as hub of sorts, or to cross-TOC between related projects, it'd be nice if it were possible for toctree to use intersphinx when references are not found locally (and are not URLs)

[andreacassioli]

That would be nice. I agree.

[erget]

I just ran up against this wall myself - this would be a great feature!

[moorchegue]

+1!

[mniak]

+2!

[Tzaphkiel]

interrested as well !

[curtisma]

+3!

[moorchegue]

We missed this feature request's 5 year anniversary!

[ryanraba]

I need this also. For my purposes, there is a bit of a workaround though.

In conf.py, add:

os.system("wget https://raw.githubusercontent.com/whatever/you/need/thefile")

This will pull down the file to the sphinx build directory whenever sphinx executes. Now in your toctree, just list thefile like any local file

Not actually using intersphinx at all

[davbeek]

+4!

[tsibley]

I'm interested in this functionality, and might be willing/able to implement it, but would love a quick feasibility check from someone more experienced with Sphinx first. Does anyone have a sense of what this would take to support? Does someone with experience with Sphinx internals and intersphinx think it's doable without sweeping design changes to toctree and/or intersphinx?

[tk0miya]

At present, there are no way to fetch toctrees of the external document because the inventory file (that is used to resolve the cross references by intersphinx) does not contain them. And intersphinx is only invoked on the resolving-phase. So there are no chance to insert external toctrees to the local one. So you need to design and implement the whole of features for this issue.

Additionally, we need to discuss how external toctrees are represented in non-HTML outputs. So I think it would be better to go to implement this as a 3rd party extension, at first.

[xmo-odoo]

@tk0miya I don't know about other commenters but my original use-case was really just having intersphinx links as part of the toctree, so in the same way you can :ref:comparison manual <python:comparisons> you could do something similar in a TOC and it'd link to a remote document rather than the local documentation.

Though maybe you mean the very structuring of Sphinx' toctrees would require the ability to graft entire toctrees?

[tk0miya]

Ah, sorry. I wrongly commented yesterday. If your goal is creating a link to the external document in toctree like :ref: does, it's not so difficult. The inventory file already contains these labels. For that use-case, my large concern is representation in non-HTML outputs. In HTML output, the toctree behaves like a collection of hyperlinks. But, not so in other outputs. It is used to combine all of document in toctree to single document (PDF, man page, and so on).

[tsibley]

Good to point out that those two things are separate! I'm interested in both. :-)

How do non-HTML outputs handle external links in a toctree already?

[tsibley]

@tk0miya Your summaries are really helpful, thank you!

[tsibley]

How do non-HTML outputs handle external links in a toctree already?

Asked too quick, as I just answered my own question. Looks like sphinx.builders.manpage and sphinx.builders.latex for example both use inline_all_toctrees() from sphinx.util.nodes, which iterates over the toctree's list of included local files. The directive's implementation tracks an entry for external links, but doesn't add to the corresponding list of local files since there of course isn't one. So it seems external links are just silently skipped during inlining. Presumably it'd be ok for intersphinx references in TOC trees to work similarly?

[xmo-odoo]

Asked too quick, as I just answered my own question. Looks like sphinx.builders.manpage and sphinx.builders.latex for example both use inline_all_toctrees() from sphinx.util.nodes, which iterates over the toctree's list of included local files. The directive's implementation tracks an entry for external links, but doesn't add to the corresponding list of local files since there of course isn't one. So it seems external links are just silently skipped during inlining.

It's a completely separate matter, but since latex supports hyperlinks it ought be possible not to skip them no?

[tsibley]

It's a completely separate matter, but since latex supports hyperlinks it ought be possible not to skip them no?

Seems that way to me, yes. Other builders could render them the same way they render other external links too. The source of inline_all_toctrees() is pretty readable if you're curious:

sphinx/sphinx/util/nodes.py

Lines 397 to 428 in 6d5478b

	def inline_all_toctrees(builder: "Builder", docnameset: Set[str], docname: str,
	tree: nodes.document, colorfunc: Callable, traversed: List[str]
	) -> nodes.document:
	"""Inline all toctrees in the *tree*.
	Record all docnames in *docnameset*, and output docnames with *colorfunc*.
	"""
	tree = cast(nodes.document, tree.deepcopy())
	for toctreenode in tree.traverse(addnodes.toctree):
	newnodes = []
	includefiles = map(str, toctreenode['includefiles'])
	for includefile in includefiles:
	if includefile not in traversed:
	try:
	traversed.append(includefile)
	logger.info(colorfunc(includefile) + " ", nonl=True)
	subtree = inline_all_toctrees(builder, docnameset, includefile,
	builder.env.get_doctree(includefile),
	colorfunc, traversed)
	docnameset.add(includefile)
	except Exception:
	logger.warning(__('toctree contains ref to nonexisting file %r'),
	includefile, location=docname)
	else:
	sof = addnodes.start_of_file(docname=includefile)
	sof.children = subtree.children
	for sectionnode in sof.traverse(nodes.section):
	if 'docname' not in sectionnode:
	sectionnode['docname'] = includefile
	newnodes.append(sof)
	toctreenode.parent.replace(toctreenode, newnodes)
	return tree

[Tzaphkiel]

My use case is a bit different, I would like to maintain the SDMX Standard's documentation from the various repositories it comprises of under one main documentation site (HTML + PDF downloads). I would like to leave the oportunity and responsability to each "sub-"repository to update their documentation but have only one TOC.

• sdmx-im : contains the TOC & base documentation
  • sdmx-ml : contains the doc for the XML format
  • sdmx-json : contains the doc for the JSON format
  • sdmx-rest : contains the doc for the API
  • ...

and the TOC has sections that correspond to the main documentation and repositories.

This would mean that the sub-section in the TOC would be expanded from the TOC hierarchy of the sub-repositories.

See here a trial publication

[tsibley]

@Tzaphkiel

This would mean that the sub-section in the TOC would be expanded from the TOC hierarchy of the sub-repositories.

Yep, I'd like this as well. If you end up implementing it as a Sphinx extension (as suggested by @tk0miya above), would you drop a note here?

[Tzaphkiel]

Just a precision, sub-repository here for me is just another repository of my choosing (it happens to be in the same organization in github).
It is also a sub-project from the readthedocs.org project's perspective.

[tk0miya]

So it seems external links are just silently skipped during inlining. Presumably it'd be ok for intersphinx references in TOC trees to work similarly?

Are you saying the external links (including intersphinx) are shown only on HTML output? If so, I don't prefer your proposal. I still don't understand the external links are parts of the "table of contents". And I think Sphinx is a document converter that converts single-input into multi-format. But proposed behavior (mark-up) is designed only for HTML. So I feel odd...

[tsibley]

Are you saying the external links (including intersphinx) are shown only on HTML output? If so, I don't prefer your proposal.

I was describing what Sphinx currently does for most formats (latex, epub, etc… even singlehtml) when rendering toctrees that contain external links. For example, an index.rst with:

.. toctree::

   one
   external <https://sphinx-doc.org>
   two

will omit the sphinx-doc.org entry in the rendered table of contents and combined output document, except in the HTML output format.

I agree having external links in a table of contents is conceptually a little weird, particularly for output formats other than HTML (although even single-page HTML omits them). It's a bit like a section titled "About Y" in a manual that only contains the single sentence "Refer to document X for more information on Y." I do think it'd make more sense conceptually if intersphinx could embed the external toctrees too.

[tk0miya]

Indeed. Currently, a toctree directive accepts an external link as its entry, and it is suppressed in non-HTML formats. TBH, I feel that is a bad design. It is surprising behavior to me that Sphinx automatically removes my contents silently on building to some output. I'd not like to add such a similar feature.

[tsibley]

TBH, I feel that is a bad design. It is surprising behavior to me that Sphinx automatically removes my contents silently on building to some output. I'd not like to add such a similar feature.

That's very fair. I was surprised to find out it silently dropped some contents too.

Your point about not adding to the problem by extending that behaviour make sense. One way to approach it in non-HTML formats might be including a section for the entry using the entry title which contains just "See <url>". Does that feel ok to you?

[tk0miya]

One way to approach it in non-HTML formats might be including a section for the entry using the entry title which contains just "See ".

What happens with your example?

.. toctree::

   one
   external <https://sphinx-doc.org>
   two

Is it expanded like this?

Contents of one
===============

blah blah blah ...

See `external <https://sphinx-doc.org>`_

Contents of two
===============

blah blah blah ...

I think a collection of hyperlinks (including cross-references) and table-of-contents are different things. So it should be better to separate them if possible.

[tsibley]

I was thinking it would be expanded using the given title for the external link, so:

Contents of one
===============

blah blah blah ...

External
========

See <https://sphinx-doc.org>.

Contents of two
===============

blah blah blah ...

For an intersphinx :ref:, it would pull the actual reference/document title from an objects.inv inventory file and use that title by default if none was given.

(and a new toctree inventory file could even let it include the sub-toctree of the referenced section, if applicable)

I think a collection of hyperlinks (including cross-references) and table-of-contents are different things. So it should be better to separate them if possible.

Nod. I understand this desire. But since toctree is very special and central to Sphinx, its lack of support for external refs makes it hard to create unified "umbrella" documentation projects which are comprised of several sub-projects.

[tk0miya]

I was thinking it would be expanded using the given title for the external link, so:

I don't know it makes sense to everyone. But I feel interesting.

[RabidCicada]

This seems very cool to me!

[acceleratxr]

+1

I just spent the last two hours trying to figure out why intersphinx wasn't working only to find this thread... using intersphinx links in a toctree should be a given.

[pkulev]

same here 😢 couple of hours totally wasted

[amotl]

Hi there,

feeling to be in the same boat like you and others who are discussing similar topics over at ¹, you may enjoy to hear we started to work on a more versatile linktree directive.

• Add "link tree" element, using directive linktree tech-writing/sphinx-design-elements#11

If you would like to join the conversation about it, in order not to hijack this one too much, you are welcome to add your ideas on the issue we just created.

• Discussing and shaping the linktree directive tech-writing/sphinx-design-elements#14

With kind regards,
Andreas.

Footnotes

1. https://stackoverflow.com/questions/27979803/external-relative-link-in-sphinx-toctree-directive ↩