Divina: linkable rectangles in images

[chocolatkey]

Japanese manga magazines use HTML/SVG-native <a> elements with <rect>, inside of the <svg> of a fixed layout EPUBs. When loading the EPUBs into a web engine, this feature magically works, given a decent EPUB renderer (feature shown here highlighted by dev tools):

The XHTML of this page (taken from a real live EPUB) is available here: https://gist.github.com/chocolatkey/e86d79b43b4450d1bcf75c47b9dad4e2

This lets users easily get to a chapter in the publication, despite there being no HTML table of contents in the spine, which is always the case with e.g. manga.
Proprietary reading solutions in Japan have implemented this feature during conversion to a divina-like format for web, live example here: https://viewer-trial.bookwalker.jp/03/12/viewer.html?cid=9bed8d15-ca00-44d6-bdc0-f5051ee1f4b0&cty=1 . The representation in their JSON looks like this (the LinkList):

Another example of a manga reader (for the same publication) that implements this same feature: https://booklive.jp/bviewer/s/?cid=411736_059

I think having this feature in divina would help increase adoption in those situations, and it is not particularly difficult to implement this feature in a web reader (or native one).

[mickael-menu]

That's a great use case.

I think this could go beyond images and work with any kind of media type (most useful for binary formats of course). For example, an audiobook could have some links to characters description available for each characters in a "scene". For this, a media fragment can determine the duration of the link.

Think of X-Ray by Amazon (available on Kindle but also Prime video).

It reminds me of the discussion on the Synchronized Narration spec, at its core it's pretty similar: a way to map two (or more) locators. In the case of Synchronized Renditions, it is used to synchronize the playback of multiple resources, while here it is to display links. Maybe there's an opportunity for a shared JSON schema.

[HadrienGardeur]

First of all, aside from an implementation using SVG, I believe that this can also be implemented in HTML with image maps.
Image maps have also been mentioned in the context of a11y for FXL as they also allow to associate text to areas of an image.

In the case of Divina, where we don't use HTML or SVG at all, I think that this can be implemented using a similar approach.
For example:

{
  "href": "http://example.org/page1.jpeg",
  "type": "image/jpeg",
  "map": [
    {
      "href": "#xywh=0,0,300,200",
      "title": "Panel 1"
    },
    {
      "href": "#xywh=300,200,310,200",
      "title": "Panel 2"
    }
  ]
}

This would open a number of questions of course:

• do we need a new link property (map) or can we re-use an existing one (children)?
• since href is used to define the area, how do we handle their destination when they have one?
• how do we handle areas that overlap one another?

[HadrienGardeur]

This one came back during TPAC at the breakout session dedicated to manga/comics.

Here's @shiestyle slide about it:

I'm not a fan of the approach of mixing bitmaps with HTML/SVG and this would be forbidden in the Divina profile.

Back in 2021, we didn't had our on-going work on Guided Navigation. Maybe this is how we should handle this use case as well?

Something like this could actually work:

{
  "role": ["imageMap"],
  "imgref": "toc.jpg",
  "children": [
    {
      "imgref": "toc.jpg#xywh=0,0,300,200",
      "href": "chapter1.jpg"
    },
    {
      "imgref": "toc.jpg#xywh=300,200,310,200",
      "href": "chapter2.jpg"
    }
  ]
}

[HadrienGardeur]

I'm going back to that one again.

Thinking about this once again, I'm not necessarily convinced that this is a good fit for Guided Navigation Documents.

Based on my understanding of how this is used in Japan, I would say that:

• these links on regions of an image are very sparsely used
• they're only present on a few pages at most
• and they're only relevant when you're reading content without guided navigation

Taking all of these points in consideration, they are maybe better expressed in the manifest directly.

Let's try something in the Link Object directly:

{
  "readingOrder": [
    {
      "href": "cover.jpg",
      "rel": "cover",
      "type": "image/jpeg"
    },
    {
      "href": "toc.jpg",
      "rel": "contents",
      "type": "image/jpeg",
      "properties": {
        "map": [
          {
            "href": "page1.jpg",
            "fragment": "xywh=753,1085,100,100"
          }
       ]
      }
    },
    {
      "href": "page1.jpg",
      "type": "image/jpeg"
    }
  ]
}

@chocolatkey would something like this work in your opinion? Are you aware of non-rectangular shapes used in some of these implementations?