TLS for DNSLink websites loaded via public subdomain gateways

[lidel]

By default, people will point DNSLink website at their own go-ipfs and set up proper TLS cert for the domain.
This is how https://en.wikipedia-on-ipfs.org works.

Problems when loading DNSLink website from a public gateway

There are known problems when someone tries to load DNSLink website from an alternative, public gateway:

• path-based gateway at https://ipfs.io/ipns/en.wikipedia-on-ipfs.org works fine, but does not provide Origin isolation and can't be used for dapps and advanced websites
• subdomain gateway at https://dweb.link/ipns/en.wikipedia-on-ipfs.org returns redirect to http://en.wikipedia-on-ipfs.org.ipns.dweb.link which provides Origin isolation, but also returns TLS error due to the way certs work.
  • TLS certificates support only a single level of wildcards (*.ipns.dweb.link and *.ipfs.dweb.link). This works fine for CIDs, which are a single DNS label, but DNSLink names have . in them, which introduces more than one level.

Constraints

I think the main criteria for DNSLink representation are:

1. DNS label representing FQDN with DNSLink should remain human-readable
2. When loaded in web browser DNSLink website should get proper Origin isolation between content roots
3. Subdomain URLs should work with any HTTP client

Solving the TLS problem

👉 Below is a summary of options I see, would love to hear what others think / do sanity check on this / propose your own.

(A) Mining wildcard certificates on the fly

My understanding is that to support DNSLink in subdomains at our public gateway without this TLS error we would have to create some magical orchestration which "mines certificate" on first load.

While it should be technically possible via some nginx+lua hackery (needs sanity check), it may violate ToS of services like Let's Encrypt. This means we would have to work with CA, make our case and ensure they won't ban us when we do it. If we have green light, we could either document the setup for other gateways to use, or consider implementing the magic it in go-ipfs' gateway itself.

💔 I don't like this because it brings complexity/centralization of PKI deep into go-ipfs/gateway logic, ale we should avoid that if we can.

(B) Do nothing, live with TLS errors

We could just live with the TLS error, and when someone points it out say "to be independent of centralized PKI you need to Install IPFS Desktop and IPFS Companion".

💔 Avoiding the problem is bad, but also I worry people won't understand nuance why, and just take a note that "IPFS breaks/ignores web security", which is a pretty bad meme.

(C) Encode FQDN to a string that fits in a single DNS label

This may be the least painful fix so far: if we come up with a way of encoding domain names to something that fits in a single DNS label (max 63 characters), then https://dweb.link/ipns/foo.tld would redirect to https://{encoded-foo-tld}.ipfs.dweb.link and there would be no TLS error (at last for domains shorter than 63 characters).

💚 The encoding step could be added to go-ipfs (just like we convert to base36 if base32 is too long) and that way every gateway would support this with pre-existing TLS certs (no additional setup, seamless upgrade).

Challenges:

• FQDN encoding needs to be both human-readable and non-spoofable (how to tell foo-bar.tld vs foo.bar.tld apart if both are represented as foo-bar-tld etc)
• Encoding means one no longer can just add/remove .ipns.example.com suffix to get the original domain name,
  but its still a lesser evil for public gateways. And when native IPFS is used on localhost subdomains, we can keep original domain names because there is no TLS.

[bertrandfalguiere]

What about (C) with and an escape character? For exemple, with - as an escape character, foo-bar.tld becomes foo--bar-tld and foo.bar.tls become foo-bar-tld.

[Gozala]

I have faced this exact dilemma in my lunet experiment and have settled on a following compromise:

Loading https://lunet.link/en.wikipedia-on-ipfs.org does not redirect to anything instead:

1. Looks up dnslink record for the path en.wikipedia-on-ipfs.org
   • If it maps to /ipns/Qm..hash/ use origin https://Qm..hash.celestal.link
   • If it maps to /ipfs/Qm...hash/ derive origin by hashing /ipns/en.wikipedia-on-ipfs.org e.g drv...hash.celestal.link.
2. Serve page with a content like <iframe style="width: 100%; height: 100%" src="https://drv..hash.lunet.link" />

Which achieved following:

1. User would still see nice URLs like https://lunet.link/en.wikipedia-on-ipfs.org.
2. Every page was origin separated where origin was IPNS public key, with fallback to a domain hash.

   Note: I think public keys are much better option than domain names because

   • It multiple different domains to share the origin
   • To migrate across domain names preserving origin (and storage)
   • Decentralized as in one can generate a key pair
   • Record updates could be signed with a private key to ensure integrity

3. On could share link that looked and worked as expected.

It had following downsides:

1. Reflecting URL changes e.g. via pushState required some JS trickery.
2. curl-ing URL would not give you an actual content, but rather iframe with some celestal.link.

   That could probably be addressed via redirect based on user agent and other headers.

That is all to suggest that I think it's best to decouple UX and Origin separation problems from each other. E.g dweb.link could be used to do the Origin separation and another domain could provide human meaningful URLs on top. I don't believe both could be achieved at the same time.

If I were to choose between some domain escaping logic and use of public keys in it's place I would much rather go with later as in my experience deriving origins from domain names did not really provided good UX, on the contrary it resembled spoofing attacks.

[lidel]

I feel that lunet operated at higher abstraction layers, to which we don't have access here (no keys etc).

I think the main criteria for DNSLink representation are (updated first comment):

1. DNS label representing FQDN with DNSLink should remain human-readable
   • Frankly, DNSLink already looks like "spoofing attack": http://docs.libp2p.io.ipns.localhost:8080
     I don't think http://docs-libp2p-io.ipns.dweb.link would be much worse. I'd say it looks LESS suspicious.
2. When loaded in web browser DNSLink website should get proper Origin isolation between content roots
3. Subdomain URLs should work with any HTTP client
   • Solution should work at HTTP level, not HTML or JS
   • Either no iframes or we are forced to do even more user-agent sniffing ( which we hoped to move away from)
     • while https://dweb.link/ipns/dnslink.site.example.com could return HTML with:

       <iframe src=""https://dnslink-site-example-com.ipns.dweb.link` />

---

Note: DNSlink representation in subdomain on public gateway will be a niche use case.
Most of the people will load DNSLink website wither from localhost gateway or original domain.

I am leaning towards keeping this very simple and implementing a variant of (C) which supports encoded domains:

• Processing https://dweb.link/ipns/dnslink.site.example.com would detect HTTPS scheme and return a redirect to https://dnslink-site-example-com.ipns.dweb.link, removing the TLS problem and nothing more

[Gozala]

I feel that lunet operated at higher abstraction layers, to which we don't have access here (no keys etc).

That is fare assessment. I don't think this is in conflict with what I was trying to suggest however. The way I see it https://${derived_origin(domain)}.dweb.link is an equivalent of what lunet used to load inside iframes (that is https://${derive_origin(domain)}.celestal.link).

That higher level abstraction layer could be built on top of this.

I do still however think that it would be highly beneficial if derive_origin function could be extended beyond domain.replace(/./g, '-'). Specifically what I'm suggesting it could do is following:

1. Read TXT DNS record similar to (or extended version of) dns_link that provides public key to be used in place of domain.replace(/./g, '-'), and return that.
2. If public key is not present fall back to the substitution based derivation.

That enables:

1. Site to optionally migrate from domain to domain without loosing user data cache.
2. Loading https://dweb.link/ipns/${pub_key_for_foo_com} and https://dweb.link/ipns/foo.com would result in the same origin.

But it does makes redirect URLS less readable as in https://foo-com.ipns.dweb.link vs https://Qm...hash.dweb.link, however since providing a public key is optional I think it's reasonable to give domain owner a choice here.

All that said, it may be doing simple char substitution in this iteration and considering future extensions (like mentioned public keys base origins) is best compromise. I did however wanted to point out than in my prior experience I found simple char substitution problematic because it did not play well with IPNS.

[lidel]

Ok, I am leaning towards shipping support for simple substitution in go-ipfs ~0.9:
https://dweb.link/ipns/my.v-long.example.com → https://my-v--long-example-com.ipns.dweb.link

Immediate need is to make subdomain gateways drop-in replacement for all paths without surprises or need for custom TLS setup (including ones with DNSLink name).

Extending DNSLink with optional pubkey is exciting, but requires more analysis and I feel we need to park it until we have time to work on IPNS itself.