Locale related APIs -- language matching

[zbraniecki]

UPDATE: Latest Proposal - Nov 2015

I started working on a spec and polyfill for Intl.Locale API.

I should have some proposal for this next week. Currently I have five functions on it:

• isStructurallyValidLanguageTag(locale)
• canonicalizeLocaleList(locales)
• resolveLocale(availableLocales, requestedLocales, options, relevantExtensionKeys, localeData)
• prioritizeAvailableLocales(availableLocales, requestedLocales, defaultLocale)
• getDirection(locale)

The rationale for those five are:

• isStructurallyValidTag
  This allows testing language tag for sanitation/validation purposes.
• canonicalizeLocaleList
  This is the core function allowing all language negotiation operations
• resolveLocale

this exposes the core function used by all Intl formatters that allows people to write their own custom formatters or polyfills.

• prioritizeAvailableLocales

This is basic language negotiation function for all localization libraries. It takes requestedLocales list (navigator.languages or custom), availableLocales (app provided locales), and a fallback defaultLocale (default app locale, not default host env locale) and returns the list of prioritized available locales that best matches user requested locales.

• getDirection

Simple function allowing l10n libraries to set html.dir based on their negotiated locale.

@caridy - One question I'd like to get help with is about using Set instead of Array. In all cases where we carry a list of locales, we always want it deduplicated and Set does just that. It would be sweet to use it instead of an Array, but it's API seems to be significantly less user friendly than that of the Array - it's impossible to just retrieve locales[0], it's not easy to map etc.
Should I stick to Array or try to use Set?

[caridy]

Two things:

• You can certainly use a Set internally as part of the logic, but let's stick to an array for arguments, I haven't seen any precedent of using a Set as arguments in Ecma.
• Instead of implementing a new polyfill (once we settle on the api) you can just submit a PR for Intl.js, which already implements those abtract operations.

[zbraniecki]

sure! thanks for the feedback :)

[zbraniecki]

@caridy - I started playing with Intl.js and ended up separating src/locale.js out of src/core.js and then carved out the common functions into src/utils.js because src/core.js is huuge :)
Not sure if you're interested in any sort of refactor like that - if not, I'll bring it back into src/core.js.

Branch is here: https://github.com/zbraniecki/Intl.js/tree/locale

I also wrote a simple polyfill just for Intl.Locale based on Gecko implementation - https://github.com/zbraniecki/IntlLocale

Let's now talk about the API and get it agreed upon (before I start the daunting task of working on the spec)

[caridy]

@zbraniecki I added the module system into Intl.js to be able to split it into small chunks, I didn't get to split core into small chunks. In other words, go for it, we will appreciate it. Just open a PR there, and we can discuss the details.

In the other hand, I'm more interested on the API discussion :)

[zbraniecki]

@caridy Cool. Do you have any comments on the API I proposed in comment 1? I'd like to make Locale a good unified language negotiation API for other Intl APIs and localization frameworks.

I believe that with those five functions are enough to achieve that. On top of that, all of that code except of getDirection is already in Intl API, just not exposed, so it should not cause any cost to the vendors deploying it.

[zbraniecki]

I guess it's not the right idea to at the same time refactor a major piece of code and introduce new feature for debate in it :)

So I updated my locale branch to just introduce IntlLocale to Intl.js polyfill now - https://github.com/zbraniecki/Intl.js/tree/locale

@caridy - does it look like something you may be interested in?

[caridy]

update: we need to cook the proposal for the next meeting in two weeks. will try to coordinate some time to work on this.

[zbraniecki]

We've talked about this with @caridy and he mentioned that Intl formatters may be interested in switching to something like prioritizeLocales for language negotiation.

This would mean that we don't have to expose resolveLocale at all, and just exposing prioritizeLocales will be enough for both - localization frameworks and for custom intl formatters to match the behavior of the platform ones.

The only thing coming to my mind that we may want to use depth parameter to limit the length of the resolved chain - if Intl formatter will only take the first one anyway, there's no reason to resolve the whole chain. At the same time l10n frameworks operate in more uncertainty (l20n for example falls back when a single l10n string cannot be resolved in a given language at runtime) so they need full "depth".

That would leave us with:

• isStructurallyValidLanguageTag
• canonicalizeLocaleList
• prioritizeLocales(availableLocales, requestedLocales, defaultLocale, depth)
• getDirection

(and later)

• firstDayOfTheWeek (or weekendInfo returning firstDay, weekendStarts and weekendEnds)

wrt. prioritizeLocales you can see an implementation here: https://github.com/l20n/l20n.js/blob/v1.0.x/lib/l20n/intl.js#L434

It uses LookupAvailableLocales which is a new function that acts exactly like LookupSupportedLocales but instead of returning a requested locale that has a match in available ones, it returns the available locale that has a match in requested ones.
So if the user requests 'sv-SE' and we have 'sv', LookupSupportedLocales will return 'sv-SE', while the new function will return 'sv'. Which is what l10n and intl can later use to load the right resources.

[rxaviers]

I'm interested to know the exact algorithm and how this relates to https://github.com/rxaviers/ecma402-fix-lookup-matcher please.

[caridy]

@rxaviers that's precisely the reason why we will abstract the operation to produce the parent list, then we can update that to take into consideration the proper algo step.

[caridy]

Abstract Locale Operations - Nov 2015