Default hour-cycle computation can resolve to non-preferred hour format

[anba]

https://github.com/tc39/test262/blob/master/test/intl402/DateTimeFormat/prototype/resolvedOptions/hourCycle-default.js is currently failing in SpiderMonkey, revealing a potential spec issue.

Reduced test case:

function hourCycle(hour12) {
  return new Intl.DateTimeFormat("en", {hour: "numeric", hour12}).resolvedOptions().hourCycle;
}

print("default:", hourCycle(undefined));
print("hour12=true:", hourCycle(true));
print("hour12=false:", hourCycle(false));

Returns in SpiderMonkey:

default: h12
hour12=true: h12
hour12=false: h23

But in V8 (and per current spec):

default: h12
hour12=true: h12
hour12=false: h24

InitializeDateTimeFormat, step 30.d for hcDefault=h12, computes hc=hc12 for hour12=true resp. hc=hc24 for hour12=false.

But when we're consulting CLDR's supplementalData.xml for "US" [1], the preferred hour formats are either "h" or "H", which means either "hc12" or "hc23". So "hc24" is likely an unexpected result for US, because "k" isn't listed in the preferred hour formats for that region. (See [2] for "h", "H", and "k".)

[1] Adding likely subtags to the "en" locale results in "en-Latn-US".
[2] https://unicode.org/reports/tr35/tr35-dates.html#dfst-hour

[anba]

cc @zbraniecki

[mobz]

See also https://bugs.chromium.org/p/chromium/issues/detail?id=1045791

[littledan]

This issue seems somewhat important--it is a sort of user-observable "bug". Should we put it on the ES2021 milestone?

[mobz]

I consider it a bug. The assumption in the spec is that h11 maps to h23 and h12 maps to h24, as a logical preference is wrong.

We can see from various bugs in date formatting packages that are following the spec that this is surprising.

hourCycle options map to keys in https://unicode.org/reports/tr35/tr35-dates.html#dfst-hour
K = h11 = 0-11
h = h12 = 1-12
H = h23 = 0-23
k = h24= 1-24
Fall back formatting preferences for different regions can be obtained http://cldr.unicode.org/
We can see (data below) that

• all regions use either h12 or h23 as the default
• all regions have h23 as an allowed value
• zero regions have h24 as an allowed value
• one region (JP) has h11 (K) as an allowed value

there is not a single region on the planet that provides h24 as an allowed format.

either;

• hcDefault should only apply for hour12=true,
• The value for hourCycle when hour12=false should be h23 in all cases
  or
• the value for hcDefault should be determined based on the value of hour12
  or
• hcDefault should be an ordered array of values and we should match the first one that matches hour12 setting

all these algorithms should evaluate to the same thing.
(formatting part b/B= am/AM etc)

grep "<hours.*allowed" ./cldr/common/supplemental/supplementalData.xml
<hours preferred="H" allowed="H" regions="AX BQ CP CZ DK FI ID IS ML NE RU SE SJ SK"/>
<hours preferred="H" allowed="H h" regions="001"/>
<hours preferred="H" allowed="H h hb hB" regions="AC AI BW BZ CC CK CX DG FK GB GG GI IE IM IO JE LT MK MN MS NF NG NR NU PN SH SX TA ZA"/>
<hours preferred="H" allowed="H h hB" regions="CF CM LU NP PF SC SM SN TF VA ca_ES fr_CA gl_ES it_CH it_IT"/>
<hours preferred="H" allowed="H h hB hb" regions="AR CL CR CU EA GT HN IC KG KM LK MA MX NI PY SV UY af_ZA es_BO es_BR es_EC es_ES es_GQ es_PE"/>
<hours preferred="H" allowed="H h K" regions="JP"/>
<hours preferred="H" allowed="H hb hB h" regions="AF LA"/>
<hours preferred="H" allowed="H hB" regions="AD AM AO AT AW BE BF BJ BL BR CG CI CV DE EE FR GA GF GN GP GW HR IL IT KZ MC MD MF MQ MZ NC NL PM PT RE RO SI SR ST TG TR WF YT"/>
<hours preferred="H" allowed="H hB h" regions="AZ BA BG CH GE LI ME RS UA UZ XK"/>
<hours preferred="H" allowed="H hB h hb" regions="BO EC ES GQ PE"/>
<hours preferred="H" allowed="H hB hb h" regions="LV TL zu_ZA"/>
<hours preferred="H" allowed="hB H" regions="CD IR"/>
<hours preferred="H" allowed="hB hb H h" regions="KE MM TZ UG"/>
<hours preferred="h" allowed="h H" regions="AS BT DJ ER GH IN LS PG PW SO TO VU WS"/>
<hours preferred="h" allowed="h H hb hB" regions="CY GR"/>
<hours preferred="h" allowed="h H hB" regions="AL TD"/>
<hours preferred="h" allowed="h H hB hb" regions="CO DO KP KR NA PA PR VE"/>
<hours preferred="h" allowed="h hb H hB" regions="AG AU BB BM BS CA DM FJ FM GD GM GU GY JM KI KN KY LC LR MH MP MW NZ SB SG SL SS SZ TC TT UM US VC VG VI ZM en_001"/>
<hours preferred="h" allowed="h hB H" regions="BD PK"/>
<hours preferred="h" allowed="h hB hb H" regions="AE BH DZ EG EH IQ JO KW LB LY MR OM PH PS QA SA SD SY TN YE ar_001"/>
<hours preferred="h" allowed="hb hB h H" regions="BN MY"/>
<hours preferred="h" allowed="hB h H" regions="hi_IN kn_IN ml_IN te_IN"/>
<hours preferred="h" allowed="hB h H hb" regions="KH"/>
<hours preferred="h" allowed="hB h hb H" regions="ta_IN"/>
<hours preferred="h" allowed="hB hb h H" regions="CN HK MO TW ET gu_IN mr_IN pa_IN"/>

[zbraniecki]

So, if I'm correct, SpiderMonkey enforces h12/h23 while V8 enforces h12/h24 for hour12 true/false, right?
My preference would be to make the fallback depend on the locale's preferred value.

[anba]

SpiderMonkey uses the following hour symbols in the skeleton:

• No option present: "j"
• hour12 = true: "h"
• hour12 = false: "H"
• hourCycle = h11: "h", plus modifying the resolved pattern to use the hour symbol "K".
• hourCycle = h12: "h", plus modifying the resolved pattern to use the hour symbol "h".
• hourCycle = h23: "H", plus modifying the resolved pattern to use the hour symbol "H".
• hourCycle = h24: "H", plus modifying the resolved pattern to use the hour symbol "k".

(Input skeletons don't differentiate between "h" and "K" resp. "H" and "k".)

With that approach we don't strictly enforce h12/h23, which is visible for "ja":

js> print(new Intl.DateTimeFormat("en", {hour:"numeric", hour12:true, timeZone:"UTC"}).format(new Date("2020-01-01T12:00Z"))) 
12 PM
js> print(new Intl.DateTimeFormat("ja", {hour:"numeric", hour12:true, timeZone:"UTC"}).format(new Date("2020-01-01T12:00Z")))
午後0時

[ziyunfei]

I was bitten by this V8 change early this week https://bugs.chromium.org/p/chromium/issues/detail?id=1045791#c7.

Here was my quick fix :

[ray007]

What I'm missing with the hourCycle option is a 0-24.
When defining time-ranges, I might want an end-time of 0:55, but also 24:00 when it's midnight.

[littledan]

@ziyunfei This is exactly what { hourCycle: "h23" } is for. Your RegExp seems like it could be pretty unfortunate if the number 24 shows up somewhere else in the datetime!

@ray007 Can you say more about in which locales this is appropriate?

[ray007]

@ray007 Can you say more about in which locales this is appropriate?

I know that at least with german speaking locales having midnight as endtime with 24:00 is common.
So the time-range for a fullday event would be 00:00 - 24:00.

No idea about other locales.

[littledan]

@ray007 In this case, it's hard for me to see what sort of setting for Intl.DateTimeFormat would permit both 0:00 and 24:00 to be generated with the same formatter...

[ray007]

@ray007 In this case, it's hard for me to see what sort of setting for Intl.DateTimeFormat would permit both 0:00 and 24:00 to be generated with the same formatter...

No need.
It's either 0:00 .. 23:59 (start time) or 0:01 .. 24:00 (end time). 2 formatters needed.