diff --git a/CLAUDE.md b/CLAUDE.md index 2ff895679469d..a37a3d99e1442 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -160,7 +160,7 @@ Custom event loop implementation for Bun's runtime requirements ``` ### When a test fails, and after an upstream sync -`JSTests/`, `LayoutTests/js` and the PerformanceTests collections are upstream's tests, and this fork has changed 124 of them. `JSTests/BUN-TEST-DIFFERENCES.md` lists every one by reason: the behaviors the fork has on purpose (with the fork code behind each and the files whose expectations follow it), the tests narrowed to the modes or builds they can pass in, and the ones skipped with `TODO(bun)` for a known gap. Read it before changing a test's expectation or skipping one: a failure that matches an entry there means the test is what to change; one that matches none is a regression in the engine. Its last section is the procedure for sorting a new failure. When a test is changed, add it to that file, and put a `// Bun:` or `// TODO(bun):` comment on the changed line saying what upstream expects. `JSTests/CLAUDE.md` and `JSTests/README.md` are upstream's files: leave them as they are. +`JSTests/`, `LayoutTests/js` and the PerformanceTests collections are upstream's tests, and this fork has changed 125 of them. `JSTests/BUN-TEST-DIFFERENCES.md` lists every one by reason: the behaviors the fork has on purpose (with the fork code behind each and the files whose expectations follow it), the tests narrowed to the modes or builds they can pass in, and the ones skipped with `TODO(bun)` for a known gap. Read it before changing a test's expectation or skipping one: a failure that matches an entry there means the test is what to change; one that matches none is a regression in the engine. Its last section is the procedure for sorting a new failure. When a test is changed, add it to that file, and put a `// Bun:` or `// TODO(bun):` comment on the changed line saying what upstream expects. `JSTests/CLAUDE.md` and `JSTests/README.md` are upstream's files: leave them as they are. ## Development Tips diff --git a/JSTests/BUN-TEST-DIFFERENCES.md b/JSTests/BUN-TEST-DIFFERENCES.md index a99a4a1ea283a..8cb37c9bd2ac3 100644 --- a/JSTests/BUN-TEST-DIFFERENCES.md +++ b/JSTests/BUN-TEST-DIFFERENCES.md @@ -1,10 +1,10 @@ # Where this fork's tests differ from upstream's, and why -`JSTests/`, `LayoutTests/js` and the PerformanceTests collections are upstream's tests. This fork changes 124 of them and +`JSTests/`, `LayoutTests/js` and the PerformanceTests collections are upstream's tests. This fork changes 125 of them and three runner scripts. Every change is listed here by the reason for it, so that after an upstream sync a failing test can be sorted into one of three piles: -1. **The fork behaves differently on purpose.** Update the expectation. Sections A1–A9 list every such behavior, how to +1. **The fork behaves differently on purpose.** Update the expectation. Sections A1–A10 list every such behavior, how to recognize it in a failure, and the fork code that causes it. If a failure matches one of these and the fork code is still there, the test is the thing to change. 2. **The test cannot pass in a mode or build we run.** Skip it in that mode only, with the reason. Section B. @@ -114,6 +114,40 @@ Rules for an edit to an upstream test: - Fork code: `DecoderStringTable::jsStringFor` (`runtime/CachedTypes.h:132`): loading a cache interns nothing. - File: `stress/jsstring-definitely-atom-bit.js`: `$skipModes << "bytecode-cache"`. The other modes run it unchanged. +### A10. `Intl.DateTimeFormat` prints ICU's own era and year before the epoch of an islamic or the coptic calendar + +- Upstream replaces ICU's text in `format()` and `formatToParts()`, and so in `formatRange()` of two Numbers or Dates + that print as one date (that arm calls `format()`). It does not in `formatRangeToParts()`, in `formatRange()` of + Temporal values, or in a true range. For `islamic-civil`, `islamic-tbla` and `islamic-umalqura` the era becomes the + English `Before Hijra` and the year becomes 1 − year, and for `coptic` the era `Anno Martyrum` is added + (`proposalEraOverrideString`, `ICU4C-WORKAROUND: rdar://182953351`; upstream's commits are 318428@main + (99473681ff5e), 318847@main (b2ec9a4586ee) and 319172@main (171864159318)). That makes test262's + `formatToParts/era.js` pass. +- The fork removes the override. In these four calendars the methods print one text for one date, in the formatter's + language and in the era width that was asked for. For a Number or a Date it is the text V8 prints on the same ICU. + With the override one formatter gave two texts for one date, and the first hours of 1 AH were + `Muh. 1, 0 Before Hijra`. `@js-temporal/polyfill` 0.5.1 reads its calendar fields from `formatToParts()`: with the + override `Temporal.PlainYearMonth.from("0622-07-25[u-ca=islamic-civil]")` did not return (1.4 GB after 10 s). +- Fork code: the `#pragma GCC poison` at the top of `runtime/IntlDateTimeFormat.cpp`. It is the one line that + `git grep -nE 'EraOverride|proposalEraOverride|flippedEraOverrideYear|formatWithEraOverride|182953351' -- Source/JavaScriptCore` + prints. Anything more is the override, back from a merge: take it out again, callers included, and leave the tests as + they are. +- At a sync, a conflict in `runtime/IntlDateTimeFormat.cpp` whose fork side is empty resolves to the fork side, and a + call that upstream added to the override goes with it. This holds until the ICU that Bun bundles selects these eras + itself, also if upstream only gates its override on a version of ICU. +- Recognize: the test expects `Before Hijra` or `Anno Martyrum` and a year counted back from 1, and gets ICU's + `Anno Hegirae` / `AH` (for coptic, no era) and ICU's year: `1 Before Hijra` → `0 Anno Hegirae`, `333 Before Hijra` → + `-332 Anno Hegirae`, `85 Anno Martyrum` → `85 `. +- Files: `stress/intl-datetimeformat.js`, 12 expectations. test262's + `test/intl402/DateTimeFormat/prototype/formatToParts/era.js` fails: the era of ISO 600 and the era of ISO 2025 are one + text. No job runs test262, so its entries in `test262/expectations-linux.yaml` (made by + `Tools/Scripts/test262-runner --save`) and in `test262/expectations.yaml` (the same entry, not run on a Mac) are + for a run by hand. V8 and SpiderMonkey skip that file. ICU has no era before the Hijra (ICU-23327) and still has + two coptic eras (ICU-23277). +- The section ends when the bundled ICU selects these eras: `stress/intl-datetimeformat.js` then fails on the lanes + with the bundled ICU. Take upstream's expectations again, and remove the two test262 entries and the pragma. + `stress/bun-intl-datetimeformat-pre-epoch-era-one-text.js` holds before and after that. + --- ## B. The test cannot pass in a mode or build we run: narrowed, not changed @@ -180,6 +214,7 @@ Not upstream's: a sync neither brings nor changes them. Most are named `stress/b | `stress/bun-object-assign-calls-indexed-setters-on-the-chain.js` | `Object.assign()` stores with `[[Set]]` on its fast path too: an indexed setter on the target's prototype chain is called, and a read-only element there refuses the store | | `stress/bun-arrays-become-prototypes-below-immutable-prototypes.js` | with `Object.prototype` and `Array.prototype` immutable and a setter under an index on a class prototype, an array or an object with elements that becomes a prototype keeps the storage it has; reads, stores and changes of length on it and through its heirs answer as the language says | | `stress/bun-weak-map-and-set-shrunk-before-eden-collection.js` | the buffer of a `WeakMap` or `WeakSet` counts in the heap's size as it is behind a collection (`WeakMapImpl::reconcileWeakReferencesAtGCEnd`; upstream counts it as the last visit found it), but an eden collection never leaves that size below what the collection before it left: a buffer that shrank stops counting at the next full collection | +| `stress/bun-intl-datetimeformat-pre-epoch-era-one-text.js` | for a date before the epoch of an islamic or the coptic calendar, `format()`, `formatToParts()`, `formatRange()` and `formatRangeToParts()` of one `Intl.DateTimeFormat` print one text, for a Number, a Date and the Temporal types; every instant of one day prints one date; each end of a range has the year that the date has alone (A10) | | the last block of `stress/regexp-test-sticky-first-char-filter.js` (the file is upstream's; nothing of upstream's text changes) | the first-character filter of a sticky `test()` comes from the first sets of the alternatives (`yarr/YarrFirstFollowSets.*`): behind a leading lookbehind, forward reference or lookahead, and in groups nested more than eight deep, a byte that passes the filter still reaches the matcher | `jsc` cannot compile a program ahead of time by itself (the compiler's input comes from the front end, in Bun's repository), so @@ -189,7 +224,7 @@ the programs that test ahead-of-time compilation are in Bun's `test/aot`. ## E. After a sync: a failing test, step by step -1. Does the difference match A1–A9 exactly (same text, same shape)? Check that the fork code named there is still in the +1. Does the difference match A1–A10 exactly (same text, same shape)? Check that the fork code named there is still in the tree, then update the expectation with a `// Bun:` comment and add the file to its list here. 2. Does it fail only in one mode or only with ASan, for a reason in the test and not the engine (it counts JIT compiles, depends on allocation never failing, needs more than 300 s)? Narrow it as in B. diff --git a/JSTests/stress/bun-intl-datetimeformat-pre-epoch-era-one-text.js b/JSTests/stress/bun-intl-datetimeformat-pre-epoch-era-one-text.js new file mode 100644 index 0000000000000..8e35d82387f49 --- /dev/null +++ b/JSTests/stress/bun-intl-datetimeformat-pre-epoch-era-one-text.js @@ -0,0 +1,112 @@ +//@ requireOptions("--useTemporal=1") +// Bun: not upstream's. This fork has no era override for a date before the epoch of an islamic-* or the coptic calendar +// (BUN-TEST-DIFFERENCES.md, "Intl.DateTimeFormat prints ICU's own era and year"): every method of one +// Intl.DateTimeFormat prints ICU's own era and year. Upstream's override changes what format() and formatToParts() +// print, and so formatRange() of two Numbers or Dates that print as one date. It does not change formatRangeToParts(), +// formatRange() of Temporal values, or a true range. +// The checks compare the methods with each other and name no era text, so they hold for every version of ICU. + +function expect(label, got, want) +{ + if (got !== want) + throw new Error(`${label}: expected ${JSON.stringify(want)}, got ${JSON.stringify(got)}`); +} + +function join(parts) +{ + return parts.map(part => part.value).join(""); +} + +function yearOf(parts, source) +{ + const part = parts.find(part => part.type === "year" && (!source || part.source === source)); + return part && part.value; +} + +const msPerHour = 3600000; +const msPerDay = 24 * msPerHour; + +const calendars = ["islamic-civil", "islamic-tbla", "islamic-umalqura", "coptic"]; +const locales = ["en", "sv", "ar", "ja"]; +const shapes = [ + { era: "long", year: "numeric", month: "short", day: "numeric" }, + { era: "short", year: "numeric", month: "short", day: "numeric" }, + { era: "narrow", year: "numeric", month: "short", day: "numeric" }, + // No era option: the CLDR patterns of these calendars have an era field next to a year. + { year: "numeric", month: "short", day: "numeric" }, + { dateStyle: "full" }, +]; +// Before the epoch of each calendar: ISO 284 for coptic, ISO 622 for islamic-*. +const isoDates = ["-000400-01-15", "0200-01-01", "0283-08-01"]; + +function valuesFor(isoDate, calendar) +{ + const plainDate = Temporal.PlainDate.from(isoDate); + const epochMilliseconds = plainDate.toZonedDateTime("UTC").epochMilliseconds; + return [ + ["Number", epochMilliseconds], + ["Date", new Date(epochMilliseconds)], + ["Instant", Temporal.Instant.fromEpochMilliseconds(epochMilliseconds)], + ["PlainDate", plainDate.withCalendar(calendar)], + ["PlainDateTime", plainDate.toPlainDateTime("12:00").withCalendar(calendar)], + ["PlainYearMonth", plainDate.withCalendar(calendar).toPlainYearMonth()], + ]; +} + +let checked = 0; +for (const calendar of calendars) { + for (const locale of locales) { + for (const shape of shapes) { + const format = new Intl.DateTimeFormat(locale, { calendar, timeZone: "UTC", ...shape }); + const label = `${locale} ${calendar} ${JSON.stringify(shape)}`; + + // One value, four methods, one text. + for (const isoDate of isoDates) { + for (const [kind, value] of valuesFor(isoDate, calendar)) { + const text = format.format(value); + const rangeParts = format.formatRangeToParts(value, value); + expect(`${label} ${kind} ${isoDate} formatToParts`, join(format.formatToParts(value)), text); + expect(`${label} ${kind} ${isoDate} formatRange`, format.formatRange(value, value), text); + expect(`${label} ${kind} ${isoDate} formatRangeToParts`, join(rangeParts), text); + for (const part of rangeParts) + expect(`${label} ${kind} ${isoDate} source of the ${part.type} part`, part.source, "shared"); + ++checked; + } + } + + // Each end of a true range has the year that formatToParts() gives that date. + const start = Temporal.PlainDate.from(isoDates[0]).toZonedDateTime("UTC").epochMilliseconds; + const end = start + 366 * msPerDay; + const rangeParts = format.formatRangeToParts(start, end); + expect(`${label} range, year of the start`, yearOf(rangeParts, "startRange"), yearOf(format.formatToParts(start))); + expect(`${label} range, year of the end`, yearOf(rangeParts, "endRange"), yearOf(format.formatToParts(end))); + } + } +} +expect("values checked", checked, calendars.length * locales.length * shapes.length * isoDates.length * 6); + +// Every instant of one day in the formatter's time zone prints the same date, on the days around the epoch too. +// A range of two such instants is that one date, in either order. +const epochs = [ + ["islamic-civil", Date.UTC(622, 6, 19)], + ["islamic-umalqura", Date.UTC(622, 6, 19)], + ["islamic-tbla", Date.UTC(622, 6, 18)], + ["coptic", Date.UTC(284, 7, 29)], +]; +for (const [calendar, epoch] of epochs) { + for (const shape of shapes) { + const format = new Intl.DateTimeFormat("en", { calendar, timeZone: "UTC", ...shape }); + const texts = new Set; + for (let day = -2; day <= 1; ++day) { + const midnight = epoch + day * msPerDay; + const text = format.format(midnight); + const label = `${calendar} ${JSON.stringify(shape)} day ${day}`; + for (let hour = 1; hour < 24; ++hour) + expect(`${label} hour ${hour}`, format.format(midnight + hour * msPerHour), text); + expect(`${label} range`, format.formatRange(midnight + 3 * msPerHour, midnight + 12 * msPerHour), text); + expect(`${label} reversed range`, format.formatRange(midnight + 12 * msPerHour, midnight + 3 * msPerHour), text); + texts.add(text); + } + expect(`${calendar} ${JSON.stringify(shape)} dates of four days`, texts.size, 4); + } +} diff --git a/JSTests/stress/intl-datetimeformat.js b/JSTests/stress/intl-datetimeformat.js index acb824e0ce033..61c4d3fde5433 100644 --- a/JSTests/stress/intl-datetimeformat.js +++ b/JSTests/stress/intl-datetimeformat.js @@ -350,23 +350,28 @@ shouldBe(Intl.DateTimeFormat('en-u-ca-islamic-rgsa', { timeZone: 'America/Los_An { const opts = { year: 'numeric', era: 'long', timeZone: 'UTC' }; - // islamic-civil/islamic-umalqura epoch is ISO 622-07-19T07:52:58.000Z; islamic-tbla's is one - // day earlier. A date in February 622 is before both real epochs but was previously missed by - // an overly early hardcoded threshold, so it incorrectly reported "Anno Hegirae". - // Year is 1 - year (no year zero), matching Temporal's eraYear; used to show the raw ICU year (0). - shouldBeOneOf(Intl.DateTimeFormat('en-u-ca-islamic-civil', opts).format(Date.UTC(622, 1, 1)), ['Before Hijra 1', '1 Before Hijra']); - // Well before the epoch, not just at the boundary -- the only case that distinguishes "1 - year" from other conventions. - shouldBeOneOf(Intl.DateTimeFormat('en-u-ca-islamic-civil', opts).format(Date.UTC(300, 0, 1)), ['Before Hijra 333', '333 Before Hijra']); - // year:"2-digit": the flip must use the true year, not ICU's already-truncated 2-digit text - // (raw year -199 renders as "-99" under 2-digit; flipping -99 gives 100, not the correct - // 00 = (1 - (-199)) mod 100). - shouldBeOneOf(Intl.DateTimeFormat('en-u-ca-islamic-civil', { year: '2-digit', era: 'long', timeZone: 'UTC' }).format(Date.UTC(429, 0, 1)), ['Before Hijra 00', '00 Before Hijra']); + // Bun: this fork has no era override for a date before the epoch of an islamic-* or the coptic calendar + // (BUN-TEST-DIFFERENCES.md, "Intl.DateTimeFormat prints ICU's own era and year"). Every method prints ICU's era + // and year. In format() and formatToParts() upstream prints "Before Hijra" with the year 1 - year for islamic-*, + // and adds "Anno Martyrum" for coptic. Each expectation below that differs names upstream's value. The long era + // names are the CLDR 48 data of ICU 78. + // A date in February 622 is before the epoch of the three islamic calendars. ICU's year for it is 0. + // Bun: upstream expects '1 Before Hijra'. + shouldBeOneOfForICUVersion(78, Intl.DateTimeFormat('en-u-ca-islamic-civil', opts).format(Date.UTC(622, 1, 1)), ['0 Anno Hegirae']); + // Well before the epoch, not just at the boundary. + // Bun: upstream expects '333 Before Hijra'. + shouldBeOneOfForICUVersion(78, Intl.DateTimeFormat('en-u-ca-islamic-civil', opts).format(Date.UTC(300, 0, 1)), ['-332 Anno Hegirae']); + // year:"2-digit": ICU renders its year -199 as "-99". + // Bun: upstream expects '00 Before Hijra'. + shouldBeOneOfForICUVersion(78, Intl.DateTimeFormat('en-u-ca-islamic-civil', { year: '2-digit', era: 'long', timeZone: 'UTC' }).format(Date.UTC(429, 0, 1)), ['-99 Anno Hegirae']); // ICU < 78 has no distinct long-form "Anno Hegirae" era string for islamic-civil in this locale and falls back to the short form ("AH"). shouldBeOneOfForICUVersion(78, Intl.DateTimeFormat('en-u-ca-islamic-civil', opts).format(Date.UTC(650, 0, 1)), ['Anno Hegirae 29', '29 Anno Hegirae']); - shouldBeOneOf(Intl.DateTimeFormat('en-u-ca-islamic-tbla', opts).format(Date.UTC(622, 1, 1)), ['Before Hijra 1', '1 Before Hijra']); - shouldBeOneOf(Intl.DateTimeFormat('en-u-ca-islamic-umalqura', opts).format(Date.UTC(622, 1, 1)), ['Before Hijra 1', '1 Before Hijra']); - // coptic AM epoch is ISO 284-08-29T07:52:58.000Z. - shouldBeOneOf(Intl.DateTimeFormat('en-u-ca-coptic', opts).format(Date.UTC(200, 0, 1)), ['Anno Martyrum 85', '85 Anno Martyrum']); + // Bun: upstream expects '1 Before Hijra' for both. + shouldBeOneOfForICUVersion(78, Intl.DateTimeFormat('en-u-ca-islamic-tbla', opts).format(Date.UTC(622, 1, 1)), ['0 Anno Hegirae']); + shouldBeOneOfForICUVersion(78, Intl.DateTimeFormat('en-u-ca-islamic-umalqura', opts).format(Date.UTC(622, 1, 1)), ['0 Anno Hegirae']); + // Before the coptic epoch ICU 78 selects an era that CLDR 48 gives no name, so the era is empty. + // Bun: upstream expects '85 Anno Martyrum'. + shouldBeOneOfForICUVersion(78, Intl.DateTimeFormat('en-u-ca-coptic', opts).format(Date.UTC(200, 0, 1)), ['85 ']); // ICU < 78 has no distinct long-form "Anno Martyrum" era string for coptic in this locale and falls back to the generic "ERA1" placeholder. shouldBeOneOfForICUVersion(78, Intl.DateTimeFormat('en-u-ca-coptic', opts).format(Date.UTC(400, 0, 1)), ['Anno Martyrum 116', '116 Anno Martyrum']); } @@ -386,40 +391,44 @@ shouldBe(Intl.DateTimeFormat('en-u-ca-islamic-rgsa', { timeZone: 'America/Los_An const pre = Date.UTC(622, 1, 1); // islamic-civil pre-Hijra const base = { year: 'numeric', month: 'long', day: 'numeric', timeZone: 'UTC' }; - // Width ignored: long/short/narrow all produce the same text. - for (const width of ['long', 'short', 'narrow']) { + // The era part has the requested width. + // Bun: upstream expects 'Before Hijra' for the three widths. + for (const [width, expected] of [['long', 'Anno Hegirae'], ['short', 'AH'], ['narrow', 'AH']]) { const eraPart = Intl.DateTimeFormat('en-u-ca-islamic-civil', { ...base, era: width }).formatToParts(pre).find(p => p.type === 'era'); - shouldBe(eraPart && eraPart.value, 'Before Hijra'); + if (width !== 'long' || icuVersion >= 78) + shouldBe(eraPart && eraPart.value, expected); } - // The year part's VALUE must also be flipped -- formatToParts is a separate codepath from - // format()'s string splicing, with its own copy of this logic. + // The year part is ICU's year. + // Bun: upstream expects '1'. const yearPart = Intl.DateTimeFormat('en-u-ca-islamic-civil', { ...base, era: 'long' }).formatToParts(pre).find(p => p.type === 'year'); - shouldBe(yearPart && yearPart.value, '1'); + shouldBe(yearPart && yearPart.value, '0'); - // Locale ignored: ja gets the English text glued onto Japanese-formatted output. + // The text is in the formatter's language, with the era where the locale's pattern has it. + // Bun: upstream expects a text that includes the English 'Before Hijra'. const ja = Intl.DateTimeFormat('ja-u-ca-islamic-civil', { ...base, era: 'long' }).format(pre); - if (!ja.includes('Before Hijra')) - throw new Error(`islamic-civil pre-epoch ja locale expected (incorrect) "Before Hijra", got: ${ja}`); + if (icuVersion >= 78) + shouldBe(ja, 'AH0年7月10日'); - // Pattern position ignored: ja's real era position is first (see the post-epoch date, which - // ICU formats natively), but the override always appends last instead. + // ja puts the era first: a date after the epoch, for which ICU has an era name. const postParts = Intl.DateTimeFormat('ja-u-ca-coptic', { ...base, era: 'long' }).formatToParts(Date.UTC(400, 0, 1)).map(p => p.type); shouldBe(postParts[0], 'era'); // ICU4C-WORKAROUND: rdar://183226206 - below ICU 78, the coptic calendar emits a native - // era field even for pre-AM dates (ICU 78+ emits none there), so the override replaces - // that field in place instead of appending a new part. Only verifying current (78+) + // era field even for pre-AM dates (ICU 78+ emits none there). Only verifying current (78+) // behavior here; not pinning down the old-ICU shape. + // Bun: upstream expects a last part of type 'era', which its override appends. const preParts = Intl.DateTimeFormat('ja-u-ca-coptic', { ...base, era: 'long' }).formatToParts(Date.UTC(200, 0, 1)).map(p => p.type); if (icuVersion >= 78) - shouldBe(preParts[preParts.length - 1], 'era'); + shouldBe(preParts.join(), 'year,literal,month,literal,day,literal'); } // FIXME: rdar://182953351 (icu-issues/05) - formatRange/formatRangeToParts don't apply the // pre-Hijra/pre-AM override at all, unlike format()/formatToParts. Wait for ICU to select the // right era natively rather than threading this through more call sites; flip to `if (true)` // once fixed. +// Bun: this fork has no override, so the four methods print one text today, which +// bun-intl-datetimeformat-pre-epoch-era-one-text.js checks. This block stays disabled until ICU selects these eras. if (false) { const rangeOpts = { year: 'numeric', month: 'long', day: 'numeric', era: 'long', timeZone: 'UTC' }; diff --git a/JSTests/test262/expectations-linux.yaml b/JSTests/test262/expectations-linux.yaml index c4411996380b0..ef1f648ad88eb 100644 --- a/JSTests/test262/expectations-linux.yaml +++ b/JSTests/test262/expectations-linux.yaml @@ -58,6 +58,9 @@ test/built-ins/TypedArray/prototype/slice/speciesctor-return-same-buffer-with-of test/built-ins/Uint8Array/prototype/setFromBase64/trailing-garbage.js: default: 3 strict mode: 3 +test/intl402/DateTimeFormat/prototype/formatToParts/era.js: + default: 3 + strict mode: 3 test/intl402/Locale/prototype/getHourCycles/region-priority.js: default: 3 strict mode: 3 diff --git a/JSTests/test262/expectations.yaml b/JSTests/test262/expectations.yaml index baf80b1d1e975..b3f8c49ee9abc 100644 --- a/JSTests/test262/expectations.yaml +++ b/JSTests/test262/expectations.yaml @@ -55,6 +55,9 @@ test/built-ins/Proxy/construct/trap-is-not-callable-realm.js: test/built-ins/Uint8Array/prototype/setFromBase64/trailing-garbage.js: default: 3 strict mode: 3 +test/intl402/DateTimeFormat/prototype/formatToParts/era.js: + default: 3 + strict mode: 3 test/intl402/Locale/prototype/getHourCycles/region-priority.js: default: 3 strict mode: 3 diff --git a/Source/JavaScriptCore/runtime/IntlDateTimeFormat.cpp b/Source/JavaScriptCore/runtime/IntlDateTimeFormat.cpp index 04a3afd3c3b7c..169ab180eb02a 100644 --- a/Source/JavaScriptCore/runtime/IntlDateTimeFormat.cpp +++ b/Source/JavaScriptCore/runtime/IntlDateTimeFormat.cpp @@ -61,6 +61,13 @@ #include #define U_HIDE_DRAFT_API 1 +#if USE(BUN_JSC_ADDITIONS) +// This fork has no era override for a date before the epoch of an islamic or the coptic calendar: every method prints +// ICU's own era and year (JSTests/BUN-TEST-DIFFERENCES.md, "Intl.DateTimeFormat prints ICU's own era and year"). +// A merge that brings upstream's override back does not compile. +#pragma GCC poison proposalEraOverrideString formatWithEraOverride flippedEraOverrideYear +#endif + WTF_ALLOW_UNSAFE_BUFFER_USAGE_BEGIN namespace JSC { @@ -1395,139 +1402,6 @@ static void LEGACY_NODELETE replaceNarrowNoBreakSpaceOrThinSpaceWithNormalSpace( } } -// ICU4C-WORKAROUND: rdar://182953351 - Islamic calendars never expose a second era -// (architecturally locked to one); Coptic has a second era but never selects it. -// Substitute CLDR's "Before Hijra"/"Anno Martyrum" text ourselves. islamic-tbla's -// epoch differs from civil/umalqura's by a day, hence the separate branch. -// needsYearFlip: islamic's raw year has no year-zero skip, unlike Gregorian's native "Before -// Christ" or Temporal's own eraYear for this era (both use `1 - year`). Coptic's year is already correct. -// FIXME: doesn't cover formatRange()/formatRangeToParts(); wait for ICU -// to fix era selection natively rather than extending this further. -struct EraOverride { - ASCIILiteral text; - bool needsYearFlip { false }; -}; -static std::optional proposalEraOverrideString(const String& calendar, double epochMs) -{ - if (calendar == "islamic-civil"_s || calendar == "islamic-umalqura"_s) { - constexpr double islamicEpoch = -42521558822000.0; // ISO 622-07-19T07:52:58.000Z - if (epochMs < islamicEpoch) - return EraOverride { "Before Hijra"_s, true }; - return std::nullopt; - } - if (calendar == "islamic-tbla"_s) { - constexpr double islamicTblaEpoch = -42521645222000.0; // ISO 622-07-18T07:52:58.000Z - if (epochMs < islamicTblaEpoch) - return EraOverride { "Before Hijra"_s, true }; - return std::nullopt; - } - if (calendar == "coptic"_s) { - constexpr double copticEpoch = -53184182822000.0; // ISO 284-08-29T07:52:58.000Z - if (epochMs < copticEpoch) - return EraOverride { "Anno Martyrum"_s }; - return std::nullopt; - } - return std::nullopt; -} - -// ICU4C-WORKAROUND: rdar://182953351 - the flipped "Before Hijra" year (1 - year, matching -// Temporal's eraYear and ICU's native "Before Christ"), read from a clone of the format's own -// calendar rather than parsed from ICU's rendered text -- under year:"2-digit" that text is -// already lossily truncated, so flipping it instead of the real value would be wrong. -static std::optional flippedEraOverrideYear(UDateFormat* format, double value, bool yearIsTwoDigit) -{ - UErrorCode status = U_ZERO_ERROR; - auto calendar = std::unique_ptr>(ucal_clone(udat_getCalendar(format), &status)); - if (U_FAILURE(status)) [[unlikely]] - return std::nullopt; - ucal_setMillis(calendar.get(), value, &status); - int32_t year = ucal_get(calendar.get(), UCAL_YEAR, &status); - if (U_FAILURE(status)) [[unlikely]] - return std::nullopt; - int32_t flipped = 1 - year; - if (!yearIsTwoDigit) - return String::number(flipped); - int32_t twoDigit = ((flipped % 100) + 100) % 100; - auto digits = String::number(twoDigit); - return digits.length() < 2 ? makeString('0', digits) : digits; -} - -// ICU4C-WORKAROUND: rdar://182953351 - see proposalEraOverrideString. Shared by both format() overloads. -static JSValue formatWithEraOverride(JSGlobalObject* globalObject, UDateFormat* format, double value, const EraOverride& eraOverride, bool yearIsTwoDigit) -{ - VM& vm = globalObject->vm(); - auto scope = DECLARE_THROW_SCOPE(vm); - - UErrorCode fieldsStatus = U_ZERO_ERROR; - auto fields = std::unique_ptr(ufieldpositer_open(&fieldsStatus)); - if (U_FAILURE(fieldsStatus)) [[unlikely]] - return throwTypeError(globalObject, scope, "failed to open field position iterator"_s); - Vector result; - fieldsStatus = callBufferProducingFunction(udat_formatForFields, format, value, result, fields.get()); - if (U_FAILURE(fieldsStatus)) [[unlikely]] - return throwTypeError(globalObject, scope, "failed to format date value"_s); - replaceNarrowNoBreakSpaceOrThinSpaceWithNormalSpace(result); - - // Find the era field, and (only if needed) the year field, so both splice in one pass. - int32_t eraBegin = -1, eraEnd = -1; - int32_t yearBegin = -1, yearEnd = -1; - int32_t beginIndex = 0, endIndex = 0; - while (true) { - auto fieldType = ufieldpositer_next(fields.get(), &beginIndex, &endIndex); - if (fieldType < 0) - break; - if (fieldType == UDAT_ERA_FIELD) { - eraBegin = beginIndex; - eraEnd = endIndex; - } else if (eraOverride.needsYearFlip && fieldType == UDAT_YEAR_FIELD) { - yearBegin = beginIndex; - yearEnd = endIndex; - } - } - - StringView resultView(result.span()); - if (eraBegin < 0) { - // ICU can already leave a trailing space where the empty era slot would go; - // only add our own separator if one isn't already there. - StringBuilder builder; - builder.append(resultView); - if (resultView.isEmpty() || !WTF::isUnicodeWhitespace(resultView[resultView.length() - 1])) - builder.append(' '); - builder.append(String(eraOverride.text)); - return jsString(vm, builder.toString()); - } - - // Replace the era field, and the year field too when needsYearFlip (see struct above). - String yearReplacement; - if (yearBegin >= 0) { - if (auto flipped = flippedEraOverrideYear(format, value, yearIsTwoDigit)) - yearReplacement = *flipped; - } - - struct ReplacedSpan { - int32_t begin; - int32_t end; - String replacement; - }; - ReplacedSpan spans[2] = { { eraBegin, eraEnd, String(eraOverride.text) }, { -1, -1, { } } }; - if (!yearReplacement.isNull()) - spans[1] = { yearBegin, yearEnd, yearReplacement }; - if (spans[1].begin >= 0 && spans[1].begin < spans[0].begin) - std::swap(spans[0], spans[1]); - - StringBuilder builder; - int32_t cursor = 0; - for (auto& span : spans) { - if (span.begin < 0) - continue; - builder.append(resultView.substring(cursor, span.begin - cursor)); - builder.append(span.replacement); - cursor = span.end; - } - builder.append(resultView.substring(cursor, resultView.length() - cursor)); - return jsString(vm, builder.toString()); -} - // https://tc39.es/proposal-temporal/#sec-formatdatetime JSValue IntlDateTimeFormat::format(JSGlobalObject* globalObject, double value) const { @@ -1539,11 +1413,6 @@ JSValue IntlDateTimeFormat::format(JSGlobalObject* globalObject, double value) c if (!std::isfinite(value)) return throwRangeError(globalObject, scope, "date value is not finite in DateTimeFormat format()"_s); - // ICU4C-WORKAROUND: rdar://182953351 - see proposalEraOverrideString; only when era requested. - auto eraOverride = m_impl->m_era != Era::None ? proposalEraOverrideString(m_impl->m_calendar, value) : std::nullopt; - if (eraOverride) - RELEASE_AND_RETURN(scope, formatWithEraOverride(globalObject, m_impl->m_dateFormat.get(), value, *eraOverride, m_impl->m_year == Year::TwoDigit)); - Vector result; auto status = callBufferProducingFunction(udat_format, m_impl->m_dateFormat.get(), value, result, nullptr); if (U_FAILURE(status)) [[unlikely]] @@ -1616,7 +1485,7 @@ static ASCIILiteral partTypeString(UDateFormatField field) return "unknown"_s; } -static JSValue buildFormattedDateTimeParts(JSGlobalObject* globalObject, UDateFormat* format, double value, JSString* sourceType, std::optional eraOverride = std::nullopt, bool yearIsTwoDigit = false) +static JSValue buildFormattedDateTimeParts(JSGlobalObject* globalObject, UDateFormat* format, double value, JSString* sourceType) { VM& vm = globalObject->vm(); auto scope = DECLARE_THROW_SCOPE(vm); @@ -1646,7 +1515,6 @@ static JSValue buildFormattedDateTimeParts(JSGlobalObject* globalObject, UDateFo int32_t previousEndIndex = 0; int32_t beginIndex = 0; int32_t endIndex = 0; - bool sawEra = false; while (previousEndIndex < resultLength) { auto fieldType = ufieldpositer_next(fields.get(), &beginIndex, &endIndex); if (fieldType < 0) @@ -1664,43 +1532,10 @@ static JSValue buildFormattedDateTimeParts(JSGlobalObject* globalObject, UDateFo if (fieldType >= 0) { auto type = jsNontrivialString(vm, partTypeString(UDateFormatField(fieldType))); - JSString* valueStr; - { - // ICU4C-WORKAROUND: rdar://182953351 - replace era field text with override for islamic-* pre-Hijra / coptic pre-AM. - if (eraOverride && fieldType == UDAT_ERA_FIELD) { - valueStr = jsNontrivialString(vm, String(eraOverride->text)); - sawEra = true; - } else if (eraOverride && eraOverride->needsYearFlip && fieldType == UDAT_YEAR_FIELD) { - // ICU4C-WORKAROUND: rdar://182953351 - flip the year the same way formatWithEraOverride does. - auto flipped = flippedEraOverrideYear(format, value, yearIsTwoDigit); - valueStr = flipped ? jsString(vm, *flipped) : jsString(vm, resultStringView.substring(beginIndex, endIndex - beginIndex)); - } else - valueStr = jsString(vm, resultStringView.substring(beginIndex, endIndex - beginIndex)); - } + auto value = jsString(vm, resultStringView.substring(beginIndex, endIndex - beginIndex)); JSObject* part = sourceType - ? createIntlPartObjectWithSource(globalObject, type, valueStr, sourceType) - : createIntlPartObject(globalObject, type, valueStr); - parts->putDirectIndex(globalObject, parts->length(), part); - RETURN_IF_EXCEPTION(scope, { }); - } - } - - { - // ICU4C-WORKAROUND: rdar://182953351 - append era override as a distinct part when ICU emitted no era field (coptic pre-AM). - if (eraOverride && !sawEra) { - if (resultStringView.isEmpty() || !WTF::isUnicodeWhitespace(resultStringView[resultLength - 1])) { - auto separator = jsSingleCharacterString(vm, static_cast(' ')); - JSObject* separatorPart = sourceType - ? createIntlPartObjectWithSource(globalObject, literalString, separator, sourceType) - : createIntlPartObject(globalObject, literalString, separator); - parts->putDirectIndex(globalObject, parts->length(), separatorPart); - RETURN_IF_EXCEPTION(scope, { }); - } - auto type = jsNontrivialString(vm, "era"_s); - auto valueStr = jsNontrivialString(vm, String(eraOverride->text)); - JSObject* part = sourceType - ? createIntlPartObjectWithSource(globalObject, type, valueStr, sourceType) - : createIntlPartObject(globalObject, type, valueStr); + ? createIntlPartObjectWithSource(globalObject, type, value, sourceType) + : createIntlPartObject(globalObject, type, value); parts->putDirectIndex(globalObject, parts->length(), part); RETURN_IF_EXCEPTION(scope, { }); } @@ -2577,11 +2412,6 @@ JSValue IntlDateTimeFormat::format(JSGlobalObject* globalObject, JSValue x) cons if (!tempFormat) [[unlikely]] return throwTypeError(globalObject, scope, "DateTimeFormat has no fields applicable to this Temporal type"_s); - // ICU4C-WORKAROUND: rdar://182953351 - see proposalEraOverrideString; only when era requested. - auto eraOverride = m_impl->m_era != Era::None ? proposalEraOverrideString(m_impl->m_calendar, record.value) : std::nullopt; - if (eraOverride) - RELEASE_AND_RETURN(scope, formatWithEraOverride(globalObject, tempFormat, record.value, *eraOverride, m_impl->m_year == Year::TwoDigit)); - Vector result; auto fmtStatus = callBufferProducingFunction(udat_format, tempFormat, record.value, result, nullptr); if (U_FAILURE(fmtStatus)) [[unlikely]] @@ -2611,9 +2441,7 @@ JSValue IntlDateTimeFormat::formatToParts(JSGlobalObject* globalObject, JSValue return throwTypeError(globalObject, scope, "DateTimeFormat has no fields applicable to this Temporal type"_s); } - // ICU4C-WORKAROUND: rdar://182953351 - see proposalEraOverrideString; only when era requested. - auto eraOverride = m_impl->m_era != Era::None ? proposalEraOverrideString(m_impl->m_calendar, record.value) : std::nullopt; - RELEASE_AND_RETURN(scope, buildFormattedDateTimeParts(globalObject, fmt, record.value, nullptr, eraOverride, m_impl->m_year == Year::TwoDigit)); + RELEASE_AND_RETURN(scope, buildFormattedDateTimeParts(globalObject, fmt, record.value, nullptr)); } std::unique_ptr