Always report unmeasurable variance for mapped types

[ahejlsberg]

Fixes #49852.

[ahejlsberg]

@typescript-bot test this
@typescript-bot user test this inline
@typescript-bot run dt
@typescript-bot perf test faster

[typescript-bot]

Heya @ahejlsberg, I've started to run the diff-based user code test suite on this PR at e9b71d0. You can monitor the build here.

Update: The results are in!

[typescript-bot]

Heya @ahejlsberg, I've started to run the extended test suite on this PR at e9b71d0. You can monitor the build here.

[typescript-bot]

Heya @ahejlsberg, I've started to run the parallelized Definitely Typed test suite on this PR at e9b71d0. You can monitor the build here.

[typescript-bot]

Heya @ahejlsberg, I've started to run the abridged perf test suite on this PR at e9b71d0. You can monitor the build here.

Update: The results are in!

[typescript-bot]

@ahejlsberg
The results of the perf run you requested are in!

Here they are:

Comparison Report - main..49894

Metric	main	49894	Delta	Best	Worst
Angular - node (v14.15.1, x64)
Memory used	335,438k (± 0.01%)	335,412k (± 0.01%)	-26k (- 0.01%)	335,352k	335,481k
Parse Time	2.06s (± 0.60%)	2.06s (± 0.55%)	+0.00s (+ 0.19%)	2.04s	2.09s
Bind Time	0.90s (± 0.44%)	0.90s (± 0.72%)	+0.00s (+ 0.33%)	0.89s	0.91s
Check Time	5.78s (± 0.52%)	5.80s (± 0.53%)	+0.02s (+ 0.42%)	5.76s	5.89s
Emit Time	6.36s (± 0.61%)	6.44s (± 0.69%)	+0.08s (+ 1.23%)	6.36s	6.52s
Total Time	15.10s (± 0.43%)	15.21s (± 0.40%)	+0.11s (+ 0.72%)	15.09s	15.33s
Compiler-Unions - node (v14.15.1, x64)
Memory used	192,832k (± 0.01%)	192,787k (± 0.08%)	-45k (- 0.02%)	192,150k	192,923k
Parse Time	0.85s (± 0.78%)	0.85s (± 0.55%)	-0.00s (- 0.00%)	0.84s	0.86s
Bind Time	0.57s (± 1.01%)	0.57s (± 0.83%)	0.00s ( 0.00%)	0.56s	0.58s
Check Time	6.69s (± 0.40%)	6.70s (± 0.46%)	+0.01s (+ 0.10%)	6.63s	6.75s
Emit Time	2.48s (± 0.79%)	2.51s (± 1.15%)	+0.03s (+ 1.29%)	2.46s	2.59s
Total Time	10.59s (± 0.41%)	10.63s (± 0.52%)	+0.04s (+ 0.40%)	10.51s	10.72s
Monaco - node (v14.15.1, x64)
Memory used	325,716k (± 0.01%)	325,698k (± 0.01%)	-19k (- 0.01%)	325,656k	325,761k
Parse Time	1.58s (± 0.66%)	1.59s (± 0.84%)	+0.01s (+ 0.38%)	1.56s	1.62s
Bind Time	0.79s (± 0.82%)	0.79s (± 0.86%)	+0.01s (+ 0.89%)	0.79s	0.82s
Check Time	5.67s (± 0.54%)	5.69s (± 0.59%)	+0.02s (+ 0.35%)	5.60s	5.75s
Emit Time	3.37s (± 1.12%)	3.37s (± 0.78%)	+0.00s (+ 0.12%)	3.32s	3.43s
Total Time	11.41s (± 0.44%)	11.44s (± 0.47%)	+0.03s (+ 0.31%)	11.30s	11.53s
TFS - node (v14.15.1, x64)
Memory used	288,800k (± 0.01%)	288,781k (± 0.01%)	-20k (- 0.01%)	288,731k	288,820k
Parse Time	1.34s (± 2.13%)	1.33s (± 1.27%)	-0.01s (- 0.74%)	1.30s	1.37s
Bind Time	0.75s (± 1.27%)	0.77s (± 3.53%)	+0.02s (+ 2.95%)	0.74s	0.86s
Check Time	5.34s (± 0.41%)	5.33s (± 0.43%)	-0.01s (- 0.13%)	5.28s	5.38s
Emit Time	3.57s (± 2.08%)	3.53s (± 1.91%)	-0.03s (- 0.95%)	3.45s	3.69s
Total Time	11.00s (± 0.73%)	10.97s (± 0.78%)	-0.03s (- 0.24%)	10.81s	11.19s
material-ui - node (v14.15.1, x64)
Memory used	446,674k (± 0.00%)	454,933k (± 0.00%)	+8,259k (+ 1.85%)	454,884k	454,981k
Parse Time	1.87s (± 0.40%)	1.88s (± 0.66%)	+0.01s (+ 0.54%)	1.86s	1.92s
Bind Time	0.73s (± 0.61%)	0.73s (± 1.71%)	+0.00s (+ 0.27%)	0.70s	0.76s
Check Time	13.11s (± 0.72%)	13.33s (± 0.62%)	+0.22s (+ 1.69%)	13.11s	13.50s
Emit Time	0.00s (± 0.00%)	0.00s (± 0.00%)	0.00s ( NaN%)	0.00s	0.00s
Total Time	15.71s (± 0.61%)	15.95s (± 0.57%)	+0.23s (+ 1.49%)	15.70s	16.09s
xstate - node (v14.15.1, x64)
Memory used	541,296k (± 0.00%)	546,870k (± 0.00%)	+5,574k (+ 1.03%)	546,846k	546,922k
Parse Time	2.60s (± 0.50%)	2.60s (± 0.26%)	-0.01s (- 0.23%)	2.58s	2.61s
Bind Time	1.15s (± 1.69%)	1.16s (± 0.99%)	+0.01s (+ 0.96%)	1.13s	1.19s
Check Time	1.55s (± 0.38%)	1.81s (± 0.79%)	+0.26s (+17.01%)	1.78s	1.84s
Emit Time	0.07s (± 4.66%)	0.07s (± 4.66%)	0.00s ( 0.00%)	0.07s	0.08s
Total Time	5.37s (± 0.53%)	5.64s (± 0.26%)	+0.27s (+ 5.01%)	5.61s	5.68s

System

Machine Name	ts-ci-ubuntu
Platform	linux 4.4.0-210-generic
Architecture	x64
Available Memory	16 GB
Available Memory	15 GB
CPUs	4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz

Hosts

• node (v14.15.1, x64)

Scenarios

• Angular - node (v14.15.1, x64)
• Compiler-Unions - node (v14.15.1, x64)
• Monaco - node (v14.15.1, x64)
• TFS - node (v14.15.1, x64)
• material-ui - node (v14.15.1, x64)
• xstate - node (v14.15.1, x64)

Benchmark	Name	Iterations
Current	49894	10
Baseline	main	10

Developer Information:

Download Benchmark

[typescript-bot]

Heya @ahejlsberg, I've run the RWC suite on this PR - assuming you're on the TS core team, you can view the resulting diff here.

[typescript-bot]

@ahejlsberg
The results of the user tests run you requested are in!

Here they are:

Comparison Report - main..refs/pull/49894/merge

fp-ts

2 of 4 projects failed to build with the old tsc

tsconfig.json

• error TS2322: Type 'Readonly<Record<string, A>>' is not assignable to type 'Readonly<Record<K, A>>'.
  • test/ReadonlyRecord.ts#L516
• error TS2322: Type 'Record<string, A>' is not assignable to type 'Record<K, A>'.
  • test/Record.ts#L486

[ahejlsberg]

There's definitely some performance impact to this. Check time increases 2% in material-ui and 17% in xstate.

[ahejlsberg]

Hmm, yeah, some pretty bad effects in lodash. Check time for this short example goes up 500% and produces errors where there were none before.

import { Collection} from "lodash";

declare let ca: Collection<any>;
declare let cs: Collection<string>;

ca = cs;
cs = ca;

[ahejlsberg]

I'm going to close this PR as it simply isn't feasible from a performance perspective.