Skip to content

Throwing RecursionOverflow in TreePickler can lead to infinite loop #16447

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
WojciechMazur opened this issue Nov 30, 2022 · 2 comments
Open

Throwing RecursionOverflow in TreePickler can lead to infinite loop #16447

WojciechMazur opened this issue Nov 30, 2022 · 2 comments
Labels
area:pickling itype:bug itype:crash stat:needs minimization Needs a self contained minimization

Comments

@WojciechMazur
Copy link
Contributor

WojciechMazur commented Nov 30, 2022 •
edited
Loading

When migrating one of Scala2 projects into Scala I ended up in infinite compilation loop. I was not yet able to minimize it, but I'm providing a thread dump of a compiler.

Compiler version

3.2.1

Minimized code

No minimization yet

Subsequent thread dump pointed to the same last method, so probably it's impossible to leave dotty.tools.dotc.core.Types$ThisType.underlying(Types.scala:2938) function call

Thread dump (click arrow to expand)

"pool-11-thread-13" #356 prio=5 os_prio=0 cpu=6580194,19ms elapsed=6618,22s tid=0x00007f081cfc4800 nid=0x77889 runnable  [0x00007f07e19c6000]
   java.lang.Thread.State: RUNNABLE
	at dotty.tools.dotc.core.Types$ThisType.underlying(Types.scala:2938)
	at dotty.tools.dotc.core.Types$TypeProxy.superType(Types.scala:1987)
	at dotty.tools.dotc.core.Types$Type.showPrefixSafely$1(Types.scala:887)
	at dotty.tools.dotc.core.Types$Type.findMember$$anonfun$1(Types.scala:891)
	at dotty.tools.dotc.core.Types$Type$$Lambda$8774/0x0000000102213c40.apply(Unknown Source)
	at dotty.tools.dotc.core.RecursionOverflow.explanation(TypeErrors.scala:47)
	at dotty.tools.dotc.core.RecursionOverflow.opsString$$anonfun$1(TypeErrors.scala:69)
	at dotty.tools.dotc.core.RecursionOverflow$$Lambda$8779/0x0000000102241840.apply(Unknown Source)
	at scala.collection.immutable.List.map(List.scala:246)
	at dotty.tools.dotc.core.RecursionOverflow.opsString(TypeErrors.scala:69)
	at dotty.tools.dotc.core.RecursionOverflow.produceMessage$$anonfun$4(TypeErrors.scala:79)
	at dotty.tools.dotc.core.RecursionOverflow$$Lambda$8775/0x0000000102210c40.apply(Unknown Source)
	at dotty.tools.dotc.reporting.NoExplanation.msg(Message.scala:154)
	at dotty.tools.dotc.reporting.Message.message(Message.scala:102)
	at dotty.tools.dotc.reporting.Message.isNonSensical(Message.scala:114)
	at dotty.tools.dotc.reporting.HideNonSensicalMessages.isHidden(HideNonSensicalMessages.scala:16)
	at dotty.tools.dotc.reporting.HideNonSensicalMessages.isHidden$(HideNonSensicalMessages.scala:10)
	at dotty.tools.dotc.reporting.AbstractReporter.isHidden(AbstractReporter.scala:8)
	at dotty.tools.dotc.reporting.Reporter.issueUnconfigured(Reporter.scala:155)
	at dotty.tools.dotc.reporting.Reporter.go$1(Reporter.scala:180)
	at dotty.tools.dotc.reporting.Reporter.issueIfNotSuppressed(Reporter.scala:199)
	at dotty.tools.dotc.reporting.Reporter.report(Reporter.scala:202)
	at dotty.tools.dotc.report$.error(report.scala:63)
	at dotty.tools.dotc.core.tasty.TreePickler.pickleTree(TreePickler.scala:662)
	at dotty.tools.dotc.core.tasty.TreePickler.pickleTree$$anonfun$3(TreePickler.scala:432)
	at dotty.tools.dotc.core.tasty.TreePickler.pickleTree$$anonfun$adapted$3(TreePickler.scala:434)
	at dotty.tools.dotc.core.tasty.TreePickler$$Lambda$6751/0x0000000101beb040.apply(Unknown Source)
	at scala.Function0.apply$mcV$sp(Function0.scala:42)
	at dotty.tools.dotc.core.tasty.TreePickler.withLength(TreePickler.scala:58)
	at dotty.tools.dotc.core.tasty.TreePickler.pickleTree(TreePickler.scala:434)
	at dotty.tools.dotc.core.tasty.TreePickler.pickleTreeUnlessEmpty(TreePickler.scala:321)
	at dotty.tools.dotc.core.tasty.TreePickler.pickleDef$$anonfun$1(TreePickler.scala:338)
	at dotty.tools.dotc.core.tasty.TreePickler.pickleDef$$anonfun$adapted$1(TreePickler.scala:340)
	at dotty.tools.dotc.core.tasty.TreePickler$$Lambda$6749/0x0000000101bea840.apply(Unknown Source)
	at scala.Function0.apply$mcV$sp(Function0.scala:42)
	at dotty.tools.dotc.core.tasty.TreePickler.withLength(TreePickler.scala:58)
	at dotty.tools.dotc.core.tasty.TreePickler.pickleDef(TreePickler.scala:340)
	at dotty.tools.dotc.core.tasty.TreePickler.pickleTree(TreePickler.scala:550)
	at dotty.tools.dotc.core.tasty.TreePickler.pickleTree$$anonfun$9$$anonfun$1(TreePickler.scala:474)
	at dotty.tools.dotc.core.tasty.TreePickler$$Lambda$6779/0x0000000101bf9840.applyVoid(Unknown Source)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
	at scala.collection.immutable.List.foreach(List.scala:333)
	at dotty.tools.dotc.core.tasty.TreePickler.pickleTree$$anonfun$9(TreePickler.scala:474)
	at dotty.tools.dotc.core.tasty.TreePickler.pickleTree$$anonfun$adapted$8(TreePickler.scala:474)
	at dotty.tools.dotc.core.tasty.TreePickler$$Lambda$6778/0x0000000101bf8440.apply(Unknown Source)
	at scala.Function0.apply$mcV$sp(Function0.scala:42)
	at dotty.tools.dotc.core.tasty.TreePickler.withLength(TreePickler.scala:58)
	at dotty.tools.dotc.core.tasty.TreePickler.pickleTree(TreePickler.scala:474)
	at dotty.tools.dotc.core.tasty.TreePickler.pickleTreeUnlessEmpty(TreePickler.scala:321)
	at dotty.tools.dotc.core.tasty.TreePickler.pickleDef$$anonfun$1(TreePickler.scala:338)
	at dotty.tools.dotc.core.tasty.TreePickler.pickleDef$$anonfun$adapted$1(TreePickler.scala:340)
	at dotty.tools.dotc.core.tasty.TreePickler$$Lambda$6749/0x0000000101bea840.apply(Unknown Source)
	at scala.Function0.apply$mcV$sp(Function0.scala:42)
	at dotty.tools.dotc.core.tasty.TreePickler.withLength(TreePickler.scala:58)
	at dotty.tools.dotc.core.tasty.TreePickler.pickleDef(TreePickler.scala:340)
	at dotty.tools.dotc.core.tasty.TreePickler.pickleTree(TreePickler.scala:565)
	at dotty.tools.dotc.core.tasty.TreePickler.pickleStats$$anonfun$2(TreePickler.scala:365)
	at dotty.tools.dotc.core.tasty.TreePickler$$Lambda$6745/0x0000000101be9040.applyVoid(Unknown Source)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
	at scala.collection.immutable.List.foreach(List.scala:333)
	at dotty.tools.dotc.core.tasty.TreePickler.pickleStats(TreePickler.scala:365)
	at dotty.tools.dotc.core.tasty.TreePickler.pickleTree$$anonfun$26(TreePickler.scala:591)
	at dotty.tools.dotc.core.tasty.TreePickler.pickleTree$$anonfun$adapted$24(TreePickler.scala:592)
	at dotty.tools.dotc.core.tasty.TreePickler$$Lambda$6761/0x0000000101bf1040.apply(Unknown Source)
	at scala.Function0.apply$mcV$sp(Function0.scala:42)
	at dotty.tools.dotc.core.tasty.TreePickler.withLength(TreePickler.scala:58)
	at dotty.tools.dotc.core.tasty.TreePickler.pickleTree(TreePickler.scala:592)
	at dotty.tools.dotc.core.tasty.TreePickler.pickleDef$$anonfun$1(TreePickler.scala:335)
	at dotty.tools.dotc.core.tasty.TreePickler.pickleDef$$anonfun$adapted$1(TreePickler.scala:340)
	at dotty.tools.dotc.core.tasty.TreePickler$$Lambda$6749/0x0000000101bea840.apply(Unknown Source)
	at scala.Function0.apply$mcV$sp(Function0.scala:42)
	at dotty.tools.dotc.core.tasty.TreePickler.withLength(TreePickler.scala:58)
	at dotty.tools.dotc.core.tasty.TreePickler.pickleDef(TreePickler.scala:340)
	at dotty.tools.dotc.core.tasty.TreePickler.pickleTree(TreePickler.scala:567)
	at dotty.tools.dotc.core.tasty.TreePickler.pickleStats$$anonfun$2(TreePickler.scala:365)
	at dotty.tools.dotc.core.tasty.TreePickler$$Lambda$6745/0x0000000101be9040.applyVoid(Unknown Source)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
	at scala.collection.immutable.List.foreach(List.scala:333)
	at dotty.tools.dotc.core.tasty.TreePickler.pickleStats(TreePickler.scala:365)
	at dotty.tools.dotc.core.tasty.TreePickler.pickleTree$$anonfun$29(TreePickler.scala:607)
	at dotty.tools.dotc.core.tasty.TreePickler.pickleTree$$anonfun$adapted$27(TreePickler.scala:607)
	at dotty.tools.dotc.core.tasty.TreePickler$$Lambda$6743/0x0000000101be7840.apply(Unknown Source)
	at scala.Function0.apply$mcV$sp(Function0.scala:42)
	at dotty.tools.dotc.core.tasty.TreePickler.withLength(TreePickler.scala:58)
	at dotty.tools.dotc.core.tasty.TreePickler.pickleTree(TreePickler.scala:607)
	at dotty.tools.dotc.core.tasty.TreePickler.pickle$$anonfun$1(TreePickler.scala:779)
	at dotty.tools.dotc.core.tasty.TreePickler$$Lambda$6742/0x0000000101be7040.applyVoid(Unknown Source)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
	at scala.collection.immutable.List.foreach(List.scala:333)
	at dotty.tools.dotc.core.tasty.TreePickler.pickle(TreePickler.scala:779)
	at dotty.tools.dotc.transform.Pickler.run$$anonfun$1$$anonfun$1(Pickler.scala:72)
	at dotty.tools.dotc.transform.Pickler$$Lambda$6741/0x0000000101be6840.applyVoid(Unknown Source)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
	at scala.collection.immutable.List.foreach(List.scala:333)
	at dotty.tools.dotc.transform.Pickler.run$$anonfun$1(Pickler.scala:110)
	at dotty.tools.dotc.transform.Pickler$$Lambda$6738/0x0000000101be4040.applyVoid(Unknown Source)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
	at scala.collection.immutable.List.foreach(List.scala:333)
	at dotty.tools.dotc.transform.Pickler.run(Pickler.scala:110)
	at dotty.tools.dotc.core.Phases$Phase.runOn$$anonfun$1(Phases.scala:316)
	at dotty.tools.dotc.core.Phases$Phase$$Lambda$6574/0x0000000100ff4040.apply(Unknown Source)
	at scala.collection.immutable.List.map(List.scala:250)
	at dotty.tools.dotc.core.Phases$Phase.runOn(Phases.scala:320)
	at dotty.tools.dotc.transform.Pickler.runOn(Pickler.scala:115)
	at dotty.tools.dotc.Run.runPhases$1$$anonfun$1(Run.scala:233)
	at dotty.tools.dotc.Run$$Lambda$5632/0x00000001018f1840.applyVoid(Unknown Source)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
	at scala.collection.ArrayOps$.foreach$extension(ArrayOps.scala:1321)
	at dotty.tools.dotc.Run.runPhases$1(Run.scala:244)
	at dotty.tools.dotc.Run.compileUnits$$anonfun$1(Run.scala:252)
	at dotty.tools.dotc.Run.compileUnits$$anonfun$adapted$1(Run.scala:261)
	at dotty.tools.dotc.Run$$Lambda$5594/0x00000001018c3440.apply(Unknown Source)
	at dotty.tools.dotc.util.Stats$.maybeMonitored(Stats.scala:68)
	at dotty.tools.dotc.Run.compileUnits(Run.scala:261)
	at dotty.tools.dotc.Run.compileSources(Run.scala:185)
	at dotty.tools.dotc.Run.compile(Run.scala:169)
	at dotty.tools.dotc.Driver.doCompile(Driver.scala:35)
	at dotty.tools.xsbt.CompilerBridgeDriver.run(CompilerBridgeDriver.java:88)
	- locked <0x00000000e036d438> (a dotty.tools.xsbt.CompilerBridgeDriver)
	at dotty.tools.xsbt.CompilerBridge.run(CompilerBridge.java:22)
	at sbt.internal.inc.AnalyzingCompiler.compile(AnalyzingCompiler.scala:91)
	at sbt.internal.inc.MixedAnalyzingCompiler.$anonfun$compile$7(MixedAnalyzingCompiler.scala:193)
	at sbt.internal.inc.MixedAnalyzingCompiler$$Lambda$5325/0x00000001015f0c40.apply$mcV$sp(Unknown Source)
	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
	at sbt.internal.inc.MixedAnalyzingCompiler.timed(MixedAnalyzingCompiler.scala:248)
	at sbt.internal.inc.MixedAnalyzingCompiler.$anonfun$compile$4(MixedAnalyzingCompiler.scala:183)
	at sbt.internal.inc.MixedAnalyzingCompiler.$anonfun$compile$4$adapted(MixedAnalyzingCompiler.scala:163)
	at sbt.internal.inc.MixedAnalyzingCompiler$$Lambda$5321/0x00000001015ed040.apply(Unknown Source)
	at sbt.internal.inc.JarUtils$.withPreviousJar(JarUtils.scala:239)
	at sbt.internal.inc.MixedAnalyzingCompiler.compileScala$1(MixedAnalyzingCompiler.scala:163)
	at sbt.internal.inc.MixedAnalyzingCompiler.compile(MixedAnalyzingCompiler.scala:211)
	at sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileInternal$1(IncrementalCompilerImpl.scala:534)
	at sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileInternal$1$adapted(IncrementalCompilerImpl.scala:534)
	at sbt.internal.inc.IncrementalCompilerImpl$$Lambda$3110/0x0000000100ee7c40.apply(Unknown Source)
	at sbt.internal.inc.Incremental$.$anonfun$apply$5(Incremental.scala:179)
	at sbt.internal.inc.Incremental$.$anonfun$apply$5$adapted(Incremental.scala:177)
	at sbt.internal.inc.Incremental$$$Lambda$3117/0x0000000100ef7040.apply(Unknown Source)
	at sbt.internal.inc.Incremental$$anon$2.run(Incremental.scala:463)
	at sbt.internal.inc.IncrementalCommon$CycleState.next(IncrementalCommon.scala:116)
	at sbt.internal.inc.IncrementalCommon$$anon$1.next(IncrementalCommon.scala:56)
	at sbt.internal.inc.IncrementalCommon$$anon$1.next(IncrementalCommon.scala:52)
	at sbt.internal.inc.IncrementalCommon.cycle(IncrementalCommon.scala:263)
	at sbt.internal.inc.Incremental$.$anonfun$incrementalCompile$8(Incremental.scala:418)
	at sbt.internal.inc.Incremental$$$Lambda$3153/0x0000000100f1a840.apply(Unknown Source)
	at sbt.internal.inc.Incremental$.withClassfileManager(Incremental.scala:506)
	at sbt.internal.inc.Incremental$.incrementalCompile(Incremental.scala:405)
	at sbt.internal.inc.Incremental$.apply(Incremental.scala:171)
	at sbt.internal.inc.IncrementalCompilerImpl.compileInternal(IncrementalCompilerImpl.scala:534)
	at sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileIncrementally$1(IncrementalCompilerImpl.scala:488)
	at sbt.internal.inc.IncrementalCompilerImpl$$Lambda$3039/0x0000000100e84840.apply(Unknown Source)
	at sbt.internal.inc.IncrementalCompilerImpl.handleCompilationError(IncrementalCompilerImpl.scala:332)
	at sbt.internal.inc.IncrementalCompilerImpl.compileIncrementally(IncrementalCompilerImpl.scala:425)
	at sbt.internal.inc.IncrementalCompilerImpl.compile(IncrementalCompilerImpl.scala:137)
	at sbt.Defaults$.compileIncrementalTaskImpl(Defaults.scala:2363)
	at sbt.Defaults$.$anonfun$compileIncrementalTask$2(Defaults.scala:2313)
	at sbt.Defaults$$$Lambda$3032/0x0000000100e83040.apply(Unknown Source)
	at sbt.internal.server.BspCompileTask$.$anonfun$compute$1(BspCompileTask.scala:30)
	at sbt.internal.server.BspCompileTask$$$Lambda$3034/0x0000000100e87040.apply(Unknown Source)
	at sbt.internal.io.Retry$.apply(Retry.scala:46)
	at sbt.internal.io.Retry$.apply(Retry.scala:28)
	at sbt.internal.io.Retry$.apply(Retry.scala:23)
	at sbt.internal.server.BspCompileTask$.compute(BspCompileTask.scala:30)
	at sbt.Defaults$.$anonfun$compileIncrementalTask$1(Defaults.scala:2311)
	at sbt.Defaults$$$Lambda$892/0x00000001006d8040.apply(Unknown Source)
	at scala.Function1.$anonfun$compose$1(Function1.scala:49)
	at scala.Function1$$Lambda$291/0x0000000100461840.apply(Unknown Source)
	at sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFunctions.scala:62)
	at sbt.internal.util.$tilde$greater$$Lambda$2610/0x0000000100d29840.apply(Unknown Source)
	at sbt.std.Transform$$anon$4.work(Transform.scala:68)
	at sbt.Execute.$anonfun$submit$2(Execute.scala:282)
	at sbt.Execute$$Lambda$2640/0x0000000100d36c40.apply(Unknown Source)
	at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:23)
	at sbt.Execute.work(Execute.scala:291)
	at sbt.Execute.$anonfun$submit$1(Execute.scala:282)
	at sbt.Execute$$Lambda$2620/0x0000000100d2f440.apply(Unknown Source)
	at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:265)
	at sbt.ConcurrentRestrictions$$anon$4$$Lambda$2627/0x0000000100d32040.apply(Unknown Source)
	at sbt.CompletionService$$anon$2.call(CompletionService.scala:64)
	at java.util.concurrent.FutureTask.run(java.base@11.0.9.1/FutureTask.java:264)
	at java.util.concurrent.Executors$RunnableAdapter.call(java.base@11.0.9.1/Executors.java:515)
	at java.util.concurrent.FutureTask.run(java.base@11.0.9.1/FutureTask.java:264)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.9.1/ThreadPoolExecutor.java:1128)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.9.1/ThreadPoolExecutor.java:628)
	at java.lang.Thread.run(java.base@11.0.9.1/Thread.java:834)```
	
	</details>
@WojciechMazur WojciechMazur added itype:bug itype:crash stat:needs triage Every issue needs to have an "area" and "itype" label labels Nov 30, 2022
@nicolasstucki nicolasstucki added stat:needs minimization Needs a self contained minimization area:pickling and removed stat:needs triage Every issue needs to have an "area" and "itype" label labels Dec 1, 2022
@WojciechMazur
Copy link
Contributor Author

The error leading to an infinite loop was caused by the issue described in #16437
Even though this case leading to the infinite loop is already fixed, there seems to be a more general issue related to reporting errors.

@odersky
Copy link
Contributor

odersky commented Dec 4, 2022

I would not say this is about reporting errors but about preventing the compiler to get into illegal states. An underlying that points to itself is illegal. We cannot check for cycles (too expensive) so the scheme is to prevent that in the first place. If the illegal state is caused by an erroneous program it's not enough to report the error we also have to prevent the illegal state internally.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area:pickling itype:bug itype:crash stat:needs minimization Needs a self contained minimization
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@odersky @nicolasstucki @WojciechMazur