Skip to content

Potential performance regression when deriving large case classess #15834

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
WojciechMazur opened this issue Aug 8, 2022 · 3 comments
Closed

Potential performance regression when deriving large case classess #15834

WojciechMazur opened this issue Aug 8, 2022 · 3 comments
Assignees
@nicolasstucki
Labels
area:typeclass-derivation itype:performance stat:needs minimization Needs a self contained minimization

Comments

@WojciechMazur
Copy link
Contributor

WojciechMazur commented Aug 8, 2022 •
edited
Loading

Based on failure in Open Community Build #2826 in library https://github.com/sirthias/borer

Scala 3.1.3 builds were stable when compiling derivation tests using -Xss=2m, however, in Scala 3.2.0-RCx they end up with StackOverflowError in the staging phase, even with increased -Xss=4m

Compiler version

3.2.0-RC3

Minimized code

No code minimization yet, based on my observations project might fail with StackOverflowError when deriving encoder here for case class using 100 fields.

Output

Stacktrace (click arrow to expand)

[error] ## Exception when compiling 1 sources to /home/wmazur/projects/virtuslab/community-build3/repo/derivation/.jvm/target/scala-3.2.0-RC3/test-classes
[error] java.lang.StackOverflowError
[error]         at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1404)
[error]         at dotty.tools.dotc.ast.TreeMapWithImplicits.transform(TreeMapWithImplicits.scala:73)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.mapOverTree$1(TreeMapWithStages.scala:79)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.transform(TreeMapWithStages.scala:149)
[error]         at dotty.tools.dotc.transform.PCPCheckAndHeal.transform(PCPCheckAndHeal.scala:61)
[error]         at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1412)
[error]         at dotty.tools.dotc.ast.TreeMapWithImplicits.transform(TreeMapWithImplicits.scala:73)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.mapOverTree$1(TreeMapWithStages.scala:79)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.transform(TreeMapWithStages.scala:149)
[error]         at dotty.tools.dotc.transform.PCPCheckAndHeal.transform(PCPCheckAndHeal.scala:61)
[error]         at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1418)
[error]         at dotty.tools.dotc.ast.TreeMapWithImplicits.transform(TreeMapWithImplicits.scala:73)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.mapOverTree$1(TreeMapWithStages.scala:79)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.transform(TreeMapWithStages.scala:149)
[error]         at dotty.tools.dotc.transform.PCPCheckAndHeal.transform(PCPCheckAndHeal.scala:61)
[error]         at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1452)
[error]         at dotty.tools.dotc.ast.TreeMapWithImplicits.transform(TreeMapWithImplicits.scala:73)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.mapOverTree$1(TreeMapWithStages.scala:79)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.transform(TreeMapWithStages.scala:149)
[error]         at dotty.tools.dotc.transform.PCPCheckAndHeal.transform(PCPCheckAndHeal.scala:61)
[error]         at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1452)
[error]         at dotty.tools.dotc.ast.TreeMapWithImplicits.transform(TreeMapWithImplicits.scala:73)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.mapOverTree$1(TreeMapWithStages.scala:79)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.transform(TreeMapWithStages.scala:149)
[error]         at dotty.tools.dotc.transform.PCPCheckAndHeal.transform(PCPCheckAndHeal.scala:61)
[error]         at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1452)
[error]         at dotty.tools.dotc.ast.TreeMapWithImplicits.transform(TreeMapWithImplicits.scala:73)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.mapOverTree$1(TreeMapWithStages.scala:79)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.transform(TreeMapWithStages.scala:149)
[error]         at dotty.tools.dotc.transform.PCPCheckAndHeal.transform(PCPCheckAndHeal.scala:61)
[error]         at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1430)
[error]         at dotty.tools.dotc.ast.TreeMapWithImplicits.transform(TreeMapWithImplicits.scala:73)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.mapOverTree$1(TreeMapWithStages.scala:79)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.transform(TreeMapWithStages.scala:149)
[error]         at dotty.tools.dotc.transform.PCPCheckAndHeal.transform(PCPCheckAndHeal.scala:61)
[error]         at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1452)
[error]         at dotty.tools.dotc.ast.TreeMapWithImplicits.transform(TreeMapWithImplicits.scala:73)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.mapOverTree$1(TreeMapWithStages.scala:79)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.transform(TreeMapWithStages.scala:149)
[error]         at dotty.tools.dotc.transform.PCPCheckAndHeal.transform(PCPCheckAndHeal.scala:61)
[error]         at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1452)
[error]         at dotty.tools.dotc.ast.TreeMapWithImplicits.transform(TreeMapWithImplicits.scala:73)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.mapOverTree$1(TreeMapWithStages.scala:79)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.transform(TreeMapWithStages.scala:149)
[error]         at dotty.tools.dotc.transform.PCPCheckAndHeal.transform(PCPCheckAndHeal.scala:61)
[error]         at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1452)
[error]         at dotty.tools.dotc.ast.TreeMapWithImplicits.transform(TreeMapWithImplicits.scala:73)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.mapOverTree$1(TreeMapWithStages.scala:79)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.transform(TreeMapWithStages.scala:149)
[error]         at dotty.tools.dotc.transform.PCPCheckAndHeal.transform(PCPCheckAndHeal.scala:61)
[error]         at dotty.tools.dotc.ast.tpd$TreeMapWithPreciseStatContexts.loop$2(tpd.scala:1212)
[error]         at dotty.tools.dotc.ast.tpd$TreeMapWithPreciseStatContexts.transformStats(tpd.scala:1212)
[error]         at dotty.tools.dotc.ast.tpd$TreeMapWithPreciseStatContexts.transformBlock(tpd.scala:1217)
[error]         at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1432)
[error]         at dotty.tools.dotc.ast.TreeMapWithImplicits.transform(TreeMapWithImplicits.scala:49)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.mapOverTree$1(TreeMapWithStages.scala:79)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.transform(TreeMapWithStages.scala:132)
[error]         at dotty.tools.dotc.transform.PCPCheckAndHeal.transform(PCPCheckAndHeal.scala:61)
[error]         at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform$$anonfun$1(Trees.scala:1513)
[error]         at scala.collection.immutable.List.mapConserve(List.scala:472)
[error]         at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1513)
[error]         at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1418)
[error]         at dotty.tools.dotc.ast.TreeMapWithImplicits.transform(TreeMapWithImplicits.scala:73)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.mapOverTree$1(TreeMapWithStages.scala:79)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.transform(TreeMapWithStages.scala:149)
[error]         at dotty.tools.dotc.transform.PCPCheckAndHeal.transform(PCPCheckAndHeal.scala:61)
[error]         at dotty.tools.dotc.transform.PCPCheckAndHeal.transform(PCPCheckAndHeal.scala:58)
[error]         at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1452)
[error]         at dotty.tools.dotc.ast.TreeMapWithImplicits.transform(TreeMapWithImplicits.scala:73)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.mapOverTree$1(TreeMapWithStages.scala:79)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.transform(TreeMapWithStages.scala:149)
[error]         at dotty.tools.dotc.transform.PCPCheckAndHeal.transform(PCPCheckAndHeal.scala:61)
[error]         at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1452)
[error]         at dotty.tools.dotc.ast.TreeMapWithImplicits.transform(TreeMapWithImplicits.scala:73)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.mapOverTree$1(TreeMapWithStages.scala:79)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.transform(TreeMapWithStages.scala:149)
[error]         at dotty.tools.dotc.transform.PCPCheckAndHeal.transform(PCPCheckAndHeal.scala:61)
[error]         at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1481)
[error]         at dotty.tools.dotc.ast.TreeMapWithImplicits.transform(TreeMapWithImplicits.scala:73)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.mapOverTree$1(TreeMapWithStages.scala:79)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.transform(TreeMapWithStages.scala:149)
[error]         at dotty.tools.dotc.transform.PCPCheckAndHeal.transform(PCPCheckAndHeal.scala:61)
[error]         at dotty.tools.dotc.ast.tpd$TreeMapWithPreciseStatContexts.loop$2(tpd.scala:1212)
[error]         at dotty.tools.dotc.ast.tpd$TreeMapWithPreciseStatContexts.transformStats(tpd.scala:1212)
[error]         at dotty.tools.dotc.ast.tpd$TreeMapWithPreciseStatContexts.transformBlock(tpd.scala:1217)
[error]         at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1432)
[error]         at dotty.tools.dotc.ast.TreeMapWithImplicits.transform(TreeMapWithImplicits.scala:49)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.mapOverTree$1(TreeMapWithStages.scala:79)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.transform(TreeMapWithStages.scala:132)
[error]         at dotty.tools.dotc.transform.PCPCheckAndHeal.transform(PCPCheckAndHeal.scala:61)
[error]         at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1452)
[error]         at dotty.tools.dotc.ast.TreeMapWithImplicits.transform(TreeMapWithImplicits.scala:73)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.mapOverTree$1(TreeMapWithStages.scala:79)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.transform(TreeMapWithStages.scala:149)
[error]         at dotty.tools.dotc.transform.PCPCheckAndHeal.transform(PCPCheckAndHeal.scala:61)
[error]         at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1452)
[error]         at dotty.tools.dotc.ast.TreeMapWithImplicits.transform(TreeMapWithImplicits.scala:73)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.mapOverTree$1(TreeMapWithStages.scala:79)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.transform(TreeMapWithStages.scala:149)
[error]         at dotty.tools.dotc.transform.PCPCheckAndHeal.transform(PCPCheckAndHeal.scala:61)
[error]         at dotty.tools.dotc.ast.tpd$TreeMapWithPreciseStatContexts.transformBlock$$anonfun$1$$anonfun$1(tpd.scala:1217)
[error]         at dotty.tools.dotc.ast.tpd$TreeMapWithPreciseStatContexts.loop$2(tpd.scala:1199)
[error]         at dotty.tools.dotc.ast.tpd$TreeMapWithPreciseStatContexts.transformStats(tpd.scala:1212)
[error]         at dotty.tools.dotc.ast.tpd$TreeMapWithPreciseStatContexts.transformBlock(tpd.scala:1217)
[error]         at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1432)
[error]         at dotty.tools.dotc.ast.TreeMapWithImplicits.transform(TreeMapWithImplicits.scala:49)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.mapOverTree$1(TreeMapWithStages.scala:79)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.transform(TreeMapWithStages.scala:132)
[error]         at dotty.tools.dotc.transform.PCPCheckAndHeal.transform(PCPCheckAndHeal.scala:61)
[error]         at dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1452)
[error]         at dotty.tools.dotc.ast.TreeMapWithImplicits.transform(TreeMapWithImplicits.scala:73)
[error]         at dotty.tools.dotc.transform.TreeMapWithStages.mapOverTree$1(TreeMapWithStages.scala:79)
.... // Last 20 lines repeat ~50 times in the stacktrace
[error] (derivation / Test / compileIncremental) java.lang.StackOverflowError

Expectation

It should be checked whether new versions of compiler does not imply significantly larger memory usage
@WojciechMazur WojciechMazur added itype:performance stat:needs minimization Needs a self contained minimization stat:needs triage Every issue needs to have an "area" and "itype" label labels Aug 8, 2022
@szymon-rd szymon-rd added area:typeclass-derivation and removed stat:needs triage Every issue needs to have an "area" and "itype" label labels Aug 16, 2022
@WojciechMazur WojciechMazur mentioned this issue Sep 6, 2022
Closed
@prolativ
Copy link
Contributor

@WojciechMazur The Community Build link seems wrong to me - that is a build of https://github.com/valskalla/odin and it was successful

@WojciechMazur
Copy link
Contributor Author

@prolativ Maybe I've copied the wrong link. I've updated the link above. It now points to the latest, still failing, build of boerer.

@WojciechMazur
Copy link
Contributor Author

Project is now passing the OpenCB, the issue might have been present due to testing infrastructure and default -Xss sizes

@WojciechMazur WojciechMazur closed this as not planned Won't fix, can't repro, duplicate, stale Jul 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@nicolasstucki nicolasstucki

Labels
area:typeclass-derivation itype:performance stat:needs minimization Needs a self contained minimization
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@nicolasstucki @szymon-rd @prolativ @WojciechMazur