Skip to content

Speedup bisection builds by skipping generation of scaladoc #18157

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 7, 2023

Conversation

WojciechMazur
Copy link
Contributor

Adds special mode for sbt builds enabled with BISECTBUILD=yes env variables. Under this mode scaladoc and static resources are not generated, though allowing to speedup local publishing by up to 50% (92s vs 189s previously).
Flag would be used when bisecting projects.

Copy link
Member

@bishabosha bishabosha left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Recently was added the publishLocalBin task to skip doc generation, would that work instead?

@WojciechMazur
Copy link
Contributor Author

Recently was added the publishLocalBin task to skip doc generation, would that work instead?

In case of bisecting that would not work, becouse this new task publishLocalBin might be missing in previous versions of the sbt build. Usage of my solution is best effort - it would have no effect for any previous builds, but would provide a signicant speedup for future builds.

@bishabosha
Copy link
Member

In case of bisecting that would not work, becouse this new task publishLocalBin might be missing in previous versions of the sbt build. Usage of my solution is best effort - it would have no effect for any previous builds, but would provide a signicant speedup for future builds.

Makes sense!

@@ -238,6 +238,7 @@ class CommitBisect(validationScript: File, shouldFail: Boolean, bootstrapped: Bo
val bisectRunScript = s"""
|scalaVersion=$$(sbt "print ${scala3CompilerProject}/version" | tail -n1)
|rm -r out
|export BISECTBUILD=yes
|sbt "clean; ${scala3Project}/publishLocal"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
|sbt "clean; ${scala3Project}/publishLocal"
|sbt "clean; set Compile / doc / skip := true; set Compile/ doc / sources := Nil; ${scala3Project}/publishLocal"

I think this single line change should be equivalent to all the other changes from this PR and it should speed up also builds of historical commits

Copy link
Contributor Author

@WojciechMazur WojciechMazur Jul 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really, the global scope definition would not propagate to the projects, so these would have any effect. We could use set every x but this one does not support scopes, instead it would apply value to every task with given name. We could use internals to properly set correct values, but that's the overkill.

However, we can use comination of 2 more specific commands to get the expected effect:

  • set scaladoc/Compile/resourceGenerators := (ThisBuild/Compile/resourceGenerators).value - to remove custom resourceGenerators requiring compilation and linking scaladoc static resources using Scala.js, we still want to keep the default ones
  • set every doc := new File("unused") - overwrite all doc tasks to no-op (seems to be unharmfull, even though we use non-exisitng file)

@WojciechMazur WojciechMazur force-pushed the build/bisect-builds branch from 60c8ed1 to ee76224 Compare July 7, 2023 12:03
@@ -238,7 +238,7 @@ class CommitBisect(validationScript: File, shouldFail: Boolean, bootstrapped: Bo
val bisectRunScript = s"""
|scalaVersion=$$(sbt "print ${scala3CompilerProject}/version" | tail -n1)
|rm -r out
|sbt "clean; ${scala3Project}/publishLocal"
|sbt "clean; set every doc := new File(\"unused\"); set scaladoc/Compile/resourceGenerators := (\`${scala3Project}\`/Compile/resourceGenerators).value; ${scala3Project}/publishLocal"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are backticks required around the first occurrence of the project name but not the second one?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, in ${scala3Project}/publishLocal we get special handling provided by sbt when running commands, however, in := {...} part we're in the scope which is not covered by special handling and we need to refer to actual symbols defined in the code. It's quite a common problem that we need to use different syntax in set and different syntax in regular commands

@WojciechMazur WojciechMazur merged commit 7ede150 into main Jul 7, 2023
@WojciechMazur WojciechMazur deleted the build/bisect-builds branch July 7, 2023 14:07
nicolasstucki added a commit that referenced this pull request Jul 10, 2023
The bisection script had a bug introduced in #18157 that was failing at
runtime when executing bisect
@Kordyjan Kordyjan added this to the 3.4.0 milestone Dec 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants