can I skip re-fetching of the repos? #210
Comments
|
I checked a recent community build run, containing 177 projects. The time spent re-fetching the various projects from their GitHub repo is exactly 56.1 seconds, in total. So you would not save much in the end, I am afraid. Here's the full list: |
eed3si9n
added
uncategorized
|
@cunei we discussed this in person in Montreal last week and I think I was able to convince you that a "don't go to GitHub, just use what you have" flag would be useful to me in community build work. and this is the reminder you asked for :-) |
|
I feel like we discussed a few other related possibilities, but I'm forgetting now what they were. I think one of them might have been, can we not only skip talking to GitHub, but also skip recomputing the inter-project dependencies entirely and just jump right into building? is there enough cached information to do that? |
|
@SethTisue You may be interested in the branch https://github.com/lightbend/dbuild/commits/wip-INPROGRESS-disableUpdates, which should be now complete. 😊 It is published as version
The speed improvements are substantial; when remote updates are disabled, the entire extraction phase for the community build (with unchanged references) takes about 25 seconds. |
|
A matching update to the community build is at: https://github.com/cunei/community-builds/commits/wip-testing-toni4 |
|
@SethTisue I made further improvements and optimizations. Now, re-extracting the 188 projects of the community build, with all cached and updates disabled, takes just 8 seconds: You can test it by using dbuild version 0.9.17-toni22. |
Why not wrap these various behaviors of Git and sbt skipping update in to something called Also, why not use |
|
I'll have to look at this in January |
|
the |
now that the community build has over 160 projects in it, when I'm testing changes locally, doing
git fetchon all the repos to see whether each project's dependencies need to be re-extracted takes a while.this is by far the biggest factor causing a slow feedback loop on typical changes.
is there a way to tell dbuild "I don't care about any remote changes right now, just use whatever you already have locally"?
The text was updated successfully, but these errors were encountered: