need better documentation on doing local runs

[SethTisue]

(accumulated random notes on this from the last few months)

though it may not even be worth documenting this better until something is done about the performance issues (its's #152)

I guess setting up local Artifactory helps performance; I haven't tried it yet. there is doc linked from #58

often, running dbuild with -d is key in order to see what's going on (I mean if it's the community build itself you're troubleshooting, and not a compiler bug)

document how to fix the scala version at a given SHA, to prevent global rebuilds when someone merges a Scala PR? the Jenkins jobs for community builds have a field for this, not sure how it's done locally.

how to run just one project? e.g. ./dbuild-0.9.1/bin/dbuild -n community.dbuild sbinary works, sort of, but does bypassing the usual scripts/jobs/integrate/community-build method still get you the right resolvers? need to double check what value the scripts directory script adds

also building just one project doesn't really save you that much time since all projects are still extracted which takes a while. I've been working around this by commenting out big swathes of common.conf and/or community.dbuild

[matanox]

Nice doc! I'm consistently grinding close to a halt with ivy resolutions. (indeed with the -d option I can see something is happening v.s. just being stuck). Wondering whether a local proxy gives any gain beyond the caching already performed by default by ivy. I would assume that it would not make any difference in the single machine scenario.

Thanks for suggesting -d!

[gkossakowski]

Local artifactory helps to speed up resolution for a few reasons:

• it appears to me that Ivy will resolve the same artifact multiple times; Artifactory is good at resolving only once
• when you add repos to Artifactory you can check the option to prefetch jars in the background triggered by poms requests
• Ivy cache might get corrupted and you have no option but start over
• Ivy cache is stored per dbuild version so upgrading dbuild requires recreating Ivy cache

[matanox]

I assume you are right, many thanks. But I can't really fathom why would
ivy resolve the same artifact multiple times.

On Tue, Dec 8, 2015 at 2:57 AM, Grzegorz Kossakowski <
[email protected]> wrote:

Local artifactory helps to speed up resolution for a few reasons:

• it appears to me that Ivy will resolve the same artifact multiple
  times; Artifactory is good at resolving only once
• when you add repos to Artifactory you can check the option to
  prefetch jars in the background triggered by poms requests
• Ivy cache might get corrupted and you have no option but start over
• Ivy cache is stored per dbuild version so upgrading dbuild requires
  recreating Ivy cache

—
Reply to this email directly or view it on GitHub
#162 (comment)
.

[matanox]

Running for 24 hours and going, several restarts required after occasional
akka http timeouts over resolution.
Any idea why so much duplication in the resolution logging?

[scala-swing] [info] Resolving org.apache#apache;4 ...
[scala-swing] [info] Resolving net.databinder#dispatch-futures_2.10;0.8.10...
[scala-swing] [info] Resolving net.databinder#dispatch-futures_2.10;0.8.10...
[scala-swing] [info] Resolving org.scala-lang#scala-actors;2.10.2 ...
[scala-swing] [info] Resolving org.scala-lang#scala-actors;2.10.2 ...
[scala-swing] [info] Resolving commons-io#commons-io;2.4 ...
[scala-swing] [info] Resolving commons-io#commons-io;2.4 ...
[scala-swing] [info] Resolving org.apache.commons#commons-parent;25 ...
[scala-swing] [info] Resolving org.apache.commons#commons-parent;25 ...
[scala-swing] [info] Resolving org.apache#apache;9 ...
[scala-swing] [info] Resolving org.apache#apache;9 ...
[scala-swing] [info] Resolving com.jcraft#jsch;0.1.50 ...
[scala-swing] [info] Resolving com.jcraft#jsch;0.1.50 ...

And I am very curious why the entire resolution process starts over when rerunning dbuild, it looks like no caching is taking place, or dbuild is invalidating the previous resolution. It would be nice being able to tell it not to invalidate prior resolutions, when doing a rerun. Is this last aspect a pure sbt one?

[SethTisue]

@matanster you may find the ivy resolution picture improved now if you tried again — the list of the resolvers in the config file is now much shorter

[SethTisue]

how to run just one project

or several (comma-separated); improved and documented in #331

[SethTisue]

this is an open-ended ticket, but I'm basically satisfied with the current documentation, so I'm going to go ahead and close it.

it isn't documented yet how to specify the Scala nightly you want, but I'll get to that as part of scala/scala-jenkins-infra#189

[SethTisue]

https://github.com/scala/community-builds/wiki was updated today with fresh instructions on this

[matanox]

Thanks!

Sent from my mobile

-------- Original Message --------
From:Seth Tisue [email protected]
Sent:Fri, 14 Oct 2016 23:25:43 +0300
To:scala/community-builds [email protected]
Cc:matanster [email protected],Mention [email protected]
Subject:Re: [scala/community-builds] need better documentation on doing local runs (#162)

https://github.com/scala/community-builds/wiki was updated today with fresh instructions on this

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

{"api_version":"1.0","publisher":{"api_key":"05dde50f1d1a384dd78767c55493e4bb","name":"GitHub"},"entity":{"external_key":"github/scala/community-builds","title":"scala/community-builds","subtitle":"GitHub repository","main_image_url":"https://cloud.githubusercontent.com/assets/143418/17495839/a5054eac-5d88-11e6-95fc-7290892c7bb5.png","avatar_image_url":"https://cloud.githubusercontent.com/assets/143418/15842166/7c72db34-2c0b-11e6-9aed-b52498112777.png","action":{"name":"Open in GitHub","url":"https://github.com/scala/community-builds"}},"updates":{"snippets":[{"icon":"PERSON","message":"@SethTisue in #162: https://github.com/scala/community-builds/wiki was updated today with fresh instructions on this"}],"action":{"name":"View Issue","url":"https://github.com/scala/community-builds/issues/162#issuecomment-253910509"}}}