Cache broken with GitVersionTask 3.6.0 #942
Comments
|
@tpluscode glad I am not alone and not crazy. I am getting the exact same error It takes forever when the cache is not working. |
|
I have the same problem here. My build time went from Even while building an assembly, the name of the cache file keeps on changing for every step: |
|
Hey, thanks for posting the issue. Not sure why this is happening, the code to calculate the hash is: The result is then hashed. As far as I know the only change is including the config file contents in the hash. Thoughts @asbjornu ? |
|
Can you try 3.6.1, we received a PR which cleaned up and fixed some issues around cache. |
|
Still broken: |
|
@JakeGinnivan I made a local build of GitVersion, where I added a bunch of debug info: The difference is when getting the last directory write. If you look at the timestamps, the write to |
|
After some more tests: In It is a side effect of commit acce06c: the git directory is always normalized in a build server. |
|
I have the problem too. It just happened that I was finally able to get my big solution into TeamCity right as 3.6.0 was available. |
|
Seems like the problem persists in 3.6.1 - my builds are taking 20 min instead of a few minutes or less\ |
|
Awesome investigation @DanielRose If someone wants to pick this up that would be great. |
|
@DanielRose what does it mean that the git directory is normalized? Is it running this? https://github.com/GitTools/GitTools.Core/blob/244cca386321d6f8c90cf31cd3c4734c8932b4da/src/GitTools.Core/GitTools.Core.Shared/Git/Helpers/GitRepositoryHelper.cs |
Removed redundant call to gitPreparer.Initialise since it's now called earlier
|
@tofutim Yes, that is what is being run. However, I don't understand the reason for that. Perhaps it is necessary to run that (once?) for dynamic repositories? |
|
@rubenmamo Hi Ruben, can you shed some light on this? I am running TeamCity and the new changes make each instance of GitVersionTask (in every assembly!) to compute the gitversion instead of using the task. Three questions: (1) what does it mean to NormalizeGitDirectory, (2) why does it change the repository (each time it is run), and (3) why force buildServer to do this? In my case I have a single Visual Studio solution with about 10-12 projects each calling GitVersionTask. Update. (1) the notes say It seems that this may be true for some build servers, but not so for TeamCity, which will checkout branches as needed. So buildServer != null should not be used in generally. (2) perhaps there is something in normalization that is tied to the datetime of normalization, which then changes the cache key Seems that this is the best thing to fix, if you normalize the repo, shouldn't it always produce the same cache key? (3) We need @rubenmamo to tell us which build server he was targeting and perhaps filter on that. At least if the buildserver.Contains("TeamCity") we should NOT normalize. Or maybe the condition should be
I'm not sure I can make any further progress. Hopefully this is helpful to someone. **Update 2. I forgot to add that it is not a problem for the cachekey to change, but it is a problem for the cachekey to change if you normalize a second time. In my first run,
In the second run (same repo, just run GitVersion again),
This gives me the idea that maybe you should put a marker that says this repo has been normalized. If it has been normalized already, do not re-normalize. |
|
Hi All, The change in question was done to get gitversion to work again when using dynamic repositories on team city as this was broken in an earlier commit which was not calling the initialize method before using stuff which is initialized in that particular method. In my environment I cannot use agent side checkout in TeamCity and call the gitversion command line as the first build step (I'm not using the ms build task) and hence I need to use dynamic repositories which meant I couldn't use the latest version of gitversion. The change I made Was to get the initialize method to be called earlier (previously it was called in executeinternal) and hence I did not change its structure apart from modifying how it determines that it's a dynamic repository. Hope this helps. Ruben |
|
@rubenmamo I would like to get your feedback on a possible solution. Given,
instead of forcing all build servers to normalize, the normalization test should instead be for gitPreparer.IsDynamicGitRepository so that we have
(actually that IsNullOrWhitesSpace could also use gitPreparer.IsDynamicGitRepository). BTW, for your TeamCity setup, how are you doing it? Are you doing the .git checkout server-side? I am doing the .git checkout agent-side. Update. Scratch that, undoing that breaks some of the tests. Maybe the real problem is that repeated normalization changes the hash. Why does it change the hash? Because: Maybe using timestamp in .git/refs directory is not enough. Question. How would one test for the cache in the tests? |
|
As I stated above, the problem is that every run of |
|
The problem here is that normalizeGitDirectory touches the LastDirectoryWrite on .git/refs. So even though the content is the same, the Hash ends up different because of The test looks like this: Update. Thinking something like this |
|
@tofutim, for my TeamCity setup I'm using Server Side Checkout (not as I incorrectly stated in my previous comment which I've now amended). I could not get Agent Side Checkout to work, if I remember correctly I was having issues with using TFS as my Git Server since it requires authentication (it might have been something else, not 100% sure). Since I'm using Server Side Checkout I need to use alternate repositories as document here: (http://gitversion.readthedocs.io/en/latest/build-server-support/build-server/teamcity/). I did this particular change because the application was failing with an unhandled Exception when using Dynamic Repositories. I traced it down to the way GitPreparer was being initialised and the calls made to GitPreparer. The ExecuteCore was calling GitPreparer.IsDynamicGitRepository before calling GItPreparer.Initialise and GitPreparer.IsDynamicGitRepository requires Initialise to have been called beforehand. I was wary of changing the way GitPreparer is initialised as this would probably have affected other areas and opted to call the initialise earlier. This has obviously caused the issue with the cache. I tried to keep the code similar to what was present before as I did not understand the full ramifications of doing other more structural changes. |
|
@rubenmamo Your change is fine. There might be repeated normalization over multiple assemblies (in GitVersionTask) but I think is ok. The biggest problem is that the cache hash (haha) is computed on the timestamp of refs, which the author admits maybe not enough. The real deal is to traverse the refs, which I'm working on. PR soon. Incidentally, I was able to get agent checkout to work by using uploaded key. |
|
Pls someone do a sanity check on my PR. Thanks! |
|
Releasing 3.6.2 now. Thanks everyone for your help! |
|
Updated to 3.6.2. Cache is working again on my build server. Thanks! |
|
Hi everybody, i have still same problem with 3.6.5 version on tfs build server. I got same error with extension task , than i tried to add a commandline task and call gitversion.exe by command prompt and i got same error again.
. Do you have a suggestion? Am i have to back an older version recommended above ? |
|
@nzrytmn: GitVersion should calculate the same version number with or without the cache. The only thing the cache should affect, is performance. If you're experiencing different version numbering depending, I would be inclined to conclude that that's another issue. Also, the line you've quoted is not an error, but expected behaviour if the version should infact change. If running GitVersion on the same commit several times in a row results in the same cache miss being logged, that's a problem, but as I said, that should not affect the version calculation. |
|
Thanks for information @asbjornu , yes i know it is not an error, but "not found" message makes me feel something is not correct; by the way the main problem is the version number is not increasing that i expect. For my perspective the problem is version tool always calculates version number from base tag and because of this it is calculating same version for each release, it couldn't keep mind the old releases were deployed with same version it gave,
|
|
@nzrytmn: What versioning mode are you using? If it's |
|
@asbjornu I tried both mode, but it is calculating same version number for each release. But you are right, it may be because of my configuration settings. This is my GitVersion.yml file for one of my samples. branches: {
master:
regex: master
mode: ContinuousDeployment
tag: ''
increment: Patch
prevent-increment-of-merged-branch-version: true
track-merge-target: false
tracks-release-branches: false
is-release-branch: false
release:
regex: release?[/-]
mode: ContinuousDeployment
tag: beta
increment: Patch
prevent-increment-of-merged-branch-version: true
track-merge-target: false
tracks-release-branches: false
is-release-branch: true
feature:
regex: feature?[/-]
mode: ContinuousDeployment
tag: useBranchName
increment: Inherit
prevent-increment-of-merged-branch-version: false
track-merge-target: false
tracks-release-branches: false
is-release-branch: false
pull-request:
regex: (pull|pull\-requests|pr)[/-]
mode: ContinuousDeployment
tag: PullRequest
increment: Inherit
prevent-increment-of-merged-branch-version: false
tag-number-pattern: '[/-](?<number>\d+)[-/]'
track-merge-target: false
tracks-release-branches: false
is-release-branch: false
hotfix:
regex: hotfix(es)?[/-]
mode: ContinuousDeployment
tag: beta
increment: Patch
prevent-increment-of-merged-branch-version: false
track-merge-target: false
tracks-release-branches: false
is-release-branch: false
support:
regex: support[/-]
mode: ContinuousDeployment
tag: ''
increment: Patch
prevent-increment-of-merged-branch-version: true
track-merge-target: false
tracks-release-branches: false
is-release-branch: false
develop:
regex: dev(elop)?(ment)?$
mode: ContinuousDeployment
tag: unstable
increment: Minor
prevent-increment-of-merged-branch-version: false
track-merge-target: true
tracks-release-branches: true
is-release-branch: false
}
next-version: 1.2
assembly-versioning-scheme: MajorMinorPatch
assembly-informational-format: '{InformationalVersion}'
mode: ContinuousDeployment
increment: Inherit
continuous-delivery-fallback-tag: ci
tag-prefix: '[vV]'
major-version-bump-message: '\+semver:\s?(breaking|major)'
minor-version-bump-message: '\+semver:\s?(feature|minor)'
patch-version-bump-message: '\+semver:\s?(fix|patch)'
no-bump-message: '\+semver:\s?(none|skip)'
legacy-semver-padding: 4
build-metadata-padding: 4
commits-since-version-source-padding: 4
commit-message-incrementing: Enabled
ignore:
sha: []
commits-before: yyyy-MM-ddTHH:mm:ss |
|
@nzrytmn: Can you please try deleting the |
|
@nzrytmn I am experiencing a basically identical issue, did you ever resolve it? |
|
Having the exact same issues running 3.6.4. Nothing in the working directory is changing in-between TeamCity builds, the .yml used for the cache can't be found and GitVersion will go off and re-calculate a repo. Taking a build from 4 minutes to 33 minutes. |
|
@fenneh: Can you please upgrade to v4 beta 12 and let us know if that fixes the issue? |
I've just updated the MSBuild task package and apparently the cache is broken. Each target (
WriteVersionInfoToBuildLog,WriteVersionInfoToBuildLogandGitVersion) in each built project recalculates the version number from scratch. Every time I seeAnd every time (during one build) the
XYZ123part is different. Worked good in version 3.5.4The text was updated successfully, but these errors were encountered: