Skip to content

modify cache key calc to use entire contents of .git/refs instead of timestamp #969

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jul 26, 2016

Conversation

tofutim
Copy link
Contributor

@tofutim tofutim commented Jul 24, 2016

I didn't see the note in rebase that the range is excluding the first one - so it took me a while to figure out how to do it. Ref: #967 (original pull request) - should resolve #942

@tofutim tofutim changed the title Cachehash modify cache key calc to use entire contents of .git/refs instead of timestamp Jul 24, 2016
}

// lifted from https://msdn.microsoft.com/en-us/library/bb513869.aspx
public static List<string> TraverseTree(string root)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this need to be public?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No

On Jul 24, 2016, at 11:07, Asbjørn Ulsberg [email protected] wrote:

In src/GitVersionCore/GitVersionCacheKeyFactory.cs:

  •        return GetHash(dotGitDirectory, lastGitRefsChangedTicks.ToString());
    
  •        return GetHash(traverse.ToArray());
    
  •    }
    
  •    // lifted from https://msdn.microsoft.com/en-us/library/bb513869.aspx
    
  •    public static List<string> TraverseTree(string root)
    
    Does this need to be public?


You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rename to what it actually does, ex. CalculateDirectoryContents.

@JakeGinnivan
Copy link
Contributor

How heavy is this?

Could we just hash the refs instead as well as HEAD.

var targetUrl = "https://github.com/GitTools/GitVersion.git";
var targetBranch = "refs/head/master";
var gitPreparer = new GitPreparer(targetUrl, null, new Authentication(), false, fixture.RepositoryPath);
//var cacheKey0 = GitVersionCacheKeyFactory.Create(fileSystem, gitPreparer, null);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove this line instead of commenting.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left it there as a reminder that the cache key changes after Initialize, but ok.

@DanielRose
Copy link
Contributor

@JakeGinnivan This checks the files and directories in refs only. On my TeamCity (with a bit more logging than in this PR):

[WriteVersionInfoToBuildLog] INFO [07/25/16 8:36:28:23] Begin: Creating cache key
[WriteVersionInfoToBuildLog]   INFO [07/25/16 8:36:28:23] dotGitDirectory: C:\TeamCity_BuildAgent_1\work\7b9073d83bcec7a9\.git
[WriteVersionInfoToBuildLog]   INFO [07/25/16 8:36:28:24] TraverseTree: Checking directory: C:\TeamCity_BuildAgent_1\work\7b9073d83bcec7a9\.git\refs
[WriteVersionInfoToBuildLog]   INFO [07/25/16 8:36:28:24] TraverseTree: Files in directory: 
[WriteVersionInfoToBuildLog]   INFO [07/25/16 8:36:28:24] TraverseTree: Checking directory: C:\TeamCity_BuildAgent_1\work\7b9073d83bcec7a9\.git\refs\heads
[WriteVersionInfoToBuildLog]   INFO [07/25/16 8:36:28:24] TraverseTree: Files in directory: 
[WriteVersionInfoToBuildLog]   INFO [07/25/16 8:36:28:24] TraverseTree: Checking directory: C:\TeamCity_BuildAgent_1\work\7b9073d83bcec7a9\.git\refs\tags
[WriteVersionInfoToBuildLog]   INFO [07/25/16 8:36:28:24] TraverseTree: Files in directory: 
[WriteVersionInfoToBuildLog]   INFO [07/25/16 8:36:28:24] traverse: C:\TeamCity_BuildAgent_1\work\7b9073d83bcec7a9\.git\refs;C:\TeamCity_BuildAgent_1\work\7b9073d83bcec7a9\.git\refs\heads;C:\TeamCity_BuildAgent_1\work\7b9073d83bcec7a9\.git\refs\tags
[WriteVersionInfoToBuildLog]   INFO [07/25/16 8:36:28:25] gitSystemHash: FC02E007672D8D4A3726FE7CAC83808A28FD2C73
[WriteVersionInfoToBuildLog]   INFO [07/25/16 8:36:28:25] IsDynamicGitRepository: False
[WriteVersionInfoToBuildLog]   INFO [07/25/16 8:36:28:25] Returning Project Root from DotGitDirectory: C:\TeamCity_BuildAgent_1\work\7b9073d83bcec7a9\.git - C:\TeamCity_BuildAgent_1\work\7b9073d83bcec7a9
[WriteVersionInfoToBuildLog]   INFO [07/25/16 8:36:28:25] configFileHash: 
[WriteVersionInfoToBuildLog]   INFO [07/25/16 8:36:28:25] repositorySnapshotHash: 5DA1B05446AFA0ACF49FDA53234AF7B18BEDA6BC
[WriteVersionInfoToBuildLog]   INFO [07/25/16 8:36:28:25] overrideConfigHash: 
[WriteVersionInfoToBuildLog]   INFO [07/25/16 8:36:28:25] compositeHash: 7D1E057FD5147AB25412FF7C7A757B3A01EBB869
[WriteVersionInfoToBuildLog] INFO [07/25/16 8:36:28:26] End: Creating cache key (Took: 22.00ms)

22ms for the first run. Afterwards the data is cached by the filesystem or so, and it takes 8ms.

Locally:

INFO [07/25/16 8:48:47:58] Begin: Creating cache key
  INFO [07/25/16 8:48:47:58] dotGitDirectory: C:\Git\GitVersionTaskTest\.git
  INFO [07/25/16 8:48:47:58] TraverseTree: Checking directory: C:\Git\GitVersionTaskTest\.git\refs
  INFO [07/25/16 8:48:47:58] TraverseTree: Files in directory:
  INFO [07/25/16 8:48:47:58] TraverseTree: Checking directory: C:\Git\GitVersionTaskTest\.git\refs\heads
  INFO [07/25/16 8:48:47:58] TraverseTree: Files in directory: C:\Git\GitVersionTaskTest\.git\refs\heads\master
  INFO [07/25/16 8:48:47:58] TraverseTree: master: 41, 25.07.2016 08:35:44, 25.07.2016 08:35:44 14b36548dd2f39d7004196fd3759adb513971f73
  INFO [07/25/16 8:48:47:59] TraverseTree: Checking directory: C:\Git\GitVersionTaskTest\.git\refs\remotes
  INFO [07/25/16 8:48:47:59] TraverseTree: Files in directory:
  INFO [07/25/16 8:48:47:59] TraverseTree: Checking directory: C:\Git\GitVersionTaskTest\.git\refs\remotes\origin
  INFO [07/25/16 8:48:47:59] TraverseTree: Files in directory: C:\Git\GitVersionTaskTest\.git\refs\remotes\origin\master
  INFO [07/25/16 8:48:47:59] TraverseTree: master: 41, 25.07.2016 08:35:51, 25.07.2016 08:35:51 14b36548dd2f39d7004196fd3759adb513971f73
  INFO [07/25/16 8:48:47:59] TraverseTree: Checking directory: C:\Git\GitVersionTaskTest\.git\refs\tags
  INFO [07/25/16 8:48:47:59] TraverseTree: Files in directory:
  INFO [07/25/16 8:48:47:59] traverse: C:\Git\GitVersionTaskTest\.git\refs;C:\Git\GitVersionTaskTest\.git\refs\heads;C:\Git\GitVersionTaskTest\.git\refs\heads\master: 14b36548dd2f39d7004196fd3759adb513971f73;C:\Git\GitVersionTaskTest\.git\refs\remotes;C:\Git\GitVersionTaskTest\.git\refs\remotes\origin;C:\Git\GitVersionTaskTest\.git\refs\remotes\origin\master: 14b36548dd2f39d7004196fd3759adb513971f73;C:\Git\GitVersionTaskTest\.git\refs\tags
  INFO [07/25/16 8:48:47:59] gitSystemHash: 7A635C869DB2D88E5F5212103448522CE007745E
  INFO [07/25/16 8:48:47:60] IsDynamicGitRepository: False
  INFO [07/25/16 8:48:47:60] Returning Project Root from DotGitDirectory: C:\Git\GitVersionTaskTest\.git - C:\Git\GitVersionTaskTest
  INFO [07/25/16 8:48:47:60] configFileHash:
  INFO [07/25/16 8:48:47:64] repositorySnapshotHash: 5DA1B05446AFA0ACF49FDA53234AF7B18BEDA6BC
  INFO [07/25/16 8:48:47:64] overrideConfigHash:
  INFO [07/25/16 8:48:47:64] compositeHash: 0B4D44B657F494B42591D17720965CFCF082A928
INFO [07/25/16 8:48:47:64] End: Creating cache key (Took: 62.00ms)

62ms (first run), afterwards 22ms.

@JakeGinnivan
Copy link
Contributor

Thanks @DanielRose, should have looked closer. I thought was more than just refs :P

@maxraab
Copy link
Contributor

maxraab commented Jul 26, 2016

We're going to introduce GitVersion and our first ci build (with GitLab CI) took about 47 minutes. Using this PR, the total time reduces to 2.5 minutes. I think this solves the problem.

{
files = Directory.GetFiles(currentDir);
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove these empty lines (line 93 as well).

@JakeGinnivan JakeGinnivan merged commit 399ad48 into GitTools:support/3.6.0 Jul 26, 2016
@JakeGinnivan
Copy link
Contributor

I've fixed up the style issues

@tofutim tofutim deleted the cachehash branch July 26, 2016 15:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants