When pushing only the first branch/tag gets PR/Release marked

[mixman68]

• Gitea version (or commit ref): 1.8.1
• Git version: 2.11.0
• Operating system: Debian 9 stretch
• Database (use [x]):
  • PostgreSQL
  • MySQL
  • MSSQL
  • SQLite
• Can you reproduce the bug at https://try.gitea.io:
  • Yes (provide example URL) https://try.gitea.io/djgreg13/bugtag
  • No
  • Not relevant
• Log gist:

Description

If i push a commit in master and a tag with --all options, the front is desynchronised with my repo.
https://try.gitea.io/djgreg13/bugtag is the example, i push master and tag with --all

If i clone the repo, i see the tag, but on front no tags is shown in Releases page

If i push only a new tag, an automatic entry will be created on Releases page

I except to see in this page tag 1.0.0 (pushed with --all) and tag 1.0.1 (pushed with --tags) or i see only 1.0.1

Screenshots

you can see on https://try.gitea.io/djgreg13/bugtag

[zeripath]

This is due to the breaks in:

gitea/cmd/hook.go

Lines 193 to 217 in 54bd63c

	if newCommitID != git.EmptySHA && strings.HasPrefix(refFullName, git.BranchPrefix) {
	branch := strings.TrimPrefix(refFullName, git.BranchPrefix)
	repo, pullRequestAllowed, err := private.GetRepository(repoID)
	if err != nil {
	log.GitLogger.Error("get repo: %v", err)
	break
	}
	if !pullRequestAllowed {
	break
	}
	baseRepo := repo
	if repo.IsFork {
	baseRepo = repo.BaseRepo
	}
	if !repo.IsFork && branch == baseRepo.DefaultBranch {
	break
	}
	pr, err := private.ActivePullRequest(baseRepo.ID, repo.ID, baseRepo.DefaultBranch, branch)
	if err != nil {
	log.GitLogger.Error("get active pr: %v", err)
	break
	}

[zeripath]

Most of these should probably be continue

[zeripath]

However this problem is fixed in my #6993

[mrsdizzie]

Wouldn't this be happening in:

gitea/models/update.go

Lines 187 to 282 in 181b7c9

	func pushUpdate(opts PushUpdateOptions) (repo *Repository, err error) {
	isNewRef := opts.OldCommitID == git.EmptySHA
	isDelRef := opts.NewCommitID == git.EmptySHA
	if isNewRef && isDelRef {
	return nil, fmt.Errorf("Old and new revisions are both %s", git.EmptySHA)
	}
	repoPath := RepoPath(opts.RepoUserName, opts.RepoName)
	gitUpdate := exec.Command("git", "update-server-info")
	gitUpdate.Dir = repoPath
	if err = gitUpdate.Run(); err != nil {
	return nil, fmt.Errorf("Failed to call 'git update-server-info': %v", err)
	}
	owner, err := GetUserByName(opts.RepoUserName)
	if err != nil {
	return nil, fmt.Errorf("GetUserByName: %v", err)
	}
	repo, err = GetRepositoryByName(owner.ID, opts.RepoName)
	if err != nil {
	return nil, fmt.Errorf("GetRepositoryByName: %v", err)
	}
	gitRepo, err := git.OpenRepository(repoPath)
	if err != nil {
	return nil, fmt.Errorf("OpenRepository: %v", err)
	}
	if err = repo.UpdateSize(); err != nil {
	log.Error("Failed to update size for repository: %v", err)
	}
	var commits = &PushCommits{}
	if strings.HasPrefix(opts.RefFullName, git.TagPrefix) {
	// If is tag reference
	tagName := opts.RefFullName[len(git.TagPrefix):]
	if isDelRef {
	err = pushUpdateDeleteTag(repo, gitRepo, tagName)
	if err != nil {
	return nil, fmt.Errorf("pushUpdateDeleteTag: %v", err)
	}
	} else {
	// Clear cache for tag commit count
	cache.Remove(repo.GetCommitsCountCacheKey(tagName, true))
	err = pushUpdateAddTag(repo, gitRepo, tagName)
	if err != nil {
	return nil, fmt.Errorf("pushUpdateAddTag: %v", err)
	}
	}
	} else if !isDelRef {
	// If is branch reference
	// Clear cache for branch commit count
	cache.Remove(repo.GetCommitsCountCacheKey(opts.RefFullName[len(git.BranchPrefix):], true))
	newCommit, err := gitRepo.GetCommit(opts.NewCommitID)
	if err != nil {
	return nil, fmt.Errorf("gitRepo.GetCommit: %v", err)
	}
	// Push new branch.
	var l *list.List
	if isNewRef {
	l, err = newCommit.CommitsBeforeLimit(10)
	if err != nil {
	return nil, fmt.Errorf("newCommit.CommitsBeforeLimit: %v", err)
	}
	} else {
	l, err = newCommit.CommitsBeforeUntil(opts.OldCommitID)
	if err != nil {
	return nil, fmt.Errorf("newCommit.CommitsBeforeUntil: %v", err)
	}
	}
	commits = ListToPushCommits(l)
	}
	if opts.RefFullName == git.BranchPrefix+repo.DefaultBranch {
	UpdateRepoIndexer(repo)
	}
	if err := CommitRepoAction(CommitRepoActionOptions{
	PusherName: opts.PusherName,
	RepoOwnerID: owner.ID,
	RepoName: repo.Name,
	RefFullName: opts.RefFullName,
	OldCommitID: opts.OldCommitID,
	NewCommitID: opts.NewCommitID,
	Commits: commits,
	}); err != nil {
	return nil, fmt.Errorf("CommitRepoAction: %v", err)
	}
	return repo, nil
	}

Calling pushUpdateAddTag() etc...?

gitea/models/update.go

Line 111 in 181b7c9

func pushUpdateAddTag(repo *Repository, gitRepo *git.Repository, tagName string) error {

I suspect what happens is that for whatever reason sometimes when a tag gets pushed that pushUpdateAddTag() fails or isn't called, and then the tag is present on the remote and never run through pushUpdateAddTag() again. There are several places pushUpdateTag() could error, and several more even before that call, but you would have already needed error logging turned on to see I think.

Migrating or mirroring these example repos like the one above shows the proper releases (both 1.0.0 and 1.0.1) because it runs SyncReleasesWithTags (which then runs pushUpdateAddTag() for each tag )which does see that tag and create a proper release:

https://try.gitea.io/mrsdizzie/bugtag

FWIW the initial comment says "I except to see in this page tag 1.0.0 (pushed with --all) "
but git push --all doesn't push tags. However, it should have then pushed both tags on the next git push --tags (which it does in my testing, and I've yet to be able to reproduce pushing a tag, or several tags at once, that doesn't generate a release for each tag).

I want to find the cause of this and fix in case of a bug, but I also wonder if it would be helpful to add a button in the repo settings that can run SyncReleasesWithTags in case of any errors like this, which would at least provide a way to recover.

[zeripath]

That never gets called.

Look again at hook post-receive.

If you are pushing multiple things the breaks will stop the processing of everything after them.

To reproduce this just clone his repo and git push --tags to a local Gitea - only the first tag received will get a release made.

[zeripath]

#6993 fixes this because the private.HookPostReceive is called for every line in the push - but if you wanted to fix it without that just change those breaks to continue and it's fixed, however I'm not doing that as I'm certain that #6993 is a better way of doing hooks.

[mrsdizzie]

Hm when I clone the test repo and push it to a local repo then do git push --tagsit shows both of the tags as releases.

In addition, I can do something like this:

$ git tag test1
$ git tag test2
$ git tag test3
$ git push --tags

and it will create a release for test1, test2, test3 so I don't quite understand the only first tag gets processed part. Those hooks are running but they aren't stopping pushing multiple tags.

Are you not seeing that?

[zeripath]

Weird I don't get that on master. Did you push --all before pushing --tags?

[zeripath]

Yeah you're right the breaks shouldn't affect tag only pushing as they should only due for branch pushing.

I was definitely able to reproduce the problem on master with:

git push --all local
git push --tags local

[mrsdizzie]

Interesting, if I do the same thing it shows the release fine. Just so we're on the same page, I'm creating a new empty repo locally ("reproduce") and then doing this:

[mrsdizzie@mbp ~/Documents/source/github ] >  git clone https://try.gitea.io/djgreg13/bugtag.git
Cloning into 'bugtag'...
remote: Enumerating objects: 3, done.
remote: Counting objects: 100% (3/3), done.
remote: Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
[mrsdizzie@mbp ~/Documents/source/github ] >  cd bugtag/
[mrsdizzie@mbp ~/Documents/source/github/bugtag (master) ] >  git remote add local http://localhost:3000/mrsdizzie/reproduce.git
[mrsdizzie@mbp ~/Documents/source/github/bugtag (master) ] >  git push local --all
Enumerating objects: 3, done.
Counting objects: 100% (3/3), done.
Writing objects: 100% (3/3), 212 bytes | 212.00 KiB/s, done.
Total 3 (delta 0), reused 0 (delta 0)
To http://localhost:3000/mrsdizzie/reproduce.git
 * [new branch]      master -> master
[mrsdizzie@mbp ~/Documents/source/github/bugtag (master) ] >  git push local --tags
Total 0 (delta 0), reused 0 (delta 0)
To http://localhost:3000/mrsdizzie/reproduce.git
 * [new tag]         1.0.0 -> 1.0.0
 * [new tag]         1.0.1 -> 1.0.1

And then I can see both releases in Gitea. Tried via SSH too just to rule out any possible difference. Do you get any logging from the pushUpdate function on your end? Or maybe you see something obvious that I'm missing in the way I am testing it?

[zeripath]

Yeah that's what I did.

Hmm. I'll have to try it again.

Maybe it's an intermittent locking Heisenbug?

[zeripath]

Just realised that in post-receive we probably shouldn't stop on error because it's too late as the commits are in the repo - so we have to complete.

It's almost like we need a transaction log or some way of getting Gitea to just refresh the whole repo - as you suggested. (This would also be useful for allowing gitea to just take over repos.)

[mrsdizzie]

I'd be curious if you have error logging on to see if there is anything coming from the pushUpdate or pushUpdateAddTag functions (as there are places it could exit before creating the release but after pushing the tag). Or just anywhere I guess! I was hoping to be able to reproduce it as you are to see whats actually happening but I can't :(

And yea perhaps SyncReleasesWithTags should be made available to the user in some way, because as of now even if you found why the release wasn't made and fixed it (or explained why it was a valid error) there is no way to make it try again.