Skip to content

Old git cli (maybe < 2) can not do IsEmpty check correctly #19086

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
IT-AlexKor opened this issue Mar 14, 2022 · 4 comments
Closed

Old git cli (maybe < 2) can not do IsEmpty check correctly #19086

IT-AlexKor opened this issue Mar 14, 2022 · 4 comments
Labels
issue/confirmed Issue has been reviewed and confirmed to be present or accepted to be implemented issue/workaround it is or has a workaround type/bug

Comments

@IT-AlexKor
Copy link
Contributor

Gitea Version

1.16.3

Git Version

1.8.3.1

Operating System

CentOS Linux 7 (Core)

How are you running Gitea?

Self-hosted instance in ProxMox

configuration:

APP_NAME = <ERASED>
RUN_USER = git
RUN_MODE = prod

[security]
INTERNAL_TOKEN     = <ERASED>
INSTALL_LOCK       = true
SECRET_KEY         = <ERASED>
PASSWORD_HASH_ALGO = pbkdf2
DISABLE_GIT_HOOKS  = false

[database]
DB_TYPE  = postgres
HOST     = <ERASED>
NAME     = <ERASED>
USER     = <ERASED>
PASSWD   = <ERASED>
SCHEMA   =
SSL_MODE = disable
CHARSET  = utf8
PATH     = /var/lib/gitea/data/gitea.db
LOG_SQL  = false

[repository]
ROOT = /var/lib/gitea/data/gitea-repositories

[server]
SSH_DOMAIN       = <ERASED>
DOMAIN           = <ERASED>
HTTP_PORT        = 3000
ROOT_URL         = https://<ERASED>/
DISABLE_SSH      = false
SSH_PORT         = 22
LFS_START_SERVER = true
LFS_CONTENT_PATH = /var/lib/gitea/data/lfs
LFS_JWT_SECRET   = <ERASED>
OFFLINE_MODE     = false

[mailer]
ENABLED = true
HOST    = <ERASED>
FROM    = <ERASED>
USER    =
PASSWD  =

[service]
REGISTER_EMAIL_CONFIRM            = false
ENABLE_NOTIFY_MAIL                = false
DISABLE_REGISTRATION              = false
ALLOW_ONLY_EXTERNAL_REGISTRATION  = false
ENABLE_CAPTCHA                    = false
REQUIRE_SIGNIN_VIEW               = true
DEFAULT_KEEP_EMAIL_PRIVATE        = false
DEFAULT_ALLOW_CREATE_ORGANIZATION = true
DEFAULT_ENABLE_TIMETRACKING       = true
NO_REPLY_ADDRESS                  = noreply.localhost

[picture]
DISABLE_GRAVATAR        = false
ENABLE_FEDERATED_AVATAR = true

[openid]
ENABLE_OPENID_SIGNIN = true
ENABLE_OPENID_SIGNUP = true

[session]
PROVIDER = file

[log]
MODE      = console
LEVEL     = info
ROOT_PATH = /var/lib/gitea/log
ROUTER    = console

[metrics]
ENABLED = true

[mirror]
MIN_INTERVAL = 1m

[indexer]
ISSUE_INDEXER_TYPE = elasticsearch
ISSUE_INDEXER_CONN_STR = http://<ERASED>:9200
ISSUE_INDEXER_NAME = gitea_issues
ISSUE_INDEXER_PATH = indexers/issues.bleve
REPO_INDEXER_ENABLED = true
REPO_INDEXER_TYPE = elasticsearch
REPO_INDEXER_CONN_STR = http://<ERASED>:9200
REPO_INDEXER_NAME = gitea_codes
REPO_INDEXER_PATH = indexers/repos.bleve
UPDATE_BUFFER_LEN = 20
MAX_FILE_SIZE = 1048576
REPO_INDEXER_INCLUDE =
REPO_INDEXER_EXCLUDE = resources/bin/**
STARTUP_TIMEOUT = 30s

Database

PostgreSQL

Can you reproduce the bug on the Gitea demo site?

No

Log Gist

No response

Description

Steps to reproduce

  1. Prepare repo on bitbucket with some commits/code
  2. Migrate repo from Bitbucket (with mirroring enabled) via https clone url
  3. After migration ends look at the migrated repository page. It will look like there is no code in repo
  4. Go into repo dir on file system and check via "git log" command that this repo has some commits (and code too)

In DB i see that row with imported repository (in "repository" table) has value "is_empty = true". I think that this behavior caused by change in isempty method 913d6f3

Migration was fine on version 1.15.8, today we update our instance and this new behavior was appered

Example of old code (before commit 913d6f3)
[root@ci-gitea .git]# git rev-list --all --count --max-count=1
1
[root@ci-gitea .git]# git show-ref --head ^HEAD$
[root@ci-gitea .git]

Screenshots

No response
@IT-AlexKor
Copy link
Contributor Author

Current workaround - update migrated repos with "is_empty = true" to "is_empty = false", after that repo content is displayed properly. Any cron jobs wasn't help, Add to reindex queue wasn't help too

2022/03/14 19:17:12 ...dules/git/command.go:141:RunWithContext() [D] /usr/bin/git clone --mirror --quiet -- https://<ERASED>@bitbucket.org/<ERASED>/<ERASED>.git /var/lib/gitea/data/gitea-repositories/<ERASED>/<ERASED>.git
2022/03/14 19:17:15 ...dules/git/command.go:143:RunWithContext() [D] /var/lib/gitea/data/gitea-repositories/<ERASED>/<ERASED>.git: /usr/bin/git update-server-info
2022/03/14 19:17:15 ...dules/git/command.go:143:RunWithContext() [D] /var/lib/gitea/data/gitea-repositories/<ERASED>/<ERASED>.git: /usr/bin/git rev-parse
2022/03/14 19:17:15 ...dules/git/command.go:143:RunWithContext() [D] /var/lib/gitea/data/gitea-repositories/<ERASED>/<ERASED>.git: /usr/bin/git show-ref --head ^HEAD$
2022/03/14 19:17:15 ...dules/git/command.go:143:RunWithContext() [D] /var/lib/gitea/data/gitea-repositories/<ERASED>/<ERASED>.git: /usr/bin/git cat-file --batch
2022/03/14 19:17:15 ...dules/git/command.go:143:RunWithContext() [D] /var/lib/gitea/data/gitea-repositories/<ERASED>/<ERASED>.git: /usr/bin/git cat-file --batch-check
2022/03/14 19:17:15 models/repo.go:291:updateRepoSize() [I] [SQL] SELECT COALESCE(sum("size"),0) FROM "lfs_meta_object" WHERE (repository_id = $1) [68] - 2.936611ms
2022/03/14 19:17:15 ...s/repository/repo.go:157:MigrateRepositoryGitData() [I] [SQL] UPDATE "repository" SET "size" = $1 WHERE "id"=$2 [20745 68] - 2.728419ms
2022/03/14 19:17:15 ...odels/repo/mirror.go:135:InsertMirror() [I] [SQL] INSERT INTO "mirror" ("repo_id","interval","enable_prune","updated_unix","next_update_unix","lfs_enabled","lfs_endpoint") VALUES ($1,$2,$3,$4,$5,$6,$7) RETURNING "id" [68 0s true 1647274635 1647274635 false ] - 2.83418ms
2022/03/14 19:17:15 ...ns/gitea_uploader.go:116:CreateRepo() [I] [SQL] BEGIN TRANSACTION [] - 1.071294ms
2022/03/14 19:17:15 ...s/repository/repo.go:197:MigrateRepositoryGitData() [I] [SQL] UPDATE "repository" SET "owner_id" = $1, "owner_name" = $2, "lower_name" = $3, "name" = $4, "description" = $5, "website" = $6, "original_service_type" = $7, "original_url" = $8, "default_branch" = $9, "num_watches" = $10, "num_stars" = $11, "num_forks" = $12, "num_issues" = $13, "num_closed_issues" = $14, "num_pulls" = $15, "num_closed_pulls" = $16, "num_milestones" = $17, "num_closed_milestones" = $18, "num_projects" = $19, "num_closed_projects" = $20, "is_private" = $21, "is_empty" = $22, "is_archived" = $23, "is_mirror" = $24, "status" = $25, "is_fork" = $26, "fork_id" = $27, "is_template" = $28, "template_id" = $29, "size" = $30, "is_fsck_enabled" = $31, "close_issues_via_commit_in_any_branch" = $32, "topics" = $33, "trust_model" = $34, "avatar" = $35, "updated_unix" = $36 WHERE "id"=$37 [10 <ERASED> <ERASED> <ERASED>   1 https://<ERASED>@bitbucket.org/<ERASED>/<ERASED>.git  1 0 0 0 0 0 0 0 0 0 0 true true false true 1 false 0 false 0 20745 false false null default  1647274635 68] - 2.255227ms
2022/03/14 19:17:15 models/repo.go:291:updateRepoSize() [I] [SQL] SELECT COALESCE(sum("size"),0) FROM "lfs_meta_object" WHERE (repository_id = $1) [68] - 2.759688ms
2022/03/14 19:17:15 models/repo.go:709:UpdateRepository() [I] [SQL] UPDATE "repository" SET "size" = $1 WHERE "id"=$2 [20745 68] - 2.428638ms
2022/03/14 19:17:15 ...igrations/migrate.go:201:migrateRepository() [I] [SQL] COMMIT [] - 1.403757ms
2022/03/14 19:17:15 ...dules/git/command.go:143:RunWithContext() [D] /var/lib/gitea/data/gitea-repositories/<ERASED>/<ERASED>.git: /usr/bin/git rev-parse
2022/03/14 19:17:15 ...igrations/migrate.go:206:migrateRepository() [T] migrating topics
2022/03/14 19:17:15 ...dules/git/command.go:143:RunWithContext() [D] /var/lib/gitea/data/gitea-repositories/<ERASED>/<ERASED>.git: /usr/bin/git cat-file --batch
2022/03/14 19:17:15 ...dules/git/command.go:143:RunWithContext() [D] /var/lib/gitea/data/gitea-repositories/<ERASED>/<ERASED>.git: /usr/bin/git cat-file --batch-check
2022/03/14 19:17:15 ...igrations/migrate.go:467:migrateRepository() [I] [SQL] BEGIN TRANSACTION [] - 986.644µs
2022/03/14 19:17:15 ...ns/gitea_uploader.go:977:Finish() [I] [SQL] INSERT INTO issue_index (group_id, max_index) VALUES ($1,1) ON CONFLICT (group_id) DO UPDATE SET max_index = issue_index.max_index+1 [68] - 2.872318ms
2022/03/14 19:17:15 models/issue.go:1047:RecalculateIssueIndexForRepo() [I] [SQL] SELECT  MAX("index") FROM "issue" WHERE (repo_id=$1) LIMIT 1 [68] - 3.144796ms
2022/03/14 19:17:15 ...igrations/migrate.go:467:migrateRepository() [I] [SQL] UPDATE "issue_index" SET max_index=$1 WHERE group_id=$2 [0 68] - 1.951991ms
2022/03/14 19:17:15 ...igrations/migrate.go:129:MigrateRepository() [I] [SQL] COMMIT [] - 1.664575ms
2022/03/14 19:17:15 models/repo.go:1026:repoStatsCorrectNumWatches() [I] [SQL] UPDATE "repository" SET num_watches=(SELECT COUNT(*) FROM "watch" WHERE repo_id=$1 AND mode<>2) WHERE id=$2 [68 68] - 3.692319ms
2022/03/14 19:17:15 models/repo.go:1030:repoStatsCorrectNumStars() [I] [SQL] UPDATE "repository" SET num_stars=(SELECT COUNT(*) FROM "star" WHERE repo_id=$1) WHERE id=$2 [68 68] - 2.638365ms
2022/03/14 19:17:15 models/repo.go:1083:repoStatsCorrectNumIssues() [I] [SQL] UPDATE "repository" SET num_issues=(SELECT COUNT(*) FROM "issue" WHERE repo_id=$1 AND is_pull=$2) WHERE id=$3 [68 false 68] - 3.102515ms
2022/03/14 19:17:15 models/repo.go:1087:repoStatsCorrectNumPulls() [I] [SQL] UPDATE "repository" SET num_pulls=(SELECT COUNT(*) FROM "issue" WHERE repo_id=$1 AND is_pull=$2) WHERE id=$3 [68 true 68] - 2.905033ms
2022/03/14 19:17:15 models/repo.go:1096:repoStatsCorrectNumClosedIssues() [I] [SQL] UPDATE "repository" SET num_closed_issues=(SELECT COUNT(*) FROM "issue" WHERE repo_id=$1 AND is_closed=$2 AND is_pull=$3) WHERE id=$4 [68 true false 68] - 3.09832ms
2022/03/14 19:17:15 models/repo.go:1100:repoStatsCorrectNumClosedPulls() [I] [SQL] UPDATE "repository" SET num_closed_pulls=(SELECT COUNT(*) FROM "issue" WHERE repo_id=$1 AND is_closed=$2 AND is_pull=$3) WHERE id=$4 [68 true true 68] - 2.578161ms
2022/03/14 19:17:15 models/repo.go:1238:UpdateRepoStats() [I] [SQL] UPDATE "label" SET num_issues=(SELECT COUNT(*) FROM "issue_label" WHERE label_id=id) WHERE repo_id=$1 [68] - 2.534305ms
2022/03/14 19:17:15 models/repo.go:1238:UpdateRepoStats() [I] [SQL] UPDATE "label" SET num_closed_issues=(SELECT COUNT(*) FROM "issue_label","issue" WHERE "issue_label".label_id="label".id AND "issue_label".issue_id="issue".id AND "issue".is_closed=$1) WHERE "label".repo_id=$2 [true 68] - 3.061631ms
2022/03/14 19:17:15 models/repo.go:1060:milestoneStatsCorrectNumIssuesRepo() [I] [SQL] SELECT "milestone".id FROM "milestone" WHERE "milestone".num_closed_issues!=(SELECT COUNT(*) FROM "issue" WHERE "issue".milestone_id="milestone".id AND "issue".is_closed=$1) OR "milestone".num_issues!=(SELECT COUNT(*) FROM "issue" WHERE "issue".milestone_id="milestone".id) AND "milestone".repo_id = $2 [true 68] - 3.266192ms
2022/03/14 19:17:15 ...ns/gitea_uploader.go:986:Finish() [I] [SQL] UPDATE "repository" SET "status" = $1, "updated_unix" = $2 WHERE "id"=$3 [[0 1647274635](tel:01647274635) 68] - 3.146106ms
2022/03/14 19:17:15 ...i/v1/repo/migrate.go:211:Migrate() [T] Repository migrated: <ERASED>/<ERASED>

@noerw noerw added the type/bug label Mar 21, 2022
@wxiaoguang
Copy link
Contributor

wxiaoguang commented Mar 22, 2022 •
edited
Loading

This problem is caused by a very old git client. The problem is git show-ref --head ^HEAD$ is not supported by old git client.

Use 2.x won't have any problem.

@IT-AlexKor
Copy link
Contributor Author

You are right, I missed it - will try to update. Thank you

@wxiaoguang wxiaoguang changed the title Possibly invalid behavior of isempty detection after repo was migrated from BitBucket Old git cli (maybe <= 2) can not do IsEmpty check correctly Apr 6, 2022
@wxiaoguang wxiaoguang changed the title Old git cli (maybe <= 2) can not do IsEmpty check correctly Old git cli (maybe < 2) can not do IsEmpty check correctly Apr 6, 2022
@wxiaoguang wxiaoguang added issue/confirmed Issue has been reviewed and confirmed to be present or accepted to be implemented issue/workaround it is or has a workaround labels Apr 6, 2022
@IT-AlexKor
Copy link
Contributor Author

Works great. Thanky you for support and sorry for late update

@wxiaoguang wxiaoguang mentioned this issue Apr 29, 2022
Closed
@go-gitea go-gitea locked and limited conversation to collaborators May 3, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
issue/confirmed Issue has been reviewed and confirmed to be present or accepted to be implemented issue/workaround it is or has a workaround type/bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@wxiaoguang @noerw @IT-AlexKor