Add support for sha256 repositories

[AdamMajer]

Currently only SHA1 repositories are supported by Gitea. This adds support for alternate SHA256 with the additional aim of easier support for additional hash types in the future.

Fixes: #13794
Limited by: go-git/go-git#899
Depend on: #28138

[AdamMajer]

As additional comments for this PR,

• It's WIP. It compiles now and passes tests (unit tests and the sqlite tests) and I've managed to import a SHA256 repo and things are not exploding anymore
• Many things are not yet tested
• This is my first project where I need to use Go :-)
• This is my first attempt to understand Gitea
• GoGit is completely unsupported by this patch

Comments are welcome. I do not expect this to land soon and I expect some changes.

[techknowlogick]

Whoa! This is a huge accomplishment! Thank you <3

[AdamMajer]

There is a funny issue I've reported upstream (git) now. And it's funny by semi-blocking here.

If you do,

git init --object-format=sha256 a
git clone a b
git-receive-pack --advertise-refs a
git-receive-pack --advertise-refs b

You will see respective results,

00cb0000000000000000000000000000000000000000000000000000000000000000
capabilities^{}report-status report-status-v2 delete-refs side-band-64k quiet atomic ofs-delta object-format=sha256 agent=git/2.35.3

00b10000000000000000000000000000000000000000
capabilities^{}report-status report-status-v2 delete-refs side-band-64k quiet atomic ofs-delta object-format=sha1 agent=git/2.35.3

Basically, the git-clone uses git-upload-pack to fetch the remote objects, but this doesn't send the header if there are no objects. git-receive-pack, used as part of push, does send a NULL OID in case of empty repository, so hash type can be determined. git-upload-pack does not, so git-clone initializes uses default hash, and this is sha1.

Gitea uses git-clone everywhere as ways of checking out a branch and inserting README.md or LICENSE or whatever. This of course fails on an empty repository. So, to make SHA256 usable without errors everywhere, the git-clone needs to have have the hash passed to it. Fortunately, this is possible by using GIT_DEFAULT_HASH env variable, otherwise there would be no workaround. But, this will make the patch unnecessarily more intrusive, and it will not fix the case where you do a git clone of remote empty sha256 repository to initialize it. You have to do a git init --object-type=sha256 instead. At least until upsteam git issue is fixed.

TL;DR: This means that if you create a SHA256 repository with nothing in it and clone it, you will get a SHA1 repository. This then will fail to push.

[AdamMajer]

Looks like we have a preliminary patch for this issue upstream now git/git@8b214c2

[AdamMajer]

Looks like go-git/go-git#707 is starting point for SHA256 go-git implementation. I'll refactor my patches to align more with those changes.

[AdamMajer]

The patch is now rebased on top of latest main and I've added DB migration for larger hash fields that I've tested with postgres. Next step is to finally add gogit support.

[6543]

is this you @AdamMajer -> https://sha2git.com ?!?

[AdamMajer]

is this you @AdamMajer -> https://sha2git.com ?!?

No, I'm not related to this.

Aside, now I have a little more time to work on this so hopefully we can have this merged soon ™️ 🤞

[AdamMajer]

@6543 50c5488 should be reverted, it probably will break things

[6543]

@6543 50c5488 should be reverted, it probably will break things

well the insert yes ... did add a count check

as for the other one ... see #23894 (comment)

we either have to update it first or we dont

[6543]

ok I'll put my hands away again ;)

#23894 (comment)
Y/N?

[6543]

Well avter a 360° commit session we are back ... to start diff 😆

could you add at least a codecomment that the update statement only exist to catch strange edgecases if they exist ... ?

[AdamMajer]

Thanks for all the work in helping this patch to finally land. Extra credits go to @lunny for all the help!

[6543]

wow thanks for the hard work to all - we have initiall support ... now we can build ontop that e.g. for go-git ...

[denyskon]

Great that this finally landed! Did you by any chance check the label in repo heeder on mobile? It could be broken as we display icons instead of labels there. If not, I'll check later...

[6543]

no i did not checked the mobile ui 😅 - that's then for a followup if it's broken :)

[AdamMajer]

wow thanks for the hard work to all - we have initiall support ... now we can build ontop that e.g. for go-git ...

GoGit needs to implement new API for ability to support different hashes. Currently they only support sha256 if compiled with sha256 buildtag, and at same time this drops sha1 support. This is a no-go (pun kind of intended). Proper support needs to be implemented for their next major version since API changes are required. Then we can add that here, which should be easy. See: go-git/go-git#706

Any issues here with this patch, please open a new issue.