Packager.io configuration for automated Debian/Ubuntu/Centos package builds

[Janhouse]

I wanted to migrate from Gogs to Gitea but noticed that there are no prebuilt Debian packages.
Gogs is successfully using packager.io to automatically build packages for multiple distributions.
I copied their configuration, bump up the buildpack version and disabled Debian 7 since it has reached end of life.
Currently with this configuration most distribution packages build successfully and I successfully installed Gitea using one of those packages on Debian 9.

You can check the generated packages here: https://packager.io/gh/Janhouse/gitea

If you merge this, you will have to create packager.io account and set it to build only release branch/tags.

[codecov-io]

Codecov Report

❗ No coverage uploaded for pull request base (master@c1da790). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master    #6671   +/-   ##
=========================================
  Coverage          ?   41.17%           
=========================================
  Files             ?      423           
  Lines             ?    58416           
  Branches          ?        0           
=========================================
  Hits              ?    24051           
  Misses            ?    31187           
  Partials          ?     3178

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c1da790...c5e6e01. Read the comment docs.

[strk]

LGTM

[lunny]

@Janhouse Thanks for your PR. We have that account and repository see https://packager.io/gh/go-gitea/gitea .

[lunny]

Another question. How to configure to add build branches release/*?

[Janhouse]

@lunny, you can set up branches to build in https://packager.io/gh/go-gitea/gitea/edit
Of course it will build successfully only after merging .pkgr.yml in that branch.

[lunny]

Have added release/*, but cannot find the build configuration when git tagged.

[Janhouse]

@lunny, probably because those release branches/tags don't have the .pkgr.yml file.
You might have to exclude all the previous release tags/branches, and probably also release/-dev, release/-rc*. Or maybe just create a separate "stable" branch that would have the latest stable release for packager.io builds and include just that.

[lunny]

It seems https://packager.io/gh/go-gitea/gitea/edit is buggy, when save the changes, it said successfully but not.

[lafriks]

Do we really need support for Ubuntu 12.04? We should probably drop that and instead add 18.04

[Janhouse]

@lafriks you are correct, since I don't use Ubuntu myself, I didn't pay attention to it. Packager also supports other distributions, would be great if someone looked at those as well at some point.

[lunny]

I think we shouldn't merge this before that site fix the bug https://packager.io/gh/go-gitea/gitea/edit can not save the configuration change.

[Janhouse]

@lunny, that is weird because it saved changes just fine on Firefox for me. Maybe you have some extension that interferes?

[kolaente]

So we're essentially waiting for snap fixes now?

[lunny]

@kolaente No. this is not related with snap I think.

[Janhouse]

@lunny any decisions made? I think you still have to merge this to create properly working packages that put the custom config under /etc/gitea/ and so on.
And then the question is - would you create a "stable" branch that would be used for these packages?

[lunny]

@Janhouse we have no special one stable branch, we have multiple stable branches. All branches named release/v* are stable branches.

[Janhouse]

@lunny you should probably create a stable "latest" release branch that contains the latest release that is not alpha/beta/rc so that it could be distributed through packages. It is probably not a very good idea to provide packages for master branch, since it does not have clear version and could have potentially broken things in there.

[tboerger]

There are no plans to provide a separate "stable" branch. We got already the release branches.

[Janhouse]

@tboerger in that scenario you can't really automatically provide decent packages for distributions with latest stable version. If you wanted to, you would have to tell which version is the latest one (Is it 1.8.0 or 1.9.0, or maybe some totally different one). And if you provided packages for all of the different minor versions that you currently support, at some point you will stop supporting those (no security/bug fixes) and there is no way to let the user know that they should update to a different version.

Let's use Linux kernel as an example. They have multiple versions that they support (as seen on https://www.kernel.org/ ) but the "mainline" one is the latest one that is considered stable and it is used by most rolling release distros. Could you please consider doing something similar with Gitea so that we don't have to come here frequently to check for new stable versions?

My idea was that you have a separate branch or tag named "stable", "latest" or whatever else you wish that would contain most recent version that is out of release candidate stage. Currently it would point to the same commit that is tagged with v1.8.0, but once 1.9.0 is released, it would point to the same commit as that tag.

[kolaente]

Couldn't we automate the update of a stable branch with drone? To something like every time we push to a release/v* branch, it would push to that stable branch. We'd maybe need a way to tell drone it should only update patches to the latest, not the last version though.

[lunny]

@kolaente It seems we can do that automatically via drone. But the latest should be a branch or a tag? Maybe someone could write a drone plugin to do that.

[kolaente]

@lunny A branch, since the stable needs to get updated constantly.

[tboerger]

Have fun with resolving the conflicts on that approach. If package.io isn't capable to use the gitea versioning it sound to me like the simply wrong tool.

I have anyway already said that I would prefer proper packaging in favor of packager.io because of the strange executable wrapping and stuff like that.

Build packages with fpm, publish them to bintray or to a custom repository and that's it.

[techknowlogick]

I agree with @tboerger. We should build our own packages with (n)fpm, and https://packagecloud.io/ and https://bintray.com/ are both very good options for hosting.

[tboerger]

The drone plugins for bintray/packagecloud just got to be updated. Building a new plugin for nfpm would be pretty easy.

[Janhouse]

Packager.io treats each branch as a separate update repository. All you have to do is create that branch to build from. From @kolaente I understand that drone has similar requirement. Or someone will have to maintain the "stable" branch name/tag in some other place.

If you prefer other tool to package with then go for it but with this PR already makes it work with packager.io. That system handles almost everything automatically and is free for open source projects.
@tboerger, do you have some concerns regarding that tool?

[Janhouse]

Then I'm closing this PR hoping that someone configures it with packagecloud/bintray.
Automatic updates using distribution built in package managers are very desired.

[lafriks]

We need something that would have single repository and build packages only for tags

[techknowlogick]

I've just published: https://github.com/techknowlogick/drone-nfpm which means we can use drone to build rpms and debs on just tags.

[Janhouse]

@lafriks, but for which tag? Tag stucture for Gitea doesn't identify the latest stable version, unless you use some regexp and compare numbers across all tags, or specify the tag somewhere to build packages for.

[tboerger]

We don't need to care about it in that case, ayn tag gets a deb/rpm, publishes it to the repository and the package managers decide on their own which version is the latest one.

[Janhouse]

@tboerger, seems like you also have dev and rc tags that can have higher version number. Should package manager install those as updates?

[tboerger]

If we really publish a dev/git package it will have a separate package name since this can be potentially broken. I'm not sure if we really want to publish rc versions in the same repo as regular stable versions.

[Janhouse]

Yes. Then if you want to automate building packages from all https://github.com/go-gitea/gitea/ tags, you probably want to filter out those -dev and -rc tags since they are there.

[zeripath]

If we do do this I think we really must provide the package in such a way that Gitea obeys the FHS either by shadowing the real gitea binary behind a script that sets the default environment variables or we recompile it so that you don't need the environment variables.

People will be extremely confused that Gitea is not behaving in a normal fashion if we do not do this. Eg. #6855

[tboerger]

No need to recompile or something like that. Just a proper init script and default config

[zeripath]

You cannot place the Gitea binary directly in /usr/bin on a FHS compliant distribution and expect it to work.

It simply will not work and expecting people to set environment variables is not going to cut it I'm afraid. We regularly get semi-irate users who struggle for hours because Gitea does not behave like they expect.

So either we hide it away the weirdness with a script - or we recompile so it's FHS compliant by default but what we absolutely must not do is encourage people to plonk Gitea in /usr/bin

Examples of weirdnesses:

• Gitea will panic if you try to run it from a non-readable working directory. Although it will never resolve against the working directory. This is due to Macaron's init. I have written a pr to mitigate against this Change the working directory if we can't stat it  #6559 but it's clearly not an ideal solution and my Macaron pr is languishing Remove the init panic go-macaron/macaron#178 . Macaron's init runs extremely early due to poor package dependency management on our behalf - e.g models depends on macaron... (!) - but even if we fix that we'd likely still need to do a temporary directory shimmy in init()!!
• Gitea resolves its paths against the location of the Gitea binary - not the working directory, this is only settable through an environment variable which is somewhat opaquely labelled GITEA_WORK_DIR. This should be settable in main as an option and I think I might put a PR against this v. soon - at least then it would be discoverable through the application help.
• Gitea looks for its configuration app.ini in a totally non-standard location for FHS systems, yes it's settable but then you have to remember to set it every time you want to run Gitea.

Yes all of these are settable through environment variables, yes we have the documentation, but it's non-standard and therefore unexpected. If we're providing packages we must comply to the expectations of the FHS.

So again, we must either shadow Gitea behind a script to hide some of this weirdness or we build with the appropriate settings so that Gitea is at least partially FHS compliant by default or we're going to suffer a lot of complaints.