Skip to content

Conversation

@salvadorbs
Copy link

@salvadorbs salvadorbs commented Nov 17, 2022

Hi @davidbannon, I wrote a github workflow to automate the build of libqt6pas and the creation of the its packages. It runs on every push (and pull request) event, but could be limited to just release event (git tag). Let me know if it can be useful.

@davidbannon
Copy link
Owner

Wow, thanks, I am somewhat pushed for time right now, in the middle of a release of may app and had a lot of trouble with the Ubuntu PPA (#!!$#$). But will look closely at this when I can. Just have to get the Debian release done now ...

I am unsure about how much compute time github gives us (for free), the build time is quite long, an hour on my laptop (?) so maybe we need to consider what that means. And I am also encouraging Z to assign a new release number every time he makes a change, that has been a problem on Qt5 and I hope it is no longer one on Qt6.

But I am very keen to see how you did it !

Thanks for now ...

Davo

@salvadorbs
Copy link
Author

salvadorbs commented Nov 19, 2022

Wow, thanks, I am somewhat pushed for time right now, in the middle of a release of may app and had a lot of trouble with the Ubuntu PPA (#!!$#$). But will look closely at this when I can. Just have to get the Debian release done now ...
No problem. Take all the time you need. 😄

If you see my fork (see November 17th builds), build takes up to 30 minutes. With github free plan, you have 2000 minutes/month. About 65 builds.

Since this repo is not synchronized with the original one, we can better manage when to push the new changes and therefore the automatic build. However, we can always change the policy, and only start the build when a tag is created (and automatically upload the files created by it to the tag/release).

I wonder if it is possible to speed up the build by using parallel threads (for example, "make -j8" = 8 threads). I do some experiments with this PR.

Edit: Well, it works! Last build only lasted 13 minutes! If you want, test on your laptop to see if there are any improvements (github action should theoretically not do any build caching, but I want more test).

@davidbannon
Copy link
Owner

Salvador, an extended challenge, please have a look at #3

Its actually a long read, in summary, how much fine control do you get when you run your workflow ? Sorry, I still have not looked at it closely.

The library MUST be built on Fedora 35 (or unsupported U21.10, or one of the risky IMHO RH EL9 clones). The debs need to be built on a deb based machine. I am unsure where this all leads I must admit....

Davo

@salvadorbs
Copy link
Author

Well, Github only has these operating systems available for workflows:

For example, I'm using Ubuntu 22.04 lts in my workflow because i'm just experimenting with lazarus qt6 for my program on kde neon (ubuntu 22.04 based).

As you see, we are quite forced to use Ubuntu for Linux.

@davidbannon
Copy link
Owner

Yes, I thought that would be the case. I was using U2204 but we found that the resulting library was dependent on GLIBC 2.35 (that U2204 uses). That excluded things like

  • U21.10, out of support anyway
  • Fedora 35, comes out of support in a month or so.
  • RedHat Enterprise Linux et al, probably supported for several years from now !

I personally feel uncomfortable doing all this recent change really just for EL but we cannot ignore it. Has a lot of users and if we want to keep Lazarus relevent, cannot drop some....

Honestly, I feel a workflow approach here is probably unnecessary, based on libqt5pas experience, it does not change much. Right now we are on our third package release of libqt6pas but thats because of my mistakes and Z's changing of version numbers. So, a workflow would have made those harder to fix, not easier.

But, Salvador, I do like what you have done here, I may, with your permission copy some of it to my own project and maybe, not too far away, we will actually start using it here too. But right now, I see libqt6pas as just a transitional thing, maybe useful until the distros catch up.

Thanks for your contribution, valued ! (please do not delete this pull request, needs to stay ...)

Davo

@salvadorbs
Copy link
Author

salvadorbs commented Nov 20, 2022

I'm trying building in Ubuntu 20.04 (see https://github.com/salvadorbs/libqt6pas/actions/runs/3510238526/jobs/5879869222). Yes, custom repo backports. Brrr.... but I want to try it for glory and money (okok, I mean experience). 😄

Honestly, I feel a workflow approach here is probably unnecessary, based on libqt5pas experience, it does not change much. Right now we are on our third package release of libqt6pas but thats because of my mistakes and Z's changing of version numbers. So, a workflow would have made those harder to fix, not easier.

No problem. I will probably continue to use the workflow on a personal level. It's easier for me to have an automatic build instead of doing it manually locally (I have a vm with kde neon, so the performance isn't great).

But, Salvador, I do like what you have done here, I may, with your permission copy some of it to my own project and maybe, not too far away, we will actually start using it here too.

Yes, "my" workflow is yours! It is very simple (I don't have experience in these things. I'm learning).

But right now, I see libqt6pas as just a transitional thing, maybe useful until the distros catch up.

Yes, but main distro are very slow. I hate to think when Lazarus 2.4 will be released and the time it takes for the major distros to make the necessary packages.

@salvadorbs salvadorbs force-pushed the master branch 11 times, most recently from 8f7a956 to eeef10a Compare December 19, 2022 11:05
@davidbannon
Copy link
Owner

Oh, wow, you have been having some fun Salvador ! You might need to explain few things to me please ?

First, a suggestion, I suspect you are concentrating at the wrong end of the process, by responding to a push, that implies that the workflow is triggered by me (or you) updating this github repository. But in fact, it needs to be setup to respond when Zelkio changes the version number in the Qt6Pas.pro file over in gitlab. Building is easy but spotting when Z makes a change, not so !

I have a script qt6update.bash that downloads the cbindings and compares Qt6Pas.pro, if they differ it updates my local git tree. In fact, all it needs to do is download Qt6Pas.pro from https://gitlab.com/freepascal.org/lazarus/lazarus/-/blob/main/lcl/interfaces/qt6/cbindings/Qt6Pas.pro and compare that. Dead easy in a bash script but I had quick look at running a bash script in a runner but could not see how the get the script there in the first place.

OH, (smacks his forehead), I see how we get the script there, have it in the source tree ! So, so slow sometimes ....

Ideally, if that script, running in a github runner found the version changed, all it needs do is raise an issue here and i/you would be notified.

Anyway, my questions, honestly, I have no idea what can be done with YAML, so if my questions sound silly, it because they are !

Brilliant you can build a Windows binary too !
Brilliant idea to use that PPA to feed Qt62 to U2004, I just tried it, builds OK, have you tested it ? The PPA offers 6.2.2 but Z was adament that we needed to use 6.2.3. Bad things happened when we used 6.2.4 for example. Sadly, that means we need to test widely ...

  1. Right now, you use the deb-package script, that should be removed, it depends on alien and produces very poor RPMs. Need to use package-lib - it makes stand alone RPMs and has better error checking. Sorry !
  2. Where do the libraries and packages end up when the runner finished ? I am concerned that it will commit them to the repo, if that happens every time we push, then we add 4 to 8meg per linux architecture to the git commit history, I guess the windows DLL is the same size and an arm64 (at least) about the same too. Having this trigger off a push might just be too much.
  3. Can we save the packages back somewhere outside the src tree ? They need to pop up under Releases....

Seriously exciting stuff !

I wonder if I should establish another, separate github project with a different name so we can experiment there, I'd rather not risk adding those binaries to the commit history unless we establish its unavoidable ? Hmm, might be a way to strip large files out of git commit history perhaps ?

I am going to run some tests on the libs made under Ubuntu2004 and the PPA, that contribution of yours is already enormous !

Davo

@davidbannon
Copy link
Owner

One more thing Salvaror, you also pushed some changes that Zelko made to the cpp code in the last day or so, generally, I don't do a new release unless he has reved the version number. Otherwise, we end up with two versions out in the wild with the same version number but a different behavour. So, unless there is a really good reason, I'd wait until Z gives it a new number.

If its urgent, we can do something like 6.2.3a but neither of our scripts will handle that !

Zelko had some problems with the numbering in Qt5 because it started from a very funny base (including a ~) so we discussed numbering quite carefully here.

Davo

@salvadorbs
Copy link
Author

salvadorbs commented Dec 20, 2022

Oh, wow, you have been having some fun Salvador ! You might need to explain few things to me please ?

I'm just trying to see if it's possible to use Lazarus qt6 on Windows. I wanted to write to you, but then I went locally, and I haven't pushed the latest changes yet.

First, a suggestion, I suspect you are concentrating at the wrong end of the process, by responding to a push, that implies that the workflow is triggered by me (or you) updating this github repository. But in fact, it needs to be setup to respond when Zelkio changes the version number in the Qt6Pas.pro file over in gitlab. Building is easy but spotting when Z makes a change, not so !

To keep git history clean, I prefer to edit the last commit and force push it instead of creating lots of different commits. At every push force, workflow runs and I see if it works.

Brilliant idea to use that PPA to feed Qt62 to U2004, I just tried it, builds OK, have you tested it ? The PPA offers 6.2.2 but Z was adament that we needed to use 6.2.3. Bad things happened when we used 6.2.4 for example. Sadly, that means we need to test widely ...

With the Windows build, I discovered the existence of Install QT Action (https://github.com/marketplace/actions/install-qt), which in turn uses AQTInstall (https://github.com/miurahr/aqtinstall). Given the problems with apt, the ideal would be to use this action and to have more control over the version of qt to use (see future 6.5, https://forum.lazarus.freepascal.org/index.php/topic,61237.msg464055.html#msg464055).

Right now, you use the deb-package script, that should be removed, it depends on alien and produces very poor RPMs. Need to use package-lib - it makes stand alone RPMs and has better error checking. Sorry !

No problem. The part of the workflow for Linux, in my opinion, needs to be redone (see above + cache to speed up).

Where do the libraries and packages end up when the runner finished ? I am concerned that it will commit them to the repo, if that happens every time we push, then we add 4 to 8meg per linux architecture to the git commit history, I guess the windows DLL is the same size and an arm64 (at least) about the same too. Having this trigger off a push might just be too much.

For now, after a successful workflow run, it will upload compiled lib and packages as action artifact in a zip file (see https://github.com/salvadorbs/libqt6pas/blob/35d143335f898b6e26b9919ed1b8a1d6d0653b39/.github/workflows/main.yml#L90)

Can we save the packages back somewhere outside the src tree ? They need to pop up under Releases....

Yes, see https://github.com/marketplace/actions/gh-release

One more thing Salvaror, you also pushed some changes that Zelko made to the cpp code in the last day or so, generally, I don't do a new release unless he has reved the version number. Otherwise, we end up with two versions out in the wild with the same version number but a different behavour. So, unless there is a really good reason, I'd wait until Z gives it a new number.

Yes, I knew. https://forum.lazarus.freepascal.org/index.php/topic,61645.0.html, But I needed a test machine to see if the changes were ok or not. Then I switched to my computer… sob.

Anyway, this pull request (or if you prefer, my fork repo) is only experimental. So it must not be merged to your repo for any reason. I'll keep working on it and IF we get a good result (that everyone likes), I'll create a new merge request with a clean git history. And I would do the same thing for libqt5pas as well.

I hope I have answered everything and not forgotten anything. Let me know if you don't understand something, unfortunately my English isn't the best.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants