revive pytest-html maintenance

[ssbarnea]

pytest-html needs an active pool of maintainers as it slowly degraded.

I am offering to help with most maintenance tasks and especially around making/keeping CI/CD green and reviewing upcoming contributions.

pytest-html is a key ingredient in testing implementation we use on several openstack projects, also directly used with my pytest-molecule plugin, so I have a strong interest in keep in it working.

I hope others will also be interested to join this, as it much better to avoid me becoming another single point of failure.

One of the first things I want to do is to migrate testing to Github Actions as it far more reliable and flexible than Travis, and assure we have a devel job with tests with unreleased versions of major dependencies, like pytest (so we avoid surprises when they release). like we had with pytest-6.0.0. Also Github Actions can take care of release process in a safe way, which involve using a bot account to publish releases.

PS. I do not yet have the rights on the project.

[ssbarnea]

@pytest-html-admin please assist.

[webknjaz]

@pytest-dev/pytest-html-admin @pytest-dev/pytest-html-developers ^

[ssbarnea]

Indeed, as non member I cannot mention these teams and when I hover over the links I get "Hovercard not available", clicking gives a 404, which is usually what a private team would do (common but confusing security measure)

[davehunt]

I don't agree that pytest-html maintenance needs 'reviving' or that it's slowly degraded. I'm certainly not an active maintainer these days, but @BeyondEvil has been doing a great job of keeping the plugin going, and without him I'm certain the plugin would be in a far worse state. That said, there's always room for improvement, and it sounds like you have some good ideas. I encourage you to work with @BeyondEvil to get these implemented, and I welcome you to the project.

[ssbarnea]

Yes @BeyondEvil did a great job and without him the project would be in a very poor state. Still this does not mean that the project does not need more attention, #315 got stalled/blocked because lack of a simple rebase.

https://github.com/pytest-dev/pytest-html/pull/320/files adds testing with devel. There is also a LOT of work to be done on maintenance, for example this project is not tested at all with pytho3.8, which is the current stable version of python. Scheduling daily or weekly runs on master can be easily activated by anyone using travis interface.

[RibeiroAna]

Hi there, unfortunately since last year I could not find time to contribute to Pytest-HTML since I have a full time job and this year was not particularly easy for me (I broke my leg at begging of the year, before all the pandemic thing). I still have a lot of interest in the Pytest project and I hope to return to the project whenever it's possible. By now I don't know what I can do to contribute since I have been away of the project for so long.

[nicoddemus]

Hi everyone!

Indeed, as non member I cannot mention these teams and when I hover over the links I get "Hovercard not available", clicking gives a 404, which is usually what a private team would do (common but confusing security measure)

TBH I was not aware of that, I thought those teams were public and everyone could mention them.

I particular have seen @ssbarnea's work in general and would welcome him as a contributor to pytest-html. If nobody objects I would be happy to send him an invitation (although his "Collaborator" badge on the top of his comment frames already say he has been invited to contribute to the project, so perhaps @davehunt already did it?)

Still this does not mean that the project does not need more attention, #315 got stalled/blocked because lack of a simple rebase.

Indeed that's unfortunate... having more contributors with write access certainly helps mitigating this.

As for myself I try to help here whenever I can, but my time is already spread thin over pytest itself and other plugins where I'm the sole maintainer, so I would be more than happy to have more people joining pytest-html. 👍

[nicoddemus]

I broke my leg at begging of the year

Ouch! Sorry to hear that @RibeiroAna. Espero que nessa altura tu já esteja melhor!

[gnikonorov]

I don’t mind taking on an active role in maintaining this plugin if no one objects. I’ve been meaning to contribute to it for a while.

[nicoddemus]

@gnikonorov you have my vote, you have already proven a valuable member to the pytest core team, I'm sure you can greatly help out pytest-html as well. 👍

[BeyondEvil]

I particular have seen @ssbarnea's work in general and would welcome him as a contributor to pytest-html. If nobody objects I would be happy to send him an invitation (although his "Collaborator" badge on the top of his comment frames already say he has been invited to contribute to the project, so perhaps @davehunt already did it?)

Hi @nicoddemus

That was me (attempting) to add @ssbarnea as a "full" contributor. Obviously I failed.

I attempted to do it the correct way just now (or at least what I perceive to be the correct way, it doesn't seem to be documented anywhere), but it looks like I don't have the correct permissions?

I'm helping @davehunt to maintain:

pytest-html
pytest-selenium
pytest-variables
pytest-base-url
pytest-metadata

I'm in the admin and developers teams for all of the above plugins, but it seems I don't have the correct permissions to add others to those groups.

It's up to yourself and Dave to decide if and what access I should have to those teams, but being an admin of a plugin should at least give me the ability to add contributors to the -developers teams imo.

I'm also more than happy to add @gnikonorov as a contributor to the plugin and team.

[BeyondEvil]

Hi there, unfortunately since last year I could not find time to contribute to Pytest-HTML since I have a full time job and this year was not particularly easy for me (I broke my leg at begging of the year, before all the pandemic thing). I still have a lot of interest in the Pytest project and I hope to return to the project whenever it's possible. By now I don't know what I can do to contribute since I have been away of the project for so long.

Sorry to hear that! I hope you're feeling better now! @RibeiroAna

One way to contribute in a big way, is to be available to look at PRs. Even tho you've been away, I don't think the plugin has changed in any significant way. And, seeing how you've been a long time contributor, your input will be greatly appreciated! 🙏

One thing that has frustrated me, which is true for all the plugins I help maintain, is the slow turnaround/feedback-loop. I'll create a PR (maintenance, bug fix, new feature, whatever) and it can be days (sometimes even weeks) before I get any feedback or an approval. This means that if a change is requested or a discussion needs to happen - it can take a long time before the PR is finally merged.

This leads to a couple of issues:

• We risk loosing community support because they might think that the project has been abandoned. Something that @ssbarnea mentioned in an email to the pytest-dev mailinglist.
• Turnaround/feedback-loop delay escalates even further due to contex-switching in the sense that if too much time passes between feedback etc. it takes longer to re-understand what it is you're dealing with, and you risk pushing the work even further into the future.

I'm the first to admit that I'm guilty of all of the above and I understand we live busy lives, I do too!

But a problem not raised, is a problem not fixed.

[BeyondEvil]

One of the first things I want to do is to migrate testing to Github Actions as it far more reliable and flexible than Travis, and assure we have a devel job with tests with unreleased versions of major dependencies, like pytest (so we avoid surprises when they release). like we had with pytest-6.0.0. Also Github Actions can take care of release process in a safe way, which involve using a bot account to publish releases.

I think more or less all pytest-dev plugins (incl. pytest itself) uses Travis, so I'm against moving to Github Actions unless there's very good reason to do so.

I haven't experienced any reliability issues with Travis, and I do believe they support scheduled runs - I'm just not sure if they are available on the free/OS tier. That is something I'm going to look into right now.

I do however agree that we need "devel" testing!

As far as release process goes, apart from some manual steps (basically changelog stuff), I don't see any issues with the current process. But I'm happy to hear detailed feedback on how Github Actions would improve that process.

[ssbarnea]

I am really happy to see everyone stepping in to help here. It make me quite positive about the future as we identified few things to improve. Yes, very often the admins or even cores forget the user experience interacting with a repo can be quite limiting, it happened to me several times and I ended up creating and using a spare github account for validating different scenarios.

The good part is that some time ago GitHub made the process easier with a set of default access rights. "Read" is useless for opensource as anyone already has it but the rest are important to know.

Triage can be given without much worry, i allows user to perform secretary work: open/edit/close issues, open/edit/close PRs but NOT voting review or merges. Usually I send the invite to Triage right away when someone takes interest in project (after checking a little his social history), sometimes I do even without being asked.

Write is mainly a core, so he perform voting reviews and can push. If the repository is well configured to require green CI and one review, giving this right is also easy, as they would be very unlikely to do mistakes alone (they would not be able to merge their own PRs).

Admin gives full rights and is usually needed for some plumbing work like enabling/disabling different CI/CD integrations, editing repository topics or managing the branch access. I find myself editing these quite often of some repositories that I nurture on ansible-community but in general is not needed. An Admin needs to know very well how github ACL works and be a trustable person, not someone that would ever go rogue.

[webknjaz]

I would be happy to send him an invitation (although his "Collaborator" badge on the top of his comment frames already say he has been invited to contribute to the project

Colaborator means that he's been added to the repository, not the org. In the org settings, he'll show up as an "outside" contributor unless invited to the org teams specifically... Then, the label would be Member.

[webknjaz]

As far as release process goes, apart from some manual steps (basically changelog stuff), I don't see any issues with the current process. But I'm happy to hear detailed feedback on how Github Actions would improve that process.

GHA just gives more free resources in general + job logs are embedded in the GH UI which is nice + they have built-in artifacts that you can save from jobs. Also, flexibility seems higher.

[webknjaz]

I haven't experienced any reliability issues with Travis, and I do believe they support scheduled runs - I'm just not sure if they are available on the free/OS tier. That is something I'm going to look into right now.

You can enable cron runs in Travis for free. Just go to the project settings there and scroll down, then select a branch and an interval.

[BeyondEvil]

I haven't experienced any reliability issues with Travis, and I do believe they support scheduled runs - I'm just not sure if they are available on the free/OS tier. That is something I'm going to look into right now.

You can enable cron runs in Travis for free. Just go to the project settings there and scroll down, then select a branch and an interval.

I don't have the settings menu in Travis, can you give me that level of access @nicoddemus ? Seems @davehunt couldn't. 🤷‍♂️

[The-Compiler]

I think more or less all pytest-dev plugins (incl. pytest itself) uses Travis, so I'm against moving to Github Actions unless there's very good reason to do so.

FWIW pytest itself switched to GitHub Actions in December 2019 for almost all its jobs, the only thing still done via Travis is tests for Python 3.5.0 and .1 as those patch versions apparently aren't easily available via GitHub Actions. A couple of plugins I looked at (pytest-timeout, pytest-randomly, pytest-twisted, pytest-mock, pytest-flask, probably more) are using it as well.

[webknjaz]

@The-Compiler have you tried setup-python v2 or deadsnakes? I think they may have fixed that by adding more interpreters downloadable on the fly...

[The-Compiler]

@webknjaz Only 3.5.4 is available via GHA, and 3.5.7 via deadsnakes. Let's take this to pytest-dev/pytest#6369 though.

[BeyondEvil]

I think more or less all pytest-dev plugins (incl. pytest itself) uses Travis, so I'm against moving to Github Actions unless there's very good reason to do so.

FWIW pytest itself switched to GitHub Actions in December 2019 for almost all its jobs, the only thing still done via Travis is tests for Python 3.5.0 and .1 as those patch versions apparently aren't easily available via GitHub Actions. A couple of plugins I looked at (pytest-timeout, pytest-randomly, pytest-twisted, pytest-mock, pytest-flask, probably more) are using it as well.

Well, there you go. ☺️

I can look into moving to GA. Hopefully that would also solve the permissions issue with Travis.

[The-Compiler]

Here's a quite simple example of running tests/linters (though via tox) and then deploying to PyPI, if that's of any help: https://github.com/The-Compiler/pytest-mccabe/blob/master/.github/workflows/main.yml

I guess it'd also make sense to use the account of @BeyondEvil (and an API token instead of username/password) rather than @davehunt for releases?

[ssbarnea]

Please do not use personal accounts for releases, make a service/bot account for this purpose and configure its token in correct places. Personal account usage is less safe, misleading (as the person trigering the release is not necessary the same and also decreases trust related to having a reliable release pipelines. Ideally we should have a "pytest" bot account for the entire pytest-dev organization, as this would allow configuring the token in a single place instead each project. Anyway it would make sense for pytest to lead the way with that practice and allow other projects to follow-up.

[BeyondEvil]

I agree with @ssbarnea about having a pytest-bot shared between all the pytest-dev projects. This is good for other things as well, like integrations.

What needs to be solved however is how to share those credentials.

Maybe use something like 1Password or Lastpass? But I’m not sure a free/OS tier allows for multiple users.

[ssbarnea]

I am already using 1Password and I do have a free team license they offered me for use with https://github.com/pycontribs/ organization, so if anyone wants I can create a team there and nobody needs to install 1Password or pay for it. Alternatively, you can ask them for another team account for PyTest organization too. AFAIK, LastPass or Bitwarden do not have such plans, which is really sad, especially for the second one which is the most open-source one.

[The-Compiler]

Anyway it would make sense for pytest to lead the way with that practice and allow other projects to follow-up.

I opened https://github.com/pytest-dev/pytest/issues/7585. The PyPI token can just be configured as an organization-wide GitHub secret, then it can be used in GitHub Actions by any pytest-dev repo. The "real" account details still would need to be shared in some different way, but I think it'd make more sense to have things like that shared with only a handful of core developers (just like e.g. access to the pytest.org DNS).

[nicoddemus]

Just found time to read this now, sorry for joining late.

I've commented on https://github.com/pytest-dev/pytest/issues/7585, I'm not sure having a shared token is a good idea in the sense that it might not really make things more convenient and will probably be less secure. I suggest we move the discussion about shared token to that thread to avoid having the discussion happen in two places. 👍

As for @ssbarnea being a member, I think the proper way to do it is to add him to the https://github.com/orgs/pytest-dev/teams/pytest-html-developers team. Can you try to add him there @BeyondEvil?

Right now @ssbarnea is listed as a "read-only" collaborator:

[BeyondEvil]

As for @ssbarnea being a member, I think the proper way to do it is to add him to the https://github.com/orgs/pytest-dev/teams/pytest-html-developers team. Can you try to add him there @BeyondEvil?

Right now @ssbarnea is listed as a "read-only" collaborator:

I tried adding him in that team as outlined here, but I'm either blind (can't find any add button or the like, or I don't have the correct perms/access.

I'll see if I can at least give him write perms.

Edit: @ssbarnea now has write perm.

This is my "team"-view:

[nicoddemus]

@BeyondEvil is also an admin already in this view:

[webknjaz]

@davehunt I just checked in @aio-libs and found that in the team view I can select a member with a checkbox and then there's a menu on top that has "Change role..."

But it looks like on your screenshot those checkboxes are missing.

[webknjaz]

@nicoddemus on your screenshot he does not have a maintainer label, but is just a member of that group (meaning that he has no power over its members).

[nicoddemus]

Yeah he's on the "admin" team, but you are right he's not a maintainer! Thanks!

Changed both @BeyondEvil and @RibeiroAna to maintainer, thanks @webknjaz!

But I see the distinction now: "teams" don't imply any access rights, it is just a grouping of users.

[BeyondEvil]

Yeah he's on the "admin" team, but you are right he's not a maintainer! Thanks!

Changed both @BeyondEvil and @RibeiroAna to maintainer, thanks @webknjaz!

But I see the distinction now: "teams" don't imply any access rights, it is just a grouping of users.

Do you mind making me a ”maintainer” of the other plugins as well (base-url, metadata, selenium, variables)?

If @davehunt agrees of course.

[nicoddemus]

@BeyondEvil done all, let me know if there's anything missing.

(In retrospect I don't think the existence of "admin" and "developers" groups help much... probably better to just let each plugin create teams as they see fit. It is even stranger that people outside the organization cannot ping public teams, which would be an excellent use for this).

[BeyondEvil]

Thanks @nicoddemus, I appreciate the help! 🙏

It is even stranger that people outside the organization cannot ping public teams, which would be an excellent use for this.

That is strange 🤔

[webknjaz]

But I see the distinction now: "teams" don't imply any access rights, it is just a grouping of users.

Well, I mean... They may imply certain privileges because you then grant those teams some level of access to repos. Just not within the same group.

[webknjaz]

@nicoddemus looks like with all this discussion we forgot to invite Gleb #318 (comment) too.

[nicoddemus]

Ahh indeed, thanks for the reminder, but now @BeyondEvil can take it from here I assume?

[gnikonorov]

Thank you for remembering @webknjaz

[BeyondEvil]

Ahh indeed, thanks for the reminder, but now @BeyondEvil can take it from here I assume?

Yes, sir!

[BeyondEvil]

Thank you for remembering @webknjaz

Done! Welcome to the team @gnikonorov ! 👊

With great power comes etc... 😊

Looking forward to your contributions. 👍

[gnikonorov]

Thank you @BeyondEvil! Looking forward to contributing

[ssbarnea]

@BeyondEvil @davehunt @RibeiroAna @nicoddemus Any chance for me to be added as project owner on github and pypi?

I aim to keep my involvement on CI/CD, assuring it goes as slim as possible and cover as much as possible. Related to #322

[nicoddemus]

Sure @ssbarnea, changed your role to Admin here on GH, however I don't have access to PyPI's settings.

[BeyondEvil]

I have access to PyPi, but I want an approval from @davehunt before I add you to it. :)

[davehunt]

I have access to PyPi, but I want an approval from @davehunt before I add you to it. :)

You have it, and future approval for anyone else that you want to add. 😊

[ssbarnea]

My username is the same as on github ssbarnea, only on irc I use different user: zbr.

[ssbarnea]

@BeyondEvil Maybe you can add pytest-dev and ssbarnea accounts on pypi?

For testpypi only @davehunt can do it as he is the only package owner there, see https://test.pypi.org/project/pytest-html/ -- still use of testpypi is a nice to have but not hard-requirement.

[BeyondEvil]

Done ✅

Both added as maintainer. Thanks for reminding me! 🙏 @ssbarnea

[ssbarnea]

Thanks. Apparently weekends are the best to show some love to our projects. Already raised PR to make the switch at #360

[davehunt]

For testpypi only @davehunt can do it as he is the only package owner there, see https://test.pypi.org/project/pytest-html/ -- still use of testpypi is a nice to have but not hard-requirement.

I'm happy to add another owner here. I don't see @BeyondEvil as a valid user on TestPyPI though. Let me know what user(s) to add.

[ssbarnea]

@davehunt Add me, some user as github: ssbarnea and i can take care of the rest.

[davehunt]

@davehunt Add me, some user as github: ssbarnea and i can take care of the rest.

Thanks, I've sent the invite.

[BeyondEvil]

Huh, weird @davehunt

I'll investigate and see why I'm not on testpypi

[ssbarnea]

Account database is split, dont ask me why....

On Tue, 27 Oct 2020 at 23:10, Jim Brännlund ***@***.***> wrote: Huh, weird @davehunt <https://github.com/davehunt> I'll investigate and see why I'm not on testpypi — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#318 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAZAX5KINOSC6UPL4OC3VTSM5HO5ANCNFSM4PLJJZOQ> .

-- -- /sorin

[BeyondEvil]

Weird, I do exist in test.pypi.org as BeyondEvil 🤷‍♂️ @davehunt

https://test.pypi.org/user/BeyondEvil/

[ssbarnea]

@davehunt added me to testpypi only as a maintainer so I cannot add @BeyondEvil or bots. What i did was to a token from my account limited to pytest-html to unblock the publishing.

Still, we should address the ownership of testpypi and have at least two humans as owners.

[davehunt]

Weird, I do exist in test.pypi.org as BeyondEvil 🤷‍♂️ @davehunt

Owner invitation sent.