#8455 Move mailman to newer release of Fedora or CentOS
Opened 2 years ago by smooge. Modified 2 months ago

Describe what you would like us to do:

Currently mailman3 servers are running on EL7 and are 'stuck' with a version of python which is not getting any support. Code updates to mailman3 require newer versions of python so we need to either move to EL8 or latest Fedora.

When do you need this to be done by? (YYYY/MM/DD)


Metadata Update from @smooge:
- Issue assigned to smooge

2 years ago

Metadata Update from @smooge:
- Issue priority set to: Waiting on Assignee (was: Needs Review)
- Issue tagged with: lists

2 years ago

Metadata Update from @smooge:
- Issue tagged with: backlog

2 years ago

@ngompa you had started looking at the packaging for EL8 no?

Yes, I've started working on this for almost all of our software. Mailman is already in Fedora, but I don't know where our HyperKitty and Postorius packages are...

@ngompa out of curiosity, have you had a chance to look where it's at now?

I've started looking. I think I found this: https://abompard.fedorapeople.org/current-mailman-hyperkitty-deps/

Is this what we're using? Or is it somewhere else?

Taking info from associate ticket

present deployed seems to be:

Postorius Documentation • GNU Mailman • Postorius Version 1.1.2

upstream has released up to 1.2.3


Describe what you need us to do:

Update to a later version, as there is a an 'across panels' leak of data when two mailing lists are open and an update occurs in one ... an async status message is displayed in all panels

When do you need this? (YYYY/MM/DD)

no deadline -- nice to have

When is this no longer needed or useful? (YYYY/MM/DD)

no expiration date

If we cannot complete your request, what is the impact?

a cross site exploit seems to exist

to demonstrate unsubscribe from the following mailing lists

For usage: https://lists.fedorahosted.org/archives/list/firewalld-users@lists.fedorahosted.org/

For development: https://lists.fedorahosted.org/archives/list/firewalld-devel@lists.fedorahosted.org/

then close the browser to get a fresh cache state (I use Firefox latest in CentOS)

open two tabs

choose one and enter a subscription transaction

change to the other tab

(the XSS leak is in a green box up top, saying:
Please check your inbox for further instructions

it appears in both tabs

(From #8629 ) GitHub recently sent us this message:

Hi @FedoraInfra,

On February 6th, 2020 at 00:07 (UTC) your application (Fedora Mailman) used an access token (with the User-Agent python-requests/2.9.1) as part of a query parameter to access an endpoint through the GitHub API:


Please use the Authorization HTTP header instead as using the `access_token` query parameter is deprecated.

Depending on your API usage, we'll be sending you this email reminder once every 3 days for each token and User-Agent used in API calls made on your behalf.
Just one URL that was accessed with a token and User-Agent combination will be listed in the email reminder, not all.

Visit https://developer.github.com/changes/2019-11-05-deprecated-passwords-and-authorizations-api/#authenticating-using-query-parameters for more information.

The GitHub Team

It's likely that the library that does social authentication in HyperKitty needs to be upgraded to use more recent APIs.

I'm working on updating the packages. Hopefully I'll have some progress in the coming days.

Metadata Update from @smooge:
- Assignee reset

2 years ago

Metadata Update from @cverna:
- Issue untagged with: backlog
- Issue tagged with: high-trouble, medium-gain

2 years ago

Metadata Update from @nphilipp:
- Issue assigned to nphilipp

2 years ago

As discussed on IRC, building current versions of the missing packages in COPR (and Koji as the case may be). Tracking status here.

Documenting caveats (differences to the old deployment etc.) here.

Documenting caveats (differences to the old deployment etc.) here.

I believe we have a way of shipping alternative versions of pytest... @churchyard might know something here.

But we could also simply just not run the tests on EPEL8. Not sure if we want to do that, given how much modifications we're doing...

I've been summoned. Reading the ticket now...

You can create a "compat" package with pytest4 in EPEL. People will love you for that. I can help.

Or you can use the Python 3.8 module for newer pytest, but that is another can of worms, because you don't have other deps.

Or you can figure out why does pytest-django actually need >=3.6 and have that fixed.

Let me know which way you'd like to proceed.

@churchyard In order of preference:

  1. Have pytest-django use older pytest
  2. pytest4 compat package
  3. py38 module

I don't think using the py38 module is realistic because we'd need to rebuild everything and adjust the packaging for it and deal with module building...

Package pytest-django 3.2.1?

Personally, I'd prefer a somewhat current pytest4 compat package over packaging version 3.2.x of pytest-django (which seems to be incompatible with newer pytest versions, i.e. potentially obstruct the way forward). I concur with staying away from a Python 3.8 based stack if we can avoid it (though it would intuitively mesh well with "here be current versions").

@churchyard, how would I go about such a compat package? If I'm not off track, it wouldn't be parallel-installable to the existing version, so wouldn't need special treatment in terms of file conflicts etc. Would it simply get a namespaced (S)RPM name, e.g. pytest4, python3-pytest4 with the proper Python provides so consumers like pytest-django could then do s.th. like BuildRequire: %{py3_dist pytest} >= 3.6 to get the compat package?

Pretty much what you say. I can help. In fact, if we get the python-pytest4 component, we might end up reusing it as a "lower" version compat package in Fedora, because stuff is incompatible with pytest 5. I just don't want to update to pytest 5 before the 3.9 bootstrap to avoid having to bootstrap both of them. I can submit the package for review and we can build it first in epel8 only, and after the 3.9 bootstrap, also build it in Fedora when updating pytest to 5.

Sounds good?

Sounds good?

I'm game, sounds good.

I'd need comaintainers for the epel8 branch.

Feel free to add @infra-sig. Since we need it for infra apps, it's just better to make sure everyone can maintain it.

OK, after some digging, this won't work:


Feel free to add @infra-sig. Since we need it for infra apps, it's just better to make sure everyone can maintain it.


@nphilipp I think we're going to have to go with trying to get things to work with older pytest on EPEL8. The other strategies seem basically unworkable.

@duck let's coordinate reviewing your (OSPO's) more up to date set of packages here. As a reference for others, here's your COPR and associated repo which I understand to be preliminary work for the "reviewing pipeline":

Can you add me to the respective groups so I can try my luck at updating & building the packages? My account on GitLab is @nilsph. Not sure if it's strictly needed, I could just put my changes into another repo but I guess it'd be good to have everything in one place.

@duck My name is @Conan_Kudo there as well...

I added you to the GL project, please only make changes to the api-3.1 branch (but you can make your own), so that it does not affect our production.

As for Copr, if you wish to be able to trigger builds could you make a request to be a member? (I cannot add people in the UI, it works the other way around only).

@duck I guess I have to be a member of the pizza-cats FAS group to work with the COPR repo, but according to FAS, the group's invite only. As an admin of the group, the group page should show you an "Add User" field where you can put in my FAS user name (nphilipp).

The pizza-cats group is for the OSPO Comminfra Team, to manage all our resources (even if right now I did not migrate them all yet), so I cannot use this group.

But you can make a request for a Copr Project and I will accept it. Once loggued in if you go to the project's Settings you can check for Request Permissions.

The Other way around is to create a new FAS group for the Mailman maintenance that we could use (I suppose) in src.fedoraproject.org too. It would be nice to have proper group maintenance on all resources.

So I finally got mailman3 to build fine again. Interestingly Click decided to uselessly change its quoting it broke the tests and I had to backport a recent version for EL8. So now it's fine on both rawhide and EL8. I'm going to create a PR on the official package.

python-mailman-client requires vcrpy and pytest-services for its test suite so that's my next steps.

The pizza-cats group is for the OSPO Comminfra Team, to manage all our resources (even if right now I did not migrate them all yet), so I cannot use this group.


But you can make a request for a Copr Project and I will accept it. Once loggued in if you go to the project's Settings you can check for Request Permissions.

Ah, wasn't aware of that. I've requested build permissions there.

The Other way around is to create a new FAS group for the Mailman maintenance that we could use (I suppose) in src.fedoraproject.org too. It would be nice to have proper group maintenance on all resources.

I think for taking care of stuff in koji, bodhi, Bugzilla, it's fine if we just use the existing groups, i.e. infra-sig on our end (I assume you have something comparable on the OSPO side).

I think for taking care of stuff in koji, bodhi, Bugzilla, it's fine if we just use the existing groups, i.e. infra-sig on our end (I assume you have something comparable on the OSPO side).

That's the goal of the newly created pizza-cats group, but limited to our custom stuff needed for our communities.

Should I join the infra-sig group?

Currently the mailman3 package and possibly some deps (I don't think he ever uploaded hyperkitty and postorius) are owned by abompard and it would be nice to get a change of ownership since he won't be involved anymore. I guess the python sig is taking care too, and some deps might not be useful anymore, thus some cleanup might be needed.

I hit a build issue and reported a bug to Copr:
Not reply yet but that's write annoying (or should I say blocking).
If you have any idea, please tell me or comment on the ticket.

I've finally managed to fix mailman3 package in Fedora again: https://bodhi.fedoraproject.org/updates/FEDORA-2020-9d4ef73a14

Next up: Postorius and HyperKitty

thanks @ngompa . last time I had time to work on this I made changes in this package for python-mailman-client and there were problems because python3-zope-i18nmessageid was not provided anymore by any of the dependencies. The fact that Fedora is a very fast moving target is really a pain honestly.

The problem I see with just bumping the version to an unreleased version is that we have no idea if this will work properly with Hyperkitty and Postorius. Versions need to be coordinated and that's a combination that's not supported upstream. Also this build is not tested on EL8, which is our (OSPO) target. I would have preferred backporting patches but that's way more work.

OK, we're lucky, the API version is still 3.1, so inter-communication between the components should work. Now we can see if the build of other components work too.

Metadata Update from @smooge:
- Issue tagged with: ops

2 years ago

Metadata Update from @kevin:
- Issue tagged with: mini-iniative

a year ago

It would be nice to have all those packages in the main fedora repo when the time comes.

That's the goal! :wink:

Know where a issue is to do that with the rest of the Fedora Web infrastructure? That would make it very convenient to update to the latest. (and safer, too)

So far Fedora has been too fast a moving target for me to handle this alone. Also there seem to be a confusion between being able to build and having the set all working properly together. Last time I checked mailman-client's tests were failing so that mean the software is not going to work well.

The real struggle is our patches on the stack. Porting those is painful. Unfortunately, I can't drop some of them, because they deal with non-upstream functionality and alternative Python dependencies. :frowning:

How hard is it to get those upstream?

Well I made it so hyperkitty can be updated for lists.fedoraproject.org some one just has to make a project on Fedora.

Currently it is 404: https://src.fedoraproject.org/rpms/HyperKitty


The main problematic patch that hasn't been upstreamed yet is this one: https://gitlab.com/mailman/mailman/-/merge_requests/721

Looks like this is still pending upstream. ;(

@ngompa is there a checklist of things people could help with here? aside from the patch?

Getting HyperKitty and Postorius packaged would be great. That can be done while I work through Mailman itself.


In the past at OSPO we started packaging a newer version, continuing Aurelien Bompard's work further to the Python 3 world. We do not plan to use the fast-moving Fedora target but EL-8 for stability but we wanted to collaborate on Fedora integration. This has proved just too much and Fedora was really changing too fast and I spent more timr trying to patch deps that kept breaking than finishing packaging the rest of the deps tree. Also when EL8 was just out so many packages were missing. I opened bugs but most were unanswered and when the packages were finally there the BR were not closed and I had no idea it was available. Nils Philippsen proposed to help and we started organizing but soon after he vanished without a trace. So when other projects came I just simply quit.

I'm trying to see if I could reboot the project now but this time I'm going to focus on EL8 only to focus on our production but if anyone wants to collaborate, you're welcome.

Our packaging repo is here:
and we use Copr to build them.

Most notably you'll find postorius and hyperkitty packages from Aurelien and updated to fix some problems. I'm having a look at newer version to see how the deps evolved.


Ironically, working on EL8 was killing me because the Mailman project focuses on supporting newer components rather aggressively, which is why I focused on Fedora.

I agree this is a problem, especially because Python 3.6 is rather old now and even when a project advertise 3.6+ certain base components like setuptools or pytests in the distro are not recent enough. This attempt may fail indeed.

I 'm not sure what to do about that but upgrading all the instances we manage for various communities every so often is clearly not something I'm looking for either. Having packages that build fine is not sufficient to ensure the service is going to work well; I'm not following closely but I hope there are plans for runtime tests (I read something about reusing autopkgtest in the past, not sure where it's going). I was hoping to get a newer version than the EL7 build we made. It's working fine but there are some fixes and a GDPR feature to delete/anonymize accounts that are missing.

The other option is using epel9 but that's not there and even if Copr supports it partially getting all the deps in is gonna take years again (my experience with epel8).

Of course if we did a very good job in Fedora in time to get it in EL10 that would be great and future-proof but that's really going to be too late for what we already have to maintain.

If I was to be pragmatic I would use another distro I know well where the work has already been done and is working fine, with an Ansible role only needing minor changes, but I was told that was not the way to go…

Getting it working in EPEL9 is going to be significantly easier than EPEL8, since the default Python is 3.9. If we get stuff into Fedora, branching them into EPEL9-next and then EPEL9 final will be pretty straightforward.

I guess I could work on a specific Fedora release but working with rawhide is what I tried already and before I could complete anything a new major version of Python came in and once again unreleased patches needed to be backported, or new ones created and then you're not building on the version you're targeting anymore. 3.10 is already out, so how long do we have to complete just about everything before it becomes the default? How is this going to work?

Nowadays, Python versions are released yearly, so the next Python version will arrive for Fedora Linux 37. It's a lot less painful than the last time, when Python versions were coming irregularly and quickly.

Some servers were just restarted today. That shouldn't hurt no?

@ngompa I'm indeed facing problems (need new attrs, pytest, etc etc). I guess I can give a try to your proposal :-). I'm going to start again on our Copr space and I'll try to push what I can do. If you have any advice, I'm all ears.

I can try helping with postorius

I've created a tracker on Bugzilla for the packages we need:
https://bugzilla.redhat.com/show_bug.cgi?id=2030061 (alias: MailmanTracker)

python-hyperkitty was just packaged

added python-postorius fedora link above.

Looks like we are down to just 4 bugs on the tracker?

Perhaps we could make some kind of coordinated push to get it over the line now?

How best can we help?

The four bugs currently open are requests for co-maintainership. Is there anything we can do to move those forward?

Also, we need to package up python3-django3 in Fedora to fix postorius and hyperkitty FTIs in Fedora.

2030928 - done just waiting on ?
2031302 - appears done also?
2031342 - done, added @salimma as admin
2033064 - is django... a bit of doom there about 'must support for 10 years'

Metadata Update from @kevin:
- Issue tagged with: blocked

2 months ago

Login to comment on this ticket.

Boards 2
ops Status: Backlog
mini-initative Status: Backlog