#8455 Move mailman to newer release of Fedora or CentOS
Opened a year ago by smooge. Modified a month ago

Describe what you would like us to do:

Currently mailman3 servers are running on EL7 and are 'stuck' with a version of python which is not getting any support. Code updates to mailman3 require newer versions of python so we need to either move to EL8 or latest Fedora.


When do you need this to be done by? (YYYY/MM/DD)

Meh..



Metadata Update from @smooge:
- Issue assigned to smooge

a year ago

Metadata Update from @smooge:
- Issue priority set to: Waiting on Assignee (was: Needs Review)
- Issue tagged with: lists

a year ago

Metadata Update from @smooge:
- Issue tagged with: backlog

a year ago

@ngompa you had started looking at the packaging for EL8 no?

Yes, I've started working on this for almost all of our software. Mailman is already in Fedora, but I don't know where our HyperKitty and Postorius packages are...

@ngompa out of curiosity, have you had a chance to look where it's at now?

I've started looking. I think I found this: https://abompard.fedorapeople.org/current-mailman-hyperkitty-deps/

Is this what we're using? Or is it somewhere else?

Taking info from associate ticket

present deployed seems to be:

Postorius Documentation • GNU Mailman • Postorius Version 1.1.2

upstream has released up to 1.2.3

https://postorius.readthedocs.io/en/latest/news.html

Describe what you need us to do:

Update to a later version, as there is a an 'across panels' leak of data when two mailing lists are open and an update occurs in one ... an async status message is displayed in all panels

When do you need this? (YYYY/MM/DD)

no deadline -- nice to have

When is this no longer needed or useful? (YYYY/MM/DD)

no expiration date

If we cannot complete your request, what is the impact?

a cross site exploit seems to exist

to demonstrate unsubscribe from the following mailing lists

For usage: https://lists.fedorahosted.org/archives/list/firewalld-users@lists.fedorahosted.org/

For development: https://lists.fedorahosted.org/archives/list/firewalld-devel@lists.fedorahosted.org/

then close the browser to get a fresh cache state (I use Firefox latest in CentOS)

open two tabs

choose one and enter a subscription transaction

change to the other tab

(the XSS leak is in a green box up top, saying:
Please check your inbox for further instructions
)

it appears in both tabs

(From #8629 ) GitHub recently sent us this message:

Hi @FedoraInfra,

On February 6th, 2020 at 00:07 (UTC) your application (Fedora Mailman) used an access token (with the User-Agent python-requests/2.9.1) as part of a query parameter to access an endpoint through the GitHub API:

https://api.github.com/user

Please use the Authorization HTTP header instead as using the `access_token` query parameter is deprecated.

Depending on your API usage, we'll be sending you this email reminder once every 3 days for each token and User-Agent used in API calls made on your behalf.
Just one URL that was accessed with a token and User-Agent combination will be listed in the email reminder, not all.

Visit https://developer.github.com/changes/2019-11-05-deprecated-passwords-and-authorizations-api/#authenticating-using-query-parameters for more information.

Thanks,
The GitHub Team

It's likely that the library that does social authentication in HyperKitty needs to be upgraded to use more recent APIs.

I'm working on updating the packages. Hopefully I'll have some progress in the coming days.

Metadata Update from @smooge:
- Assignee reset

9 months ago

Metadata Update from @cverna:
- Issue untagged with: backlog
- Issue tagged with: high-trouble, medium-gain

8 months ago

Metadata Update from @nphilipp:
- Issue assigned to nphilipp

7 months ago

As discussed on IRC, building current versions of the missing packages in COPR (and Koji as the case may be). Tracking status here.

Documenting caveats (differences to the old deployment etc.) here.

Documenting caveats (differences to the old deployment etc.) here.

I believe we have a way of shipping alternative versions of pytest... @churchyard might know something here.

But we could also simply just not run the tests on EPEL8. Not sure if we want to do that, given how much modifications we're doing...

I've been summoned. Reading the ticket now...

You can create a "compat" package with pytest4 in EPEL. People will love you for that. I can help.

Or you can use the Python 3.8 module for newer pytest, but that is another can of worms, because you don't have other deps.

Or you can figure out why does pytest-django actually need >=3.6 and have that fixed.

Let me know which way you'd like to proceed.

@churchyard In order of preference:

  1. Have pytest-django use older pytest
  2. pytest4 compat package
  3. py38 module

I don't think using the py38 module is realistic because we'd need to rebuild everything and adjust the packaging for it and deal with module building...

Package pytest-django 3.2.1?

Personally, I'd prefer a somewhat current pytest4 compat package over packaging version 3.2.x of pytest-django (which seems to be incompatible with newer pytest versions, i.e. potentially obstruct the way forward). I concur with staying away from a Python 3.8 based stack if we can avoid it (though it would intuitively mesh well with "here be current versions").

@churchyard, how would I go about such a compat package? If I'm not off track, it wouldn't be parallel-installable to the existing version, so wouldn't need special treatment in terms of file conflicts etc. Would it simply get a namespaced (S)RPM name, e.g. pytest4, python3-pytest4 with the proper Python provides so consumers like pytest-django could then do s.th. like BuildRequire: %{py3_dist pytest} >= 3.6 to get the compat package?

Pretty much what you say. I can help. In fact, if we get the python-pytest4 component, we might end up reusing it as a "lower" version compat package in Fedora, because stuff is incompatible with pytest 5. I just don't want to update to pytest 5 before the 3.9 bootstrap to avoid having to bootstrap both of them. I can submit the package for review and we can build it first in epel8 only, and after the 3.9 bootstrap, also build it in Fedora when updating pytest to 5.

Sounds good?

Sounds good?

I'm game, sounds good.

I'd need comaintainers for the epel8 branch.

Feel free to add @infra-sig. Since we need it for infra apps, it's just better to make sure everyone can maintain it.

OK, after some digging, this won't work:

Sorry.

Feel free to add @infra-sig. Since we need it for infra apps, it's just better to make sure everyone can maintain it.

👍

@nphilipp I think we're going to have to go with trying to get things to work with older pytest on EPEL8. The other strategies seem basically unworkable.

@duck let's coordinate reviewing your (OSPO's) more up to date set of packages here. As a reference for others, here's your COPR and associated repo which I understand to be preliminary work for the "reviewing pipeline":

Can you add me to the respective groups so I can try my luck at updating & building the packages? My account on GitLab is @nilsph. Not sure if it's strictly needed, I could just put my changes into another repo but I guess it'd be good to have everything in one place.

@duck My name is @Conan_Kudo there as well...

I added you to the GL project, please only make changes to the api-3.1 branch (but you can make your own), so that it does not affect our production.

As for Copr, if you wish to be able to trigger builds could you make a request to be a member? (I cannot add people in the UI, it works the other way around only).

@duck I guess I have to be a member of the pizza-cats FAS group to work with the COPR repo, but according to FAS, the group's invite only. As an admin of the group, the group page should show you an "Add User" field where you can put in my FAS user name (nphilipp).

The pizza-cats group is for the OSPO Comminfra Team, to manage all our resources (even if right now I did not migrate them all yet), so I cannot use this group.

But you can make a request for a Copr Project and I will accept it. Once loggued in if you go to the project's Settings you can check for Request Permissions.

The Other way around is to create a new FAS group for the Mailman maintenance that we could use (I suppose) in src.fedoraproject.org too. It would be nice to have proper group maintenance on all resources.

So I finally got mailman3 to build fine again. Interestingly Click decided to uselessly change its quoting it broke the tests and I had to backport a recent version for EL8. So now it's fine on both rawhide and EL8. I'm going to create a PR on the official package.

python-mailman-client requires vcrpy and pytest-services for its test suite so that's my next steps.

The pizza-cats group is for the OSPO Comminfra Team, to manage all our resources (even if right now I did not migrate them all yet), so I cannot use this group.

Understood.

But you can make a request for a Copr Project and I will accept it. Once loggued in if you go to the project's Settings you can check for Request Permissions.

Ah, wasn't aware of that. I've requested build permissions there.

The Other way around is to create a new FAS group for the Mailman maintenance that we could use (I suppose) in src.fedoraproject.org too. It would be nice to have proper group maintenance on all resources.

I think for taking care of stuff in koji, bodhi, Bugzilla, it's fine if we just use the existing groups, i.e. infra-sig on our end (I assume you have something comparable on the OSPO side).

I think for taking care of stuff in koji, bodhi, Bugzilla, it's fine if we just use the existing groups, i.e. infra-sig on our end (I assume you have something comparable on the OSPO side).

That's the goal of the newly created pizza-cats group, but limited to our custom stuff needed for our communities.

Should I join the infra-sig group?

Currently the mailman3 package and possibly some deps (I don't think he ever uploaded hyperkitty and postorius) are owned by abompard and it would be nice to get a change of ownership since he won't be involved anymore. I guess the python sig is taking care too, and some deps might not be useful anymore, thus some cleanup might be needed.

I hit a build issue and reported a bug to Copr:
https://bugzilla.redhat.com/show_bug.cgi?id=1848938
Not reply yet but that's write annoying (or should I say blocking).
If you have any idea, please tell me or comment on the ticket.

I've finally managed to fix mailman3 package in Fedora again: https://bodhi.fedoraproject.org/updates/FEDORA-2020-9d4ef73a14

Next up: Postorius and HyperKitty

thanks @ngompa . last time I had time to work on this I made changes in this package for python-mailman-client and there were problems because python3-zope-i18nmessageid was not provided anymore by any of the dependencies. The fact that Fedora is a very fast moving target is really a pain honestly.

The problem I see with just bumping the version to an unreleased version is that we have no idea if this will work properly with Hyperkitty and Postorius. Versions need to be coordinated and that's a combination that's not supported upstream. Also this build is not tested on EL8, which is our (OSPO) target. I would have preferred backporting patches but that's way more work.

OK, we're lucky, the API version is still 3.1, so inter-communication between the components should work. Now we can see if the build of other components work too.

Login to comment on this ticket.