The bodhi puppet module needs to be ported to ansible. We should probably do it as-is, and then make the necessary changes for bodhi 2.0 after.
Here is what I have done by now. I tried to port bodhi from puppet to ansible in a way that would resemble current puppet classes. I added a new role named bodhi with few nested roles within it. So at the moment it looks like this:
roles/bodhi/
roles/bodhi/app/
roles/bodhi/masher/
roles/bodhi/masher/jobrunner/
roles/bodhi/masher/epelmasher/
With such a roles structure I tried to imitate the puppet's class inheritance. So for example if you would like to include role epelmasher in a playbook you would include the following roles:
roles/bodhi
roles/bodhi/masher
roles/bodhi/masher/epelmasher.
Basically you just include all roles on the way to epelmasher and you just need to know the path. So that is how things are at the moment.
The second alternative I had in mind was to have the following roles:
roles/bodhi/base
with a few extra variables in playbooks to control tasks. What do you think guys? Give me as much feedback as possible. To me it doesn't matter if we keep my current solution or if we use something else, just tell what you think, what would you prefer and what you want me to do.
So, a bit of background and my thoughts: :)
Currently we have 6 machines bodhi is setup on in puppet (Luke: please correct me if I am wrong with any of this)
app01, app02, app03, app04: These machines are the ones maintainers hit to manage updates, the public/anon web pages people can see updates on, provide karma, etc.
releng04: This is the machine that does 2 things: Fedora updates pushes (so it needs masher config) and periodic jobs (like checking for old buildroot overrides or telling maintainers that a update has spent enough time in testing, etc), so it needs jobrunner.
relepel01: EPEL updates, so this just needs epel masher config.
In moving to ansible, we want to drop the app servers and make 2 (at least) new machines... bodhi01 and bodhi02 that handle what the app servers did. The other two I guess can stay the same, or we could rename them 'fedoraupdates01' 'epelupdates01' or something.
To my mind it would be nice if we had just 3 roles... the app (goes on bodhi01/02), the masher (goes on the two releng machines, possibly just with variables to decide if it has epel or fedora mash configs, or possibly just installs them all to both), the jobrunner (goes on releng04). This allows us flexability if we want to move the jobrunner somewhere, and such.
Does that make sense or help? Luke: any other thoughts? Should we make any changes for bodhi 2.0 too? if so, what?
Replying to [comment:4 kevin]:
So, a bit of background and my thoughts: :) Currently we have 6 machines bodhi is setup on in puppet (Luke: please correct me if I am wrong with any of this) app01, app02, app03, app04: These machines are the ones maintainers hit to manage updates, the public/anon web pages people can see updates on, provide karma, etc. releng04: This is the machine that does 2 things: Fedora updates pushes (so it needs masher config) and periodic jobs (like checking for old buildroot overrides or telling maintainers that a update has spent enough time in testing, etc), so it needs jobrunner.
Yup, sounds right. Even though it does 2 things, it only inherits from the single bodhi::masher::jobrunner subclass.
relepel01: EPEL updates, so this just needs epel masher config. In moving to ansible, we want to drop the app servers and make 2 (at least) new machines... bodhi01 and bodhi02 that handle what the app servers did. The other two I guess can stay the same, or we could rename them 'fedoraupdates01' 'epelupdates01' or something. To my mind it would be nice if we had just 3 roles... the app (goes on bodhi01/02), the masher (goes on the two releng machines, possibly just with variables to decide if it has epel or fedora mash configs, or possibly just installs them all to both), the jobrunner (goes on releng04). This allows us flexability if we want to move the jobrunner somewhere, and such. Does that make sense or help? Luke: any other thoughts? Should we make any changes for bodhi 2.0 too? if so, what?
This makes sense to me. In the future, splitting the 'jobrunner' out into simple cron jobs is probably the way to go, so having the flexibility to run it elsewhere is fine. Right now it's simply a 1-line config file change to enable these jobs (even though we have seperate config templates with lots of duplicate options), so it could also potentially be narrowed down to just app & masher roles, having a 'jobrunner' boolean option to enable them.
I decided to change everything to just two roles and to control certain tasks in roles through variables. Now there are:
Extra boolean variables are jobrunner, epelmasher and bodhiapp. These are by default set to false. You can change their values, when you include a role in playbook. I included dependency in masher role which automatically includes base role.
Have a look and comment it.
attachment bodhi_ansible_port
Looks good, but I'm confused about the 'bodhiapp' boolean. It looks like all it does is apply the hotfix from ticket [ticket:3755], but that's really only needed for the mashers.
I just applied that bodhi hotfix upstream, so you can remove it from the playbook
OK I will do that. Is there anything else I can improve?
A few more things:
We could probibly drop all the old mash files from the old eol releases now i would think?
We need a playbooks/group/bodhi.yml playbook... this should create the instances as the first play, then configure them with the role(s) in the second. Look at any of the other existing group playbooks.
Along with that, we need inventory/group_vars/bodhi and host_vars/releng04, relepel01, bodhi01, bodhi02. I can get you information for ip address, etc on those when needed.
The instances also need nfs mounts. Look at tasks/koji/releng_config.yml for how to do that. Perhaps we should split that out to a nfs_client role since we use it there and here and in builders too.
Take a look at the existing manifests/nodes/releng04 and relepel01 in puppet, there's a number of misc other scripts we need to port over, but I guess those all shouldn't be in the bodhi role. Perhaps releng04/relepel01 should be new releng role and just include the bodhi masher stuff. Thats likely the way to go. So, perhaps we concentrate on getting the bodhi role done and bodhi01/02 and then look at releng04/relepel01 next after those are done.
OK, I will try to come up with something though I will probably need help. I will keep you informed.
Here is playbooks/groups/bodhi.yml. NFS mount is missing at the moment. I will do that tomorrow. Guys could you have a look and tell me which roles and tasks in plays are OK and which are not. And what do I add.
attachment bodhi.yml
Minor nits:
Hosts is seperated with : and not ,
Instead of seperate plays for each of the types, you can instead use 'when:' that just checks for either hostname or a variable and only runs that role when it's true. Then, all those plays can collapse into 2 (one to make the machines, the second to configure them, with conditional when's).
Here are few new things. Comments, suggestions? Variables' names in new nfs role?
attachment bodhi
attachment bodhi.2.yml
attachment nfs_role.yml
service: fedocal
should be service: bodhi ?
I think we need mod_wsgi on all of them... even the application ones.
For the nfs mount, we also need a weird route to the netapp... look in roles/koji_builder/tasks/main.yml starting at "name: route config for netapp network"
Can you do a diff against current ansible HEAD so we can see the full changes? I think we are getting close here. ;)
Sorry for fedocal, I didn't notice. Mod_wsgi will be added and nfs mount changed.
What is missing at the moment is host variables, domain names in inventory and perhaps some changes in dns repo. That's how I see it. But besides that basic bodhi porting is probably done.
After we are done with basic porting I suggest that we do releng04 and relepel01.
Had troubles with diff, so here is patch.
attachment 0001-Porting-bodhi-to-ansible.patch
I added that task from koji to nfs_client role. I also added two more tasks that seem relevant to that "route config" tasks. If they are not needed I will remove them.
Quite the patch now at this point. ;)
So, from a lookover it looks ok to me, but we may run into various little issues.
I'd like to commit this next week and then see if we can bring up a bodhi01.stg first with it. If we get that all going, then bring up a bodhi01/02 in production and switch bodhi web traffic from the app servers over to those. Once thats all stable, we can look at moving on to relepel01 and releng01. I think those will be a seperate playbook (or in with the existing releng one) and just include the needed bodhi roles.
Thanks a bunch for working on this. If you have time next week to work on it on irc we can push things forward and I can have you tweak any issues we run into...
Replying to [comment:20 kevin]:
Quite the patch now at this point. ;) So, from a lookover it looks ok to me, but we may run into various little issues. I'd like to commit this next week and then see if we can bring up a bodhi01.stg first with it. If we get that all going, then bring up a bodhi01/02 in production and switch bodhi web traffic from the app servers over to those. Once thats all stable, we can look at moving on to relepel01 and releng01. I think those will be a seperate playbook (or in with the existing releng one) and just include the needed bodhi roles.
Sounds good
Yeah, +1. Thanks for doing this janeznemanic, I look forward to testing it out next week.
Kevin, shall we try to apply things tomorrow? What about host variables? Have you taken care of that? How are we going to handle time zone difference? I'm in CET. I have time after 17.00 h UTC, is that fine with you?
Tomorrow might work, but I have a meeting at 18UTC... so we could work on it from 17-18 and see where we are at at the end of that?
I still need to allocate ips for the new bodhi01/02's... I can do that tomorrow morning. ;)
ok. I got sidetracked, but finally got to pushing this in. ;)
We now have a bodhi01.stg instance up and running.
Thanks a bunch for all the work on this.
Login to comment on this ticket.