#87 create roles for beaker server and lab controller in fedora infra ansible
Closed: Fixed None Opened 9 years ago by tflink.

beaker-dev is working now but it's largely deployed and maintained by hand. One of the next steps is to deploy and maintain the server and lab controller with fedora infra's ansible repo.

Create roles for beaker server and lab controller.


I have created a patch for this. Please have a look.
{F20848}

Apologies for the delay on getting back to you on this - kept getting sidetracked.

Comments on the patch:
does SELinux really need to be turned off on the beaker server?
this doesn't handle the public/private hostname differences that we have (beaker01.qa.fedoraproject.org internally, beaker.fedoraproject.org externally for production)
assuming that we're going with the local-to-beaker-server mariadb instance, that role has already been written in the infra ansible repo
what do we want to do with / on the beaker server? I have a static HTML landing page on the current beaker-dev setup but that doesn't appear to be handled here
all http traffic will go through a proxy that handles the https encrytpion/certs so all traffic coming directly out of the beaker server should be http
I'd prefer to see role names of beaker/server and beaker/labcontroller instead of keeping them flat - it keeps from adding more dirs to the roles/ dir than we have to

Other than that stuff, I don't see any problems with it. I'm sure we'll run into problems when actually deploying the thing (I've yet to see a new system deploy 100% without changes in ansible) but we'll cross that bridge when we get there.

The patch is also missing some infra-specific bits for http proxying, backups, ACLs etc. but I can take care of that part when applying it. Just to make sure I'm not forgetting anything, does anything outside of the local networks need to access the beaker server by something other than HTTPS?

Comments on the patch:
* does SELinux really need to be turned off on the beaker server?

Yes, its required on the beaker server

  • this doesn't handle the public/private hostname differences that we have (beaker01.qa.fedoraproject.org internally, beaker.fedoraproject.org externally for production)

I donnot know the differences about those two servers. In ansible/inventory file, there is a group called beaker so I guess we could add a new group called beaker-pub.
Then I assume there would be two playbooks in the playbooks/groups, one is for beaker internally and one is for beaker publicly.

  • assuming that we're going with the local-to-beaker-server mariadb instance, that role has already been written in the infra ansible repo

if this is the way we are going, I could add those two tasks in the beaker server role.

  • name: create the beaker database
    mysql_db: name=beaker state=present

  • name: create beaker user
    mysql_user: name={{beaker_server_admin_user}} password={{beaker_server_admin_pass}} priv=beaker.*:ALL,GRANT state=present

  • what do we want to do with / on the beaker server? I have a static HTML landing page on the current beaker-dev setup but that doesn't appear to be handled here

Sorry, I am not sure what you mean but beaker server role should only handles the things in https://beaker-project.org/docs/admin-guide/installation.html. Any custom work could
be done in the playbook where using this role.

  • all http traffic will go through a proxy that handles the https encrytpion/certs so all traffic coming directly out of the beaker server should be http

I guess this is another custom change for beaker server.

  • I'd prefer to see role names of beaker/server and beaker/labcontroller instead of keeping them flat - it keeps from adding more dirs to the roles/ dir than we have to

yes, that can be changed.

Other than that stuff, I don't see any problems with it. I'm sure we'll run into problems when actually deploying the thing (I've yet to see a new system deploy 100% without changes in ansible) but we'll cross that bridge when we get there.

The patch is also missing some infra-specific bits for http proxying, backups, ACLs etc. but I can take care of that part when applying it. Just to make sure I'm not forgetting anything, does anything outside of the local networks need to access the beaker server by something other than HTTPS?

! In #435#5722, @mjia wrote:
Comments on the patch:
* does SELinux really need to be turned off on the beaker server?

Yes, its required on the beaker server

Oof. I suspect there's a good reason for this but I dislike setting that precedent.

  • this doesn't handle the public/private hostname differences that we have (beaker01.qa.fedoraproject.org internally, beaker.fedoraproject.org externally for production)

I donnot know the differences about those two servers. In ansible/inventory file, there is a group called beaker so I guess we could add a new group called beaker-pub.
Then I assume there would be two playbooks in the playbooks/groups, one is for beaker internally and one is for beaker publicly.

There is only one server but it has 2 different hostnames depending on where you're trying to reach it from. From inside the fedora infra - the production host would be beaker01.qa.fedoraproject.org. From outside the fedora infra, that same host would be reached as beaker.fedoraproject.org.

However, it looks like that is already taken care of (tg.url_domain and tg.lab_domain). For some reason, I was remembering a different option name - my bad.

  • assuming that we're going with the local-to-beaker-server mariadb instance, that role has already been written in the infra ansible repo

if this is the way we are going, I could add those two tasks in the beaker server role.

  • name: create the beaker database
    mysql_db: name=beaker state=present

  • name: create beaker user
    mysql_user: name={{beaker_server_admin_user}} password={{beaker_server_admin_pass}} priv=beaker.*:ALL,GRANT state=present

Sounds like a plan to me.

  • what do we want to do with / on the beaker server? I have a static HTML landing page on the current beaker-dev setup but that doesn't appear to be handled here

Sorry, I am not sure what you mean but beaker server role should only handles the things in https://beaker-project.org/docs/admin-guide/installation.html. Any custom work could
be done in the playbook where using this role.

OK, I thought you were going to be involved in the setup of the whole system - hence the question. I can figure something out later, though - not that big of a deal and won't take long.

  • all http traffic will go through a proxy that handles the https encrytpion/certs so all traffic coming directly out of the beaker server should be http

I guess this is another custom change for beaker server.

Custom? All it means is not redirecting any traffic to https. I think that's a minor change in the apache config, no?

! In #435#5723, @tflink wrote:

! In #435#5722, @mjia wrote:
Comments on the patch:
* does SELinux really need to be turned off on the beaker server?

Yes, its required on the beaker server

Oof. I suspect there's a good reason for this but I dislike setting that precedent.

We have an old open RFE to write a policy module for Beaker: https://bugzilla.redhat.com/show_bug.cgi?id=1074384

Writing to /var/www/beaker is denied from httpd_t and for beaker-provision, writing to /var/lib/tftpboot is denied even from unconfined_service_t for some reason. So we have always just required it to be disabled until we get around to writing a proper policy.

Chris Long recently started working on a Beaker SELinux policy for our internal deployment so we may have some good news on that front soon, I will check with him how far he got.

! In #435#5721, @tflink wrote:
* what do we want to do with / on the beaker server? I have a static HTML landing page on the current beaker-dev setup but that doesn't appear to be handled here

Actually I think we should move Beaker to / and drop the /bkr prefix. There is really no good reason for the prefix, assuming that Beaker is deployed on a dedicated hostname (which you probably always want to do anyway). I just never got around to changing the defaults.

! In #435#5723, @tflink wrote:
Custom? All it means is not redirecting any traffic to https. I think that's a minor change in the apache config, no?

Yes we can just drop the HTTPS redirection stuff in the Apache config. The reverse proxy will do the redirect-to-HTTPS dance for us, for clients coming in from outside the test network, right?

! In #435#5721, @tflink wrote:
Just to make sure I'm not forgetting anything, does anything outside of the local networks need to access the beaker server by something other than HTTPS?

Nope, clients using the web UI or the CLI only need HTTPS to the Beaker server.

! In #435#5725, @dcallagh wrote:

! In #435#5721, @tflink wrote:
* what do we want to do with / on the beaker server? I have a static HTML landing page on the current beaker-dev setup but that doesn't appear to be handled here

Actually I think we should move Beaker to / and drop the /bkr prefix. There is really no good reason for the prefix, assuming that Beaker is deployed on a dedicated hostname (which you probably always want to do anyway). I just never got around to changing the defaults.

Sounds like a plan to me - less work to create a static landing page :)

! In #435#5726, @dcallagh wrote:

! In #435#5723, @tflink wrote:
Custom? All it means is not redirecting any traffic to https. I think that's a minor change in the apache config, no?

Yes we can just drop the HTTPS redirection stuff in the Apache config. The reverse proxy will do the redirect-to-HTTPS dance for us, for clients coming in from outside the test network, right?

Yeah, the proxy hosts handle the redirect to https

I'm still futzing around trying to get a beaker virthost and VM clients created in ansible so that's one less thing to worry about in this patch. At some point, FAS authentication will need to be addressed but that can wait for another time.

Yeah sorting out the FAS/OpenID auth is fairly high up on our list, so that people can start logging in and submitting jobs.

Matt recently also wrote up some playbooks for setting up Beaker test VMs in a dedicated libvirt network, it's intended for dev environments but maybe it will help with what you're working on? https://git.beaker-project.org/cgit/beaker-in-a-box/tree/

that helps a lot, actually. I was trying to do something similar and kept having problems with variable access and/or evaulation order. matt's doing something slightly different than what I was doing and it works :)

Upload a new patch to address the above comments.

{F21324}

Matt in /etc/beaker/server.cfg you have server.webpath = "/" but I think it will need to be uncommented.

Also I think we can just drop the beaker-transfer stuff, so that it's just left disabled. If at some point in future we need to turn it on the playbook can be updated.

! In #435#5791, @dcallagh wrote:
Matt in /etc/beaker/server.cfg you have server.webpath = "/" but I think it will need to be uncommented.

I have this line commented out in my dev working environment and it looks all good. Also I have a look on the beaker-devel and it does not have this line. So I am wondering beaker is using "/" for the server webpath by default.

Also I think we can just drop the beaker-transfer stuff, so that it's just left disabled. If at some point in future we need to turn it on the playbook can be updated.

This playbook is checking whether the beaker-transfer is configured on the server. If you donnot have it on, basically it will do nothing. If you want to turn it on the existing servers, you just need to change the correspond settings in the server.cfg and rerun the playbook. So I think it would be good to leave it there.

I didn't think that the lab controller hub and the beaker server could share the same URL - or am I misunderstanding the lab controller config (HUB_URL in labcontroller.conf.j2)?

Other than that question, it looks pretty good to me overall. Some bits still need some minor tweaking (email setup, password placement, various minutia) but I can do that when I apply the patch or after you apply it.

! In #435#5796, @tflink wrote:
I didn't think that the lab controller hub and the beaker server could share the same URL - or am I misunderstanding the lab controller config (HUB_URL in labcontroller.conf.j2)?

Other than that question, it looks pretty good to me overall. Some bits still need some minor tweaking (email setup, password placement, various minutia) but I can do that when I apply the patch or after you apply it.

The HUB_URL in the lab controller configuration file is the URL of your Beaker server without the trailing slash.

! In #435#5796, @tflink wrote:
I didn't think that the lab controller hub and the beaker server could share the same URL - or am I misunderstanding the lab controller config (HUB_URL in labcontroller.conf.j2)?

Other than that question, it looks pretty good to me overall. Some bits still need some minor tweaking (email setup, password placement, various minutia) but I can do that when I apply the patch or after you apply it.

Tim, thanks for looking it. Please go ahead to apply it since I donnot have the push permission to the fedora infras ansible repo.

Massaged patch applied to infra's ansible repo and relevant usernames/passwords generated in secure files.

https://infrastructure.fedoraproject.org/cgit/ansible.git/commit/?id=1b784e777817d164dee425e5b2f1c17cc16a8634

Since this task was all about the ansible changes, I'm closing it and will continue to update T475 as it progresses

Metadata Update from @tflink:
- Issue tagged with: infrastructure

6 years ago

Login to comment on this ticket.

Metadata