#9152 staging noggin deployment planning
Opened 3 months ago by kevin. Modified 2 months ago

Greetings.

We are now ready to build up our staging env again and I figured I would file a ticket to coordinate noggin deployment along with all the other things we need for it.

Some questions:

  • Does noggin work/will it work in openshift? If so, I can do a openshift deployment first, if not, we can just do it in a vm.

  • unfortunately (or perhaps fortunately), we didn't save the old staging ipa server, so I did a new deployment from scratch in a vm. (ipa01.stg.iad2.fedoraproject.org).
    Does noggin need anything from the ipa server configuration wise? The playbook is currently failing on:
    ipa: ERROR: Host 'id.stg.fedoraproject.org' does not have corresponding DNS A/AAAA record, but it does... not sure whats going on there.

  • I'm assuming noggin needs ipa and ipsilon and a proxy, any other services?

Things we need to figure out:

  • Should we just start from 0 for now? (ie, have admins make accounts, etc) or do we want to try and migrate data from prod?

  • we need to figure out ssh access/replacement for fasClient

  • we need to figure out sudo access/replacement for pam_url

cc @abompard @pingou @puiterwijk


Metadata Update from @mohanboddu:
- Issue priority set to: Waiting on Assignee (was: Needs Review)
- Issue tagged with: groomed, medium-gain, medium-trouble

3 months ago

Does noggin work/will it work in openshift? If so, I can do a openshift deployment first, if not, we can just do it in a vm.

Yes, The two webapps will run in OpenShift. I have the yaml files I used for the CommuniShift and RH instance deployment so I can reuse them for staging. If you make me a role folder in Ansible I'll put them there. I'll also need a couple secrets obviously.

unfortunately (or perhaps fortunately), we didn't save the old staging ipa server, so I did a new deployment from scratch in a vm. (ipa01.stg.iad2.fedoraproject.org). Does noggin need anything from the ipa server configuration wise?

I'll will need to have the freeipa-fas plugin installed, but IPA should be installable without it, and we can add it later.

The playbook is currently failing on: ipa: ERROR: Host 'id.stg.fedoraproject.org' does not have corresponding DNS A/AAAA record, but it does... not sure whats going on there.

Hmm, not sure either.

I'm assuming noggin needs ipa and ipsilon and a proxy, any other services?

I'll need to connect to the RabbitMQ servers for Fedora Messaging, but I think that's all.

Should we just start from 0 for now? (ie, have admins make accounts, etc) or do we want to try and migrate data from prod?

We can try the migration script.

we need to figure out ssh access/replacement for fasClient
we need to figure out sudo access/replacement for pam_url

That should just be running ipa-client-install, I believe.

Metadata Update from @abompard:
- Issue untagged with: groomed, medium-gain, medium-trouble
- Issue priority set to: Needs Review (was: Waiting on Assignee)

3 months ago

Metadata Update from @abompard:
- Issue tagged with: groomed, medium-gain, medium-trouble

3 months ago

Metadata Update from @smooge:
- Issue priority set to: Waiting on Assignee (was: Needs Review)

3 months ago

Note that I also have the Noggin stack packaged as RPMs and it can be deployed to a container or a VM with it. This also includes freeipa-fas plugin. Without the plugin installed and FreeIPA reconfigured with it, Noggin will break pretty badly.

Oh, and here's the COPR where I've built all this: https://copr.fedorainfracloud.org/coprs/ngompa/fedora-aaa/

I'm basically waiting on @abompard's approval before upstreaming these into Fedora itself.

ok, we now have a staging openshift cluster up and running. It doesn't have any web or remote access yet however (that needs some firewall rules setup for staging proxies).

That said, I think we can start working on deploying noggin there anytime.

I assume ipsilon needs some config adjustment to talk only to IPA and not fas? We should do that too and deploy it in openshift.

Also, does IPA need any changes?

@abompard since we have no auth currently or web interface, how about I just put your ssh key for root on os-control01.stg.iad2.fedoraproject.org ? You can login there as root and ssh to os-master01.stg.iad2.fedoraproject.org (or any of the cluster) to debug things. Is that acceptable? Or do you just need to deploy via playbook and don't need any more access? Anything else you need? Once we have noggin up and ipsilon, we can look at sorting out our ssh / local admin accounts plans.

Yeah I'll also need access to the FreeIPA server to deploy the extension (or rather: be allowed to run the playbook that will do it, probably the same as the freeipa one)

Then I can start on ipsilon & noggin. I have the openshift yaml files for noggin but I haven't written playbooks, I can start with that once I have access. I haven't used root on openshift yet, I hope I won't break things...

Status update:

  • Noggin and FASJSON are deployed in staging openshift, but I can't check that they actually work because there's currently no way to get to the web UIs (if I understand correctly the proxies aren't set up)

  • Ipsilon is not deployed, and I'll work on that today. It needs to be configured to pull information from IPA/LDAP and not FAS, so there are configuration changes to do to the current playbook/role. I'd welcome help from someone who know Ipsilon well to figure out those changes and how to deploy them, because we can't use the nice --ipa installation switch with containers

Ipsilon does have a plugin to get info from LDAP, but I don't think we can use it as-is, because the FAS info plugin seems to do more stuff (be careful @nphilipp or @ryanlerch , the infofas plugin that is actually deployed is a modified version that lives in ansible/roles/ipsilon/files/infofas.py) especially around the AWS roles.

I'm not sure how much of a IPA client setup Ipsilon will need to switch to IPA-based authentication and information, but we can't run ipa-client-install in a container, so we'll have to figure something out. For FASJSON I'm using a lightweight system that gets the IPA CA cert and a service keytab, but I'm not sure it'll be sufficient for Ipsilon. Maybe the IPA folks ( @cheimes ?) can shed a light on that as they've probably had container-based IPA clients in the past already.

Finally, the IPA server in staging seems to be up and running fine with the FAS plugin installed. There are almost no users at the moment, so we can start testing the import script, but it's there and machines where fasClient used to run can be enrolled to enable SSH access.

That's all for me today! I'll be off most of next week but @nphilipp and @ryanlerch are around (hop, right under the bus, you're welcome).

Ipsilon is not deployed, and I'll work on that today. It needs to be configured to pull information from IPA/LDAP and not FAS, so there are configuration changes to do to the current playbook/role. I'd welcome help from someone who know Ipsilon well to figure out those changes and how to deploy them, because we can't use the nice --ipa installation switch with containers

I believe @hellcp has a working Ipsilon configuration with FreeIPA that he could share to template for the container deployment.

That'd be very nice, thanks.

@puiterwijk is more qualified to talk about IPA integration for Ipsilon. As far as I remember Ipsilon uses SSSD, mod_auth_gssapi, and mod_lookup_identity. The lookup identity module depends on SSSD's info pipe and D-Bus. At a minimum you have to configure authselect as authselect select sssd, configure SSSD, enable info-pipe in SSSD config, and have a minimal IPA installation with default.conf, ipa.crt, /etc/krb5.keytab, and a keytab for HTTPd.

We met today to move this deployment forward.

I have created a db-fas01.stg.iad2.fedoraproject.org that has the prod fas db loaded in it.

You should now be able to deploy a fas in staging openshift and migrate from it to noggin.

Should we keep this open? Or close until we see more to do? The sssd part of things still needs to be sorted tho.

Thanks Kevin!

I would like to keep this open for now in case the team run into any issues deploying so we have a reference of what was asked.

Happy to open a separate issue for sssd if that would help too.

Login to comment on this ticket.

Metadata