This is a request for resources for new Factory 2 service: On Demand Compose Service (ODCS).
Focus doc for ODCS for more background information: https://fedoraproject.org/wiki/Infrastructure/Factory2/Focus/ODCS Sources: https://pagure.io/odcs
Requested 4 VMs: stg and prod, a frontend and a backend for each. Spec: - 2 vCPUs - 4GB ram for stg/prod - 20 GB HDD
Related setup requests: * We'll need /mnt/koji read-only access. * We need a gluster/nfs share between the frontend and backend nodes in both stg and prod.
@ralph is the infra sponsor.
See #6167 for the associated security audit.
Like in #6183 (which was for freshmaker), here can I get a :+1: from another sysadmin to start provisioning these VMS even though the security audit of the app code isn't complete? Of course, we'll be around to fix any issues found before we go live. Getting a head start on deployment configuration work would be helpful at this point.
Metadata Update from @pingou: - Issue tagged with: security
Metadata Update from @pingou: - Issue tagged with: request-for-resources
As in https://pagure.io/fedora-infrastructure/issue/6183 :thumbsup:
OK - I created stg nodes and I claimed IPs for the prod node, although they're commented out in the ansible inventory so they won't be created.
FYI - a missing piece here is that we need a nfs share served from the backend to the frontend (in staging and eventually prod).
@ralph You could look at the docker-registry.yml or ask.yml playbooks as to how we usually set up gluster shared storage.
OK - first attempt at gluster shared storage to be found here.
After a few tweaks, it works. /srv/odcs is the currently mounted shared location.
/srv/odcs
@qwan, @cqi, @jkaluza - can you guys move forwards with setting up roles for the app?
@ralph: We will need database and keytab to contact Koji before deploying. I will also try writing the ODCS SOP today.
The openidc staging client and scopes have been added here: https://id.stg.fedoraproject.org/openidc/.well-known/openid-configuration
The staging database is created, with corresponding secrets in the secrets repo.
After talking in IRC, we see now that we don't actually need a keytab to talk to koji. All requests can be done anonymously.
$ curl https://odcs.stg.fedoraproject.org/odcs/1/composes/ { "items": [], "meta": { "first": "http://odcs.stg.fedoraproject.org/odcs/1/composes/?per_page=10&page=1", "last": "http://odcs.stg.fedoraproject.org/odcs/1/composes/?per_page=10&page=0", "next": null, "page": 1, "pages": 0, "per_page": 10, "prev": null, "total": 0 } }
OK - I missed a crucial piece here.
The odcs-backend01.stg and odcs-backend01.phx2 VMs need read-only access to /mnt/koji.
odcs-backend01.stg
odcs-backend01.phx2
/mnt/koji
The odcs-backend01.stg (stg) VM is expecting to find the content in /mnt/koji/vol/prod/, but it can be mounted somewhere else like in /mnt/fedora_koji_prod just like on koji01.stg.
/mnt/koji/vol/prod/
/mnt/fedora_koji_prod
koji01.stg
[koji01 ~][STG]# ls -alh /mnt/koji/vol/prod lrwxrwxrwx. 1 root root 26 Jun 16 00:05 /mnt/koji/vol/prod -> /mnt/fedora_koji_prod/koji
I guess this needs a ticket to change the network ACLs to allow the read-only mount from these hosts?
For the record, @puiterwijk has asked us to fill out the details here a bit more formally... so I'm going to use his template from infra-docs#64
Phase I
Phase II
Phase III
Phase IV
the fedora_koji netapp volume is exported ro to 10.5.127.0/24... ie, the entire storage network.
Just adding a eth1 (installing with a two_nics option and a eth1 ip set) should allow you to mount it.
Metadata Update from @puiterwijk: - Issue untagged with: security
Mounting /mnt/koji is now set in staging. As the other systems, it has /mnt/koji which is the staging koji (/mnt/fedora_koji/koji), and it has /mnt/koji/vol/prod, which is a symlink to the prod NFS mount on /mnt/fedora_koji_prod.
One headsup: all of these NFS mounts (including /mnt/koji in staging) are readonly. For production, an NFS IP will need to be assigned before the role can be enabled.
@ralph: I will also need rights to do "sudo journalctl -u odcs-backend.service" on the backend nodes, would it be possible to set it up?
@jkaluza, yes. It should be set now in ansible.
OK - ODCS should be up here: https://odcs.fedoraproject.org/api/1/composes/
What else do we need before closing this out?
Anything else?
For nagios checks, added:
I think we're good to close this now?
Yep. :clap:
Metadata Update from @kevin: - Issue close_status updated to: Fixed - Issue status updated to: Closed (was: Open)
Login to comment on this ticket.