#5466 Fedora RelEng Automation
Closed: Fixed 2 years ago Opened 3 years ago by maxamillion.

The Fedora Release Engineering Team would like to be able to start to automate various tasks using Ansible[0], some of these tasks are to be executed manually where as others are meant to be run automatically either via a fedmsg event[1][2] or via a cron job somewhere in the infrastructure.

I would like to propose integration into the rbac-playbook utility to accommodate the execution of RelEng Ansible playbooks by hand but would like to have automated tasks run completely hands off. I would also like to propose having a releng-specific ansible user that would have it's own set of ssh keys and would be managed by the Fedora Infrastructure Ansible playbooks that would allow for automatic execution of jobs that are automated based on schedule or fedmsg events.

[0] - https://pagure.io/releng-automation
[1] - https://fedoraproject.org/wiki/Changes/ReleaseEngineeringAutomationWorkflowEngine
[2] - https://github.com/maxamillion/loopabull


Couldn't we use the similar approach as we already use for running ansible playbook from fedmsg events?

It's basically a fedmsg consumer that sudos to run the ansible playbook.

advantages:

  • no need for a separate key/user
  • to update the consumer, just update the rpm

inconvenient:

  • the consumer cannot pass arguments to the playbook as this would change the command called and thus would not be allowed in the sudoers

Agreed during the meeting:

  • !RelEng Automation will be done using the fedmsg consumer in Fedora Infra and contribute code to add required functionality
  • !RelEng Automation ansible pagure repo will be brought in under the Fedora Infra ansible playbooks repository once this moves to pagure
    • Details need to be sorted on role paths and such

After further discussion, we're likely going to use Loopabull and Patrick is going to review and security audit.

There was a discussion a little while back on this topic between Patrick and I, the notes were captured in an etherpad here. I have copied the notes directly below for posterity.

RelEng Automation / Loopabull in Fedora Infra

Background:
    https://pagure.io/releng-automation


Agenda:
    - Where can/should Loopbull run in Fedora Infra?
    - How do we want to limit access to what all Loopabull can touch/control?
        - FAS Account specifically for Loopabull and playbooks run as that user via rbac-playbook?
        - Custom set of ssh keys with permissions to do releng group things?
        - Other?
    - How will/should we handle pulling the ansible playbooks from the releng-automation pagure repo?
        - Are there any security concerns around this?

Notes:
    - Sudo permissions on an ansbile playbook are hard because it's kind of ALL or nothing
    - We could use the command module to run 'sudo $some_command' as needed and only permit the automation account to run that set of commands
        - This would require documentation of which commands are allowed to be used in ansible automation tasks and a process by which to request new commands be added (via the infra team)
        - Filesystem permissions are hard for things like /mnt/koji because it's NFS
    - Signing requests from playbooks run by loopabull is also going to be a pain point
    - We could have a pagure repo (potentially called fedora-releng-utils) that would house a series of small utilities to perform specific tasks that would require priv escalation, we could then assign sudo permissions for these specific utils and automate them with ansible
        - Maybe https://pagure.io/flr ?
    - Loopabull would need to listen for pagure fedmsgs for releng-automation repo and git pull to locally update it's ansible playbooks/modules
    - Permissions on https://pagure.io/releng-automation need to not include @releng just in case someone gets added to the FAS group but isn't yet trusted with all things releng
        - Starting list:
            - dgilmore
            - mboddu
            - maxamillion
            - threebean
            - nirik
            - pbrobinson

Action Items:
    - Create small command line tools for specific tasks in https://pagure.io/flr
    - Fix permissions to releng-automation
    - Document flr and process to get new commands added to sudo list
        - Create an infra ticket with the what (which new script), the where (what machine does this need to run on), and the who (the automation user), and justification for needing this added
    - Request for a new VM to run loopabull from
        - loopabull01
    - Package new version of loopabull
    - Figure out signing requests?
        - In the magical future

So, we have the RFR in https://pagure.io/fedora-infrastructure/issue/5670

I don't think we need this also... closing out.

:rabbit2:

Metadata Update from @kevin:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

2 years ago

Login to comment on this ticket.

Metadata