#420 Create a basic Vagrant development environment.
Opened 5 years ago by bowlofeggs. Modified 4 months ago
bowlofeggs/koji vagrant  into  master

file modified
+4
@@ -8,3 +8,7 @@ 

  .idea

  .DS_STORE

  .vagrant

+ noarch

+ koji-1.*

+ koji-*.tar.bz2

+ Vagrantfile

file added
+70
@@ -0,0 +1,70 @@ 

+ # -*- mode: ruby -*-

+ # vi: set ft=ruby :

+ 

+ # On your host:

+ # git clone https://pagure.io/koji.git

+ # cd koji

+ # cp Vagrantfile.example Vagrantfile

+ # vagrant up

+ # vagrant ssh

+ 

+ VAGRANTFILE_API_VERSION = "2"

+ 

+ Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|

+  config.vm.box_url = "https://download.fedoraproject.org/pub/fedora/linux/releases/25/CloudImages/x86_64/images/Fedora-Cloud-Base-Vagrant-25-1.3.x86_64.vagrant-libvirt.box"

I think we should use CentOS 7 (https://app.vagrantup.com/centos) or 8 (https://bugs.centos.org/view.php?id=16455) so it does not go out of date.

+  config.vm.box = "f25-cloud-libvirt"

+ 

+  # Forward traffic on the host to the development server on the guest.

+  config.vm.network "forwarded_port", guest: 80, host: 8080

+ 

+  # This is an optional plugin that, if installed, updates the host's /etc/hosts

+  # file with the hostname of the guest VM. In Fedora it is packaged as

+  # ``vagrant-hostmanager``

+  if Vagrant.has_plugin?("vagrant-hostmanager")

+      config.hostmanager.enabled = true

+      config.hostmanager.manage_host = true

+  end

+ 

+  # Vagrant can share the source directory using rsync, NFS, or SSHFS (with the vagrant-sshfs

+  # plugin). Consult the Vagrant documentation if you do not want to use SSHFS.

+  config.vm.synced_folder ".", "/vagrant", disabled: true

+  config.vm.synced_folder ".", "/home/vagrant/koji", type: "sshfs", sshfs_opts_append: "-o nonempty"

+ 

+  # To cache update packages (which is helpful if frequently doing `vagrant destroy && vagrant up`)

+  # you can create a local directory and share it to the guest's DNF cache. Uncomment the lines below

+  # to create and use a dnf cache directory

+  #

+  # Dir.mkdir('.dnf-cache') unless File.exists?('.dnf-cache')

+  # config.vm.synced_folder ".dnf-cache", "/var/cache/dnf", type: "sshfs", sshfs_opts_append: "-o nonempty"

+ 

+  # Comment this line if you would like to disable the automatic update during provisioning

+  config.vm.provision "shell", inline: "sudo dnf upgrade -y"

+ 

+  # bootstrap and run with ansible

+  config.vm.provision "shell", inline: "sudo dnf -y install python2-dnf libselinux-python"

+  config.vm.provision "ansible" do |ansible|

+      ansible.playbook = "devel/ansible/playbook.yml"

+  end

+ 

+ 

+  # Create the koji dev box

+  config.vm.define "koji" do |koji|

+     koji.vm.host_name = "koji-dev.example.com"

+ 

+     koji.vm.provider :libvirt do |domain|

+         # Season to taste

+         domain.cpus = 4

+         domain.graphics_type = "spice"

+         domain.memory = 1024

+         domain.video_type = "qxl"

+ 

+         # Uncomment the following line if you would like to enable libvirt's unsafe cache

+         # mode. It is called unsafe for a reason, as it causes the virtual host to ignore all

+         # fsync() calls from the guest. Only do this if you are comfortable with the possibility of

+         # your development guest becoming corrupted (in which case you should only need to do a

+         # vagrant destroy and vagrant up to get a new one).

+         #

+         # domain.volume_cache = "unsafe"

+     end

+  end

+ end

@@ -0,0 +1,1 @@ 
alcmb commented 5 years ago

Looks like a temporary file that can be safely removed from the pull request.

+ koji

@@ -0,0 +1,9 @@ 

+ ---

+ - hosts: all

+   become: true

+   become_method: sudo

+   vars:

+   roles:

+     - core

+     - db

+     - dev

@@ -0,0 +1,13 @@ 

+ ---

+ - name: Install basic packages

+   dnf:

+       name: "{{ item }}"

+       state: present

+   with_items:

+       - bash-completion

+       - dstat

+       - fedora-easy-karma

+       - htop

+       - screen

+       - tmux

+       - tree

@@ -0,0 +1,39 @@ 

+ ---

+ - name: Install database packages

+   dnf:

+       name: "{{ item }}"

+       state: present

+   with_items:

+       - libsemanage-python

+       - postgresql-server

+ 

+ - name: Initialize PostgreSQL

+   command: postgresql-setup initdb

+   args:

+       creates: /var/lib/pgsql/data/pg_hba.conf

+ 

+ - replace:

+     dest: /var/lib/pgsql/data/pg_hba.conf

+     regexp: "host    all             all             127.0.0.1/32            ident"

+     replace: "host    all             all             127.0.0.1/32            trust"

+ 

+ - replace:

+     dest: /var/lib/pgsql/data/pg_hba.conf

+     regexp: "host    all             all             ::1/128                 ident"

+     replace: "host    all             all             ::1/128                 trust"

+ 

+ - service:

+     name: postgresql

+     state: started

+     enabled: yes

+ 

+ - name: Create a database for Koji

+   shell: runuser -l postgres -c 'createdb koji' && touch /home/vagrant/.koji-db-created

+   args:

+       creates: /home/vagrant/.koji-db-created

Here's how I did this in my playbooks:

- postgresql_user:
    name: koji
    password: test
  become_user: postgres

- postgresql_db:
    name: koji
    owner: koji
  become_user: postgres

+ 

+ - name: Allow httpd to connect to postgres

+   seboolean:

+       name: httpd_can_network_connect_db

+       state: yes

+       persistent: yes

@@ -0,0 +1,20 @@ 

+ # .bashrc

+ 

+ # Source global definitions

+ if [ -f /etc/bashrc ]; then

+         . /etc/bashrc

+ fi

+ 

+ # Uncomment the following line if you don't like systemctl's auto-paging feature:

+ # export SYSTEMD_PAGER=

+ 

+ shopt -s expand_aliases

+ 

+ function ktest {

+     find /home/vagrant/koji -name "*.pyc" -delete;

+     pushd /home/vagrant/koji && make test; popd

+ }

+ 

+ export PYTHONWARNINGS="once"

+ 

+ cd koji

@@ -0,0 +1,3 @@ 

+ set expandtab

+ set tabstop=4

+ set shiftwidth=4

@@ -0,0 +1,7 @@ 

+ 

+ Welcome to the Koji development environment! Here are some helpful commands:

+ 

+ ktest:      Run Koji's test suite.

+ 

+ Happy hacking!

+ 

@@ -0,0 +1,75 @@ 

+ HOME                    = .

+ RANDFILE                = .rand

+ 

+ [ca]

+ default_ca              = ca_default

+ 

+ [ca_default]

+ dir                     = .

+ certs                   = $dir/certs

+ crl_dir                 = $dir/crl

+ database                = $dir/index.txt

+ new_certs_dir           = $dir/newcerts

+ certificate             = $dir/%s_ca_cert.pem

+ private_key             = $dir/private/%s_ca_key.pem

+ serial                  = $dir/serial

+ crl                     = $dir/crl.pem

+ x509_extensions         = usr_cert

+ name_opt                = ca_default

+ cert_opt                = ca_default

+ default_days            = 3650

+ default_crl_days        = 30

+ default_md              = sha256

+ preserve                = no

+ policy                  = policy_match

+ 

+ [policy_match]

+ countryName             = match

+ stateOrProvinceName     = match

+ organizationName        = match

+ organizationalUnitName  = optional

+ commonName              = supplied

+ emailAddress            = optional

+ 

+ [req]

+ default_bits            = 1024

+ default_keyfile         = privkey.pem

+ default_md              = sha256

+ distinguished_name      = req_distinguished_name

+ attributes              = req_attributes

+ x509_extensions         = v3_ca # The extentions to add to the self signed cert

+ string_mask             = MASK:0x2002

+ 

+ [req_distinguished_name]

+ countryName                     = Country Name (2 letter code)

+ countryName_default             = AT

+ countryName_min                 = 2

+ countryName_max                 = 2

+ stateOrProvinceName             = State or Province Name (full name)

+ stateOrProvinceName_default     = Vienna

+ localityName                    = Locality Name (eg, city)

+ localityName_default            = Vienna

+ 0.organizationName              = Organization Name (eg, company)

+ 0.organizationName_default      = My company

+ organizationalUnitName          = Organizational Unit Name (eg, section)

+ commonName                      = Common Name (eg, your name or your server\'s hostname)

+ commonName_max                  = 64

+ emailAddress                    = Email Address

+ emailAddress_max                = 64

+ 

+ [req_attributes]

+ challengePassword               = A challenge password

+ challengePassword_min           = 4

+ challengePassword_max           = 20

+ unstructuredName                = An optional company name

+ 

+ [usr_cert]

+ basicConstraints                = CA:FALSE

+ nsComment                       = "OpenSSL Generated Certificate"

+ subjectKeyIdentifier            = hash

+ authorityKeyIdentifier          = keyid,issuer:always

+ 

+ [v3_ca]

+ subjectKeyIdentifier            = hash

+ authorityKeyIdentifier          = keyid:always,issuer:always

+ basicConstraints                = CA:true

@@ -0,0 +1,101 @@ 

+ ---

+ - name: Install dev packages

+   dnf:

+       name: "{{ item }}"

+       state: present

+   with_items:

+       - git

+       - python

+       - python-devel

+       - python-ipdb

+       - python-psycopg2

+       - python-rpdb

+       - python2-coverage

+       - python2-sphinx

+       - rpm-build

+       - vim-enhanced

+ 

+ - name: Build the RPMs

+   shell: make test-rpm

+   args:

+     chdir: /home/vagrant/koji

+     creates: /home/vagrant/koji/noarch

+ 

+ - name: Install dev packages

+   shell: dnf install -y /home/vagrant/koji/noarch/*.rpm

+ 

+ - name: Create /etc/pki/koji

+   file:

+       path: "/etc/pki/koji/{{ item }}"

+       owner: root

+       group: root

+       mode: 0755

+       state: directory

+   with_items:

+       - ""

+       - certs

+       - private

+       - confs

+ 

+ - name: Install ssl.cnf

+   copy:

+       src: ssl.cnf

+       dest: /etc/pki/koji/ssl.cnf

+       mode: 0644

+       owner: root

+       group: root

+ 

+ - name: Create the CA private key

We can replace these openssl commands (and ssl.cnf) with koji-ssl-admin

+   shell: openssl genrsa -out /etc/pki/koji/private/koji_ca_cert.key 2048

+ 

+ - name: Create the CA

+   shell: openssl req -config /etc/pki/koji/ssl.cnf -new -x509 -subj "/C=US/ST=Oregon/L=Portland/O=IT/CN=koji.example.com" -days 3650 -key /etc/pki/koji/private/koji_ca_cert.key -out /etc/pki/koji/koji_ca_cert.crt -extensions v3_ca

+ 

+ - name: import the koji db schema

+   shell: cat /usr/share/doc/koji/docs/schema.sql | runuser -l postgres -c 'psql koji' && touch /home/vagrant/.db-imported

+   args:

+       creates: /home/vagrant/.db-imported

+ 

+ - name: Configure koji to use our local postgres db

+   replace:

+     dest: /etc/koji-hub/hub.conf

+     regexp: "^DBUser = koji$"

+     replace: "DBUser = postgres"

+ 

+ - replace:

+     dest: /etc/koji-hub/hub.conf

+     regexp: "^#DBHost = .*$"

+     replace: "DBHost = 127.0.0.1"

+ 

+ - replace:

+     dest: /etc/koji-hub/hub.conf

+     regexp: "^#DBPass = .*$"

+     replace: "DBPass = doesntmatter"

+ 

+ - name: Install the .bashrc

+   copy:

+       src: .bashrc

+       dest: /home/vagrant/.bashrc

+       mode: 0644

+       owner: vagrant

+       group: vagrant

+ 

+ - name: Install the .vimrc

+   copy:

+       src: .vimrc

+       dest: /home/vagrant/.vimrc

+       mode: 0644

+       owner: vagrant

+       group: vagrant

+ 

+ - name: Install the motd

+   copy:

+       src: motd

+       dest: /etc/motd

+       mode: 0644

+ 

+ - name: Start and enable the httpd service

+   service:

+       name: httpd

+       state: started

+       enabled: yes

This commit introduces a simple Vagrant environment to aid in
development of Koji. It is a simple scaffold at this point - it
creates an empty database, builds RPMs, installs them, creates a
CA, and starts httpd.

Signed-off-by: Randy Barlow randy@electronsweatshop.com

I'm having trouble getting this to work with the instructions. Maybe we can expand the lines int he example vagrantfile into their own little doc. It would be nice to let folks know:

  • what tools/etc are needed (eg. vagrant, vagrant-libvirt, vagrant-sshfs)
  • link to a vagrant howto guide

There is also some ongoing work on https://github.com/release-engineering/koji-dojo/tree/vagrant branch. Maybe it would make sense to consolidate these together (not checked yet).

Looks like Koji-dojo repo should be a better location for submitting this kind
of work. That repo already contains an assortment of ways to install Koji.

Looks like a temporary file that can be safely removed from the pull request.

https://github.com/release-engineering/koji-dojo/tree/vagrant somewhat intersects with the current pull request, but they don't have too much in common. The former is a vagrant wrapper for koji-dojo, and PR420 is a plain vagrant-based installation. So it's rather an alternative than a duplicate.

For the Ansible part, there are also koji-* roles existing in Ansible galaxy: https://galaxy.ansible.com/list#/roles?page=1&page_size=10&autocomplete=koji
I didn't look at them closely, but those could probably be used instead of the devel/ansible/roles/* introduced in this PR.

Here's how I did this in my playbooks:

- postgresql_user:
    name: koji
    password: test
  become_user: postgres

- postgresql_db:
    name: koji
    owner: koji
  become_user: postgres

I think we should use CentOS 7 (https://app.vagrantup.com/centos) or 8 (https://bugs.centos.org/view.php?id=16455) so it does not go out of date.

We can replace these openssl commands (and ssl.cnf) with koji-ssl-admin

Incidentally I'm right now working on this due to my personal needs. Ill share what I have in a few days.