PR#89: Add virtualenv wrapper tutorial to readme - kiskadee

kiskadee

#89 Add virtualenv wrapper tutorial to readme

Merged 7 years ago by davidcarlos. Opened 7 years ago by lucasandrade.

kiskadeemes/kiskadee 70-virtualenv-wrapper into master

Merge with master

Lucas Andrade • 7 years ago

bb5f10c

Added virtualenv tutorial to README

Lucas Andrade • 7 years ago

4e02a7a

AUTHORS

file modified

		`@@ -1,2 +1,7 @@`
		`+ Adailson Santos <adailson2@gmail.com>`
		`Athos Ribeiro <athoscr@fedoraproject.org>`
		`+ Daniel Moura <danmoura17@gmail.com>`
		`David Carlos <ddavidcarlos1392@gmail.com>`
		`+ Fabio Teixeira <fabio1079@gmail.com>`
		`+ Lucas Andrade <lucasandradeunb@gmail.com>`
		`+ Vitor Borges <vitorbertulucci@gmail.com>`

CHANGELOG.md

file modified

+11

		`@@ -1,5 +1,16 @@`
		`# Change Log`

		`+ ## [Unreleased]`
		`+ ### Added`
		`+ - Several API improvements.`
		`+ - Include ansible playbooks for easy deploy.`
		`+ - Use alembic to manage migrations.`
		`+ ### Changed`
		`+ - Convert files to firehose in memory.`
		`+ - Use PostgreSQL JSON type to save analyses.`
		`+ ### Removed`
		`+ - Python2 support.`
		`+`
		`## [0.3.1] - 2017-09-07`
		`### Fixed`
		`- Set package version.`

Jenkinsfile

file modified

+29 -18

		`@@ -2,35 +2,46 @@`
		`agent any`

		`stages {`
		`- stage('Build') {`
		`+`
		`+ stage('Create Virtualenv') {`
		`steps {`
		`sh 'virtualenv -p /usr/bin/python3 .'`
		`- sh 'source bin/activate && pip install -e .'`
		`+ sh 'source bin/activate && pip install -e .'`
		`}`
		`}`
		`- stage('build-docker-images') {`
		`+`
		`+ stage('Build Docker Images') {`
		`steps {`
		`sh 'source bin/activate && make analyzers'`
		`}`
		`}`
		`- stage('Test') {`
		`+`
		`+ stage('Run Tests') {`
		`steps {`
		`sh "source bin/activate && make check"`
		`}`
		`+ post {`
		`+ success {`
		`+ // publish html`
		`+ publishHTML target: [`
		`+ allowMissing: false,`
		`+ alwaysLinkToLastBuild: false,`
		`+ keepAll: true,`
		`+ reportDir: 'htmlcov',`
		`+ reportFiles: 'index.html',`
		`+ reportName: 'coverage report'`
		`+ ]`
		`+ }`
		`+ }`
		`+ }`

		`- post {`
		`- success {`
		`- // publish html`
		`- publishHTML target: [`
		`- allowMissing: false,`
		`- alwaysLinkToLastBuild: false,`
		`- keepAll: true,`
		`- reportDir: 'htmlcov',`
		`- reportFiles: 'index.html',`
		`- reportName: 'coverage report'`
		`- ]`
		`- }`
		`- }`
		`+ stage('Deploy') {`
		`+ steps {`
		`+ sh "if [ '${env.BRANCH_NAME}' = 'master' ] \`
		`+ \|\| [ '${env.BRANCH_NAME}' = 'continuos_dep' ]; then \`
		`+ source bin/activate && make deploy INVENTORY=production; \`
		`+ fi"`
		`+ }`
		`}`
		`- }`
		`+ }`
		`}`

Makefile

file modified

		`@@ -23,5 +23,9 @@`
		`popd; \`
		`done`

		`+ deploy:`
		`+ echo "Installing kiskadee...";`
		`+ ansible-playbook -i playbook/${INVENTORY} playbook/site.yml -f 10;`
		`+`
		`clean:`
		`rm -rf htmlcov .coverage`

README.md

file modified

+73 -4

		`@@ -22,15 +22,19 @@`
		`- openssl-devel`
		`- python3-devel`
		`- gcc`
		`- - redhat-rpm-config python-pip`
		`- - python-pip`
		`+ - redhat-rpm-config`
		`+ - python3-pip`

		`### Virtual Environment`

		`Create a [virtualenv](https://virtualenv.pypa.io/en/stable/) to kiskadee.`
		`The virtualenv package will create a isolated environment`
		`- for our python dependencies. To know more about virtualenvs and why to`
		`- use them we suggest [reading this article](Python Virtual Environments - a Primer).`
		`+ for our python dependencies.`
		`+`
		`+ You can use virtualenv or virtualenvwrapper, so just choose one`
		`+ of your preference and follow the tutorial bellow.`
		`+`
		`+ #### virtualenvwrapper`

		`sudo pip install virtualenv virtualenvwrapper`

		`@@ -55,6 +59,36 @@`
		`pip install -e .`
		`pip install "fedmsg[consumers]"`

		`+`
		`+ #### virtualenv`
		`+`
		`+ sudo pip install virtualenv`
		`+ virtualenv -p /usr/bin/python3 .`
		`+ source bin/activate`
		`+`
		`+ Install the python dependencies using pip`
		`+`
		`+ pip install -e .`
		`+ pip install "fedmsg[consumers]"`
		`+`
		`+ To know more about virtualenvs and why to`
		`+ use them we suggest [reading this article](Python Virtual Environments - a Primer).`
		`+`
		`+ ### Environment variables`
		`+`
		`+ Kiskadee database migration tool(alembic) get its database configuration`
		`+ from a environment variable named DATABASE_TYPE.`
		`+ If this variable is not defined, then it will assume its running on a developemnt`
		`+ environment, but for others environments such as test, homologation or`
		`+ production be sure to set which one are being used.`
		`+`
		`+ Only set DATABASE_TYPE if kiskadee is running on a non development environment.`
		+ ```bash
		`+ export DATABASE_TYPE=db_test`
		+ ```
		`+`
		`+ > To see which data each one of those alembic will use take a look on: util/kiskadee.conf`
		`+`
		`### Docker Images`

		`To run the static analyzers, you must have`
		`@@ -238,6 +272,41 @@`
		`[page](https://apps.fedoraproject.org/datagrepper/raw?category=anitya.)`
		`For more info about the Anitya service, read kiskadee documentation.`

		`+ ## Migrations`
		`+`
		`+ Kiskadee uses alembic as its tool for database migration, it has a solid`
		`+ documentation on: http://alembic.zzzcomputing.com/en/latest`
		`+`
		`+ For short, the most used commands are:`
		`+`
		`+ To create a new migration`
		+ ```bash
		`+ alembic revision -m "migration description"`
		+ ```
		`+`
		`+ To autogenerate a new migration`
		+ ```bash
		`+ alembic revision --autogenerate`
		+ ```
		`+`
		`+ or`
		`+`
		+ ```bash
		`+ alembic revision --autogenerate -m "some migration description"`
		+ ```
		`+`
		`+ To execute the migrations`
		+ ```bash
		`+ alembic upgrade head`
		`+ alembic upgrade +2`
		`+ alembic upgrade -1`
		`+ alembic upgrade some_revision_id+2`
		+ ```
		`+`
		`+ Downgrading`
		+ ```bash
		`+ alembic downgrade base`
		+ ```

		`## License`
		`Copyright (C) 2017 the AUTHORS (see the AUTHORS file)`

Vagrantfile

file added

+10

		`@@ -0,0 +1,10 @@`
		`+ # -- mode: ruby --`
		`+ # et ft=ruby :`
		`+ #`
		`+ Vagrant.configure("2") do \|config\|`
		`+ config.vm.box = "fedora/25-cloud-base"`
		`+ config.vm.box_version = "20161122"`
		`+`
		`+ config.vm.network "public_network", dev: "virbr0",`
		`+ mode: "bridge", type: "bridge", libvirt__network_name: "default"`
		`+ end`

alembic.ini

file added

+74

		`@@ -0,0 +1,74 @@`
		`+ # A generic, single database configuration.`
		`+`
		`+ [alembic]`
		`+ # path to migration scripts`
		`+ script_location = alembic`
		`+`
		`+ # template used to generate migration files`
		`+ # file_template = %%(rev)s_%%(slug)s`
		`+`
		`+ # timezone to use when rendering the date`
		`+ # within the migration file as well as the filename.`
		`+ # string value is passed to dateutil.tz.gettz()`
		`+ # leave blank for localtime`
		`+ # timezone =`
		`+`
		`+ # max length of characters to apply to the`
		`+ # "slug" field`
		`+ #truncate_slug_length = 40`
		`+`
		`+ # set to 'true' to run the environment during`
		`+ # the 'revision' command, regardless of autogenerate`
		`+ # revision_environment = false`
		`+`
		`+ # set to 'true' to allow .pyc and .pyo files without`
		`+ # a source .py file to be detected as revisions in the`
		`+ # versions/ directory`
		`+ # sourceless = false`
		`+`
		`+ # version location specification; this defaults`
		`+ # to alembic/versions. When using multiple version`
		`+ # directories, initial revisions must be specified with --version-path`
		`+ # version_locations = %(here)s/bar %(here)s/bat alembic/versions`
		`+`
		`+ # the output encoding used when revision files`
		`+ # are written from script.py.mako`
		`+ # output_encoding = utf-8`
		`+`
		`+ # sqlalchemy.url = driver://user:pass@localhost/dbname`
		`+`
		`+`
		`+ # Logging configuration`
		`+ [loggers]`
		`+ keys = root,sqlalchemy,alembic`
		`+`
		`+ [handlers]`
		`+ keys = console`
		`+`
		`+ [formatters]`
		`+ keys = generic`
		`+`
		`+ [logger_root]`
		`+ level = WARN`
		`+ handlers = console`
		`+ qualname =`
		`+`
		`+ [logger_sqlalchemy]`
		`+ level = WARN`
		`+ handlers =`
		`+ qualname = sqlalchemy.engine`
		`+`
		`+ [logger_alembic]`
		`+ level = INFO`
		`+ handlers =`
		`+ qualname = alembic`
		`+`
		`+ [handler_console]`
		`+ class = StreamHandler`
		`+ args = (sys.stderr,)`
		`+ level = NOTSET`
		`+ formatter = generic`
		`+`
		`+ [formatter_generic]`
		`+ format = %(levelname)-5.5s [%(name)s] %(message)s`
		`+ datefmt = %H:%M:%S`

alembic/README

file added

		`@@ -0,0 +1,1 @@`
		`+ Generic single-database configuration.`
		`\ No newline at end of file`

alembic/env.py

file added

+85

		`@@ -0,0 +1,85 @@`
		`+ """This file is used to setup and override some of alembic configurations."""`
		`+`
		`+ from __future__ import with_statement`
		`+ from alembic import context`
		`+ from sqlalchemy import engine_from_config, pool`
		`+ from logging.config import fileConfig`
		`+ from os import environ`
		`+`
		`+ from kiskadee.database import get_database_uri`
		`+ from kiskadee.model import Base`
		`+`
		`+ # this is the Alembic Config object, which provides`
		`+ # access to the values within the .ini file in use.`
		`+ config = context.config`
		`+`
		`+ # Interpret the config file for Python logging.`
		`+ # This line sets up loggers basically.`
		`+ fileConfig(config.config_file_name)`
		`+`
		`+ # add your model's MetaData object here`
		`+ # for 'autogenerate' support`
		`+ # from myapp import mymodel`
		`+ # target_metadata = mymodel.Base.metadata`
		`+ target_metadata = Base.metadata`
		`+`
		`+ # other values from the config, defined by the needs of env.py,`
		`+ # can be acquired:`
		`+ # my_important_option = config.get_main_option("my_important_option")`
		`+ # ... etc.`
		`+`
		`+ try:`
		`+ database_type = environ['DATABASE_TYPE']`
		`+ except KeyError:`
		`+ database_type = 'db_development'`
		`+`
		`+ db_uri = get_database_uri(database_type)`
		`+ config.set_main_option('sqlalchemy.url', db_uri)`
		`+`
		`+`
		`+ def run_migrations_offline():`
		`+ """Run migrations in 'offline' mode.`
		`+`
		`+ This configures the context with just a URL`
		`+ and not an Engine, though an Engine is acceptable`
		`+ here as well. By skipping the Engine creation`
		`+ we don't even need a DBAPI to be available.`
		`+`
		`+ Calls to context.execute() here emit the given string to the`
		`+ script output.`
		`+`
		`+ """`
		`+ url = config.get_main_option("sqlalchemy.url")`
		`+ context.configure(`
		`+ url=url, target_metadata=target_metadata, literal_binds=True)`
		`+`
		`+ with context.begin_transaction():`
		`+ context.run_migrations()`
		`+`
		`+`
		`+ def run_migrations_online():`
		`+ """Run migrations in 'online' mode.`
		`+`
		`+ In this scenario we need to create an Engine`
		`+ and associate a connection with the context.`
		`+`
		`+ """`
		`+ connectable = engine_from_config(`
		`+ config.get_section(config.config_ini_section),`
		`+ prefix='sqlalchemy.',`
		`+ poolclass=pool.NullPool)`
		`+`
		`+ with connectable.connect() as connection:`
		`+ context.configure(`
		`+ connection=connection,`
		`+ target_metadata=target_metadata`
		`+ )`
		`+`
		`+ with context.begin_transaction():`
		`+ context.run_migrations()`
		`+`
		`+`
		`+ if context.is_offline_mode():`
		`+ run_migrations_offline()`
		`+ else:`
		`+ run_migrations_online()`

alembic/script.py.mako

file added

+26

		`@@ -0,0 +1,26 @@`
		`+ """${message}.`
		`+`
		`+ Revision ID: ${up_revision}`
		`+ Revises: ${down_revision \| comma,n}`
		`+ Create Date: ${create_date}`
		`+`
		`+ """`
		`+ from alembic import op`
		`+ import sqlalchemy as sa`
		`+ ${imports if imports else ""}`
		`+`
		`+ # revision identifiers, used by Alembic.`
		`+ revision = ${repr(up_revision)}`
		`+ down_revision = ${repr(down_revision)}`
		`+ branch_labels = ${repr(branch_labels)}`
		`+ depends_on = ${repr(depends_on)}`
		`+`
		`+`
		`+ def upgrade():`
		`+ """TODO: Add an upgrade description."""`
		`+ ${upgrades if upgrades else "pass"}`
		`+`
		`+`
		`+ def downgrade():`
		`+ """TODO: Add a downgrade description."""`
		`+ ${downgrades if downgrades else "pass"}`

alembic/versions/48dd292b5a80_.py

file added

+72

		`@@ -0,0 +1,72 @@`
		`+ """Auto generate a migration to the following tables.`
		`+`
		`+ * reports`
		`+ * analysis`
		`+ * versions`
		`+ * analyzers`
		`+`
		`+ Revision ID: 48dd292b5a80`
		`+ Revises: d2989db85795`
		`+ Create Date: 2017-10-19 10:32:42.008412`
		`+`
		`+ """`
		`+ from alembic import op`
		`+ import sqlalchemy as sa`
		`+`
		`+`
		`+ # revision identifiers, used by Alembic.`
		`+ revision = '48dd292b5a80'`
		`+ down_revision = 'd2989db85795'`
		`+ branch_labels = None`
		`+ depends_on = None`
		`+`
		`+`
		`+ def upgrade():`
		`+ """Create tables: reports, analysis, versions, analyzers."""`
		`+ # ### commands auto generated by Alembic - please adjust! ###`
		`+ op.create_table(`
		`+ 'analyzers',`
		`+ sa.Column('id', sa.Integer(), nullable=False),`
		`+ sa.Column('name', sa.Unicode(length=255), nullable=False),`
		`+ sa.Column('version', sa.Unicode(length=255), nullable=True),`
		`+ sa.PrimaryKeyConstraint('id'),`
		`+ sa.UniqueConstraint('name')`
		`+ )`
		`+ op.create_table(`
		`+ 'versions',`
		`+ sa.Column('id', sa.Integer(), nullable=False),`
		`+ sa.Column('number', sa.Unicode(length=100), nullable=False),`
		`+ sa.Column('package_id', sa.Integer(), nullable=False),`
		`+ sa.ForeignKeyConstraint(['package_id'], ['packages.id'], ),`
		`+ sa.PrimaryKeyConstraint('id'),`
		`+ sa.UniqueConstraint('number', 'package_id')`
		`+ )`
		`+ op.create_table(`
		`+ 'analysis',`
		`+ sa.Column('id', sa.Integer(), nullable=False),`
		`+ sa.Column('version_id', sa.Integer(), nullable=False),`
		`+ sa.Column('analyzer_id', sa.Integer(), nullable=False),`
		`+ sa.Column('raw', sa.JSON(), nullable=True),`
		`+ sa.ForeignKeyConstraint(['analyzer_id'], ['analyzers.id'], ),`
		`+ sa.ForeignKeyConstraint(['version_id'], ['versions.id'], ),`
		`+ sa.PrimaryKeyConstraint('id')`
		`+ )`
		`+ op.create_table(`
		`+ 'reports',`
		`+ sa.Column('id', sa.Integer(), nullable=False),`
		`+ sa.Column('analysis_id', sa.Integer(), nullable=False),`
		`+ sa.Column('results', sa.JSON(), nullable=True),`
		`+ sa.ForeignKeyConstraint(['analysis_id'], ['analysis.id'], ),`
		`+ sa.PrimaryKeyConstraint('id')`
		`+ )`
		`+ # ### end Alembic commands ###`
		`+`
		`+`
		`+ def downgrade():`
		`+ """Drop the tables created at upgrade."""`
		`+ # ### commands auto generated by Alembic - please adjust! ###`
		`+ op.drop_table('reports')`
		`+ op.drop_table('analysis')`
		`+ op.drop_table('versions')`
		`+ op.drop_table('analyzers')`
		`+ # ### end Alembic commands ###`

alembic/versions/50988af48b09_create_fetcher_table.py

file added

+40

		`@@ -0,0 +1,40 @@`
		`+ """create fetcher table.`
		`+`
		`+ Revision ID: 50988af48b09`
		`+ Revises: 6daaf1d5cfee`
		`+ Create Date: 2017-10-17 16:48:26.940294`
		`+`
		`+ """`
		`+ from alembic import op`
		`+ import sqlalchemy as sa`
		`+`
		`+`
		`+ # revision identifiers, used by Alembic.`
		`+ revision = '50988af48b09'`
		`+ down_revision = '6daaf1d5cfee'`
		`+ branch_labels = None`
		`+ depends_on = None`
		`+`
		`+`
		`+ def upgrade():`
		`+ """Create table fetchers."""`
		`+ op.create_table(`
		`+ 'fetchers',`
		`+ sa.Column(`
		`+ 'id',`
		`+ sa.Integer,`
		`+ sa.Sequence(`
		`+ 'fetchers_id_seq',`
		`+ optional=True`
		`+ ),`
		`+ primary_key=True`
		`+ ),`
		`+ sa.Column('name', sa.Unicode(255), nullable=False, unique=True),`
		`+ sa.Column('target', sa.Unicode(255), nullable=True),`
		`+ sa.Column('description', sa.UnicodeText)`
		`+ )`
		`+`
		`+`
		`+ def downgrade():`
		`+ """Drop table fetchers."""`
		`+ op.drop_table('fetchers')`

alembic/versions/6daaf1d5cfee_create_package_table.py

file added

+34

		`@@ -0,0 +1,34 @@`
		`+ """create Package table.`
		`+`
		`+ Revision ID: 6daaf1d5cfee`
		`+ Revises:`
		`+ Create Date: 2017-10-17 15:48:05.193085`
		`+`
		`+ """`
		`+ from alembic import op`
		`+ import sqlalchemy as sa`
		`+`
		`+ # revision identifiers, used by Alembic.`
		`+ revision = '6daaf1d5cfee'`
		`+ down_revision = None`
		`+ branch_labels = None`
		`+ depends_on = None`
		`+`
		`+`
		`+ def upgrade():`
		`+ """Create table packages."""`
		`+ op.create_table(`
		`+ 'packages',`
		`+ sa.Column(`
		`+ 'id',`
		`+ sa.Integer,`
		`+ sa.Sequence('packages_id_seq', optional=True),`
		`+ primary_key=True`
		`+ ),`
		`+ sa.Column('name', sa.Unicode(255), nullable=False)`
		`+ )`
		`+`
		`+`
		`+ def downgrade():`
		`+ """Drop table packages."""`
		`+ op.drop_table('packages')`

alembic/versions/d2989db85795_add_relation_between_fetchers_and_.py

file added

+40

		`@@ -0,0 +1,40 @@`
		`+ """add relation between fetchers and packages tables.`
		`+`
		`+ Revision ID: d2989db85795`
		`+ Revises: 50988af48b09`
		`+ Create Date: 2017-10-17 16:59:09.495015`
		`+`
		`+ """`
		`+ from alembic import op`
		`+ import sqlalchemy as sa`
		`+`
		`+`
		`+ # revision identifiers, used by Alembic.`
		`+ revision = 'd2989db85795'`
		`+ down_revision = '50988af48b09'`
		`+ branch_labels = None`
		`+ depends_on = None`
		`+`
		`+`
		`+ def upgrade():`
		`+ """Add fetch_id to packages and set a unique constraint for it."""`
		`+ with op.batch_alter_table("packages") as batch_op:`
		`+ batch_op.add_column(`
		`+ sa.Column(`
		`+ 'fetcher_id',`
		`+ sa.Integer,`
		`+ sa.ForeignKey('fetchers.id'),`
		`+ nullable=False`
		`+ )`
		`+ )`
		`+ batch_op.create_unique_constraint(`
		`+ 'name_and_fetcher_id_unique',`
		`+ ['name', 'fetcher_id']`
		`+ )`
		`+`
		`+`
		`+ def downgrade():`
		`+ """Undo add fetcher_id to packages."""`
		`+ with op.batch_alter_table("packages") as batch_op:`
		`+ op.drop_constraint('name_and_fetcher_id_unique', 'packages')`
		`+ batch_op.drop_column('fetcher_id')`

kiskadee/__init__.py

file modified

+1 -1

		`@@ -28,7 +28,7 @@`
		`import logging`
		`import sys`

		`- __version__ = '0.3.1'`
		`+ __version__ = '0.3.2~dev'`

		`_my_path = os.path.dirname(os.path.realpath(__file__))`

kiskadee/database.py

file modified

+17 -12

		`@@ -16,19 +16,24 @@`
		`Base.metadata.bind = self.engine`

		`def _create_engine(self, db):`
		`- driver = kiskadee.config[db]['driver']`
		`- username = kiskadee.config[db]['username']`
		`- password = kiskadee.config[db]['password']`
		`- hostname = kiskadee.config[db]['hostname']`
		`- port = kiskadee.config[db]['port']`
		`- dbname = kiskadee.config[db]['dbname']`
		`- return create_engine('%s://%s:%s@%s:%s/%s' % (driver,`
		`- username,`
		`- password,`
		`- hostname,`
		`- port,`
		`- dbname))`
		`+ uri = get_database_uri(db)`
		`+ return create_engine(uri)`

		`def _create_session(self, engine):`
		`DBSession = orm.sessionmaker(bind=engine)`
		`return DBSession()`
		`+`
		`+`
		`+ def get_database_uri(db):`
		`+ """Return the Database URI of the current session."""`
		`+ config = kiskadee.config[db]`
		`+`
		`+ driver = config['driver']`
		`+ username = config['username']`
		`+ password = config['password']`
		`+ hostname = config['hostname']`
		`+ port = config['port']`
		`+ dbname = config['dbname']`
		`+`
		`+ return '%s://%s:%s@%s:%s/%s' % (driver, username, password,`
		`+ hostname, port, dbname)`

playbook/development

file added

		`@@ -0,0 +1,2 @@`
		`+ [vm]`
		`+ 192.168.121.148`

playbook/production

file added

		`@@ -0,0 +1,2 @@`
		`+ [vm]`
		`+ 172.24.0.55`

playbook/roles/common/tasks/main.yml

file added

+19

		`@@ -0,0 +1,19 @@`
		`+ - name: install yum`
		`+ dnf:`
		`+ name: yum`
		`+`
		`+ - name: install fedora dependencies`
		`+ dnf: name={{ item }} state=installed`
		`+ with_items:`
		`+ - openssl-devel`
		`+ - python2-devel`
		`+ - python3-devel`
		`+ - gcc`
		`+ - redhat-rpm-config`
		`+ - python3-pip`
		`+ - dnf-plugins-core`
		`+ - git`
		`+ - postgresql-server`
		`+ - postgresql-contrib`
		`+ - libselinux-python`
		`+ ignore_errors: yes`

playbook/roles/database/tasks/main.yml

file added

+63

		`@@ -0,0 +1,63 @@`
		`+ - name: Enable postgresql`
		`+ service:`
		`+ name: postgresql`
		`+ enabled: yes`
		`+`
		`+ - name: initialize postgresql configuration`
		`+ command: postgresql-setup initdb`
		`+ ignore_errors: yes`
		`+`
		`+ - name: Start postgresql`
		`+ service:`
		`+ name: postgresql`
		`+ state: started`
		`+`
		`+ - name: setup database password`
		`+ shell: psql -U postgres -c "create user kiskadee with PASSWORD 'kiskadee';"`
		`+ become: true`
		`+ become_user: postgres`
		`+ ignore_errors: yes`
		`+`
		`+ - name: save database password`
		`+ shell: echo "localhost:5432:kiskadee:kiskadee:kiskadee" > ~/.pgpass`
		`+ become: true`
		`+ become_user: kiskadee`
		`+`
		`+ - name: change pgpass permissions`
		`+ file:`
		`+ path: /home/kiskadee/.pgpass`
		`+ mode: 0600`
		`+`
		`+ - name: create development database`
		`+ shell: psql -U postgres -c "create database kiskadee with owner kiskadee;"`
		`+ become: true`
		`+ become_user: postgres`
		`+ ignore_errors: yes`
		`+`
		`+ - name: grant privileges on development database`
		`+ shell: psql -U postgres -c "grant all privileges on database kiskadee to kiskadee"`
		`+ become: true`
		`+ become_user: postgres`
		`+`
		`+ - name: create test database`
		`+ shell: psql -U postgres -c "create database kiskadee_test with owner kiskadee;"`
		`+ become: true`
		`+ become_user: postgres`
		`+ ignore_errors: yes`
		`+`
		`+ - name: grant privileges on test database`
		`+ shell: psql -U postgres -c "grant all privileges on database kiskadee_test to kiskadee"`
		`+ become: true`
		`+ become_user: postgres`
		`+`
		`+ - name: configure pg_hba.conf`
		`+ template:`
		`+ src: pg_hba.conf`
		`+ dest: /var/lib/pgsql/data/pg_hba.conf`
		`+ force: yes`
		`+ mode: 0644`
		`+`
		`+ - name: Start postgresql`
		`+ service:`
		`+ name: postgresql`
		`+ state: restarted`

playbook/roles/database/templates/pg_hba.conf

file added

		`@@ -0,0 +1,8 @@`
		`+ local all postgres peer`
		`+ # "local" is for Unix domain socket connections only`
		`+ local all kiskadee password`
		`+ local all kiskadee_test password`
		`+ # IPv4 local connections:`
		`+ host all all 127.0.0.1/32 password`
		`+ # IPv6 local connections:`
		`+ host all all ::1/128 password`

playbook/roles/docker/tasks/main.yml

file added

+16

		`@@ -0,0 +1,16 @@`
		`+ - name: Configure docker repository`
		`+ command: dnf config-manager --add-repo https://download.docker.com/linux/fedora/docker-ce.repo`
		`+`
		`+ - name: Install docker engine`
		`+ dnf: name=docker-ce`
		`+`
		`+ - name: Start docker service`
		`+ service:`
		`+ name: docker`
		`+ state: started`
		`+`
		`+ - name: Change docker socket permitions`
		`+ file:`
		`+ path: /var/run/docker.sock`
		`+ mode: 0777`
		`+`

playbook/roles/git/tasks/main.yml

file added

+14

		`@@ -0,0 +1,14 @@`
		`+ - name: Create kiskadee user`
		`+ user:`
		`+ name: kiskadee`
		`+ group: wheel`
		`+ shell: /bin/bash`
		`+`
		`+ - name: Clone kiskadee repository`
		`+ git:`
		`+ repo: https://pagure.io/kiskadee.git`
		`+ dest: /home/kiskadee/kiskadee`
		`+ force: yes`
		`+ become: yes`
		`+ become_user: kiskadee`
		`+`

playbook/roles/python/tasks/main.yml

file added

+50

		`@@ -0,0 +1,50 @@`
		`+ - name: Instal virtualenvwrapper package`
		`+ pip:`
		`+ name: virtualenvwrapper`
		`+ executable: pip3`
		`+`
		`+ # TODO: do this only once`
		`+ - name: source virtualenvwrapper`
		`+ shell: echo source /usr/bin/virtualenvwrapper.sh >> /home/kiskadee/.bashrc`
		`+`
		`+ # TODO: do this only once`
		`+ - name: Export workon env`
		`+ shell: echo export WORKON_HOME=/home/kiskadee/Envs >> /home/kiskadee/.bashrc`
		`+`
		`+ # TODO: do this only once`
		`+ - name: Export python version`
		`+ shell: echo export VIRTUALENVWRAPPER_PYTHON="/usr/bin/python3" >> /home/kiskadee/.bashrc`
		`+`
		`+ - name: Change virtualenv path permitions`
		`+ file:`
		`+ path: /home/kiskadee/Envs`
		`+ owner: kiskadee`
		`+ state: directory`
		`+ mode: 0755`
		`+`
		`+ - name: Create kiskadee virtualenv`
		`+ environment:`
		`+ WORKON_HOME: /home/kiskadee/Envs`
		`+ VIRTUALENVWRAPPER_PYTHON: "/usr/bin/python3"`
		`+ shell: source /usr/bin/virtualenvwrapper.sh && mkvirtualenv --python=/usr/bin/python3 kiskadee`
		`+ chdir: /home/kiskadee/kiskadee`
		`+ become: true`
		`+ become_user: kiskadee`
		`+`
		`+ - name: Install kiskadee dependencies`
		`+ environment:`
		`+ WORKON_HOME: /home/kiskadee/Envs`
		`+ VIRTUALENVWRAPPER_PYTHON: "/usr/bin/python3"`
		`+ shell: source /usr/bin/virtualenvwrapper.sh && workon kiskadee && pip install -e . && pip install "fedmsg[consumers]"`
		`+ become: true`
		`+ become_user: kiskadee`
		`+ args:`
		`+ chdir: /home/kiskadee/kiskadee/`
		`+`
		`+ - name: build docker images`
		`+ make:`
		`+ chdir: /home/kiskadee/kiskadee`
		`+ target: analyzers`
		`+ become: true`
		`+ become_user: kiskadee`
		`+`

playbook/site.yml

file added

+10

		`@@ -0,0 +1,10 @@`
		`+ ---`
		`+ # TODO: fix a static IP on Vagrantfile`
		`+ - hosts: vm`
		`+ remote_user: root`
		`+ roles:`
		`+ - common`
		`+ - docker`
		`+ - git`
		`+ - python`
		`+ - database`

requirements.txt

file modified

		`@@ -18,3 +18,5 @@`
		`flask-cors`
		`coverage`
		`nose`
		`+ ansible`
		`+ alembic`

lucasandrade commented 7 years ago

Signed-off-by: lucasandrade lucasandradeunb@gmail.com
Signed-off-by: DaniloBarros dan.b412@gmail.com

athoscr commented 7 years ago

Thanks for the PR!

Note that this information should really go in the docs, under the doc directory. But since this README file is already quite bloated, let's work to merge this. Any help migrating the information in the README file in the documentation will be much appreciated though.

Note that you added information on how to isolate the development environment with virtualenv wrapper but removed the information for running it using just virtualenv. Since the information is already there, I'd rather keep guidance for both strategies. Any thoughts?

1 new commit added