PR#123: database migrations should be a pre-deployment hook - waiverdb

waiverdb

#123 database migrations should be a pre-deployment hook

Merged 6 years ago by ralph. Opened 6 years ago by dcallagh.

dcallagh/waiverdb issue-121 into master

database migrations should be a pre-deployment hook

Dan Callaghan • 6 years ago

0c12a70

file modified

+11 -4

		`@@ -170,6 +170,17 @@`
		`selector:`
		`environment: "test-${TEST_ID}"`
		`service: web`
		`+ strategy:`
		`+ type: Rolling`
		`+ rollingParams:`
		`+ pre:`
		`+ failurePolicy: Abort`
		`+ execNewPod:`
		`+ containerName: web`
		`+ command: [ /bin/sh, -i, -c, "waiverdb wait-for-db && waiverdb db upgrade" ]`
		`+ volumes:`
		`+ - config-volume`
		`+ - secret-volume`
		`template:`
		`metadata:`
		`labels:`
		`@@ -181,10 +192,6 @@`
		`image: "docker-registry.engineering.redhat.com/factory2/waiverdb:${WAIVERDB_APP_VERSION}"`
		`ports:`
		`- containerPort: 8080`
		`- lifecycle:`
		`- postStart:`
		`- exec:`
		`- command: [ /bin/sh, -i, -c, "waiverdb db upgrade" ]`
		`volumeMounts:`
		`- name: config-volume`
		`mountPath: /etc/waiverdb`

waiverdb/manage.py

file modified

+21

		`@@ -1,7 +1,10 @@`
		`# SPDX-License-Identifier: GPL-2.0+`

		`+ import time`
		`import click`
		`from flask.cli import FlaskGroup`
		`+ from sqlalchemy.exc import OperationalError`
		`+ from waiverdb.models import db`


		`def create_waiver_app(_):`
		`@@ -14,5 +17,23 @@`
		`pass`


		`+ @cli.command(name='wait-for-db')`
		`+ def wait_for_db():`
		`+ """`
		`+ Wait until database server is reachable.`
		`+ """`
		`+ poll_interval = 10 # seconds`
		`+ while True:`
		`+ try:`
		`+ db.engine.connect()`
		`+ except OperationalError as e:`
		`+ click.echo('Failed to connect to database: {}'.format(e))`
		`+ click.echo('Sleeping for {} seconds...'.format(poll_interval))`
		`+ time.sleep(poll_interval)`
		`+ click.echo('Retrying...')`
		`+ else:`
		`+ break`
		`+`
		`+`
		`if __name__ == '__main__':`
		`cli() # pylint: disable=E1120`

dcallagh commented 6 years ago

... not a post-start pod hook for the web app. This way, we can see the
logs of the migration pod, and if the migration fails the deployment
will be aborted.

The database migration pod has to first wait for the Postgres pod to
finish starting up, because in a freshly created environment (as in the
tests) the migration pod is started concurrently with the Postgres pod.

Fixes #121.