#96 prep for release 2.10
Closed 3 years ago by scoady. Opened 3 years ago by scoady.
scoady/elections release-2.10  into  production

file modified
+2
@@ -5,6 +5,8 @@ 

  build/

  dist/

  .coverage

+ coverage.xml

+ client_secrets.json

  alembic.ini

  .vagrant/

  .tox/

file modified
+33 -9
@@ -16,6 +16,7 @@ 

  [Vagrant](https://www.vagrantup.com/ "Vagrant by Hashicorp"), a powerful and

  useful tool for creating development environments on your workstation.

  

+ 

  ### Using Vagrant

  

  You can quickly start hacking on the Fedora Elections web application using the
@@ -46,6 +47,7 @@ 

  Once that is running, go to [localhost:5005](http://localhost:5005/) in your

  browser to see your running Fedora Elections test instance.

  

+ 

  ### A note about fonts

  

  Fedora Elections uses web fonts hosted in Fedora's infrastructure that might
@@ -104,30 +106,40 @@ 

  git clone https://pagure.io/elections.git

  ```

  

- ### Configure the application

+ ### Install pip requirements w/ tox for testing

  

- An example configuration file is provided [here](https://pagure.io/elections/blob/master/f/files/fedora-elections.cfg "files/fedora-elections.cfg").

+ Set up venv (replacing `<base_path_for_venv>` and `<venv_name>`):

  

- ### Create database

+ ```

+ pip install --user virtualenv tox

+ mkvirtualenv <base_path_for_venv>/<venv_name>

+ . <base_path_for_venv>/<venv_name>

+ ```

  

- Run:

+ Install requirements:

  

  ```

- python createdb.py

+ pip install -r requirements.txt

  ```

  

+ ### Configure the application

+ 

+ An example configuration file is provided [here](https://pagure.io/elections/blob/master/f/files/fedora-elections.cfg "files/fedora-elections.cfg").

+ 

  ### Register the application using openid-connect

  

- Run:

+ From root of project, run:

  

  ```

- oidc-register https://iddev.fedorainfracloud.org/ http://localhost:5005/oidc_callback

+ oidc-register https://iddev.fedorainfracloud.org/ http://localhost:5005

  ```

  

- Copy the corresponding ``client_secrets.json`` in the sources:

+ ### Create database

+ 

+ Run:

  

  ```

- cp client_secrets.json fedora_elections/client_secrets.json

+ python createdb.py

  ```

  

  ### Create a local configuration file
@@ -192,6 +204,18 @@ 

  ```

  

  

+ ### Running tests

+ fedora-elections uses `tox` to simplify testing, and support testing across multiple environments.

+ 

+ Refer to earlier in this section on "How to launch Fedora Elections" in this document to get set up with `tox`

+ 

+ To run tests, simply run:

+ 

+ ```

+ tox

+ ```

+ 

+ 

  ## How to contribute

  

  As mentioned earlier, this project is primarily hosted on [Pagure](https://pagure.io/elections "Fedora Infrastructure Elections application").

file modified
+16 -10
@@ -25,7 +25,7 @@ 

  #

  from __future__ import unicode_literals, absolute_import

  

- __version__ = '2.9'

+ __version__ = '2.10'

  

  import logging  # noqa

  import os  # noqa
@@ -43,6 +43,7 @@ 

  import munch  # noqa

  import six  # noqa

  

+ from fasjson_client import Client

  from fedora.client import AuthError, AppError  # noqa

  from fedora.client.fas2 import AccountSystem  # noqa

  from flask_oidc import OpenIDConnect  # noqa
@@ -64,13 +65,18 @@ 

  

  APP.wsgi_app = fedora_elections.proxy.ReverseProxied(APP.wsgi_app)

  

- # FAS for usernames.

- FAS2 = AccountSystem(

-     APP.config['FAS_BASE_URL'],

-     username=APP.config['FAS_USERNAME'],

-     password=APP.config['FAS_PASSWORD'],

-     insecure=not APP.config['FAS_CHECK_CERT']

- )

+ if APP.config.get('FASJSON'):

+     ACCOUNTS = Client(

+         url=APP.config['FAS_BASE_URL']

+     )

+ else:

+     # FAS for usernames.

+     ACCOUNTS = AccountSystem(

+         APP.config['FAS_BASE_URL'],

+         username=APP.config['FAS_USERNAME'],

+         password=APP.config['FAS_PASSWORD'],

+         insecure=not APP.config['FAS_CHECK_CERT']

+     )

  

  

  # modular imports
@@ -210,8 +216,8 @@ 

                  'email': OIDC.user_getfield('email') or '',

                  'timezone': OIDC.user_getfield('zoneinfo'),

                  'cla_done':

-                     'http://admin.fedoraproject.org/accounts/cla/done'

-                     in (OIDC.user_getfield('cla') or []),

+                     'FPCA'

+                     in (OIDC.user_getfield('agreements') or []),

              })

          flask.g.fas_user = flask.session.fas_user

      else:

file modified
+47 -35
@@ -31,13 +31,21 @@ 

  import flask

  from sqlalchemy.exc import SQLAlchemyError

  from fedora.client import AuthError

+ from fedora_elections_messages import (

+     NewElectionV1,

+     EditElectionV1,

+     NewCandidateV1,

+     EditCandidateV1,

+     DeleteCandidateV1,

+ )

  

  from fedora_elections import fedmsgshim

  from fedora_elections import forms

  from fedora_elections import models

  from fedora_elections import (

-     APP, SESSION, FAS2, is_authenticated, is_admin

+     APP, SESSION, ACCOUNTS, is_authenticated, is_admin

  )

+ from fasjson_client.errors import APIError

  

  

  def election_admin_required(f):
@@ -108,13 +116,11 @@ 

  

          SESSION.commit()

  

-         fedmsgshim.publish(

-             topic="election.new",

-             msg=dict(

+         fedmsgshim.publish(NewElectionV1(body=dict(

                  agent=flask.g.fas_user.username,

                  election=election.to_json(),

-                 )

-         )

+             )

+         ))

  

          flask.flash('Election "%s" added' % election.alias)

          return flask.redirect(flask.url_for(
@@ -196,13 +202,11 @@ 

              SESSION.delete(admingrp)

  

          SESSION.commit()

-         fedmsgshim.publish(

-             topic="election.edit",

-             msg=dict(

+         fedmsgshim.publish(EditElectionV1(body=dict(

                  agent=flask.g.fas_user.username,

                  election=election.to_json(),

              )

-         )

+         ))

          flask.flash('Election "%s" saved' % election.alias)

          return flask.redirect(flask.url_for(

              'admin_view_election', election_alias=election.alias))
@@ -230,9 +234,14 @@ 

          fas_name = None

          if election.candidates_are_fasusers:  # pragma: no cover

              try:

-                 fas_name = FAS2.person_by_username(

-                     form.name.data)['human_name']

-             except (KeyError, AuthError):

+                 if APP.config.get('FASJSON'):

+                     user = ACCOUNTS.get_user(

+                         username=form.name.data).result

+                     fas_name = f"{user['givenname']} {user['surname']}"

+                 else:

+                     fas_name = ACCOUNTS.person_by_username(

+                         form.name.data)['human_name']

+             except (KeyError, AuthError, APIError):

                  flask.flash(

                      'User `%s` does not have a FAS account.'

                      % form.name.data, 'error')
@@ -251,14 +260,12 @@ 

          SESSION.add(candidate)

          SESSION.commit()

          flask.flash('Candidate "%s" saved' % candidate.name)

-         fedmsgshim.publish(

-             topic="candidate.new",

-             msg=dict(

+         fedmsgshim.publish(NewCandidateV1(body=dict(

                  agent=flask.g.fas_user.username,

                  election=candidate.election.to_json(),

                  candidate=candidate.to_json(),

              )

-         )

+         ))

          return flask.redirect(flask.url_for(

              'admin_view_election', election_alias=election.alias))

  
@@ -286,9 +293,14 @@ 

              fas_name = None

              if election.candidates_are_fasusers:  # pragma: no cover

                  try:

-                     fas_name = FAS2.person_by_username(

-                         candidate[0])['human_name']

-                 except (KeyError, AuthError):

+                     if APP.config.get('FASJSON'):

+                         user = ACCOUNTS.get_user(

+                             username=candidate[0]).result

+                         fas_name = f"{user['givenname']} {user['surname']}"

+                     else:

+                         fas_name = ACCOUNTS.person_by_username(

+                             candidate[0])['human_name']

+                 except (KeyError, AuthError, APIError):

                      SESSION.rollback()

                      flask.flash(

                          'User `%s` does not have a FAS account.'
@@ -317,14 +329,13 @@ 

                  candidates_name.append(cand.name)

              else:

                  flask.flash("There was an issue!")

-             fedmsgshim.publish(

-                 topic="candidate.new",

-                 msg=dict(

+                 continue

+             fedmsgshim.publish(NewCandidateV1(body=dict(

                      agent=flask.g.fas_user.username,

                      election=cand.election.to_json(),

                      candidate=cand.to_json(),

                  )

-             )

+             ))

  

          SESSION.commit()

          flask.flash('Added %s candidates' % len(candidates_name))
@@ -356,9 +367,14 @@ 

  

          if election.candidates_are_fasusers:  # pragma: no cover

              try:

-                 candidate.fas_name = FAS2.person_by_username(

-                     candidate.name)['human_name']

-             except (KeyError, AuthError):

+                 if APP.config.get('FASJSON'):

+                     user = ACCOUNTS.get_user(

+                         username=candidate.name).result

+                     candidate.fas_name = f"{user['givenname']} {user['surname']}"

+                 else:

+                     candidate.fas_name = ACCOUNTS.person_by_username(

+                         candidate.name)['human_name']

+             except (KeyError, AuthError, APIError):

                  SESSION.rollback()

                  flask.flash(

                      'User `%s` does not have a FAS account.'
@@ -370,14 +386,12 @@ 

  

          SESSION.commit()

          flask.flash('Candidate "%s" saved' % candidate.name)

-         fedmsgshim.publish(

-             topic="candidate.edit",

-             msg=dict(

+         fedmsgshim.publish(EditCandidateV1(body=dict(

                  agent=flask.g.fas_user.username,

                  election=candidate.election.to_json(),

                  candidate=candidate.to_json(),

              )

-         )

+         ))

          return flask.redirect(flask.url_for(

              'admin_view_election', election_alias=election.alias))

  
@@ -407,14 +421,12 @@ 

              SESSION.delete(candidate)

              SESSION.commit()

              flask.flash('Candidate "%s" deleted' % candidate_name)

-             fedmsgshim.publish(

-                 topic="candidate.delete",

-                 msg=dict(

+             fedmsgshim.publish(DeleteCandidateV1(body=dict(

                      agent=flask.g.fas_user.username,

                      election=candidate.election.to_json(),

                      candidate=candidate.to_json(),

                  )

-             )

+             ))

          except SQLAlchemyError as err:

              SESSION.rollback()

              APP.logger.debug('Could not delete candidate')

@@ -19,6 +19,8 @@ 

  # You will want to change this for your install

  SECRET_KEY = 'change me'

  

+ FASJSON = False

+ 

  FAS_BASE_URL = 'https://admin.stg.fedoraproject.org/accounts/'

  FAS_USERNAME = ''

  FAS_PASSWORD = ''
@@ -27,7 +29,7 @@ 

  

  OIDC_CLIENT_SECRETS = os.path.join(os.path.dirname(

      os.path.abspath(__file__)), '..', 'client_secrets.json')

- OIDC_SCOPES = ['openid', 'email', 'profile', 'fedora']

+ OIDC_SCOPES = ['openid', 'email', 'profile', 'https://id.fedoraproject.org/scope/groups', 'https://id.fedoraproject.org/scope/agreements']

  OIDC_OPENID_REALM = 'http://localhost:5005/oidc_callback'

  

  LOGGING = {

@@ -15,13 +15,9 @@ 

  _log = logging.getLogger(__name__)

  

  

- def publish(topic, msg):  # pragma: no cover

-     _log.debug('Publishing a message for %r: %s', topic, msg)

+ def publish(message):  # pragma: no cover

+     _log.debug('Publishing a message for %r: %s', message.topic, message.body)

      try:

-         message = fedora_messaging.api.Message(

-             topic='fedora_elections.%s' % topic,

-             body=msg

-         )

          fedora_messaging.api.publish(message)

          _log.debug("Sent to fedora_messaging")

      except PublishReturned as e:

file modified
+10 -4
@@ -10,8 +10,9 @@ 

  

  

  from fedora.client import AuthError

- from fedora_elections import SESSION, FAS2, APP

+ from fedora_elections import SESSION, ACCOUNTS, APP

  from fedora_elections.models import Election

+ from fasjson_client.errors import APIError

  

  

  class ElectionForm(FlaskForm):
@@ -155,9 +156,14 @@ 

          if fasusers:  # pragma: no cover

              # We can't cover FAS integration

              try:

-                 title = \

-                     FAS2.person_by_username(candidate.name)['human_name']

-             except (KeyError, AuthError) as err:

+                 if APP.config.get('FASJSON'):

+                     user = ACCOUNTS.get_user(

+                         username=candidate.name).result

+                     title = f"{user['givenname']} {user['surname']}"

+                 else:

+                     title = ACCOUNTS.person_by_username(

+                         candidate.name)['human_name']

+             except (KeyError, AuthError, APIError) as err:

                  APP.logger.debug(err)

          if candidate.url:

              title = '%s <a href="%s" target="_blank" rel="noopener noreferrer">[Info]</a>' % (title, candidate.url)

@@ -267,6 +267,7 @@ 

          ''' Return a json representation of this object. '''

          return dict(

              name=self.name,

+             fas_name=self.fas_name,

              url=self.url,

          )

  

@@ -15,6 +15,11 @@ 

  ## application, including all elections past, present and future

  FEDORA_ELECTIONS_ADMIN_GROUP = 'elections'

  

+ # Elections directly connects to the accounts backend to get

+ # details of nominees when adding them to an election. 

+ # if FASJSON is false, elections will connect to FAS2. if FASJSON is

+ # True, elections will connect to FASJSON

+ FASJSON = False

  

  ## Fedora-elections can integrate with FAS to retrieve information about the

  ## candidates, the following configuration keys are required for this

file modified
+8 -1
@@ -1,7 +1,7 @@ 

  %define modname fedora_elections

  

  Name:           fedora-elections

- Version:        2.9

+ Version:        2.10

  Release:        1%{?dist}

  Summary:        Fedora elections application

  
@@ -112,6 +112,13 @@ 

  

  

  %changelog

+ * Mon Mar 29 2021 Stephen Coady <scoady@redhat.com> 2.10-1

+ - Update to 2.10

+ - add fasjson support

+ - general bugfixes

+ - make use of fedora messaging schemas

+ - use new oidc scopes

+ 

  * Mon Nov 18 2019 Ben Cotton <bcotton@fedoraproject.org> 2.9-1

  - Update to 2.9 (2.8 existed, in a sense)

  - Open "more info" links in a new window

file modified
+2
@@ -14,3 +14,5 @@ 

  gunicorn

  psycopg2

  rsa==4.0  # last version that supports python2

+ fasjson_client

+ fedora_elections_messages

file modified
+10 -1
@@ -26,6 +26,12 @@ 

      '--port', '-p', default=5005,

      help='Port for the flask application.')

  parser.add_argument(

+     '--cert', '-s', default=None,

+     help='Filename of SSL cert for the flask application.')

+ parser.add_argument(

+     '--key', '-k', default=None,

+     help='Filename of the SSL key for the flask application.')

+ parser.add_argument(

      '--host', default="127.0.0.1",

      help='Hostname to listen on. When set to 0.0.0.0 the server is available \

      externally. Defaults to 127.0.0.1 making the it only visable on localhost')
@@ -47,4 +53,7 @@ 

      os.environ['FEDORA_ELECTIONS_CONFIG'] = config

  

  APP.debug = True

- APP.run(host=args.host, port=int(args.port))

+ if args.cert and args.key:

+     APP.run(host=args.host, port=int(args.port), ssl_context=(args.cert, args.key))

+ else:

+     APP.run(host=args.host, port=int(args.port))

file modified
+1 -1
@@ -28,7 +28,7 @@ 

      install_requires=[

          'Flask', 'SQLAlchemy>=0.7', 'python-fedora', 'kitchen',

          'python-openid', 'python-openid-teams', 'python-openid-cla',

-         'Flask-wtf', 'wtforms',

+         'Flask-wtf', 'wtforms', 'fedora-elections-messages',

      ],

      test_suite="tests",

  )

file modified
+2
@@ -218,6 +218,7 @@ 

              {

                  'name': 'Ralph',

                  'url': 'https://fedoraproject.org/wiki/User:Ralph',

+                 'fas_name': None,

              }

          )

  
@@ -227,6 +228,7 @@ 

              {

                  'name': 'Kevin',

                  'url': 'https://fedoraproject.org/wiki/User:Kevin',

+                 'fas_name': None,

              }

          )

  

file modified
+41 -26
@@ -33,6 +33,10 @@ 

  

  import flask

  from mock import patch, MagicMock

+ from fedora_messaging.testing import mock_sends

+ from fedora_elections_messages import (

+     NewElectionV1, EditElectionV1, NewCandidateV1, EditCandidateV1, DeleteCandidateV1,

+ )

  

  sys.path.insert(0, os.path.join(os.path.dirname(

      os.path.abspath(__file__)), '..'))
@@ -318,8 +322,9 @@ 

                      'csrf_token': csrf_token,

                  }

  

-                 output = self.app.post(

-                     '/admin/new', data=data, follow_redirects=True)

+                 with mock_sends(NewElectionV1):

+                     output = self.app.post(

+                         '/admin/new', data=data, follow_redirects=True)

                  self.assertEqual(output.status_code, 200)

                  output_text = output.get_data(as_text=True)

                  self.assertTrue(
@@ -357,8 +362,9 @@ 

                      'csrf_token': csrf_token,

                  }

  

-                 output = self.app.post(

-                     '/admin/new', data=data, follow_redirects=True)

+                 with mock_sends(NewElectionV1):

+                     output = self.app.post(

+                         '/admin/new', data=data, follow_redirects=True)

                  self.assertEqual(output.status_code, 200)

                  output_text = output.get_data(as_text=True)

                  self.assertTrue(
@@ -502,8 +508,9 @@ 

                      'csrf_token': csrf_token,

                  }

  

-                 output = self.app.post(

-                     '/admin/test_election/', data=data, follow_redirects=True)

+                 with mock_sends(EditElectionV1):

+                     output = self.app.post(

+                         '/admin/test_election/', data=data, follow_redirects=True)

                  self.assertEqual(output.status_code, 200)

                  output_text = output.get_data(as_text=True)

                  self.assertTrue(
@@ -580,8 +587,9 @@ 

                      'csrf_token': csrf_token,

                  }

  

-                 output = self.app.post(

-                     '/admin/test_election2/', data=data, follow_redirects=True)

+                 with mock_sends(EditElectionV1):

+                     output = self.app.post(

+                         '/admin/test_election2/', data=data, follow_redirects=True)

                  self.assertEqual(output.status_code, 200)

                  output_text = output.get_data(as_text=True)

                  self.assertTrue(
@@ -619,8 +627,9 @@ 

                      'csrf_token': csrf_token,

                  }

  

-                 output = self.app.post(

-                     '/admin/test_election2/', data=data, follow_redirects=True)

+                 with mock_sends(EditElectionV1):

+                     output = self.app.post(

+                         '/admin/test_election2/', data=data, follow_redirects=True)

                  self.assertEqual(output.status_code, 200)

                  output_text = output.get_data(as_text=True)

                  self.assertTrue(
@@ -705,8 +714,9 @@ 

                      'csrf_token': csrf_token,

                  }

  

-                 output = self.app.post(

-                     '/admin/test_election3/', data=data, follow_redirects=True)

+                 with mock_sends(EditElectionV1):

+                     output = self.app.post(

+                         '/admin/test_election3/', data=data, follow_redirects=True)

                  self.assertEqual(output.status_code, 200)

                  output_text = output.get_data(as_text=True)

                  self.assertTrue(
@@ -734,8 +744,9 @@ 

                      'csrf_token': csrf_token,

                  }

  

-                 output = self.app.post(

-                     '/admin/test_election3/', data=data, follow_redirects=True)

+                 with mock_sends(EditElectionV1):

+                     output = self.app.post(

+                         '/admin/test_election3/', data=data, follow_redirects=True)

                  self.assertEqual(output.status_code, 200)

                  output_text = output.get_data(as_text=True)

                  self.assertTrue(
@@ -829,9 +840,10 @@ 

                      'csrf_token': csrf_token,

                  }

  

-                 output = self.app.post(

-                     '/admin/test_election/candidates/new', data=data,

-                     follow_redirects=True)

+                 with mock_sends(NewCandidateV1):

+                     output = self.app.post(

+                         '/admin/test_election/candidates/new', data=data,

+                         follow_redirects=True)

                  self.assertEqual(output.status_code, 200)

                  output_text = output.get_data(as_text=True)

                  self.assertTrue(
@@ -919,9 +931,10 @@ 

                      'csrf_token': csrf_token,

                  }

  

-                 output = self.app.post(

-                     '/admin/test_election/candidates/new/multi', data=data,

-                     follow_redirects=True)

+                 with mock_sends(NewCandidateV1, NewCandidateV1, NewCandidateV1):

+                     output = self.app.post(

+                         '/admin/test_election/candidates/new/multi', data=data,

+                         follow_redirects=True)

                  self.assertEqual(output.status_code, 200)

                  output_text = output.get_data(as_text=True)

                  self.assertTrue(
@@ -1029,9 +1042,10 @@ 

                      'csrf_token': csrf_token,

                  }

  

-                 output = self.app.post(

-                     '/admin/test_election/candidates/1/edit', data=data,

-                     follow_redirects=True)

+                 with mock_sends(EditCandidateV1):

+                     output = self.app.post(

+                         '/admin/test_election/candidates/1/edit', data=data,

+                         follow_redirects=True)

                  self.assertEqual(output.status_code, 200)

                  output_text = output.get_data(as_text=True)

                  self.assertTrue(
@@ -1121,9 +1135,10 @@ 

                      'csrf_token': csrf_token,

                  }

  

-                 output = self.app.post(

-                     '/admin/test_election4/candidates/10/delete', data=data,

-                     follow_redirects=True)

+                 with mock_sends(DeleteCandidateV1):

+                     output = self.app.post(

+                         '/admin/test_election4/candidates/10/delete', data=data,

+                         follow_redirects=True)

                  self.assertEqual(output.status_code, 200)

                  output_text = output.get_data(as_text=True)

                  self.assertTrue(

file modified
+1 -1
@@ -1,5 +1,5 @@ 

  [tox]

- envlist = py27,diff-cover

+ envlist = py36,py37,py38,diff-cover

  skipsdist = True

  

  [testenv]

no initial comment

@pingou I combined the merge and release in one PR in the interest of time, hope that's OK. If you want me to split them just shout.

rebased onto 0811869

3 years ago

@scoady, @pingou and I (well pingou did the work, I just merged a few PRs) cleaned up the repos, updated tests, and the readme. Can you rebase?

It'd probably be easier to split the merge and release separately (and go from develop->master->staging->productions)

Ideally, I'd like to get a fix for #73 in before we do another release, despite the fact that I've nearly entirely ignored this repo for the last year :-(

The original idea for staging and production was to rebase them on the top of develop or master when we want to update the version running in openshift (ie: no merge commits)

@pingou yeah - I noticed when I made this patch that the process you just outlined hadn't really been followed so the branches were out of step. What I can do then is bring develop and master in line with each other and then also rebase staging. I'll create a separate PR for each and the only one I'll bump the version on is prod.

@bcotton no problem, it's up to you the order and if you want to wait, but just FYI elections isn't fully working in prod at the moment as you can't log in without this fix.

I'll close this and create all the PRs now to do the above anyway.

Actually, I just read the readme properly and it says to use force pushing so someone with access will need to do that - this was the complete wrong approach to getting a release. Develop is up to date so a merge on master and then a force push to staging and then prod in turn is whats needed. Closing this now.

Pull-Request has been closed by scoady

3 years ago