PR#239: use python-versioneer for managing project version - fedora-qa/blockerbugs

fedora-qa / blockerbugs

#239 use python-versioneer for managing project version

Merged 2 years ago by kparal. Opened 2 years ago by kparal.

feature/versioneer into develop

use python-versioneer for managing project version

Kamil Páral • 2 years ago

d2b6a0b

.gitattributes

file added

		`@@ -0,0 +1,1 @@`
		`+ blockerbugs/_version.py export-subst`

MANIFEST.in

file modified

		`@@ -1,2 +1,4 @@`
		`recursive-include blockerbugs/templates *`
		`recursive-include blockerbugs/static *`
		`+ include versioneer.py`
		`+ include blockerbugs/_version.py`

Makefile

file modified

		`@@ -90,11 +90,15 @@`
		`rm -f pep8.out`
		`rm -f pylint.out`

		+ # FIXME: Rewrite this to avoid parsing the spec file, e.g. by using `git describe`.
		+ # Alternatively, don't use `git archive` but use `setup.py sdist` instead.
		`.PHONY: archive`
		`archive: $(SRC)-$(VERSION).tar.gz`

		`.PHONY: $(SRC)-$(VERSION).tar.gz`
		`$(SRC)-$(VERSION).tar.gz:`
		`+ if [ -z "$(shell git tag --points-at $(GITBRANCH))" ]; then echo \`
		`+ "WARNING: Creating archive from an untagged commit, the app version will be 'unknown'"; fi`
		`git archive $(GITBRANCH) --prefix=$(SRC)-$(VERSION)/ \| gzip -c9 > $@`
		`mkdir -p build/$(VERSION)-$(RELEASE)`
		`mv $(SRC)-$(VERSION).tar.gz build/$(VERSION)-$(RELEASE)/`

README.md

file modified

+2 -6

		`@@ -17,12 +17,8 @@`
		`## Documentation`

		To see a complete documentation of this project, look into `docs/source/`
		`- directory. You can also run this command in the project root directory:`
		- ```
		`- make docs`
		- ```
		`-`
		- And then you'll find the rendered docs in `docs/_build/html/index.html`.
		`+ directory. If you want to see rendered pages, build them according to`
		+ instructions in `development.rst` in that directory.

		`## Deployment`

blockerbugs/__init__.py

file modified

+11 -2

		`@@ -10,9 +10,13 @@`
		`import sqlalchemy`

		`from . import config`
		`+ from . import _version`

		`- # the version as used in setup.py and docs`
		`- __version__ = "1.5"`
		`+ __version__ = _version.get_versions()['version']`
		`+ if __version__ == '0+unknown':`
		`+ # something is wrong, probably running from a tarball created from an untagged release`
		`+ # we'll print an error a bit later, once logging is set up`
		`+ __version__ += '.{}'.format(_version.get_versions()['full-revisionid'])`

		`# Flask App`
		`app: Flask = Flask(__name__)`
		`@@ -105,6 +109,11 @@`

		`setup_logging()`

		`+ # version check`
		`+ if _version.get_versions()['error']:`
		`+ app.logger.warn('Could not reliably figure out app version, the error was: {}'.format(`
		`+ _version.get_versions()['error']))`
		`+`
		`# database`
		`if app.config['SHOW_DB_URI']:`
		`app.logger.debug('using DBURI: %s' % app.config['SQLALCHEMY_DATABASE_URI'])`

blockerbugs/_version.py

file added

+644

		`@@ -0,0 +1,644 @@`
		`+`
		`+ # This file helps to compute a version number in source trees obtained from`
		`+ # git-archive tarball (such as those provided by githubs download-from-tag`
		`+ # feature). Distribution tarballs (built by setup.py sdist) and build`
		`+ # directories (produced by setup.py build) will contain a much shorter file`
		`+ # that just contains the computed version number.`
		`+`
		`+ # This file is released into the public domain. Generated by`
		`+ # versioneer-0.21 (https://github.com/python-versioneer/python-versioneer)`
		`+`
		`+ """Git implementation of _version.py."""`
		`+`
		`+ import errno`
		`+ import os`
		`+ import re`
		`+ import subprocess`
		`+ import sys`
		`+ from typing import Callable, Dict`
		`+`
		`+`
		`+ def get_keywords():`
		`+ """Get the keywords needed to look up the version information."""`
		`+ # these strings will be replaced by git during git-archive.`
		`+ # setup.py/versioneer.py will grep for the variable names, so they must`
		`+ # each be defined on a line of their own. _version.py will just call`
		`+ # get_keywords().`
		`+ git_refnames = "$Format:%d$"`
		`+ git_full = "$Format:%H$"`
		`+ git_date = "$Format:%ci$"`
		`+ keywords = {"refnames": git_refnames, "full": git_full, "date": git_date}`
		`+ return keywords`
		`+`
		`+`
		`+ class VersioneerConfig:`
		`+ """Container for Versioneer configuration parameters."""`
		`+`
		`+`
		`+ def get_config():`
		`+ """Create, populate and return the VersioneerConfig() object."""`
		`+ # these strings are filled in when 'setup.py versioneer' creates`
		`+ # _version.py`
		`+ cfg = VersioneerConfig()`
		`+ cfg.VCS = "git"`
		`+ cfg.style = "pep440"`
		`+ cfg.tag_prefix = ""`
		`+ cfg.parentdir_prefix = "blockerbugs-"`
		`+ cfg.versionfile_source = "blockerbugs/_version.py"`
		`+ cfg.verbose = False`
		`+ return cfg`
		`+`
		`+`
		`+ class NotThisMethod(Exception):`
		`+ """Exception raised if a method is not valid for the current scenario."""`
		`+`
		`+`
		`+ LONG_VERSION_PY: Dict[str, str] = {}`
		`+ HANDLERS: Dict[str, Dict[str, Callable]] = {}`
		`+`
		`+`
		`+ def register_vcs_handler(vcs, method): # decorator`
		`+ """Create decorator to mark a method as the handler of a VCS."""`
		`+ def decorate(f):`
		`+ """Store f in HANDLERS[vcs][method]."""`
		`+ if vcs not in HANDLERS:`
		`+ HANDLERS[vcs] = {}`
		`+ HANDLERS[vcs][method] = f`
		`+ return f`
		`+ return decorate`
		`+`
		`+`
		`+ def run_command(commands, args, cwd=None, verbose=False, hide_stderr=False,`
		`+ env=None):`
		`+ """Call the given command(s)."""`
		`+ assert isinstance(commands, list)`
		`+ process = None`
		`+ for command in commands:`
		`+ try:`
		`+ dispcmd = str([command] + args)`
		`+ # remember shell=False, so use git.cmd on windows, not just git`
		`+ process = subprocess.Popen([command] + args, cwd=cwd, env=env,`
		`+ stdout=subprocess.PIPE,`
		`+ stderr=(subprocess.PIPE if hide_stderr`
		`+ else None))`
		`+ break`
		`+ except OSError:`
		`+ e = sys.exc_info()[1]`
		`+ if e.errno == errno.ENOENT:`
		`+ continue`
		`+ if verbose:`
		`+ print("unable to run %s" % dispcmd)`
		`+ print(e)`
		`+ return None, None`
		`+ else:`
		`+ if verbose:`
		`+ print("unable to find command, tried %s" % (commands,))`
		`+ return None, None`
		`+ stdout = process.communicate()[0].strip().decode()`
		`+ if process.returncode != 0:`
		`+ if verbose:`
		`+ print("unable to run %s (error)" % dispcmd)`
		`+ print("stdout was %s" % stdout)`
		`+ return None, process.returncode`
		`+ return stdout, process.returncode`
		`+`
		`+`
		`+ def versions_from_parentdir(parentdir_prefix, root, verbose):`
		`+ """Try to determine the version from the parent directory name.`
		`+`
		`+ Source tarballs conventionally unpack into a directory that includes both`
		`+ the project name and a version string. We will also support searching up`
		`+ two directory levels for an appropriately named parent directory`
		`+ """`
		`+ rootdirs = []`
		`+`
		`+ for _ in range(3):`
		`+ dirname = os.path.basename(root)`
		`+ if dirname.startswith(parentdir_prefix):`
		`+ return {"version": dirname[len(parentdir_prefix):],`
		`+ "full-revisionid": None,`
		`+ "dirty": False, "error": None, "date": None}`
		`+ rootdirs.append(root)`
		`+ root = os.path.dirname(root) # up a level`
		`+`
		`+ if verbose:`
		`+ print("Tried directories %s but none started with prefix %s" %`
		`+ (str(rootdirs), parentdir_prefix))`
		`+ raise NotThisMethod("rootdir doesn't start with parentdir_prefix")`
		`+`
		`+`
		`+ @register_vcs_handler("git", "get_keywords")`
		`+ def git_get_keywords(versionfile_abs):`
		`+ """Extract version information from the given file."""`
		`+ # the code embedded in _version.py can just fetch the value of these`
		`+ # keywords. When used from setup.py, we don't want to import _version.py,`
		`+ # so we do it with a regexp instead. This function is not used from`
		`+ # _version.py.`
		`+ keywords = {}`
		`+ try:`
		`+ with open(versionfile_abs, "r") as fobj:`
		`+ for line in fobj:`
		`+ if line.strip().startswith("git_refnames ="):`
		`+ mo = re.search(r'=\s"(.)"', line)`
		`+ if mo:`
		`+ keywords["refnames"] = mo.group(1)`
		`+ if line.strip().startswith("git_full ="):`
		`+ mo = re.search(r'=\s"(.)"', line)`
		`+ if mo:`
		`+ keywords["full"] = mo.group(1)`
		`+ if line.strip().startswith("git_date ="):`
		`+ mo = re.search(r'=\s"(.)"', line)`
		`+ if mo:`
		`+ keywords["date"] = mo.group(1)`
		`+ except OSError:`
		`+ pass`
		`+ return keywords`
		`+`
		`+`
		`+ @register_vcs_handler("git", "keywords")`
		`+ def git_versions_from_keywords(keywords, tag_prefix, verbose):`
		`+ """Get version information from git keywords."""`
		`+ if "refnames" not in keywords:`
		`+ raise NotThisMethod("Short version file found")`
		`+ date = keywords.get("date")`
		`+ if date is not None:`
		`+ # Use only the last line. Previous lines may contain GPG signature`
		`+ # information.`
		`+ date = date.splitlines()[-1]`
		`+`
		`+ # git-2.2.0 added "%cI", which expands to an ISO-8601 -compliant`
		`+ # datestamp. However we prefer "%ci" (which expands to an "ISO-8601`
		`+ # -like" string, which we must then edit to make compliant), because`
		`+ # it's been around since git-1.5.3, and it's too difficult to`
		`+ # discover which version we're using, or to work around using an`
		`+ # older one.`
		`+ date = date.strip().replace(" ", "T", 1).replace(" ", "", 1)`
		`+ refnames = keywords["refnames"].strip()`
		`+ if refnames.startswith("$Format"):`
		`+ if verbose:`
		`+ print("keywords are unexpanded, not using")`
		`+ raise NotThisMethod("unexpanded keywords, not a git-archive tarball")`
		`+ refs = {r.strip() for r in refnames.strip("()").split(",")}`
		`+ # starting in git-1.8.3, tags are listed as "tag: foo-1.0" instead of`
		`+ # just "foo-1.0". If we see a "tag: " prefix, prefer those.`
		`+ TAG = "tag: "`
		`+ tags = {r[len(TAG):] for r in refs if r.startswith(TAG)}`
		`+ if not tags:`
		`+ # Either we're using git < 1.8.3, or there really are no tags. We use`
		`+ # a heuristic: assume all version tags have a digit. The old git %d`
		`+ # expansion behaves like git log --decorate=short and strips out the`
		`+ # refs/heads/ and refs/tags/ prefixes that would let us distinguish`
		`+ # between branches and tags. By ignoring refnames without digits, we`
		`+ # filter out many common branch names like "release" and`
		`+ # "stabilization", as well as "HEAD" and "master".`
		`+ tags = {r for r in refs if re.search(r'\d', r)}`
		`+ if verbose:`
		`+ print("discarding '%s', no digits" % ",".join(refs - tags))`
		`+ if verbose:`
		`+ print("likely tags: %s" % ",".join(sorted(tags)))`
		`+ for ref in sorted(tags):`
		`+ # sorting will prefer e.g. "2.0" over "2.0rc1"`
		`+ if ref.startswith(tag_prefix):`
		`+ r = ref[len(tag_prefix):]`
		`+ # Filter out refs that exactly match prefix or that don't start`
		`+ # with a number once the prefix is stripped (mostly a concern`
		`+ # when prefix is '')`
		`+ if not re.match(r'\d', r):`
		`+ continue`
		`+ if verbose:`
		`+ print("picking %s" % r)`
		`+ return {"version": r,`
		`+ "full-revisionid": keywords["full"].strip(),`
		`+ "dirty": False, "error": None,`
		`+ "date": date}`
		`+ # no suitable tags, so version is "0+unknown", but full hex is still there`
		`+ if verbose:`
		`+ print("no suitable tags, using unknown + full revision id")`
		`+ return {"version": "0+unknown",`
		`+ "full-revisionid": keywords["full"].strip(),`
		`+ "dirty": False, "error": "no suitable tags", "date": None}`
		`+`
		`+`
		`+ @register_vcs_handler("git", "pieces_from_vcs")`
		`+ def git_pieces_from_vcs(tag_prefix, root, verbose, runner=run_command):`
		`+ """Get version from 'git describe' in the root of the source tree.`
		`+`
		`+ This only gets called if the git-archive 'subst' keywords were not`
		`+ expanded, and _version.py hasn't already been rewritten with a short`
		`+ version string, meaning we're inside a checked out source tree.`
		`+ """`
		`+ GITS = ["git"]`
		`+ TAG_PREFIX_REGEX = "*"`
		`+ if sys.platform == "win32":`
		`+ GITS = ["git.cmd", "git.exe"]`
		`+ TAG_PREFIX_REGEX = r"\*"`
		`+`
		`+ _, rc = runner(GITS, ["rev-parse", "--git-dir"], cwd=root,`
		`+ hide_stderr=True)`
		`+ if rc != 0:`
		`+ if verbose:`
		`+ print("Directory %s not under git control" % root)`
		`+ raise NotThisMethod("'git rev-parse --git-dir' returned error")`
		`+`
		`+ # if there is a tag matching tag_prefix, this yields TAG-NUM-gHEX[-dirty]`
		`+ # if there isn't one, this yields HEX[-dirty] (no NUM)`
		`+ describe_out, rc = runner(GITS, ["describe", "--tags", "--dirty",`
		`+ "--always", "--long",`
		`+ "--match",`
		`+ "%s%s" % (tag_prefix, TAG_PREFIX_REGEX)],`
		`+ cwd=root)`
		`+ # --long was added in git-1.5.5`
		`+ if describe_out is None:`
		`+ raise NotThisMethod("'git describe' failed")`
		`+ describe_out = describe_out.strip()`
		`+ full_out, rc = runner(GITS, ["rev-parse", "HEAD"], cwd=root)`
		`+ if full_out is None:`
		`+ raise NotThisMethod("'git rev-parse' failed")`
		`+ full_out = full_out.strip()`
		`+`
		`+ pieces = {}`
		`+ pieces["long"] = full_out`
		`+ pieces["short"] = full_out[:7] # maybe improved later`
		`+ pieces["error"] = None`
		`+`
		`+ branch_name, rc = runner(GITS, ["rev-parse", "--abbrev-ref", "HEAD"],`
		`+ cwd=root)`
		`+ # --abbrev-ref was added in git-1.6.3`
		`+ if rc != 0 or branch_name is None:`
		`+ raise NotThisMethod("'git rev-parse --abbrev-ref' returned error")`
		`+ branch_name = branch_name.strip()`
		`+`
		`+ if branch_name == "HEAD":`
		`+ # If we aren't exactly on a branch, pick a branch which represents`
		`+ # the current commit. If all else fails, we are on a branchless`
		`+ # commit.`
		`+ branches, rc = runner(GITS, ["branch", "--contains"], cwd=root)`
		`+ # --contains was added in git-1.5.4`
		`+ if rc != 0 or branches is None:`
		`+ raise NotThisMethod("'git branch --contains' returned error")`
		`+ branches = branches.split("\n")`
		`+`
		`+ # Remove the first line if we're running detached`
		`+ if "(" in branches[0]:`
		`+ branches.pop(0)`
		`+`
		`+ # Strip off the leading "* " from the list of branches.`
		`+ branches = [branch[2:] for branch in branches]`
		`+ if "master" in branches:`
		`+ branch_name = "master"`
		`+ elif not branches:`
		`+ branch_name = None`
		`+ else:`
		`+ # Pick the first branch that is returned. Good or bad.`
		`+ branch_name = branches[0]`
		`+`
		`+ pieces["branch"] = branch_name`
		`+`
		`+ # parse describe_out. It will be like TAG-NUM-gHEX[-dirty] or HEX[-dirty]`
		`+ # TAG might have hyphens.`
		`+ git_describe = describe_out`
		`+`
		`+ # look for -dirty suffix`
		`+ dirty = git_describe.endswith("-dirty")`
		`+ pieces["dirty"] = dirty`
		`+ if dirty:`
		`+ git_describe = git_describe[:git_describe.rindex("-dirty")]`
		`+`
		`+ # now we have TAG-NUM-gHEX or HEX`
		`+`
		`+ if "-" in git_describe:`
		`+ # TAG-NUM-gHEX`
		`+ mo = re.search(r'^(.+)-(\d+)-g([0-9a-f]+)$', git_describe)`
		`+ if not mo:`
		`+ # unparsable. Maybe git-describe is misbehaving?`
		`+ pieces["error"] = ("unable to parse git-describe output: '%s'"`
		`+ % describe_out)`
		`+ return pieces`
		`+`
		`+ # tag`
		`+ full_tag = mo.group(1)`
		`+ if not full_tag.startswith(tag_prefix):`
		`+ if verbose:`
		`+ fmt = "tag '%s' doesn't start with prefix '%s'"`
		`+ print(fmt % (full_tag, tag_prefix))`
		`+ pieces["error"] = ("tag '%s' doesn't start with prefix '%s'"`
		`+ % (full_tag, tag_prefix))`
		`+ return pieces`
		`+ pieces["closest-tag"] = full_tag[len(tag_prefix):]`
		`+`
		`+ # distance: number of commits since tag`
		`+ pieces["distance"] = int(mo.group(2))`
		`+`
		`+ # commit: short hex revision ID`
		`+ pieces["short"] = mo.group(3)`
		`+`
		`+ else:`
		`+ # HEX: no tags`
		`+ pieces["closest-tag"] = None`
		`+ count_out, rc = runner(GITS, ["rev-list", "HEAD", "--count"], cwd=root)`
		`+ pieces["distance"] = int(count_out) # total number of commits`
		`+`
		`+ # commit date: see ISO-8601 comment in git_versions_from_keywords()`
		`+ date = runner(GITS, ["show", "-s", "--format=%ci", "HEAD"], cwd=root)[0].strip()`
		`+ # Use only the last line. Previous lines may contain GPG signature`
		`+ # information.`
		`+ date = date.splitlines()[-1]`
		`+ pieces["date"] = date.strip().replace(" ", "T", 1).replace(" ", "", 1)`
		`+`
		`+ return pieces`
		`+`
		`+`
		`+ def plus_or_dot(pieces):`
		`+ """Return a + if we don't already have one, else return a ."""`
		`+ if "+" in pieces.get("closest-tag", ""):`
		`+ return "."`
		`+ return "+"`
		`+`
		`+`
		`+ def render_pep440(pieces):`
		`+ """Build up version string, with post-release "local version identifier".`
		`+`
		`+ Our goal: TAG[+DISTANCE.gHEX[.dirty]] . Note that if you`
		`+ get a tagged build and then dirty it, you'll get TAG+0.gHEX.dirty`
		`+`
		`+ Exceptions:`
		`+ 1: no tags. git_describe was just HEX. 0+untagged.DISTANCE.gHEX[.dirty]`
		`+ """`
		`+ if pieces["closest-tag"]:`
		`+ rendered = pieces["closest-tag"]`
		`+ if pieces["distance"] or pieces["dirty"]:`
		`+ rendered += plus_or_dot(pieces)`
		`+ rendered += "%d.g%s" % (pieces["distance"], pieces["short"])`
		`+ if pieces["dirty"]:`
		`+ rendered += ".dirty"`
		`+ else:`
		`+ # exception #1`
		`+ rendered = "0+untagged.%d.g%s" % (pieces["distance"],`
		`+ pieces["short"])`
		`+ if pieces["dirty"]:`
		`+ rendered += ".dirty"`
		`+ return rendered`
		`+`
		`+`
		`+ def render_pep440_branch(pieces):`
		`+ """TAG[[.dev0]+DISTANCE.gHEX[.dirty]] .`
		`+`
		`+ The ".dev0" means not master branch. Note that .dev0 sorts backwards`
		`+ (a feature branch will appear "older" than the master branch).`
		`+`
		`+ Exceptions:`
		`+ 1: no tags. 0[.dev0]+untagged.DISTANCE.gHEX[.dirty]`
		`+ """`
		`+ if pieces["closest-tag"]:`
		`+ rendered = pieces["closest-tag"]`
		`+ if pieces["distance"] or pieces["dirty"]:`
		`+ if pieces["branch"] != "master":`
		`+ rendered += ".dev0"`
		`+ rendered += plus_or_dot(pieces)`
		`+ rendered += "%d.g%s" % (pieces["distance"], pieces["short"])`
		`+ if pieces["dirty"]:`
		`+ rendered += ".dirty"`
		`+ else:`
		`+ # exception #1`
		`+ rendered = "0"`
		`+ if pieces["branch"] != "master":`
		`+ rendered += ".dev0"`
		`+ rendered += "+untagged.%d.g%s" % (pieces["distance"],`
		`+ pieces["short"])`
		`+ if pieces["dirty"]:`
		`+ rendered += ".dirty"`
		`+ return rendered`
		`+`
		`+`
		`+ def pep440_split_post(ver):`
		`+ """Split pep440 version string at the post-release segment.`
		`+`
		`+ Returns the release segments before the post-release and the`
		`+ post-release version number (or -1 if no post-release segment is present).`
		`+ """`
		`+ vc = str.split(ver, ".post")`
		`+ return vc[0], int(vc[1] or 0) if len(vc) == 2 else None`
		`+`
		`+`
		`+ def render_pep440_pre(pieces):`
		`+ """TAG[.postN.devDISTANCE] -- No -dirty.`
		`+`
		`+ Exceptions:`
		`+ 1: no tags. 0.post0.devDISTANCE`
		`+ """`
		`+ if pieces["closest-tag"]:`
		`+ if pieces["distance"]:`
		`+ # update the post release segment`
		`+ tag_version, post_version = pep440_split_post(pieces["closest-tag"])`
		`+ rendered = tag_version`
		`+ if post_version is not None:`
		`+ rendered += ".post%d.dev%d" % (post_version+1, pieces["distance"])`
		`+ else:`
		`+ rendered += ".post0.dev%d" % (pieces["distance"])`
		`+ else:`
		`+ # no commits, use the tag as the version`
		`+ rendered = pieces["closest-tag"]`
		`+ else:`
		`+ # exception #1`
		`+ rendered = "0.post0.dev%d" % pieces["distance"]`
		`+ return rendered`
		`+`
		`+`
		`+ def render_pep440_post(pieces):`
		`+ """TAG[.postDISTANCE[.dev0]+gHEX] .`
		`+`
		`+ The ".dev0" means dirty. Note that .dev0 sorts backwards`
		`+ (a dirty tree will appear "older" than the corresponding clean one),`
		`+ but you shouldn't be releasing software with -dirty anyways.`
		`+`
		`+ Exceptions:`
		`+ 1: no tags. 0.postDISTANCE[.dev0]`
		`+ """`
		`+ if pieces["closest-tag"]:`
		`+ rendered = pieces["closest-tag"]`
		`+ if pieces["distance"] or pieces["dirty"]:`
		`+ rendered += ".post%d" % pieces["distance"]`
		`+ if pieces["dirty"]:`
		`+ rendered += ".dev0"`
		`+ rendered += plus_or_dot(pieces)`
		`+ rendered += "g%s" % pieces["short"]`
		`+ else:`
		`+ # exception #1`
		`+ rendered = "0.post%d" % pieces["distance"]`
		`+ if pieces["dirty"]:`
		`+ rendered += ".dev0"`
		`+ rendered += "+g%s" % pieces["short"]`
		`+ return rendered`
		`+`
		`+`
		`+ def render_pep440_post_branch(pieces):`
		`+ """TAG[.postDISTANCE[.dev0]+gHEX[.dirty]] .`
		`+`
		`+ The ".dev0" means not master branch.`
		`+`
		`+ Exceptions:`
		`+ 1: no tags. 0.postDISTANCE[.dev0]+gHEX[.dirty]`
		`+ """`
		`+ if pieces["closest-tag"]:`
		`+ rendered = pieces["closest-tag"]`
		`+ if pieces["distance"] or pieces["dirty"]:`
		`+ rendered += ".post%d" % pieces["distance"]`
		`+ if pieces["branch"] != "master":`
		`+ rendered += ".dev0"`
		`+ rendered += plus_or_dot(pieces)`
		`+ rendered += "g%s" % pieces["short"]`
		`+ if pieces["dirty"]:`
		`+ rendered += ".dirty"`
		`+ else:`
		`+ # exception #1`
		`+ rendered = "0.post%d" % pieces["distance"]`
		`+ if pieces["branch"] != "master":`
		`+ rendered += ".dev0"`
		`+ rendered += "+g%s" % pieces["short"]`
		`+ if pieces["dirty"]:`
		`+ rendered += ".dirty"`
		`+ return rendered`
		`+`
		`+`
		`+ def render_pep440_old(pieces):`
		`+ """TAG[.postDISTANCE[.dev0]] .`
		`+`
		`+ The ".dev0" means dirty.`
		`+`
		`+ Exceptions:`
		`+ 1: no tags. 0.postDISTANCE[.dev0]`
		`+ """`
		`+ if pieces["closest-tag"]:`
		`+ rendered = pieces["closest-tag"]`
		`+ if pieces["distance"] or pieces["dirty"]:`
		`+ rendered += ".post%d" % pieces["distance"]`
		`+ if pieces["dirty"]:`
		`+ rendered += ".dev0"`
		`+ else:`
		`+ # exception #1`
		`+ rendered = "0.post%d" % pieces["distance"]`
		`+ if pieces["dirty"]:`
		`+ rendered += ".dev0"`
		`+ return rendered`
		`+`
		`+`
		`+ def render_git_describe(pieces):`
		`+ """TAG[-DISTANCE-gHEX][-dirty].`
		`+`
		`+ Like 'git describe --tags --dirty --always'.`
		`+`
		`+ Exceptions:`
		`+ 1: no tags. HEX[-dirty] (note: no 'g' prefix)`
		`+ """`
		`+ if pieces["closest-tag"]:`
		`+ rendered = pieces["closest-tag"]`
		`+ if pieces["distance"]:`
		`+ rendered += "-%d-g%s" % (pieces["distance"], pieces["short"])`
		`+ else:`
		`+ # exception #1`
		`+ rendered = pieces["short"]`
		`+ if pieces["dirty"]:`
		`+ rendered += "-dirty"`
		`+ return rendered`
		`+`
		`+`
		`+ def render_git_describe_long(pieces):`
		`+ """TAG-DISTANCE-gHEX[-dirty].`
		`+`
		`+ Like 'git describe --tags --dirty --always -long'.`
		`+ The distance/hash is unconditional.`
		`+`
		`+ Exceptions:`
		`+ 1: no tags. HEX[-dirty] (note: no 'g' prefix)`
		`+ """`
		`+ if pieces["closest-tag"]:`
		`+ rendered = pieces["closest-tag"]`
		`+ rendered += "-%d-g%s" % (pieces["distance"], pieces["short"])`
		`+ else:`
		`+ # exception #1`
		`+ rendered = pieces["short"]`
		`+ if pieces["dirty"]:`
		`+ rendered += "-dirty"`
		`+ return rendered`
		`+`
		`+`
		`+ def render(pieces, style):`
		`+ """Render the given version pieces into the requested style."""`
		`+ if pieces["error"]:`
		`+ return {"version": "unknown",`
		`+ "full-revisionid": pieces.get("long"),`
		`+ "dirty": None,`
		`+ "error": pieces["error"],`
		`+ "date": None}`
		`+`
		`+ if not style or style == "default":`
		`+ style = "pep440" # the default`
		`+`
		`+ if style == "pep440":`
		`+ rendered = render_pep440(pieces)`
		`+ elif style == "pep440-branch":`
		`+ rendered = render_pep440_branch(pieces)`
		`+ elif style == "pep440-pre":`
		`+ rendered = render_pep440_pre(pieces)`
		`+ elif style == "pep440-post":`
		`+ rendered = render_pep440_post(pieces)`
		`+ elif style == "pep440-post-branch":`
		`+ rendered = render_pep440_post_branch(pieces)`
		`+ elif style == "pep440-old":`
		`+ rendered = render_pep440_old(pieces)`
		`+ elif style == "git-describe":`
		`+ rendered = render_git_describe(pieces)`
		`+ elif style == "git-describe-long":`
		`+ rendered = render_git_describe_long(pieces)`
		`+ else:`
		`+ raise ValueError("unknown style '%s'" % style)`
		`+`
		`+ return {"version": rendered, "full-revisionid": pieces["long"],`
		`+ "dirty": pieces["dirty"], "error": None,`
		`+ "date": pieces.get("date")}`
		`+`
		`+`
		`+ def get_versions():`
		`+ """Get version information or return default if unable to do so."""`
		`+ # I am in _version.py, which lives at ROOT/VERSIONFILE_SOURCE. If we have`
		`+ # __file__, we can work backwards from there to the root. Some`
		`+ # py2exe/bbfreeze/non-CPython implementations don't do __file__, in which`
		`+ # case we can only use expanded keywords.`
		`+`
		`+ cfg = get_config()`
		`+ verbose = cfg.verbose`
		`+`
		`+ try:`
		`+ return git_versions_from_keywords(get_keywords(), cfg.tag_prefix,`
		`+ verbose)`
		`+ except NotThisMethod:`
		`+ pass`
		`+`
		`+ try:`
		`+ root = os.path.realpath(__file__)`
		`+ # versionfile_source is the relative path from the top of the source`
		`+ # tree (where the .git directory might live) to this file. Invert`
		`+ # this to find the root from __file__.`
		`+ for _ in cfg.versionfile_source.split('/'):`
		`+ root = os.path.dirname(root)`
		`+ except NameError:`
		`+ return {"version": "0+unknown", "full-revisionid": None,`
		`+ "dirty": None,`
		`+ "error": "unable to find root of source tree",`
		`+ "date": None}`
		`+`
		`+ try:`
		`+ pieces = git_pieces_from_vcs(cfg.tag_prefix, root, verbose)`
		`+ return render(pieces, cfg.style)`
		`+ except NotThisMethod:`
		`+ pass`
		`+`
		`+ try:`
		`+ if cfg.parentdir_prefix:`
		`+ return versions_from_parentdir(cfg.parentdir_prefix, root, verbose)`
		`+ except NotThisMethod:`
		`+ pass`
		`+`
		`+ return {"version": "0+unknown", "full-revisionid": None,`
		`+ "dirty": None,`
		`+ "error": "unable to compute version", "date": None}`

blockerbugs/controllers/main.py

file modified

+2 -1

		`@@ -27,7 +27,7 @@`
		`import itertools`

		`from blockerbugs import app, __version__`
		`- from blockerbugs.util import bz_interface, pagure_bot`
		`+ from blockerbugs.util import bz_interface, pagure_bot, misc`
		`from blockerbugs.models.bug import Bug`
		`from blockerbugs.models.milestone import Milestone`
		`from blockerbugs.models.update import Update`
		`@@ -41,6 +41,7 @@`
		`def before_request():`
		`g.milestones = Milestone.query.filter_by(active=True).order_by(Milestone.name).all()`
		`g.version = __version__`
		`+ g.version_date = misc.version_date()`


		`def get_recent_modifications(milestoneid):`

blockerbugs/templates/layout.html

file modified

+1 -1

		`@@ -142,7 +142,7 @@`
		`{% endif %}`
		`<br /> This application is open source! Visit its`
		`<a class="text-white-50" href="https://pagure.io/fedora-qa/blockerbugs">project page</a>.`
		`- <br /> Version {{ g.version }}`
		`+ <br /> Version {{ g.version }} ({{ g.version_date }})`
		`<div class="fedora-footer"></div>`
		`</span>`
		`</div>`

blockerbugs/util/misc.py

file added

+17

		`@@ -0,0 +1,17 @@`
		`+ """Miscellaneous helper functions"""`
		`+`
		`+ from typing import Optional`
		`+`
		`+ from blockerbugs import _version`
		`+`
		`+`
		`+ def version_date() -> Optional[str]:`
		`+ """Return the date (just the date portion) of the app version identifier, as returned by`
		+ versioneer. The output format is `YYYY-MM-DD`. If the date isn't known, return `None`.
		`+ """`
		`+ date = _version.get_versions()['date']`
		`+ if not date:`
		`+ return None`
		`+ parts = date.split(sep='T')`
		`+ assert len(parts) == 2`
		`+ return parts[0]`

docs/source/conf.py

file modified

+8 -5

		`@@ -11,7 +11,6 @@`
		`# All configuration values have a default; values that are commented out`
		`# serve to show the default.`

		`- import sys, os`

		`# If extensions (or modules to document with autodoc) are in another directory,`
		`# add these directories to sys.path here. If the directory is relative to the`
		`@@ -40,17 +39,21 @@`
		`master_doc = 'index'`

		`# General information about the project.`
		`- project = u'Fedora Blocker Bug Tracker'`
		`- copyright = u'2021, Fedora QA'`
		`+ project = 'Fedora BlockerBugs App'`
		`+ copyright = '2022, Fedora QA'`

		`# The version info for the project you're documenting, acts as replacement for`
		`# \|version\| and \|release\|, also used in various other places throughout the`
		`# built documents.`
		`#`
		`# The short X.Y version.`
		`- version = '1.5'`
		`+ # version =`
		`# The full version, including alpha/beta/rc tags.`
		`- release = '1.5'`
		`+ # release =`
		`+`
		`+ # ## Load the version through python-versioneer ##`
		`+ import blockerbugs # NOQA: E402`
		`+ version = release = blockerbugs.__version__`

		`# The language for content autogenerated by Sphinx. Refer to documentation`
		`# for a list of supported languages.`

docs/source/development.rst

file modified

+10

		`@@ -200,6 +200,16 @@`
		`mypy`


		`+ Building documentation`
		`+ ======================`
		`+`
		`+ While the virtualenv is active, you can build all documentation with this command::`
		`+`
		`+ make docs`
		`+`
		+ You can then inspect the resulting docs in ``docs/_build/html``. Not everything is built as HTML, so check ``docs/source`` as well.
		`+`
		`+`
		`More info on development`
		`========================`

docs/source/development_tasks.rst

file modified

+17 -1

		`@@ -124,7 +124,7 @@`
		match and are correct. ``blockerbugs.spec`` should have a useful statement
		`in the changelog corresponding to the changes in this latest release`

		`- * Create a git tag on master`
		`+ * Create an annotated git tag on master`

		`* Push the changes in master and the new tag to origin`

		`@@ -279,3 +279,19 @@`
		`time a change is detected::`

		`compass watch`
		`+`
		`+`
		`+ Updating python-versioneer`
		`+ ==========================`
		`+`
		+ BlockerBugs uses `python-versioneer`_ to automatically discover program version based on tags in git. This tool is statically installed as ``versioneer.py`` in root, uses configuration in ``setup.cfg`` and creates ``blockerbugs/_version.py`` as part of its install process.
		`+`
		+ If you wish to update the included versioneer copy, simply follow the `project instructions`__, i.e.::
		`+`
		`+ pip install -U versioneer`
		`+ # check if setup.cfg needs updating`
		`+ versioneer install`
		`+`
		`+`
		`+ .. _python-versioneer: https://github.com/python-versioneer/python-versioneer`
		`+ .. __: https://github.com/python-versioneer/python-versioneer#updating-versioneer`

requirements-development.txt

file modified

		`@@ -9,3 +9,6 @@`
		`munch-stubs`
		`types-requests`
		`types-mock`
		`+`
		`+ ## documentation`
		`+ sphinx`

setup.cfg

file modified

		`@@ -47,3 +47,11 @@`

		`[coverage:run]`
		`source = blockerbugs`
		`+`
		`+ [versioneer]`
		`+ VCS = git`
		`+ style = pep440`
		`+ versionfile_source = blockerbugs/_version.py`
		`+ versionfile_build = blockerbugs/_version.py`
		`+ tag_prefix =`
		`+ parentdir_prefix = blockerbugs-`

setup.py

file modified

+4 -13

		`@@ -1,11 +1,11 @@`
		`"""Install the application"""`

		`import codecs`
		`- import re`
		`import os`
		`-`
		`from setuptools import setup, Command # type: ignore`

		`+ import versioneer`
		`+`
		`here = os.path.abspath(os.path.dirname(__file__))`


		`@@ -25,17 +25,8 @@`
		`return codecs.open(os.path.join(here, *parts), 'r').read()`


		`- def find_version(*file_paths):`
		`- version_file = read(*file_paths)`
		`- version_match = re.search(r"^__version__ = ['\"]([^'\"]*)['\"]",`
		`- version_file, re.M)`
		`- if version_match:`
		`- return version_match.group(1)`
		`- raise RuntimeError("Unable to find version string.")`
		`-`
		`-`
		`setup(name='blockerbugs',`
		`- version=find_version('blockerbugs', '__init__.py'),`
		`+ version=versioneer.get_version(),`
		`description='Web application for tracking blocker and nth bugs in Fedora releases',`
		`author='Tim Flink',`
		`author_email='tflink@fedoraproject.org',`
		`@@ -48,7 +39,7 @@`
		`package_dir={'blockerbugs': 'blockerbugs'},`
		`entry_points=dict(console_scripts=['blockerbugs=blockerbugs.cli:main']),`
		`include_package_data=True,`
		`- cmdclass={'test': PyTest},`
		`+ cmdclass=versioneer.get_cmdclass({'test': PyTest}),`
		`install_requires=[`
		`'alembic',`
		`'Flask-Admin',`

versioneer.py

file added

+2109

The added file is too large to be shown here, see it at: versioneer.py

kparal commented 2 years ago

Instead of defining the app version statically, let python-versioneer discover it
automatically for us from git. It uses a similar version format as git describe,
i.e. the latest tag + number of additional commits + shorthash.

Versioneer is installed statically, so that it also works if executed outside of
git (in that case, depending on how it was created, it might only display a git
hash without additional attributes). During development and with proper tagged
releases, or when installed directly from the git repo (as in OpenShift), it
should display the version correctly and also the commit date as a bonus, in the
web footer.

When building docs, it is now recommended to install sphinx into virtualenv (or
not use venv at all), otherwise the version can't be determined.

One unresolved issue is that specfile versioning is not integrated, i.e. it still
needs to be updated manually. The Makefile support is also poor. I suppose we'll
want to get rid of the spec file and possibly large portion of the Makefile soon
anyway, and that's why I haven't invested time to fix it at this moment. Since
we don't use it now, it shouldn't be much of a problem.

Fixes: https://pagure.io/fedora-qa/blockerbugs/issue/236

zuul commented 2 years ago

Build succeeded.

blockerbugs-pytest : SUCCESS in 1m 53s

frantisekz commented 2 years ago

One unresolved issue is that specfile versioning is not integrated, i.e. it still
needs to be updated manually. The Makefile support is also poor. I suppose we'll
want to get rid of the spec file and possibly large portion of the Makefile soon
anyway, and that's why I haven't invested time to fix it at this moment. Since
we don't use it now, it shouldn't be much of a problem.

Yeah, both spec and makefile could be dropped basically right now, we have no intention of getting this packaged into Fedora, we're deploying it without using rpm build in the process at all, it'll just rot there forever. I'll create a PR for that!

Just thinking if it's really necessary to add a ~3k LoC for something that can be achieved by

echo `git show -s --format=%ct | (echo -n @ && cat) | date +'%Y-%m-%d' -f  -`".git"`git rev-parse --short HEAD`

I am not saying I am strictly against it, just wondering if something like the bash one-liner above wouldn't be a better solution... eg. do we care about executing it outside of git? In the deployment pipeline, it'd always be executed from a git tree, same for development.

So that could be output of os.system (or, more specifically something that allows you to capture stdout) added to setup.py and/or init if you don't run setup.py for development. There is probably a better solution that I just don't see on Friday afternoon where it could be placed.

PS. I can try to make PoC PR for this proposed solution.

Edited 2 years ago by frantisekz

adamwill commented 2 years ago

FWIW, in my projects I use setuptools-scm and a simple bash script for doing releases which looks like this:

#!/bin/bash

baddeps=""
# check deps
python3 -m build.__init__ || baddeps="python3-build"
if [ -n "${baddeps}" ]; then
    echo "${baddeps} must be installed!"
    exit 1
fi

if [ "$#" != "1" ]; then
    echo "Must pass release version!"
    exit 1
fi

version=$1
name=fedfind
sed -i -e "s,version=\".*\",version=\"${version}\", g" setup.py
sed -i -e "s,__version__ = \".*\",__version__ = \"${version}\", g" src/${name}/__init__.py
git add setup.py src/${name}/__init__.py
git commit -s -m "Release ${version}"
git push
git tag -a -m "Release ${version}" ${version}
git push origin ${version}
python3 -m build .
twine upload -r pypi dist/${name}-${version}*

That uses build for the tarball and wheel generation, which follows the relatively-new PEP517 spec. It requires having these bits in pyproject.toml too:

[build-system]
requires = ["setuptools>=40.6.0", "setuptools-scm", "wheel"]
build-backend = "setuptools.build_meta"

I find Hynek Schlawack is a good reference for this kind of Python packaging/metadata stuff - he has an article that he keeps updated quite regularly for the latest new stuff (like PEP 517 and 518 and so on). The approach he takes in that article (and his real-world projects like attrs) is to set the version 'canonically' in __init__.py and have setup.py find it from there - I don't know how he handles git tags exactly.

Some of his other articles are helpful too, like this one on testing with tox and this one on the issue of metadata duplication in general.

kparal commented 2 years ago

I am not saying I am strictly against it, just wondering if something like the bash one-liner above wouldn't be a better solution...

I inspected the following two projects in close detail:
https://github.com/python-versioneer/python-versioneer
https://github.com/jbweston/miniver

Miniver is much shorter version (200 lines), but it has fewer features (for example, you can't determine the release/commit date, just the git describe-like identifier), and currently they implemented some questionable approaches which in our case make the version identifier quite confusing (some tags are not detected, depending how we merged branches, and the number of commits above the tag also doesn't match).

eg. do we care about executing it outside of git?

That's one of the reasons why I went with an existing project instead of re-inventing the wheel. If I make a tarball (a git archive, python setup.py sdist, etc), I wanted the project to show the version properly (as long as you made the tarball from a tagged commit).

Also, I think even on our OpenShift, you do install the project, don't you? I.e. you run python setup.py install somewhere (or perhaps the automagic process does that for you). Meaning you're not running directly from git. And in that case, your "bash one-liners" won't work, and you need to make sure the version gets exported during project installation. And that's exactly what those projects take care of (among other things).

I was also a bit hesitant to hardcode this to a single use case, because you never know when an alternative approach might get handy (e.g. during debugging, if we want to upload it to pypi, etc). Having the install approach working is a strong must, I think, and the tarball approach is just a safety measure, in case we need to change deployment quickly in the future. In that case, at least basics would still be working, instead of reworking a lot of source code in a hurry.

So yes, I considered using short snippets (usually not one liners :-)) found on stackoverflow, but it seemed easier to statically install a script which can do more corner case handling than our short snippet, and the length of the script doesn't really bother me that much (I'd prefer miniver, if it was a bit better, but the versioneer script is still quite readable if needed, I looked at the source code quite extensively when playing with it).

Last week, I also had a quick look at:
https://github.com/pypa/setuptools_scm/
but it seemed too complex and requiring more changes to blockerbugs. But I'll look at it again. Thanks @adamwill for a detailed info.

kparal commented 2 years ago

I spent considerable time with setuptools_scm. I have to say its documentation is probably decent for people who already know a lot in the area of python packaging, but very bad for someone who doesn't know much about these things (that's me). Some of the articles that @adamwill shared are super interesting, but they make me think that I'd need to study much more about python packaging and we'd need to redo a lot of our setup.py/setup.cfg/requirements.txt etc configuration, in order to be able to use it properly.

After hopping over a number of obstacles, I made it work at least in some basic sense, but I have the following issues with it:
a) I can't make it work dynamically. It seems to create some manifests in blockerbugs.egg-info/ at install time, and then read it from there, when I import it at runtime (using the recommended approach of from importlib.metadata import version). So if I add some new commits, it doesn't show them, unless I reinstall the project. I don't know what I'm missing, but currently that's useless. However, when I run python -m setuptools_scm on the command line, it does compute the version dynamically correctly!
b) I tried to do from setuptools_scm import get_version instead (which doesn't seem to be the proper way to do it, but still), and that computes the version properly, I think (at the cost of a runtime dependency), but completely ignores the content the configuration provided in pyproject.toml! The command line execution correctly honors it.
c) It is not possible to show a commit date, unless your checkout is dirty (so in real deployment, never). Versioneer can do it, and I find it useful.

So overall I'm disappointed with it at the moment. I also looked at fedfind how Adam does it, and he hardcodes the version at two places (setup.py and __init__.py), which is not the way I want to handle things. I don't want the version number to be written anywhere (which is what versioneer allows). So I'm not even sure if setuptools_scm supports this at all?

It seems I'd need to invest several more days to study python packaging in detail, and play with setuptools_scm more to figure out if it can actually do dynamic versioning in runtime, and I'm not sure if this is a good time to make that time investment, honestly. If somebody else can make it work, I'll be happy to look at it.

adamwill commented 2 years ago

Ah, yeah, if that's what you want, I'm not sure of the best way to achieve it. I haven't tried that myself for anything yet.

frantisekz commented 2 years ago

Also, I think even on our OpenShift, you do install the project, don't you? I.e. you run python setup.py install somewhere (or perhaps the automagic process does that for you). Meaning you're not running directly from git. And in that case, your "bash one-liners" won't work, and you need to make sure the version gets exported during project installation.

Yes, you do install the project, from a git tree. I think we should define a scope here somehow, because, in order to just fix the #236 , you can just take the output from that bash script and bake it into the layout.html during setup.py. De we care about anything else here?

I'll explore this a bit and make en example PR how that could work, the idea is that it'd be the above mentioned bash line and one string replace on the footer section of layout.html during setup.py .

Edited 2 years ago by frantisekz

kparal commented 2 years ago

I think we should define a scope here somehow

My intention is:

Stop hardcoding project version in all config files and derive the version dynamically. This removes manual work and also allows us to move to continuous deployment e.g. on staging.
Show the version in a git describe style or similar, because it's nice and readable.
If possible, also allow the web UI to show the commit date, because it again improves the experience and saves work (figuring out the commit date from its hash).
Make this system work:
a) when run from a git checkout, i.e. in development setup
b) when installed (setup.py install or possibly from pypi in some future)
c) when downloaded as a git archive and executed from there (which is a common way to getting snapshots from git forges like pagure/github/etc).
I find it important to cover all these approaches, because that gives a consistent approach which makes the project more reliable (less special-casing), and better tested (some code paths are not executed e.g. just during deployment).
Make sure the system also works when building documentation.

I know it's more than it was described in #236, but this PR was not meant as a direct fix just for that. Rather, it moves the project in the way I'd like it to move, which also fixes #236 :-)

I'm fine with experimenting with alternative approaches, or waiting for Frantisek's "one liners" :-) But I don't honestly prefer to reinvent the wheel. The changes are very minimal in this PR, it only seems large because there's versioneer.py statically dumped in the project root, and _version.py generated alongside it. But that's not our code, and it's basically not different from any other library that we already import in requirements.txt (it might even be possible to make versioneer a runtime dependency and not dump it in our project, but it's not the recommended approach, so I haven't even tried). I prefer to use a library which is already maintained by someone instead of maintaining the code (and updating it with future python and git changes) ourselves. Sure, I could've used a locally patched miniver code, which would be smaller than versioneer, but I don't see any benefits in that. I'd have to maintain it myself.

Rather than discussing whether to include that many LoC, I'd rather hear whether the goals of this PR are reasonable, or whether I omitted something important. Are there are concerns except the apparent extra LoC?

I'll explore this a bit and make en example PR how that could work

Knock yourself out :-) I think you'll end up writing a new version of miniver, though.

kparal commented 2 years ago

Commit 416a500 fixes this pull-request

Pull-Request has been merged by kparal

2 years ago

kparal commented 2 years ago

I decided to merge this, so that it doesn't block future work. If somebody comes with an alternative solution, I'm happy to look at it. In the meantime, let's try if versioneer works for us.