Issue #119: find most recent images using filename convention - taskotron

taskotron

#119 find most recent images using filename convention

Closed: Fixed Opened Aug 19, 2015 by tflink.

While at Flock 2015, @mkrizek, @jskladan and I spent some time talking about how to go about finding specific images to use for disposable clients given a template name (minimal, server, workstation etc.). The best thing that we were able to come up with was to start using a filename convention where the build date, fedora release and template name were in the image's filename in a predictable way.

An example of this could be 20150818-f23-minimal.qcow2 which follows the format of <build date>-<fedora release>-<template name>.qcow2. While using filenames like this isn't perfect, it is a relatively easy solution that acheives most of our goals:

possible to download images in a separate process from launching
can sort by build date in filename to find the most recent image for the (release, template) tuple that we should have from the formula
easy to write cron jobs to remove all but the most recent X images to keep space consumption under control
easy to remove images from virthosts if something is broken in a new image, code will automagically fall back to the previous image

For this ticket, write code that looks in a directory (/var/lib/testcloud/cache is testcloud's default) for images and picks the newest one for a given (fedora release, image template) tuple

somebody commented Aug 19, 2015

This ticket is a duplicate of https://pagure.io/taskotron/libtaskotron/issue/263

somebody commented Aug 19, 2015

This ticket had assigned some Differential requests:
D618
D528

lbrabec commented Oct 12, 2015

As @kparal proposed in D528, "datestamp" can be too coarse, we can use iso8601:
<iso8601 datetime>-<release>-<template>.qcow2 like in 2015-10-08T11:42:22-f23-minimal.qcow2

This approach, together with @tflink's suggestion that the code should return testcloud.Image instance,
needs small adjustment in testcloud - the uri parsing in testcloud.Image has to support colons in filename/path.

Are we going to use the "datetimestamp" in iso8601 format?

tflink commented Oct 12, 2015

or we could use - (like 20151012-1, 20151012-2 and so on) and not require changes to the testcloud codebase.

@kparal, wouldn't doing this accomplish most of what you were looking to do with your comment in D528?

kparal commented Oct 13, 2015

! In #575#8213, @tflink wrote:
or we could use - (like 20151012-1, 20151012-2 and so on) and not require changes to the testcloud codebase.

@kparal, wouldn't doing this accomplish most of what you were looking to do with your comment in D528?

Yes. I don't mind either way. But the new filename should be determined automatically (so in case of versioning, it should find the latest version number and increase it by one), we should not require people to specify it every time. That is easier with timestamp and a bit harder with revision numbers, but of course the latter is easier to read.

Also, maybe we should use an underscore instead of a dash, so that we can easily split it? I.e. 20151012_1-f23-minimal.qcow2.

Is it better to skip the revision for the first image of the day, or is it not worth it?
20151012-f23-minimal.qcow2
20151012_2-f23-minimal.qcow2
20151012_3-f23-minimal.qcow2

tflink commented Oct 13, 2015

! In #575#8216, @kparal wrote:

! In #575#8213, @tflink wrote:
or we could use - (like 20151012-1, 20151012-2 and so on) and not require changes to the testcloud codebase.

@kparal, wouldn't doing this accomplish most of what you were looking to do with your comment in D528?

Yes. I don't mind either way. But the new filename should be determined automatically (so in case of versioning, it should find the latest version number and increase it by one), we should not require people to specify it every time. That is easier with timestamp and a bit harder with revision numbers, but of course the latter is easier to read.

Completely agreed, consistency is more important than any format we could choose.

Also, maybe we should use an underscore instead of a dash, so that we can easily split it? I.e. 20151012_1-f23-minimal.qcow2.

Is it better to skip the revision for the first image of the day, or is it not worth it?
20151012-f23-minimal.qcow2
20151012_2-f23-minimal.qcow2
20151012_3-f23-minimal.qcow2

Let's keep the same format for all the images and add the revision - easier to parse.

lbrabec commented Nov 3, 2015

Closing (D618).

Metadata Update from @tflink:
- Issue tagged with: infrastructure

Aug 4, 2017

Metadata

Assignee

None

Tags

Blocking

None

Depending on

None

Priority

Low

taskotron

#119 find most recent images using filename convention Closed: Fixed Opened Aug 19, 2015 by tflink.

Metadata

infra

#119 find most recent images using filename convention

Closed: Fixed Opened Aug 19, 2015 by tflink.