@mvadkert, @martinpitt, please have a look.

@psss: So the how: restraint and snapshots: true are the trick how to give a fresh testbed to each of the test/one, test/two, test/three? And dependencies.yml and selenium.yml would run once, then a snapshot is taken? That would be wonderful indeed!

@martinpitt Actually, how shouldn't be important for this to work, it could be any other method supported by the CI system (e.g. runners like sti, pytest or simple execute command), snapshots should do the trick "alone" - CI system should be smart enough to prepare fresh environment before startinghow for each test.

Yes, that's my idea / proposal how this could work. Separation of the testing stages could allow us to provide different / multiple ways how to implement each stage plus we could have generic attributes such as snapshot which would denote if the particular implementation should enable snapshoting functionality. So this would not be limited to restraint only. Another option would be to configure snaphost under provision as it is closely related to how the snapshoting would be done.

It fits more to running tests, IMHO, than to provision, since provision should be a one-time thing, cast to the stone, than runners take over.

And a bit of bike-shedding: how about something like isolate-tests, isolation: true or something similar? snapshot might be too OpenStack/OpenShift/container-ish, while user's wish "run my tests isolated" better eflects something with "isolate" substring :)

yep, I like isolation more :)

Isolation makes sense. What about isolate: true? Example updated.

Merging based on IRC ack from @happz and @mvadkert.