#3 [RFE] Mark relics for auto-submission
Opened 5 years ago by netvor. Modified 5 years ago

It would be nice to have some sort of implicit relic management system, so that people don't have to submit relics explicitly. Few ideas:

  • -e and -o options to jat__cmd() could cause the created files to be auto-submitted.

  • jat__pstart() or even jat__sinit() (via jattool, maybe via FMF parameters) could have a way to name out relics to be auto-collected during jat__pend() and jat__sinit().

  • jat__pstart() already has -C argument; this could be taken into consideration.

Some of the design challenges are:

  • Don't make it so complicated & messy that users will easily lose track of what will be submitted.

    If there's a relatively simple, cleanly written test, it should be obvious from peeking at the test how many and what kind of relics it will generate. IOW, the names of files should be visible in the code and seeing test code on one side and result on the other should make it clear which relic is from where.

  • Avoid submitting too much.

    Eg. the auto-submission via jat__pstart -C <DIR> would be issue if the <DIR> is eg. /usr.

  • Have an easy conflict avoidance mechanism.

    People should be motivated to keep tests simple, so it would be good if we had a way to avoid conflicts that would be easy enough so that they could grasp it and learn to rely on it.

    IOW, they should not have to name their relics with complicated names "some-file-with-$RANDOM-and-unique-name.err" just to avoid overwriting on submission.

    A possible style would be that every phase would have a dedicated auto-submission sub-directory named according to phase id and/or case id. That way user would only have to think of the naming within single phase, and even the final tree would have mostly deterministic structure.

    (This idea could be even extended to assert level, possibly to the point where user does not have to think of filenames at all, eg. an assert jat__cmd -c foo some_cmd called under phase that set bar caseid, would generate foo.err, foo.out and foo.es, and just automatically submit them under bar directory.)


Login to comment on this ticket.

Metadata