fedora-infrastructure

#11433 Hosting for Fedora OpenQA Dataset
Closed: Fixed with Explanation by kevin. Opened by tflink.

Closed: Fixed with Explanation
Reopen Issue

Describe what you would like us to do:

I've been working on an experiment involving data I extracted from Fedora's OpenQA instance and I will be presenting this experiment as part of a talk I'm giving at flock. I would like to make the data and code available to anyone who's interested and hosting it on a Fedora-associated domain makes sense to me if that's possible.

The problem is that the dataset is not small - the full dataset before any processing is 102 GB. I could post only the subsampled, processed data (778M) but I'd prefer to post the raw data if I can.

Is this something that could be reasonably done or should I focus on getting the dataset under 50GB so that I can post it to Zenodo?

When do you need this to be done by?

2023/08/02 would be ideal - I'm giving a talk at Flock about using this dataset and I'd like to be able to post links to the code and data

I underestimated xz's ability to compress. After 4 hours, it was able to reduce that 102GB dataset to 50GB if that makes a difference.

Sure. Options:

  1. /srv/web/infra/ on infrastructure.fedoraproject.org (ie, batcave01)

  2. /srv/pub/alt/ on secondary01, ie dl.fedoraproject.org/pub/alt/ (and it would get mirrored by mirrors that do alt)

How long do you need the data to be there? Forever? Some time after the talk?

And what would be good for ''?

I'm of two minds WRT data retention - the first is an academic saying "it should be archived" and the other is a realist which says "nobody is going to use this, it doesn't really matter". So, I guess at least until the end of the year and we can revisit if anyone actually downloads the data or does something with it.

somename: fedora-openqa-data

Bah, I didn't answer your first question. I don't think there is any reason to mirror the data - I really don't expect many people to be interested. /srv/web/infra will be fine, IMHO.

ok, made /srv/web/infra/fedora-openqa-data and it's owned by your user. :)

Metadata Update from @kevin:
- Issue close_status updated to: Fixed with Explanation
- Issue status updated to: Closed (was: Open)
Metadata
None
None
None
None
Lowest
Static archive. pagure.io was sunset in 2026; this is a read-only snapshot.
Red Hat Logo

Fedora is sponsored by Red Hat. Learn more about the relationship between Red Hat and Fedora »

© Red Hat, Inc. and others.