Needs figuring out by someone who knows the system, but something we should do

I have full access to backups, and know Weblate hosting=2E
There is a passphrase to decrypt the backup and a private ssh key availabl=
e=2E
What do you need?

So, just skimming the docs... we should probibly store a copy of the passphrase and private/public ssh key for the backups in our private repo.

If we want to store a copy of the data it sounds like we would have to setup a borg repository and ask it to sync the backup there?

Is there some way to just download the existing weblate backups from the cloud? Or manually download one and save it somewhere we control?

Thanks for any info.

FYI: Kevin was made super-admin of Weblate to see the available info

Considering Kevin's busy schedule for the coming weeks/months, I can try to help.

@jibecfed should we try to catch up and discuss what we can do and how?

thank you for followin this, if you want, you can find me on fedora-i18n on IRC
please login on https://translate.fedoraproject.org so I can make you admin, to allow you to see borg information

there isn't much to do:
keep one backup per year for long term analysis (we only need the database for this)
keep last two months backup for operational safety (we need all files, including huge git repositories)

@jibecfed I've logged in, I'll try to catch you sometime this week to discuss what we can do :)

I gave Pingou admin rights to access security info (passphrase, private key)

------------------------------------------------------------------------------
Archive name: 2020-05-28T04:00:00
Archive fingerprint: b25253817f5aef083af74d49a4363f02f8f236836f0aaf751c2972c1ba976249
Time (start): Thu, 2020-05-28 04:00:10
Time (end):   Thu, 2020-05-28 04:01:39
Duration: 1 minutes 29.48 seconds
Number of files: 220586
Utilization of max. archive size: 0%
------------------------------------------------------------------------------
                       Original size      Compressed size    Deduplicated size
This archive:                4.59 GB              2.17 GB            215.13 MB
All archives:               66.36 GB             32.19 GB              5.96 GB

                       Unique chunks         Total chunks
Chunk index:                  638878              3131182
------------------------------------------------------------------------------

Here is how I downloaded the latest archive: BORG_RSH="ssh -i ~/.ssh/id_rsa_translate_fpo" borg extract ssh://u164666-sub8@u164666-sub8.your-storagebox.de:23/./backups::2020-05-28T04:00:00

It creates an home folder: with the following size:

[jb@localhost hop]$ du --max-depth=1 -h
5,1G    ./home
5,1G    .

Here is the detail:

[jb@localhost hop]$ du --max-depth=1 -h home/weblate/data/
4,0K    home/weblate/data/cache
2,7G    home/weblate/data/vcs
849M    home/weblate/data/whoosh
332M    home/weblate/data/memory
120K    home/weblate/data/celery
1,3G    home/weblate/data/backups
12K home/weblate/data/fonts
28K home/weblate/data/ssh
37M home/weblate/data/media
32K home/weblate/data/home
5,1G    home/weblate/data/

I don't think we can try to cherrypick what we want to backup, as biggest folders are vcs and backups, which contains git repositories and the database, which is the core of the application and represents 4 GB already.

You may expect all this to grow a lot over time (mostly because of the localization of fedora docs).

@pingou did you do anything here?

Do we want to try and automate this? or just manually grab it once per month? It can be placed on backup01 in /fedora_backups/

@kevin I have the access and some level of understanding on how it works, but I need to sync with you on where to run and place the files.

@smooge you've added the dev tag to this one, what do you think we have to develop for this ticket?

Well, we need a script/cronjob/something if we want to do this in an automated way.