PR#6969: Add script to automate torrent generation - releng

releng

#6969 Add script to automate torrent generation

Closed 2 years ago by humaton. Opened 6 years ago by yahzaa.

yahzaa/releng master into master

Add script to automate torrent generation

Chandrakant Kumar • 6 years ago

8f3f7a3

scripts/create-torrent.sh

file added

+60

		`@@ -0,0 +1,60 @@`
		`+ #!/bin/bash`
		`+`
		`+ # Get torrent.fedoraproject.org username`
		`+ read -p 'torrent.fedoraproject.org username: ' USERNAME`
		`+`
		`+ ORG = "Fedora"`
		`+ BASE_DIR = "/srv/torrent/new"`
		`+ TORRENT_DIR = "$BASE_DIR/$ORG"`
		`+ ARCHIVE_NAME = "archived.gzip"`
		`+ HOSTNAME = 'torrent.fedoraproject.org'`
		`+`
		`+ # Archive and upload the files`
ausil commented 6 years ago we just need to rsync the content, no tar or compresseion
		`+`
		`+ tar cvzf $ARCHIVE_NAME $@`
		`+ sha1sum $ARCHIVE_NAME > $ARCHIVE_NAME.sha1`
		`+ scp $ARCHIVE_NAME $ARCHIVE_NAME.sha1 $USERNAME@$HOSTNAME:$TORRENT_DIR`
		`+`
		`+ ssh $USERNAME@$HOSTNAME << EOF`
		`+`
		`+ if [ ! -d $TORRENT_DIR ]; then`
		`+ mkdir -p $TORRENT_DIR`
		`+ fi`
		`+`
		`+ cd $TORRENT_DIR && tar -xvzf $ARCHIVE_NAME && rm $ARCHIVE_NAME`
ausil commented 6 years ago This is not needed
		`+`
		`+ rsync -avhHP rsync://download-ib01.fedoraproject.org/fedora-stage/24_RC-1.2///iso/ /srv/torrent/new/fedora/`
ausil commented 6 years ago the compose "24_RC-1.2" needs to be a variable that ou pass in as an argument to the command, it changes for every release
		`+`
		`+ for iso in $(ls iso);`
kellin commented 6 years ago 'ls' should not be used to generate arrays for loops. Swap this to 'find'.
		`+ do`
		`+ dest=$(echo $iso\|sed -e 's\|-1.2.iso\|\|g' );`
ausil commented 6 years ago you need to calculate the 1.2 i the substitution as it changes every release
		`+ mkdir $dest;`
		`+ mv $iso $dest;`
		`+ done`
		`+`
		`+ for checksum in $(ls CHECKSUM)`
		`+ do`
		`+ for file in $(grep "SHA256 (" $checksum \|sed -e 's\|SHA256 (\|\|g' -e 's\|-1.2.\|\|g' -e 's\|-2..\|\|g' )`
ausil commented 6 years ago the values here need to be determined each release
		`+ do`
		`+ cp $checksum $file`
		`+ done`
		`+ done`
		`+`
		`+ sha256sum -c SHA256SUMS 2>&1 \| grep OK`
		`+`
		`+ if [ ! $? -eq "0" ];`
		`+ echo "sha1sum verification failed"`
		`+ exit`
		`+ fi`
		`+`
		`+ /srv/torrent/new/maketorrent .`
		`+ wget https://pagure.io/releng/raw/master/f/scripts/fedora-torrent-ini.py`
		`+ chmod u+x fedora-torrent-ini.py`
		`+ ./fedora-torrent-ini.py > $ORG.ini`
		`+ if [ ! -d /srv/torrent/www/torrents/$ORG ]; then`
		`+ mkdir -p /srv/torrent/www/torrents/$ORG`
		`+ fi`
		`+ cp *.torrent /srv/torrent/www/torrents/$ORG/`
		`+ cd /srv/torrent/torrent-generator/ && mv $TORRENT_DIR/$ORG.ini .`
		`+ mv $TORRENT_DIR /srv/torrent/btholding/`
		`+ EOF`

yahzaa commented 6 years ago

https://pagure.io/releng/issue/5223

ausil commented on line 16 of scripts/create-torrent.sh 6 years ago

we currently run a rsync from the torrent box to get the isos and checksums in place
rsync -avhHP rsync://download-ib01.fedoraproject.org/fedora-stage/24_RC-1.2///iso/ /srv/torrent/new/fedora/

ausil commented on line 19 of scripts/create-torrent.sh 6 years ago

all checksum files are anmed ending in CHECKSUM and are sha256 hashed

ausil commented on line 27 of scripts/create-torrent.sh 6 years ago

We have been running two loops to put everything in place
for iso in $(ls iso); do dest=$(echo $iso|sed -e 's|-1.2.iso||g' ); mkdir $dest; mv $iso $dest; done
for checksum in $(ls CHECKSUM); do for file in $(grep "SHA256 (" $checksum |sed -e 's|SHA256 (||g' -e 's|-1.2.||g' -e 's|-2..||g' ); do cp $checksum $file ; done; done