#6969 Add script to automate torrent generation
Opened 3 years ago by yahzaa. Modified 2 years ago
yahzaa/releng master  into  master

Add script to automate torrent generation
Chandrakant Kumar • 3 years ago  
@@ -0,0 +1,60 @@ 

+ #!/bin/bash

+ 

+ # Get torrent.fedoraproject.org username

+ read -p 'torrent.fedoraproject.org username: ' USERNAME

+ 

+ ORG = "Fedora"

+ BASE_DIR = "/srv/torrent/new"

+ TORRENT_DIR = "$BASE_DIR/$ORG"

+ ARCHIVE_NAME = "archived.gzip"

+ HOSTNAME = 'torrent.fedoraproject.org'

+ 

+ # Archive and upload the files
ausil commented 3 years ago

we just need to rsync the content, no tar or compresseion

+ 

+ tar cvzf $ARCHIVE_NAME $@

+ sha1sum $ARCHIVE_NAME > $ARCHIVE_NAME.sha1

+ scp $ARCHIVE_NAME $ARCHIVE_NAME.sha1 $USERNAME@$HOSTNAME:$TORRENT_DIR

+ 

+ ssh $USERNAME@$HOSTNAME << EOF

+    

+     if [ ! -d $TORRENT_DIR ]; then

+         mkdir -p $TORRENT_DIR

+     fi

+ 

+     cd $TORRENT_DIR && tar -xvzf $ARCHIVE_NAME && rm $ARCHIVE_NAME
ausil commented 3 years ago

This is not needed

+ 

+     rsync -avhHP rsync://download-ib01.fedoraproject.org/fedora-stage/24_RC-1.2///iso/ /srv/torrent/new/fedora/
ausil commented 3 years ago

the compose "24_RC-1.2" needs to be a variable that ou pass in as an argument to the command, it changes for every release

+ 

+     for iso in $(ls iso); 

'ls' should not be used to generate arrays for loops. Swap this to 'find'.

+     do 

+         dest=$(echo $iso|sed -e 's|-1.2.iso||g' ); 
ausil commented 3 years ago

you need to calculate the 1.2 i the substitution as it changes every release

+         mkdir $dest; 

+         mv $iso $dest; 

+     done

+ 

+     for checksum in $(ls CHECKSUM)

+     do 

+         for file in $(grep "SHA256 (" $checksum |sed -e 's|SHA256 (||g' -e 's|-1.2.||g' -e 's|-2..||g' )
ausil commented 3 years ago

the values here need to be determined each release

+         do 

+             cp $checksum $file

+         done

+     done

+ 

+     sha256sum -c SHA256SUMS 2>&1 | grep OK

+ 

+     if [ ! $? -eq "0" ];

+         echo "sha1sum verification failed"

+         exit

+     fi

+ 

+     /srv/torrent/new/maketorrent .

+     wget https://pagure.io/releng/raw/master/f/scripts/fedora-torrent-ini.py

+     chmod u+x fedora-torrent-ini.py

+     ./fedora-torrent-ini.py > $ORG.ini

+     if [ ! -d /srv/torrent/www/torrents/$ORG ]; then

+         mkdir -p /srv/torrent/www/torrents/$ORG

+     fi

+     cp *.torrent /srv/torrent/www/torrents/$ORG/

+     cd /srv/torrent/torrent-generator/ && mv $TORRENT_DIR/$ORG.ini .

+     mv $TORRENT_DIR /srv/torrent/btholding/

+ EOF

we currently run a rsync from the torrent box to get the isos and checksums in place
rsync -avhHP rsync://download-ib01.fedoraproject.org/fedora-stage/24_RC-1.2///iso/ /srv/torrent/new/fedora/

all checksum files are anmed ending in CHECKSUM and are sha256 hashed

We have been running two loops to put everything in place
for iso in $(ls iso); do dest=$(echo $iso|sed -e 's|-1.2.iso||g' ); mkdir $dest; mv $iso $dest; done
for checksum in $(ls
CHECKSUM); do for file in $(grep "SHA256 (" $checksum |sed -e 's|SHA256 (||g' -e 's|-1.2.||g' -e 's|-2..||g' ); do cp $checksum $file ; done; done

The issue - https://pagure.io/releng/issue/5223 explains torrent generation for only one iso.

sorry the process described there has not been updated. the rsync has only been used for the last few releases

rebased onto 8f3f7a3

3 years ago

we just need to rsync the content, no tar or compresseion

the compose "24_RC-1.2" needs to be a variable that ou pass in as an argument to the command, it changes for every release

you need to calculate the 1.2 i the substitution as it changes every release

the values here need to be determined each release

'ls' should not be used to generate arrays for loops. Swap this to 'find'.

@yahzaa care to address comments and rebase? Or shall we just close this?

Metadata