#12030 adjust torrent_hashes.py to non bittorrent world
Closed: Fixed 9 months ago by kevin. Opened 10 months ago by kevin.

When I moved torrent01 to rhel9, I was able to drop the old 'bittorrent' package. This is a thing that was free, then went closed source, so we had been running the last free version on rhel7 before.
I'd like to keep it gone. ;)

The torrent_hashes.py script is a script called from cron that extracts the hashes from all the .torrent files so we can add them to opentrackers allowlist.

Currently it tries to import Bittorrent and fails, so we need to adjust it to not use that.
I made a quick scratch build of python-fastbencode for epel9 and tried to use that, but it wasn't working right.

@jnsamyak was able to run it fine on fedora and we got the allowlist updated for now, but we need to fix that script up.

We can either figure out whats wrong with that package on epel9, fix that and ask it be branched for epel9, adjust the script to use it

or

come up with some other way to generate the hashes for the allowlist.


Totally, it doesn't make sense to carry on with BitTorrent due to its new policy in place, and on the way, we discovered that we can achieve the same functionality with python-fastbencode

On debugging & fixing things yesterday, the script below was invented/rewritten to work for us in Fedora Python virtual env, which I think works perfectly fine, since we tested it as well! The only problem is that the python-fastbencode rpm seems to have some issues for sure.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
#!/usr/bin/python
#
# This script extracts the hashes from all the .torrent 
# files so we can add them to the opentrackers allowlist.

import os
import sys
import hashlib
from optparse import OptionParser
from fastbencode import bencode, bdecode

def torrent_hash(fname):
    try:
        with open(fname, 'rb') as f:
            d = bdecode(f.read())
        hash_val = hashlib.sha1(bencode(d[b'info'])).hexdigest().upper()
        fn = os.path.basename(fname)
        return f"{hash_val} - {fn}"
    except FileNotFoundError:
        print(f"Error: File '{fname}' not found.", file=sys.stderr)
    except IsADirectoryError:
        print(f"Error: '{fname}' is a directory.", file=sys.stderr)
    except Exception as e:
        print(f"Error reading or decoding hash from {fname}: {e}", file=sys.stderr)
    return None

def main():
    parser = OptionParser(usage=sys.argv[0] + " [options] [torrentfiles] ...")
    parser.add_option("-o", "--output", type="string", metavar="FILE",
                      dest="output", default='-',
                      help="write hashes to FILE, default=stdout")
    (options, args) = parser.parse_args()

    outfd = sys.stdout
    if options.output != '-':
        try:
            outfd = open(options.output, 'w')
        except Exception as e:
            print(f"Error: unable to open output file {options.output}: {e}", file=sys.stderr)
            return 1

    for a in args:
        try:
            hash_val = torrent_hash(a)
            if hash_val:
                outfd.write(hash_val + '\n')
        except Exception as e:
            print(f"Error reading hash from {a}: {e}", file=sys.stderr)

    return 0

if __name__ == "__main__":
    sys.exit(main())

I think we should go with checking what's wrong with the epel9, I also checked other options last night that we could use but those RPMs don't exist in the fedora and hence it's of no use anyway, should we ping the maintainer once?

By the way, there were some other issues with migration as well, and hence we had to update the fedora-ini-torrent script as well, here's the PR for that: #12027

Metadata Update from @jnsamyak:
- Issue tagged with: high-gain, medium-trouble, ops

10 months ago

fedora-torrent-ini.py in ansible repo also needs a lot of help.

I ran 2to3 on it, but it then fails trying to sort releases. I just commented that out since the order is fine without sorting, but we need to fix this script in ansible.

It's the one that takes the torrent files and generates a .ini file for you...

Metadata Update from @jnsamyak:
- Issue assigned to jnsamyak

10 months ago

Oh sorry, I meant roles/torrent/files/torrent-generator/torrent-generator in the ansible repo. :( Sorry for confusion there...

I think we should close this as well, since we have the ansible PR merged as well for this: https://pagure.io/fedora-infra/ansible/pull-request/1935#? Or we have something else as well?

Metadata Update from @kevin:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

9 months ago

Log in to comment on this ticket.

Metadata
Boards 1
Ops Status: Backlog