The test started failing around the time python-dns update was available: python-dns-2.4.2-2.fc39
https://bodhi.fedoraproject.org/updates/FEDORA-2024-391ed3a61d

I will try to run the tests with/without the update in order to confirm.

Same issue in test_backup_and_restore_TestBackupAndRestoreWithDNSSEC and test_backup_and_restore_TestBackupReinstallRestoreWithDNSSEC: after the zone is created, it is not signed.

Confirmed that the problem is in python3-dns:

Using this python script similar to the method used in the test to check if a zone is signed:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52

#!/usr/bin/python3
import dns.name
import os
import sys
from ipapython.dnsutil import DNSResolver

def resolve_with_dnssec(nameserver, query, rtype="SOA"):
    res = DNSResolver()
    res.nameservers = [nameserver]
    res.lifetime = 10  # wait max 10 seconds for reply

    # enable Authenticated Data + Checking Disabled flags
    res.set_flags(dns.flags.AD | dns.flags.CD)

    # enable EDNS v0 + enable DNSSEC-Ok flag
    res.use_edns(0, dns.flags.DO, 0)

    ans = res.resolve(query, rtype)
    return ans

def get_RRSIG_record(nameserver, query, rtype="SOA"):
    ans = resolve_with_dnssec(nameserver, query, rtype=rtype)
    return ans.response.find_rrset(
        ans.response.answer, dns.name.from_text(query),
        dns.rdataclass.IN, dns.rdatatype.RRSIG,
        dns.rdatatype.from_text(rtype))

def is_record_signed(nameserver, query, rtype="SOA"):
    try:
        get_RRSIG_record(nameserver, query, rtype=rtype)
    except KeyError:
        print("KeyError")
        return False
    except dns.exception.DNSException as e:
        print("DNSException {}".format(e))
        return False
    return True


if len(sys.argv) < 3:
    print("Usage: {} resolver-ip-addr zone".format(
        os.path.basename(sys.argv[0])))
    sys.exit(1)

resolver = sys.argv[1]
zone = sys.argv[2]

print("Using resolver {} to query zone {}".format(resolver, zone))
if is_record_signed(resolver, zone, rtype='SOA'):
    print("Signed")
else:
    print("Not Signed")

If the script is called with python3-dns-2.4.2-1:

[root@server]# python3 resolverec.py 10.0.184.28 dnssec.test
Using resolver 10.0.184.28 to query zone dnssec.test
Signed
[root@server]# rpm -qa python3-dns
python3-dns-2.4.2-1.fc39.noarch

Now if the package is updated to python3-dns-2.4.2-2:

[root@server]# dnf update -y --enablerepo=updates-testing python3-dns
[...]
Upgrading:
 python3-dns       noarch       2.4.2-2.fc39        updates-testing       586 k
[...]
Complete!
[root@server]# python3 resolverec.py 10.0.184.28 dnssec.test
Using resolver 10.0.184.28 to query zone dnssec.test
DNSException The resolution lifetime expired after 10.110 seconds: Server Do53:10.0.184.28@53 answered The DNS operation timed out.
Not Signed
[root@server]# rpm -qa python3-dns
python3-dns-2.4.2-2.fc39.noarch
[root@server]# 

I'm not sure why it causes trouble here but the only thing in the update is a backported patch to fix one CVE, see https://src.fedoraproject.org/rpms/python-dns/c/5e8cb9a90477341f636acf4d598327b7f5a3a7b2?branch=f39

Could you please help me to debug the problem? The CVE fix is waiting.

@lbalhar in order to reproduce on a fedora 39 machine (replace 10.11.5.160 with your lab DNS server):

# hostnamectl set-hostname server.ipa.test
# IPADDR=`hostname -I | cut -d' ' -f1`
# echo "$IPADDR server.ipa.test" >> /etc/hosts

# dnf update -y
# dnf install -y freeipa-server-dns

# ipa-server-install --domain ipa.test --realm IPA.TEST --setup-dns --forwarder 10.11.5.160 -a Secret123 -p Secret123 -U
# echo Secret123 | kinit admin
# ipa-dns-install --dnssec-master  --forwarder 10.11.5.160 -U
# ipa dnszone-add dnssec.test. --skip-overlap-check --dnssec true --ttl 1 --default-ttl 1

At this point if you run the python script provided in my earlier comment, the output should look like the following:

# python3 /tmp/resolverec.py $IPADDR dnssec.test.
Using resolver 10.0.185.196 to query zone dnssec.test.
Signed

Now upgrade python3-dns to the version in updates-testing, an re-run the same python script:

# dnf update -y --enablerepo=updates-testing python3-dns
# python3 /tmp/resolverec.py $IPADDR dnssec.test.
Using resolver 10.0.185.196 to query zone dnssec.test.
DNSException The resolution lifetime expired after 10.108 seconds: Server Do53:10.0.185.196@53 answered The DNS operation timed out.
Not Signed

In order to debug you can use tcpdump and wireshark and compare the DNS requests generated with the 2 versions of python3-dns.

Thank you. This seems to be much more complex than I anticipated.

Could you please test the newer version of python-dns (2.6.1) from Fedora 40? IIRC it's not fully backward-compatible but it's installable on F39. If there is the same bug as in the version build for F39, the issue has to be reported upstream. If the problem disappears with the version 2.6.1 build for F40, the problem is in my patch.

Install command: dnf install -y https://kojipkgs.fedoraproject.org//packages/python-dns/2.6.1/1.fc40/noarch/python3-dns-2.6.1-1.fc40.noarch.rpm

Hi @lbalhar
with python3-dns-2.6.1-1.fc40 there is no issue.

New update rebasing python-dns to 2.6.1 in F39 is ready: https://bodhi.fedoraproject.org/updates/FEDORA-2024-3b4c7849ab

@lbalhar the update shows good results as the nightly don't fail any more (see PR #3764 with only known issues unrelated to dnssec). Closing this ticket