#3962 Failure in prod of the single-package packager workflow
Opened by packagerbot. Modified

A run of monitor-gating has just failed in prod for the single-package workflow.

The suspects are 'waiverdb, datagrepper, datagrepper, koji'.

Full log:

06:39:59 - Cloning as packagerbot the git repo: rpms/dummy-test-package-gloster
06:40:01 - Cloning as packagerbot the git repo: rpms/dummy-test-package-gloster                       [DONE]
06:40:01 - Switching to branch: rawhide
06:40:02 - Switching to branch: rawhide                                                               [DONE]
06:40:02 - Bumping release of: dummy-test-package-gloster.spec
06:40:02 - Bumping release of: dummy-test-package-gloster.spec                                        [DONE]
06:40:02 - Committing changes
06:40:02 - Committing changes                                                                         [DONE]
06:40:02 - Getting nevr
06:40:03 - Getting nevr                                                                               [DONE]
06:40:03 - Upcoming build : dummy-test-package-gloster-0-9232.fc37                                    [DONE]
06:40:03 - Pushing changes
06:40:10 - Pushing changes                                                                            [DONE]
06:40:10 - The build should be found on: https://koji.fedoraproject.org/koji/search?match=glob&type=build&terms=dummy-test-package-gloster-0-9232.fc37 [DONE]
06:40:10 - The build should be found in: https://koji.fedoraproject.org/koji/search?match=glob&type=package&terms=dummy-test-package-gloster [DONE]
06:40:10 - Building the package
06:47:14 - Building the package                                                                       [DONE]
06:47:14 - Retrieving koji tags
06:47:15 - Retrieving koji tags: ['f37-updates-candidate', 'f37-updates-testing-pending'] found while expected ['updates-candidate', 'signing-pending'] [DONE]
06:47:15 - Retrieving update created from https://bodhi.fedoraproject.org/updates/?builds=dummy-test-package-gloster-0-9232.fc37
06:47:15 - Retrieving update created from https://bodhi.fedoraproject.org/updates/?builds=dummy-test-package-gloster-0-9232.fc37 [DONE]
06:47:15 - Update automatically created : https://bodhi.fedoraproject.org/updates/FEDORA-2022-cb6a1fb8f5 [DONE]
06:47:15 - Retrieving koji tags
06:47:16 - Retrieving koji tags: ['f37-updates-candidate', 'f37-updates-testing-pending'] found while expected ['signing-pending', 'testing-pending'] [DONE]
06:47:16 - Checking datagrepper for bodhi to CI messages
06:47:16 - bodhi to CI results in datagrepper returned  - ran for: 0s                                 [DONE]
06:47:16 - Checking datagrepper for CI (running) messages
06:47:16 - CI (running) results in datagrepper returned running - ran for: 0s                         [DONE]
06:47:16 - Checking datagrepper for CI (complete) messages
06:49:48 - CI (complete) results in datagrepper returned needs_inspection - ran for: 151s             [DONE]
06:49:48 - Retrieving koji tags
06:49:48 - Retrieving koji tags: ['f37-updates-candidate', 'f37-updates-testing-pending'] found while expected ['testing-pending'] [DONE]
06:49:48 - CI results query from https://resultsdb.fedoraproject.org/api/v2.0/results&page=0          [DONE]
06:49:48 - Checking resultsdb(phx) for CI results
06:49:48 - CI results in resultsdb(phx) returned org.centos.prod.ci.koji-build.test.running - ran for: 0s [DONE]
06:49:48 - Checking datagrepper for resultsdb messages
06:49:48 - resultsdb results in datagrepper returned FAILED - ran for: 0s                             [DONE]
06:49:48 - Checking datagrepper for greenwave messages
06:52:50 - greenwave results in datagrepper returned False - ran for: 181s                            [DONE]
06:52:50 - Retrieving koji tags
06:52:51 - Retrieving koji tags: ['f37-updates-candidate', 'f37-updates-testing-pending'] found while expected ['testing-pending'] [DONE]
06:52:51 - Waiving test results for bodhi update
06:53:00 - Waiving test results for bodhi update                                                      [FAILED]
06:53:00 - Checking datagrepper for waiverdb messages
09:11:03 - No messages in data-grepper on https://apps.fedoraproject.org/datagrepper/raw?topic=org.fedoraproject.prod.waiverdb.waiver.new&start=1656484636.123927&row_per_page=10&page=1 (psycopg2.OperationalError) server closed the connection unexpectedly
    This probably means the server terminated abnormally
    before or while processing the request.
[SQL: SELECT count(*) AS count_1 
FROM (SELECT messages.id AS messages_id, messages.msg_id AS messages_msg_id, messages.i AS messages_i, messages.topic AS messages_topic, messages.timestamp AS messages_timestamp, messages.certificate AS messages_certificate, messages.signature AS messages_signature, messages.category AS messages_category, messages.username AS messages_username, messages.crypto AS messages_crypto, messages.source_name AS messages_source_name, messages.source_version AS messages_source_version, messages.msg AS messages_msg, messages.headers AS messages_headers 
FROM messages 
WHERE messages.timestamp BETWEEN %(timestamp_1)s AND %(timestamp_2)s AND messages.topic = %(topic_1)s) AS anon_1]
[parameters: {'timestamp_1': datetime.datetime(2022, 6, 29, 6, 37, 16, 123927), 'timestamp_2': datetime.datetime(2022, 6, 29, 9, 11, 3, 783888), 'topic_1': 'org.fedoraproject.prod.waiverdb.waiver.new'}]
(Background on this error at: https://sqlalche.me/e/14/e3q8)
09:11:33 - No messages in data-grepper on https://apps.fedoraproject.org/datagrepper/raw?topic=org.fedoraproject.prod.waiverdb.waiver.new&start=1656484636.123927&row_per_page=10&page=2 (psycopg2.OperationalError) server closed the connection unexpectedly
    This probably means the server terminated abnormally
    before or while processing the request.
[SQL: SELECT count(*) AS count_1 
FROM (SELECT messages.id AS messages_id, messages.msg_id AS messages_msg_id, messages.i AS messages_i, messages.topic AS messages_topic, messages.timestamp AS messages_timestamp, messages.certificate AS messages_certificate, messages.signature AS messages_signature, messages.category AS messages_category, messages.username AS messages_username, messages.crypto AS messages_crypto, messages.source_name AS messages_source_name, messages.source_version AS messages_source_version, messages.msg AS messages_msg, messages.headers AS messages_headers 
FROM messages 
WHERE messages.timestamp BETWEEN %(timestamp_1)s AND %(timestamp_2)s AND messages.topic = %(topic_1)s) AS anon_1]
[parameters: {'timestamp_1': datetime.datetime(2022, 6, 29, 6, 37, 16, 123927), 'timestamp_2': datetime.datetime(2022, 6, 29, 9, 11, 33, 877097), 'topic_1': 'org.fedoraproject.prod.waiverdb.waiver.new'}]
(Background on this error at: https://sqlalche.me/e/14/e3q8)
09:12:03 - No messages in data-grepper on https://apps.fedoraproject.org/datagrepper/raw?topic=org.fedoraproject.prod.waiverdb.waiver.new&start=1656484636.123927&row_per_page=10&page=1 (psycopg2.OperationalError) server closed the connection unexpectedly
    This probably means the server terminated abnormally
    before or while processing the request.
[SQL: SELECT count(*) AS count_1 
FROM (SELECT messages.id AS messages_id, messages.msg_id AS messages_msg_id, messages.i AS messages_i, messages.topic AS messages_topic, messages.timestamp AS messages_timestamp, messages.certificate AS messages_certificate, messages.signature AS messages_signature, messages.category AS messages_category, messages.username AS messages_username, messages.crypto AS messages_crypto, messages.source_name AS messages_source_name, messages.source_version AS messages_source_version, messages.msg AS messages_msg, messages.headers AS messages_headers 
FROM messages 
WHERE messages.timestamp BETWEEN %(timestamp_1)s AND %(timestamp_2)s AND messages.topic = %(topic_1)s) AS anon_1]
[parameters: {'timestamp_1': datetime.datetime(2022, 6, 29, 6, 37, 16, 123927), 'timestamp_2': datetime.datetime(2022, 6, 29, 9, 12, 3, 939163), 'topic_1': 'org.fedoraproject.prod.waiverdb.waiver.new'}]
(Background on this error at: https://sqlalche.me/e/14/e3q8)
09:12:34 - No messages in data-grepper on https://apps.fedoraproject.org/datagrepper/raw?topic=org.fedoraproject.prod.waiverdb.waiver.new&start=1656484636.123927&row_per_page=10&page=2 (psycopg2.OperationalError) server closed the connection unexpectedly
    This probably means the server terminated abnormally
    before or while processing the request.
[SQL: SELECT count(*) AS count_1 
FROM (SELECT messages.id AS messages_id, messages.msg_id AS messages_msg_id, messages.i AS messages_i, messages.topic AS messages_topic, messages.timestamp AS messages_timestamp, messages.certificate AS messages_certificate, messages.signature AS messages_signature, messages.category AS messages_category, messages.username AS messages_username, messages.crypto AS messages_crypto, messages.source_name AS messages_source_name, messages.source_version AS messages_source_version, messages.msg AS messages_msg, messages.headers AS messages_headers 
FROM messages 
WHERE messages.timestamp BETWEEN %(timestamp_1)s AND %(timestamp_2)s AND messages.topic = %(topic_1)s) AS anon_1]
[parameters: {'timestamp_1': datetime.datetime(2022, 6, 29, 6, 37, 16, 123927), 'timestamp_2': datetime.datetime(2022, 6, 29, 9, 12, 34, 63563), 'topic_1': 'org.fedoraproject.prod.waiverdb.waiver.new'}]
(Background on this error at: https://sqlalche.me/e/14/e3q8)
09:15:06 - No messages in data-grepper on https://apps.fedoraproject.org/datagrepper/raw?topic=org.fedoraproject.prod.waiverdb.waiver.new&start=1656484636.123927&row_per_page=10&page=8 (psycopg2.OperationalError) server closed the connection unexpectedly
    This probably means the server terminated abnormally
    before or while processing the request.
[SQL: SELECT count(*) AS count_1 
FROM (SELECT messages.id AS messages_id, messages.msg_id AS messages_msg_id, messages.i AS messages_i, messages.topic AS messages_topic, messages.timestamp AS messages_timestamp, messages.certificate AS messages_certificate, messages.signature AS messages_signature, messages.category AS messages_category, messages.username AS messages_username, messages.crypto AS messages_crypto, messages.source_name AS messages_source_name, messages.source_version AS messages_source_version, messages.msg AS messages_msg, messages.headers AS messages_headers 
FROM messages 
WHERE messages.timestamp BETWEEN %(timestamp_1)s AND %(timestamp_2)s AND messages.topic = %(topic_1)s) AS anon_1]
[parameters: {'timestamp_1': datetime.datetime(2022, 6, 29, 6, 37, 16, 123927), 'timestamp_2': datetime.datetime(2022, 6, 29, 9, 15, 6, 49230), 'topic_1': 'org.fedoraproject.prod.waiverdb.waiver.new'}]
(Background on this error at: https://sqlalche.me/e/14/e3q8)
09:16:06 - No messages in data-grepper on https://apps.fedoraproject.org/datagrepper/raw?topic=org.fedoraproject.prod.waiverdb.waiver.new&start=1656484636.123927&row_per_page=10&page=1 (psycopg2.OperationalError) server closed the connection unexpectedly
    This probably means the server terminated abnormally
    before or while processing the request.
[SQL: SELECT count(*) AS count_1 
FROM (SELECT messages.id AS messages_id, messages.msg_id AS messages_msg_id, messages.i AS messages_i, messages.topic AS messages_topic, messages.timestamp AS messages_timestamp, messages.certificate AS messages_certificate, messages.signature AS messages_signature, messages.category AS messages_category, messages.username AS messages_username, messages.crypto AS messages_crypto, messages.source_name AS messages_source_name, messages.source_version AS messages_source_version, messages.msg AS messages_msg, messages.headers AS messages_headers 
FROM messages 
WHERE messages.timestamp BETWEEN %(timestamp_1)s AND %(timestamp_2)s AND messages.topic = %(topic_1)s) AS anon_1]
[parameters: {'timestamp_1': datetime.datetime(2022, 6, 29, 6, 37, 16, 123927), 'timestamp_2': datetime.datetime(2022, 6, 29, 9, 16, 6, 481229), 'topic_1': 'org.fedoraproject.prod.waiverdb.waiver.new'}]
(Background on this error at: https://sqlalche.me/e/14/e3q8)
09:47:27 - waiverdb results not found in datagrepper - ran for: 10466s                                [FAILED]
09:47:27 - Checking datagrepper for greenwave messages
09:47:29 - greenwave results not found in datagrepper - ran for: 1s                                   [FAILED]
09:47:29 - Retrieving koji tags
10:02:45 - Retrieving koji tags: [] found while expected f36                                          [FAILED]
Ran for 12166 seconds (202.77 minutes)

Metadata