#3366 PostgreSQL 14: KeyError: <class 'decimal.Decimal'>, TypeError: cannot marshal <class 'decimal.Decimal'> objects
Closed: Fixed 2 months ago by tkopecek. Opened 3 months ago by amessina.

After upgrading from Fedora 35 to Fedora 36 (koji-1.28.1-1.fc36.noarch, python3-3.10.4-1.fc36.x86_64), I now see the following xmlrpc.client.Fault for any connections to the hub (kojira, kojid, kojiweb):

[Sat May 21 15:00:50.947731 2022] [wsgi:error] 2022-05-21 15:00:50,947 [WARNING] m=getSessionInfo u=linux-ws1.messinet.com p=1090 r=fd00::1:46338 koji.xmlrpc: Traceback (most recent call last):
[Sat May 21 15:00:50.947794 2022] [wsgi:error]   File "/usr/lib64/python3.10/xmlrpc/client.py", line 522, in __dump
[Sat May 21 15:00:50.947807 2022] [wsgi:error]     f = self.dispatch[type(value)]
[Sat May 21 15:00:50.947825 2022] [wsgi:error] KeyError: <class 'decimal.Decimal'>
[Sat May 21 15:00:50.947843 2022] [wsgi:error] 
[Sat May 21 15:00:50.947851 2022] [wsgi:error] During handling of the above exception, another exception occurred:
[Sat May 21 15:00:50.947860 2022] [wsgi:error] 
[Sat May 21 15:00:50.947868 2022] [wsgi:error] Traceback (most recent call last):
[Sat May 21 15:00:50.947882 2022] [wsgi:error]   File "/usr/share/koji-hub/kojixmlrpc.py", line 257, in _wrap_handler
[Sat May 21 15:00:50.947893 2022] [wsgi:error]     response = dumps(response, methodresponse=1, marshaller=Marshaller)
[Sat May 21 15:00:50.947901 2022] [wsgi:error]   File "/usr/lib/python3.10/site-packages/koji/xmlrpcplus.py", line 83, in dumps
[Sat May 21 15:00:50.947910 2022] [wsgi:error]     data = m.dumps(params)
[Sat May 21 15:00:50.947919 2022] [wsgi:error]   File "/usr/lib64/python3.10/xmlrpc/client.py", line 514, in dumps
[Sat May 21 15:00:50.947928 2022] [wsgi:error]     dump(v, write)
[Sat May 21 15:00:50.947936 2022] [wsgi:error]   File "/usr/lib64/python3.10/xmlrpc/client.py", line 536, in __dump
[Sat May 21 15:00:50.947945 2022] [wsgi:error]     f(self, value, write)
[Sat May 21 15:00:50.947954 2022] [wsgi:error]   File "/usr/lib64/python3.10/xmlrpc/client.py", line 607, in dump_struct
[Sat May 21 15:00:50.947963 2022] [wsgi:error]     dump(v, write)
[Sat May 21 15:00:50.947972 2022] [wsgi:error]   File "/usr/lib64/python3.10/xmlrpc/client.py", line 526, in __dump
[Sat May 21 15:00:50.947981 2022] [wsgi:error]     raise TypeError("cannot marshal %s objects" % type(value))
[Sat May 21 15:00:50.947991 2022] [wsgi:error] TypeError: cannot marshal <class 'decimal.Decimal'> objects

I did not see this issue on Fedora 35. Any idea how to begin troubleshooting?

This may be related to the change in PostgreSQL 14 (https://www.postgresql.org/docs/14/release-14.html) "Change EXTRACT() to return type numeric instead of float8"

Temporarily, I have added the recommendation from https://www.psycopg.org/docs/faq.html#faq-float into /usr/lib/python3.10/site-packages/koji/db.py and my local Koji instance is running again.

Metadata Update from @tkopecek:
- Custom field Size adjusted to None
- Issue set to the milestone: 1.29.1

3 months ago

Second option is that we could live with Decimal internally and update xmlrpc ExtendedMarshaller to encode it as a float there. I'm a bit more inclined to convert it to float ASAP than handling it on xmlrpc layer. @mikem ?

Change EXTRACT() to return type numeric instead of float8 (Peter Eisentraut)

This avoids loss-of-precision issues in some usages. The old behavior can still be obtained by using the old underlying function date_part().

So, seems like we could change EXTRACT(EPOCH FROM $field) to date_part('epoch', $field).

That's a bit more invasive than the customized adapter, but might be more broadly compatible overall (assuming my reading is correct and that date_part really still returns floats).

I agree that we should address this when the data comes out of the db rather than wait for xmlrpc marshalling. The latter would risk breaking code inside the hub.

Metadata Update from @tkopecek:
- Issue tagged with: testing-ready

2 months ago

Metadata Update from @jobrauer:
- Issue tagged with: testing-done

2 months ago

Login to comment on this ticket.

Related Pull Requests
  • #3388 Merged 2 months ago