For some reason when trying to import tickets from https://fedorahosted.org/389/ it fails with this error from fedorahosted:
$ pgimport fedorahosted https://fedorahosted.org/389/ --tags Enter your FAS Username: mreynolds Password: Repo available
1 - 389.git
Choose the import destination repo [1]: ERROR: Error in response: {u'message': u'ServiceException details : unsupported type for timedelta microseconds component: unicode', u'code': -32603, u'name': u'JSONRPCError'}
When I try other repos it works fine, only "389" has issues. I would gladly help if you want to do live debugging, etc. I can be found on IRC "mreynolds" (internal #dirsec, freenode #389)
ok. Digging around and adding some debugging, this is the traceback trac outputs:
2016-11-18 18:13:31,284 Trac[xml_rpc] ERROR: Traceback (most recent call last): File "/usr/lib/python2.6/site-packages/tracrpc/web_ui.py", line 158, in _rpc_process result = (XMLRPCSystem(self.env).get_method(method_name)(req, args))[0] File "/usr/lib/python2.6/site-packages/tracrpc/api.py", line 197, in __call__ result = self.callable(req, *args) File "/usr/lib/python2.6/site-packages/tracrpc/ticket.py", line 140, in get t = model.Ticket(self.env, id) File "/usr/lib/python2.6/site-packages/trac/ticket/model.py", line 68, in __init__ self._fetch_ticket(tkt_id, db) File "/usr/lib/python2.6/site-packages/trac/ticket/model.py", line 121, in _fetch_ticket self.values[field] = from_utimestamp(value) File "/usr/lib/python2.6/site-packages/trac/util/datefmt.py", line 98, in from_utimestamp return _epoc + timedelta(microseconds=ts or 0) ServiceException: unsupported type for timedelta microseconds component: unicode
I see an old trac ticket where someone hit this with wiki pages: https://trac.edgewall.org/ticket/10287 but the solution there doesn't work for us.
debugging says it's running this sql:
SELECT t.id AS id,t.summary AS summary,t.owner AS owner,t.type AS type,t.status AS status,t.priority AS priority,t.milestone AS milestone,t.time AS time,t.changetime AS changetime,priority.value AS priority_value FROM ticket AS t LEFT OUTER JOIN enum AS priority ON (priority.type='priority' AND priority.name=priority) ORDER BY COALESCE(t.id,0)=0,t.id
Running a query for just time and updated I see the following tickets that are messed up:
SELECT t.id AS id,t.time AS time,t.changetime AS changetime FROM ticket AS t
... 47291|query_constraints|[{'component': [u'Security - Access Control (ACL)'], 'milestone': [u'1.2.11.20'], 'status': [u'reopened', u'accepted', u'new', u'assigned']}] 47292|query_tickets|628 47293|query_href|/389/query?component=Security+-+Access+Control+(ACL)&milestone=1.2.11.20&status=reopened&status=accepted&status=new&status=assigned&order=priority&group=status 47295|query_href|/389/query?component=Replication+-+General&status=new&status=assigned&status=accepted&status=reopened&milestone=1.3.2&order=owner&group=status&desc=1 47297|query_constraints|[{'component': [u'Replication - General'], 'status': [u'new', u'assigned', u'accepted', u'reopened'], 'milestone': [u'1.3.2']}] ...
So, we need to delete or fix these tickets... Is there any other record (emails/list, etc) that might tell us what these are and if we can delete them or should try and fix them? They don't seem to contain any useful data. ;(
sqlite> select * from ticket where id = '47291'; 47291|0|query_constraints|[{'component': [u'Security - Access Control (ACL)'], 'milestone': [u'1.2.11.20'], 'status': [u'reopened', u'accepted', u'new', u'assigned']}]||||||||||||| sqlite> select * from ticket where id = '47292'; 47292|0|query_tickets|628||||||||||||| sqlite> select * from ticket where id = '47293'; 47293|0|query_href|/389/query?component=Security+-+Access+Control+(ACL)&milestone=1.2.11.20&status=reopened&status=accepted&status=new&status=assigned&order=priority&group=status||||||||||||| sqlite> select * from ticket where id = '47294'; 47294|0|query_time|1363880649||||||||||||| sqlite> select * from ticket where id = '47295'; 47295|0|query_href|/389/query?component=Replication+-+General&status=new&status=assigned&status=accepted&status=reopened&milestone=1.3.2&order=owner&group=status&desc=1||||||||||||| sqlite> select * from ticket where id = '47297'; 47297|0|query_constraints|[{'component': [u'Replication - General'], 'status': [u'new', u'assigned', u'accepted', u'reopened'], 'milestone': [u'1.3.2']}]|||||||||||||
:hotdog:
Kevin,
So a few years ago something changed in the trac database, and our ticket numbers jumped from 400 something to 47298. So all of the tickets listed above (47290 - 47297) never really existed (perhaps they were used or testing?). Anyway they can be deleted, but I don't know how to do that.
Did you see any other tickets that were invalid?
Thanks, Mark
Nope. Only those ones.
I can manually delete them from command line. (After taking a backup).
Will do so now and then have you try the import again...
ok, done. Can you try the import again?
$ pgimport fedorahosted https://fedorahosted.org/389/ --tags Enter your FAS Username: mreynolds Password:
1 - clone-nunc-test.git 2 - nunc-test.git 3 - rest389.git 4 - ds.git
Choose the import destination repo [1]: 4 Updated ds.git with issue :1/49046 Updated ds.git with issue :2/49046 Updated ds.git with issue :3/49046 Updated ds.git with issue :4/49046 Updated ds.git with issue :5/49046 Updated ds.git with issue :6/49046 Updated ds.git with issue :7/49046 Updated ds.git with issue :8/49046 Updated ds.git with issue :9/49046 Updated ds.git with issue :10/49046 Updated ds.git with issue :11/49046 Updated ds.git with issue :12/49046 Updated ds.git with issue :13/49046 Updated ds.git with issue :14/49046 Updated ds.git with issue :15/49046 Updated ds.git with issue :16/49046 Updated ds.git with issue :17/49046 Updated ds.git with issue :18/49046 Updated ds.git with issue :19/49046 Updated ds.git with issue :20/49046 Updated ds.git with issue :21/49046 Updated ds.git with issue :22/49046 Updated ds.git with issue :23/49046 Updated ds.git with issue :24/49046 ERROR: Error in response: {u'message': u"Attachment 'dshang_result.tar.gz' not found", u'code': 404, u'name': u'JSONRPCError'}
Two issues:
[1] The ticket number is not incrementing (not sure if that's a pagure issue or not) [2] Missing attachment is breaking the import
The attachment there on ticket 25 doesn't exist indeed. Any idea what happened to it?
Either we should:
1) create some kind of placeholder attachment there so the import goes on. (Or if there's another copy of it I can just put it in place if you can get it to me)
or
2) open a issue against pagure-importer to ignore missing attachments and go on.
:ping_pong:
I'm currently looking into [2] - playing around with the code now. I'm not sure yet if the error in the response is blocking the import of that ticket
Yeah I fixed the missing attachment issue in the pagure importer. So we are good to go! Thanks for your help Kevin, please close this out.
Great. Please do let us know if you run into any further issues we can assist with.
:japanese_castle:
@kevin changed the status to Closed
Closed
Yeah I fixed the missing attachment issue in the pagure importer.
Would it better if that fix gets into importer project?