dusty / failed-composes

Clone
Source Code
GIT

#793 Fedora-Rawhide-20180917.n.0 DOOMED
Opened 5 years ago by dustymabe. Modified 5 years ago

Open
Close issue as:
fixed invalid timeout killed known_issue

pungi.global.log

  • Compose run failed because: - No Task ID, look at log statement
[ERROR   ] Compose run failed: ERROR running command: pungi-gather --config=/mnt/koji/compose/rawhide/Fedora-Rawhide-20180917.n.0/work/aarch64/pungi/Workstation.aarch64.comps.conf --greedy=build --arch=aarch64 --profiler

This is a crash in the depsolver, and the log is empty: https://kojipkgs.fedoraproject.org/compose/rawhide/Fedora-Rawhide-20180917.n.0/work/aarch64/pungi/Workstation.aarch64.comps.log

From dmesg output:

[219087.902563] pungi-gather[55087]: segfault at 31 ip 0000000000000031 sp 00007ffcbcb98e88 error 14 in python3.6[55cae97ee000+2000]

ooh that looks fun.. who can we pull in to help debug?

I don't know if there even is a way to debug it now unless there is a coredump. I'm not sure those are enabled on the machine. If we get, then maybe Python team might be able to help.

 
# coredumpctl info 55087
           PID: 55087 (pungi-gather)
           UID: 0 (root)
           GID: 0 (root)
        Signal: 11 (SEGV)
     Timestamp: Mon 2018-09-17 07:03:50 UTC (8h ago)
  Command Line: /usr/bin/python3 -s /usr/bin/pungi-gather --config=/mnt/koji/compose/rawhide/Fedora-Rawhide-20180917.n>
    Executable: /usr/bin/python3.6
 Control Group: /system.slice/crond.service
          Unit: crond.service
         Slice: system.slice
       Boot ID: adde520444cd4727816b76b1a7971fa8
    Machine ID: 5340b4e68c7b42f5b7e8a51489ef8227
      Hostname: rawhide-composer.phx2.fedoraproject.org
       Storage: /var/lib/systemd/coredump/core.pungi-gather.0.adde520444cd4727816b76b1a7971fa8.55087.1537167830000000.>
       Message: Process 55087 (pungi-gather) of user 0 dumped core.

                Stack trace of thread 55087:
                #0  0x0000000000000031 n/a (n/a)

not much use. ;( We do have the core file if someone wants me to put it somewhere.

If you can share it somewhere, I'll take a look if there is something to find out about the cause.

So, this is happening again (happened the last 3 runs).

https://infrastructure.fedoraproject.org/infra/tmp/core.pungi-gather.0.adde520444cd4727816b76b1a7971fa8.48705.1537748246000000.lz4

is the core from the one yesterday.

Here's the trace with debuginfo:

#0  0x000055b756598830 in ?? ()
#1  0x00007f24b949e2cf in visit_decref (op=<optimized out>, data=<optimized out>, op=<optimized out>, 
    data=<optimized out>) at /usr/src/debug/python3-3.6.6-1.fc28.x86_64/Modules/gcmodule.c:388
#2  0x00007f24b94c7d94 in subtract_refs (containers=<optimized out>)
    at /usr/src/debug/python3-3.6.6-1.fc28.x86_64/Modules/gcmodule.c:403
#3  collect () at /usr/src/debug/python3-3.6.6-1.fc28.x86_64/Modules/gcmodule.c:961
#4  0x00007f24b959fdfd in collect_with_callback (generation=0)
    at /usr/src/debug/python3-3.6.6-1.fc28.x86_64/Modules/gcmodule.c:1136
#5  0x00007f24b959feb0 in collect_generations () at /usr/src/debug/python3-3.6.6-1.fc28.x86_64/Modules/gcmodule.c:1159
#6  0x00007f24b94c89dc in _PyObject_GC_Alloc (basicsize=<optimized out>, use_calloc=0)
    at /usr/src/debug/python3-3.6.6-1.fc28.x86_64/Modules/gcmodule.c:1737
#7  _PyObject_GC_Malloc () at /usr/src/debug/python3-3.6.6-1.fc28.x86_64/Modules/gcmodule.c:1747
#8  0x00007f24b94c89fd in _PyObject_GC_New (tp=tp@entry=0x7f24b98ed6e0 <PyDict_Type>)
    at /usr/src/debug/python3-3.6.6-1.fc28.x86_64/Modules/gcmodule.c:1759
#9  0x00007f24b95130f2 in new_dict (values=0x0, keys=0x7f24b4f650d8)
    at /usr/src/debug/python3-3.6.6-1.fc28.x86_64/Objects/dictobject.c:586
#10 _PyDict_NewPresized () at /usr/src/debug/python3-3.6.6-1.fc28.x86_64/Objects/dictobject.c:1388
#11 0x00007f24b957f144 in _PyEval_EvalFrameDefault ()
    at /usr/src/debug/python3-3.6.6-1.fc28.x86_64/Python/ceval.c:2747
#12 0x00007f24b9502d5a in _PyFunction_FastCall (globals=<optimized out>, nargs=1, args=<optimized out>, 
    co=<optimized out>) at /usr/src/debug/python3-3.6.6-1.fc28.x86_64/Python/ceval.c:4947
#13 fast_function () at /usr/src/debug/python3-3.6.6-1.fc28.x86_64/Python/ceval.c:4947
#14 0x00007f24b954acc2 in call_function () at /usr/src/debug/python3-3.6.6-1.fc28.x86_64/Python/ceval.c:4851
#15 0x00007f24b957b12a in _PyEval_EvalFrameDefault ()
    at /usr/src/debug/python3-3.6.6-1.fc28.x86_64/Python/ceval.c:3328
#16 0x00007f24b9502d5a in _PyFunction_FastCall (globals=<optimized out>, nargs=2, args=<optimized out>, 
    co=<optimized out>) at /usr/src/debug/python3-3.6.6-1.fc28.x86_64/Python/ceval.c:4947
#17 fast_function () at /usr/src/debug/python3-3.6.6-1.fc28.x86_64/Python/ceval.c:4947
#18 0x00007f24b954acc2 in call_function () at /usr/src/debug/python3-3.6.6-1.fc28.x86_64/Python/ceval.c:4851
#19 0x00007f24b957b12a in _PyEval_EvalFrameDefault ()
    at /usr/src/debug/python3-3.6.6-1.fc28.x86_64/Python/ceval.c:3328
#20 0x00007f24b954d058 in _PyEval_EvalCodeWithName (qualname=0x0, name=<optimized out>, closure=0x0, kwdefs=0x0, 
    defcount=0, defs=0x0, kwstep=2, kwcount=<optimized out>, kwargs=<optimized out>, kwnames=<optimized out>, 
    argcount=<optimized out>, args=<optimized out>, locals=<unknown at remote 0x7f24b9ab2288>, 
    globals=<unknown at remote 0x7f24b9ab2288>, _co=<unknown at remote 0x7f24b8641300>)
    at /usr/src/debug/python3-3.6.6-1.fc28.x86_64/Python/ceval.c:4180
#21 PyEval_EvalCodeEx () at /usr/src/debug/python3-3.6.6-1.fc28.x86_64/Python/ceval.c:4180
#22 0x00007f24b954df0b in PyEval_EvalCode (co=co@entry=<unknown at remote 0x7f24b8641300>, 
    globals=globals@entry=<unknown at remote 0x7f24b9ab2288>, locals=locals@entry=<unknown at remote 0x7f24b9ab2288>)
    at /usr/src/debug/python3-3.6.6-1.fc28.x86_64/Python/ceval.c:731
#23 0x00007f24b95e77f2 in run_mod () at /usr/src/debug/python3-3.6.6-1.fc28.x86_64/Python/pythonrun.c:1025
#24 0x00007f24b95e93ca in PyRun_FileExFlags () at /usr/src/debug/python3-3.6.6-1.fc28.x86_64/Python/pythonrun.c:978
#25 0x00007f24b95ea128 in PyRun_SimpleFileExFlags ()
    at /usr/src/debug/python3-3.6.6-1.fc28.x86_64/Python/pythonrun.c:420
#26 0x00007f24b95ea2e9 in PyRun_AnyFileExFlags () at /usr/src/debug/python3-3.6.6-1.fc28.x86_64/Python/pythonrun.c:81
#27 0x00007f24b95eb1fe in run_file (
    p_cf=<error reading variable: Dwarf Error: could not find partial DIE containing offset 0x316 [in module /usr/lib/debug/usr/bin/python3.6-3.6.6-1.fc28.x86_64.debug]>, filename=<optimized out>, fp=0x55b755f69bc0)
    at /usr/src/debug/python3-3.6.6-1.fc28.x86_64/Modules/main.c:344
#28 Py_Main () at /usr/src/debug/python3-3.6.6-1.fc28.x86_64/Modules/main.c:814

I ran it with PYTHONMALLOC=debug and of course it didn't crash.

Next time I'll file a pungi issue with all this info...

Login to comment on this ticket.

Metadata
None
None
None
None
Powered by Pagure 5.13.3
Documentation • File an Issue • About • SSH Hostkey/Fingerprint
© Red Hat, Inc. and others.