#2597 F35 Change: Debuginfod By Default
Closed: Accepted 3 months ago by churchyard. Opened 4 months ago by bcotton.

Fedora users / developers who need to debug/trace distro binaries can make use of the recently activated elfutils-debuginfod servers to automatically fetch debugging data and source code, instead of having to use # sudo dnf commands.


I added the meeting tag, because I think we should discuss the privacy and security implications of this.

Metadata Update from @zbyszek:
- Issue tagged with: meeting

4 months ago

Placeholder -0 while further discussion happens.

Random privacy/security thought after the meeting:

Open arbitrary fedora binary via gdb.
Admire the immediate downloading of debuginfo and source code.

What if there was a consent prompt? Something like:

Open arbitrary fedora binary via gdb.

Gdb will download and execute debuginfo files from https://debuginfod.fedoraproject.org/
For the information that will be shared with the server see https://debuginfod.fedoraproject.org/privacy
Allow this? [No/Yes/Never/Always]
> Always
Gdb will always download and execute debuginfo files from https://debuginfod.fedoraproject.org/ -- you can revisit this decision at any time by running `debuginfod --forget https://debuginfod.fedoraproject.org`

Admire the immediate downloading of debuginfo and source code.

What if there was a consent prompt?

Consider that there are a dozen+ applications that use the client code library, so new user interface at the per-consumer level would involve a lot of tooling & upstream changes. GDB for example already includes explicit diagnostics to show users downloads in progress, so there is no surprise. But it's not a confirmation, it's a diagnostic. Systemtap's stap-prep tool prints $DEBUGINFOD_URLS, but the main tool just uses it quietly if set.

If instead one aims at the client library, doing interactive tty UX conversations from within a low level place is not generally appropriate. Or do you happen to know such a distro framework or precedent?

No, I don't really, I just thought that getting user consent might be required by legal/etc.

A patch for one-time user notification (not interactive consent) has been drafted:
https://sourceware.org/pipermail/elfutils-devel/2021q2/003749.html

Awesome, thanks. Coudl you please paste a an example gdb session that has this? How does the user know what to do if they are paranoid and want to opt-out?

% gdb /bin/ls
[...]
Type "apropos word" to search for commands related to "word"...
Reading symbols from /bin/ls...

NOTICE: This system is configured to auto-download debuginfo from:
https://debuginfod.elfutils.org/
NOTICE: Set your $DEBUGINFOD_URLS differently if desired.

[sleeps here]
[...]

GDB accepts an interrupt at this point.

Awesome, that makes me +1 for the change.

Metadata Update from @churchyard:
- Issue close_status updated to: Accepted
- Issue status updated to: Closed (was: Open)

3 months ago

Could you explain what is required for the warning?

The example provided can be implemented in gdb. Is it enough if gdb provides that warning as in https://pagure.io/fesco/issue/2597#comment-730463 ?

I don't like having the library do extra file accesses, a warning and sleep:
https://sourceware.org/pipermail/elfutils-devel/2021q2/003770.html
It isn't the job of a library to do such interactive warnings, but of the programs using the libraries and/or the installer that sets the environment variable to opt in for the user.

Metadata Update from @churchyard:
- Issue status updated to: Open (was: Closed)

3 months ago

I am relieved :-) to report that Mark (elfutils maintainer) would approve an alternate method whereby elfutils' /etc/profile.d shell script fragments print out a one-time notification to the user. The patch below also tries to send a desktop notification (via /usr/bin/notify-send). However, this option is not tied to the point of use of an actual debugger-type tool. WDYT?

https://sourceware.org/pipermail/elfutils-devel/2021q2/003782.html

I am relieved :-) to report that Mark (elfutils maintainer) would approve an alternate method whereby elfutils' /etc/profile.d shell script fragments print out a one-time notification to the user. The patch below also tries to send a desktop notification (via /usr/bin/notify-send).

I'm confused. Wouldn't this script be executed on any login? Is it somehow tied to gdb or systemd-coredump being executed?

--

This was mentioned before (by @ngompa IIRC): can we please get a splash page under
https://debuginfod.fedoraproject.org/ ? It is important because if Fedora systemds start autoconnecting to that service, things will be easier if people immediately see an explanation what it is. https://debuginfod.debian.net seems like a good blueprint.

I'm confused. Wouldn't this script be executed on any login? Is it somehow tied to gdb or systemd-coredump being executed?

Yes, it would hit every user's first login, which seems broad. If this is unacceptable, and if the elfutils maintainer vetoes debuginfod-client library code even as a temporary fedora-only patch, then we are stuck with having to instrument calling applications. What would you say is a minimum set?

can we please get a splash page under https://debuginfod.fedoraproject.org

Can do - maybe just a redirect into the wiki? - just needs some proxy configuration work.

can we please get a splash page under https://debuginfod.fedoraproject.org/

Thanks to help from @kevin, this now redirects to this wiki page.

Please don't use wiki pages as landing pages. Make a static page and host it there.

Basically, something along the lines of a page like this one should be hosted on debuginfod.fedoraproject.org.

May I ask why? This is not a page that will be fetched by debuginfod client software, only curious users, by hand, so the load should be negligible. Keeping it in the wiki lets us easily keep it up to date with usage advice, or independent status. Adding a static file would require adding a new piece of web fileserver whereas there isn't one anywhere now, so it'd be an additional thing to maintain.

Because a wiki page is a terrible landing page because it's easily mutable. Not to mention, it couples the landing page for debuginfod to the wiki system, and if the latter goes down, then you have no landing page.

Because a wiki page is a terrible landing page because it's easily mutable.

Only by FAS account holders, with an audit trail.

Not to mention, it couples the landing page for debuginfod to the wiki system, and if the latter goes down, then you have no landing page.

That seems OK -- the landing page is not important for functioning, and the wiki system is robust.

Did the redirect maybe break things? I now get (No debugging symbols found in .gnu_debugdata for ...) for every source file:

$ sleep 10& DEBUGINFOD_URLS='https://debuginfod.fedoraproject.org/' DEBUGINFOD_PROGRESS=1 gdb -p $(pgrep sleep)
[1] 508
GNU gdb (GDB) Fedora 10.1-18.fc35
...
Attaching to process 508
Reading symbols from /usr/bin/sleep...
Reading symbols from .gnu_debugdata for /usr/bin/sleep...
(No debugging symbols found in .gnu_debugdata for /usr/bin/sleep)
Reading symbols from /lib64/libc.so.6...
(No debugging symbols found in /lib64/libc.so.6)
Reading symbols from /lib64/ld-linux-x86-64.so.2...
(No debugging symbols found in /lib64/ld-linux-x86-64.so.2)
0x00007f9a206a6e2a in clock_nanosleep@GLIBC_2.2.5 () from /lib64/libc.so.6
Missing separate debuginfos, use: dnf debuginfo-install coreutils-8.32-22.fc35.x86_64

Did the redirect maybe break things?

Argh, yes ....

% curl https://debuginfod.stg.fedoraproject.org/buildid/998feb12788309aa27465206ce4209be13ef5d9a/debuginfo
<title>302 Found</title>

Interestingly, the metrics work. Fixing....

https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/Z7OTP34QOGFU2HI7X4SDIOA6BGQPN4IR/

* #2597 F35 Change: Debuginfod By Default  (mhroncok, 17:04:41)
  * AGREED: Change is approved as originally proposed to FESCo (+5,2,-0)
    (mhroncok, 17:21:12)
  * if there are problems, fesco can revert this decision in the future
    (true for all fesco's decision, but wanted to be explicit here)
    (mhroncok, 17:21:44)
  * ACTION: fche to do their best to add a warning to at least gdb
    (mhroncok, 17:22:09)
  * ACTION: zbyszek to continue the discussion on how to make this safer
    (on the mailing list)  (mhroncok, 17:22:46)
  * ACTION: fche to make sure this is loud enough in the release notes /
    announcements etc.  (mhroncok, 17:23:59) 

Metadata Update from @churchyard:
- Issue close_status updated to: Accepted
- Issue status updated to: Closed (was: Open)

3 months ago

Login to comment on this ticket.

Metadata