#1657 No space left on device
Closed: Fixed 3 years ago by praiskup. Opened 3 years ago by jankratochvil.

https://copr.fedorainfracloud.org/coprs/jankratochvil/lldb/build/1886281/
In July 2020 I had to disable *-debuginfo build as it started to fail due to small disk size. The build is about 50GB locally, rpmbuild install part may require some more space.
praiskup: ah, no ...

mock_chroot_tmpfs   81G  780M   80G   1% /var/lib/mock/epel-8-x86_64-1610959337.030000/root

=> https://download.copr.fedorainfracloud.org/results/jankratochvil/lldb/epel-8-x86_64/01886281-lldb-experimental/hw_info.log.gz
80GB may really not be enough. IMO 120GB would be enough.


$ du -shc lldb-experimental-12.0.0/ *.rpm *.x86_64
95G lldb-experimental-12.0.0/
1.3G    *.rpm
6.4G    *.x86_64 (unpacked *.rpm)
102G    total

So 100GB would not be enough, maybe 120GB with some reserve for near future?

Metadata Update from @praiskup:
- Issue assigned to praiskup

3 years ago

Metadata Update from @praiskup:
- Issue tagged with: RFE, ansible

3 years ago

https://pagure.io/fedora-infra/ansible/c/d4e1b2d5bca9a251732cb1b51745f27a3c06ee6e?branch=main

But I realized we probably need to regenerate the builder images
for the change. Commit 66451136c2f097a4ae0d194bffa6e71765a578c7
(copr-be: provision: don't create SWAP when preparing image) and
so I'm testing with 37c0246379e2a88e7281beb398b000abd2a3c04e.

I expect production will be fixed tomorrow soonest.

Done, there should be 120G now. Thank you for the report.

Metadata Update from @praiskup:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

3 years ago

Unfortunately it is not enough:
https://copr.fedorainfracloud.org/coprs/jankratochvil/lldb/build/1890432/
https://download.copr.fedorainfracloud.org/results/jankratochvil/lldb/fedora-33-x86_64/01890432-lldb-experimental/

mock_chroot_tmpfs  129G  257M  129G   1% /var/lib/mock/fedora-33-x86_64-1611220613.743397/root
+ /usr/bin/cmake --install x86_64-redhat-linux-gnu
...
  No space left on device.

It is already the last installation phase so I expect not much more disk size is needed.
I have tried a rebuild locally but it is only 98GB - BUILDROOT/ is empty despite mock -N:

mock --enable-network -r fedora-33-x86_64 --uniqueext=lldbsize -N --rebuild lldb-experimental-12.0.0-0.20210121snap2.fc33.src.rpm
98G /var/lib/mock/fedora-33-x86_64-lldbsize

Metadata Update from @jankratochvil:
- Issue status updated to: Open (was: Closed)

3 years ago

I can add 20G more, probably not more. We are in the process of adding new builders
on hardware with limited disk space, and even this will likely force us to limit the number
of new builders..

The system isn't ready to work with builders with different parameters - as user you
can not ask for high-performance builder. So if we had two kinds of VMs, you would
receive random VMs and random failures.

20GB should be definitely OK but if it really has some negative impact on COPR I am fine with WONTFIXing it (and decreasing it back to 80GB or whatever). I can live without debuginfo in these builds. I have also some doubts if anyone else besides me is using these LLDB builds anyway.

Updated, there's 140G. Please reopen if it doesn't help (this is suspicious enough to properly debug, if you claim that locally it is enough to have just 100G of storage).

It has failed even with 140GB. So I rather rechecked the .spec and made a shared library build (not static libraries build). That first into 55GB locally (although the static build fits into 102GB locally). It builds now fine in 140GB but I expect much less would be sufficient:
https://download.copr.fedorainfracloud.org/results/jankratochvil/lldb/fedora-rawhide-x86_64/01917686-lldb-experimental/builder-live.log.gz

+ du -shc ...
46G /builddir/build/BUILD/lldb-experimental-13.0.0
4.2G    /builddir/build/BUILDROOT/lldb-experimental-13.0.0-0.20210129snap0.fc34.x86_64
50G total

The static build was intentional in the past for easier debugging but I see it causes more trouble than benefits.

Can we face some problem with memory management/storage management in gcc/rpmbuild? If you claim that locally you fit in 100G it is weird 140 is still not enough in Copr.

But nevermind, if the dynamic linking is good enough here, I am not against.

Metadata Update from @praiskup:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

3 years ago

Log in to comment on this ticket.

Metadata