An optional feature for Fedora 33 Btrfs by default proposal is transparent compression.
Summary of benefits:
Summary of ways to enable compression:
-o compress=zstd:1
Example 1:
Example 2:
NOCOMPRESS
btrfs property set
A call for testing could be useful in helping to make decisions, and provide some "how to" documentation to encourage it.
Metadata Update from @ngompa: - Issue set to the milestone: Fedora 34 - Issue tagged with: Dev
A call for testing could be useful
Using btrfs and compression all this time since we start testing it in Fedora and i don't want go back to ext4. :) Here some benchmarks which i did for test case with mock and where Chris explain me more interesting stuff. More powerful CPU will benefit even more in this case for sure. Using zstd:1 on HDD and SSD (non-NVMe).
mock
zstd:1
The impact on battery life for a laptop is expected to be small. But is this just assumption? Is there a reliable way to measure it?
@salimma is interested in driving this for F34.
Metadata Update from @ngompa: - Issue assigned to salimma
Ideally there'd be a way to specify a different compression level for the installation, than what appears in the fstab. Why?
This makes it possible to use a higher level of compression for the installation than for run-time. This in particular could come in handy if we use Btrfs seed/sprout images. We'd want to use level 11+ for that seed, but use something else like 1 or 3 in fstab.
Further, we could use compress-force for the installation. And compress for fstab.
compress-force
compress
If we ever get reliable detection of rotational media, we could also have a higher value.
This is a nice to have. It's not blocking.
btrfs-progs does not have any way of showing the effect of compression. Currently 'compsize' does this and is in Fedora repository. Is it enough to include compsize by default in order to enable compression by default? Or should btrfs-progs have this functionality?
https://fedoraproject.org/wiki/Changes/BtrfsTransparentCompression is the F34 proposal to implement this, cc @salimma
compsize should definitely be in the default package set, if this is done. It should also be given a user-friendliness pass, TBH, if it's going to be an enduser tool. Currently:
compsize
-h
--help
$ compsize --help
compsize: unrecognized option '--help'
$ compsize
Usage: compsize file-or-dir1 [file-or-dir2 ...]
The man page, OTOH, is waaay too technical. The second paragraph reads:
As talking about compression ratio for a partial extent doesn't quite make any sense, every used extent is considered in its entirety. Every extent is also counted exactly once, even if it's reflinked multiple times.
A few paragraphs later it's showing ASCII-art diagrams of file storage layouts.
I suppose there's no way to expose the compressed size via an existing, well-known tool such as du, huh?
du
Filed as https://github.com/kilobyte/compsize/issues/37
Edit: (And I see the first issue is already addressed in the current code. In fact, it was addressed back in February, but there hasn't been a release since. I've requested one.)
# compsize /var /var/lib/nfs/rpc_pipefs/gssd/clntXX/info: Not btrfs (or SEARCH_V2 unsupported). #
I'm seeing this on Fedora Server, works fine on Fedora Workstation. I haven't configured NFS on this system, so I don't know what this is about. But even if someone has enabled NFS, it should skip over this gracefully.
compsize -x seems to work around this; consistent with rsync -x.
compsize -x
Could https://hackmd.io/kIMJv7yHSiKoAq1MPcCMdw be moved into the wiki (on a separate page if necessary) so we have it for the record (and under a Fedora-approved content license for that matter, as hackmd.io does not provide a license that I can see)?
I've copied it into the wiki (and I wrote it in the first place, so no licensing issue there)
Where did you copy it to? I don't see it in the Change document wiki page?
https://fedoraproject.org/wiki/Changes/BtrfsByDefault#Simple_Analysis_of_btrfs_zstd_compression_level
In the wrong place. I'll fix it
For clarity, I just added a bullet to the compression section with a link out to this wiki page: https://fedoraproject.org/wiki/Changes/BtrfsByDefault/CompressionLevelAnalysis
Hopefully I'm keeping relatively within the standards here, but I hope folks'll feel free to move wherever appropriate!
This page should probably be subheaded under the transparent compression change instead...
Sorry, we're doing performance reviews and it's been a wild week. I totally missed the correct wiki page: https://fedoraproject.org/wiki/Changes/BtrfsTransparentCompression#Simple_Analysis_of_btrfs_zstd_compression_level
Metadata Update from @chrismurphy: - Issue set to the milestone: Fedora 35 (was: Fedora 34)
This issue is fixed in Fedora 34, so I'll close it.
There's a weird bug that I discovered late in the F34 cycle, which is flagged for Common Bugs: btrfs compression isn't set if the user renames a mountpoint's subvolume
Metadata Update from @chrismurphy: - Issue close_status updated to: Fixed - Issue set to the milestone: Fedora 34 (was: Fedora 35) - Issue status updated to: Closed (was: Open)
Login to comment on this ticket.