#1013 Document py_reproducible_pyc_path macro
Closed 3 years ago by churchyard. Opened 3 years ago by lbalhar.
lbalhar/packaging-committee byte-comp  into  master

@@ -418,3 +418,23 @@ 

  ....

  

  Note that this *does disable* the compilation of files in `+/usr/lib(64)?/pythonX.Y/+`.

+ 

+ == Byte compilation reproducibility

+ 

+ For two Python files with the exact same content and metadata, byte compilation might produce different results.

+ The resulting `.pyc` files are functionally identical but are not bit-by-bit identical.

+ In most cases, internal Python refference counter is here to blame because it might have a different internal state during each byte compilation.

s/refference/reference/

LGTM otherwise.

and "is to blame here" sounds better than "is here to blame" :)

+ If you want a deeper explanation, take a look at https://bugzilla.redhat.com/show_bug.cgi?id=1686078#c2[this Bugzilla comment].

+ 

+ This inconvenience might cause a problem in Koji where noarch packages built as a part of an arch build might be rejected because they have different content.

+ 

+ To workaround this issue, BuildRequire marshalparser `BuildRequires: /usr/bin/marshalparser` (a tool that makes `.pyc` files more reproducible)

+ and instruct it to process the `.pyc` files in certain paths by setting the `%py_reproducible_pyc_path` macro:

+ 

+ ....

+ %global py_reproducible_pyc_path %{buildroot}%{_datadir}/llamafarm/plugins

+ ....

+ 

+ With that setting, marshalparser recursively finds all byte-compiled Python files in `%{buildroot}%{_datadir}/llamafarm/plugins` and tries to fix them.

+ This happens at the very end of the build process when all previous byte compilation steps are finished.

+ If marshalparser cannot parse some of the cache files, the build fails.

Us e a more specific example? E.g. %global py_reproducible_pyc_path %{buildroot}%{_datadir}/llamafarm/plugins -- a more specific example serves better because readers have a better idea how does the argument work.

In the guidelines, we usually focus less on the story ("we have created") and concentrate on the "hows" and "whats", e.g. say "To workaround this issue, BuildRequire marshalparser (a tool that makes pyc files more reproducible) and instruct it to process the pyc files in certain paths by setting the %py_reproducible_pyc_path macro".
Does that make sense?

The general message here is good, please consider to switch to https://sembr.org/ when you amend the changes. (If you don't want to, don't worry, somebody from the FPC will do that before the merge.)

rebased onto 7d3ac1f73e1377e66312ce6705b0668b9bca7dfa

3 years ago

Fixed. I wasn't sure about newlines because some parts of the document uses them and some not so I went with what's simpler for me.

Fixed.

Thanks!

I wasn't sure about newlines because some parts of the document uses them and some not so I went with what's simpler for me.

Only new or revisited parts of the text follow sembr.

With my FPC hat on: +1.

Metadata Update from @churchyard:
- Pull-request tagged with: meeting

3 years ago

I wanted to push some AsciiDoc escape stuff in here, but Pagure doesn't let me, so I won't.

- `%{buildroot}%{_datadir}/llamafarm/plugins`
+ `+%{buildroot}%{_datadir}/llamafarm/plugins+`

etc...

+1. I'm fine with the change/addition, but it feels like this just happen automatically when you build a python package.

but it feels like this just happen automatically when you build a python package.

This is only opt in for now, because the tool might not be able to parse everything yet. Also, it needs to be BuildRequired explicitly.

s/refference/reference/

LGTM otherwise.

and "is to blame here" sounds better than "is here to blame" :)

rebased onto 5cddf8c

3 years ago

Pull-Request has been closed by churchyard

3 years ago