There was a question on IRC about whether to package the pycache directory and I realized that it doesn't seem to be mentioned in the guidelines.
I suppose I'd expect to find mention of it, and a bit of an example spec, under https://fedoraproject.org/wiki/Packaging:Python#Byte_compiling but I don't see any mention of pycache at all.
Unfortunately I don't know enough about python packaging to say what the right thing to do would be. Any hints? I'll write up a draft if I know the basics of what it should say.
Indeed, they should be packaged. pycache is a relatively new thing. It basically replaces the former practice of having .pyc and .pyo byte compiled files live next to the .py source files on the filesystem. It allows storing python byte code built from the same source code for several different python interpreters. We aren't currently using the new abilities that this provides but we do use the compiled byte code. Since upstream has moved to shipping byte code this way we need to ship the directories.
This upstream PEP describes why upstream has moved to this format and how it is laid out on the filesystem: https://www.python.org/dev/peps/pep-3147/
It's possible that in the future we could use the enhanced features that this provides to help ship python libraries for cpython, pypy, jython, and other implementations of python. On the other hand we may continue to keep those in separate directory structures. But the option is available for us to explore when the time is right.
CC'ing bkabrda in case he has anything to add.
I think that Toshio summed it up very well, nothing to add. I can only confirm that we do need to ship these directories for the reasons mentioned by Toshio.
There is the question where {{{%{python3_sitearch}/pycache}}} should be packaged though.
I'd say it should be owned by python-libs and normal python packages must not directly include it - only the own files ''inside'' that directory.
What's your opinion bkabrda, toshio?
Replying to [comment:3 tomspur]:
There is the question where {{{%{python3_sitearch}/pycache}}} should be packaged though. I'd say it should be owned by python-libs and normal python packages must not directly include it - only the own files ''inside'' that directory. What's your opinion bkabrda, toshio?
Do you mean {{{%{python3_sitearch}/site-packages/pycache}}}?
If so, then AFAICS both this directory and its {{{%{python3_sitelib}/site-packages/pycache}}} counterpart are owned by python3.
It is a valid point that these shouldn't be owned by Python extension modules. Extension modules should (or rather "must") only own {{{pycache}}} directories in subdirectories of {{{site-packages}}}.
Replying to [comment:4 bkabrda]:
Replying to [comment:3 tomspur]: There is the question where {{{%{python3_sitearch}/pycache}}} should be packaged though. I'd say it should be owned by python-libs and normal python packages must not directly include it - only the own files ''inside'' that directory. What's your opinion bkabrda, toshio? Do you mean {{{%{python3_sitearch}/site-packages/pycache}}}?
{{{%{python3_sitearch}/pycache}}} evaluates to {{{/usr/lib64/python3.4/site-packages/pycache}}}, so I mean the former.
If so, then AFAICS both this directory and its {{{%{python3_sitelib}/site-packages/pycache}}} counterpart are owned by python3. It is a valid point that these shouldn't be owned by Python extension modules. Extension modules should (or rather "must") only own {{{pycache}}} directories in subdirectories of {{{site-packages}}}.
+1
Replying to [comment:5 tomspur]:
Replying to [comment:4 bkabrda]: Replying to [comment:3 tomspur]: There is the question where {{{%{python3_sitearch}/pycache}}} should be packaged though. I'd say it should be owned by python-libs and normal python packages must not directly include it - only the own files ''inside'' that directory. What's your opinion bkabrda, toshio? Do you mean {{{%{python3_sitearch}/site-packages/pycache}}}? {{{%{python3_sitearch}/pycache}}} evaluates to {{{/usr/lib64/python3.4/site-packages/pycache}}}, so I mean the former.
Ah, sorry, I got it confused. I was talking about the directory you originally meant, of course.
Looks like I need to try and cook up a draft.
OK, finally. Here's a strawman: https://fedoraproject.org/wiki/User:Tibbs/PycacheDraft#Byte_compiling
Diff from the current python guidelines: https://fedoraproject.org/w/index.php?title=User%3ATibbs%2FPycacheDraft&diff=402243&oldid=402019
Basically that section would have been hilariously complicated if I had preserved the existing "there's more than one way to do it" set of examples so I just condensed it down to the simplest one.
Also, note the large list of packages that directly own the top level pycache directory that, if we decide this is the right way to go, would not follow the guideline.
This was discussed in this weeks meeting (http://meetbot.fedoraproject.org/fedora-meeting-1/2015-02-05/fpc.2015-02-05-17.02.txt):
Announcement text:
The byte compilation section of the Python packaging guidelines was rewritten to include information about packaging the pycache directories generated by newer Python versions. https://fedoraproject.org/wiki/Packaging:Python#Byte_compiling https://fedorahosted.org/fpc/ticket/494
Metadata Update from @tomspur: - Issue assigned to tibbs
Login to comment on this ticket.