From a FPC viewpoint, there's no difference between a library written in C or java or python or PHP. So it's likely that this code counts as a bundled library. You can probably make an argument based on how much code is copied -- for instance, taking a few lines from a library (as long as the licenses are compatible) is probably not something we should be spending time on. Once past that level, though, you are dealing with things that can be readily identified by their API. FPC has never examined whether copying a single function from a library is an issue or whether copying a class or file or directory. Personally, I would say that at the function level and above we should definitely be using the system version of the library.

From reading the bug it sounds like some identifiable libraries have been found beneath the wp-includes directory. Is that the only places that your review found issues as well? From reading your comment, I'm not sure if the problems can be localized like that or if they're interspersed throughout the whole code.

I had a look at some file in wp-includes directory and maybe a third seems to be copied from somewhere. This is all based on the header in the comment. The following files seem to be copied from somewhere and heavily modified:

• http://core.trac.wordpress.org/browser/branches/2.8/wp-includes/canonical.php
• http://core.trac.wordpress.org/browser/branches/2.8/wp-includes/wp-db.php
• http://core.trac.wordpress.org/browser/branches/2.8/wp-includes/Text/Diff.php

The following files are more or less verbatim copies, with smaller modifications:
* http://core.trac.wordpress.org/browser/branches/2.8/wp-includes/atomlib.php
* http://core.trac.wordpress.org/browser/branches/2.8/wp-includes/class-IXR.php
* http://core.trac.wordpress.org/browser/branches/2.8/wp-includes/class-phpass.php
* http://core.trac.wordpress.org/browser/branches/2.8/wp-includes/class-phpmailer.php
* http://core.trac.wordpress.org/browser/branches/2.8/wp-includes/class-pop3.php
* http://core.trac.wordpress.org/browser/branches/2.8/wp-includes/class-simplepie.php
* http://core.trac.wordpress.org/browser/branches/2.8/wp-includes/class-smtp.php
* http://core.trac.wordpress.org/browser/branches/2.8/wp-includes/class-snoopy.php
* http://core.trac.wordpress.org/browser/branches/2.8/wp-includes/gettext.php
* http://core.trac.wordpress.org/browser/branches/2.8/wp-includes/kses.php
* http://core.trac.wordpress.org/browser/branches/2.8/wp-includes/rss.php
* http://core.trac.wordpress.org/browser/branches/2.8/wp-includes/streams.php

Lots of code in the javascript directory seems to be also copied from somewhere:
* http://core.trac.wordpress.org/browser/branches/2.8/wp-includes/js

I have not looked at the other directories, but it is possible that there is just as much external code as in wp-includes.

Took a look at a couple from each category:

The first set is heavily modified as you say. Examined wp-db.php. Highly conflicted here. On the one hand, there's a lot of customization. Instead of using the code as a library and subclassing to get any additional functionality they need, wp customizes the code inside the library. You can still see the original API and code but there's different error messages and additional functions embedded in the code. The version of the library that's being used here is pretty old. There's one more release in the 1.x line and the current stream of development is at 2.05. This is bad coding style and upstream should be told that if they keep doing stuff like this they won't be able to leverage open source development properly.

From the second set I looked at class-IXR.php and gettext.php. These both look like things that we should be using system libraries instead of the included files.

My feeling would be that the second set of packages Must be changed to use system libraries. The first set should go to FPC in order to clarify where to draw the line between bundle libraries and snippets of code taken from other projects. It is possible that all of the files in the first set are considered snippets in this respect but we (FPC) would need to decide that together rather than me making a lone decision. Want me to start the process on that? Does it affect your needs or is the second set still too big? Do you still want FESCo to render an judgement?

The javascript indeed has many bundled libraries :-( At the moment, there are no javascript packaging guidelines so we've been letting them through while telling packagers that once we have javascript guidelines we'll need to pull those libraries out as well (I'm thinking we'll treat javascript libraries like statically linked libraries where the javascript libraries come from another package that tracks the javascript library's upstream but the javascript is included in the app package at build time. When the javascript library is updated, the applications would need to be rebuilt to pull in the new javascript library versions. This would be a compromise between bundling (pretty much the only way it's done right now) and dynamic libraries (by patching applications to reference a URL where we install the javascript.))

We would like to defer this and look at it again at next weeks meeting.
Hopefully packaging folks can look and provide input on where the line should be drawn here.

I'm seeing if we can get quorum for FPC to discuss and evaluate the three heavily modified libraries listed above. There's several possibilities that can come out of that:

1. We could generate an update to the guidelines that clarify where to draw the line between bundled libraries and copied code snippets, probably with recommendations for each of the three heavily modified items.
2. We could say that these are all bundled libraries and make a recommendation to FESCo on whether to accept them under an exception or not.
3. We could let FESCo make the decision of whether these qualify as bundled libraries (via the current exception process) and then either try to draw out a Guideline from that or merely let our list of exceptions grow.

At this point I don't know which it will be.

Adrian, I also need to know what your feeling on the second, longer list of non-heavily modified libraries is. Since those seem to be very much bundled libraries, FESCo and FPC members can look at whether there's grounds for an exception there but any clarification of what constitutes a bundled library are unlikely to affect them.

From my point of view it would be important to let wordpress exist as it is right now. It is such a complicated piece of software (for me at least), that not doing what upstream does (copying it all together) seems to me difficult and dangerous. Especially dangerous when it comes to security updates.

Usually I would agree that it would be much more secure if it would use the installed system wide libraries. With wordpress, however, I would argue, as long it is maintained upstream, the best thing to do is to stay as close to upstream as possible.

Adrian,
Any thought to contacting wordpress upstream and asking them about these local copies and if they are interested in NOT having them be that way?

It is possible the current wordpress maintainers are not the older wordpress maintainers and that they don't actually know they are carting around other code OR are interested in NOT doing that anymore.

FPC discussed this at today's meeting and recommends 5:1 that the non-heavily modified libraries be unbundled (ie, no exception granted). There doesn't seem to be anything about them that makes it different than any other package bundling libraries and seeing as wordpress is network facing is at higher risk than other packages. The one dissenter thought that it would be appropriate to grandfather in libraries that are not already packaged for Fedora.

The heavily modified libraries are a more difficult issue. FPC thinks there is a line somewhere that can be drawn but we aren't sure yet. We'll be discussing this more next meeting to try and hash out some criteria. Possible criteria that we'll have to evaluate API changes, bugfixes that aren't upstream, needed by more than one package, dead upstream.... If anyone has more criteria they'd like to throw out there, feel free to add that and we'll consider whether it's a good idea or not.

The log is here:
http://meetbot.fedoraproject.org/fedora-meeting/2010-02-03/fedora-meeting.2010-02-03-16.04.log.html#l-73

Upstream ticket: http://core.trac.wordpress.org/ticket/12124

Replying to [comment:9 adrian]:

Upstream ticket: http://core.trac.wordpress.org/ticket/12124

Upstream closed it as ''wontfix''.

FPC talked about possible criteria but could not arrive at any decision on what criteria could be consistently applied so it's back to FESCo to analyze the code in those three files and make a decision. We did agree to a basic plan that would seem to fit this case:

1. Package up the verbatim copied bits as proper upstream components, and make WP dep on them (with symlinks if needed).
2. Generate diffs of the changes made to forked components and submit them to the upstreams (library upstreams in this case) for consideration. If those upstreams rejected the diffs, I'd try to at least open a dialog with WP on the topic

From looking at the wp code, some of #2 is definitely not suitable for upstream but I haven't looked at all the code. The other files would need to be looked at in the same light. Some of the things could be brought up with WP again if we have a better php programmer look into it but it seems that without our doing patches to prove out the concepts to WP upstream, it wouldn't be possible.

At the 2010-03-09 FESCo meeting:
{{{
* AGREED: wordpress should unbundle libraries that are upstream
identical. Can bundle 3 heavily modified php files. (nirik,
20:11:32)
}}}

Adrian: Please let us know what you can/are willing to do here now.

Removing meeting keyword.

The best would probably be if there is a new (or co-) maintainer who wants to dive into this. Maybe the maintainer from wordpress-mu is interested. I have not heard anything from him yet but there is also bug open for wordpress-mu for the same issue and as it is probably the same code it needs to be fixed as well as wordpress.

Adding the wordpress-mu owner here for comment.

Bret: Would you be willing to undertake this for both wordpress-mu/wordpress?

Being the maintainer of another headaches generating PHP package, I'd like to note a couple things; Debian is somehow unbundling the libs, or at least it looks like in the package page: http://packages.debian.org/unstable/web/wordpress; also http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=504242 confirms this.

Additionally, I think we should note in the upstream report that Debian (so probably also Ubuntu and other derivatives) are already doing the same; finally, they also claim it's a maintenance burden to guarantee their code will be working with system libs: in reality, they should just clearly state which versions are required, then it's just our job to make sure required libraries are available in the distro.

Unfortunately, their current package unbundles only parts of TinyMCE (not even the whole thing).

It seems it is not possible to find a solution here. I will write to the devel list asking for a new maintainer.

Swamped at work atm; working on getting a co-maintainer for -mu, might have bandwidth to look at it then.

As nothing happened here for almost three month I will ask on devel for a new maintainer (co-maintainer). With the newest wordpress release it seems that wordpress and wordpress-mu has been merged, so one of the packages (wordpress or wordpress-mu) has to be retired anyway.

Thanks Adrian.

Hopefully we can find a group of folks willing to work on this. Please do post.

Ten months past, wordpress receives no update in fedora after this blocker. I think we should be tolerant about bundling system libs. At least, it should not be a blocker for normal updates of existing packages, I also suggest bug reporter about bundling libraries to offer a help of unbudling libraries since some packagers don't have the ability of doing this.

Well, lots has changed and I don't know what the current status is.
wordpress and wordpress-mu have merged. Does the new tree have this issue?
Or something worse/different?

I would love to see a group of people work on the unbundling and updating wordpress. Perhaps try and reopen discussion on the mailing list?
We also have a outstanding fedora engineering services ticket pending to help with this.

Replying to [comment:22 supercyper]:

Ten months past, wordpress receives no update in fedora after this blocker. I think we should be tolerant about bundling system libs. At least, it should not be a blocker for normal updates of existing packages, I also suggest bug reporter about bundling libraries to offer a help of unbudling libraries since some packagers don't have the ability of doing this.

Correction, there have been no updates released. I posted links in both bugs to updated versions on 7/13 requesting testing before I committed them, and have received no reply.

There's not very many people still cc'ed on those bugs. ;(

Perhaps:

• Scratch build and put up a repo at repos.fedoraproject.org with the updated packages?
• Announce on the devel and/or test lists that you would like testers?

I have a wordpress-mu install here, will test that package and provide feedback in the bug.

I sort of envisioned the maintainer doing that, or at least testing and possibly pushing to rawhide. I'll email -devel.

Can you file this over on the FPC trac now?

https://fedorahosted.org/fpc/newticket

We are having FPC handle bundled library requests now. Thanks.

I'll note that some of these have been fixed now, but I am not sure what the entire state is, please file a FPC ticket to track that?