#105 Exception to package bundled libraries in Spring RTS.

Created 6 years ago by gilboa
Modified 7 months ago

The spring RTS games include a number of bundled libraries (Bug report [1]). Some are official sourced, and others are more-or-less code extracts that's included in the Spring RTS binary as static library due to Spring developers convenience.

I'll start with a general comment: The Spring game engine and the games that are built on it, are both single play and multi-player games; Changes to the code, even minor ones (E.g. The GCC 4.6 fix I introduced in F15) tends to break multi-player gaming or worse, tag a play as cheater. As such, replacing a library that does X with a library that does X, but uses a different code path (E.g. different MD5 implementation) can have adverse effects on multi-player gaming and must be ACK'ed by the upstream developers.
I've approached the upstream developers regarding the use of bundled library and its being looked at [2] - future Spring game engine releases should allow downstream maintainers to integrate system libraries instead of the built in libraries if/when such a library exists (More on that, later).

List of libraries:
1. Lua.
2. MD5.
3. 7z.

a. Has the library behavior been modified? If so, how has it been modified?
1. Lua: From the developers:
"Lua has patches applied, must link to streflop, and is configured different from stock Lua (most importantly we need lua_Number to be float and not double). Lua is particularly important because parts of the game code may be written in it, which must yield exactly identical results (also floating point operations!) on all platforms."

  1. MD5:
    The M5 implementation is a 3 function, single source file, free license implementation of the MD5 checksum algorithm. Given the nature of the software, its very hard to ascertain if the implementation was somehow modified. Beyond that, as far as I can see, there's no official upstream.

  2. 7z: The spring game engine uses a public domain implementation of the 7z compression algorithm. The library itself is a partial extract from the 7zip LZMA SDK.

b. Why haven't the changes been pushed to the upstream library?
1. Lua: Changes are specific to the Spring game engine.
2. MD5: No upstream.
3. 7z: As far as I understand, the Spring game engine only requires a small set of functionality from the 7z LZMA SDK; there's little use for the upstream in such a partial package.

c. Have the changes been proposed to the Fedora package maintainer for the library?
1. Lua: No, little use outside of the Spring game engine.
2. MD5: Not packaged in Fedora.
3. 7z: Not packaged in Fedora.

d. Could we make the forked version the canonical version within Fedora?
1. Lua: Unlikely, these changes will most likely break other Lua applications.
2. MD5: Given the scale of this implementation and the availability of better, well-supported alternatives, I greatly doubt it.
3. 7z: Possibly; If required by other potential 7z users. (Though the 7zip project itself is not cross-platform, limiting the use of 7z to the LZMA SDK). Alternatively, I could simply use the lzma-devel package, but this will require no-so-minor changes to the Spring engine code.

e. Are the changes useful to consumers other than the bundling application?
1. Lua: Unlikely. Changes are specific the Spring game engine.
2. MD5: Given the scope of the library, unlikely.
3. 7z: Unlikely (lzma-devel, see above).

f. Is upstream keeping the base library updated or are they continuously one or more versions behind the latest upstream release?
1. Lua: Yes. Current bundled version is Lua 5.1.
2. MD5: No. But given the nature of the code, the lack of upstream maintainer should not prove be an obstacle.
3. 7z: As far as I could check by looking at the actual bundled source files, yes.

g. What is the attitude of upstream towards bundling?
1. Lua: No alternative.
2. MD5: Using alternative implementations was considered, but given the scope of the bundled code and the cross-platform nature of the Spring engine itself (They rather have a clean, single source build environment on both Linux and Windows), I greatly doubt that they replace this implementation.
3. 7z: Upstream is willing to support compile-time configurable alternative implementation of 7z in future Spring game engine releases - but this will require a full LZMA SDK package in Fedora.

h. Overview of the security ramifications of bundling?
1. Lua: As long as the Spring developers closely follows upstream Lua releases, minimal.
2. MD5: MD5 itself is known to be vulnerable to spoofing, so using a different implementation will have negligible effect on the vulnerability of the Spring game engine.
3. 7z: Extracted source files seems identical to the current upstream release.

i. Does the maintainer of the Fedora package of the library being bundled have any comments about this?
1. Lua: Haven't checked. (Given the patchset and lack of outside users for the modified version, I doubt that it'll interest him).
2. MD5: None exist.
3. 7z: No direct alternative.

j. Is there a plan for unbundling the library at a later time?
1. Lua: Depending if/when the Spring game engine developers no longer require a patched version of Lua.
2. MD5: Unlikely. (I could simply patch the build system and include the code inside the main binary or alternatively, create a shared lib out of it - but both seem like a mere formality).
3. 7z: Yes. Pending upstream support for compile time alternatives.


7z is in Fedora, or its about to be: https://bugzilla.redhat.com/show_bug.cgi?id=596461
Is it feasible to use that package instead?

MD5 is already approved as copylib, so that's a non-issue here.

I think the Lua issue is the biggest one.

In today's meeting we approved the exception for the forked lua bundling (+1:6, 0:0, -1:1), md5 bundling approved (+1:7, 0:0, -1:0).

The spring rts package must add:

Provides: bundled(lua) = X.Y.Z (where X.Y.Z is the base lua version)
Provides: bundled(md5-$IMPLEMENTATION) (where $IMPLEMENTATION is the type of md5 implementation being used, see https://fedoraproject.org/wiki/Packaging:No_Bundled_Libraries#cite_note-1

Remaining issues:
We hope that the lzma-sdk package will be able to be used in place of the bundled copy. If this is not plausible, we will revisit that bundling exception.
Streflop bundling decision postponed, pending research on packaging it. I'm going to try to package that for Fedora today.

So, packaging streflop was easy enough, but hooking it into spring was... not trivial. CMake gives me a massive headache, I don't understand the motivation behind it.

FYI: http://spot.fedorapeople.org/streflop-0.3-1.fc15.src.rpm

Perhaps gilboa (or the spring upstream) can figure out how to make spring check for the streflop libs/headers instead of just building the bundled copy.

Contacted spring upstream about streflop and lzma-sdk, waiting to answer.
If OK w/ upstream, I'll patch the spring build-sys and add the missing detection until the upstream release.

Short update.
Spring 84 (that will hopefully be released by upstream next week) will include the "provides" tags.
As for having lzma-sdk being a compile-time option, upstream concerned by the lack of uniformity between the Windows lzma-sdk and the Linux version. They are currently search for alternatives.

I'm still waiting for OK for having streflop and lzma-sdk alternative compile time option.
Hopefully upstream will have more time to assist me once 84 has been released.

  • Gilboa

Announce text for the lua exception grant:

Spring RTS includes a forked and bundled copy of Lua which has Spring RTS specific patches applied, must link to streflop, and is configured differently from stock Lua (most importantly it needs lua_Number to be a float and not a double). Lua is particularly important because parts of the game code may be written in it, which must yield exactly identical results (also floating point operations!) on all platforms.

As a result, it has been granted a bundling exception for lua. The Spring RTS package must include Provides: bundled(lua) = X.Y.Z (where X.Y.Z is the base lua version), until the bundling issue is resolved (if ever).

7 months ago

Metadata Update from @spot:
- Issue assigned to spot

Login to comment on this ticket.