freeipa

Clone
Source Code
GIT

#6418 Build system refactoring
Closed None Opened 8 years ago by pspacek.

Closed
Reopen Issue

The build system in FreeIPA 4.4 has following problems:

  • It is ineffective. This costs developer time every time we build IPA. Every developer typically builds IPA several times a day. Also, it slows down CI because it needs to build the packages as well.
  • The build system does not directly support rapid development approaches: Every patch-copy-test cycle requires either manual work (which is error prone) or full build (which is very slow). It slows down rapid prototyping.
    • Repeated build with minimal changes (like changing one file) takes long time, which prevents us from doing rapid devel cycle.
    • Theoretically the process should be very fast as proper Makefile would detect what can be skipped.
    • In reality the build takes 88 % of the time from the first build!
  • Is not reliable. Often, "git clean -xdf" is needed to get the build to pass.
  • It is hard to maintain and debug.

The proposal is to re-write the build system using current best practices and standard tools. Re-write from scratch is deemed to be more effecient than fixing bugs one by one because parts of current system are unnecessairly complex or can be replaced with standard solutions.

We forgot to create a ticket for this refactoring effort, so I've created one restrospectively and added all the relevant pull requests to it.

pr 132, master:

  • 4cd83fb Port all setup.py to setuptools
  • e12a70a Remove ipapython/ipa.conf

pr171, master:

  • 329f080 Build: pass down LIBDIR definition from RPM SPEC to Makefile
  • 41da873 Build: adjust include paths in daemons/ipa-kdb/tests/ipa_kdb_tests.c
  • 25dab77 Build: promote daemons/configure.ac to top-level configure.ac
  • b0cb6af Build: transform util directory to libutil convenience library
  • 6e1d777 Build: merge asn1/configure.ac to top-level configure.ac
  • 5e028b5 Build: merge ipatests/man/configure.ac to top-level configure.ac
  • 927ddcb Build: merge install/configure.ac into top-level configure.ac
  • 0d37619 Build: move translations from install/po/ to top-level po/
  • 881ab4e Build: remove Transifex support
  • 0d7d6f3 Build: merge client/configure.ac into top-level configure.ac
  • 1f16486 Build: modernize POPT library detection
  • fd3176a Build: modernize SASL library detection
  • b12da59 Build: modernize CURL library detection
  • 1e0143c Build: modernize XMLRPC-client library detection
  • 8acb1f3 Build: cleanup INI library detection
  • c8be979 Build: use libutil convenience library for client
  • 1a4f287 Build: use default error handling for PKG_CHECK_MODULES
  • 38628e4 Build: split API/ACI checks into separate Makefile targets
  • f25c0cb Build: split out egg-info Makefile target from version-update target
  • c954d0e Build docs: update platform selection instructions
  • c70a287 Build: replace ipaplatform magic with symlinks generated by configure
  • 3e79d8a Docs: update docs about ipaplatform to match reality

pr159, master:

  • 21395d1 spec file: clean up BuildRequires
  • 1077743 spec file: do not include BuildRequires for lint by default
  • 0d370a9 pylint: enable the import-error check
  • 0b91735 ipaserver: remove ipalib import from setup.py
  • 9b53423 makeapi, makeaci: do not fail on missing imports
  • 9477e39 client: remove unused libcurl build dependency
  • cc5ad6b pwpolicy: do not run klist on import

pr213, master:

  • c48e5fd Build: move version handling from Makefile to configure
  • 0a17155 Build: replace hand-made Makefile with one generated by Automake
  • 81da45f Build: fix module name in ipaserver/setup.py
  • 8de11b0 Build: Makefiles for Python packages
  • 3d6b8f8 Build: fix ipaplatform detection for out-of-tree builds
  • 3a41b3b Build: fix make install in asn1 subdirectory
  • 820fd4c Build: respect --prefix for systemdsystemunitdir
  • 24feae4 Build: fix Makefile.am files to separate source and build directories
  • b8d81ba Build: remove non-existing README files from Makefile.am
  • fa8a468 Build: rename project from ipa-server to freeipa
  • 7282776 Build: fix distdir target for translations
  • 2f67128 Build: fix man page distribution
  • 021a52d Build: fix distribution and installation of static files in top-level directory
  • f95098b Build: stop build when a step in web UI build fails
  • 441acf7 Build: fix distribution of static files for web UI
  • 24525fd Web UI: Remove offline version of Web UI
  • b910683 Build: fix distribution and installation of update LDIFs
  • 04be250 Build: fix distribution of libexec scripts
  • deec97a Build: cleanup unused LDIFs from install/share
  • a125370 Build: Remove spurious EXTRA_DIST from install/share/Makefile.am
  • dabc65f Build: fix distribution of oddjob files
  • 886d916 Build: fix distribution of install/REDME.schema file
  • c951a49 Build: fix distribution of asn1/asn1c files
  • 0d5fe1d Build: fix distribution of client header file
  • 125bf25 Build: fix distribution of daemon/ipa-kdb files
  • 684a2c6 Build: fix distribution of ipa-slapi-plugins/common files
  • 4fb2f53 Build: fix distribution of daemons/ipa-slapi-plugins/ipa-cldap files
  • 14bce67 Build: fix distribution of daemons/ipa-slapi-plugins/ipa-exdom-extop files
  • c1652f9 Build: fix distribution of daemons/ipa-slapi-plugins/ipa-otp-counter files
  • 278cda7 Build: fix distribution of daemons/ipa-slapi-plugins/ipa-otp-lasttoken files
  • 53cd71a Build: fix distribution of daemons/ipa-slapi-plugins/ipa-pwd-extop files
  • 74820fe Build: fix distribution of daemons/ipa-slapi-plugins/ipa-sidgen files
  • 39b17ef Build: fix distribution of daemons/ipa-slapi-plugins/ipa-winsync files
  • f229bb5 Build: fix distribution of daemons/ipa-slapi-plugins/topology files
  • 312e780 Build: integrate daemons/dnssec into build system
  • 8ffd3bd Build: remove ancient checks/check-ra.py
  • d3cab75 Build: integrate contrib directory into build system
  • a027bf7 Build: remove init/SystemV directory
  • 288d624 Build: integrate init and init/systemd into build system
  • 6cb0271 Build: create /var/run directories at install time
  • cc63825 Build: distribute doc directory
  • dc5699a Build: do not compress manual pages at install time
  • 4ce3aa3 Build: remove unused dirs /var/cache/ipa/{sysupgrade,sysrestore} from SPEC
  • 75a944e Build: fix KDC proxy installation and remove unused kdcproxy.conf
  • fee9bbd Build: add make rpms target and convenience script makerpms.sh
  • 2df9877 Build: remove obsolete instructions about BuildRequires from BUILD.txt
  • 14c1c8d Build: add Python lint target
  • f31a489 Build: add JS lint target
  • b54e9e8 Build: add makeaci lint target
  • e3b537a Build: add makeapi lint target
  • 4498998 Build: add polint target for i18n tests
  • c0674e8 Build: fix make clean for web UI

master:

  • 9bb6d86 gitignore: ignore tar ball

master:

  • d20f6a5 Build: fix make clean to remove build artifacts from top-level directory

master:

  • 394edf5 Build: remove unused and redundant code from configure.ac and po/Makefile.in
  • f6f5708 Build: IPA_VERSION_IS_GIT_SNAPSHOT checks if source directory is Git repo
  • 3dc5d2c Build: use POSIX 1003.1-1988 (ustar) file format for tar archives
  • a691b7d Build: IPA_VERSION_IS_GIT_SNAPSHOT re-generates version number on RPM build
  • 0023fb5 Build: add make srpms target
  • 8c81c6c Build: update IPA_VERSION_IS_GIT_SNAPSHOT to comply with PEP440
  • 961773b Build: pass down %{release} from SPEC to configure

master:

  • 636aaa7 Build: move client directory handling from SPEC to Makefile.am
  • 2091857 Build: move server directory handling from SPEC to Makefile.am
  • 1fa0ed9 Build: move web UI file installation from SPEC to Makefile.am
  • 5a53734 Build: document what should be in %install section of SPEC file

master:

  • 27e7a89 Build: workaround bug 1005235 related to Python paths in auto-generated Requires
  • 46b6b9e Build: support --enable-silent-rules for Python packages
  • 1cbd823 Build: enable silent build in makerpms.sh
  • d568372 Build: remove incorrect use of MAINTAINERCLEANFILES
  • e2060e8 Build: update makerpms.sh to use same paths as rpmbuild
  • 6b9977f Build: fix file dependencies for make-css.sh

master:

  • 28c5e12 Build: ignore rpmbuild for lint target

master:

  • 4842231 Build: use standard infrastructure for translations
  • d40c376 Build: support strip-po target for translations
  • 166257e Build: clean in po/ removes *~ files as well
  • 9ef5a7d Build: automatically generate list of files to be translated in configure
  • 8a79625 Build: integrate translation system tests again
  • 4c13383 Build: remove ipa.pot from Git as it can be re-generated at any time

master:

  • 132b475 Build: workaround bug while calling parallel make from rpmbuild

master:

  • e5686c0 Lowered the version of gettext

As far as I can tell the main refactoring work was done.
The last missing thing is client-only build which is tracked separately in ticket:6517.

master:

  • 1b85e59 spec file: do not define with_lint inside a comment

To be able to build rpms in an environment controlled by a FreeIPA domain we need to use a better tar format.
3dc5d2c change configure to use the ustar format, but that is not enough as it limits UID/GID size to values less than 2 million.

master:

  • 2bc01ec Use the tar Posix option for tarballs

master:

  • eb1f05d spec file: revert to the previous Release tag

Metadata Update from @pspacek:
- Issue assigned to pspacek
- Issue set to the milestone: FreeIPA 4.5
8 years ago

master:

  • 2a4f7f2 CONFIGURE: Improve detection of xmlrpc_c flags

Metadata Update from @mbasti:
- Issue close_status updated to: None (was: Fixed)
8 years ago

Log in to comment on this ticket.

Metadata
None
None
None
normal
None
None
None
None
refactoring
None
None
Build System
None
https://github.com/freeipa/freeipa/pull/155 https://github.com/freeipa/freeipa/pull/169 https://github.com/freeipa/freeipa/pull/132 https://github.com/freeipa/freeipa/pull/171 https://github.com/freeipa/freeipa/pull/213 https://github.com/freeipa/freeipa/pull/220 https://github.com/freeipa/freeipa/pull/226 https://github.com/freeipa/freeipa/pull/233 https://github.com/freeipa/freeipa/pull/236 https://github.com/freeipa/freeipa/pull/238 https://github.com/freeipa/freeipa/pull/260
None
None
None
None
None
None
http://www.freeipa.org/page/V4/Build_system_refactoring
None
Powered by Pagure 5.14.1
Documentation • File an Issue • About • SSH Hostkey/Fingerprint
© Red Hat, Inc. and others.