symboldb

Created 5 years ago
Maintained by fweimer
Symbol extraction from RPM packages for ELF, Python, and Java
Members 1
Prerequisites
─────────────

In addition to the usual C++ build environment, you need the following
development packages:

- cmake
- createrepo
- curl-devel
- diffutils
- elfutils-devel
- elfutils-libelf-devel
- expat-devel
- gawk (for /usr/bin/awk)
- libarchive-devel
- nss-devel
- openssl-devel
- postgresql-contrib
- postgresql-devel
- postgresql-server
- publican
- python
- python3
- rpm-devel
- vim-common (for /usr/bin/xxd)
- xmlto
- zlib-devel

Note that elfutils version 0.159 or later is required.

xmlto and publican are optional, but if they are missing, the
documentation will not be built.  A missing python3 package (and
python3 executable) will lead to some test failures.

Building
────────

Create a build directory, run cmake and make:

  mkdir build
  cd build
  cmake ..
  make
  cd ..
  build/runtests

Alternatively, you can use the Ninja build tool:

  mkdir build
  cd build
  cmake -G Ninja -DCMAKE_MAKE_PROGRAM=ninja-build ..
  ninja-build
  cd ..
  build/runtests

Running
───────

╔════════════════════════════════════════════════════════════════════╗
║ NOTE: The database schema is still evolving.  I do not provide     ║
║ automated schema migration support at this stage because I do not  ║
║ assume that there are any users of this code.  Please drop me a    ║
║ note (at <fweimer@redhat.com>) if you use this software, and I     ║
║ will be more careful about database schema changes.                ║
╚════════════════════════════════════════════════════════════════════╝

symboldb honors the PGUSER, PGDATABASE etc. variables generally used
by PostgreSQL applications.  If you want to change the defaults, you
have to set those environment variables.  The database schema is
currently hard-coded to "symboldb".

To create a database for your user account (named "USER"), run the
following commands as the "postgres" user:

  createuser USER
  createdb --lc-collate=C USER
  psql -c "CREATE EXTENSION pg_trgm" USER

If the final command files with an error message, your system likely
lacks the postgresql-contrib package.

The symboldb SQL schema contains COLLATE "C" annotations in many
places, so re-using an existing database with another collation scheme
is possible without a performance loss.

If you plan to load large repositories (such as Fedora releases), you
need to tune your PostgreSQL installation.  At least the very least,
increase the "shared_buffers" and "checkpoint_segments" values.

Afterwards, as "USER", run "psql" to check if you can access the
database.  The following commands have to be invoked as "USER", too.

This command creates the "symboldb" database schema:

    symboldb --create-schema

This adds RPM files to the database, skipping already imported RPMs
(based on the content hash):

    symboldb --load-rpm *.rpm

Multiple load commands can be executed in parallel, but imports will
occasionally fail (due to an aborted database transaction) if the same
RPM file is loaded concurrently.

RPMs from a set of of composes can be grouped into package set using:

    symboldb --create-set=Fedora/rawhide/x86_64 \
      http://download.fedoraproject.org/pub/fedora/linux/development/rawhide/x86_64/os/

To actually load the RPMs in these repositories, use this command

    symboldb --update-set Fedora/rawhide/x86_64

This only downloads the latest version for each package
name/architecture combination.  RPMs are cached in a directory under
~/.cache/symboldb by default.

A single package set can cover multiple architectures.  The name of
the package set, Fedora/rawhide/x86_64 in the example, just follows a
naming convention and is not interpreted by the tool.

A symboldb.1 manual page will be written to the build directory.
Examples for SQL queries are found in the doc/examples directory.


  -- Florian Weimer <fweimer@redhat.com>