#124 Startup with big amount of data in LDAP is slow
Opened 10 years ago by pspacek. Modified 7 years ago

DNS zones are not visible until initial synchronization between LDAP and bind-dyndb-ldap is done. The delay for 10000 records in DNSSEC signed zone on laptop machine can be ~ 0.5 minute.

Trivial benchmark from [list]]([https://www.redhat.com/archives/freeipa-devel/2013-May/msg00250.html|mailing):

Command: /usr/bin/time dnssec-signzone -n 1 -o example.net example.net
Signing was limited to single core (parameter -n 1).

Unsigned zone: 327 285 bytes, ~ 10 000 A records and several other records
Signed zone: 10 847 688 bytes
Results:
38.28user 0.09system 0:38.80elapsed 98%CPU (0avgtext+0avgdata 18032maxresident)k
0inputs+21200outputs (0major+4646minor)pagefaults 0swaps

Wow, it is pretty slow.

CPU: Intel(R) Core(TM) i7-2620M CPU @ 2.70GHz
Operating memory: 4 GB of DDR3 @ 1333 MHz

This can be solved partially solved by caching:

Design

BIND start-up sequence

  1. Load saved (cached) version of zones from filesystem if they are present.
    • This ensures quick start-up and saves computing power (think about re-creating of DNSSEC signatures).
    • Zone will not be served to clients if data in filesystem cache are older than X seconds (interval is specified by SOA expiry field in particular zone).
    • BIND will start with disabled dynamic updates. This ensures that no changes will get lost during initial database synchronization.
  2. Re-synchronize with LDAP: ticket:125
  3. Consistency between LDAP and local database is restored - enable dynamic updates.

Questions

How often should we save the cache from operating memory to disk?

  • On shutdown only?
  • On start-up (after initial synchronization) and on shutdown?
  • Periodically? How often? At the end of periodical re-synchronization?
  • Each N updates?
  • If N % of the database was changed? (pspacek's favorite)

Results of planning meeting held on 2014-11-04.

Results of planning meeting held on 2014-11-04.

Implementation requires additional information from meta-database - we need to store information where are cached data stored, last processed syncrepl cookie etc.

These tickets are not directly related to upcomming FreeIPA 4.4 features.

Moving tickets as triaged on 2016-02-11 with mkosek. These tickets are not going to be implemented without strong need/good justification.

I'm setting priority to 'critical' so these tickets stay on top of The Backlog bucket.

Metadata Update from @pspacek:
- Issue assigned to someone
- Issue set to the milestone: The Backlog

7 years ago

Login to comment on this ticket.

Metadata