We should see if your build of bdb with tas disabled helps this situation.

Filter cache - any shared global data structure will need to have some sort of locking, or use some sort of lock-free API

new idl - yes, it is the optimal choice for most situations, especially if you have large ID lists

with tas disabled the performance did improve a lot, but the stacks remained almost the same (only without the __db_tas_mutex layer), so I thought it could be possible the get rid of these call.

Of course there would be some locking on an other level if don't find the lock free magic.

I would have thought to have a large ID list in one record would be better then multiple records with the same key

Replying to [comment:3 lkrispen]:

I would have thought to have a large ID list in one record would be better then multiple records with the same key

Not sure what you mean here. This is how the (new) ID lists work in 389: http://port389.org/wiki/Database_Architecture

I see the benefit for adding or deleting an id in the list. But in the db3 file we have:
db_dump -d a /var/lib/dirsrv/slapd-vpn1/db/example/givenName.db | grep =alexia
[238] 7120 len: 8 data: =alexia\0
[240] 7120 len: 8 data: =alexia\0
[242] 7120 len: 8 data: =alexia\0
[244] 7120 len: 8 data: =alexia\0
[246] 7120 len: 8 data: =alexia\0
[248] 7120 len: 8 data: =alexia\0
[250] 7120 len: 8 data: =alexia\0
[252] 7120 len: 8 data: =alexia\0
[254] 7120 len: 8 data: =alexia\0
[256] 7120 len: 8 data: =alexia\0
[258] 7120 len: 8 data: =alexia\0
[260] 7120 len: 8 data: =alexia\0
[262] 7120 len: 8 data: =alexia\0
[264] 7120 len: 8 data: =alexia\0
[266] 7120 len: 8 data: =alexia\0
so it blows up the btree pages and to retrieve the ids for the key alexia you need to access many records. DB_MULTIPLE helps, but think it could still be overhead

Ah, I see. In the case where you have very small ID list per key, but large numbers of values matching your search filter, then perhaps the new IDL implementation adds overhead.

389-ds-base is moving from Pagure to Github. This means that new issues and pull requests
will be accepted only in 389-ds-base's github repository.

This issue has been cloned to Github and is available here:
- https://github.com/389ds/389-ds-base/issues/540

If you want to receive further updates on the issue, please navigate to the github issue
and click on subscribe button.

Thank you for understanding. We apologize for all inconvenience.