#50547 Enable BDB MVCC mode (DB_MULTIVERSION)
Opened 11 months ago by firstyear. Modified 11 months ago

Issue Description

According to the official BDB documentation a flag called DB_MULTIVERSION exists which enables copy-on-write (aka multiversion concurrency control) in the DB. This is the same strategy that is used by LMDB to achieve high performance operation with large numbers of concurrent reads.

We should consider enabling this flag in the project, and attempting to understand if we have locking strategies in place that may interfer with this process (IE rwlocks).


+1, but we need performance tests first.

Metadata Update from @mhonek:
- Custom field origin adjusted to None
- Custom field reviewstatus adjusted to None
- Issue tagged with: Performance

11 months ago

I actually already have a patch, and I'm doing load tests but it's uncovered a segfault in libdb so I have to struggle with debuginfo now :) perhaps not as easy to enable as I had hoped.

I confirm Matus' request: we need performance data first. And in your PR you state: "However, we still have poor concurrency and locking strategies all over the code base, so many of the benefits of this may not be visible until other changes are made".

I don't see any benefit of introducing a change like this without having done the investigation of the mentioned problems in the server first and demonstrating a real impact on performance.

Also many people are on vacation right now, please don't push and give them a chance to review

I confirm Matus' request: we need performance data first. And in your PR you state: "However, we still have poor concurrency and locking strategies all over the code base, so many of the benefits of this may not be visible until other changes are made".

I'm doing performance tests and in the process found a segfault in libdb which I'm investigating. I always planned to provided proper numbers with this change.

I don't see any benefit of introducing a change like this without having done the investigation of the mentioned problems in the server first and demonstrating a real impact on performance.

Silly William thought it would be a quick change, but apparently a crash in libdb was awaiting me. I'm getting to a point where I may discontinue the investigation because of it as my time may be better elsewhere.

Also many people are on vacation right now, please don't push and give them a chance to review

I ... why ... ? I'm not sure I follow - I would never push code like this unless it has been reviewed by the team and accepted by them? Good to know people are on vacation - there certainly is no rush for this code at all.

Metadata Update from @mreynolds:
- Issue set to the milestone: 1.4 backlog

11 months ago

Login to comment on this ticket.

Metadata