In a replication session a changelog buffer is used to read key(csn)/value(changes) pairs from the changelog database using a cursor and the DB_MULTIPLE_KEY flag.
The intention is to reduce database access, but the drawback in an active server is that while processing the buffer changes may be received which are inserted at the beginning of the buffer. So the effect is that at teh next bufferload the buffer start is reset and many chaneg which were already sent are processed again and skipped.
With the fix for #48766 the buffer size used was limited and only if data didn't fit it was temporarily increased. Now, with the latest tests for reliab15, it showed that for large entries this can become inefficient as well, requiring many reallocs and buffer loads.
There is no optimal buffer size for all scenarios, so it would be best if the buffer size adapts to the effect of the last iteration.
I propose do do somthing like:
introduce two new fields, buf_record_cnt_last and buf_record_skipped_last start with an arbitary not too big buffer
in clcache_cursor_get do:
buf_processd = buf_buf_record_cnt - buf_buf_record_cnt_last buf_skipped = buf_record_skipped - buf_record_skipped_last if buf_processed < MIN ==> increase buffer if buf_skipped / buf_processed < XX % ==>increase (almost all entries were sent) if buf_skipped / buf_processed > yy % ==> reduce buffer (too many skipped)
Metadata Update from @lkrispen: - Issue set to the milestone: 1.3.6.0
Metadata Update from @mreynolds: - Custom field component reset (from Replication - General) - Issue close_status updated to: None - Issue set to the milestone: 1.3.7.0 (was: 1.3.6.0)
Metadata Update from @mreynolds: - Custom field component adjusted to None - Custom field reviewstatus adjusted to None - Issue set to the milestone: 1.4 backlog (was: 1.3.7.0) - Issue tagged with: RFE
389-ds-base is moving from Pagure to Github. This means that new issues and pull requests will be accepted only in 389-ds-base's github repository.
This issue has been cloned to Github and is available here: - https://github.com/389ds/389-ds-base/issues/2014
If you want to receive further updates on the issue, please navigate to the github issue and click on subscribe button.
subscribe
Thank you for understanding. We apologize for all inconvenience.
Metadata Update from @spichugi: - Issue close_status updated to: wontfix - Issue status updated to: Closed (was: Open)
Login to comment on this ticket.