Bug 1435663: deadlock between write and search operation in schema-compat plugin
Problem description:
Schema compat is a betxn pre/post op plugin managing maps.
The maps are protected by a RW lock.
A typical deadlock scenario is when a read thread (SRCH) holding
the map lock needs a ressource (DB page) acquired by an
write thread (MOD/ADD/DEL..) and that write thread needs to update
the map and so acquire the map lock.
So far we have been able to reduce the frequency of those scenarios
(but not eliminate them) by restricting the scope of operations
that need to acquire the lock.
But scoping is not systematically possible like described in
https://bugzilla.redhat.com/show_bug.cgi?id=1435663#c16
Fix description:
The fix implements a plugin RW reentrant lock 'plugin_lock'.
To do this it uses a thread private variables (thread_plugin_lock_status and
thread_plugin_lock_count to remember the current status of the lock
(free, read_acquired, write_acquired).
Then for write operations (even if it is out of the scope of SC), the lock is acquired in write
in the BE_preop operation (note not in BETXN_preop) and only released
in the BE_postop (postop/bepostop).
So a write operation acquires (exclusive) the maps lock before acquiring DB pages.
So read and write operation will acquire locks in the same order.
Design is https://www.freeipa.org/page/V4_slapi_nis_locking
Signed-off-by: Thierry Bordaz <tbordaz@redhat.com>