#51134 Ticket 51131 - improve mutex alloc in conntable
Closed by spichugi. Opened by firstyear.
firstyear/389-ds-base 51131-connlockinit  into  master

Download 51134.patch

Bug Description: We previously did delayed allocation
of mutexs, which @tbordaz noted can lead to high usage
of the pthread mutex init routines. This was done under
the conntable lock, as well as cleaning the connection

Fix Description: rather than delayed allocation, we
initialise everything at start up instead, which means
that while startup may have a delay, at run time we have
a smaller and lighter connection allocation routine,
that is able to release the CT lock sooner.

https://pagure.io/389-ds-base/issue/51131

Author: William Brown william@blackhats.net.au

Review by: ???

I noticed no change in performance during basic tests:

before: 35 passed, 3 skipped, 150 warnings in 160.68s (0:02:40)
after: 35 passed, 3 skipped, 150 warnings in 159.78s (0:02:39)

This change is too small to be significant, during a ldclt search it may show up.

LGTM but I'd like @tbordaz to have a look as well...

I was testing a similar patch, but instead of alloc on startup I was doing alloc on demand (when a connection is selected for the first time).

I agree with the overall fix but I have a doubt with the connection state. Connections are CONN_STATE_INIT. But it remains functions testing CONN_STATE_FREE, especially connection_table_as_entry and connection_table_move_connection_out_of_active_list). Aren't they impacted ?

Alloc on select will have similar issues because select is in a single thread. You're just putting the same resource contention into the same spot ....

In theory now with the changes, the CONN_STATE is a no-op and shouldn't be needed OR we can expand it to have more states because the current flagging system is really hard to understand. I didn't want to change too much at once.

Alloc on select will have similar issues because select is in a single thread. You're just putting the same resource contention into the same spot ....

Actually I meant it was allocated on demand but never freed unless at shutdown

In theory now with the changes, the CONN_STATE is a no-op and shouldn't be needed OR we can expand it to have more states because the current flagging system is really hard to understand. I didn't want to change too much at once.

I agree conn_state looks useless now. It only flags that the mutex/monitor have been initialized but with the patch they are initialized for all connections at startup. would you mind to remove that dead/useless code in the patch ?

It's not useless, I want to move some of the connection flagging into it, because at the moment the current flags and how we check what a connection is doing is really messy. But I didn't want to overload this patch.

Thanks for the explanation. The tests and the patch are looking good to me. ACK

@firstyear the patch looks good. Do you want to update it ?

rebased onto 7e1d80f63c0254ddec697f684057bb899b7e8aad

Pull-Request has been merged by firstyear

389-ds-base is moving from Pagure to Github. This means that new issues and pull requests
will be accepted only in 389-ds-base's github repository.

This pull request has been cloned to Github as issue and is available here:
- https://github.com/389ds/389-ds-base/issues/4187

If you want to continue to work on the PR, please navigate to the github issue,
download the patch from the attachments and file a new pull request.

Thank you for understanding. We apologize for all inconvenience.

Pull-Request has been closed by spichugi

Metadata