#1729 Enumerating large number of users makes sssd_be hog the cpu for a long time.
Closed: worksforme 4 years ago Opened 9 years ago by pbrezina.

https://bugzilla.redhat.com/show_bug.cgi?id=888739 (Red Hat Enterprise Linux 6)

Description of problem:
Enumerating large number of users makes sssd_be hog the cpu for a long time.

Version-Release number of selected component (if applicable):
1.9.2-46

How reproducible:
Always

Steps to Reproduce:
1. I have 30k users in ldap server.
2. Set enumerate=true in sssd.conf and restart sssd.
3. Check the cpu usage and try to login as a local user.

Actual results:
sssd_be consumes almost 99% of cpu forever(I observed for 20 mins)
 4850 root      20   0  387m 172m 3464 R 99.3 17.3   1:22.40 sssd_be

During this time, a simple login/logout of a local user took almost 2 mins:

# time su - testlocal
[testlocal@dhcp201-200 ~]$ logout

real    1m46.156s
user    0m0.007s
sys     0m0.013s


Expected results:
Other operations should not be slowed down during enumeration.

Additional info:

Fields changed

blockedby: =>
blocking: =>
coverity: =>
design: =>
design_review: => 0
feature_milestone: =>
fedora_test_page: =>
milestone: NEEDS_TRIAGE => SSSD 1.11 beta
testsupdated: => 0

Fields changed

milestone: SSSD 1.11 beta => SSSD 1.12 beta

Fields changed

cc: => jwm@horde.net
changelog: =>
review: => 0
selected: =>

This /might/ be helped by the improvements as scoped in ticket #2602 and because we might also add the syncrepl as part of #2062, we might also want to add an option to use syncrepl per-client for enumeration (not be default, though, that would trash the server).

However, enumeration is not the default and wouldn't work well with large environments anyway, so I think this is only a nice-to-have ticket for 1.14.

mark: => 0
milestone: SSSD 1.14 beta => SSSD 1.14 backlog
sensitive: => 0

Since the 1.14 branch is transitioning into maintenance mode and new functionality is being developed in master which will become 1.15 eventually, I'm mass-moving tickets from the 1.14 backlog milestone to the "Future releases" milestone.

milestone: SSSD 1.14 backlog => SSSD Future releases (no date set yet)

Metadata Update from @pbrezina:
- Issue set to the milestone: SSSD Future releases (no date set yet)

5 years ago

I'm putting the issue into the 1.16 milestone (aka the 'next upstream version') because I believe the performance enhancements that @sbose had been working on would also result in improved performance when it comes to enumeration and at the same time, the release that introduced the timestamp cache also introduced an enumeration regression, so the enhancements that the ts cache brought were shadowed by that regression.

Metadata Update from @jhrozek:
- Custom field design_review reset (from 0)
- Custom field mark reset (from 0)
- Custom field patch reset (from 0)
- Custom field review reset (from 0)
- Custom field sensitive reset (from 0)
- Custom field testsupdated reset (from 0)
- Issue close_status updated to: None
- Issue set to the milestone: SSSD 1.16.0 (was: SSSD Future releases (no date set yet))

5 years ago

Which regression do you mean?

Metadata Update from @lslebodn:
- Custom field design_review reset (from false)
- Custom field mark reset (from false)
- Custom field patch reset (from false)
- Custom field review reset (from false)
- Custom field sensitive reset (from false)
- Custom field testsupdated reset (from false)

5 years ago

The cleanup issue with the timestamp cache made testing everything enumeration-related too unreliable, so the downstream bug was dropped from 7.3. Look at the comments in the downstream bug for more details.

Metadata Update from @jhrozek:
- Custom field design_review reset (from false)
- Custom field mark reset (from false)
- Custom field patch reset (from false)
- Custom field review reset (from false)
- Custom field sensitive reset (from false)
- Custom field testsupdated reset (from false)

5 years ago

Metadata Update from @jhrozek:
- Custom field design_review reset (from false)
- Custom field mark reset (from false)
- Custom field patch reset (from false)
- Custom field review reset (from false)
- Custom field sensitive reset (from false)
- Custom field testsupdated reset (from false)
- Issue tagged with: performance

4 years ago

Since we are required to release a new upstream tarball no later than Friday Oct-20, I'm moving tickets that will not be closed by that date to the next milestone, 1.16.1

Metadata Update from @jhrozek:
- Custom field design_review reset (from false)
- Custom field mark reset (from false)
- Custom field patch reset (from false)
- Custom field review reset (from false)
- Custom field sensitive reset (from false)
- Custom field testsupdated reset (from false)
- Issue set to the milestone: SSSD 1.16.1 (was: SSSD 1.16.0)

4 years ago

The performance should be quite better with the enhancements done in 1.16.1. If you still seee unsatisfactory performance with 1.16.1, please open a new ticket.

Metadata Update from @jhrozek:
- Custom field design_review reset (from false)
- Custom field mark reset (from false)
- Custom field patch reset (from false)
- Custom field review reset (from false)
- Custom field sensitive reset (from false)
- Custom field testsupdated reset (from false)

4 years ago

Metadata Update from @jhrozek:
- Custom field design_review reset (from false)
- Custom field mark reset (from false)
- Custom field patch reset (from false)
- Custom field review reset (from false)
- Custom field sensitive reset (from false)
- Custom field testsupdated reset (from false)
- Issue close_status updated to: worksforme
- Issue status updated to: Closed (was: Open)

4 years ago

SSSD is moving from Pagure to Github. This means that new issues and pull requests
will be accepted only in SSSD's github repository.

This issue has been cloned to Github and is available here:
- https://github.com/SSSD/sssd/issues/2771

If you want to receive further updates on the issue, please navigate to the github issue
and click on subscribe button.

Thank you for understanding. We apologize for all inconvenience.

Login to comment on this ticket.

Metadata