#1207 Improve behavior of LDAP paging control with servers with paging limits
Closed: Invalid None Opened 12 years ago by sgallagh.

This ticket has been split from ticket #1202 to address the more complicated parts of this fix.

We added support for the Simple Paged Search in ticket #658 and SSSD 1.5.8. When we did so, we chose to default to always requesting paged searches for all requests. However, it turns out that RFC 2696 does not mandate that the LDAP server supports more than one paging search at a time on the same connection. Some LDAP servers (notably 389 DS 1.2.9 and older) allow only a single paged search to be in progress on a request at one time. This results in receiving an error from LDAP that the server refuses to perform the operation. Once ticket #1201 is fixed, this will result in offline operation (right now it's misbehaving as if it received zero results). We need to avoid both of these events.

RFC 2696 does specify that if the result size being returned is smaller than the requested page size that the paging control should be ignored. This would mitigate the problem, but it appears that 389 DS also chooses to not ignore the control in this case.

In general, we need to find ways to minimize our usage of the paging control where available. I propose the following changes:

  1. Add a boolean option to {{{ldap_generic_search_send()}}} to specify whether to use the paging control (when available). -- done in Ticket #1202
  2. Do not request the paging control when we expect a single-valued return from the request.
  3. Always request the paging control for enumeration requests. -- done in Ticket #1202
  4. Serialize paging requests by maintaining a queue of lookups requiring paging and allow only one to operate at a time.{{{[1]}}}
  5. If it can be reasonably determined that replies will never exceed 500 entries, also omit the paging control{{{[2]}}}

{{{[1]}}}: This will likely introduce apparent performance issues when performing initgroups() requests, since they will no longer be able to run in parallel. We must always assume that the user may belong to many groups.

{{{[2]}}}: I'm not 100% sure about this one. It's possible for a server to manually set its page size smaller than 500, though I doubt very many would do so. On the other hand, I can't come up with any ideas offhand for lookups of greater than a single entry that could not therefore result in a potentially-infinite set of results. So this may be a moot point.


Setting 1.9 minor for now. If not completed in the 1.9.x timeframe, this should become 1.10 critical.

milestone: NEEDS_TRIAGE => SSSD 1.9.0
priority: major => minor

Fields changed

milestone: SSSD 1.9.0 => SSSD 1.10.0
priority: minor => critical

Fields changed

milestone: SSSD 1.10.0 => SSSD 1.10 beta

Capturing today's IRC discussion on the topic so it doesn't get lost:

- (12:50:10 PM) sgallagh: simo: At the end of the day, though, we need to figure out a real long-term solution for SSSD.
- (12:50:27 PM) simo: sgallagh: serializing paged request perhaps
- (12:50:33 PM) simo: not great for latency
- (12:50:45 PM) simo: but well better correct and slow than fast and broken :)
- (12:51:09 PM) sgallagh: simo: Maybe we could do a test on rootdse lookup where we see how many we can request at once before getting the error
- (12:51:37 PM) simo: sgallagh: maybe or simply have a conservative default value and allow an option in sssd.conf to increase it
- (12:51:53 PM) sgallagh: simo: The only acceptable conservative default is 1, unfortunately
- (12:52:05 PM) simo: a test may not work well if it depends on multiple concurrent clients
- (12:52:21 PM) sgallagh: simo: No, the limitation on this is per-LDAP-connection
- (12:52:27 PM) sgallagh: In SSSD we only initiate one
- (12:52:34 PM) simo: or we could catch the error and just retry and scale down
- (12:52:39 PM) simo: worst case we get to 1
- (12:53:08 PM) simo: we start with like 16 concurrent allowed
- (12:53:15 PM) sgallagh: simo: Maybe a sufficient answer on the short-term is to treat this error as a retry event that doesn't mark the server as down?
- (12:53:23 PM) simo: and if we get errors we half to 8 than 4 then 2 than 1
- (12:53:42 PM) simo: sgallagh: yeah that would also work, it is the same thing as serializing anyway
- (12:54:01 PM) simo: except we may not be fair
- (12:54:09 PM) simo: an ordered queue would be fair
- (12:54:15 PM) sgallagh: Yeah, good point
- (12:54:16 PM) simo: we just bounce a req backj though
- (12:54:26 PM) simo: however it should be ok for most cases
- (12:54:32 PM) simo: certainly better than what we have now

design: =>
design_review: => 0
fedora_test_page: =>

Fields changed

selected: => Not need

Moving tickets that are not a priority for SSSD 1.10 into the next release.

milestone: SSSD 1.10 beta => SSSD 1.11 beta

Fields changed

mark: => 0

So far moving to 1.13 backlog, but suspect moving even further, since this ticket hasn't been requested in years.

changelog: =>
milestone: SSSD 1.13 beta => SSSD 1.13 backlog
review: => 1

Fields changed

priority: critical => minor

Fields changed

priority: minor => trivial

Mass-moving tickets not planned for any immediate release and re-setting priority.

milestone: SSSD 1.13 backlog => SSSD Deferred
priority: trivial => major

Most of this work was done already, so I suggest we just close this ticket.

sensitive: => 0

Fields changed

resolution: => wontfix
status: new => closed

Metadata Update from @sgallagh:
- Issue assigned to sgallagh
- Issue marked as depending on: #1202
- Issue set to the milestone: SSSD Patches welcome

7 years ago

SSSD is moving from Pagure to Github. This means that new issues and pull requests
will be accepted only in SSSD's github repository.

This issue has been cloned to Github and is available here:
- https://github.com/SSSD/sssd/issues/2249

If you want to receive further updates on the issue, please navigate to the github issue
and click on subscribe button.

Thank you for understanding. We apologize for all inconvenience.

Login to comment on this ticket.

Metadata