#1720 Stop using in-memory hash tables for netgroups and autofs maps
Closed: Invalid 2 years ago by pbrezina. Opened 8 years ago by jhrozek.

We're using hash tables kept in-memory for processing netgroups and autofs maps. The primary reason for using them is that ldb doesn't guarantee ordering of results. The current tdb back end does, but if we switched to a different one, we'd be in trouble.

For both autofs and netgroups the API looks conceptually like this:

set_entry(name);
do {
  value = get_entry(name);
} while(value);
end_entry(name);

If we could retain the ordering of the results without the hash table, we would get rid of a lot of complexity and maybe even potential race conditions.


Fields changed

description: We're using hash tables kept in-memory for processing netgroups and autofs maps. The primary reason for using them is that ldb doesn't guarantee ordering of results. The current tdb back end does, but if we switched to a different one, we'd be in trouble.

For both autofs and netgroups the API looks conceptually like this:
{{{
set_entry(name);
do {
value = get_entry(name);
} while(value);
end_entry(value);
}}}

If we could retain the ordering of the results without the hash table, we would get rid of a lot of complexity and maybe even potential race conditions. => We're using hash tables kept in-memory for processing netgroups and autofs maps. The primary reason for using them is that ldb doesn't guarantee ordering of results. The current tdb back end does, but if we switched to a different one, we'd be in trouble.

For both autofs and netgroups the API looks conceptually like this:
{{{
set_entry(name);
do {
value = get_entry(name);
} while(value);
end_entry(name);
}}}

If we could retain the ordering of the results without the hash table, we would get rid of a lot of complexity and maybe even potential race conditions.

How big the table is usually?
Would a collection be an option?
It preservers the order.

Replying to [comment:2 dpal]:

How big the table is usually?

Depends on server data.

Would a collection be an option?

No, the reason to remove the hash table is to avoid duplicating this information in both ldb and another place. If we were to keep it in 2 place the hash table would be ok.

Fields changed

milestone: NEEDS_TRIAGE => SSSD 1.10 beta
summary: RFE: stop using in-memory hash tables for netgroups and autofs maps => [RFE] stop using in-memory hash tables for netgroups and autofs maps

Replying to [comment:5 dpal]:

Ticket has been cloned to Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=889210
Bug is closed because this is a task.

summary: [RFE] stop using in-memory hash tables for netgroups and autofs maps => Stop using in-memory hash tables for netgroups and autofs maps
type: enhancement => task

Fields changed

selected: => Not need

Moving tickets that are not a priority for SSSD 1.10 into the next release.

milestone: SSSD 1.10 beta => SSSD 1.11 beta

Fields changed

milestone: SSSD 1.11 beta => SSSD 1.10 beta
priority: major => minor

Fields changed

milestone: SSSD 1.10 beta => NEEDS_TRIAGE
review: => 0

Fields changed

milestone: NEEDS_TRIAGE => SSSD 1.12 beta

Fields changed

mark: => 0

Fields changed

changelog: =>
milestone: SSSD 1.13 beta => SSSD 1.13 backlog

Mass-moving tickets not planned for the next two releases.

Please reply with a comment if you disagree about the move..

milestone: SSSD 1.13 backlog => SSSD 1.15 beta

Metadata Update from @jhrozek:
- Issue set to the milestone: SSSD Future releases (no date set yet)

4 years ago

Metadata Update from @thalman:
- Custom field design_review adjusted to on (was: 0)
- Custom field mark adjusted to on (was: 0)
- Custom field patch adjusted to on (was: 0)
- Custom field review adjusted to on (was: 0)
- Custom field sensitive adjusted to on
- Custom field testsupdated adjusted to on (was: 0)
- Issue close_status updated to: Invalid
- Issue status updated to: Closed (was: Open)

2 years ago

The complexity was removed with cache_req switch.

Metadata Update from @pbrezina:
- Issue status updated to: Open (was: Closed)

2 years ago

Metadata Update from @pbrezina:
- Issue close_status updated to: Invalid
- Issue status updated to: Closed (was: Open)

2 years ago

SSSD is moving from Pagure to Github. This means that new issues and pull requests
will be accepted only in SSSD's github repository.

This issue has been cloned to Github and is available here:
- https://github.com/SSSD/sssd/issues/2762

If you want to receive further updates on the issue, please navigate to the github issue
and click on subscribe button.

Thank you for understanding. We apologize for all inconvenience.

Login to comment on this ticket.

Metadata