#50559 Do not index virtual attributes
Opened 2 months ago by vashirov. Modified a month ago

Issue Description

Recently I've seen a customer case where ldapsearch didn't return any results with the filter (nsrole=cn=nested_role,dc=example,dc=com). Turned out they had an index for nsrole, which is a virtual attribute. Similar situation can happen with CoS attributes: http://www.port389.org/docs/389ds/howto/howto-classofservice.html#indexing-limitations

We should prevent users from creating an index for virtual attributes or at least complain loudly in the logs about it.
For CoS attributes it might be tricky, but for nsrole and similar internal attributes we know for sure that are always virtual attributes, we should do it.


I agree, if we cannot manage an index to an attribute we should prevent creating it.

It might be difficult to catch all cases, but we should do wha tis feasable

Metadata Update from @lkrispen:
- Custom field origin adjusted to None
- Custom field reviewstatus adjusted to None

2 months ago

I think the obvious question is how do we internally determine what is and is not a virtual attribute?

It would be valid for cos to template virtual attributes on an OU for say mail, but then for mail to be real-attributes on another OU subtree?

So maybe there is something I missing here about the indexing process and being able to distinguish these two ....

I think the obvious question is how do we internally determine what is and is not a virtual attribute?

well, it could be determined by looking up the cos and role definition, or by configuring a list of non-index attributes, with defaults containing the known ones, eg nsrole - like Viktor suggested.

It would be valid for cos to template virtual attributes on an OU for say mail, but then for mail to be real-attributes on another OU subtree?

The index is per backend and if you have cos in one subtree then there will be no index, and bad luck for the other subtree. But with an index, the non-cos subtree would be fast, the other would also be fast but potentially incorrect - which is worse

So maybe there is something I missing here about the indexing process and being able to distinguish these two ....

The problem is that using an index for virtual attributes can give incorrect results, so any method to avoid this, even if incomplete, is better than what we have

I think something that has always frustrated me about LDAP is that attributes really have no distinction between real and virtual at a schema level - we simply can't know if an attribute is real or virtual because some people may also manually set nsrole as well etc.

I understand that indexing the virtual ones can give incorrect results, so I agree we should do something. Incorrect results really are not acceptable. I'm just concerned that by doing this we'll prevent indexing of real attributes also that should be indexed such as the provided subtree example. It would be easy to see a case where a subtree has cos generated cn or uid, that then causes uid to be un-indexed, which would lead to many issues given how common uid= queries are from something like SSSD.

So honestly, I think the current state is a problem, but I think that "not indexing" the attributes leads to potential more harmful cases in deployments, and surprising and non-obvious behaviour.

Because I can't be completely negative without offering solutions, a possibly-acceptable middle ground is that:

  • Any attribute in a cos template is not indexed
  • Indexes for the BE of an attribute in an existing cos template are removed and the removal is logged with reasons why (IE upgrade/existing template case)
  • Attempts to recreate these indexes are blocked/error message explaining the cos template issue, and possibly to advise split backend
  • Any attempt to create a cos template where an index exists for the attribute is also blocked/rejected.

Thoughts?

Because I can't be completely negative without offering solutions, a possibly-acceptable middle ground is that:

Any attribute in a cos template is not indexed
Indexes for the BE of an attribute in an existing cos template are removed and the removal is logged with reasons why (IE upgrade/existing template case)
Attempts to recreate these indexes are blocked/error message explaining the cos template issue, and possibly to advise split backend
Any attempt to create a cos template where an index exists for the attribute is also blocked/rejected.

Thoughts?

These four points sound reasonable , especially the last one is important, looking at the issue from the other end: if crating a cos for an indexed attr is rejected, it puts the responsibility to decide index vs cos on the admin

They also seem like good places to start with tests cases also I think.

Metadata Update from @mreynolds:
- Issue set to the milestone: 1.4.2

a month ago

Login to comment on this ticket.

Metadata