#49539 Add task to verify indexes
Opened 2 years ago by mreynolds. Modified 2 years ago

Issue Description

There are requests for the ability to "verify" an index is working as expected.

One way to do this would be to create a new Slapi Task that does an internal search, and verify the search was indexed or not indexed(notes=U/A), number of entries returned, etc.

Something like:

dn: cn=uid verify task, cn=verify index,cn=tasks,cn=confing
attrname: uid
indextpye: eq
search-base: dc=example,dc=com
search-scope: subtree
search-filter: (uid=mreynolds)
require-index: on
nentries: 1

It has also been requested this be built into the UI Monitoring section.

Metadata Update from @mreynolds:
- Custom field component adjusted to None
- Custom field origin adjusted to RHCust
- Custom field reviewstatus adjusted to None
- Custom field type adjusted to enhancement
- Custom field version adjusted to None

2 years ago

I'm curious about this because isn't this just "db2index but without actually doing the indexing?"

Why not just do db2index?

How would this actually check? Would it do a full id2entry scan vs an indexed lookup? That could be SUPER expensive ....

Maybe you have an idea in mind that I'm missing.

This is a customer request that management wants us to get into 1.4.0. So the customer had issues where a index was defined but was never actually indexed (something like that), so they got incorrect search results, or it was "unindexed" when it should of been "indexed". Nathan has more details from the customer.

Running db2index on a 10 million entry database will grind everything to halt for a "verification" test. We need something "simple". Basically a sanity test. Do a search that you know should be indexed, or do a search where we know the expected results, and make sure we get the expected results.

A full blown test would be using dbscan to look at all the id's in id2entry and then scanning indexes for each index type, not fun, and not in the scope of what they need.

Really this is a placeholder so we don't lose track of this RFE (however we decide to implement it). I was just throwing out a possible solution/approach in the ticket description, but there are many ways we could do this.

I wonder if the underlying problem isn't with index usage of indexes that are not reindexed.
in my memory,
If an index is added, but is not reindexed it should be flagged offline and not be used, but if the server is restarted this flag is lost and the index is strated to be updated and used for searches.

Ahhh so the issue is they ADDED and index but did not re-index, so when we try to lookup, we find empty.

I found this bug years ago :)

Perhaps we need something like "index state" where until it's been flagged as db2index once, we treat it as unindexed?

Login to comment on this ticket.