#48252 db2index creates index entry from deleted records
Closed: wontfix None Opened 5 years ago by tscherf.


1) Create the entry

# ldapadd -cvvvv -D "cn=Directory Manager" -W -f /root/ldif/create-posix-user2.ldif
ldap_initialize( <DEFAULT> )
add objectClass:
        top
        account
        posixAccount
        shadowAccount
add cn:
        Non Secure User
add uid:
        user80
add uidNumber:
        80
add gidNumber:
        80
add homeDirectory:
        /home/insecure
add loginShell:
        /bin/bash
add userpassword:
        {CLEAR}redhat
adding new entry "uid=user80,ou=People,dc=coe,dc=muc,dc=redhat,dc=com"
modify complete

2) Check the index file

# dbscan -f /var/lib/dirsrv/slapd-tscherf54/db/userRoot/gidNumber.db4 |grep "=80"
=80

3) Delete the entry

# ldapdelete -vx -D "cn=Directory Manager" -W "uid=user80,ou=People,dc=coe,dc=muc,dc=redhat,dc=com"
ldap_initialize( <DEFAULT> )
deleting entry "uid=user80,ou=People,dc=coe,dc=muc,dc=redhat,dc=com"

4) Verify the value from deleted entry has been removed from index

# dbscan -f /var/lib/dirsrv/slapd-tscherf54/db/userRoot/gidNumber.db4 |grep "=80"
#

5) Run db2index again

# /usr/lib64/dirsrv/slapd-tscherf54/db2index.pl  -n userRoot -D "cn=Directory Manager" -W -t gidNumber
adding new entry "cn=db2index_2015_8_19_10_34_41, cn=index, cn=tasks, cn=config"

6) Value from removed entry shows up in index file again

# dbscan -f /var/lib/dirsrv/slapd-tscherf54/db/userRoot/gidNumber.db4 |grep "=80"
=80

7) ldapsearch fails to find the removed entry and there is also nothing related in the error log:

ldapsearch -x gidNumber=80

extended LDIF

LDAPv3

base <dc=coe,dc=muc,dc=redhat,dc=com> (default) with scope subtree

filter: gidNumber=80

requesting: ALL

search result

search: 2
result: 0 Success

numResponses: 1


I tried to verify the issue on a single master system but I failed. The issue is only there when tombstone entries exists in the database. When I enable the USN-Plugin on a single master system, the issue pops up again.

Replying to [comment:1 tscherf]:

I tried to verify the issue on a single master system but I failed. The issue is only there when tombstone entries exists in the database. When I enable the USN-Plugin on a single master system, the issue pops up again.

Please note that USN plugin turns an entry to a tombstone entry to store the USN value in the deleted entry even if the server is a standalone server. Thus, the behaviour sounds consistent.

Replying to [comment:2 nhosoi]:

Replying to [comment:1 tscherf]:

I tried to verify the issue on a single master system but I failed. The issue is only there when tombstone entries exists in the database. When I enable the USN-Plugin on a single master system, the issue pops up again.

Please note that USN plugin turns an entry to a tombstone entry to store the USN value in the deleted entry even if the server is a standalone server. Thus, the behaviour sounds consistent.

Right, that was the reason why I enabled the plugin to confirm the issue has to do with tombstone records in the changelog.

I will now clone this bug into BZ since I also have a customer reporting this issue.

the fix now completely skips indexing of attributes except tombstonecsn and rdn-
But don't we need the (objectclass=nstombstone) index ?

Replying to [comment:11 lkrispen]:

the fix now completely skips indexing of attributes except tombstonecsn and rdn-
But don't we need the (objectclass=nstombstone) index ?

Thanks, Ludwig. Indeed. The special treatment is needed...
$ ldapsearch -LLLx ... -b "dc=example,dc=com" "(objectclass=nstombstone)" dn
dn: nsuniqueid=ed2d6596-478711e5-9d10cc15-0a6ddd94,uid=tuser,ou=People,dc=example,dc=com

$ db2index.pl -t objectclass ...

$ ldapsearch ... -b "dc=example,dc=com" "(objectclass=nstombstone)" dn <== returns none. :(
$

Reviewed by Mark (Thank you!!)

Attachment 0001-Ticket-48252-lib389-adding-get_bin_dir-and-dbscan.patch​ added
git patch file (master) -- lib389
Pushed to master:
2f78164..c48301a master -> master
commit c48301a63c1ea13ebd904e4a30a71fa53165d0da

Attachment 0001-Ticket-48252-db2index-creates-index-entry-from-delet.patch​ added
git patch file (master)
Attachment 0002-Ticket-48252-CI-test-added-test-cases-for-ticket-482.patch​ added
git patch file (master) -- CI test
Pushed to master:
2dbbb9d..b011ae3 master -> master
commit ca4d38d
commit b011ae3

Pushed to 389-ds-base-1.3.4:
626f2d7..2231ee4 389-ds-base-1.3.4 -> 389-ds-base-1.3.4
commit 86b9627
commit 2231ee4

Pushed to 389-ds-base-1.3.3:
7f63b58..ef0a295 389-ds-base-1.3.3 -> 389-ds-base-1.3.3
commit ef0a295

Note: the patch cannot be applied to 1.2.11 directly. Backport is needed.

Reviewed by Rich (Thank you!!)

Pushed to master:
ee96ec7..4277f2b master -> master
commit 4277f2b

Pushed to 389-ds-base-1.3.4:
326f73f..8c9c2fc 389-ds-base-1.3.4 -> 389-ds-base-1.3.4
commit 8c9c2fc

Pushed to 389-ds-base-1.3.3:
90cf9fd..f10e2e6 389-ds-base-1.3.3 -> 389-ds-base-1.3.3
commit f10e2e6

Pushed to 389-ds-base-1.2.11:
1e2793d..6a1f2f0 389-ds-base-1.2.11 -> 389-ds-base-1.2.11
commit 6a1f2f0

Metadata Update from @nhosoi:
- Issue assigned to nhosoi
- Issue set to the milestone: 1.2.11.33

4 years ago

389-ds-base is moving from Pagure to Github. This means that new issues and pull requests
will be accepted only in 389-ds-base's github repository.

This issue has been cloned to Github and is available here:
- https://github.com/389ds/389-ds-base/issues/1583

If you want to receive further updates on the issue, please navigate to the github issue
and click on subscribe button.

Thank you for understanding. We apologize for all inconvenience.

Metadata Update from @spichugi:
- Issue close_status updated to: wontfix (was: Fixed)

9 months ago

Login to comment on this ticket.

Metadata