#47490 Schema replication between DS versions may overwrite newer base schema
Closed: Fixed None Opened 5 years ago by nkinder.

The FreeIPA project has encountered problems with schema replication between DS versions. Schema replication is very basic, and can end up overwriting base schema from a newer DS with that of the older DS.

For details, see this FreeIPA ticket:

https://fedorahosted.org/freeipa/ticket/3778

Here is the current status

  • steps to reproduce

{{{
Install 1.2.11
Create 'master'

# craft the schema to be like 1.2.10
# so that 'nsSaslMapping' does NOT authorize 'nsSaslMapPriority'
stop 'master'
edit <instance>/schema/01core389.ldif
<remove 'MAY ( nsSaslMapPriority )' from 'nsSaslMapping' definition
start 'master'

#
# create a consumer with a schema where 'nsSaslMapping' authorizes 'nsSaslMapPriority'
#
Install 1.3.2
Create consumer
<initialize replication and the consumer>

# Entry update does not update the schema
create user t1
<check replication is working>
<check the consumer schema is unchanged
    ldapsearch ...-p <consumer>... -b "cn=schema" "(&(objectclass=ldapsubentry)(objectclass=subschema)" nscpentrywsi
>

# Now update the schema on the master
<add a dummy attributeType on master schema>

# Here the master schema
#  - contains the new attributeType with the related nsSchemaCSN
#  - 99user.ldif contains the new attributeType with the related nsSchemaCSN
#
# The consumer schema is unchanged

# propagate the schema to the consumer
Create user t2

# Here the consumer schema
#   - contains the new attributeType with the related nsSchemaCSN
#   - 99user.ldif contains the all the objectclasses/attributeTypes with the related nsSchemaCSN

}}}

  • My understand is that when deploying DS, masters schemas should be first copied on all topology instances. This is the reason why, when there is no nsSchemaCSN the suppliers assum the consumer schema is identical.
    When a schema is updated online, it will be propagated on the all topology. The updated schema should be the proper one because the version of DS is not checked at this point but only the nsSchemaCSN.

Here are the next steps

- Evaluate if it is possible to take into account the DS version when pushing the schema

ack

What about attribute types? Supplier can remove attributes on consumer. Supplier can change attribute definition to be "more restrictive" - change the syntax, change from multi-valued to single-valued, etc.

Also, this change is all or nothing. Either all of the schema is added, or none of it. Would it be possible to accept some schema changes but reject others?

Hi Rich,

Thanks for the review. I agree with you that the fix only address 'objectclasses' and is 'all or nothing' :-[

Regarding the 'attributeTypes', it is possible to check before pushing the schema that for a given attributes type
the consumer attribute type is not a superset of the supplier one. For example like you mentioned directory string
a superset of IA5 or DN syntax, multi-value vs single value,...
Should I do this in the same fix or open an other ticket for 'attributeTypes' ?

Regarding a partial push of the schema, I think it would require a global schema check after each push component.
For example:

{{{

Supplier 1              Supplier 2

OC_A
Must: A1, A2 Must: A1

OC_B
Supperior: OC_A Supperior: OC_A
Must: A3 Must: A3, A4

OC_C
Supperior: OC_B Supperior: OC_B
May: A4, A5 May: A5

OC_D
Supperior: OC_A Supperior: OC_A
Must: A10, A11 Must A10
}}}

Supplier 1 pushes its schema to Supplier 2. OC_B on S2 is a superset of S1, 
so OC_B should not be pushed. A limitation could be that any OC child of OC_B
should not be pushed. Else OC_C would have A4 as MAY but also as MUST (inherited from OC_B).
But I think this limitation is not suffisant.
A schema push S1->S2 would look like:

{{{
Supplier 1 Supplier 2
OC_A
Must: A1, A2 Must: A1, A2

OC_B
Supperior: OC_A Supperior: OC_A
Must: A3 Must: A3, A4

OC_C
Supperior: OC_B Supperior: OC_B
May: A4, A5 May: A5

OC_D
Supperior: OC_A Supperior: OC_A
Must: A10, A11 Must A10, A11
}}}

If a replication session S2->S1 occurs and S2 tries to push its schema.
this time OC_B on S1 is not a superset of OC_B on S2. That would mean

{{{
Supplier 1 Supplier 2
OC_A
Must: A1, A2 Must: A1, A2

OC_B
Supperior: OC_A Supperior: OC_A
Must: A3, A4 Must: A3, A4

OC_C
Supperior: OC_B Supperior: OC_B
May: A4, A5 May: A5

OC_D
Supperior: OC_A Supperior: OC_A
Must: A10, A11 Must A10, A11
}}}

Finally, S1 will be able to push OC_C on S2 and we will have A4 in OC_C both on
MAY and MUST on each server.

To prevent this, I think the global schema coherency should be checked as soon as
a component is pushed.

Ok, yes, partial schema push will really complicate matters. The immediate issue is with objectclass replication, so please open another ticket for the attribute types replication.

Also, make sure with the freeipa and dogtag team that your proposed patch will solve their problems.

Push in Master

git merge ticket47490
Updating bded301..9ca1e3c
Fast-forward
ldap/servers/plugins/replication/repl5_connection.c | 28 +++++++
ldap/servers/slapd/proto-slap.h | 2 +
ldap/servers/slapd/schema.c | 320 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-------
ldap/servers/slapd/slapi-private.h | 5 ++
4 files changed, 324 insertions(+), 31 deletions(-)

git push origin master
Enter passphrase for key '/home/tbordaz/.ssh/id_rsa_fedora':
Counting objects: 21, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (11/11), done.
Writing objects: 100% (11/11), 5.09 KiB, done.
Total 11 (delta 9), reused 0 (delta 0)
To ssh://git.fedorahosted.org/git/389/ds.git
bded301..71151cf master -> master

commit 71151cf
Author: Thierry bordaz (tbordaz) tbordaz@redhat.com
Date: Wed Sep 18 15:25:41 2013 +0200

IPA team successfully tested on F19 and verified that they hit the fix with the logs messages:
[08/Oct/2013:09:30:15 -0400] schema - Local objectClasses must not be overwritten (set replication log for additional info)

Push in 1.3.2 branch for 1.3.2.1 milestone

git push origin 389-ds-base-1.3.2
Counting objects: 21, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (11/11), done.
Writing objects: 100% (11/11), 5.09 KiB, done.
Total 11 (delta 9), reused 0 (delta 0)
To ssh://git.fedorahosted.org/git/389/ds.git
ba3fb44..6473608 389-ds-base-1.3.2 -> 389-ds-base-1.3.2

commit 6473608
Author: Thierry bordaz (tbordaz) tbordaz@redhat.com
Date: Wed Sep 18 15:25:41 2013 +0200

git merge ticket47490
Updating 0e0848a..73bbaa7
Fast-forward
dirsrvtests/tickets/ticket47490_test.py | 117 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-------------
1 file changed, 98 insertions(+), 19 deletions(-)

git push origin master
Counting objects: 9, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (5/5), done.
Writing objects: 100% (5/5), 2.01 KiB, done.
Total 5 (delta 3), reused 0 (delta 0)
To ssh://git.fedorahosted.org/git/389/ds.git
0e0848a..73bbaa7 master -> master

commit 73bbaa7
Author: Thierry bordaz (tbordaz) tbordaz@redhat.com
Date: Fri Nov 28 19:11:54 2014 +0100

Metadata Update from @tbordaz:
- Issue assigned to tbordaz
- Issue set to the milestone: 1.3.2.1

2 years ago

Login to comment on this ticket.

Metadata