#47437 Some attributes in cn=config should not be multivalued
Closed: Fixed None Opened 6 years ago by rmeggins.

Ticket was cloned from Red Hat Bugzilla (product Red Hat Enterprise Linux 7): Bug 982597

Description of problem:
Some attributes in cn=config are multivalued when they clearly should not.
Examples of such attributes are nsslapd-require-secure-binds,
nsslapd-disk-monitoring and nsslapd-disk-monitoring-threshold.

Version-Release number of selected component (if applicable):
389-ds-base-1.2.11.15-16.el6_4.x86_64

How reproducible:
always

Steps to Reproduce:

1. On fresh instance:
[jrusnack@dstet 6.0]$ grep nsslapd-disk-monitoring
/etc/dirsrv/slapd-dstet/dse.ldif
[jrusnack@dstet 6.0]$ ldapsearch -D "cn=directory manager" -w Secret123 -b
"cn=config " -s base -LLL nsslapd-disk-monitoring
dn: cn=config
nsslapd-disk-monitoring: off

2. Try replace, works OK:

[jrusnack@dstet 6.0]$ ldapmodify  -h localhost -p 389 -D "cn=directory manager"
-w Secret123 <<EOF
dn: cn=config
changetype: modify
replace: nsslapd-disk-monitoring
nsslapd-disk-monitoring: on
EOF
modifying entry "cn=config"

[jrusnack@dstet 6.0]$ ldapsearch -D "cn=directory manager" -w Secret123 -b
"cn=config " -s base -LLL nsslapd-disk-monitoring
dn: cn=config
nsslapd-disk-monitoring: on

[jrusnack@dstet 6.0]$ grep nsslapd-disk-monitoring
/etc/dirsrv/slapd-dstet/dse.ldif
nsslapd-disk-monitoring: off

3. Try adding value (succeeds, now there are two values):
[jrusnack@dstet 6.0]$ ldapmodify  -h localhost -p 389 -D "cn=directory manager"
-w Secret123 <<EOF
dn: cn=config
changetype: modify
add: nsslapd-disk-monitoring
nsslapd-disk-monitoring: on
EOF

modifying entry "cn=config"

[jrusnack@dstet 6.0]$ ldapsearch -D "cn=directory manager" -w Secret123 -b
"cn=config " -LLL -s base nsslapd-disk-monitoring
dn: cn=config
nsslapd-disk-monitoring: on

[jrusnack@dstet 6.0]$ grep nsslapd-disk-monitoring
/etc/dirsrv/slapd-dstet/dse.ldif
nsslapd-disk-monitoring: off
nsslapd-disk-monitoring: on

4. Try adding value again:
[jrusnack@dstet 6.0]$ ldapmodify  -h localhost -p 389 -D "cn=directory manager"
-w Secret123 <<EOF
dn: cn=config
changetype: modify
add: nsslapd-disk-monitoring
nsslapd-disk-monitoring: on
EOF
modifying entry "cn=config"
ldap_modify: Type or value exists (20)


Actual results:
It is possible to set two values for some attributes in cn=config.

Expected results:
Boolean attributes like nsslapd-disk-monitoring and numerical attributes like
nsslapd-disk-monitoring-threshold should be single-valued and any other value
should be refused.

Additional info:
Complete list of affected attributes is not known - requires investigation.

There are 133 configuration attributes that do not have schema definitions. Most of these should be single valued.

Is this going to cause problems with schema replication with earlier versions?

Replying to [comment:7 rmeggins]:

Is this going to cause problems with schema replication with earlier versions?

Only if older versions misused their config attributes. How far back should I test? 1.2.9? 1.2.10? ...

Replying to [comment:8 mreynolds]:

Replying to [comment:7 rmeggins]:

Is this going to cause problems with schema replication with earlier versions?

Only if older versions misused their config attributes. How far back should I test? 1.2.9? 1.2.10? ...

The reason I ask is that many times when we change schema it causes problems with replication.

The earliest supported release is 1.2.11

Replying to [comment:9 rmeggins]:

Replying to [comment:8 mreynolds]:

Replying to [comment:7 rmeggins]:

Is this going to cause problems with schema replication with earlier versions?

Only if older versions misused their config attributes. How far back should I test? 1.2.9? 1.2.10? ...

The reason I ask is that many times when we change schema it causes problems with replication.

Just curious, what kinds of problems have you seen?

In this case, I'm not changing any pre-existing schema/objectclasses, just adding new attributes. I would think it wouldn't pose much risk, but we'll see. I'm going to test replication next...

The earliest supported release is 1.2.11

Replying to [comment:10 mreynolds]:

Replying to [comment:9 rmeggins]:

Replying to [comment:8 mreynolds]:

Replying to [comment:7 rmeggins]:

Is this going to cause problems with schema replication with earlier versions?

Only if older versions misused their config attributes. How far back should I test? 1.2.9? 1.2.10? ...

The reason I ask is that many times when we change schema it causes problems with replication.

Just curious, what kinds of problems have you seen?

If I remember correctly, we have had a few tickets around problems with schema compatability when replicating with older versions. I seem to recall problems with ipa and/or dogtag in this area.

In this case, I'm not changing any pre-existing schema/objectclasses, just adding new attributes. I would think it wouldn't pose much risk, but we'll see. I'm going to test replication next...

The earliest supported release is 1.2.11

My understanding is that since https://fedorahosted.org/389/attachment/ticket/47541/0001-Ticket-47541-Replication-of-the-schema-may-overwrite.patch there is a control before pushing/accepting a schema if the attributes in the consumer schema are not superset of the supplier schema.

If we add those definitions in master, I would expect then a master instance acting as a consumer should refuse the update of its schema from a 1.3.2, 1.3.1 or 1.2.11 where those attributes are not defined.
If this is the case, then we can probably workaround this by adding "schemaUpdateAttributeAccept: <attribute>" in the entry 'dn: cn=consumerUpdatePolicy,cn=replSchema,cn=config'

Replying to [comment:12 tbordaz]:

My understanding is that since https://fedorahosted.org/389/attachment/ticket/47541/0001-Ticket-47541-Replication-of-the-schema-may-overwrite.patch there is a control before pushing/accepting a schema if the attributes in the consumer schema are not superset of the supplier schema.

If I'm correct this patch is only in 1.3.3

If we add those definitions in master, I would expect then a master instance acting as a consumer should refuse the update of its schema from a 1.3.2, 1.3.1 or 1.2.11 where those attributes are not defined.

Can you clarify this? You mean a replica is a dedicated consumer, and the other a single master? Or you simply mean two masters?

If I stop a server, update the schema (the new 01core389.ldif), and start it up, why wouldn't it replicate to the other masters?

Anyway I just got some beaker boxes setup, so I want to make sure I am testing all of this correctly. Also, it looks like you haven't checked in 47676 into master(which is where 47541 resides). Any pans to do so?

If this is the case, then we can probably workaround this by adding "schemaUpdateAttributeAccept: <attribute>" in the entry 'dn: cn=consumerUpdatePolicy,cn=replSchema,cn=config'

I just added 135 new attributetypes, I would hope adding all of these attrs would not be necessary. We'll see.

Thanks!

Hi Mark,

You are right my update was confusing, because I used 'master' to name the branch not the role of the replica.

My understanding is that if you add new attributes in master branch. Then the master branch will contain 47541, 47676 (that I pushed yesterday) and 47437 (this ticket). A replica (from the master branch) will have those 135 new attributes types (47437). If this replica acts as a consumer and a supplier (let's say from 1.3.1) try to update the schema through replication, then I suspect 47541 to prevent this update. In fact the schema from the supplier (1.3.1) will be detected as a subset of the schema from the consumer (because of those attribute).

I agree it worth testing.

regards

Replying to [comment:15 tbordaz]:

Hi Mark,

You are right my update was confusing, because I used 'master' to name the branch not the role of the replica.

My understanding is that if you add new attributes in master branch. Then the master branch will contain 47541, 47676 (that I pushed yesterday) and 47437 (this ticket). A replica (from the master branch) will have those 135 new attributes types (47437). If this replica acts as a consumer and a supplier (let's say from 1.3.1) try to update the schema through replication, then I suspect 47541 to prevent this update. In fact the schema from the supplier (1.3.1) will be detected as a subset of the schema from the consumer (because of those attribute).

I agree it worth testing.

Ok, almost everything seems to work fine. I tested these scenarios:

1.3.3 --> 1.3.3

 - Stopped server, replaced 01core389.ldif, started server.  Made a schema change, and updated an entry -> schema replicated to the other 1.3.3 server.

1.3.3 --> 1.3.1

 - 1.3.3 already has the new schema, but 1.3.1 does not.  Made a schema change, and updated an entry -> schema replicated to 1.3.1

1.3.1 --> 1.3.3

 - 1.3.1 already has the new schema, but 1.3.3 does not.  Made schema change on 1.3.1, and then updated a entry.  Schema replicated to 1.3.3.

1.3.3 --> 1.2.11

 - 1.3.3 already has the new schema, but 1.2.11 does not..  Made schema change to 1.3.3, and updated an entry -> schema replicated to 1.2.11

1.2.11 --> 1.3.3

 - 1.2.11 already has the new schema, but 1.3.3 does not.  Made schema change on 1.2.11, and then updated an entry.  Schema replicated to 1.3.3.

These two do not work, but I think this is the expected result:

1.3.1 <--> 1.3.3

 - 1.3.1 already has new schema, but 1.3.3 does not.  Make a schema update on 1.3.3, and then update an entry on 1.3.3.  Schema replicates breaks.

1.2.11 <--> 1.3.3

 - 1.2.11 already has new schema, but 1.3.3 does not.  Make a schema update on 1.3.3, and then update an entry on 1.3.3.  Schema replication breaks.

regards

Replying to [comment:16 mreynolds]:

Replying to [comment:15 tbordaz]:

Hi Mark,

You are right my update was confusing, because I used 'master' to name the branch not the role of the replica.

My understanding is that if you add new attributes in master branch. Then the master branch will contain 47541, 47676 (that I pushed yesterday) and 47437 (this ticket). A replica (from the master branch) will have those 135 new attributes types (47437). If this replica acts as a consumer and a supplier (let's say from 1.3.1) try to update the schema through replication, then I suspect 47541 to prevent this update. In fact the schema from the supplier (1.3.1) will be detected as a subset of the schema from the consumer (because of those attribute).

I agree it worth testing.

Ok, almost everything seems to work fine. I tested these scenarios:

Below, when you say "schema replicated to the other server", does that include the new single valued attribute schema, or just the one single change you made? If the former, what happens if by some mis-configuration the customer has more than one value in one of these attributes? Does this cause their server not to be able to be restarted?

I found one attribute that does not appear to be single valued:
{{{
attributeTypes: ( 2.16.840.1.113730.3.1.2268 NAME 'nsslapd-accesslog-list' DESC 'Netscape defined attribute type' SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 SINGLE-VALUE X-ORIGIN 'Netscape Directory Server' )
}}}

This is defined in libglobs.c as CONFIG_CHARRAY, which I believe is represented as a multi-valued attribute in cn=config. I believe any CONFIG_CHARRAY or CONFIG_SPECIAL_REFERRALLIST is a multi-valued attribute. Can you double check?

1.3.3 --> 1.3.3

 - Stopped server, replaced 01core389.ldif, started server.  Made a schema change, and updated an entry -> schema replicated to the other 1.3.3 server.

1.3.3 --> 1.3.1

 - 1.3.3 already has the new schema, but 1.3.1 does not.  Made a schema change, and updated an entry -> schema replicated to 1.3.1

1.3.1 --> 1.3.3

 - 1.3.1 already has the new schema, but 1.3.3 does not.  Made schema change on 1.3.1, and then updated a entry.  Schema replicated to 1.3.3.

1.3.3 --> 1.2.11

 - 1.3.3 already has the new schema, but 1.2.11 does not..  Made schema change to 1.3.3, and updated an entry -> schema replicated to 1.2.11

1.2.11 --> 1.3.3

 - 1.2.11 already has the new schema, but 1.3.3 does not.  Made schema change on 1.2.11, and then updated an entry.  Schema replicated to 1.3.3.

These two do not work, but I think this is the expected result:

1.3.1 <--> 1.3.3

 - 1.3.1 already has new schema, but 1.3.3 does not.  Make a schema update on 1.3.3, and then update an entry on 1.3.3.  Schema replicates breaks.

1.2.11 <--> 1.3.3

 - 1.2.11 already has new schema, but 1.3.3 does not.  Make a schema update on 1.3.3, and then update an entry on 1.3.3.  Schema replication breaks.

regards

Replying to [comment:17 rmeggins]:

Replying to [comment:16 mreynolds]:

Replying to [comment:15 tbordaz]:

Hi Mark,

You are right my update was confusing, because I used 'master' to name the branch not the role of the replica.

My understanding is that if you add new attributes in master branch. Then the master branch will contain 47541, 47676 (that I pushed yesterday) and 47437 (this ticket). A replica (from the master branch) will have those 135 new attributes types (47437). If this replica acts as a consumer and a supplier (let's say from 1.3.1) try to update the schema through replication, then I suspect 47541 to prevent this update. In fact the schema from the supplier (1.3.1) will be detected as a subset of the schema from the consumer (because of those attribute).

I agree it worth testing.

Ok, almost everything seems to work fine. I tested these scenarios:

Below, when you say "schema replicated to the other server", does that include the new single valued attribute schema, or just the one single change you made?

Once the "schema is replicated", if you look at the user99.ldif on the receiving end it has the converged set of schema, including the update that triggered it. So both server's schemas(from the cn=schema viewpoint) are identical.

If the former, what happens if by some mis-configuration the customer has more than one value in one of these attributes? Does this cause their server not to be able to be restarted?

Probably, if it is a configuration attribute is under cn=config. I will test that as well.

I found one attribute that does not appear to be single valued:
{{{
attributeTypes: ( 2.16.840.1.113730.3.1.2268 NAME 'nsslapd-accesslog-list' DESC 'Netscape defined attribute type' SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 SINGLE-VALUE X-ORIGIN 'Netscape Directory Server' )
}}}

This is defined in libglobs.c as CONFIG_CHARRAY, which I believe is represented as a multi-valued attribute in cn=config. I believe any CONFIG_CHARRAY or CONFIG_SPECIAL_REFERRALLIST is a multi-valued attribute. Can you double check?

Ok, I thought I checked them all, but I obviously missed one(of 135). I will double the list, and make sure everything is correct.

1.3.3 --> 1.3.3

 - Stopped server, replaced 01core389.ldif, started server.  Made a schema change, and updated an entry -> schema replicated to the other 1.3.3 server.

1.3.3 --> 1.3.1

 - 1.3.3 already has the new schema, but 1.3.1 does not.  Made a schema change, and updated an entry -> schema replicated to 1.3.1

1.3.1 --> 1.3.3

 - 1.3.1 already has the new schema, but 1.3.3 does not.  Made schema change on 1.3.1, and then updated a entry.  Schema replicated to 1.3.3.

1.3.3 --> 1.2.11

 - 1.3.3 already has the new schema, but 1.2.11 does not..  Made schema change to 1.3.3, and updated an entry -> schema replicated to 1.2.11

1.2.11 --> 1.3.3

 - 1.2.11 already has the new schema, but 1.3.3 does not.  Made schema change on 1.2.11, and then updated an entry.  Schema replicated to 1.3.3.

These two do not work, but I think this is the expected result:

1.3.1 <--> 1.3.3

 - 1.3.1 already has new schema, but 1.3.3 does not.  Make a schema update on 1.3.3, and then update an entry on 1.3.3.  Schema replicates breaks.

1.2.11 <--> 1.3.3

 - 1.2.11 already has new schema, but 1.3.3 does not.  Make a schema update on 1.3.3, and then update an entry on 1.3.3.  Schema replication breaks.

regards

I tried the following TC. Install M1 (master branch 1.3.3) and M2 (1.3.1). Then update the schema on M2 and add an entry on M2.

The entry is replication. Fine. But the schema not with errors like:

{{{
[10/Feb/2014:13:53:34 +0100] schema - Attribute sslVersionMin is not allowed in 'nsEncryptionConfig' of the remote supplier schema
[10/Feb/2014:13:53:34 +0100] schema - Attribute sslVersionMax is not allowed in 'nsEncryptionConfig' of the remote supplier schema
[10/Feb/2014:13:53:34 +0100] schema - Fail to retrieve in the remote supplier schema [nsSchemaPolicy or 2.16.840.1.113730.3.2.328]
[10/Feb/2014:13:53:34 +0100] schema - Attribute nsRoleScopeDN is not allowed in 'nsRoleDefinition' of the remote supplier schema
[10/Feb/2014:13:53:34 +0100] schema - Attribute winSyncDirectoryFilter is not allowed in 'nsDSWindowsReplicationAgreement' of the remote supplier schema
[10/Feb/2014:13:53:34 +0100] schema - Attribute winSyncWindowsFilter is not allowed in 'nsDSWindowsReplicationAgreement' of the remote supplier schema
[10/Feb/2014:13:53:34 +0100] schema - Attribute winSyncSubtreePair is not allowed in 'nsDSWindowsReplicationAgreement' of the remote supplier schema
[10/Feb/2014:13:53:34 +0100] schema - Local objectClasses must not be overwritten (set replication log for additional info)

}}}

Schema is not updated M2->M1, because checking Objectclasses the schema of M1 is a superset of M2.
Before looking at the new Attributes, the schema update is rejected because of the Objectclasses.
So adding new attributes to the master branch does not create a new problem.

I think one step I was missing in your test is 'x.y.z already has new schema'. Before doing your test, do you copy the new schema files on the former instance ?

Replying to [comment:19 tbordaz]:

I tried the following TC. Install M1 (master branch 1.3.3) and M2 (1.3.1). Then update the schema on M2 and add an entry on M2.

The entry is replication. Fine. But the schema not with errors like:

{{{
[10/Feb/2014:13:53:34 +0100] schema - Attribute sslVersionMin is not allowed in 'nsEncryptionConfig' of the remote supplier schema
[10/Feb/2014:13:53:34 +0100] schema - Attribute sslVersionMax is not allowed in 'nsEncryptionConfig' of the remote supplier schema
[10/Feb/2014:13:53:34 +0100] schema - Fail to retrieve in the remote supplier schema [nsSchemaPolicy or 2.16.840.1.113730.3.2.328]
[10/Feb/2014:13:53:34 +0100] schema - Attribute nsRoleScopeDN is not allowed in 'nsRoleDefinition' of the remote supplier schema
[10/Feb/2014:13:53:34 +0100] schema - Attribute winSyncDirectoryFilter is not allowed in 'nsDSWindowsReplicationAgreement' of the remote supplier schema
[10/Feb/2014:13:53:34 +0100] schema - Attribute winSyncWindowsFilter is not allowed in 'nsDSWindowsReplicationAgreement' of the remote supplier schema
[10/Feb/2014:13:53:34 +0100] schema - Attribute winSyncSubtreePair is not allowed in 'nsDSWindowsReplicationAgreement' of the remote supplier schema
[10/Feb/2014:13:53:34 +0100] schema - Local objectClasses must not be overwritten (set replication log for additional info)

}}}

Schema is not updated M2->M1, because checking Objectclasses the schema of M1 is a superset of M2.

Shouldn't they already be in sync (no supersets)?

Before looking at the new Attributes, the schema update is rejected because of the Objectclasses.
So adding new attributes to the master branch does not create a new problem.

I think one step I was missing in your test is 'x.y.z already has new schema'. Before doing your test, do you copy the new schema files on the former instance ?

This what I do for all of my test cases, in which I'm trying to reproduce a valid upgrade scenario.

Setup two servers, and get replication working (schema should be in sync at this point). Stop one server, replace 01core389.ldif(or any file), start server. Make another schema change(ldapmodify/console), and update an entry on that same server. Everything does replicate. Doesn't matter which server I start with(1.3.1 or 1.3.3).

If the schema is in sync, it should not matter what I add to any of the servers. If this is not true, then I feel there is a flaw in our schema replication logic.

I agree with you, after a valid upgrade scenario (copy the schema files), you can update the schema and make it replicated even from an older version. Thanks for your explanations

Replying to [comment:21 tbordaz]:

I agree with you, after a valid upgrade scenario (copy the schema files), you can update the schema and make it replicated even from an older version. Thanks for your explanations

To be clear, I wasn't trying to argue, I was just telling you how I was testing the server. But this does bring up something else. Once schema replication is broken, there doesn't seem to be an easy way to fix it, or get it back in sync - not without making a bunch of manual changes. I wonder if some kind of schema resync task could be created(enhancement to schema-reload plugin?). This out of the scope for this ticket, but was curious what you think about this.

Replying to [comment:17 rmeggins]:

Below, when you say "schema replicated to the other server", does that include the new single valued attribute schema, or just the one single change you made? If the former, what happens if by some mis-configuration the customer has more than one value in one of these attributes? Does this cause their server not to be able to be restarted?

Having pre-existing duplicate single-valued attributes does not prevent the server from starting. It doesn't complain at all. So this should not unexpectedly break the server for customers who have "invalid" configurations, and upgrade.

Replying to [comment:23 mreynolds]:

Replying to [comment:17 rmeggins]:

Below, when you say "schema replicated to the other server", does that include the new single valued attribute schema, or just the one single change you made? If the former, what happens if by some mis-configuration the customer has more than one value in one of these attributes? Does this cause their server not to be able to be restarted?

Having pre-existing duplicate single-valued attributes does not prevent the server from starting. It doesn't complain at all. So this should not unexpectedly break the server for customers who have "invalid" configurations, and upgrade.

Ok. Ack.

Replying to [comment:22 mreynolds]:

Replying to [comment:21 tbordaz]:

I agree with you, after a valid upgrade scenario (copy the schema files), you can update the schema and make it replicated even from an older version. Thanks for your explanations

To be clear, I wasn't trying to argue, I was just telling you how I was testing the server. But this does bring up something else. Once schema replication is broken, there doesn't seem to be an easy way to fix it, or get it back in sync - not without making a bunch of manual changes. I wonder if some kind of schema resync task could be created(enhancement to schema-reload plugin?). This out of the scope for this ticket, but was curious what you think about this.

Yes, I agree. Please open a ticket for that.

git merge ticket47437
Updating 02169ef..22a8572
Fast-forward
ldap/schema/01core389.ldif | 135 +++++++++++++++++++++++++++++++++++++++++
ldap/servers/slapd/libglobs.c | 32 +++++-----
2 files changed, 151 insertions(+), 16 deletions(-)

git push origin master
Counting objects: 15, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (8/8), done.
Writing objects: 100% (8/8), 2.80 KiB, done.
Total 8 (delta 6), reused 0 (delta 0)
To ssh://git.fedorahosted.org/git/389/ds.git
02169ef..22a8572 master -> master

commit 22a8572
Author: Mark Reynolds mreynolds@redhat.com
Date: Mon Feb 10 11:18:07 2014 -0500

Metadata Update from @tbordaz:
- Issue assigned to mreynolds
- Issue set to the milestone: 1.3.3 - 1/14 (January)

2 years ago

Login to comment on this ticket.

Metadata