I'm testing the new version of 389-ds (1.4.1.3). I have the following test environment: - FreeIPA server 4.7.2 - 389-ds-base 1.4.1.2 with its legacy-tools
legacy-tools
During rpm upgrade of 389-ds the postscript triggers, but fails with:
... Processing /usr/share/dirsrv/updates/60removeLegacyReplication.ldif ... Running updateDSInstance stage runinst update /usr/share/dirsrv/updates/60upgradeconfigfiles.pl Running updateDSInstance stage runinst update /usr/share/dirsrv/updates/60upgradeschemafiles.pl Running updateDSInstance stage runinst update /usr/share/dirsrv/updates/70upgradefromldif.pl Running updateDSInstance stage runinst update /usr/share/dirsrv/updates/80upgradednformat.pl age runinst update /usr/share/dirsrv/updates/60upgradeschemafiles.pl +Running updateDSInstance stage runinst update /usr/share/dirsrv/updates/70upgradefromldif.pl +Running updateDSInstance stage runinst update /usr/share/dirsrv/updates/80upgradednformat.pl [28/May/2019:13:58:25.218277711 +0300] - ERR - slapd_system_isFIPS - Can not access /proc/sys/crypto/fips_enabled - assuming FIPS is OFF [28/May/2019:13:58:25.219829621 +0300] - ERR - slapd_system_isFIPS - Can not access /proc/sys/crypto/fips_enabled - assuming FIPS is OFF [28/May/2019:13:58:25.410390985 +0300] - ERR - dse_read_one_file - The entry cn=schema in file /etc/dirsrv/slapd-IPASL-IPATEST/schema/99user.ldif (lineno: 1) is invalid, error code 20 (Type or value exists) - attribute type nsUserRDNComponent: Does not match the OID "nsUserRDNComponent-oid". Another attribute type is already using the name or OID. [28/May/2019:13:58:25.412020837 +0300] - ERR - setup_internal_backends - Please edit the file to correct the reported problems and then restart the server. sort: could not read: /var/lib/dirsrv/slapd-IPASL-IPATEST/ldif/changelog_dn_norm_sp.txt: There is no such file or directory Undefined subroutine &DSUpdate::Package00000000009::debug called at /usr/share/dirsrv/updates/80upgradednformat.pl line 216.
An upgrade invocation is:
/usr/sbin/setup-ds.pl -u -d -l /var/log/dirsrv/upgrade.log -s General.UpdateMode=offline
I will look into this. The rpm upgrade should not be calling setup-ds.pl as it is no longer needed in 1.4.x (if it is there is a bug in the rpm specfile). This also means the user does not need to call setup-ds.pl either. I'm assuming in your case this fails during the rpm install, correct?
Metadata Update from @mreynolds: - Custom field origin adjusted to None - Custom field reviewstatus adjusted to None
My RPM spec file is very similar to one provided by upstream (I mean rpm/389-ds-base.spec.in). Let's assume it's the same in part of legacy-tools.
rpm/389-ds-base.spec.in
The rpm upgrade should not be calling setup-ds.pl as it is no longer needed in 1.4.x (if it is there is a bug in the rpm specfile)
That's really great! But currently, legacy-tools obsoletes 389-ds <= 1.4.0.9 as well as FreeIPA (4.7.2) has an explicit dependency on it. So, all the master instances which come up from that old versions have legacy-tools and thus, upgrade script runs for all of them.
So, I can just skip this step (upgrade) for 389-ds 1.4.0.9+, right? Perhaps, to prevent a possible harm of this script some sort of version check needs to be added to spec file.
I'm assuming in your case this fails during the rpm install, correct?
To be exact this happens during rpm upgrade.
@vashirov @tbordaz I think this is an effect of #49960 and "schema learning". If, for the latter, have attributes with string oids moved to 99user.ldif aand now upgrade schema ldifs and have the same attribute with a numeric oid we do get problems
Given the way that 99user.ldif is intended to work, we could allow that single file to duplicate and over-ride existing schema items if the OIDS match to allow the schema-learning/replication to still work effectively.
but the OIDs do not match, "schema learning" moves attributes of the core schema to 99user.ldif, and now for some has a ois like "attribute-oid". After the upgrade the attribute in the core schema file has a numeric oid - so they do not match.
We have an ongoing discussion on schema management in a replicated topology, maybe we need to try to make some progress there
@lkrispen , my understanding is that schema learning happens in the early phase of replication session. Here it is about upgrading a standalone instance. Possibly schema learning is involved but it is not clear to me how it is playing a role outside of replication.
where do you see that it is a standalone instance, it is an IPA server, and I can't see that it is a single master. And without schema learning, how would nsUserRDNComponent end up in 99user.ldif ?
The root cause of the this ticket is #47749 where standard definitions can be propagated in 99user.ldif
During upgrade of 389-ds-base-legacy-tools, 60upgradeschemafiles.pl scripts update the standard definitions (e.g. /usr/share/dirsrv/schema/50ns-admin.ldif). This standard definition changed because of #49960. So as a consequence of #47749 and #49960 we have a definition of attribute name with two different OIDs one a string and one numeric. At startup this conflicting definition is detected.
Note that the upgrade of 389-ds-base-legacy-tools was unexpected (in 1.4. this package has no reason to be installed) but it helped to discover this bug.
we can backout #49960 (easy) or fix #47749 (more difficult). But there are possibly others impacts (need to think more about it).
Part of the reason to avoid the pl upgrade scripts, is we can not assert they were run in a container on startup. So either we'd need to add an "update" framework to lib389 to support this, or we need the server to "assume" it has to self-upgrade itself. I'd think #47749 is "more correct" despite the difficulty, because there may be other scenarios that could occur. But I also understand that there are other pressures and time concerns here for customers.
I'm worried that if we have a change in say /usr/share/dirsrv/schema/00core.ldif, but that same oid/attr is in 99user.ldif, that the value in 99user.ldif is not used. There are lots of ways for this to go wrong I think (/usr/share is newer than 99user.ldif, or 99user.ldif is newer than /usr/share). How to detect and manage this seems like it's key to this problem, where when we read the same OID twice, we choose the "newer" item. I know we have some things to achieve this, but I think it'll be important we can test this scenario here.
Perhaps the solution is if we see the same OID twice in schema, we just "merge" the content somehow?
Metadata Update from @mreynolds: - Issue set to the milestone: 1.4.1
#49960 was backed out in 1.4.1.4 - hence fixing this issue. Closing ticket.
Metadata Update from @mreynolds: - Issue close_status updated to: fixed - Issue status updated to: Closed (was: Open)
389-ds-base is moving from Pagure to Github. This means that new issues and pull requests will be accepted only in 389-ds-base's github repository.
This issue has been cloned to Github and is available here: - https://github.com/389ds/389-ds-base/issues/3468
If you want to receive further updates on the issue, please navigate to the github issue and click on subscribe button.
subscribe
Thank you for understanding. We apologize for all inconvenience.
Metadata Update from @spichugi: - Issue close_status updated to: wontfix (was: fixed)
Login to comment on this ticket.