#48984 lib389: add paths module
Closed: wontfix None Opened 7 years ago by firstyear.

On various platforms and install prefixs, lib389 is crippled by statically coded paths.

This will add a module which is able to from the info inf created by ds, derive the paths for the installed platform we are on. Depends on #48983


Hi William,
it is great that you've decided to add this feature. Very useful, in my opinion. :)

But we would face an issue, when we'll go for testing older versions, because they don't have this default file.

Possible solutions, if default.inf wasn't found.
Do not raise exception there, because it would break older versions testing.
{{{
90 def _get_defaults_loc(self, search_paths):
91 for spath in search_paths:
92 if os.path.isfile(spath):
93 return spath
94 raise IOError('defaults.inf not found in any wellknown location!')
}}}

Instead of this:
1. Return some default values from lib389;
2. Use some precreated default.inf from lib389 setup;
3. Another resolution from you.

We also have overlap with functions in util.py (get_sbin_dir, get_data_dir, etc), can these be combined or moved?

The DS patch for the CI tests looks good. +100 on removing all the installation prefix stuff! But there is still redundancy with functions in util.py

Replying to [comment:4 mreynolds]:

The DS patch for the CI tests looks good. +100 on removing all the installation prefix stuff! But there is still redundancy with functions in util.py

Those will be removed soon, I just don't want to break everything "just yet"!

Replying to [comment:2 spichugi]:

Hi William,
it is great that you've decided to add this feature. Very useful, in my opinion. :)

But we would face an issue, when we'll go for testing older versions, because they don't have this default file.

Possible solutions, if default.inf wasn't found.
Do not raise exception there, because it would break older versions testing.
{{{
90 def _get_defaults_loc(self, search_paths):
91 for spath in search_paths:
92 if os.path.isfile(spath):
93 return spath
94 raise IOError('defaults.inf not found in any wellknown location!')
}}}

If we don't raise the exception, we have no path data ...

Instead of this:
1. Return some default values from lib389;
2. Use some precreated default.inf from lib389 setup;
3. Another resolution from you.

There are some issues here. Without this file, we should not be assuming anything about ds in the environment, in fact it is these assumptions about DS and the files in it that led to this change. I'm pretty sick of hardcoded file path hacks causing me problems :)

I think that the best solution here is we get the defaults.inf from the latest version, and make it available in the test systems for older versions (but only on the test systems). Previous versions haven't changed their filesystem paths much, so this is likely safe. The caveat here is if on a test system you are not doing an RPM based install of 389-ds, or you are doing some prefixed installation.

Alternately, we backport the defaults.inf file to older versions of DS for testing, as it's a minimal and non-invasive change.

Finally, were we to go with the "default values" as you suggest, they would basically assume:

  • ds version 1.3.5 or less.
  • standard installed RPM locations.
  • You can't have a prefixed install (ie you can't install to /opt/dirsrv).
  • Some kind of safety barrier like an ENV flag to activate their presence.

I think that were you to use this and needed a prefix on the ds packages, it would be problematic.

Replying to [comment:6 firstyear]:

Instead of this:
1. Return some default values from lib389;
2. Use some precreated default.inf from lib389 setup;
3. Another resolution from you.

There are some issues here. Without this file, we should not be assuming anything about ds in the environment, in fact it is these assumptions about DS and the files in it that led to this change. I'm pretty sick of hardcoded file path hacks causing me problems :)
Unfortunately, we don't have a possibility of going straight into the future and burning all bridges behind us. We should care about compatibility with older versions that we should test and maintain.

I think that the best solution here is we get the defaults.inf from the latest version, and make it available in the test systems for older versions (but only on the test systems). Previous versions haven't changed their filesystem paths much, so this is likely safe. The caveat here is if on a test system you are not doing an RPM based install of 389-ds, or you are doing some prefixed installation.

Alternately, we backport the defaults.inf file to older versions of DS for testing, as it's a minimal and non-invasive change.

I am agree with you and I like the idea of installing default.inf with the CI set up for older versions. I will cooperate with my team on it tomorrow and also, I think, we should wait for the Mark's reply (he has an upstream CI of his own and may be he would like to test older versions too sometimes).

Meanwhile, I've tested your patches with manually created '''/usr/share/dirsrv/inf/defaults.inf''' and it seems okay. '''lib389.paths''' catches it.

Thank you one more time. :)

Finally, were we to go with the "default values" as you suggest, they would basically assume:

  • ds version 1.3.5 or less.
  • standard installed RPM locations.
  • You can't have a prefixed install (ie you can't install to /opt/dirsrv).
  • Some kind of safety barrier like an ENV flag to activate their presence.

I think that were you to use this and needed a prefix on the ds packages, it would be problematic.

Replying to [comment:7 spichugi]:

Replying to [comment:6 firstyear]:

Instead of this:
1. Return some default values from lib389;
2. Use some precreated default.inf from lib389 setup;
3. Another resolution from you.

There are some issues here. Without this file, we should not be assuming anything about ds in the environment, in fact it is these assumptions about DS and the files in it that led to this change. I'm pretty sick of hardcoded file path hacks causing me problems :)
Unfortunately, we don't have a possibility of going straight into the future and burning all bridges behind us. We should care about compatibility with older versions that we should test and maintain.

I think that the best solution here is we get the defaults.inf from the latest version, and make it available in the test systems for older versions (but only on the test systems). Previous versions haven't changed their filesystem paths much, so this is likely safe. The caveat here is if on a test system you are not doing an RPM based install of 389-ds, or you are doing some prefixed installation.

Alternately, we backport the defaults.inf file to older versions of DS for testing, as it's a minimal and non-invasive change.

I am agree with you and I like the idea of installing default.inf with the CI set up for older versions. I will cooperate with my team on it tomorrow and also, I think, we should wait for the Mark's reply (he has an upstream CI of his own and may be he would like to test older versions too sometimes).

For nightly upstream CI testing it is always master branch. There are no intentions to test 1.3.5 or earlier. However, that doesn't mean we wouldn't want to run it in the future(after backporting some complex fix). Also, I know Viktor was planning on doing downstream testing using our CI tests. So yes I think this should be backported.

Meanwhile, I've tested your patches with manually created '''/usr/share/dirsrv/inf/defaults.inf''' and it seems okay. '''lib389.paths''' catches it.

Thank you one more time. :)

Finally, were we to go with the "default values" as you suggest, they would basically assume:

  • ds version 1.3.5 or less.
  • standard installed RPM locations.
  • You can't have a prefixed install (ie you can't install to /opt/dirsrv).
  • Some kind of safety barrier like an ENV flag to activate their presence.

I think that were you to use this and needed a prefix on the ds packages, it would be problematic.

Okay, so how about I merge this, then I reopen #48983 with the intent to backport it to the other branches. What branches do we need this in? 1.2.11 and 1.3.5?

Replying to [comment:11 firstyear]:

Okay, so how about I merge this, then I reopen #48983 with the intent to backport it to the other branches. What branches do we need this in? 1.2.11 and 1.3.5?

Sounds good. I've discussed the thing with my teammates and we all agree with this course of actions. :)
Thank you!

Replying to [comment:11 firstyear]:

Okay, so how about I merge this, then I reopen #48983 with the intent to backport it to the other branches. What branches do we need this in? 1.2.11 and 1.3.5?

Well all our CI tests were started in 1.3.3. So it should be in 1.3.3 and up. As for 1.2.11 I suspect there will be major failures. But I also believe Viktor wants to fix this by adding versions to tests so they don't run on certain versions of DS, but I don't know if he was referring to 1.2.11, or not.

Yes, I was talking about 1.2.11. It would be nice to have defaults.inf in RHEL6. The next release will be a maintenance release (i.e. bug and security fixes only, no new features), but given that this feature is a non-invasive change and doesn't touch the server itself, I think it can make it.

Well, the eventual goal, if you look at defaults.inf, you can see that it supports a 'version' field. So we can have tests and lib389 gated on versions of ds in the future to avoid the issue.

commit 5fd286450c6cc1edc701a0ae7362756852cc65cb
Writing objects: 100% (13/13), 4.19 KiB | 0 bytes/s, done.
Total 13 (delta 8), reused 0 (delta 0)
To ssh://git.fedorahosted.org/git/389/lib389.git
bf4172b..5fd2864 master -> master

commit da91421
commit 088fe3a
Compressing objects: 100% (67/67), done.
Writing objects: 100% (68/68), 8.95 KiB | 0 bytes/s, done.
Total 68 (delta 60), reused 0 (delta 0)
To ssh://git.fedorahosted.org/git/389/ds.git
fcc80bd..da91421 master -> master

Seems reasonable to me. Ack

Agreed, it makes sense.

Metadata Update from @firstyear:
- Issue assigned to firstyear
- Issue set to the milestone: lib389 1.0.3

7 years ago

Metadata Update from @mreynolds:
- Issue close_status updated to: None (was: Fixed)
- Issue set to the milestone: None (was: lib389 1.0.3)

4 years ago

389-ds-base is moving from Pagure to Github. This means that new issues and pull requests
will be accepted only in 389-ds-base's github repository.

This issue has been cloned to Github and is available here:
- https://github.com/389ds/389-ds-base/issues/2043

If you want to receive further updates on the issue, please navigate to the github issue
and click on subscribe button.

Thank you for understanding. We apologize for all inconvenience.

Metadata Update from @spichugi:
- Issue close_status updated to: wontfix

3 years ago

Login to comment on this ticket.

Metadata