#7534 Investigate failures to restore 389-ds attriubtes on upgrade failure
Closed: fixed 3 years ago by rcritten. Opened 5 years ago by rcritten.

Issue

ipa-server-upgrade disables the 389-ds listeners so the server is "quiet" during upgrade. Twice this week I've run into users who had to manually restore the values after a failed upgrade.

This should be bullet-proof and is something we've looked at before but it is still happening. I have a memory that German or Thorsten mentioned that this is common as well.

It very well may be a regression with the switch to the new installer framework.

mail thread https://lists.fedoraproject.org/archives/list/freeipa-users@lists.fedorahosted.org/thread/7YNTBRSRJKU2IXQH5CEVZQPDEKOCKHDV/

I don't know how to reproduce this.


This is because there is no call to __restore_config() upon exception.

There used to be a monster try/except around the entire upgrade process where no matter what the listeners were restored. In the switch to the admintool framework this was lost and while exceptions are caught there is no call to restore the configuration upon failure.

The double underscores are not at all needed as this class will/should never be subclassed anyway.

At least __restore_config() needs to be promoted to a public function and called during all except handlers (if not moving it to a finally with some care that it does nothing if the values have already been restored).

Metadata Update from @rcritten:
- Issue priority set to: critical
- Issue set to the milestone: FreeIPA 4.7

5 years ago

Metadata Update from @rcritten:
- Issue assigned to rcritten

5 years ago

Ah ok, I missed the run_after_failure bit in ipaserver/install/service.py so it looks like it should be restoring things properly (but doesn't)

Metadata Update from @frenaud:
- Custom field rhbz adjusted to https://bugzilla.redhat.com/show_bug.cgi?id=1569011

5 years ago

The only way I've been able to reproduce this is with keyboard interrupt ^C. If one spams ^C then it can break out of all exception handlers.

Metadata Update from @rcritten:
- Issue set to the milestone: FreeIPA 4.7.1 (was: FreeIPA 4.7)

5 years ago

FreeIPA 4.7 has been released, moving to FreeIPA 4.7.1 milestone

Metadata Update from @rcritten:
- Issue set to the milestone: FreeIPA 4.7.2 (was: FreeIPA 4.7.1)

5 years ago

FreeIPA 4.7.1 has been released, moving to FreeIPA 4.7.2 milestone

Metadata Update from @rcritten:
- Issue set to the milestone: None (was: FreeIPA 4.7.2)

3 years ago

master:

  • 2f8eb73 Use a state to determine if a 389-ds upgrade is in progress

ipa-4-8:

  • ae4e549 Use a state to determine if a 389-ds upgrade is in progress

Metadata Update from @rcritten:
- Issue close_status updated to: fixed
- Issue status updated to: Closed (was: Open)

3 years ago

Metadata Update from @rcritten:
- Custom field affects_doc adjusted to on
- Custom field knownissue adjusted to on
- Issue status updated to: Open (was: Closed)

3 years ago

There is still some exposure to killing. Someone reproduced this by killing the process in the parser.parse() call within __restore_config (line 230) so the values were restored from the backup but the new dse.ldif never written.

I'm not sure this can ever be 100% bullet-proof since it can be externally killed but if rather than calling restore_state() on the values in __restore_config we use get_state() which will leave them in the state file. Then the last step is to pop upgrade-in-progress and then the rest.

If the values have been restored and the new ldif written and copied then it's only upgrade-in-progress that really matters. The rest will be overwritten.

master:

  • ba9b950 Move where the restore state is marked during IPA server upgrade

ipa-4-9:

  • 20055dd Move where the restore state is marked during IPA server upgrade

ipa-4-8:

  • 9ecd7c2 Move where the restore state is marked during IPA server upgrade

Metadata Update from @rcritten:
- Issue close_status updated to: fixed
- Issue status updated to: Closed (was: Open)

3 years ago

Login to comment on this ticket.

Metadata