#1125 Shutdown fedmsg-hub when SQLAlchemy can no longer resolve DNS records
Merged 8 months ago by lucarval. Opened 8 months ago by mprahl.

@@ -39,6 +39,7 @@ 

  import fedmsg.consumers

  import moksha.hub

  import six

+ import sqlalchemy.exc

  

  from module_build_service.utils import module_build_state_from_msg

  import module_build_service.messaging

@@ -160,6 +161,13 @@ 

          try:

              with models.make_session(conf) as session:

                  self.process_message(session, msg)

+         except sqlalchemy.exc.OperationalError as error:

+             if 'could not translate host name' in str(error):

+                 log.exception(

+                     "SQLAlchemy can't resolve DNS records. Scheduling fedmsg-hub to shutdown.")

+                 self.shutdown()

+             else:

+                 raise

          except Exception:

              log.exception('Failed while handling {0!r}'.format(msg))

  

Occasionally, SQLAlchemy can no longer resolve the IP address of the MBS database. This exception will repeat for every new message that is encountered until fedmsg-hub is restarted. This causes fedmsg-hub to shutdown and then systemd will start it back up again, thus resolving the issue.

Change string quotation marks (' -> "), so you don't have to use escaping.

The surrounding code uses single quotes, and I think it's better to remain consistent than switching back and forth.

The surrounding code uses single quotes, and I think it's better to remain consistent than switching back and forth.

PEP8: In Python, single-quoted strings and double-quoted strings are the same. This PEP does not make a recommendation for this. Pick a rule and stick to it. When a string contains single or double quote characters, however, use the other one to avoid backslashes in the string. It improves readability.

The surrounding code uses single quotes, and I think it's better to remain consistent than switching back and forth.

PEP8: In Python, single-quoted strings and double-quoted strings are the same. This PEP does not make a recommendation for this. Pick a rule and stick to it. When a string contains single or double quote characters, however, use the other one to avoid backslashes in the string. It improves readability.

If you feel that strongly, I'll change it. It really doesn't matter though in my opinion.

rebased onto 25cda8d

8 months ago

Pull-Request has been merged by lucarval

8 months ago