frostyx / copr / copr

Forked from copr/copr 7 years ago
Clone

05b161e [backend] simplify logging through redis

3 files Authored by praiskup 5 years ago, Committed by praiskup 5 years ago,
    [backend] simplify logging through redis
    
    After 3d642d711eb9cf1e10d224fa4d379c5edf6fc34a commit, I started
    to observe weird errors (see below).  So while debugging, I
    noticed that the filters are unnecessary complicated, and that we
    can bypass LogErrors through Redis queue a more convenient way:
    
    -> start with LogRecord
    -> if traceback -> encode traceback as msg (as before)
    -> else -> encode message (as before)
    -> drop some fields which are not json.dump()able
    -> and then send to redis
    
    On the RedisLogHandler side it is then only matter of
    logging.makeLogRecord() call and logger.handle(record).
    
    While I was on it, I removed the router_logger mechanism, and
    created separate logger for each daemon (this is actually faster,
    because not all the loggers have to go through all log records).
    
    Hopefully this means final fight with RedisLog* stuff, should be
    much solid.
    
    copr_run_logger.py[26166]: --- Logging error ---
    copr_run_logger.py[26166]: Traceback (most recent call last):
    copr_run_logger.py[26166]:   File "/usr/lib64/python3.6/logging/__init__.py", line 994, in emit
    copr_run_logger.py[26166]:     msg = self.format(record)
    copr_run_logger.py[26166]:   File "/usr/lib64/python3.6/logging/__init__.py", line 840, in format
    copr_run_logger.py[26166]:     return fmt.format(record)
    copr_run_logger.py[26166]:   File "/usr/lib64/python3.6/logging/__init__.py", line 585, in format
    copr_run_logger.py[26166]:     record.exc_text = self.formatException(record.exc_info)
    copr_run_logger.py[26166]:   File "/usr/lib64/python3.6/logging/__init__.py", line 535, in formatException
    copr_run_logger.py[26166]:     traceback.print_exception(ei[0], ei[1], tb, None, sio)
    copr_run_logger.py[26166]:   File "/usr/lib64/python3.6/traceback.py", line 104, in print_exception
    copr_run_logger.py[26166]:     type(value), value, tb, limit=limit).format(chain=chain):
    copr_run_logger.py[26166]:   File "/usr/lib64/python3.6/traceback.py", line 477, in __init__
    copr_run_logger.py[26166]:     if (exc_value and exc_value.__cause__ is not None
    copr_run_logger.py[26166]: AttributeError: 'str' object has no attribute '__cause__'
    copr_run_logger.py[26166]: Call stack:
    copr_run_logger.py[26166]:   File "/usr/bin/copr_run_logger.py", line 15, in <module>
    copr_run_logger.py[26166]:     main()
    copr_run_logger.py[26166]:   File "/usr/bin/copr_run_logger.py", line 11, in main
    copr_run_logger.py[26166]:     handler.run()
    copr_run_logger.py[26166]:   File "/usr/share/copr/backend/daemons/log.py", line 135, in run
    copr_run_logger.py[26166]:     self.handle_msg(raw)
    copr_run_logger.py[26166]:   File "/usr/share/copr/backend/daemons/log.py", line 120, in handle_msg
    copr_run_logger.py[26166]:     self.router_logger.log(level, msg, extra={"event": event})
    copr_run_logger.py[26166]: Message: 'Healtcheck failed for VM 172.25.150.214 with error Connection broke:\nOUT:\n\nERR:\nssh: connect to host 172.25.150.214 port 22: No route to host\n>
    
        
file modified
+14 -69
file modified
+14 -19