fedora-infrastructure

#6929 Something in fedmsg/badges is failing over and over again
Closed: Fixed Opened by churchyard.

Closed: Fixed
Reopen Issue

Since I've been added to the sysadmin-badges group, I receive this e-mail:

Message

[2018-05-10 19:13:09][fedmsg.crypto.utils ERROR]

Authorization/routing_policy error.  Topic u'org.fedoraproject.prod.bodhi.update.comment'.  Signer 'bodhi-bodhi-backend01.phx2.fedoraproject.org'.

Process Details

  • host: badges-backend01.phx2.fedoraproject.org
  • PID: 27352
  • name: fedmsg-hub
  • command: /usr/bin/python2 /usr/bin/fedmsg-hub
  • msg_id:

Callstack that lead to the logging statement

  File "/usr/bin/fedmsg-hub", line 9 in <module>
    load_entry_point('fedmsg==1.1.0', 'console_scripts', 'fedmsg-hub')()
  File "/usr/lib/python2.7/site-packages/fedmsg/commands/hub.py", line 122 in hub
    command.execute()
  File "/usr/lib/python2.7/site-packages/fedmsg/commands/__init__.py", line 67 in execute
    return self.run()
  File "/usr/lib/python2.7/site-packages/fedmsg/commands/hub.py", line 100 in run
    framework=False,
  File "/usr/lib/python2.7/site-packages/moksha/hub/__init__.py", line 109 in main
    reactor.run(installSignalHandlers=False)
  File "/usr/lib64/python2.7/site-packages/twisted/internet/base.py", line 1169 in run
    self.mainLoop()
  File "/usr/lib64/python2.7/site-packages/twisted/internet/base.py", line 1181 in mainLoop
    self.doIteration(t)
  File "/usr/lib64/python2.7/site-packages/twisted/internet/epollreactor.py", line 379 in doPoll
    log.callWithLogger(selectable, _drdw, selectable, fd, event)
  File "/usr/lib64/python2.7/site-packages/twisted/python/log.py", line 84 in callWithLogger
    return callWithContext({"system": lp}, func, *args, **kw)
  File "/usr/lib64/python2.7/site-packages/twisted/python/log.py", line 69 in callWithContext
    return context.call({ILogContext: newCtx}, func, *args, **kw)
  File "/usr/lib64/python2.7/site-packages/twisted/python/context.py", line 118 in callWithContext
    return self.currentContext().callWithContext(ctx, func, *args, **kw)
  File "/usr/lib64/python2.7/site-packages/twisted/python/context.py", line 81 in callWithContext
    return func(*args,**kw)
  File "/usr/lib64/python2.7/site-packages/twisted/internet/posixbase.py", line 626 in _doReadOrWrite
    why = selectable.doRead()
  File "/usr/lib/python2.7/site-packages/txzmq/connection.py", line 281 in doRead
    log.callWithLogger(self, self.messageReceived, message)
  File "/usr/lib64/python2.7/site-packages/twisted/python/log.py", line 84 in callWithLogger
    return callWithContext({"system": lp}, func, *args, **kw)
  File "/usr/lib64/python2.7/site-packages/twisted/python/log.py", line 69 in callWithContext
    return context.call({ILogContext: newCtx}, func, *args, **kw)
  File "/usr/lib64/python2.7/site-packages/twisted/python/context.py", line 118 in callWithContext
    return self.currentContext().callWithContext(ctx, func, *args, **kw)
  File "/usr/lib64/python2.7/site-packages/twisted/python/context.py", line 81 in callWithContext
    return func(*args,**kw)
  File "/usr/lib/python2.7/site-packages/txzmq/pubsub.py", line 72 in messageReceived
    self.gotMessage(message[1], message[0])
  File "/usr/lib/python2.7/site-packages/moksha/hub/zeromq/zeromq.py", line 197 in chain_over_moksha_callbacks
    f(_body, _topic)
  File "/usr/lib/python2.7/site-packages/moksha/hub/zeromq/zeromq.py", line 224 in intercept
    return callback(ZMQMessage(_topic, _body))
  File "/usr/lib/python2.7/site-packages/moksha/hub/api/consumer.py", line 167 in _consume_json
    return self._consume(message_as_dict)
  File "/usr/lib/python2.7/site-packages/fedmsg/consumers/__init__.py", line 276 in _consume
    self.validate(message)
  File "/usr/lib/python2.7/site-packages/fedmsg/consumers/__init__.py", line 270 in validate
    if not fedmsg.crypto.validate(message['body'], **self.hub.config):
  File "/usr/lib/python2.7/site-packages/fedmsg/crypto/__init__.py", line 244 in validate
    return backend.validate(message, **cfg)
  File "/usr/lib/python2.7/site-packages/fedmsg/crypto/x509.py", line 225 in _m2crypto_validate
    message.get('topic'), signer, routing_policy, config.get('routing_nitpicky', False))
  File "/usr/lib/python2.7/site-packages/fedmsg/crypto/utils.py", line 78 in validate_policy
    "Topic %r.  Signer %r." % (topic, signer))

This happens several hundreds time a day. Can you help me understands what's going on and what is supposed to be fixed here?

The way I read it is that bodhi (cc @bowlofeggs) tryies to emit fedmsg messages yet something is missconfigured and keeps rejecting those... ?

Today I ran a one-off script on bodhi-backend01 to fix #6925, and that script did add comments to a bunch of updates. The composer does also add comments on updates, however, so I'd think we do want to allow backend01 to do this.

Metadata Update from @bowlofeggs:
- Issue priority set to: Waiting on Assignee (was: Needs Review)
- Issue tagged with: bodhi

I have allowed this and am pushing out the new policy.

Some background for anyone who cares:

fedmsg has a policy thing where you specify exactly what fedmsgs are allowed to be sent by exactly what hosts. It has a 'permissive' mode where it errors/warns on them, but accepts them and a enforcing mode where it rejects them if they are not per the policy. We are running in the permissive mode, but we really want to clean everything up and get to the strict mode.

Metadata Update from @kevin:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

There's more:

Authorization/routing_policy error.  Topic u'org.fedoraproject.prod.bodhi.update.request.stable'.  Signer 'bodhi-bodhi03.phx2.fedoraproject.org'.

Metadata Update from @churchyard:
- Issue status updated to: Open (was: Closed)

This should be fixed now... I was trying to move some certs from old vm's into openshift, and it didn't work. I've reverted it for now...

Metadata Update from @kevin:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)
Metadata
None

bodhi

None
None
Normal
Static archive. pagure.io was sunset in 2026; this is a read-only snapshot.
Red Hat Logo

Fedora is sponsored by Red Hat. Learn more about the relationship between Red Hat and Fedora »

© Red Hat, Inc. and others.