#1610 Draft: Sign repository metadata
Opened 2 years ago by demiobenour. Modified 2 years ago
demiobenour/pungi repodata-sign  into  master

Draft: Sign repository metadata
Demi Marie Obenour • 2 years ago  
@@ -39,10 +39,45 @@ 

  from ..wrappers.scm import get_dir_from_scm

  from .base import PhaseBase

  

+ import fedora_messaging

+ 

  CACHE_TOPDIR = "/var/cache/pungi/createrepo_c/"

  createrepo_lock = threading.Lock()

  createrepo_dirs = set()

  

+ _topic = "fake_topic"

+ def sign_repodata(path, output):

+     with open(path, 'rb') as xml_file:

+         data = xml_file.read().decode('UTF-8', 'strict')

+     publish(Message(topic=_topic, body=data))

+ 

+     def messaging_callback(message):

+         if message.topic != _topic + '.finished':

+             return

+         body = message.body

+         try:

+             if body['body'] != data:

+                 return # different XML file

+         except (KeyError, TypeError):

+             return # not a message we are interested in

+         if 'error' in message.body:

+             raise fedora_messaging.exceptions.HaltConsumer(

+                     exit_code = 1,

+                     reason = f'Signing XML file failed: {message.body["error"]!r}')

+         try:

+             signature = message.body['signature']

+         except KeyError:

+             raise fedora_messaging.exceptions.HaltConsumer(

+                     exit_code = 0,

+                     reason = 'Signing XML file failed: got a response with no signature')

+         with open(output, 'wb') as signature_file:

+             output.write(signature.encode('UTF-8', 'strict'))

+         raise fedora_messaging.exceptions.HaltConsumer('Success')

+     try:

+         fedora_messaging.api.consume(messaging_callback)

+     except fedora_messaging.exceptions.HaltConsumer as s:

+         if s.exit_code:

+             raise ValueError from s

  

  class CreaterepoPhase(PhaseBase):

      name = "createrepo"
@@ -302,6 +337,9 @@ 

                  list(module_rpms),

              )

  

+     sign_repodata(os.path.join(repo_dir, 'repodata', 'repomd.xml'),

+                   os.path.join(repo_dir, 'repodata', 'repomd.xml.asc'))

+ 

      compose.log_info("[DONE ] %s" % msg)

  

  
@@ -324,6 +362,9 @@ 

          )

          run(cmd, logfile=log_file, show_cmd=True)

  

+     sign_repodata(os.path.join(repo_dir, 'repodata', 'repomd.xml'),

+                   os.path.join(repo_dir, 'repodata', 'repomd.xml.asc'))

+ 

  

  def find_file_in_repodata(repo_path, type_):

      dom = xml.dom.minidom.parse(os.path.join(repo_path, "repodata", "repomd.xml"))

This signs repository metadata. The main problem is that the topic (fake_topic) is obviously not a good choice.

Signed-off-by: Demi Marie Obenour demi@invisiblethingslab.com

The general approach is good. However, the current code couples pungi too tightly with fedora messaging, which only works for Fedora and no other deployment (that may need to use different signing backend).

Can you split the code into two parts? The createrepo phase (where you call sign_repodata) should call compose.notifier("fake_topic", path=..., output=...). The actual logic of handling the signature should be in a separate script that will be called by this notifier object. Check out pungi/scripts/wait_for_signed_ostree_handler.py.

In short: the script will get one argument when invoked. If it's not fake_topic, it should do nothing. Otherwise it can read JSON from stdin with all of the keyword arguments and do what sign_repodata does now.

The general approach is good. However, the current code couples pungi too tightly with fedora messaging, which only works for Fedora and no other deployment (that may need to use different signing backend).

Indeed.

Can you split the code into two parts? The createrepo phase (where you call sign_repodata) should call compose.notifier("fake_topic", path=..., output=...). The actual logic of handling the signature should be in a separate script that will be called by this notifier object. Check out pungi/scripts/wait_for_signed_ostree_handler.py.

Sure.

In short: the script will get one argument when invoked. If it's not fake_topic, it should do nothing. Otherwise it can read JSON from stdin with all of the keyword arguments and do what sign_repodata does now.

How will it know what topic to use? Because presumably it should not be using fake_topic.

How will it know what topic to use? Because presumably it should not be using fake_topic.

The topic is the first argument to compose.notifier. It will have to be hardcoded there.

Here's an example of how it might work:

  1. A repo is created. The createrepo phase calls compose.notifier("repodata-sign-request", path=..., output=...).
  2. This sends a message to org.fedoraproject.prod.pungi.compose.repodata.sign.request and also it would trigger the new handler script.
  3. The handler script would set up a listener to wait for repodata.sign.request.finished message and do whatever it needs to do.

A few caveats:

  • The CLI options setting up the notification scripts would need to be specified in correct order: --notification-script=/usr/bin/pungi-fedmsg-notification --notification=/usr/bin/pungi-wait-for-signed-repodata. Opposite order would mean the listener starts before the request is sent.
  • If signing is very quick, could the signature be sent before the listener is set up?
  • There are multiple repos created in parallel. Each will have a separate listener spawned, so they will need to know how to react to the correct message. (Could the listeners consume each other messages? What would ensure they eventually stop?)
Metadata