#6799 [koji] Add configuration for module build service content generator
Closed 2 years ago Opened 2 years ago by sochotni.

We will need changes/additions to koji configuration to enable imports of Modules through content generator.

Expect future updates once since this is a bit off the top of my head.

  1. Add content generator "module-build-service"
  2. Add btype "module"
  3. Configure whatever user is used by modulebuild service to have permission for the content generator APIs

Configure whatever user is used by modulebuild service to have permission for the content generator APIs

The user is mbs/mbs.fedoraproejct.org.

For more info on what this is for, see the koji-devel thread(s):
https://lists.fedoraproject.org/archives/search?mlist=koji-devel%40lists.fedorahosted.org&q=Content+generator+for+module+build+service

So a summary of assumed required calls:

  • koji grant-cg-access mbs/mbs.fedoraproejct.org module-build-service --new
  • koji call addBType module

That should be all that's needed from MBS side to work as content generator

Pull request submitted to Bodhi to look for markers provided by this content generator: https://github.com/fedora-infra/bodhi/pull/1543

@rlaliber FYI this is a module config issue we have open.

Adding the meeting label to see if there's anything that needs to be discussed here.

Metadata Update from @ralph:
- Issue tagged with: meeting

2 years ago

FYI, this was discussed today: https://meetbot.fedoraproject.org/fedora-meeting-2/2017-06-12/releng.2017-06-12-14.31.html

My read of the conversation is that this request is "approved", and that this ticket has to date been stuck on a lack of explicit policy for new content generators. releng took an action item to work on documenting that for the future.

Linking some more information about what we want to do with this: https://fedoraproject.org/wiki/Infrastructure/Factory2/Focus/Bodhi

Also - offered to write to SOP to @katec, added @ausil to that mail waiting to hear back whether that would help

Added the SOP PR: https://pagure.io/releng/pull-request/6849

I am just referencing the koji documentation for content generator requirements since they seem aligned with what I'd expect rel-eng to require.

Alternatively koji documentation could be copied and tweaked but unless specific reservations are made it would be a waste of time.

As per the policy we have set

https://docs.pagure.org/releng/sop_adding_content_generator.html

Can you answer the following questions:

https://docs.pagure.org/releng/sop_adding_content_generator.html#questions-to-ask

And one other question I would like to know, where can we access the logs?

I set myself up didn't I? :-)

The content generator is module build service - mbs.fedoraproject.org
I am not 100% sure what the support agreement is around it but given it's running in the infra and there's playbooks I'd say it's in a decent state:
https://infrastructure.fedoraproject.org/cgit/ansible.git/tree/roles/mbs

Dev team is decently spread between continents at the moment and technically if mbs is down it's only affecting module builds, not rest of koji workflows.

The module build service itself doesn't actually do any builds - it just orchestrates koji so all logs are available in there. There's a possibility that we'd improve logging of the orchestration itself and make it part of the module import itself in the future.

New content type: We are adding "module" content type which is only a modulemd [1] yaml file. File size should be in the order of KBs. The binary content (module rpms) are already in koji and are just referenced in the build metadata.

As for the generic requirements from koji docs:
* We record rpms installed on the module build system at the time of the module orchestration
* We record version of the MBS itself
* Since we call koji to do actual binary builds we don't use host software
* For the same reason as above - we won't allow network during builds or use of pip or other content directly

Mocked content generator that would be imported into Koji looks like this:
https://fedorapeople.org/~sochotni/mbs-content-generator-v2.json

In addition the mentioned modulemd file would be included as output and all rpms from the module referenced as belonging to this modulemd file.

[1] https://pagure.io/modulemd

I should have mentioned - basic information about state of orchestration is visible in mbs itself. For example:
https://mbs.fedoraproject.org/module-build-service/1/module-builds/640

I forgot that in addition to the buildroot info and host rpms etc on MBs, we also provide dist-git source URL/hash. This leads to modulemd yaml file which was used by MBS to orchestrate the builds - same way you have source url for rpm builds

Linking of the module rpms into this module build itself provides way to collect logs and track which specific branches/hashes module builds were done.

Some final quick questions : where does the json file thats imported into koji comes from?
How is it generated?
And where are the logs on its generation?

where does the json file thats imported into koji comes from?

It gets generated on the fly by the MBS in this file https://pagure.io/fm-orchestrator/blob/master/f/module_build_service/builder/KojiContentGenerator.py

How is it generated?

From the modulemd that the user used to drive the module build as a whole (plus information from koji during the module build process).

And where are the logs on its generation?

The logs can be found in the journal on mbs-backend01.phx2.fedoraproject.org. See the 'fedmsg-hub' unit. (that's where the mbs backend runs).

Importantly, see also https://fedora-infra-docs.readthedocs.io/en/latest/sysadmin-guide/sops/mbs.html

Any other questions we can answer @mohanboddu?

14:47 < dgilmore> I think the only change I would like to propose here is that mbs also uploads teh logs on teh json generation
14:47 < dgilmore> not sure how invasive that would be 
14:47 < dgilmore> threebean: any idea? 
14:48  * threebean reads up 
14:50 < threebean> eh, should be possible. 
14:50 < threebean> but will take a code change.    
14:51 < dgilmore> threebean: cool  
14:51 < threebean> will file an issue to track it. 
14:51 < threebean> can we add it in later? 
14:51 < dgilmore> I think it wou,ld be usefult to have as part of the auditability 
14:51 < dgilmore> I think so 
14:51 < threebean> (for now at least, the logs are accessible on mbs-backend01 in the journal)
14:51 < dgilmore> threebean: thats not public and open and transparent :D
14:52 < threebean> yup, yup.
14:52 < threebean> but it is auditable!
14:53 < dgilmore> #agreed that under the condition we add the logs creating the json to the upload we can add the content generator

Metadata Update from @ausil:
- Issue untagged with: meeting
- Issue assigned to mohanboddu
- Issue tagged with: f26

2 years ago

MBS pull request with the addition of build logs to the content generator outputs section:
https://pagure.io/fm-orchestrator/pull-request/601

FYI the PR is now merged and @jkaluza is working on releasing new MBS version that will be importing the logs during CG imports

FYI the MBS is now deployed - cg imports are disabled until Koji configuration is changed. So ... shall we?

Can we get an update/plan for this? Boltron is out so I assume folks should have some time now?

@sochotni Just for an example, can you link one of the logs here?

By default the build logs are stored in /tmp and it doesn't seem to be overridden in the infra repo. I don't have permissions to ssh into mbs.fedoraproject.org but if you do you should find logs in /tmp/build-X.log

Correction, I looked at the wrong default config variable - by default we don't generate per-build logs unless it's enabled by changing BUILD_LOGS_DIR from empty string. Which would happen once we enabled CG code but it's not there yet.

$ koji grant-cg-access mbs/mbs.fedoraproject.org module-build-service --new
$ koji call addBType module
None
$

The content generator has been added. We still need the logs which requires a change in koji config. Please let us know when its done or what needs to be done to get the logs and then we can close the ticket.

Thanks.

Came back to this ticket just today and it looks like this is done.

See this example build of the platform module:

https://koji.fedoraproject.org/koji/buildinfo?buildID=973644

The builds are attached here: https://kojipkgs.fedoraproject.org//packages/platform/master/20170921192141/data/logs/build.log

Metadata Update from @ralph:
- Issue status updated to: Closed (was: Open)

2 years ago

FYI, I ran these commands in staging just now:

~❯ koji --profile staging grant-cg-access mbs/mbs.stg.fedoraproject.org module-build-service --new
~❯ koji --profile staging call addBType module

Login to comment on this ticket.

Metadata