#618 Improve flexibility to generate mock config file
Opened 3 years ago by franzh. Modified a year ago

Currently the mock config options are hard coded in koji library.
Though user can add options by editing tag with extra field, the supported options are
limited (currently available options are mock.new_chroot and mock.package_manager).

It would be helpful for develop or debug if koji can have move flexibility on generating
mock config.

I have two directions to improve flexibility.

1) Move all hard coded mock options to a separate setting file, and generate
    mock config based on it. So every host can have their own basic mock config
    corresponding to the hardware/software configuration.

2) Currently, we can add mock option through extras field in a tag, we could still
    leverage this feature to apply options in large scope. However, we should only
    have to support partial set of options rather than full.

I'm highly suspicious about variant where hosts can have different mock configs. We could/should make 'koji mock-config' output to be only one existing option. I'm not against moving it (template) out of the code and store it on hub. It is the only point of authority we would have in such scenario. Then some API call would be needed that will provide template or better - resulting correct mock config to clients/builders.

@mikem ideas?

Related: #596 #339

I know you concern and I agree that client sides should share same mock config to ensure buildroots are consistent. Download mock config from hub is a good idea, however I think we could keep some flexibility on client side to avoid some corner cases like #569.

For example, if client side has a custom config file, it will download general file from the hub and merge its custom file to generate target config for buildroot. Additionally, if client side has this custom config, it should expose the file on koji web.

Currently, the options in mock config file are related to build tag extra, task method and repo info which is also from build tag. And meanwhile, client mock-config command can not exactly expose all configurations in buildroot
I'm thinking if the mock.conf info(or template) can be stored in buildroot, we could ignore all tag config and the task ran on it, and expose it from buildroot.

The other concern is whether the assumption that all builders and clients can run mock as expectation with the same mock.conf info/template is right.

Hmm, interesting idea. Simple ugly solution is to store config completely in tag extras (doesn't need schema change), but mock is quite essential, so using separate db field could make sense. buildroot repodata then can contain additional file (as we already have those EMPTY_REPO/pkglist/blocklist files), or even referenced in repodata/repomd.xml - but I don't see this approach to be much useful.

Metadata Update from @tkopecek:
- Issue set to the milestone: 1.16

3 years ago

We can add a field/few fields in the database to record each builder's mock preferences. Moreover, the hub has default mock configs recorded in the database as well. All builders download hub's config and its own config from database and combine them to final mock config. The advantages of this modification are: 1) we have flexibility on both hub and builder side, 2) we can track every builder's preferences.

Metadata Update from @mikem:
- Issue set to the milestone: 1.17 (was: 1.16)

2 years ago

Metadata Update from @julian8628:
- Issue tagged with: discussion

2 years ago

Here's my basic ideas:
1. have a basic mock.conf template(maybe jinja2) which could be on hub, or separately on cli and builders
2. have a new API such like getMockConfigFile(hostInfo, buildroot, ... )(when template on hub) directly or getMockOptions(hostinfo, buildroot, ...) (when template on cli, builder) then generating mock config file with the template and options
3. store mock options in db and it's related with buildroot
4. we should have another config file or a db table to store default mock options per task method
5. mock options for host(builder/client) can be generated based on host information(os and pkg version) and basic config.
6. mock options should be prioritized, I think it could be default<tag<task<host<force
7. use mock.cfg rather than mock cmd arguments as much as possible.

@mikem @tkopecek @franzh Could you give me any suggestions?

Metadata Update from @julian8628:
- Issue assigned to julian8628

2 years ago

I like your idea they are great, some suggestions:
for 2) builders should always use mock config retrieved from hub and ignore local config. So the hub can fully understand/control the config that builders use.

for 6) I think the priority should be host=default<tag<task<force

By the way, in certain cases (e.g. rhel8), we might need to pass environment variable into build root. I think it might be good to have such feature to allow user passing env var by using mock.environemnt.* .

Metadata Update from @julian8628:
- Assignee reset

2 years ago

Metadata Update from @tkopecek:
- Issue set to the milestone: 1.18 (was: 1.17)

2 years ago

Metadata Update from @dgregor:
- Issue tagged with: feature

2 years ago

Moving out of 1.18 and to the backlog. Not high priority and this work may get rolled up into some other refactoring that is planned.

Metadata Update from @dgregor:
- Custom field Size adjusted to None

a year ago

Metadata Update from @dgregor:
- Issue set to the milestone: None (was: 1.18)

a year ago

Login to comment on this ticket.