#837 Increase imagefactory timeout from 7200 to 20000 seconds
Closed 4 years ago by tkopecek. Opened 6 years ago by sinnykumari.
sinnykumari/koji master  into  master

file modified
+1 -1
@@ -3502,7 +3502,7 @@ 

              'imgdir': os.path.join(self.workdir, 'scratch_images'),

              'tmpdir': os.path.join(self.workdir, 'oz-tmp'),

              'verbose' : True,

-             'timeout': 7200,

+             'timeout': 20000,

              'output': 'log',

              'raw': False,

              'debug': True,

Building s390x cloud-base image in koji exceeds
imagefactory timeout and causes image build to fail.
Increasing timeout value to 20000 seconds will help to
to build image successfully on s390x.
Changing timeout value shouldn't impact build time for
other images on any architectures.

OZ log from one of the F28 nightly compose (28-20180304.n.0) - https://kojipkgs.fedoraproject.org//work/tasks/2491/25512491/oz-s390x.log

How did you arrive at the 20000 number?

Is it worth making this a config option?

How did you arrive at the 20000 number?

I was able to get successful cloud-base image build on local s390x machine which takes around 6400 seconds. s390x koji builder seems to be taking longer time than 7200 seconds. So, I picked a bit high number (20000 seconds) just to safe that build succeeds before timeout reaches.

Is it worth making this a config option?
It should be. We can know precise timeout value if we can successfully run imageafctory on s390x koji builder with increased timeout value.

FYI, timeout configs were added in oz-0.16.0, https://github.com/clalancette/oz/commit/591e5e49ecdadbbcace839abbc31ac653d6732c4

If we are going to make this configurable, can we skip passing install timeout to imagefactory to let oz determine it?

FYI, timeout configs were added in oz-0.16.0, https://github.com/clalancette/oz/commit/591e5e49ecdadbbcace839abbc31ac653d6732c4
If we are going to make this configurable, can we skip passing install timeout to imagefactory to let oz determine it?

This patch is about option "--timeout TIMEOUT " of imagefactory command. As per imagefactory documentation default timeout value is 3600 seconds. I played a bit around changing oz.cfg timeout values and it didn't affect imageafctory run timeout. I don't know much what exactly timeouts in oz.cfg refers to.

This patch is about option "--timeout TIMEOUT " of imagefactory command. As per imagefactory documentation default timeout value is 3600 seconds. I played a bit around changing oz.cfg timeout values and it didn't affect imageafctory run timeout. I don't know much what exactly timeouts in oz.cfg refers to.

I simply browsed through oz and imagefactory code some days ago. --timeout TIMEOUT of imagefactory cmd should be timouts.install in oz.cfg, if I didn't miss anything important.
refer to:
- https://github.com/clalancette/oz/blob/master/oz/Guest.py#L1558
- https://github.com/redhat-imaging/imagefactory/blob/master/imagefactory_plugins/TinMan/TinMan.py#L346
- https://github.com/redhat-imaging/imagefactory/blob/master/imgfac/ApplicationConfiguration.py#L41

if make 'timout': None in configuration dict for imagefactory, the number in oz.cfg will take effect.

I simply browsed through oz and imagefactory code some days ago. --timeout TIMEOUT of imagefactory cmd should be timouts.install in oz.cfg, if I didn't miss anything important.
refer to:
- https://github.com/clalancette/oz/blob/master/oz/Guest.py#L1558
- https://github.com/redhat-imaging/imagefactory/blob/master/imagefactory_plugins/TinMan/TinMan.py#L346
- https://github.com/redhat-imaging/imagefactory/blob/master/imgfac/ApplicationConfiguration.py#L41
if make 'timout': None in configuration dict for imagefactory, the number in oz.cfg will take effect.

Thanks @julian8628 for the commit links
I did a bit of further debugging to see why imagefactory is not picking up install timeout value specified in oz.cfg when we don't specify ' --timeout' . This is because there is a default timeout=3600 passed at https://github.com/redhat-imaging/imagefactory/blob/master/imgfac/ApplicationConfiguration.py#L66 . And oz.cfg install timeout is not being used. I will work on a patch to get oz.cfg install timeout being used when defined.

I have one question - Are we planning to remove imagefactory timeout value in koji and use install timeout from oz.cfg once we have imagefatcory fixed?

You are welcome!

I just tested on one builder by simply setting 'timeout': None at kojid.
then the install timeout(1200) in oz.cfg was accepted by oz
oz log:

Waiting for factory-build-be839aab-1ebf-468e-a06e-e294774c0190 to finish installing, 839/1200
Waiting for factory-build-be839aab-1ebf-468e-a06e-e294774c0190 to finish installing, 829/1200
Waiting for factory-build-be839aab-1ebf-468e-a06e-e294774c0190 to finish installing, 819/1200

I don't think the default 3600 at https://github.com/redhat-imaging/imagefactory/blob/master/imgfac/ApplicationConfiguration.py#L66 will take effect, because in kojid, ApplicationConfiguration is initialized by passing config dict

ApplicationConfiguration(configuration=config)

Looking at https://github.com/redhat-imaging/imagefactory/blob/master/imgfac/ApplicationConfiguration.py#L37, __new_argument_parser won't be called.

packages:

# rpm -qa oz imagefactory
imagefactory-1.1.10-3.el7eng.noarch
oz-0.16.0-2.el7eng.noarch

And, yes! I'd like to use "install timout" in oz.cfg if timeout is not specified in kojid side.

Filed one PR #841 to make this value configurable.

I don't think the default 3600 at https://github.com/redhat-imaging/imagefactory/blob/master/imgfac/ApplicationConfiguration.py#L66 will take effect, because in kojid, ApplicationConfiguration is initialized by passing config dict
ApplicationConfiguration(configuration=config)

Ah ok, I understand it now. I was testing timeout by directly calling imagefcatory command line tool and in that case default timeout defined in __new_argument_parser was used instead of picking from oz.cfg.

Looking at https://github.com/redhat-imaging/imagefactory/blob/master/imgfac/ApplicationConfiguration.py#L37, __new_argument_parser won't be called.
packages:

rpm -qa oz imagefactory

imagefactory-1.1.10-3.el7eng.noarch
oz-0.16.0-2.el7eng.noarch

And, yes! I'd like to use "install timout" in oz.cfg if timeout is not specified in kojid side.

+1

Closing this one in favour of #841

Pull-Request has been closed by tkopecek

4 years ago
Metadata