#246 Period of fedimg downtime
Closed 6 years ago Opened 7 years ago by dustymabe.

There appears to be a period of downtime where fedimg was not uploading AMIs

Fedora-Atomic-25-20170226.0.x86_64 EC2 (us-west-2) ami-5051d230 hvm gp2
Fedora-Atomic-25-20170304.0.x86_64 EC2 (ap-northeast-1) ami-8cb2e5eb hvm gp2

Attached is a run of get_ami.py which shows a gap.

Can someone investigate why this happened?

file.d.txt


Metadata Update from @dustymabe:
- Issue assigned to sayanchowdhury

7 years ago

So according to sayan, this is because of image test failures, which then delete the AMIs after the image tests run.

I got a failure when booting an AMI today. out of 4 nodes that I booted it only happened on one of them:

Mar 14 14:40:48 ip-10-0-8-252.ec2.internal cloud-init[1485]: Cloud-init v. 0.7.8 running 'init-local' at Tue, 14 Mar 2017 14:40:48 +0000. Up 44.49 seconds.
Mar 14 14:40:48 ip-10-0-8-252.ec2.internal cloud-init[1485]: 2017-03-14 14:40:48,608 - util.py[WARNING]: failed stage init-local
Mar 14 14:40:48 ip-10-0-8-252.ec2.internal cloud-init[1485]: [CLOUDINIT] util.py[DEBUG]: failed stage init-local
                                                             Traceback (most recent call last):
                                                               File "/usr/lib/python3.5/site-packages/cloudinit/cmd/main.py", line 247, in main_init
                                                                 init.fetch(existing=existing)
                                                               File "/usr/lib/python3.5/site-packages/cloudinit/stages.py", line 358, in fetch
                                                                 return self._get_data_source(existing=existing)
                                                               File "/usr/lib/python3.5/site-packages/cloudinit/stages.py", line 268, in _get_data_source
                                                                 pkg_list, self.reporter)
                                                               File "/usr/lib/python3.5/site-packages/cloudinit/sources/__init__.py", line 318, in find_source
                                                                 raise DataSourceNotFoundException(msg)
                                                             cloudinit.sources.DataSourceNotFoundException: Did not find any data source, searched classes: (DataSourceNoCloud, DataSourceConfigDrive, DataSourceOpenNebula, DataSourceDigitalOcean, DataSourceOVF, DataSourceSmartOS)

                                                             During handling of the above exception, another exception occurred:

                                                             Traceback (most recent call last):
                                                               File "/usr/lib/python3.5/site-packages/cloudinit/cmd/main.py", line 521, in status_wrapper
                                                                 ret = functor(name, args)
                                                               File "/usr/lib/python3.5/site-packages/cloudinit/cmd/main.py", line 265, in main_init
                                                                 init.apply_network_config(bring_up=not args.local)
                                                               File "/usr/lib/python3.5/site-packages/cloudinit/stages.py", line 650, in apply_network_config
                                                                 return self.distro.apply_network_config(netcfg, bring_up=bring_up)
                                                               File "/usr/lib/python3.5/site-packages/cloudinit/distros/__init__.py", line 162, in apply_network_config
                                                                 dev_names = self._write_network_config(netconfig)
                                                               File "/usr/lib/python3.5/site-packages/cloudinit/distros/rhel.py", line 73, in _write_network_config
                                                                 self._net_renderer.render_network_state("/", ns)
                                                               File "/usr/lib/python3.5/site-packages/cloudinit/net/sysconfig.py", line 395, in render_network_state
                                                                 util.write_file(dns_path, resolv_content)
                                                               File "/usr/lib/python3.5/site-packages/cloudinit/util.py", line 1744, in write_file
                                                                 with open(filename, omode) as fh:
                                                             FileNotFoundError: [Errno 2] No such file or directory: '/etc/resolv.conf'

Also attaching full journal.

Metadata Update from @dustymabe:
- Issue tagged with: infra

6 years ago

I guess we can close this - but we should keep an eye to make sure things don't regress.

Metadata Update from @dustymabe:
- Issue status updated to: Closed (was: Open)

6 years ago

Login to comment on this ticket.

Metadata
Attachments 2
Attached 7 years ago View Comment
Attached 7 years ago View Comment