Package backend :: Package daemons :: Module dispatcher :: Class Worker
[hide private]
[frames] | no frames]

Class Worker

source code


Worker process dispatches building tasks. Backend spin-up multiple workers, each worker associated to one group_id and process one task at the each moment.

Worker listens for the new tasks from :py:class:`retask.Queueu` associated with its group_id

:param Bunch opts: backend config :param queue: (:py:class:`multiprocessing.Queue`) queue to announce new events :param int worker_num: worker number :param int group_id: group_id from the set of groups defined in config :param callback: callback object to handle internal workers events. Should implement method ``log(msg)``. :param lock: (:py:class:`multiprocessing.Lock`) global backend lock

Instance Methods [hide private]
 
__init__(self, opts, events, worker_num, group_id, callback=None, lock=None)
x.__init__(...) initializes x; see help(type(x)) for signature
source code
 
event(self, topic, template, content=None)
Multi-purpose logging method.
source code
 
_announce_start(self, job)
Announce everywhere that a build process started now.
source code
 
_announce_end(self, job)
Announce everywhere that a build process ended now.
source code
 
run_ansible_playbook(self, args, name='running playbook', attempts=9)
Call ansible playbook:
source code
 
validate_vm(self)
Test connectivity to the VM
source code
 
try_spawn(self, args)
Tries to spawn new vm using ansible
source code
 
spawn_instance(self)
Spawn new VM, executing the following steps:
source code
 
terminate_instance(self)
Call the terminate playbook to destroy the building instance
source code
 
mark_started(self, job)
Send data about started build to the frontend
source code
 
return_results(self, job)
Send the build results to the frontend
source code
 
starting_build(self, job)
Announce to the frontend that a build is starting.
source code
 
spawn_instance_with_check(self)
Wrapper around self.spawn_instance() with exception checking
source code
 
init_fedmsg(self)
Initialize Fedmsg (this assumes there are certs and a fedmsg config on disk)
source code
 
on_pkg_skip(self, job)
Handle package skip
source code
 
obtain_job(self)
Retrieves new build task from queue.
source code
 
do_job(self, job)
Executes new job.
source code
 
check_vm_still_alive(self)
Ensure that if we have vm_ip it is alive.
source code
 
update_process_title(self, suffix=None) source code
 
run(self)
Worker should startup and check if it can function for each job it takes from the jobs queue run opts.setup_playbook to create the instance do the build (mockremote) terminate the instance.
source code

Inherited from multiprocessing.process.Process: __repr__, is_alive, join, start, terminate

Inherited from multiprocessing.process.Process (private): _bootstrap

Inherited from object: __delattr__, __format__, __getattribute__, __hash__, __new__, __reduce__, __reduce_ex__, __setattr__, __sizeof__, __str__, __subclasshook__

Class Methods [hide private]
 
pkg_built_before(cls, pkg, chroot, destdir)
Check whether the package has already been built in this chroot.
source code
Class Variables [hide private]

Inherited from multiprocessing.process.Process (private): _Popen

Properties [hide private]
  group_name

Inherited from multiprocessing.process.Process: authkey, daemon, exitcode, ident, name, pid

Inherited from object: __class__

Method Details [hide private]

__init__(self, opts, events, worker_num, group_id, callback=None, lock=None)
(Constructor)

source code 

x.__init__(...) initializes x; see help(type(x)) for signature

Overrides: object.__init__
(inherited documentation)

event(self, topic, template, content=None)

source code 

Multi-purpose logging method.

Logs messages to three different destinations:

  • To log file
  • The internal "events" queue for communicating back to the dispatcher.
  • The fedmsg bus. Messages are posted asynchronously to a zmq.PUB socket.

run_ansible_playbook(self, args, name='running playbook', attempts=9)

source code 

Call ansible playbook:

  • well mostly we run out of space in OpenStack so we rather try multiple times (attempts param)
  • dump any attempt failure

validate_vm(self)

source code 

Test connectivity to the VM

:param ipaddr: ip address to the newly created VM :raises: :py:class:`~backend.exceptions.CoprWorkerSpawnFailError`: validation fails

try_spawn(self, args)

source code 

Tries to spawn new vm using ansible

:param args: ansible for ansible command which spawns VM :return str: valid ip address of new machine (nobody guarantee machine availability)

spawn_instance(self)

source code 

Spawn new VM, executing the following steps:

  • call the spawn playbook to startup/provision a building instance
  • get an IP and test if the builder responds
  • repeat this until you get an IP of working builder

:param BuildJob job: :return ip: of created VM :return None: if couldn't find playbook to spin ip VM

starting_build(self, job)

source code 

Announce to the frontend that a build is starting.

:return True: if the build can start :return False: if the build can not start (build is cancelled)

spawn_instance_with_check(self)

source code 

Wrapper around self.spawn_instance() with exception checking

:param BuildJob job:

:return str: ip of spawned vm :raises:

  • :py:class:`~backend.exceptions.CoprWorkerError`: spawn function doesn't return ip
  • :py:class:`AnsibleError`: failure during anible command execution

obtain_job(self)

source code 

Retrieves new build task from queue. Checks if the new job can be started and not skipped.

do_job(self, job)

source code 

Executes new job.

:param job: :py:class:`~backend.job.BuildJob`

check_vm_still_alive(self)

source code 

Ensure that if we have vm_ip it is alive. Terminates unresponsive instance.

run(self)

source code 

Worker should startup and check if it can function for each job it takes from the jobs queue run opts.setup_playbook to create the instance do the build (mockremote) terminate the instance.

Overrides: multiprocessing.process.Process.run

Property Details [hide private]

group_name

Get Method:
unreachable.group_name(self)