502e05c runqueue: add cpu/io pressure regulation

1 file Authored by Aryaman Gupta 2 years ago, Committed by Richard Purdie 2 years ago,
    runqueue: add cpu/io pressure regulation
    
    Prevent the scheduler from starting new tasks if the current cpu or io
    pressure is above a certain threshold and there is at least one active
    task. This threshold can be specified through the
    "BB_PRESSURE_MAX_{CPU|IO}" variables in conf/local.conf.
    
    The threshold represents the difference in "total" pressure from the
    previous second. The pressure data is discussed in this oe-core commit:
       061931520b buildstats.py: enable collection of /proc/pressure data
    where one can see that the average and "total" values are available.
    From tests, it was seen that while using the averaged data was somewhat
    useful, the latency in regulating builds was too high. By taking the
    difference between the current pressure and the pressure seen in the
    previous second, better regulation occurs. Using a shorter time period
    is appealing but due to fluctations in pressure, comparing the current
    pressure to 1 second ago achieves a reasonable compromise. One can look
    at the buildstats logs, that usually sample once per second, to decide a
    sensible threshold.
    
    If the thresholds aren't specified, pressure is not monitored and hence
    there is no impact on build times. Arbitary lower limit of 1.0 results
    in a fatal error to avoid extremely long builds. If the limits are higher
    than 1,000,000, then warnings are issued to inform users that the specified
    limit is very high and unlikely to result in any regulation.
    
    The current bitbake scheduling algorithm requires that at least one
    task be active. This means that if high pressure is seen, then new tasks
    will not be started and pressure will be checked only for as long as at
    least one task is active. When there are no active tasks, an additional task
    will be started and pressure checking resumed. This behaviour means that
    if an external source is causing the pressure to exceed the threshold,
    bitbake will continue to make some progress towards the requested target.
    This violates the intent of limiting pressure but, given the current
    scheduling algorithm as described above, there seems to be no other option.
    In the case where only one bitbake build is running, the implications of
    the scheduler requirement will likely result in pressure being higher
    than the threshold. More work would be required to ensure that
    the pressure threshold is never exceeded, for example by adding pressure
    monitoring to make and ninja.
    
    Signed-off-by: Aryaman Gupta <aryaman.gupta@windriver.com>
    Signed-off-by: Randy Macleod <randy.macleod@windriver.com>
    Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
    
        
file modified
+65 -0