f9e53c7 ui: fix VNC client throttling when forced update is requested

4 files Authored by berrange 6 years ago, Committed by Michael Roth 6 years ago,
    ui: fix VNC client throttling when forced update is requested
    
    The VNC server must throttle data sent to the client to prevent the 'output'
    buffer size growing without bound, if the client stops reading data off the
    socket (either maliciously or due to stalled/slow network connection).
    
    The current throttling is very crude because it simply checks whether the
    output buffer offset is zero. This check is disabled if the client has requested
    a forced update, because we want to send these as soon as possible.
    
    As a result, the VNC client can cause QEMU to allocate arbitrary amounts of RAM.
    They can first start something in the guest that triggers lots of framebuffer
    updates eg play a youtube video. Then repeatedly send full framebuffer update
    requests, but never read data back from the server. This can easily make QEMU's
    VNC server send buffer consume 100MB of RAM per second, until the OOM killer
    starts reaping processes (hopefully the rogue QEMU process, but it might pick
    others...).
    
    To address this we make the throttling more intelligent, so we can throttle
    full updates. When we get a forced update request, we keep track of exactly how
    much data we put on the output buffer. We will not process a subsequent forced
    update request until this data has been fully sent on the wire. We always allow
    one forced update request to be in flight, regardless of what data is queued
    for incremental updates or audio data. The slight complication is that we do
    not initially know how much data an update will send, as this is done in the
    background by the VNC job thread. So we must track the fact that the job thread
    has an update pending, and not process any further updates until this job is
    has been completed & put data on the output buffer.
    
    This unbounded memory growth affects all VNC server configurations supported by
    QEMU, with no workaround possible. The mitigating factor is that it can only be
    triggered by a client that has authenticated with the VNC server, and who is
    able to trigger a large quantity of framebuffer updates or audio samples from
    the guest OS. Mostly they'll just succeed in getting the OOM killer to kill
    their own QEMU process, but its possible other processes can get taken out as
    collateral damage.
    
    This is a more general variant of the similar unbounded memory usage flaw in
    the websockets server, that was previously assigned CVE-2017-15268, and fixed
    in 2.11 by:
    
      commit a7b20a8efa28e5f22c26c06cd06c2f12bc863493
      Author: Daniel P. Berrange <berrange@redhat.com>
      Date:   Mon Oct 9 14:43:42 2017 +0100
    
        io: monitor encoutput buffer size from websocket GSource
    
    This new general memory usage flaw has been assigned CVE-2017-15124, and is
    partially fixed by this patch.
    
    Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
    Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
    Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
    Message-id: 20171218191228.31018-11-berrange@redhat.com
    Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
    (cherry picked from commit ada8d2e4369ea49677d8672ac81bce73eefd5b54)
    Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
    
        
file modified
+5 -0
file modified
+5 -0
file modified
+24 -4
file modified
+7 -0