GStreamer Quality Of Service (QoS)
Quality of Service in GStreamer is about measuring and adjusting the real-time performance of a pipeline.
The real-time performance is always measured relative to the pipeline clock and typically happens in the sinks when they synchronize buffers against the clock.
When buffers arrive late in the sink, i.e. when their running-time is smaller than that of the clock, we say that the pipeline is having a quality of service problem. These are a few possible reasons:
-
High CPU load, there is not enough CPU power to handle the stream, causing buffers to arrive late in the sink.
-
Network problems
-
Other resource problems such as disk load, memory bottlenecks etc
The measurements result in QOS events that aim to adjust the datarate in one or more upstream elements. Two types of adjustments can be made:
Short time "emergency" corrections based on latest observation in the sinks.
Long term rate corrections based on trends observed in the sinks.
It is also possible for the application to artificially introduce delay between synchronized buffers, this is called throttling. It can be used to limit or reduce the framerate, for example.
Measuring QoS
Elements that synchronize buffers on the pipeline clock will usually measure the current QoS. They will also need to keep some statistics in order to generate the QOS event.
For each buffer that arrives in the sink, the element needs to calculate how late or how early it was. This is called the jitter. Negative jitter values mean that the buffer was early, positive values mean that the buffer was late. the jitter value gives an indication of how early/late a buffer was.
A synchronizing element will also need to calculate how much time elapsed between receiving two consecutive buffers. We call this the processing time because that is the amount of time it takes for the upstream element to produce/process the buffer. We can compare this processing time to the duration of the buffer to have a measurement of how fast upstream can produce data, called the proportion. If, for example, upstream can produce a buffer in 0.5 seconds of 1 second long, it is operating at twice the required speed. If, on the other hand, it takes 2 seconds to produce a buffer with 1 seconds worth of data, upstream is producing buffers too slow and we won't be able to keep synchronization. Usually, a running average is kept of the proportion.
A synchronizing element also needs to measure its own performance in order to figure out if the performance problem is upstream of itself.
These measurements are used to construct a QOS event that is sent upstream. Note that a QoS event is sent for each buffer that arrives in the sink.
Handling QoS
An element will have to install an event function on its source pads in order to receive QOS events. Usually, the element will need to store the value of the QOS event and use it in the data processing function. The element will need to use a lock to protect these QoS values as shown in the example below. Also make sure to pass the QoS event upstream.