0.01 0.1 1
100 101 102 103 104 105
Coefficient of Variation cv(n)
n - Time Scale [Packets]
Trace Lower Bound
Markov Model: 2 Model Parameters Markov Model: 4 Model Parameters
Figure 5.1: Fit of the estimated Markov models
one of the states is always 1), whereas a more complex version also considers the loss rate for both states (Gilbert-Eliott model [74]). Figure 5.1 illustrates that the cv(n) obtained with the latter version closely matchescempiricalv (n) at all time-scales, whereas the other only matches at short time-scales. The figure also includes the lower boundcv(n) of a Poisson process with uncorrelated errors.
5.3 Video Quality Evaluation of Impaired Video Sequences
e.g., the amount of lost macroblocks or an indicator for freezing. The observed visual impairments depend on the decoder’s error concealment technique, e.g., freezing or slicing. In the case of freezing, the picture is frozen until the error is completely recovered. Depending on the used decoder version, slicing, however, conceals the error on a macroblock level using motion compensation which may result in visual (block) artifacts within the image.
Focusing on visual error patterns derived from the video decoder allows us to take a microscopic view at factors influencing quality. As this work is motivated by supporting quality studies, we aim at optimizing microscopic aspects. Ideally, this optimization should yield model generators that produce similar error patterns as measurement traces. The macroscopic perspective is then added in a later stage by quality studies using the loss generators.
5.3.1 Freezing
Visual artifacts introduced byfreezing can be fully characterized byi) the amount of freezing events, ii) the temporal extent of each event in the number of frozen frames, and iii)the total number of frozen frames per video. We show these visual property metrics combined for all five video sequences per loss model as a function of the packet loss rate in Figure 5.2.
The Markov model with two parameters (blue dotted line) departs from the baseline results (trace represented by the red solid line) and shows a high level of variability.
In particular, it yields a high variability in the freezing event length distribution for each loss rate (standard deviations not shown in Figure 5.2(b)). In contrast, the more complex Markov model with four parameters (black dashed line) provides a reasonably good fit to the baseline result and yields a lower variability.
5.3.2 Slicing
Figure 5.3 shows visual properties of the slicing concealment method, i.e.,i)the total number of frames with lost macroblocks,ii)the average number of lost macroblocks per impaired video frame with at least one erroneous macro block, iii) the total number of impairment events, and iv) the average length of each event. As in Section 5.3.1, each metric is shown as a function of the packet loss rate for each of the three loss sources: the trace representing the baseline result (red solid line), and the two Markov models fitted to the trace.
The figures show that compared to the version of the Markov chain with two param-eters, the version with four parameters provides a closer match to the considered visual error characteristics, especially at low loss rates. Only the average number of impaired macroblocks is not matched. This suggests that the high variability of video traces require adequate models to generate loss traces with high variability.
0 5 10 15 20
2 4 6 8 10 12 14 16
# Freezing Events
Packet Loss % Video Bitrate: 16 Mbit/s CBR
Original Trace Markov Model: 4 Parameters Markov Model: 2 Parameters
(a) # Freezing Events
0 50 100 150 200
2 4 6 8 10 12 14 16
Avg Freezing Event Length [Frames]
Packet Loss % Video Bitrate: 16 Mbit/s CBR
Original Trace Markov Model: 4 Parameters Markov Model: 2 Parameters
(b) Freezing Event Length
50 100 150 200 250 300 350 400
2 4 6 8 10 12 14 16
Number of Frozen Frames
Packet Loss % Video Bitrate: 16 Mbit/s CBR
Original Trace Markov Model: 4 Parameters Markov Model: 2 Parameters
(c) Total Frozen Frames
Figure 5.2: Visual video quality properties for the freezing concealment method.
5.3 Video Quality Evaluation of Impaired Video Sequences
0 50 100 150 200 250 300
4 6 8 10 12 14
# Frames With Lost Macroblocks
Packet Loss % Video Bitrate: 16 Mbit/s CBR
Original Trace Markov Model: 4 Parameters Markov Model: 2 Parameters
(a) Total Frames with Errors
1000 2000 3000 4000 5000 6000 7000
4 6 8 10 12 14
Avg. Impaired Macro Blocks
Packet Loss % Video Bitrate: 16 Mbit/s CBR
Original Trace Markov Model: 4 Parameters Markov Model: 2 Parameters
(b) Erroneous Error Blocks per Impaired Frame
0 20 40 60 80 100 120
4 6 8 10 12 14
# Impairment Events
Packet Loss % Video Bitrate: 16 Mbit/s CBR
Original Trace Markov Model: 4 Parameters Markov Model: 2 Parameters
(c) # Impairment Events
1 1.5 2 2.5 3 3.5 4 4.5
4 6 8 10 12 14
Avg. Impairment Length [Frames]
Packet Loss % Video Bitrate: 16 Mbit/s CBR
Original Trace Markov Model: 4 Parameters Markov Model: 2 Parameters
(d) Event Length
Figure 5.3: Visual video quality properties for the slicing concealment method.
As compared to metrics describing freezing events (cf. Section 5.3.1), metrics de-scribing slicing errors involve much higher variability. This is expressed by large standard deviations (not shown) for metrics shown in Figure 5.3(b) and 5.3(d). Due to overlapping standard deviations, differences in the models are not statistically significant.
5.3.3 Influence of Fitting Method
To highlight the impact of the chosen model and its fitting method, we compare the model summarized in Section 5.2 (second-order statistics) to the following models:
i) the 2-state Markov chain fitted by applying the Baum-Welch algorithm [247, 231] representing a classical fitting method, and ii) a Poisson process generating uniformly distributed packet losses. The impact of the fitting method / model is shown in Figure 5.4 for the average number of freezing events as a function of the packet loss rate. We show this dependence for all the four used video encoding bitrates.
2 4 6 8 10 12 14
0 0.5 1 1.5 2 2.5 3 3.5 4
# Freezing Events
% Packet Loss 16 Mbps 8 Mbps 4 Mbps 2 Mbps
(a) Second-order statistics
0 2 4 6 8 10 12
0 0.5 1 1.5 2 2.5 3 3.5 4
# Freezing Events
% Packet Loss 16 Mbps 8 Mbps 4 Mbps 2 Mbps
(b) Baum-Welch
0 2 4 6 8 10 12 14
0 0.5 1 1.5 2 2.5 3 3.5 4
# Freezing Events
% Packet Loss 16 Mbps 8 Mbps 4 Mbps 2 Mbps
(c) Uniform Loss
Figure 5.4: Average number of freezing events per video
The obtained results suggest that different modeling approaches do result in different visual error patterns for the same amount of lost packets. This is expressed by different amounts of freezing events per loss rate for each considered fitting technique in Figure 5.4 a), b), and c). As a consequence, it can be said that the choice of the used model and fitting technique therefore matters. The first evaluation using DVB-H traces suggests that a fitting procedure based on second-order statistics (see Section 5.2) more closely replicates visual impairment properties than classical burst length based packet loss modeling techniques (see Section 5.3.1 and 5.3.2).