5.5 Performance Evaluation
5.5.1 Performance in AWGN Channels
Phase Transition Diagrams
To assess the capability of the different algorithms with respect to the system load and pa, we first consider the phase transition diagram in the noise free case in Fig. 5.4 where the region of an Activity Error Rate (AER) below 10−3 is plotted. With AWGN channels, the instantaneous received power for all nodes is known. Therefore, it is not a surprise that the MMP and the MAP-E exhibit much better performance than the GOMP and MUSIC as MMP and MAP-E exploit the knowledge of a finite alphabet receive power. The underlying optimization problem for the MAP-E is of full rank as long as M2 ≥ N holds such that N nodes require a spreading sequence length of M = √
N only. This is validated as the MAP-E performs well with a relative spreading sequence length of M/N ≈ 0.1, corresponding to M = 10 which matches M = √
N = √
100. With increasing activity probability, the required spreading only increases moderately for the MAP-E.
The gap between the MAP-E and the MMP is only moderate and both algorithms exhibit only slight losses from increasing pa. The curve for the MMP decreases slightly for pa → 1, which is a side effect based on the genie knowledge for Nact.
Compared to that, the GOMP and MUSIC require higher spreading sequence lengths if the number of nodes increase. This behavior confirms
known results from the GOMP [Sch15]. The MUSIC algorithm shows only slight performance gains over the GOMP. Here we directly see the demand of the MUSIC for M < Nact to ensure a non-empty noise space. Increasing pa, also increases Nact and thus increasing the demand for longer spreading sequences for the MUSIC.
Performance over the SNR
To assess the performance of the algorithms in noisy environments, Fig. 5.5 plots the activity error rates achieved for two significant points from the phase transition diagram that only differ in the spreading sequence length.
Fig 5.5 (a) considers a relative spreading sequence length of M/N = 0.4.
For the GOMP and MUSIC the number of observations is too low and both fail at detecting the activity over the entire SNR range. MAP-E and the MMP enable activity detection and exhibit nearly the same performance, only differing in the error floor that the MMP has at higher SNR showing that the MAP-E approach is superior in this region.
Increasing the relative spreading to M/N = 0.4 as shown in Fig. 5.5 (b) removes the error floor of the MMP algorithm, making the performance of MMP and MAP-E to be the same. Further, the GOMP and MUSIC algorithm also show performance gains. Both algorithms converge to low activity error rates, but for higher SNRs. Most interestingly, the MUSIC nearly achieves the performance of the MAP-E and MMP. The SNR gap between these algorithms and the GOMP is significant.
Performance over the Frame Length
The superiority of the multiuser energy estimation algorithms over the GOMP is based on the fact that the noise averages out when the frame length LF increases. Therefore, the advantage over the GOMP has to be seen in conjunction with the particular frame length. To highlight this effect, we consider the activity error rate over the frame length LF for a fixed SNR of 1/σw2 = 0dB. Fig. 5.6 plots the activity error rates for this setup for a relative spreading of M/N = 0.15 (a) and M/N = 0.4 (b). On the left plot we see a strong decline in terms of activity error rates for the MMP and the MAP-E algorithm, which is almost log linear over the frame length. The error floor for the MMP appears at approximately a frame length of LF = 400 and the MMP does not gain from higher frame lengths, showing that the error floor is not caused by noise, but rather caused by multiuser interference that can not be resolved. In contrast to that the MAP-E yields performance gains as the frame length increases. With longer spreading sequences as shown in Fig. 5.6 (b) we observe a strong decline for
−20−15−10−5 0 5 10 15 20 10−4
10−3 10−2 10−1 100
1/σw2 in dB
AER
(a)
−20−15−10−5 0 5 10 15 20 10−4
10−3 10−2 10−1 100
1/σw2 in dB (b)
GOMP MUSIC MMP MAP-E
Figure 5.5: Activity Error Rates for multiuser energy estimation algorithms over the SNR in a system with N = 100, LF = 1000, pa = 0.2 and a relative spreading of M/N = 0.15 (left) and M/N = 0.4 (right).
0 200 400 600 800 1,000 10−4
10−3 10−2 10−1 100
LF
AER
(a)
0 200 400 600 800 1,000 10−4
10−3 10−2 10−1 100
LF (b)
GOMP MUSIC MP MAP-E
Figure 5.6: Activity Error Rates for multiuser energy estimation algorithms over the frame length LF in a system with pa = 0.2,1/σw2 = 0dB and a relative spreading of M/N = 0.15 (left) and M/N = 0.4 (right).
all multiuser energy estimation algorithms. This shows the strong denoising property of this approach, which is not exploited by the GOMP achieving only gains for short frames. Both analysis show that the denoising property of multiuser energy estimation is a clear advantage, that can already be expected with short frames of only LF ≈ 200 symbols.
Performance Involving Data Detection
The impact of the combination of multiuser energy detection and data detection is briefly discussed in the sequel. Therefore, the multiuser energy detector is followed by a linear least-squares data detector that estimates the data of nodes by solving the reduced multiuser system (2.21) determined by the estimate for the frame support SXˆ. The goal is to call the optimality of Nˆact = Nact into question. Varying this variable by over or underestimating Nact impacts the false alarm and missed detection rates. To investigate this impact we set ˆNact = Nact + ΔNact and consider the resulting BER.
Fig. 5.7 plots the BER for different estimates parametrized by ΔNact at a relative spreading of M/N = 0.4 and an SNR of 1/σw = 10dB. The curves
−5−4−3−2−1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 10−4
10−3 10−2 10−1 100
ΔNact
BER
GOMP MUSIC MMP MAP-E
Figure 5.7: BER at the output of a least-squares data detector followed. At a SNR of 1/σ2w = 10 dB, M/N = 0.4.
in Fig. 5.7 show a clear dependence of the BER and Nact for all multiuser energy detectors considered. In all cases the optimal value is ΔNact = 0, which matches the number of active nodes. Decreasing this value yields severe losses in terms of the BER due to the fact that with ΔNact < 0 missed detection errors occur. Increasing ΔNact leads to false alarm errors, decreasing the BER due to the false alarm SNR loss as highlighted in Section 2.4.4. However, it can remarkably be observed that this SNR loss is rather moderate for the GOMP. The reason can be found by visualizing that the activty detection performance of the GOMP is imperfect at the SNR considered. As shown in 5.5 (b), the GOMP only achieves an AER ≈ 10−3. Thus, increasing ΔNact does not only increase the false alarm SNR loss it also decreases the missed detection rate. Hence, the false alarm SNR loss is
compensated by the decreased missed detection rate, which has a positive impact on the BER.
The activity detection for MMP, MUSIC and MAP-E is nearly perfect for the SNR considered. Thus, changing ΔNact automatically either leads to a false alarm SNR loss or to missed detections. Applying one of these activity detection algorithms in combination with a least-squares data detector leads to an interference limited data detection. As a consequence, activity error control does not make sense in this setting. We will see that this effect has an even higher dominance in fading channels.