Decreasing termatiko distance by adjoining redundant rows to a measurement matrix

As it was discussed in Chapter 2, for BP decoding over the BEC one can add re-dundant rows to a parity-check matrix in order to decrease the number of stopping sets [46]. This is also the case for relaxed linear programming decoding of binary linear codes on any symmetric channel [15]. In this section, we aim to improve the recovery performance of the IPA by adding redundant rows to a measurement mat-rix, inspired by success on the BEC. However, there is one fundamental difference in the sense that the real linear combinations that are added to the measurement matrix should contain non-negative entries only. Furthermore, we would like to stress that redundant rows that we add to the measurement matrix are not used to provide new measurements, but rather used in the recovery process, which means that also measurements need to be linearly combined at the receiver. Thus, this procedure does not make the compression rate of the scheme worse, but rather potentially improve the recovery performance.

The following lemma shows that adding redundant rows to the measurement matrix does not harm IPA reconstruction performance, namely that it does not create new termatiko sets.

Lemma 43. Adding redundant measurements does not create new termatiko sets.

Proof. Let the original measurement matrix be denoted byA. Its extended version with non-negative redundant rows is denoted byA⁰. The matrixA⁰ is constructed such that the first rows ofA⁰are exactly the rows ofAand the remaining rows are real-valued linear combinations of the rows ofAwith non-negative entries.⁹

Denote also the Tanner graph corresponding toA⁰by(V⁰∪C⁰, E⁰), and letN⁰, N_T⁰ be the notation for neighbours in this Tanner graph (analogously to (1.1)).

Consider some signal vectorx and two problems, IPA(y, A) andIPA(y⁰, A), wherey=xA^|andy⁰ =xA^0|.

The set of variable nodes is the same, i.e.V =V⁰, but the set of measurement nodes is now a superset of the original set, i.e. C ⊂C⁰. The same is true for the set of edges,E⊂E⁰. It holds for allv∈V thatN⁰(v) =N_C⁰(v)∪ N_C⁰0\C(v) = N(v)∪ N_C⁰0\C(v). For allc ∈ C, we haveN⁰(c) = N(c). This in turn means thatyc=y_c⁰ forc∈C.

9Non-negativity of matrix entries is important for the correctness of the IPA.

Table7.Codewordsupportmatricessplitintotermatikosets.Verticallinesillustratehowtosplitthecodewordsintopairsofdistincttermatikosets eachofhalfthesize. MatrixCodeword weightCodewordsupportmatrixsplit H(q,4) q≥1110

" 0−6−24−12−300−12−24−6−30 03−13−4−1230−12−4−13 012−2466120−24 0219122492412021

# H(5,4) z∈F5\{0} k∈{0,2z}8

" 03k+3z2k+4z2z03k+3z2k+4z2z 03zk+4zk+2zk+4z0k+2z3z 02k+3z4z2k+2z2k+3z2k+2z04z 04k+3z4k+4z3k+2z3k+2z4k+4z4k+3z0

# H(7,4) z∈F7\{0} k∈{0,2z,4z,6z}8

" 02k+5z2k+z4z02k+5z2k+z4z 0k+2z5zk+4zk+2z0k+4z5z 06z5k+2z2k+4z2k+4z5k+2z06z 06k+3z3k+6z3k+4z3k+6z3k+4z6k+3z0

# H(q,5) q6=1112

 

0−4−18−22−6−160−6−22−18−4−16 01−8−12−3−1110−11−12−3−8 062−20−6260−6−20 0111283−1312110−18 016221864418226016

  H(11,5)10

 

0547674056 101083131080 2659062905 310108018310 4760550764

 

8.Codewordsupportmatricessplitintotermatikosets(continued).Verticallinesillustratehowtosplitthecodewordsintopairsofdistinct osetseachofhalfthesize. Codeword weightCodewordsupportmatrixsplit ,6)12

  

036254265403 065431306514 024615410625 053162521036 012346632140 041523043251

   ,6)16

  

010157660610510706 047102698708910624 0924861581069542 0389364292338460 083396710103607389 029846107049862107

   q,6) 1120

  

0−22−2−2010−812−10−3222−10−210−3222−200−8−2212 0−168−89−7171−1516−8016−1617−1591−78 0−101848−62212210−6222012−10181084 0−428167−52723194−4428167−52719230 0238286−4323436−2−26343220362838−4 0848405−3374553−8084048−35453753−8

   q,7)24

    

0−18−14−20−8−4826−1210−2260−4−228−2010−12−8−182−14 0−14−10−12−71648−69−15640−149−158−10−6−121−7 0−10−6−4−66461008−8684−610−106−8−4−600 0−6−24−511281267−16128211−54−6−20−17 0−2212−41601014126661612101202−406−214 02620−321−212161851362016181350−2212−321 061028−226−414182442062420261410−20418−428

    

Table 9.Termatiko distances of array LDPC code matricesH(q, a).

Let µ⁰ and M⁰ (with corresponding indices) be bounds in the iterations of IPA(y⁰, A⁰). In order to prove the statement of the lemma, it is enough to show Inductive Step.Consider iteration`≥1. At each step`of the IPA and for all c∈Candv ∈ N⁰(c) =N(c), we have apply these inequalities to Lines 14 and 15 in Algorithm 1 and, using properties of the functionsmin(·)andmax(·), we obtain the desired result.

From Lemma 43 it follows that adding redundant rows to the measurement matrix cannot harm the IPA. The following example shows that adding such rows can indeed improve the performance of the IPA by removing termatiko sets.

Example 44. Consider the binary measurement matrix

A=

v₁ v₂ sets defined in Theorem 35. T is a termatiko set. measurement nodec∗. T is not a termatiko set any more.

Figure 22.Adding a redundant measurementc_∗corresponding to the difference of rows c₂andc₁of the matrix in Example 44.

The corresponding Tanner graph is shown in Fig. 22a. Note that the set{v₁, v2} is a termatiko set for this matrix. However, if we add a redundant rowc∗equal to the difference of rowsc₂ andc₁,{v₁, v₂}is not a termatiko set for the extended

sincec₄violates conditions in Theorem 35:

• c4is not connected toS⁰, and

Fig. 22b illustrates the differences. 4

The following question arises: which redundant rows do we need to add in order to remove the largest number of harmful small-size termatiko sets. We pro-pose the following heuristic approach. First, fix some list of small-size termatiko sets for the original measurement matrixAand generate a pool of redundant rows which (hopefully) help to remove at least one termatiko set from the list as fol-lows.

Consider a termatiko set T from the list and its corresponding set S. A re-dundant rowr= (r1, r2, . . . , rn)for the measurement matrixAcan be uniquely

10Recall that operations are performed overR.

defined by coefficientsα₁, α₂, . . . , α_m∈Rin a linear combination r_v =X

c∈C

a_cvα_c.

However, since in the real calculations floating-point numbers are effectively ra-tional numbers, by multiplying allα’s by some common multiplier of their de-nominators, we can make them all integer, and they still produce a redundant row rwith the same support. Therefore, with no loss of generality, we assume thatα’s are integers. If original matrixAhas integer entries, the resulting extended matrix has integer entries as well, which allows for a faster IPA in applications where the signalxis integer.

There are two types of redundant rows that will be collected in the pool. The first type “breaks” the termatiko setT for sure. It has one non-zero entry in the positions inT and zeroes in entries indexed byS. The other entries ofrcan be chosen arbitrarily. More precisely, for a fixedv0 ∈T we solve the (integer) linear programming problem and is used to obtain smaller coefficients in a redundant row. In fact, for any feasible solution, the corresponding redundant row eliminates the termatiko set T. A redundant row can potentially be obtained for eachv₀ ∈ T. As a final remark, relaxing the α’s to be real numbers turns the program into a standard linear program that can be solved using the simplex method. However, as noted above, having integers (of moderate size) in the measurement matrix has some potential benefits. Thus, when the size of the program is not too large and can be solved using a standard solver in a reasonable time (which is the case in our examples), we keep the integer constraint on theα’s.

Redundant rows of the second type do not necessarily “break” T, but they have good chances for doing that. The basic idea is to make variable nodes inS to not satisfy Theorem 35. Hence, they are not included inS for the extended matrix. Hopefully, this eliminatesT as a termatiko set for the extended matrix.

Note that having several non-zero entries in positions inS is better, since all of them disappear fromS (and we do not add new ones toS). This have a greater chance of removingT. The corresponding (integer) linear program is

rv

X

v∈S

r_v ≥10|S|,

where the constants10and1000are chosen rather arbitrarily;10is used in order to make non-zero entries inr_S more likely, and the upper bounds of1000make sure the entries inrare of limited size. Note that no objective function is spe-cified, since any feasible solution will do. For each termatiko setT, this approach produces at most one redundant row.

Finally, after constructing the pool of redundant rows as described above, we start adjoining them to the matrixAone by one in a greedy manner as follows.

Let the list of termatiko sets be denoted byLISTand the pool of redundant rows byPOOL. For each rowr∈POOL, we calculate the score

score(r) = X

T∈RMV(LIST,r)

|T|,

whereRMV(LIST,r)is a subset ofLISTconsisting of termatiko sets that are not termatiko sets after adjoining rowrto the current measurement matrix. The row r^∗with the maximum score is adjoined to the measurement matrix, the termatiko sets inRMV(LIST,r) are removed from LIST, and the scores are re-calculated for the updatedLISTand measurement matrix. The procedure is continued until LISTis empty or all scores are zero (which means that no additional termatiko sets can be removed).