1.5 Notation
In this work, vectors are typeset in boldface lowercase letters, and matrices in boldface uppercase letters. Whenever a matrix or vector is described in parallel by a complex representation and a real-valued representation, we use sans-serif font (Aorx) for the complex representation and serif font with additional accents (A`orx) for the real-valued representation (see Section 2.9). Onˇ the other hand, in equations that are valid no matter whether the considered matrices and vectors are complex or real, serif font (Aorx) is used throughout. Note that the conjugate-transpose operator in this kind of equations becomes equivalent to the conventional transpose if real matrices and vectors are plugged in.
We use the following special matrices, vectors, scalars, and sets. In addition, notations for further special sets are introduced in Section 2.1.1.
0 zero matrix or vector of appropriate size 1 all-ones vector of appropriate length
ei ith canonical unit vector of appropriate length IL identity matrix of sizeL
d(A) vector containing the diagonal elements of the matrixA λ(A) vector containing the eigenvalues of the matrixA Cx covariance matrix of a random vectorx
Cx,y cross-covariance matrix of the random vectorsxandy diag(αi) diagonal matrix with diagonal elementsαi
blkdiag(A(c)) block-diagonal matrix with diagonal blocksA(1), . . . ,A(C) stack(a(c)) stacked vector[a(1),T, . . . ,a(C),T]T
ai ith element of a vectora
e Euler’s number
j imaginary unit
RN0,+ closed positive orthant ofRN, i.e.,RN0,+={x∈RN:xi ≥0, ∀i}
The following operators are used.
AT transpose of a matrix or vector
AH conjugate-transpose of a matrix or vector
A∗ complex conjugation
<(•) real part
=(•) imaginary part
<`(A) real representation of a complex matrix, see Section 2.9.5
<ˇ(a) real representation of a complex vector, see Section 2.9.2 A−1 inverse of a matrix
A+ Moore-Penrose pseudoinverse of a matrix
A12 matrix square root of a positive-semidefinite matrix tr[A] trace of a matrix
detA determinant of a matrix rank [A] rank of a matrix null[A] null space of a matrix
range[A] column space of a matrix
hA,Bi (Frobenius) inner product ofAandB, see Section 2.1.1 f ◦g function composition
E[•] expectation
h (x) differential entropy of a random vectorx, see Section 2.1.3 h (x|y) conditional differential entropy ofxconditioned ony
I (x;y) mutual information between the random vectors x and y, see Sec-tion 2.1.3
I (x;y|z) conditional mutual information ofxandyconditioned onz
|a| absolute value ofa
|X | cardinality of a setX
X ∩ Y intersection of the setsX andY S
i∈IXi union of the setsXifor alli∈ I X \ Y set difference of the setsX andY X × Y Cartesian product of the setsX andY N
i∈IXi Cartesian product of the setsXifor alli∈ I convX convex hull of the setX
projA(•) projection to a subspaceA
A⊥V orthogonal complement of a subspaceAin a vector spaceV A+B sum of the subspacesAandB, i.e.,{A+B|A∈ A, B∈ B}
A ⊕ B orthogonal sum of the subspacesAandB, i.e., the same asA+B, but with the implication thatAandBare orthogonal
Expressions involving variables with a superscript index and an exponent (or an exponent-like operator), are written in the formA(i),kfor(A(i))kandA(i),Hfor(A(i))H. Moreover, we use shorthand notations of the forms
• (ak)∀k= (ak)k=1,...,K = (a1, . . . , aK),
• (a(c))∀c= (a(c))c=1,...,C = (a(1), . . . , a(C)),
• (a)∀c= (a)c=1,...,C = (a, . . . , a)(tuple withCelements),
• (CxS)S∈S = (CxS1, . . . ,CxS
|S|)whereS = (S1, . . . ,S|S|),
• {ak}k∈K ={ak1, . . . , ak|K|}, whereK={k1, . . . , k|K|}.
Finally, we make use of the following partial orderings and preorderings.
a≥b b≤a ai≥bifor all componentsi= 1, . . . , N(a,b∈RN) ab b≺a bis majorized bya, see Section 2.1.2
A0 0≺A Ais positive-definite
A0 0A Ais positive-semidefinite (psd) AB BA A−Bis positive-semidefinite (psd) X ⊇ Y Y ⊆ X Yis a subset ofX or equal toX X ⊃ Y Y ⊂ X Yis a strict subset ofX
Chapter 2
Power Shaping Spaces and Maximum-Entropy Signals
As already mentioned in Section 1.4, the concept of so-calledpower shaping spacescan be used to formally describe maximum-entropy signals, i.e., signals which have the highest differential entropy within a certain family of signals. In this chapter, we introduce this framework, which is the mathematical foundation of many derivations in this work. After revisiting some mathematical preliminaries, we give a brief motivation and the formal definition of a power shaping space. Then, we derive fundamental properties of such spaces, and we give examples of power shaping spaces including those that are particularly relevant for the investigations that follow.
2.1 Mathematical Preliminaries
In addition to the notational conventions introduced in Section 1.5, the following definitions and results are needed for our derivations.
2.1.1 Vector Spaces of Matrices
We make intensive use of the fact that the space of complexN ×M matricesCN×M and the space of realN×MmatricesRN×M with the Frobenius inner product (e.g., [68, Sec. 5.2])
hA,Bi= tr[BHA] (2.1)
are Hilbert spaces. Note that tr[BHA] = tr[ABH], which turns out to be useful in many derivations. For spaces of tuples of matrices (Ak)∀k, (Bk)∀k, we use the inner product h(Ak)∀k,(Bk)∀ki=PK
k=1tr[BkHAk].
It is easy to verify that the spaces of Hermitian matrices
HM ={S∈CM×M |S =SH} ⊂CM×M (2.2) and real symmetric matrices
SM ={S ∈RM×M |S =ST} ⊂RM×M (2.3)
23
are linear subspaces, i.e., they are vector spaces themselves. WithinHM (and thus also in SM ⊂HM), the definition of the inner product simplifies tohA,Bi= tr[BA].
We use the following notation for spaces of block-diagonal matrices.
Definition 2.1.1. LetL = (`1, . . . , `C) ∈ NC andM = (m1, . . . , mC) ∈ NC be tuples of natural numbers. We useCL×M(orRL×M) to denote the space of complex (or real-valued) block-diagonal matrices where the diagonal blocks are matrices with sizes`1×m1. . . , `C,×mC. Accordingly, the notationHM(orSM) is used for the space of Hermitian (or real symmetric) block-diagonal matrices with block sizesm1×m1, . . . , mC×mC.
2.1.2 Majorization Theory
Majorization theory has been used to study various aspects of wireless communication systems, as summarized, e.g., in [61, 69], and it is also an important ingredient for establishing the framework of power shaping spaces. In line with the existing literature (e.g., [61,68–71]), we introduce the following definition.
Definition 2.1.2. Leta↓idenote theith largest component of a vectora. Then, a vectorx∈RM is majorized by a vectory∈RM, denoted byx≺y, if
m
X
i=1
x↓i≤
m
X
i=1
y↓i (2.4)
for allm= 1, . . . , M with equality form=M.
Intuitively speaking,x≺ymeans that the entries ofyare more spread out than those ofx while the sum over all components is the same for both vectors. According to [70, Sec. 1.A.3], we havex≺yif and only ifPM
i=1|xi−a| ≤PM
i=1|yi−a|, ∀a∈R. The following Lemma is a direct consequence of this equivalence.
Lemma 2.1.1. If xj ≺ yj for all pairs of vectors xj,yj ∈ RMj, j = 1, . . . , J, then [xT1, . . . ,xTJ]T ≺[y1T. . . ,yJT]T.
Below, we provide some further statements about majorization, which are helpful for the derivations that follow in the next sections.
Lemma 2.1.2([70, Sec. 1.A]). For any given vectorx∈RM withPM
i=1xi=α, it holds that
α
M1≺x≺αem,m∈ {1, . . . , M}.
Lemma 2.1.3([70, Th. 9.B.1]). For any Hermitian matrixS ∈HM, we haved(S)≺λ(S), i.e., the vector of diagonal elements is majorized by the vector of eigenvalues.
So-called Schur-convex and Schur-concave functions preserve the preordering of majoriza-tion [70]. We make use of the following definimajoriza-tions and properties.
Definition 2.1.3(cf. [70, Definition 3.A.1]). A functionf :x7→f(x)is said to be Schur-convex (or Schur-concave) on a setX ⊆RM ifx≺ywithx,y ∈ X implies thatf(x)≤f(y)(or thatf(x)≥f(y), respectively). The function is said to be strictly Schur-convex (or strictly Schur-concave) in case that equalityf(x) =f(y)holds only ifyis a permuted version ofx.