We consider a Hilbert spaceH. Asesquilinear formis a function B:H × H →K that is linear in the first and conjugate linear in the second argument, and that is bounded in the sense that there is a constantM such that|B(x, y)| ≤Mkxk·kyk for allx, y∈ H. IfT is a bounded linear operator on H, then B(x, y) = (T x|y) defines a sesquilinear form withM =kTkin view of the Inequality of Cauchy and Schwarz saying that|(x|y)| ≤ kxk·kyk. (For our purposes we included continuity in the definition of sesquilinearity.)
Lemma 4.11. If B is a sesquilinear form, then there exists a unique bounded operator T ofHsuch that kTk ≤M and that B(x, y) = (T x|y).
Proof. Exercise. ut
Exercise E4.3. Prove Lemma 4.11.
[Hint. Fixx∈ H. The functiony 7→B(x, y) is a bounded conjugate linear form onH. Hence there is a unique elementT x ∈ Hsuch thatB(x, y) = (T x|y) by the elementary Riesz Representation Theorem for Hilbert spaces. The function T = (x7→T x):H → His linear. Use|B(x, y)| ≤Mkxk·kykto deducekTk ≤M.]ut Lemma 4.12. Let G denote a compact group and T a bounded operator on a Hilbert G-module E. Then there is a unique bounded operator Te on E with kTek ≤ kTksuch that
(5) (T xe |y) = Z
G
(T gx|gy)dg= Z
G
(π(g)−1T π(g)(x)|y)dg.
Proof . Since π is a unitary representation, π(g)∗ =π(g)−1 and so the last two integrals in (5) are equal. The prescriptionB(x, y) = R
G(T gx | gy)dg defines a functionB which is linear inxand conjugate linear iny. Because
|(T gx|gy)| ≤ kT gxk·kgyk=kTk·kgxk·kgyk=kTk·kxk·kyk (asGacts unitarily onH!) we obtain the estimate|B(x, y)| ≤R
GkTk·kxk·kykdg= kTk·kxk·kyk. Hence B is a sesquilinear form, and so by Lemma 4.11, there is a bounded operatorTewithB(x, y) = (T xe |y) andkTek ≤ kTk. ut In any ring R, the commutant C(X) (or, in semigroup and group theory equivalently called the centralizer Z(X, R)) of a subset X ⊆ R is the set of all elements r∈ R with xr = rxfor all x∈ X. Using integration of no more than K-valued functions, we have created the operator
Te= Z
G
π(g)−1T π(g)dg,
where the integral indicates an averaging over the conjugatesπ(g)−1T π(g) ofT. It is clear that the averaging self-mapT 7→Teof Hom(H,H) is linear and bounded.
Its significance is that its image is exactly the commutant C π(G)
of π(G) in Hom(H,H). Thus it is the set of all bounded operatorsSonHsatisfyingSπ(g) = π(g)S. This is tantamount to saying thatS(gx) =g(Sx) for allg∈Gandx∈ H.
Such operators are also calledG-module endomorphismsorintertwining operators.
In the present context the commutant is sometimes denoted also by HomG(H,H).
Lemma 4.13. The following statements are equivalent for an operatorS of H:
(1) S∈HomG(H,H).
(2) S=Se.
(3) There is an operatorT such thatS =Te. Proof . (1)⇒(2) By definition, (Sxe | y) = R
G(Sgx | gy)dg. By (1) we know Sgx = gSx, and since H is a unitary G-module, (Sgx | gy) = (gSx | gy) =
= (Sx|y). Sinceγis normalized, we find (Sxe |y) = (Sx|y) for allxandyinH.
This means (2).
(2)⇒(3) Trivial.
(3)⇒(1) Letxandy be arbitrary inHandh∈G. Then (Shx|y) = (T hxe |y) =
Z
G
(T ghx|gy)dg= Z
G
T ghx|gh(h−1y) dg
= Z
G
(T gx|gh−1y)dg= (T xe |h−1y) = (Sx|h−1y) = (hSx|y) in view of the invariance ofγ and the fact thatπ(g)−1 =π(g)∗. HenceSπ(h) = π(h)S for allh∈Gand thus (1) is proved. ut
We see easily that HomG(H,H) is a closed C∗-subalgebra ofL(H).
Anorthogonal projectionofHis an idempotent operatorP satisfyingP∗= P, that is, (P x | y) = (x | P y) for all x, y ∈ H. The function P 7→ P(H) is a bijection from the set of all orthogonal projections of H to the set of all closed vector subspaces V of H. Indeed every closed vector subspace V has a unique orthogonal complementV⊥ and thus determines a unique orthogonal projection ofHwith image V and kernelV⊥.
Definition 4.14. If Gis a topological group and E a G-module, then a vector subspace V ofE is called asubmodule ifGV ⊆V. Equivalently, V is also called
aninvariant subspace. ut
Lemma 4.15. For a closed vector subspace V of a Hilbert G-module Hand the orthogonal projectionP with imageV the following statements are equivalent:
(1) V is aG-submodule.
(2) P∈HomG(H,H).
(3) V⊥ is aG-submodule.
Proof. (1)⇒(2) Letx∈ H; thenx=P x+ (1−P)xand thus
(∗) gx=gP x+g(1−P)x
for all g ∈ G. But P x ∈ V and thus gP x ∈ V since V is invariant. Since the operatorπ(g) is unitary, it preserves orthogonal complements, and thusg(1−P)x∈ V⊥. Then (∗) impliesgP x=P(gx) andg(1−P)x= (1−P)(gx)
.
(2)⇒(3) The kernel of a morphism of G-modules is readily seen to be invariant. SinceV⊥ = kerP and P is a morphism ofG-modules, clearly V⊥ is invariant.
(3)⇒(1) Assume thatV⊥ is invariant. We have seen in the preceding two steps of the proof that the orthogonal complement W⊥ of any invariant closed vector subspace W of H is invariant. Now we apply this to W = V⊥. Hence (V⊥)⊥ is invariant. But (V⊥)⊥=V, and thusV is invariant. ut
Lemma 4.16. If T is a hermitian (respectively, positive) operator on a Hilbert G-module H, then so is T.e
Proof. Forx, y∈ Hwe have (T xe |y) =
Z
G
(gT g−1x|y)dg= Z
G
(T g−1x|g−1y)dg.
If T = T∗, then (T g−1x | g−1y) = (T∗g−1x | g−1y) = (g−1x | T g−1y) = (T g−1y|g−1x) and thus (T xe | y) = (T ye |x). Hence Te is hermitian. If T is positive, thenTeis hermitian by what we just saw, and takingy=xand observing (T g−1x|g−1x)≥0 we find thatTe is positive, too. ut Next we turn to the important class of compact operators. Recall that an operator T:V → V on a Banach space is called compact if for every bounded subset B of V the imageT B is precompact. Equivalently, this says that T B is compact, sinceV is complete.
Lemma 4.17. If T is a compact operator on a Hilbert G-module H, then Te is also compact.
Proof . Let B denote the closed unit ball of H. We have to show that T Be is precompact. Since allπ(g) are unitary, we have gB =B for eachg ∈G. Hence Adef= T GBis compact sinceT is compact. Since the function (g, x)7→gx:G×H → His continuous by Lemma 4.7, the setGAis compact. The closed convex hullKof GAis compact (see Exercise E2.5 below). Now lety∈ Hbe such that Re(x|y)≤1 for allx∈K. Thenx∈Bimplies Re(T xe |y) =R
GRe(gT g−1x|y)dg≤R
Gdg= 1 sincegT g−1x∈GT GB⊆GA⊆Kfor allg∈G. HenceT xe is contained in every closed real half-space which containsK. From the Theorem of Hahn and Banach we know that a closed convex set is the intersection of all closed real half-spaces which contain it. Hence we concludeT xe ∈K and thusT Be ⊆K. This shows that
Teis compact. ut
It is instructive at this point to be aware of the information used in the pre-ceding proof: the joint continuity of the action proved in 4.7, the precompactness of the convex hull of a precompact set in a Banach space (subsequent Exercise!), the Hahn–Banach Theorem, and of course the compactness ofG.
Exercise E4.4. Show that in a Banach spaceV, the closed convex hull K of a precompact setP is compact.
[Hint. SinceV is complete, it suffices to show thatK is precompact. Thus letU be any open ball around 0. SinceP is precompact, there is a finite set F ⊆ P such that P ⊆ F +U. The convex hull S of F is compact (as the image of a finite simplex under an affine map). Hence there is a finite setF0 ⊆S such that
S⊆F0+U. Now the convex hull ofP is contained in the convex setS+U,hence in the setF0+U+U,and its closure is contained inF0+U+U+U =F0+ 3U.]ut
We can summarize our findings immediately in the following lemma.
Lemma 4.18. On a nonzero HilbertG-moduleHletxdenote any nonzero vector and T the orthogonal projection of H onto K·x. Then Te is a nonzero compact positive operator inHomG(H,H).
Proof. This follows from the preceding lemmas in view of the fact that an orthogo-nal projection onto a one-dimensioorthogo-nal subspaceK·xis a positive compact operator and that (T x|x) =kxk2>0, whence (T xe |x) =R
G(T g−1x|g−1x)dg >0. ut Now we recall some elementary facts on compact positive operators. No-tably, every compact positive nonzero operatorT has a positive eigenvalueλand the eigenspaceHλ is finite dimensional.
Exercise E4.5. Let H be a Hilbert space and T a nonzero compact positive operator. Show that there is a largest positive eigenvalueλand thatHλ is finite dimensional.
[Hint. Without loss of generality assumekTk= 1. Note kTk= sup{kT xk | kxk ≤ 1}= sup{Re(T x|y)| kxk,kyk ≤1}. SinceT is positive, 0≤(T(x+y)|x+y) = (T x|x)−2 Re(T x|y) + (T y|y), whence Re(T x|y)≤ 12 (T x|x) + (T y|y)
≤ max{(T x | x),(T y | y)}. It follows that kTk = sup{(T x | x) | kxk = 1}. Now there is a sequencexn∈ Hwith 1−n1 <(T xn|xn)≤1 andkxnk= 1. SinceT is compact there is a subsequenceyk =xn(k) such thatz = limk∈NT yk exists with kzk= 1. Now 0≤ kT yn−ynk2=kT ynk2−2·(T yn|yn) +kynk2→1−2 + 1 = 0.
Hencez= limyn andT z=z.] ut
We now have all the tools for the core theorem on the unitary representa-tions of compact groups.
Theorem 4.19. (The Fundamental Theorem on Unitary Modules)Every nonzero Hilbert G-module for a compact group G contains a nonzero finite dimensional submodule.
Proof. By Lemma 4.18 we find a nonzero compact positive operator Te which is invariant by 4.13. ButTe has a finite dimensional nonzero eigenspaceHλ for an eigenvalueλ >0 by Exercise E4.5 IfT xe =λ·x, thenT gxe =gT xe =g(λ·x) =λ·gx.
ThusHλ is the desired submodule. ut
Definition 4.20. A G-module E is called simple if it is nonzero and {0} and E are the only invariant submodules. The corresponding representation ofG is
calledirreducible. ut
Corollary 4.21. Every nonzero HilbertG-module for a compact groupGcontains a simple nonzeroG-module.
Proof. By the Fundamental Theorem on Unitary Modules 4.20, we may assume that the given moduleHis finite dimensional. Every descending chain of nonzero submodules then is finite and thus has a smallest element. It follows thatH has a nonzero minimal submodule which is necessarily simple. ut Corollary 4.22. Every nonzero Hilbert G-module for a compact group G is a Hilbert space orthogonal sum of finite dimensional simple submodules.
Proof . Let E be a HilbertG-module and consider, by virtue of Corollary 4.21 and Zorn’s Lemma, a maximal family F = {Ej | j ∈ J} of finite dimensional submodules such that j 6=k in J implies Ej ⊥Ek. Let H be the closed span of this family (that is, its orthogonal sum). ThenH is aG-module. IfH 6=E, then H⊥ is a nonzero G-module by Lemma 4.15 nonzero simple submodule K. Then F ∪ {K} is an orthogonal family of finite dimensional simple submodules which properly enlarges the maximal familyF, and this is impossible. ThusE=H, and
this proves the corollary. ut
Definition 4.23. We say that a family {Ej |j ∈J} ofG-modules, respectively, the family {πj | j ∈ J} of representations separates the points of G if for each g ∈ G with g 6= 1there is a j ∈ J such that πj(g) 6= idEj, that is, there is an
x∈Ej such thatgx6=x. ut
Corollary 4.24. If G is a compact group, then the finite dimensional simple modules separate the points.
Proof. By Example 4.9, there is a faithful HilbertG-moduleE. By Corollary 4.22, the module E is an orthogonal direct sum L
j∈JEj of simple finite dimensional submodules Ej. If g ∈ Gand g 6=1, then there is an x∈E such that gx6= x.
Writingxas an orthogonal sumP
j∈Jxj withxj ∈Ej we find at least one index j∈J such thatgxj6=xj and this is what we had to show. ut Corollary 4.25. The orthogonal and the unitary representationsπ:G→ O(n), respectively,π:G→U(n)separate the points of any compact groupG.
Proof. By Weyl’s Trick 4.5, , for a compact groupG, every finite dimensional real representation is orthogonal and every complex finite dimensional representation is unitary for a suitable scalar product. The assertion therefore is a consequence
of Corollary 4.24. ut
Corollary 4.26. Every compact group Gis isomorphic to a closed subgroup of a productQ
j∈JO(nj)and of a productQ
j∈JU(nj)of unitary groups. ut
Index
morphism of topological groups, 6 Normalized measure, 46f
Ofp-adic integers group, 14 on left invariant metrizability Theo-rem, 30
orbit, 2
orthogonal projection, 52, 54 orthogonal representation, 55 Polish, 40
positive measure, 46f probability measure, 48 profinite groups, 19 pseudometric, 32 RegularG-module, 50 regular representation, 50 representation, 49 Riemann integral, 46
Riesz Representation Theorem, 46, 51
right invariance, 27 Scalar product, 47 semigroup, 29
separates the points, 55
sesquilinear form, 50 sigma-compact, 40 simpleG-module, 54f simplex, 53
small compact open normal subgroups, 19
small compact open subgroups, 19 star-shaped, 48
strong operator topology, 50 submodule, 52
Topological group, 5 topological group action, 5 topological manifold, 4 torsion-free, 14
torsion-free group, 14 transfinite induction, 13 translation operator, 50 Unitary, 49
unitary group, 50 unitary operator, 47f, 50 unitary representation, 49f, 54f Weyl’s Trick, 47, 55
Weyl’s Trick Theorem, 47, 55 Zorn’s Lemma, 55