separately). So we obtain a polynomial-time randomized algorithm pro-ducing a solution to MAXCUT whose expected value is at least about 88% of the optimal value. 4
Remark. This algorithm is due to Goemans and Williamson [GW95].
Later, H˚astad [H˚as97] proved that no polynomial-time algorithm can produce better approximation in the worst case than about 94% un-less P=NP (also see [KV05] for an interesting conjecture whose validity would imply that approximation with ratio better than in the Goemans–
Williamson results is NP-complete).
15.6 A Tight Lower Bound by Fourier Transform
⇓NEW⇓ Here we present another class of n-point metric spaces requiring Ω(logn)
distortion for embedding into any Euclidean space, and more significantly, in the proof we will illustrate yet another powerful method of establishing lower bounds for distortion, based on harmonic analysis.
The basic scheme of the proof has much in common with the lower bounds for the Hamming cube and for expanders presented earlier. We construct an n-point metric space ( ¯V ,ρ) (for reasons mentioned later, all things in this¯ space will be denoted by letters with bars) and we define two sets of pairs E¯ ⊆!V¯
2
"and ¯F =!V¯ 2
". We bound from below the ratio
RE,¯F¯(¯ρ) =ave2(¯ρ,F¯) ave2(¯ρ,E)¯
of quadratic averages, thereby showing that a typical distance in ¯Eis consid-erably smaller than a typical distance in ¯F, i.e., ¯Econsists of “short edges.”
Finally, for any metric ¯σinduced by an Euclidean embedding ¯f: ¯V →#2, we want to bound RE,¯F¯(¯σ) from above (the “short edges” cannot be so short in the embedding), and this is where the new technique comes in: We will prove an inequality relating ¯σ2( ¯E) and ¯σ2( ¯F) using harmonic analysis. In the present proof we will use only very basic facts about Fourier coefficients, but the idea of employing harmonic analysis puts at one’s disposal many tools from this well-developed field, which have already been useful in several problems concerning low-distortion embeddings. Let us note that, as we know from Section 15.5, estimating the maximum of RE,¯F¯(¯σ) is equivalent to to estimating the second eigenvalue of a certain (Laplace) matrix, but here we will not use the language of eigenvalues.
Preliminaries on Fourier transform on the Hamming cube. LetV denote the vertex set of them-dimensional Hamming cubeCm. We consider the elements ofV asm-component vectors of 0’s and 1’s. Foru, v∈V,u+v denotes the vector inV whoseith component is (ui+vi) mod 2. We note that u+vis the same asu−v.
For a function f:V → R we define another function ˆf:V → R, the Fourier transformoff, by
fˆ(u) = 2−m$
v∈V
(−1)u·vf(v),
where the inner productu·vis defined as (u1v1+u2v2+· · ·+umvm) mod 2.
Readers acquainted with Fourier series or with Fourier transforms in other settings will find the following fact familiar:
15.6.1 Fact (Parseval’s equality). For every functionf:V →Rwe have
$
v∈V
f(v)2=$
u∈V
fˆ(u)2.
In our setting, where we always deal with finite sums, Parseval’s equality is quite simple to prove; see Exercise 1. Geometrically, the equality holds because (f(v):v ∈ V) are the coordinates of a vector in an orthonormal basis, and ( ˆf(u):u∈ V) are the coordinates of the same vector in another orthonormal basis. In textbooks on harmonic analysis one can find Parseval’s equality in a general setting, encompassing our situation but also the classical case of Fourier series of periodic real functions onR and many other useful cases. Then the proofs are more demanding, since one has to deal with issues of convergence of infinite sums or integrals.
Good codes. With the componentwise addition modulo 2 introduced above, V is a vector space over the two-element field GF(2) (scalar multiplication is trivial; the scalars are only 0 and 1, and we have 0v = 0 and 1v =v). The construction of the badly embeddable space is based on a suitable subspace C⊆V as in the next lemma.
15.6.2 Lemma. For every sufficiently largem divisible by 4 there exists a subsetC⊆V with the following properties:
(i) C is a vector subspace ofV of dimension 14m.
(ii) Every two distinct u, v ∈ C differ in at least δm components, where δ > 0 is a suitable small constant (δ = 0.01 will do, for example). In other words,uandv have Hamming distance at leastδm.
We leave a proof to Exercise 2. The lemma comes from the theory of error-correcting codes. Mathematically speaking, the main goal of this theory is to construct, for given integer parametersmand d, a subset C⊆V ={0,1}m (called acodein this context) that is as large as possible but, at the same time, hasminimum distance at leastd, meaning that every two distinct u, v∈ C have Hamming distance at leastd. (More precisely, this concerns codes over the two-element alphabet{0,1}, while coding theory also investigates codes over larger alphabets.) In many of the known constructions, C is a vector subspace ofV, and then it is called alinear code. A codeC⊆ {0,1}mwhose
15.6 A Tight Lower Bound by Fourier Transform 397 minimum distance is at least a fixed fraction ofm and such that log|C| is also at least a fixed fraction ofmis often called agood code. So Lemma 15.6.2 simply claims the existence of a good linear code (the constant 14 in part (i) is rather arbitrary and, for our purposes, it could be replaced by any smaller positive constant).
GivenC as in Lemma 15.6.2, let us put
W =C⊥ ={u∈V:u·v= 0 for allv∈C}.
This definition resembles the usual orthogonal complement in real vector spaces,except thatC∩C⊥ may contain nonzero vectors (considerm= 2 and C=C⊥ ={(0,0),(1,1)})! However, the familiar properties (C⊥)⊥ =C and dimC+ dimC⊥ =m hold; see Exercise 3. From now on, we will talk only aboutW (and forget aboutC), and we will use the following properties:
15.6.3 Lemma.
(W1) W is a vector subspace ofV of dimension 34m.
(W2) For every nonzerou∈V with less thanδmones there existsv∈W with u·v= 1.
Proof. Part (W1) is clear from dimC= m4 and dimC+dimC⊥=m. As for (W2), ifu∈V satisfiesu·v= 0 for allv∈W, thenu∈W⊥= (C⊥)⊥ =C.
Every nonzero vector inC has at leastδm ones, and (W2) follows. ! The badly embeddable space. Letmbe divisible by 4 and letW ⊂V satisfy (W1) and (W2). We are going to construct the metric space ( ¯V ,ρ).¯ First we define an equivalence relation≈onV by u≈v ifu−v∈W. The equivalence class containing u thus has thus the form u+W, and we will denote it by ¯u.
The points of ¯V are the equivalence classes: ¯V ={u:¯ u∈V}. The number of equivalence classes is ¯n=|V¯|=|V|/|W|= 2m/4.
The metric ¯ρ on ¯V is defined as the shortest-path metric of a suitable graph ¯G = ( ¯V ,E), where¯ {u,¯ v¯} ∈ E¯ if there is at least one edge of the Hamming cube Cm connecting a vertex of the equivalence classes ¯u to a vertex of the equivalence class ¯v. More formally, ¯E ={{u,¯ v¯}:{u, v} ∈ E}, whereE is the edge set ofCm.
It may be useful to think of ¯ρas follows. Let us suppose that it takes unit time to travel between the endpoints of an edge of the Hamming cube. Then the Hamming distance of two verticesu, v∈V is the minimum time required to travel fromuto v along edges. Now we imagine that every two vertices u, u(in the same equivalence class are connected by a “hyperspace link” that can be traveled instantaneously, in time 0. Then ¯ρ(¯u,v) is the shortest time¯ needed to travel from a vertex of ¯uto a vertex of ¯v using ordinary edges and hyperspace links as convenient.
The following observation will be useful later, and it may help digest the definition of ¯ρ.
15.6.4 Observation. For every two equivalence classes u,¯ ¯v ∈ V¯, one can travel from any vertex ofu¯to any vertex ofv¯in timeρ(¯¯u,¯v)by first using at most one hyperspace link and then traveling only through ordinary edges.
Sketch of proof. It suffices to note that if ¯u,v¯are equivalence classes and there is at least one edge of the Hamming cube connecting them, thenevery vertex of ¯uis connected to some vertex of ¯v. Indeed, if{u, v} ∈Eis an edge, i.e.,uand vdiffer in exactly one coordinate, and ifu( ∈u¯=u+W, thenu( andv(=v+u(−u∈v+W differ in exactly the same coordinate, and thus
{u(, v(} ∈E as well. !
We can state the main result of the section:
15.6.5 Theorem. Every embedding of the metric space ( ¯V ,ρ)¯ into a Eu-clidean space has distortion at leastΩ(m) = Ω(log ¯n).
Preparatory steps. We begin with realizing the already announced scheme of the proof. We consider the two sets of pairs, ¯E as above and ¯F = !V¯
2
". First we estimateRE,¯F¯(¯ρ). Since for{u,¯ ¯v} ∈E¯ we have ¯ρ(¯u,v) = 1, we get¯ ave2(¯ρ,E) = 1.¯
It is easy to see that a “typical” pair of vertices inV has Hamming distance at least a constant fraction ofm. The next lemma shows that although adding the hyperspace links in the construction of ¯V shortens the distances, a typical distance under ¯ρis still a constant fraction ofm.
15.6.6 Lemma. For at least half of pairs{u,¯ ¯v} ∈F¯ we haveρ(¯¯u,¯v)≥αm, whereαis a suitable positive constant.
Proof. Let us fix an arbitrary ¯u ∈ V¯; it suffices to show that only o(¯n) classes ¯v ∈ V¯ satisfy ¯ρ(¯u,v)¯ ≤ k =0αm1. Let U ⊆ V denote the union of all these classes ¯v. By Observation 15.6.4, every vertex ofU can be reached from some vertex of the class ¯u by traveling a path of length at most k in the Hamming cube. In the Hamming cube, the number of vertices at Hamming distance at most k from a fixed vertex is exactly 'k
i=0
!m i
", and thus |U| ≤ |W| ·'k
i=0
!m i
". Therefore, the number of ¯v with ¯ρ(¯u,v)¯ ≤k is no more than|U|/|W| ≤'k
i=0
!m i
". A standard estimate for the last sum is (em/k)k, and a simple calculation shows that this iso(¯n) for αsufficiently
small. !
By the lemma we have ave2(¯ρ,F¯) = Ω(m), and thusRE,¯F¯(¯ρ) = Ω(m).
Next, we turn to boundingRE,¯F¯(¯σ) from above, for any (pseudo)metric
¯
σ induced by an Euclidean embedding ¯f: ¯V → #2. We need to prove that ave2( ¯F ,σ) =¯ O(ave2( ¯E,¯σ)), and as we saw in the previous sections, it is sufficient to consider only one-dimensional embeddings ¯f: ¯V →R. Thus we want to prove
15.6 A Tight Lower Bound by Fourier Transform 399 To do so, we first “pull back” from ¯V to the original vertex set V of the cube. Given ¯f: ¯V →R, we definef:V →Rbyf(v) = ¯f(¯v) (sof is constant on every equivalence class). Then, withF =!V
2
(we note that ifuand v are in the same equivalence class, they contribute 0 to the sum). Observing that for every{u,¯ v¯} ∈E¯ the equivalence classes ¯u since increasing all values of f by the same number doesn’t change either side of (15.8), we may assume that'
v∈Vf(v) = 0, which will be useful (we already met this trick in Section 15.5).
A Fourier-analytic piece. We will express both sides in (15.8) using the Fourier coefficients off, beginning with the right-hand side. The next two lemmas are standard tools in harmonic analysis.
15.6.7 Lemma. For every functionf:V →Rwe have
where'u'1denotes the number of1’s in the zero-one vectoru.
We postpone the proof and continue with the proof of (15.8). The next lemma shows how the properties ofW are reflected in the Fourier coefficients off.
15.6.8 Lemma. Letf:V →Rbe such thatf(u) =f(v)wheneveru−v∈ W, where W satisfies (W1), (W2). Then fˆ(u) = 0 for every u ∈ V with 0<'u'1< δm. If we also assume'
v∈V f(v) = 0, thenfˆ(0) = 0 as well.
We again postpone the proof.
By combining Lemma 15.6.8 and Lemma 15.6.7, we have
$
We now turn to the left-hand side of (15.8). By calculation we already did in Section 15.5 (see (15.5)), we obtain
and by Parseval’s equality the right-hand side is|V|·'
u∈V fˆ(u)2. Altogether
which provides the desired inequality (15.8). It remains to prove the lemmas, which is a relatively straightforward manipulation of identities.
Proof of Lemma 15.6.8. First we note that ˆf(0) = 2−m'
Proof of Lemma 15.6.7. Letei∈V denote theith vector of the standard basis, with 1 at positioniand 0’s elsewhere. We setgi(v) =f(v+ei)−f(v) (this is something like a “derivative” offaccording to theith variable). Every edge inE has the form{v, v+ei}for somev and somei, and so
15.6 A Tight Lower Bound by Fourier Transform 401 the last equality being Parseval’s. We compute
ˆ
Lemma 15.6.7, as well as Theorem 15.6.5, are proved. ! Bibliography and remarks. Among many textbooks of harmonic analysis we mention K¨orner [K¨or89] or Dym and McKean [DM85], and a standard reference for error-correcting codes is Van Lint [vL99].
The construction of the metric space ( ¯V ,ρ) from the Hamming¯ cube using the equivalence≈is an instance of a generally useful con-struction of aquotient spacefrom a given metric space using an equiv-alence with finitely many classes; see, e.g., Gromov [Gro98] (we again add zero-length links between every two equivalent points and con-sider the resulting shortest-path metric). However, we note that for an arbitrary equivalence an analogue of Observation 15.6.4 no longer holds.
The material of this section is from Khot and Naor [KN05]. This paper also cites several earlier applications of harmonic analysis for dis-tortion bounds. An immediate predecessor of it is a fundamental work of Khot and Vishnoi [KV05], which we will discuss in Section 15.9.
Khot and Naor solved several open problems in low-distortion em-beddings. Perhaps most notably, they showed that the metric space {0,1}n with the edit distance needs Ω((logn)1/2−o(1)) distortion for embedding into#1, where the edit distance of twon-bit stringsv, wis the smallest number of edit operations (insertions or deletions of bits) required for converting v into w. Krauthgamer and Rabani [KR06]
improved the lower bound to Ω(logn), with a beautifully simple ar-gument. Ostrovsky and Rabani [OR05] proved an upper bound of 2O(√
lognlog logn). The need for such embeddings has strong algorith-mic motivation, since a low-distortion embedding into#1 would allow
for very fast approximate database queries (“Does the database con-tain a string very similar to a given query string?”; this is a basic problem in web searching, computational biology, etc.), or even a fast approximation of the edit distance of two given strings, whose exact computation is not easy.
In a way similar to the proof for expanders from the previous sec-tion, the proof in this section can easily be modified to show that there is no squared Euclidean metric that approximates the constructed space with distortiono(logn), and in particular, that any embedding into#1requires distortion Ω(logn).
Exercises
1. (a) Letu1(=u2∈V. Prove that'
v∈V(−1)u1·v(−1)u2·v= 0. 2
(b) Prove Fact 15.6.1 by direct calculation, substituting for ˆf(u) on the right-hand side from the definition, expanding, and using (a). 2
(c) Go through the following “more scientific” presentation of the proof.
Let Fm denote the real vector space of all functions f:V → R, where addition and scalar multiplication are defined in the natural way, by (f +g)(v) = f(v) +g(v) and (αf)(v) = α·f(v). Check that ,f, g- = '
v∈Vf(v)g(v) defines a real scalar product onFm, and that (ev:v∈V) form an orthonormal basis, where ev(v) = 1 and ev(u) = 0 foru (=v.
Verify that (ϕu:u ∈ V) is another orthonormal basis, where ϕu(v) = 2−m/2(−1)u·v, and that for every f ∈ Fm, f = '
u∈V fˆ(u)·ϕu is an expression off in this basis. 3
2. (a) LetC ⊆V be a vector subspace. Check that if every nonzerov∈C has at least d ones, then the Hamming distance of every two distinct vectors in Cis at least d. 1
(b) Prove Lemma 15.6.2 by induction as follows. Letmbe given and let d = 0δm1, where the constant δ > 0 is chosen as small as convenient for the proof. Let us say that a k-tuple (v1, . . . , vk) of vectors of V is good if v1, . . . , vk are linearly independent and every nonzero vector in their linear span has at leastdones. Given any goodk-tuple (v1, . . . , vk), k < m4, estimate the number ofv ∈V such that (v1, . . . , vk, v) is not a good (k+1)-tuple, and conclude that every goodk-tuple can be extended to a good (k+1)-tuple. How does this imply the lemma? 4
3. (a) Let Abe a matrix with mcolumns and of rankr, over any field K.
Recall (or look up) a proof that the subspace of Km consisting of all solutions toAx= 0 has dimensionm−r. 1
(b) For a set U ⊆ V = {0,1}m we define U⊥ = {v ∈ V:u·v = 0 for all v ∈ U}, where u·v is the inner product as in the text. Show that ifU is a vector subspace ofV, then dimU+ dimU⊥=m. 1 (c) Show that ifU is a vector subspace ofV, then (U⊥)⊥ =U. 2
⇑NEW⇑
15.7 Upper Bounds for"∞-Embeddings 403