Evaluation of the epi-convergence
1 2Petr Lachout
Abstract
The paper presents an equivalent characterization of the epi-convergence of lower semicontinuous functions. The proposed 'measures' naturally estimate the distance between optimal values of two optimization programs. A comparison of optimal solutions is more complex. We propose an estimation for a given function which can be taken as the excess of a set over another, being in metric space. Hence, we receive an estimate of the distance between optimal solutions.
1 The concept of the epi-convergence
Looking for the global minimum of a given function is the central problem discussed in the literature and solved in optimization theory. The lack of complete information on the objective function is the main trouble met in the practice. We have to work with approximations and, therefore, we naturally ask about the stability of our optimization problem provided varying objective function. The epi-convergence looks to be the most ecient tool for that, see 1], 2] or 8]. We present an equivalent description of the epi- convergence which can be used to derive estimate of the distance between optimal values.
Treating of the optimal solutions is more dicult. We o er a procedure which is natural in the case of metric spaces, as the example illustrates.
Let us specify the subject of our treatment. We work on the Hausdor topological space X. Therefore, we will employ open, closed and compact sets of X, the notion of nets, cluster points, limit points, Kuratowski-Painleve convergence of sets, etc. For convenience, we are giving the list of the used notation in Appendix.
We consider the function f : X ! R = ;1 +1]. Our interest is focused in con- tinuity property of its optimal value '(f) = infx2Xf(x) and its set of optimal solu- tions (f) = fx 2 X : f(x) = '(f)g. Let us note that the problem of "-optimal solutions do not need special treating. It is sucient to consider truncated function
f
" = maxff '(f) +"g, as we do in the example.
Known observation is that (f) is closed set provided f is l.s.c. (lower semicon- tinuous). To avoid any misunderstanding let us recall that f : X ! R is l.s.c. if liminfy!xf(y) f(x) for each x 2 X. Let us denote the set of all l.s.c. functions onX by LSC(X).
We consider the space LSC(X) with the epi-convergence.
De nition 1
Let be a directed set, f 2LSC(X) for each 2 and f 2LSC(X). We say that the net <f >2 epi-converges to f, provided epi(f) =K-lim2epi(f).Recallepi(f) = f(x )2X R : f(x)gand the denition ofK-lim is remembered in Appendix.
1The research is supported by Deutsche Forschungsgemeinschaft Projekt-Nr. Ro 1006/1-2, the Czech Republic Grant 201/96/0230 and the Charles University Grant 3051-10/716.
2The paper has partially been written during a stay at the Humboldt University of Berlin.
1
The epi-convergence admits a helpful characterization by means of the nets.
Proposition 1
Let f 2 LSC(X) for each belonging to the directed set and f 2 LSC(X). The net <f >2 epi-converges to f if and only if at each pointx2X both of the following conditions hold:1. Let be a directed set and ! : ! be monotone (i.e. 1 2 2 , 1 2
) !(1) !(2)) and connal (i.e. for each 2 there exists 2 such that
!()). Then we have liminf2f!()(x)f(x) whenever lim2x =x. 2. There exists directed set and x() 2 X for each 2 , 2 such that
lim()2x()=x and lim()2f(x()) =f(x).
Proof:
Fix the point x2X1. Let <f >2 epi-converges to f.
a)
Let be a directed set and ! : ! be monotone and connal. Assume lim2x =x and let us denote = liminf2f!()(x).Take the set G 2 G(X R) with (x ) 2 G. According to the denition of the liminf we have for each 2 some 2 such that !() and (x f!()(x)) 2 G. Then (x ) 2 Ls2epi(f) = epi(f) because of the epi-convergence. Therefore, f(x).
b)
According to the denition of the epi-convergence, Li2epi(f) = epi(f) and, hence, (x f(x))2Li2epi(f).Therefore, for each G2G(XR) with (x f(x))2Gthere is0 2 such that
G\epi(f)6= for each 0 . Let us take (x(G) (G))2 G\epi(f) for
0
and dene x(G) =x, (G) =f(x) whenever 0 6.
The set =nG2G(X R) (x f(x))2Gois directed by inclusion, i.e.
GH () GH :
Then we have lim(G)2x(G) =x and lim(G)2(G) =f(x).
That implies lim(G)2f(x(G)) = f(x) since alwaysf(x(G))(G)and liminf(G)2f(x(G))f(x), according to the rst part of the proof.
2. Let the functions f, 2 and f fulll the conditions.
a)
Accordingly to the second property, we have the directed set and the conver- gent net lim()2x() =x with lim()2f(x()) =f(x).Take the set G2G(X R) with (x f(x))2G. There are 0 2 and 0 2 such that x() f(x())2 G\epi(f) for each 0 and each 0 . That means (x f(x))2Li2epi(f).
b)
Take (x )2Ls2epi(f). We set= n( G)2G(X R) :epi(f)\G6= (x )2Go : 2
The set is directed by the natural ordering
(1 G1)(2 G2) , 1 2 and G1 G2
since (x ) is a cluster point. By the denition of we have the point (x(G) (G)) 2 epi(f)\G for each ( G) 2 . Then lim(G)2x(G) = x and lim(G)2(G) =. According to the rst property we conclude f(x) and therefore (x )2epi(f).
We proved epi(f) =Li2epi(f) =Ls2epi(f), which is the epi-convergence.
Q.E.D.
We write the paper to introduce "distances" between l.s.c. functions giving an equiv- alent characterization of the epi-convergence. These "distances" can be dened for each couple of real functionsf g :X !R and AX, x2X by
1(g fA x) = supf(f(x);g(y))+ :y2Ag
2(g fA x) = inffjf(x);g(y)j:y2Ag :
Proposition 2
Let u:R !R be increasing continuous, be a directed set,f 2LSC(X) for each 2, f 2LSC(X) and for each point x2X we have given the base Gx.Then the net <f >2 epi-converges to f if and only if lim
G2Gx
limsup
2
1(uf ufG x) = 0 for each x2X lim
2
2(uf ufG x) = 0 for each G2Gx x 2X :
Proof:
The function u naturally gives the bijection between LSC(X) and LSC(X) \n
f :X !u(R)opreserving the epi-convergence. Therefore, it is sucient to consider the nets of l.s.c. functions with real values, only, and show for them the characterization by nets given in the proposition 1. Fix the point x2X for that.
1. We show the equivalence between the rst property of the proposition 1 and the rst property of the proposition 2.
a)
Let limG2Gxlimsup21(f fG x) = 0.Let be directed set,!: ! be monotone and connal, and lim2x =x. ForG2Gx we have 0 2 such that x 2G and
1 f
!()
fG x f(x);f!()(x)
+
for each 0 : Accordingly to the assumption, we have
lim
G2G
x
limsup
2
1 f
!()
fG x= 0 and, therefore,
lim
2
f(x);f!()(x)
+
= 0 : That is nothing else than liminf2f!()(x)f(x).
3
b)
Let for each directed set ,! : ! monotone and connal, and lim2x =x, we have liminf2f!()(x)f(x).
For each 2, G2Gx and 0<<1 there exist x(G) 2Gsuch that
1(f fG x)< f(x);f(x(G))
+
+ :
Setting = Gx(0 1) we receive a set directed by the ordering (1 G1 1)(2 G2 2)()1 2 G1 G2 1 2 and lim(G)2x(G) =x. Therefore according to the assumption,
liminf
(G)2 f
(x(G))f(x) : Then lim(G)2 f(x);f(x(G))
+
= 0 and, consequently, limG2Gxlimsup21(f fG x) = 0.
2. We show the equivalence between the second property of the proposition 1 and the second property of the proposition 2.
a)
Let lim22(f fG x) = 0 for each G2Gx.Then for each 2, G2Gx and 0<<1 there exist x(G)2Gsuch that
2(f fG x)>f(x);f(x(G));: Consequently, lim(G)2Gx(01)x(G)=x and
lim
(G)2Gx(01) f
(x(G)) =f(x) :
b)
Let be a directed set and x() 2X be such that lim()2x() =x and lim()2f(x()) =f(x).Let G2 Gx. Then there are 0 2 and0 2 such that x() 2G for each
0
and 0 . Hence,
2(f fG x)f(x);f(x()) for each 0 and 0 .
Consequently, lim22(f fG x) = 0.
Q.E.D.
If the space X is a metric space then we can receive characterizations of the epi- convergence by means of the Hausdor distance of closed sets, see 2] and 9].
In general, the epi-convergence onLSC(X) is not induced by any topology. If the space
X is rst countable Hausdor space then epi-convergence coincides with the convergence induced by Fell topology on closed subsets inX R, see 5], theorem 5.2.10, p.140. If the spaceX is a nite dimensional space then there is a metric inducing the epi-convergence, see 2] or 5], p.161.
4
2 Stability of the optimization program
The epi-convergence implies consistency of the optimization program.
Theorem 1
Let f f 2 LSC(X) for each 2 , the net < f >2 epi-converges to f and supK2K(X)infy62Kinf2f(y)>'(f).Then lim2'(f) ='(f) and there exist G2 for any G2G(X) with (f)G such that(f)G for eachG, consequentlyLs2(f)(f). Moreover, (f) is a non-empty compact and there is a 0 2 such that (f) is a non-empty compact for each 0 2.
Proof:
See 1] or 5], theorem 5.3.6, p.160.Q.E.D.
Provided compact space X, the statement can be expressed in the following way:
the function ' is continuous and the multifunction is upper semicontinuous. The assumptions of the theorem 1 can be easily fullled for convex functions, see 6].
We are interested not only in consistency, we would like to estimate the rate of con- sistency. To receive an estimate of the distance between the optimal values we need the following simple lemma.
Lemma 1
If f g 2 LSC(X) and u : R ! R is increasing continuous then we have the estimates;
1(ug ufA x^)u'(g);u'(f)2(ug ufX x^) whenever x^2(f) and A\(g)6=.
Proof:
Let ^x2(f) and ^y2A\(g). Then we have the estimatesu'(f);u'(g) = uf(^x);ug(^y)1(ug ufA x^) and for each x2X
u'(g);u'(f) =ug(^y);uf(^x)
ug(x);uf(^x)jug(x);uf(^x)j : Minimizing along x2X we receive the estimate
u'(g);u'(f)2(ug ufX ^x) :
Q.E.D.
Theorem 2
Let f f 2 LSC(X) for each 2 , supK2K(X)infy62Kinf2f(y) > '(f),G
x be a given base at each point x2X and u:R !R be increasing continuous.
Let lim
G2G
x
limsup
2
1(uf ufG x) = 0 for each x2X 5
lim
2
2(uf ufG x) = 0 for each x2X G2Gx and let Gx 2Gx, x 2 for x2(f), 0D <+1 for 2 be such that
1(uf ufGx x)D 2(uf ufX x)D for each x, x2(f).
Then there is0 2 such that ju'(f);u'(f)jD for each0 and there exist G 2 for any G 2 G(X) with (f) G such that (f) G for each G, consequently Ls2(f) (f). Moreover, (f) is a non-empty compact and there is
1
2 such that (f) is a non-empty compact for each 1 2.
Proof:
According to the proposition 2, the net of functions epi-converges and, therefore, almost all assertions of the theorem are contained in the theorem 1. We have to prove the estimation of the distance between optimal values, only. From the theorem 1 we know that the set(f) is a non-empty compact. Then we can select a nite setI (f) such that (f) Sx2IintGx since (f) Sx2( f)intGx. Then there is 0 2 such that(f)Sx2IintGx for each 0 , according to the theorem 1. The stated assertion is straightforward, now, since the set I is nite and
;max
x2I
1(uf ufGx x^)u'(f);u'(f)2(uf ufX x^) accordingly to the lemma 1. The point ^x2(f) can be chosen arbitrarily.
Q.E.D.
The shown statement is better than the statement based on the uniform topology, assuming an open setG(f) and 0 2 such that
sup
y2G jf
(y);f(y)jD for each 0.
Then for each x 2 (f) there is Gx 2 Gx such that Gx G. For each y 2 Gx we have the estimate
(f(x);f(y))+ (f(y);f(y))+jf(y);f(y)j since x2(f) and, hence,
1(f fGx x)sup
y2G jf
(y);f(y)jD : Further,
2(f fX x)jf(x);f(x)jsup
y2G jf
(y);f(y)jD :
The distance of the optimal solution can be estimated using the following theorem.
Theorem 3
Let f f 2 LSC(X) for each 2 , supK2K(X)infy62Kinf2f(y) > '(f),G
x be a given base at each point x2X and u:R !R be increasing continuous.
Let lim
G2G
x
limsup
2
1(uf ufG x) = 0 for each x2X 6
lim
2
2(uf ufG x) = 0 for each x2X G2Gx :
Let us consider a given function d : Sx2X(Gxfxg) ! 0 +1). Suppose, we have functions : 0 +1) ! 0 +1), : 0 +1) ! 0 +1), 0 2 , an open set
Q(f) and Gx 2Gx for each x2Q, 0, such that Gx\(f)6=,
uf(x);u'(f)(d(Gxx))
uf(x);u'(f);uf(x) +u'(f)(d(Gxx)) for each x2Q, 0.
Then lim2'(f) ='(f) and there exist G2 for any G2G(X) with (f)G such that (f) G for each G, consequently Ls2(f) (f). The set (f) is a non-empty compact and there is1 2 such that (f) is a non-empty compact for each 1 2. Moreover, there is 2 2 such that
(d(Gxx)) (d(Gxx)) for each x2(f), 2.
Proof:
According to the proposition 2, the net of functions epi-convergences and, con- sequently, the assumptions of the theorem 1 are fullled. Therefore, almost all assertions of the theorem are contained in the theorem 1 and we have to show the estimate for the functiond, only.There is2 2 such that (f)Q for each 2. Taking x2(f), 2, we receive the estimate
0 =uf(x);u'(f); uf(x);u'(f);uf(x) +u'(f)
(d(Gxx));(d(Gxx)) : The received estimate coincides with the statement of the theorem.
Q.E.D.
The additional assumptions of the theorem are realistic. The function represents so called "growth condition" on the limit function. Such a condition is probably almost necessary. The second assumption of the theorem 3 can be fullled, for example, using 'Lipschitz norm'
Lip(gA) = inffL20 +1] :jg(x);g(y)jL(d(Gx)) x2A G2Gx y2Gg : Let Q (f) be an open set and 0 2 be such that Q (f) for each 0. Takingx2Q, G2Gx, GQ and ^x2(f)\G, we receive the estimate
uf(x);u'(f);uf(x) +u'(f)
uf(x);uf(^x) +uf(^x);uf(x)Lip(uf;ufQ)(d(Gx)) : 7
3 Metric spaces
This chapter is written to show the meaning of the theorem 3. Let X be a metric space with the metric. We take the collection of closed balls V(x ") =fy 2X :d(x y)"g for">0 as the base at the pointxand d(V(x ")x) =". Further, we employ the excess of the setA over the set B given by excess(A B) = supa2Ainfb2B(a b).
Theorem 4
Let f f 2 LSC(X) for each 2 , supK2K(X)infy62Kinf2f(y) > '(f) and u:R !R be increasing continuous. Letlim
"!0+
limsup
2
1(uf ufV(x ") x) = 0 for each x2X lim
2
2(uf ufV(x ") x) = 0 for each x2X ">0:
Suppose, we have functions : 0 +1)!0 +1), 0 2 and an open set Q(f) such that
uf(x);u'(f)(excess(fxg (f)))
uf(x);u'(f);uf(x) +u'(f)(excess(fxg (f))) for each x2Q, 0.
Then lim2'(f) ='(f) and there exist G2 for any G2G(X) with (f)G such that (f) G for each G, consequently Ls2(f) (f). The set (f) is a non-empty compact and there is1 2 such that (f) is a non-empty compact for each 1 2. Moreover, there is 2 2 such that
(excess((f) (f))) (excess((f) (f))) for each 2.
Proof:
The assertion is a easy consequence of the theorem 3. It is sucient to setG
x =V(x excess(fxg (f))) andd(V(x ")x) = "and consider that always there is a pointx2(f) such that excess(fxg (f)) = excess((f) (f)).
Q.E.D.
4 Example
Let us give a simple example illustrating the subject. Consider the functions fn(x) = (x;n)2+n,n 2N and f(x) =x2, where n !0 and n !0.
Evidently, f fn are l.s.c. and '(fn) =n, (fn) =fng, '(f) = 0, (f) =f0g. Our 'measures' are
1(fn fx;" x+"] x) =
8
<
: x
2
;(jx;nj;")2;n
+
if jx;nj>"
x 2
;
n
+
if jx;nj"
and
2(fn fx;" x+"] x)jf(x);fn(x)j=jx2 ;(x;n)2;nj : 8
