Ifwe dierentiate this expression with respet to
w l
,we obtainH g (w) = S g (w) +
c kl S kl w l w t k S kl
.
We onlude that if the bloks
k
andl
are linked, ondition (3.21) is equivalentto
(S kl + S kl w l w k t S kl ) S ll = S kk (S kl + S kl w l w k t S kl ) ,
k and l of mode A,
(S kl + S kl w l w k t S kl ) S ll = (S kl + S kl w l w k t S kl ) ,
k of mode A, l of mode B.
These equationsare in generalnot fullled.
As a onsequene, we advoate to be autious to use mode A. Suppose that
the algorithm is applied to dierent start vetors and that the resulting weight
vetors are dierent. There is no way to deide whih one of them is better, as
we do not know of any optimality riterion attahed to mode A. Note that the
above desribed senario is not hypothetial. We show in Setion 3.6 that the
algorithmsinmode B donot neessarily onverge to the solution of 3.1.
Notefurthermore thatwean easilymodifythe PLSalgorithmsinmode A suh
that their solutions are stationary points of sensible optimization problems. Let
us assume that all bloks are of mode A. We replae the normalization step of
the weight vetors by
w k (i+1) = 1
n k w e (i+1) k k w e k (i+1) .
Itisstraightforward toshowthat anysolutionofthe PLSpath algorithmsfullls
the Lagrangian equationsassoiated to(3.10). Inother words, by modifyingthe
normalizationstep in mode A, we obtain a stationary point of the optimization
problemattahed tomaximizingovarianesinstead of orrelations.
guaranteedthattheobtainedvetor
w
isthesolutionoftheoptimizationproblem 3.1. Chu & Watterson (1993) present a ounter examplefor the Horst sheme ifwe apply the multivariate power algorithm. It is shown that the solution of the
multivariate power algorithmdepends onthe starting value and that it is likely
that thealgorithmonverges toaloalsolution. Hana(2006) presentaounter
example for the entroid sheme. In this setion, we present a ounter example
for both the fatorial and the entroid sheme and for the both PLSalgorithms
Lohmöllerand Wold. Forthe obvious reason,we only onsider mode B.
Wepresent two examples. Letus start with the remark that the seondounter
example is not hosen beause it reets any real world situation, but in order
to make the results reproduible. In fat, ounter examples an be found easily,
and the onvergene to loal optima seems to be the generi ase (at least for
the entroid sheme) if the manifest variables are not highly orrelated. This
is shown in the rst example. In both examples, we use
K = 3
bloks ofvari-ables. Eah blok onsists of
p k = 4
variables. This implies that the matrixX = (X 1 , X 2 , X 3 )
onsists of4 × 3 = 12
olumns.Weassume that allbloks areonneted.
Weonsider bothPLSalgorithmsWoldand Lohmöllerinthefatorialsheme
and the entroid sheme respetively. This yields four dierent variants. We
run thesefourdierentalgorithms
500
times. Ineahiteration,the standardized starting vetorsw k (0)
are drawn randomly.Fairly realisti, but not exatly reproduible example
The number of examplesis
n = 50
. Eah row ofX
isa sampleof a multivariate normaldistributionwith zero meanand the ovarianematrix equal totheiden-tity matrix. In order to save omputationaltime, we rst transform the data as
desribed in(3.11),(3.12) and (3.13). In this example, the two algorithmsin the
fatorial sheme always onverge tothe set of multivariateeigenvalues
λ 1 = 0.19735 λ 2 = 0.0741 λ 3 = 0.27133 P
λ i = 0.54278 .
As this is the only solution out of the
500
experiments, this indiates that it is the global optimum.Sheme Algorithm Global Optimum LoalSolution
fatorial Wold 100 % 0 %
Lohmöller 100 % 0%
entroid Wold 46.8 % 53.2%
Lohmöller 54 % 46%
Table 3.1: Results forthe rst example
Forthe entroid sheme,werst remark thatgiventhe samestart vetor the
LohmöllerandtheWoldalgorithmanproduedierentresults. Inthis example,
dierentresults areobserved in
16.4
%of theases. Fortheentroidsheme,thealgorithmsonverged to one of the two sets of multivariateeigenvalues
Solution1
λ 1 = 0.59238 λ 2 = 0.50684 λ 3 = 0.60448 P
λ i = 1.70370 .
Solution2
λ 1 = 0.53533 λ 2 = 0.42023 λ 3 = 0.66051 P
λ i = 1.61607 .
The seond one is only a loalsolution. As we only observe these two solutions,
we onjeture that the rst one isthe globaloptimum.
For eah algorithm and eah sheme, we ount the number of experiments in
whih the algorithmsonverged to the respetive solutions. Table 3.1 illustrates
thatbothalgorithmshaveasubstantialhanetoonvergetotheloaloptimum.
Unrealisti, but reproduible example
The number of examples is
n = 12
. We dene the12 × 12
matrixX = (X 1 , X 2 , X 3 )
in the followingway:X i,j =
1 , i = j, j = i + 1 0 ,
otherwise.
We enter the olumns of the matrix
X
. In this example, we observe aonver-gene toloalsolutions inboth shemes.
Exatlyasintherstexample,theLohmöllerandtheWoldalgorithmssometimes
produedierentresults. Thishappensin
17%
oftheasesforthefatorialshemeand in
23%
of the ases forthe entroidsheme. Seondly, the resultdepends onSheme Algorithm Global Optimum LoalOptimum
fatorial Wold 87.4 % 12.6 %
Lohmöller 80.4 % 19.6 %
entroid Wold 42.8 % 57.2 %
Lohmöller 57.4 % 42.6 %
Table 3.2: Resultsfor the seondexample
the starting vetor. For the fatorial sheme, the algorithmonverges to one of
the twosets of multivariateeigenvalues
Solution1
λ 1 = 0.74718 λ 2 = 0.99703 λ 3 = 0.39944 P
λ i = 2.14365 ,
Solution2
λ 1 = 0.47979 λ 2 = 0.48329 λ 3 = 0.53733 P
λ i = 1.50041 .
The latter solution is onlya loalsolutionand we onjeture that the rst
solu-tion is the global optimum. The result is similar for the entroid sheme. The
algorithmsonverged to one of the twosets of multivariateeigenvalues.
Solution1
λ 1 = 1.13148 λ 2 = 1.31360 λ 3 = 0.97503 P
λ i = 3.42011 ,
Solution2
λ 1 = 1.00000 λ 2 = 1.00000 λ 3 = 1.00000 P
λ i = 3.00000 .
Again, foreahalgorithmand eah sheme,we ountthe numberofexperiments
inwhihthe algorithmsonverge tothe respetivesolutions. Table 3.2illustrates
thatboth algorithmshaveasubstantialhanetoonverge totheloaloptimum.