T e st s fo r th e T w o S a m p le P ro b le m

(1)

28

3 R a n d o m iz a tio n T e st s

3.1

In tr o d u c to ry E x a m p le

aHailprevention:(“GrossversuchIV”incentralSwitz.1978-83)Doesspreadingofsiveriodideintopotentialhailcloudsdiminishthetotalhailenergy?(Simpleideas,onlysimplecombinatoricsandprob.needed.)

Targetvariable:Hailenergy,measuredfornclouds.Twogroups:treatedversuscontrol.

Yi:Hailenergyforcloudi Gi= 1ifcloudiistreated,0otherwise.WeexpectYitobeusuallysmallerforGi=1thanforGi=0.

(2)

293.1

bObserved:

Yi=y ∗i 166722585501520461219Gi=g ∗i 11000110

g ∗i :randomchoiceofthecloudsthataretreated.(Inrealitytherewere216cloudsofwhich94weretreated.) Statisticaltest!H0:noeffect.(−→Proofbycontradiction!)ttestforindependentsamples?Wedonotliketoassumeanyparticulardistr.fortheYis!!

(3)

30

3.2

S ta tis tic a l id e a

aNullhypothesis=ProbabilitymodelusuallyisadistributionoftheYi.Gi=g ∗i assumedtobegiven.Randomizationtests:Girandom,Yi=y ∗i consideredasfixed!(Analysis“conditional,fiventhey ∗i s”.) Ifthetreatmenthasnoinfluenceonhailenergy,thesameobservationsy ∗i wouldresult,ifthetreatmenthadbeengivenbyg (1)=[0,1,0,0,1,1,0,1]oraccordingtoanyotherchoice.

(4)

31

Randomchoice:Eachchoiceofn/2=4elementsfromn=8hasthesameprobability

p= 84 −1= 170Thisdeterminesthenullhypothesis.

(5)

323.2 bTeststatistic:designedtoassumeextremevalueswhenthealternativeistrue.Alternative:y ∗i withg ∗i =1aregenerallysmaller.

Thg,y ∗i= 1n/2 X

i:gi=0 y ∗i − 1n/2 Xⁱ^:^g

i=1 y ∗i = 2n X_i y ∗i (1−2gi).

cWhatisthedistributionofTunderH0?

y ∗1 ,...,y ∗n given−→≤ nn/2 possiblevaluesforT.

PhThG,y ∗i=ti= #{g|Thg,y ∗i=t}

nn/2

” randomizationdistribution”

(6)

33

t

Wahrscheinlichkeit

−5000−3000−1000100030005000

0 / 70 5 / 70 10 / 70 15 / 70 20 / 70

Randomisierungs−Verteilung

t 3600380040004200440046004800

0 / 70 2 / 70 4 / 70 6 / 70 8 / 70

Randomisierungs−Vert., rechter Teil

(7)

343.2

dRejectionregion:α=5%mostextremevalues(aspreciselyaspossible).Example:{t|t≥4643.25}(onesided).

eExperiment:

Thg ∗,y ∗i= 14 (855+0+152+1219)

− 14 (16672+25+0+46)=−3629.25Aneffectinthewrongdirectionwasobserved!Nullhypothesisisnotrejected;noeffectisdemonstrated.

(8)

353.2

f

*

Assumptionofthetest:Independence

−→Randomizationamong76“potentialhaildays”

Amongthese,33havebeenassignedtothetreatment.Numberoftreateddaysisrandom.

−→Analysisconditionalonthenumberofhaildayswithtreatm.

g 7633 =36·10 20possiblechoices

−→Simulationoftherandomizationdistribution.

(9)

36

3.3

T e st s fo r th e T w o S a m p le P ro b le m

aRandomizationtestsareadequateeveniftheexperimentalproceduredoesnotcontainanyrandomization.

Assumptionsinthiscase:•TheobservationsmustbeequallydistributedunderH0and•independent

Then,thepresupposedprobabilityαoferrorofthefirstkindholdsprecisely.Therandomizationtestsareinthissensethethe“goldstandard”ofstatisticaltests.

(

*

Weakerassumption:“Exchangeability”.)

(10)

373.3 bIftheobservationsarerandom:Sample[Y1,...,Yn]−→orderedsampleY[1] ,...,Y[n]orempiricaldistributionfunction bFn(s.Bootstrap)

Distributionoftheteststatistic,conditionalon bFn,istherandomizationdistribution.

Cond.prob.ofanerrorofthefirstkind,given bFn,isα

(11)

383.3

cArbitraryteststatistic.Thedifferenceofmeansisnotrobust...

Optimalteststatistic?−→optimizepowerforalternative(s)!Needsfixed(familyof)distribution(s)

−→optimalteststatistic(e.g.,likelihoodratiotest)

dExample:Logtransformation,thendifferenceofmeans(preferablyrobustized)

(12)

39

t l

Wahrscheinlichkeit

−2.5−1.5−0.50.51.01.52.02.5

0 / 70 5 / 70 10 / 70

Rand.Vert. für log. Werte

010203040506070

0 10 20 30 40 50 60 70

Rang(tg)

Rang(^tg l)

g

Vergleich der Test−Statistiken

(13)

403.3

eRobustness.Whyshouldweusearobustteststatistic,ifthetestkeepsthelevelwithoutthis“preventivemeasure”?

fRanksumtestofWilcoxon,MannandWhitney(U-Test),

Thg,yi= X

gi=1 Ri= X

i giRi,

Quiterobust−→Firstchoiceforthe2sampleproblemDistributionoftheteststatisticunderH0asbefore.

g

*

Hailexperiment:Complicatedteststatistic,two-dimensional

−→twodimenstionalrejectionregion.

(14)

41

3.4

O n e S a m p le a n d M a tc h e d P a ir s

aExampleTranquilizer.Targetvariable:

9patients,beforeandaftertakingthetranquilizer ” HamiltondepressionscalefactorIV”.

before(X (1)i )1.830.501.622.481.681.881.553.061.30after(X (2)i )0.8780.6470.5982.051.061.291.063.141.29 Difference(−Yi)0.952-0.1471.0220.430.620.590.49-0.080.01

(15)

42

bMatchedPairs.DifferencesYi=X (2)i −X (1)idistributedsymmetricallyaround0?

H0:ForeachYi,+and–signisequallyprobable

Gi=sign,|Yi|=” Yi”ofatwosampleproblem.Foreachconfigurationg (ℓ)=[g (ℓ)1 ,...,g (ℓ)n]theprobabilityis=1/2 n.

(16)

433.4

cFixateststatisticThg,zi

gi=+1or=−1,zi>0.Randomizationdistr.PhThG,zi=ti=#{g|Thg,zi=t}/2 n

•Thg,zi=(1/n) Pi gizi=aveihyiicorrespondstothettestformatchedpairs.

•Thg,zi=#{i:gi=1}:signtest.

•Thg,zi= Pi:gi=1 Ri,Ri:rankofzi:signedranktestofWilcoxon

(17)

44 3.4

eExample:

>wilcox.test(d.tranquilizer[,1],d.tranquilizer[,2],paired=TRUE)Wilcoxonsignedranktestdata:d.tranquilizer[,1]andd.tranquilizer[,2]V=40,p-value=0.03906alternativehypothesis:truemuisnotequalto0barelysignificant.

Bewareofbefore-aftercomparisons!Adequate:Comparisonwithcontrolorcrossoverexperiment

(18)

45

3.5

E st im a to rs a n d C o n fi d e n c e In te rv a ls

aModel:Testingproblemwas:Isthedistr.symmetricaround0?Moregeneral:...symmetricaroundµ

⇔Yi−µsymmetricaround0.

Test:TeststatisticThg,y−µ1i.LargevaluesindicatedeviationfromH0:µ.

bThisyieldsanestimator:

bµ=argminµ hThg,y−µ1ii

(19)

463.5 cSignedrankTest−→Hodges-Lehmannestimator.FormWalshaverages(Xh +Xi)/2.

bµ=medh≤i h(Xh +Xi)/2i.

ExampleTranquilizer:45Walshaverages-0.1470,-0.1135,-0.0800,-0.0685,-0.0350,0.0100,...,1.022Medianbµ=0.46

(20)

47 3.5 d

*

Derivation:X[k] kthsmallestvalue.X[k] >0,Zhk=(X[h] +X[k] )/2,h<kZhk<0,if|X[h] |>|X[k] |.

#{Zhk<0}=#{h||X[h] |<|X[k] |}=R[k] −1 R[k] =#{h|Zhk>0,h≤k}.

X[k] <0=⇒Zhk<0,ifh<k.

Thg,zi= Pi:gi=1 Ri=#{[h,k]|Zhk>0,h≤k}

Nullhypothesisµ=µ0:Thg,zi= Pi:gi=1 Ri=#{[h,k]|Zhk>µ0,h≤k}

Testisleastsignificantif= n(n+1)2−→bµ=medianhZhk|h≤ki.

(21)

483.5

fConfidenceintervallforthesignedranktest:

LimitsoftheacceptanceintervalofT:

candc ′=n(n+1)/2+1−cConfidencelimits=cthandc ′thWalshaverage.

ExampleTranquilizer:c=6,c ′=40,Confidenceinterval[0.01,0.786].

(22)

49

hForageneralteststatisticThG,z ∗;µi:Let

Qhβi=PhThG,z ∗;µi>Thg ∗,z ∗;µii−β

Estimator=solutionofQhβ=0.5i=0.Confidencelimits=solutionof

Qhβ=0.025i=0andQhβ=0.975i=0.Notdifficult!

(23)

50

3.6

M o re th a n 2 S a m p le s

aSimpleAnalysisofVarianceRandomization=assignmentofobservationstogroups.Nummberofobservationsineachgroupisfixed.

Ranktheyisamongallobservations−→RiAveragetheranksovergroupsRh =avegi=h Ri.EhRh i=(n+1)/2.Formweightedmeanofsquaresofdeviations

Thg,yi= 12n(n+1) X

h nh Rh − n+12 2

Kruskal-Wallistest.2samples−→U-Test.

(24)

513.6

bMatchedSamples=blockdesign

nblocks,mtreatments.Randomization?

cExampleacidicsoils

Position

Block1234567

14.095.915.405.135.435.875.21

23.904.074.344.134.394.323.29

35.276.265.725.695.705.363.50

44.534.304.864.615.035.403.95

(25)

52

Position

pH

1234567

3.5 4.5 5.5

(26)

53 Friedmantest.Rij=Rankofobservationjinblocki.eRj=aveihRijiaveragerankofsamplej.

T= 12nm(m+1) Xmj=1 ( eRj−(m+1)/2) 2.

dPositionBlock1234567

11742563223647513276453143254671

Summe819211423216Mittel24.755.253.55.755.251.5

(27)

54

>friedman.test(t.dt)Friedmanranksumtestdata:t.dtFriedmanchi-squared=14.8,df=6,p-value=0.02199

eAnalysisofvarianceforthisexample:

>summary(aov(pH~trans+pos,data=t.d))

DfSumSqMeanSqFvaluePr(>F)trans10.120.120.170.68pos10.180.180.270.61Residuals2516.570.66

(28)

55

3.7

C o rr e la tio n a n d R e g re ss io n

aCorrelationandsimpleregression.Xi,Yi(Xirandomorfixed)Nullhypothesis:“norelationship”Randomization=matching=permutationofY.Probabilityofeachpermutation=1/n!=1/(n(n−1)...2·1).

Teststatistic:

•simple(Pearson)correlation,

•rankcorrelation,

•robustestimatoroftheregressioncoefficient,...

(29)

563.7

bMultipleRegression:PermutationofYfortestingthenullhypothesisthatthereisnorelationshipbetweenallexplanatoryvariablesandthetargetvariable.

cTimeSeries:Aretheobservationsindependent?Randomization:Permutation.Teststatistic:e.g.firstautocorrelation.

dMultipleRegression:Singlecoefficient(orseveral)

−→noproperrandomizationmodel.

(30)

573.7

e

*

Permutationsandotherrandomizations.Regressionandcorrelation:permutations.Twoormoresamples:subsets(“choices”).therearemanymorepermutations;manyofthemleadtothesamepartitionintogroups

−→samerandomizationdistribution.

Hailexperiment:Numberofpotentialhaildayswasrandom,proportionsoftreateddaysalsorandomrandomizationdistribution:Choicesof33from76days

−→conditionaltest,giventhenumberoftreatedandcontroldays.

(31)

58

MessagesRandomizationTests

•Randomizationtestskeepthelevelexactly,withoutanyassumptionsonthedistribution.(Independenceofobservationsisessentiallyassumed.)

•Theteststatisticmaybearbitrarilycomplicated.Chooseconsidering(informally)thepower.Chooserobustteststatistic(e.g.,basedonranks)!

•Confidenceintervalscanalsobeconstructed.