Learning and the replicability of priming effects Michael Ramscar
Thewayindividualsbehaveusuallydependsontheinformation availabletothem.Thisraisestwoquestions:Wheredoes informationcomefrom?And,whatdoesitmeanforitto‘be available’?Withregardsthelatterquestion,whileitisclearthat stimulican‘prime’information—allowingitsavailabilitytobe manipulated—thelimitsofprimingandthereplicabilityof primingeffectshavebecomesubjecttocontroversy.Beginning withthefirstquestion—learning,andthelearnedbasisof priming—thisreviewdescribeswhy,inthelimit,alldirect replicationsofprimingstudiesinvolvinglearnedinformation canbeexpectedtofail,andwhythismeansthatpriming researchersshouldfocusondevelopingformalaccountsof thesephenomena,ratherthanstudyingprimingeffects themselves.
Address
EberhardKarlsUniversita¨tTu¨bingen,Germany
Correspondingauthor:Ramscar,Michael(ramscar@gmail.com)
CurrentOpinioninPsychology2016,12:80–84
ThisreviewcomesfromathemedissueonSocialpriming EditedbyFritzStrackandNorbertSchwarz
http://dx.doi.org/10.1016/j.copsyc.2016.07.001 2352-250/#2016ElsevierLtd.Allrightsreserved.
Introduction
Sittingonthemorningtrainafteragoodnight’ssleep,you seeanotherpassenger yawn.Despitebeingwellrested, youmanifestaclassiccontagiousbehavior:youyawn as well.Contagiousyawningisnotuniquetohumans;seeing conspecificsyawnonvideoisenoughtopromptbonobos, chimpanzees,stump-tail macaques,geladababoons and evendogstodolikewise[1,2].
Norisyawningtheonlybehavioralresponsethatcanbe elicitedsimplybyexposing someonetoanappropriate stimulus. Studies of ‘priming’—as this process is called—have been conducted across a wide range of areasofpsychology.However,thereplicabilityofmuch ofthisresearchhasbecomethesubjectofconsiderable controversy[3–6].Afterdescribingwhatprimingis,and thekindsofprimingstudiedinpsychology,thisreview focusesontherelationshipbetweenlearningand prim- ing, describing how learning constrains the generaliz- abilityofallprimingeffects,andwhydirectreplications
ofprimingeffectsshould allbe expectedtofailinthe long run. The value of conceptual replications, and ways of improving their contribution to theory, are discussed.
Whatispriming?
When someone is asked to read a word, and to judge afterwardswhetheranotherstringoflettersisalsowordor not, if thetarget isrelated to the prime in their native language (e.g., if doctor is used to prime nurse), their judgment will be made more quickly [7]. Similarly, if peoplearepre-exposedto exhaustor exhumeinonetask, theybecome more likelyto produce thesewords when askedtocompleteexh________inalater,‘unrelated’task.
Thislatter effect persists for some time,and manifests itself even if people cannot explicitly recall their pre- exposuretotheprimes[8].
Priming appears to occur because an initial stimulus (whichneednotbeverbal,orevenverbalizable)provides information about a subsequent event. The behavioral facilitationaffordedbyaprimedependsonthedegreeto which the information it makes available reduces the uncertainty associated with that subsequentevent, and behaviorthatis influencedby theanticipationof it [9].
Accordingly, although doctor–nurse facilitates faster responses than bicycle–nurse, even greater facilitation occurs if nurse itself primes nurse [10]. Priming thus appearstobeanexampleof themore generalinfluence ofinformationanduncertaintyonbehavior,whichisseen, forexample,infrequencyeffects:Evenintheabsenceof priming,wordsthatoccurfrequently(orinadvantageous lexicalneighborhoods)facilitatefasterresponsesinmany tasks, because their frequency of occurrence results in thembeingbetterdiscriminatedbylearningascompared tootherwords,reducingtheuncertaintyassociatedwith themincontext [11].
Varietiesofpriming?
Primingresearchinpsychologyextendsfarbeyondword completionsandlexicaldecisions,andexaminestopicsas diverseas:theinfluenceofmotionlanguageontemporal judgments[12],theinfluenceofraceperceptiononobject identification[13],theeffectsofevaluationonunrelated items [14], and the effects of invoking stereotypes on behavior[15].Althoughsomeresearchershavesoughtto distinguishsomeofthesemoreabstractformsofpriming researchfromthelexicalprimingstudiesdescribedabove [5], Fergusonand Mann [14] arguethat no principled divisionbetween these lines of research can bedrawn:
languagesare themselvessocial embeddedsystemsand lexicalpriminginfluencesgoalsandbehavior.
This problem is further exacerbated by the fact that across all forms of priming research, the ‘mediators’ of priming are typically described using poorly defined terms (e.g., semantics, concepts, stereotypes, etc.
[16]),meaningthatsharpdistinctionsintheinformation provided by different primes are also difficult to draw.
Unsurprisingly, empirical studieshave shownboth that priming attributed to more abstract constructs can be empirically predictedbylexicalmeasures [17],and that lexical primingeffectscanbeinfluenced bynon-lexical factors(e.g. [18]).
Varieties ofreplication
Theseoverlapsbetweenlexicalprimingandotherforms ofpriminghighlightanimportantpoint:priminginevita- bly depends on learning and experience. That is, for anythingotherthaninstinctivebehavior,somethingmust be learned about a prime before a priming effect can occur.Thisisespeciallyrelevanttothedebateaboutthe replicability of specific priming results, because in dis- cussing thereplication of results, researchersoften dis- tinguish ‘direct’ replications, in which all of methods, materialsetc.arethesameasintheoriginalstudy,from
‘conceptual’ replications, in which the theoretical con- structs testedin anoriginalstudyare operationalizedin differentwaysinareplication[19].However,giventhe learningrequirementinpriming,itfollowsthatunlessthe participants in a second study have acquired the same relevant information as theparticipants in the first,the secondstudycannotbeadirectreplicationofthefirst(In much thesame way, areplication of an English lexical decisionexperimentthatfollowedeveryprocedureapart from using non-English speaking participants is not a directreplication).
Thisraisestwoquestions:
1. Whatinformationisrelevant toagiveneffect?
2. How can information be operationalized so that its equivalence inpopulationscanbeestablished?
Learning andpriming
Inthecaseoflexicalpriming,theanswertotheseques- tions amountsto describingwhat itis thatpeople have learnedaboutdoctorthatprovidesthemwithinformation aboutnurse.However,whereasdiscussionsinthepriming literaturetendtoassumethatthisinformationcomprises aseriesofbinaryassociationsbetweenprimesandtargets (thatarethen‘activated’inpriming[5,20,21]),themech- anisticmodelsdevelopedtoexplainandpredictassocia- tive learning itself have explicitly rejected this notion [22,23,24,25].
Research has showninsteadthat—evenin animals—
‘individual associations’ form as the result of a process that seeks to predict (or at least minimize a learner’s uncertaintyabout)eventsintheworld.Ratherthanbeing
directly associative, learning serves to discriminate the aspectsoftheenvironmentthatbestsupportpredictions from those that don’t. Computationally, this process is error-driven [23,24]:the valuesof cues thatprime ex- pectationsthataresubsequentlyviolatedaredecremented (discriminated against), while the values of cues that prime reliableexpectationsare incremented(rewarded).
Learning causes sensoryandexperiential informationto compete for relevance, and this discriminative process results in a dynamic model of the environment that is continuallyupdatedaseventsunfold[22,24].
Becauseassociationsareaspectsofadynamicsystem,the information conveyedbya prime aboutatargetcannot beadequatelypredictedorexplainedfromananalysisof therelationshipbetweentheprimeandthetargetalone.
Rather,thelearnedstrengthofanyprime–targetrelation- shipisaffectedbytherelationshipbetweenthetargetand otherprimes[26],andtherelationshipbetweentheprime andothertargets[25].Thisconsiderablycomplicatesthe taskof controllingforlearning in subjectpopulationsin primingexperiments.
Primingand theenvironment
Thesystemicnatureoflearningmeansthatthestructure of a learningenvironment is critical to what is learned (and when).Itisthusalso criticaltoassessing whatthe learned information relevantto primingis,both for the purposesofinterpretingprimingstudies,andforreplica- tions of them. The ways in which the structure of the environment helps determinethe information available to a learner at a given point in time can be easily illustrated in relation to lexical learning: When lexical dataisaggregatedinacorpus,thedistributionofwordsis highly skewed [11,27]: while a corpus of 450 million aggregatedwordsmightcontaintokensofaround3mil- liondifferentwordtypes,aroundhalfofthetokenswillbe of only 100 types [28]. Thus while most of the words peopleusearefairlyfrequent,themajorityofwordtypes arerare,andthedistributionofrarewordsisbursty[29]:
whenatopicisbroached,topicalwordsareusedandre- used with far greater probability than their aggregate frequencieswouldpredict.
Individual experienceof lower-frequency words isthus fractionated across populations: as aggregate word fre- quency decreases, exposure to words becomes more limited, ever more specific to a particular domain of experience, and ever more specific to a smaller group ofspeakers.Thismeansthatthelikelihoodthataspecific effectoflexicalpriming—thatis,fromaspecificprimeto a specific target—will replicate, is a function of the frequencyoftheitemsinvolvedinthepopulation.Italso meansthatthedegreetowhichanyprimingresultcanbe generalizedisafunctionoftheitemsused,thefrequency of theseitems,andtheirdispersalin apopulation.This explainswhydoctorprimesnurseinundergraduatelexical
decisiontasks, aswellaswhy lexicaldecisionsfor engi- neering-relatedwordsareslowerinnursesthanengineers, andhealthcare-relatedwords fasterin nursesthan engi- neers[30].
Thestructure ofthelexicalenvironmentmeansthatan individual’sknowledgeoflanguagechangesconsiderably across the lifespan [31]. This in turn predicts that priming results will not generalize (and should not be expected to replicate) between different age groups within a population, or even across the course of the same adult’s lifespan(for empirical confirmationof this predictions, see[31,32,33]).Further,if one takesthe waypeople talkaboutthe worldto reflecttheirexperi- enceofit,theseconsiderationssuggestthatthegenerality of allpriming effects shouldbe similarlysubjectto the culturalandlearning-basedconstraints describedhere.
Learningand translation
These points also apply to cross-linguistic studies of priming.The structure of thelexical environment pre- dictsthatprimingeffectsfoundinnativespeakerswillnot generalizeto(andshouldnotbeexpectedtoreplicatein) non-native speakers. Accordingly, whereas intuitively one might think that someone’s ability to recall disease after priming with doctor would be bestin their native tongue, discriminative learning models predict—and empirical results confirm—thatolder Chinese-German bilinguals’ performance on a paired-associate learning taskisbothbetterinGermanthantheirnativelanguage, and better in German than that of age-matched native Germanspeakers[32].
Further,whilethespecificlanguageemployedinaprim- ingstudyisfrequentlyassumedtobeunimportantinthe literature [3,34,35], themany differences that existbe- tweenlanguages—in thefrequencyandvarietyof pos- siblecognates,indistributionalpatterns,inthedifferent waysin whichsignals areencoded,etc.[36]—severely constrain the degree to which specific effects can be expectedtogeneralizebetweenlanguages.Forexample, adjectives—which are often employed as primes in English studies (e.g. [15])—vary considerably in both theirfrequenciesand theirdistributionacross languages [37],and an analysis of thesedifferences indicatesthat primingeffectsobtainedusingEnglishshould noteven generalize to German (a language of the same broad family), nor should an effect be expected to replicate onthebasisofasimpleword-by-wordtranslation [38].
Learning, time andculture
Because lexical priming effects have replicated in the past,contributors to theprimingdebateusually assume theyaredirectlyreplicable(e.g.[5]).However,giventhe nature of learning and the structure of the linguistic environment,itfollowsthatevensuccessfulreplications of semantic priming effects should not be considered
direct replications. In English, the modern senses of the words doctor and nurse date from the Victorian era [39].Beforethis,theworddoctormeantamanoflearning,
‘nurse’achild-caresurrogate,andwhatwewouldnowcall adoctor was aphysician [40]. Accordingly,for the same reasonsthatonewouldnotexpectdoctortoprimenursefor apersonwhoknowsnoEnglish,formostofthehistoryof theEnglishlanguage, one would notexpect thatdoctor wouldprimenurseeither.
Thishighlightsthefactthatinnaturallanguages,associa- tions between words—and between words and the world—arenotstableovertime[41].Grammars, word co-occurrencepatterns, wordvalances,etc., alsochange continuously [11]. This means that across time, the learningrelevanttoanysetoflexicalstimulialsochanges continuously, such that the ‘robust’ lexical priming effects produced by specific sets of stimuli replicate because these sets have a sufficiently large number of itemsof sufficientfrequency (such that therate of lin- guisticchangeinitemssufficientlysmall)toallowroughly similar results to be obtained over relatively lengthy periods of time. However, in thesame way that doctor would not have primed nurse in an English speaker 300 years, the continuous nature of language change meansthatatsomepoint,doctormaywellfailtoreliably primenurse(justaspilotpresumablyprimesstewardessless todaythanitoncewouldhave).
Theprocessesof languagechangemean thatanyset of lexical primes that currently produce replicable effects willalmostinevitably failto doso in thelong run. The relevantinformationlearnedbyapopulationinanygiven replicationofprimingwillthusonlybesimilartothatof thepopulationsinotherstudiestoadegree.Accordingly, itfollowsthatabsentametricforestablishingthedegrees of similarityin the relevant informationlearned by dif- ferentpopulations,replicationsoflexicalprimingeffects cannotbeconsideredtobedirectreplications(see[19]).
Conclusionsand discussion
Theexperienceofsubjectpopulationsisacriticalmeth- odologicalcomponentofprimingresearch.Andbecause of the nature of learning and experience, virtually all learned priming effects should be expected to fail to replicateovertime.Evenwhen‘reliable’lexicalpriming effectsareconsidered,theseareonlyrobustinthemedi- um term, and only because they average across higher frequency items, and because linguistic experience of higherfrequencywordsismoreconsistentacrosspopula- tions[42].Acorollaryofthisisthatwhenareliablesetof itemsfailstoreplicateintime,itwillnotnecessarilymean thatwhatseemedtobereliableeffecthasturnedouttobe illusory. Rather, because in practice all replications of learnedpriming effectsare conceptualreplications, and becauseanyattemptataconceptualreplicationmustuse itemsthatprovidethelearnedinformationrelevanttoan
effect, manyso-called ‘failures to replicate’ will simply reflectthefactthisinformationisnotafixedpropertyof items,but ratherissubjecttoconstantcultural change.
Inthelimit,thismeansthatintheabsenceofameansfor objectively determining whatthe information that pro- duces apriming effectis, and for determining that the same information is available to the population in a replication, all learned priming effects are scientifically unfalsifiable.(Whichalsomeansthatintheabsenceofan account of what the relevant information is in a set of primes,andhowitproducesaspecificeffect,reportsofa specificprimingresult—orfailurestoreplicateit —are scientifically uninformative; see also [19].) For many areasof primingresearch,thedevelopmentofaquanti- tative, mechanistic account of why a given stimulus primes agiven behavioris probablyatall order;raising questionsaboutwhetherresearchin theseareasisbest- servedbythequantitativeframeworkofexperimentation inwhichtheirresultsarecurrentlyreported.However,for lexical stimuli at least, formal models that allow the relationships betweenprimesandtargetstobeestimated nowexist[43,44,45],asdomodelsthatallowtheeffectsof experienceonprimingtobesimulated[28,31,31,46,47].
Althoughallof thesemodelshave limitations,it istobe hopedthateitherthey—ortheirfuturedevelopments— can moveprimingresearch towardapointwherestudies testandrefinethe predictionsofwell-specified scientific models, and away from cataloging what are otherwise transientculturaleffects.
Conflictof intereststatement Nothing declared.
References and recommendedreading
Papersofparticularinterest,publishedwithintheperiodofreview, havebeenhighlightedas:
ofspecialinterest ofoutstandinginterest
1. SilvaK,BessaJ,deSousaL:Auditorycontagiousyawningin domesticdogs(Canisfamiliaris):firstevidenceforsocial modulation.AnimCogn2012,15:721-724.
2. AmiciF,AureliF,CallJ:Responsefacilitationinthefourgreat apes:istherearoleforempathy? Primates2014,55:113-118.
3. DoyenS,KleinO,PichonCL,CleeremansA:Behavioralpriming:
it’sallinthemind,butwhosemind?PLoSOne2012,7:e29081.
4. BarghJA:Primingeffectsreplicatejustfine,thanks.Psychol Today2012.Retrievedfromhttp://www.psychologytoday.com/
blog/the-natural-unconscious/201205/
priming-effects-replicate-just-fine-thanks[29.06.15].
5. HarrisCR,CoburnN,RohrerD,PashlerH:Twofailuresto replicatehigh-performance-goalprimingeffects.PLOSONE 2013,8:e72467.
6. ShanksDR,NewellBR,LeeEH,BalakrishnanD,EkelundL, CenacZ,MooreC:Primingintelligentbehavior:anelusive phenomenon.PLOSONE2013,8:e56515.
7. MeyerDE,SchvaneveldtRW:Facilitationinrecognizingpairsof words:evidenceofadependencebetweenretrieval operations.JExpPsychol1971,90:227.
8. TulvingE,SchacterDL,StarkHA:Primingeffectsinword- fragmentcompletionareindependentofrecognitionmemory.
JExpPsycholLearnMemCogn1982,8:336.
9. JaegerTF,SniderNE:Alignmentasaconsequenceof expectationadaptation:syntacticprimingisaffectedbythe prime’spredictionerrorgivenbothpriorandrecent experience.Cognition2013,127:57-83.
10. ForsterKI,DavisC:Repetitionprimingandfrequency attenuationinlexicalaccess.JExpPsycholLearnMemCogn 1984,10:680.
11.
BaayenRH,MilinP,RamscarM:Frequencyinlexical processing.Aphasiology2016:1-47http://dx.doi.org/10.1080/
02687038.2016.1147767.
Reviewsthemanyproblemsinvolvedincontrollingandmeasuringlin- guisticexperience,andfordeterminingtheinfluenceoflexicalpredictors, andintroducesarangeofformalmethodsforaddressingtheseproblems.
12. RamscarM,MatlockT,DyeM:Runningdowntheclock:therole ofexpectationinourunderstandingoftimeandmotion.Lang CognProcess2010,25:589-615.
13. EberhardtJL,GoffPA,PurdieVJ,DaviesPG:Seeingblack:race, crime,andvisualprocessing.JPersSocPsychol2004,87:876.
14.
FergusonMJ,MannTC:Effectsofevaluation:anexampleof robust‘‘social’’priming.SocCogn2014,32:33-46.
Athoughtfultheoreticalreviewoftheproblemsinvolvedinconducting directreplicationsofprimingstudies.
15. BarghJA,ChenM,BurrowsL:Automaticityofsocialbehavior:
directeffectsoftraitconstructandstereotypeactivationon action.JPersSocPsychol1996,71:230.
16.
RamscarM,PortRF:Categorization(withoutcategories).In HandbookofCognitiveLinguistics.EditedbyDawbroskaE,Divjak D.DeGruyterMouton;2015.
Athoughtfultheoreticalreviewoftheproblemsinvolvedinconducting directreplicationsofprimingstudies.
17. DaviesM:Theimportanceofrobustcorporainprovidingmore realisticdescriptionsofvariationinEnglishgrammar.Linguist Vanguard2015,1:305-312.
18. ZhangY,LeeCY:Processingtalkervariabilityinsemantic/
associativepriming:doestalkervoicematter?JAcousSocAm 2015,137:2416.
19.
StroebeW,StrackF:Theallegedcrisisandtheillusionofexact replication.PerspectPsycholSci2014,9:59-71.
Athoughtfultheoreticalreviewoftheproblemsinvolvedinconducting directreplicationsofprimingstudies.
20. CesarioJ,JonasKJ:Replicabilityandmodelsofpriming:what aresourcecomputationframeworkcantellusabout expectationsofreplicability.SocCogn2014,32:124-136.
21. Schro¨derT,ThagardP:Priming:constraintsatisfactionand interactivecompetition.SocCogn2014,32:152-167.
22. RescorlaRA:Pavlovianconditioning:it’snotwhatyouthinkit is.AmPsychol1988,43:151.
23. RamscarM,YarlettD,DyeM,DennyK,ThorpeK:Theeffectsof feature-label-orderandtheirimplicationsforsymbolic learning.CognSci2010,34:909-957.
24.
RamscarM,DyeM,McCauleySM:Errorandexpectationin languagelearning:thecuriousabsenceofmousesinadult speech.Language2013,89:760-793.
Reviewstheshiftfromassociativetodiscriminativeprocessingmodelsin learning.Erroneousclaimsabouttheinapplicabilityoflearningmodelsto languagedevelopmentwerelargelyresponsibleforthemorewidespread neglectoflearningtheoryinpsychology,andthepaperspresentsformal andempiricalevidenceshowingthatdiscriminativemodelscanbeused tosuccessfullypredictdetailedandsomewhatcounterintuitivepatterns ofbehaviorinchildrenastheylearnlanguage.
25. RamscarM,DyeM,KleinJ:Childrenvalueinformativityover logicinwordlearning.PsycholSci2013,24:1017-1023.
26. ArnonI,RamscarM:Granularityandtheacquisitionof grammaticalgender:howorder-of-acquisitionaffectswhat getslearned.Cognition2012,122:292-305.
27. PiantadosiST:Zipf’swordfrequencylawinnaturallanguage:a criticalreviewandfuturedirections.PsychonBullRev2014, 21:1112-1130.
28. RamscarM,HendrixP,ShaoulC,MilinP,BaayenH:Themythof cognitivedecline:non-lineardynamicsoflifelonglearning.
TopCognSci2014,6:5-42.
29. ChurchKW,GaleWA:Poissonmixtures.NatLangEng1995, 1:163-190http://dx.doi.org/10.1017/S1351324900000139.
30. GardnerMK,RothkopfEZ,LapanR,LaffertyT:Theword frequencyeffectinlexicaldecision:findingafrequency-based component.MemCogn1987,15:24-28.
31.
RamscarM,HendrixP,LoveB,BaayenR:Learningisnot decline:thementallexiconasawindowintocognitionacross thelifespan.TheMentalLexicon2013,8:450-481.
Reviewsarangeoffactorsthatresultinprimingbeingsubjecttocon- siderablechangeevenwithinindividualsastheirexperiencegrows.Also showshowthesechangesareconsistentwith—andpredictedby— basicprinciplesofdiscriminativelearning.
32. RamscarM,SunC-C,HendrixP,BaayenRH:The
mismeasurementofmind:whatpairedassociatelearning tasksrevealaboutcognitiveperformanceacrossthelifespan.
In Proceedings,38thAnnualMeetingoftheCognitiveScience Society;Philadelphia,Pennsylvania:2016.
33.
KeuleersE,StevensM,ManderaP,BrysbaertM:Word knowledgeinthecrowd:measuringvocabularysizeandword prevalenceinamassiveonlineexperiment.QuartJExpPsychol 2015,68:1665-1692.
34. Ga´mezE,Di´azJM,MarreroH:Theuncertainuniversalityofthe MacbetheffectwithaSpanishsample.SpanishJPsychol2011, 14:156-162.
35. AveyardME:Acalltohonesty:extendingreligiousprimingof moralbehaviortoMiddleEasternMuslims.PLOSONE2014, 9:e99447.
36. EvansN,LevinsonSC:Themythoflanguageuniversals:
languagediversityanditsimportanceforcognitivescience.
BehavBrainSci2009,32:429-448.
37. DyeM,MilinP,FutrellR,RamscarM:Afunctionaltheoryof genderparadigms.In MorphologicalParadigmsandFunctions.
EditedbyKieferF,BlevinsJP,BartosH.Netherlands:Brill,Leiden;
2016.
38. RamscarM:Theunspeakableinpursuitoftheunrepeatable.
TheImportanceofBeingWrong.2015:.Retrievedfromhttps://
ramscar.wordpress.com/2015/08/05/the-unspeakable-in- pursuit-of-the-unrepeatable/[09.04.16].
39. LevineME,ThorneS,SawatzkyR:Thehistoryofknowledge development.In KnowledgeDevelopmentinNursing:Theoryand Process.EditedbyChinnPL,KramerMK.ElsevierHealth Sciences;2014.
40. OED.OxfordEnglishDictionary,OxfordUniversityPress.
41.
BaayenRH,TomaschekF,GahlS,RamscarM:TheEcclesiastes principleinlanguagechange.In TheChangingEnglish Language:PsycholinguisticPerspectives.EditedbyHundtM, MollinS,PfenningerS.Cambridge:CambridgeUniversityPress;
2016.[inpress].
42. RamscarM,PortRF:Howspokenlanguagesworkintheabsence ofaninventoryofdiscreteunits.LangSci2016,53:58-74.
43. LandauerTK,DumaisST:AsolutiontoPlato’sproblem:the latentsemanticanalysistheoryofacquisition,induction,and representationofknowledge.PsycholRev1997,104:211.
44. Kievit-KylarB,JonesMN:Visualizingmultiplewordsimilarity measures.BehavResMethods2012,44:656-674.
45. MarelliM,DinuG,ZamparelliR,BaroniM:Pickingbuttercups andeatingbuttercups:spellingalternations,semantic relatedness,andtheirconsequencesforcompound processing.ApplPsycholinguist2015,36:1421-1439.
46. BaayenRH,HendrixP,RamscarM:Sidesteppingthe combinatorialexplosion:towardsaprocessingmodelbased ondiscriminativelearning.LangSpeech2013,56:329-347.
47. BaayenRH,RamscarM:Abstraction,storageandnaive discriminativelearning.In HandbookofCognitiveLinguistics.
EditedbyDabrowskaE,DivjakD.Berlin:DeGruyterMouton;
2015:99-120.
