Ecological Models Outstanding Challenges in the Transferabilityof Review

Review

Outstanding Challenges in the Transferability of Ecological Models

Katherine L. Yates ,

*

Phil J. Bouchet,

M. Julian Caley,

Kerrie Mengersen,

Christophe F. Randin,

Stephen Parnell,

Alan H. Fielding,

Andrew J. Bamford,

Stephen Ban,

A. Márcia Barbosa,

Carsten F. Dormann,

Jane Elith,

Clare B. Embling,

Gary N. Ervin,

Rebecca Fisher,

Susan Gould,

Roland F. Graf,

Edward J. Gregr,

Patrick N. Halpin,

Risto K. Heikkinen,

Stefan Heinänen,

Alice R. Jones,

Periyadan K. Krishnakumar,

Valentina Lauria,

Hector Lozano-Montes,

Laura Mannocci,

Camille Mellin,

Mohsen B. Mesgaran,

Elena Moreno-Amat,

Sophie Mormede,

Emilie Novaczek,

Steffen Oppel,

Guillermo Ortuño Crespo,

A. Townsend Peterson,

Giovanni Rapacciuolo,

Jason J. Roberts,

Rebecca E. Ross,

Kylie L. Scales,

David Schoeman,

Paul Snelgrove,

Göran Sundblad,

Wilfried Thuiller,

Leigh G. Torres,

Heroen Verbruggen,

Lifei Wang,

Seth Wenger,

Mark J. Whittingham,

Yuri Zharikov,

Damaris Zurell,

and

Ana M.M. Sequeira

Predictivemodelsarecentraltomanyscientificdisciplinesandvitalforinforming managementinarapidlychangingworld.However,limitedunderstandingofthe accuracyandprecisionofmodelstransferredtonovelconditions(their‘trans- ferability’)underminesconfidenceintheirpredictions.Here,50expertsidentified priorityknowledgegapswhich,iffilled,willmostimprovemodeltransfers.These aresummarizedintosixtechnicalandsixfundamentalchallenges,whichunderlie the combined need to intensify research on the determinants of ecological predictability,includingspeciestraitsanddataquality,anddevelopbestprac- ticesfortransferringmodels.Ofhighimportanceistheidentificationofawidely applicablesetoftransferabilitymetrics,withappropriate toolstoquantifythe sourcesandimpactsofpredictionuncertaintyundernovelconditions.

PredictingtheUnknown

Predictionsfacilitatetheformulationofquantitative,testablehypothesesthatcanberefinedand validatedempirically[1].Predictivemodelshavethusbecomeubiquitousinnumerousscientific disciplines,includingecology[2],wheretheyprovidemeansformappingspeciesdistributions, explainingpopulationtrends,orquantifyingtherisksofbiologicalinvasionsanddiseaseoutbreaks (e.g.,[3,4]).Thepracticalvalueofpredictivemodelsinsupportingpolicyanddecisionmakinghas thereforegrownrapidly(Box1)[5].Withthathascomeanincreasingdesireto predict(see Glossary)thestateofecologicalfeatures(e.g.,species,habitats)andourlikelyimpactsuponthem [5],promptingashiftfromexplanatorymodelstoanticipatorypredictions[2].However,in manysituations,severedatadeficienciesprecludethedevelopmentofspecificmodels,andthe collectionofnewdatacanbeprohibitivelycostlyorsimplyimpossible[6].Itisinthiscontextthat interestintransferablemodels(i.e.,thosethatcanbelegitimatelyprojectedbeyondthespatialand temporalboundsoftheirunderlyingdata[7])hasgrown.

Transferredmodelsmustbalance thetradeoff betweenestimationandprediction biasand variance (homogenization versusnontransferability, sensu [8]).Ultimately, models that can

Highlights

Modelstransferredtonovelconditions couldprovidepredictionsindata-poor scenarios, contributing to more informedmanagementdecisions.

Thedeterminants ofecological pre- dictability are, however, still insuffi- cientlyunderstood.

Predictionsfromtransferredecological modelsareaffectedbyspecies’traits, sampling biases, biotic interactions, nonstationarity, and the degree of environmental dissimilarity between referenceandtargetsystems.

Wesynthesize six technicaland six fundamental challenges that, if resolved, will catalyze practical and conceptual advances in model transfers.

Weproposethatthemostimmediate obstacletoimprovingunderstanding liesintheabsenceofawidelyapplic- ablesetofmetricsforassessingtrans- ferability, and that encouraging the developmentofmodelsgroundedin well-established mechanisms offers themostimmediatewayofimproving transferability.

790 TrendsinEcology&Evolution,October2018,Vol.33,No.10 https://doi.org/10.1016/j.tree.2018.08.001

1SchoolofEnvironmentandLife Sciences,UniversityofSalford, Manchester,UK

2CentreforExcellencein

EnvironmentalDecisions,Universityof Queensland,Brisbane,QLD,Australia

3SchoolofBiologicalSciences, UniversityofWesternAustralia,35 StirlingHighway,Crawley,WA6009, Australia

4ARCCentreforExcellencein MathematicalandStatisticalFrontiers, QueenslandUniversityofTechnology, Brisbane,QLD,Australia

5SchoolofMathematicalSciences, QueenslandUniversityofTechnology, Brisbane,QLD,Australia

6DepartmentofEcologyand Evolution,UniversityofLausanne, Lausanne,Switzerland

7HaworthConservationLtd, Bunessan,IsleofMull,Scotland

8Wildfowl&WetlandsTrust, Slimbridge,Gloucestershire,GL27BT, UK

9CanadianParksandWilderness Society,#410-698SeymourStreet, Vancouver,BCV6B3K6,Canada

10CIBIO/InBIO-CentrodeInvestigação emBiodiversidadeeRecursos Genéticos,UniversidadedeÉvora, 7004-516Évora,Portugal

11Biometry&EnvironmentalSystem Analysis,UniversityofFreiburg, TennenbacherStr.4,79106Freiburg, Germany

12SchoolofBioSciences,Universityof Melbourne,VIC3010,Australia

13SchoolofBiologicalandMarine Sciences,PlymouthUniversity,Drake Circus,Plymouth,PL48AA,UK

14DepartmentofBiologicalSciences, MississippiStateUniversity,Starkville, MS39762,USA

15AustralianInstituteofMarine Science&UWAOceansInstitute, UniversityofWesternAustralia,35 StirlingHighway,Crawley,WA6009, Australia

16GriffithClimateChangeResponse Program,BuildingG01,Room2.25, GoldCoastCampus,Griffith University,ParklandsDrive,Southport, QLD4222,Australia

17ZHAWZürichUniversityofApplied Sciences,CH-8820Wädenswil, Switzerland

18InstituteforResources, Environment,andSustainability, UniversityofBritishColumbia,AERL Building,2202MainMallVancouver, BC,Canada

19SciTechEnvironmentalConsulting, 2136NapierStreet,Vancouver,BC V5L2N9,Canada

20MarineGeospatialEcologyLab,

simultaneouslyachievehighaccuracyandprecision,evenwhenpredictingintonovelcontexts, willprovidemaximumutilityfordecisionmaking[9].Todate,however,testsoftransferability acrosstaxaandgeographiclocationshavefailedtodemonstrateconsistentpatterns(Figure1), andageneralapproach todevelopingtransferablemodelsremainselusive(butsee [6,10]).

Here,weoutlinechallengesthat,ifaddressed,willimprovetheharmonization,uptake, and applicationofmodeltransfersinecology.Wearguethatmovingthefieldofmodeloftransfer- ability forward requires a two-pronged approach focused on: (i) investing in fundamental researchaimedatenhancingpredictability,and(ii)establishingtechnicalstandardsforassess- ingtransferability.

DefiningtheChallenges

WefirstidentifiedchallengesusingamodifiedDelphitechnique[11](seethesupplementary information online), and then divided them into those that reflected conceptual obstacles (‘fundamental challenges’), and those related to best practices (‘technical challenges’).

Acknowledging significant overlap and linkages between these challenges (Figure 2), we exploreeachseparatelybelow.Attemptstounderstandandenhancetransferabilityfacemany ofthesamehurdlesas ecologicalmodelinggenerally(e.g.,dataquality,stochasticity),and adheringtobestpracticerecommendations(e.g.,[12,13])isthusimperative.Wedonotfocus onthesewell-establishedstandards,butconcentrateontheadditionalchallengesposedby transferringmodels.Whilstspatialtransferabilitystudiesretainprominenceintheliterature(and thusinthismanuscript),thisisnotanindicationofrelativeimportance,butratherareflectionof theinherentdifficultiesinevaluatingmodelstransferredthroughtime.Ourreviewofpublished studies is not exhaustive, and the online supplementary information provides additional literaturerelevanttoeachchallenge.

FundamentalChallenges

IsModelTransferabilityTrait-orTaxon-Specific?

Knowingwhethermodelsaremoretransferableforsometaxonomicgroupswouldbeusefulto increaseconfidenceinpredictionsandprioritizeresourcesfor modeldevelopment(Box1).

Box1.WhyTransferModelsintheFirstPlace?

Ecologicalmodelsareextensivelyandincreasinglyusedinsupportofenvironmentalpolicyanddecisionmaking[77].

Theprocessoftransferringmodelstypicallystemsfromtheneedtosupportresourcemanagementinthefaceof pervasivedatadeficiencies,limitedresearchfunding,andacceleratingglobalchange[5].Spatialtransfershavebeen usedtoguidethedesignofprotectedareas,searchforspeciesonthebrinkofextinction,informspeciesrelocationsor reintroductions,outlinehotspotsofinvasivepests,designfieldsamplingcampaigns,andassisttheregulationofhuman activities(e.g.,[78,79]).Forinstance,cetaceandensitymodelsdevelopedofftheeastcoastoftheUnitedStateswere recentlyextrapolatedthroughoutthewesternNorthAtlantichighseastoassistthemanagementofpotentiallyharmful sonarexercisesperformedbythemilitary[80].Similarly,projections ofAsiantigermosquito (Aedes albopictus) distributionmodelsontoallcontinentshelpedidentifyareasatgreatestriskofinvasion,withimportantimplications forhumanhealth[81].Temporaltransfershavelargelybeenappliedtoforecastspecies’responsestoclimatewarming, retrospectivelydescribepristinepopulation states,characterizeevolutionarypatterns ofspeciation,quantifythe repercussionsoflandusechanges,orestimatefutureecosystemdynamics(e.g.,[68,72,82]).Despitebeingdifficult toquantify,thesocietalandeconomicgainsfromtransferringmodelscanbesubstantial,andaremostreadilyillustrated bythemitigationofcostsassociatedwithinvasivespecies[83].Forinstance,theestablishmentofthezebramussel (Dreissenapolymorpha)intheGreatLakesregionofNorthAmericahasledto$20–100millioninannualmitigation expenditure,withadditional,unquantifiednonmarketcostsensuingfromthelossofbiodiversityandecosystemservices [5].Transferredmodelsaccuratelypredictedtheestablishmentofthezebramussel5yearsbeforeitwasactually discoveredintheregion,howevermodelpredictionswerenotusedtotakepreventativeaction, illustratingthat developingatransferablemodelisonlythestartoftheroadtoinformingdecisionmakers(seeOutstandingQuestions).

Ultimately,thewidespreadneedtomakeproactivemanagementdecisionsindata-poorsituationsdrivestheneedto improveourunderstandingofmodeltransferability.Thisgoalfundamentallyrequiresbettertransferabilitymetricsand estimatesofpredictionuncertainty,whichcanassistinselectingthemostconsistentandeffectivemanagementoptions whileavoidingunanticipatedoutcomes[84].

Evidenceindicatesdiscrepanciesinmodelperformanceamongtaxawithdivergentlife-history traits,andpopulationswithdifferentagestructuresandsexratios(e.g.,[14]).Meta-analyses demonstratethatbodysizeandtrophicpositionarestrongindicatorsofecologicalpredict- ability[15],withsomestudiesalsoindicatinggreaterhurdlesinbuildingtransferablemodelsfor wide-rangingorganismswithbroadenvironmentalnichesthanfornarrow-rangingspecialists [16].Forexample,modeltransfersforbutterflieswerelessaccurateinspecieswithlongflight seasons[17].Bycontrast, models ofvascularplantswith higherdispersalabilityexhibited bettertransferabilitythanthosebuiltforendemicswithlimiteddispersalcapacity[18].Devel- opingtransferablemodelsforspecieswithgreaterbehavioraloradaptiveplasticitymightalso bemoredifficult,regardlessofspatialrangesize[8].Subsettingmovementandobservational databybehavioralstate(e.g.,foragingversusbreeding)orgroupcomposition(e.g.,presence of mother–youngpairs) prior to model calibration might improve model performanceand transferability.

WhichResponseVariablesMakeModelsMoreorLessTransferable?

Thesuperiorinformationcontentinherenttoabundancedatashouldfacilitategreatertrans- ferabilitythanmodelsofoccurrencebuiltfrompresence–absenceorpresence-onlydata,so thatmodelsofabundancemightbetterprojecttheecologicalimpactsofglobalchange[19].

Whilethishasbeenshownforsomebirds[19],fittingabundancemodelsremainsdifficultfor most taxa [20],not leastbecause counting individualsis more challengingthan recording presence–absence(despiteissuescausedbyimperfectdetectability).Accordingly,interesthas grownincomparingthepredictionsobtainedfromoccurrenceandabundancemodels,and testing the reliability of the former as a surrogate for the latter [21]. In general, stronger correlationsbetweenabundanceandoccurrenceareexpectedforrareorganisms.However, thestrengthofthisrelationshipcanbenonlinear,species-specific,andconditionalonspatial behavior,socialorganization,life-historystrategies,populationdensity,resourceavailability, andbioticinteractions[22].Moststudieshavealsoappliedmodeltransferstosinglespecies.

Community-andecosystem-levelmodelsthatfitsharedenvironmentalresponsesformultiple speciessimultaneouslycouldachievehighertransferability[23],butthispotential hasbeen inconsistently demonstrated. Integrated models that unite presence-only and presence– absencedata[24],andthosethatcombineoccupancyprobabilities(e.g.,derivedfromregional monitoring)withdensity-given-occupancy(e.g.,derivedfromtelemetry),offerfurtherpromise [25].Theformerprovidemoreaccuratepredictionsthanmodelsbasedonasingledatatype, whereasthelattercanaccountforsuitablebutunoccupiedhabitats.

ToWhatExtentDoesDataQualityInfluenceModelTransferability?

Moreaccurateand/orprecisedatashouldresultinbettertransfersontheoreticalgrounds,with evidenceshowingthattheaccuracyofspeciesrecordscanbemoreimportantfortransfer- abilitythantheirspatialextent[26].Dataofunverifiablequality(e.g.,anecdotalreportsofeasily misidentifiedspecies)shouldthereforebeavoided,evenifavailableoverbroadergeographical areas.Modeltransferscanbefurtherhamperedbyimperfectdetectability,spatialandtemporal biasesindatacollection,insufficientsamplesizes,theomissionofknowndrivers,ortheuseof proxy variables [27]. Additionally, species’ characteristics such as range size can impact positionalaccuracy, leading to erroneous predictions if analyses are conducted at scales correspondingwiththoseoftheoriginallocationalerrors[28].Themagnitudeoftheseeffectsis ultimatelyunclear,anddataqualitythereforerepresentsasubstantialsourceofuncertainty[29].

Simulationstudies basedon virtualspecies with knownreference information representa criticalresourceintacklingthisknowledgegap.

NicholasSchooloftheEnvironment, DukeUniversity,Durham,NC27708, USA

21FinnishEnvironmentInstitute, BiodiversityCentre,POBox140,FIN- 00251Helsinki,Finland

22DHI,EcologyandEnvironment Department,AgernAllé5,DK-2970 Hørsholm,Denmark

23TheEnvironmentInstituteand SchoolofBiologicalSciences, UniversityofAdelaide,Adelaide,SA 5005,Australia

24CenterforEnvironmentandWater, ResearchInstitute,KingFahd UniversityofPetroleumandMinerals, Dhahran31261,SaudiArabia

25Institutoperl'AmbienteMarino Costiero,IAMC-CNR,Mazaradel Vallo,Trapani,Italy

26CSIROOceans&Atmosphere, IndianOceanMarineResearch Centre,TheUniversityofWestern Australia,Crawley,WA6009,Australia

27UMRMARBEC(IRD,Ifremer,Univ.

Montpellier,CNRS),InstitutFrançais deRecherchepourl'Exploitationdela Mer,Av.JeanMonnet,CS30171, 34203Sète,France

28AustralianInstituteofMarine Science,PMBNo3,Townsville4810, QLD,Australia

29DepartmentofPlantSciences, UniversityofCalifornia,Davis,One ShieldsAvenue,Davis,CA95616,USA

30DepartamentodeSistemasy RecursosNaturales,Universidad PolitécnicadeMadrid,Ciudad Universitaria,s/n,28040,Madrid,Spain

31NationalInstituteofWaterand AtmosphericResearch(NIWA),301 EvansBayParade,Wellington6012, NewZealand

32MarineGeomaticsResearchLab, DepartmentofGeography,Memorial UniversityofNewfoundland,St.John’s, NL,Canada

33RSPBCentreforConservation Science,RoyalSocietyforthe ProtectionofBirds,TheDavid AttenboroughBuilding,Pembroke Street,CambridgeCB23QZ,UK

34BiodiversityInstitute,Universityof Kansas,Lawrence,KS66045,USA

35UniversityofCalifornia,Merced,5200 NLakeRd,Merced,CA95230,USA

36SchoolofScience&Engineering,The UniversityoftheSunshineCoast, Maroochydore,QLD4558,Australia

37CentreforAfricanConservation Ecology,DepartmentofZoology, NelsonMandelaUniversity,Port Elizabeth,SouthAfrica

38DepartmentofOceanSciencesand DepartmentofBiology,Memorial UniversityofNewfoundland,St.John's,

NLA1C5S7,Canada

39DepartmentofAquaticResources, SwedishUniversityofAgricultural Sciences(SLU),Stångholmsvägen2, 17893Drottningholm,Sweden

40UniversitéGrenobleAlpes,CNRS, Laboratoired’EcologieAlpine(LECA), GrenobleF-38000,France

41MarineMammalInstitute,Department ofFisheriesandWildlife,OregonState University,2030SoutheastMarine ScienceDr.,Newport,OR97365,USA

42DepartmentofEcologyand EvolutionaryBiology,Universityof Toronto,Toronto,ONM5S3B2, Canada

43GulfofMaineResearchInstitute, Portland,ME04101,USA

44OdumSchoolofEcology,University ofGeorgia,Athens,GA30601,USA

45Biology,SchoolofNaturaland EnvironmentalSciences,Newcastle University,Newcastle-Upon-Tyne,NE1 7RU,UK

46PacificRimNationalParkReserve, ParksCanadaAgency,Box280, Ucluelet,BCV0R3A0,Canada

47SwissFederalResearchInstitute WSL,Dept.LandscapeDynamics, Zuercherstrasse111,CH-8903 Birmensdorf,Switzerland

48Humboldt-UniversitätzuBerlin, GeographyDept.,UnterdenLinden6, D-10099Berlin,Germany

49IOMRCandTheUniversityof WesternAustraliaOceansInstitute, UniversityofWesternAustralia, Crawley,WA6009,Australia yJointfirstauthors

*Correspondence:

K.L.Yates@Salford.ac.uk(K.L.Yates).

HowCanSamplingBeOptimizedtoMaximizeModelTransferability?

Samplesencompassingthefullrangeofenvironmentalconditionsandtheirpossiblecombi- nationsshouldavoidincompletenichecharacterizationandimprovetransferability(Box2).

However,dataare often collectedopportunisticallyand pooledduringanalysis, suchthat model building ought to account for uneven sampling in environmental space (e.g., by includingrandomeffects,orthrough explicitbalancing methodsthatcapturethe intensity and distribution of sampling effort [30]). Importantly, data resolution influences model fit, prediction,andbyextension,transferability.Forexample,poorlyresolvedpredictorsmight notcaptureimportantaspectsofaspecies’ecology,andrelateonlyindirectlytoobserved patternsofoccurrenceandbiogeography[31,32].Wherepossible,thescale(s)overwhichthe processesofinterestoperateshouldthereforedrivepredictorchoice, withsensitivitytests advisable[31].Ashabitatavailability,andthusperceivedpreference,alsooftenlinktoscale [33], modelswillbe sensitiveto the extentof the studyregion, especiallyfor fragmented habitats and steep environmental gradients [8]. As such, combining geographically and environmentallydistinctregionsoughttoincreasemodeltransferability[34].Temporalrepli- cationinsamplingcanalsohelpbycapturingnaturalvariabilityandstochasticprocesses,as wellasalleviatingimperfectdetectabilityandfalsenegativerates.Whenresourcesarelimited, samplingshouldideallyfocusondesignsthataddressexistingdatalimitationsandmaximize informationgain.

HowDoesModelComplexityInfluenceModelTransferability?

Excessively complex models risk overfitting training data and can erroneously attribute patternstosamplingorenvironmentalnoise[35],leadingtopredictionsthat arebiasedor toospecific tothereference systemtobetransferable[36].Greatertransferability isthus generallyexpectedinparsimoniousmodelswithsmoothunivariateresponsecurvesandfew predictors[37].However,whilesimplemodelshavebeenshowntoleadtobettertransfer- ability,theycanalsoyieldmisleadingpredictionswhentransferredtonewcontexts,implying thatsimplicityisnotalwaysbeneficial[38,39].Ultimately,simpleandcomplexmodelsserve different purposes [40], and in some instances, a preference for accurate and precise predictionsoverecologicalinterpretabilitymightbejustifiable,makingcomplexmodelsmore appropriate[41].Complexmodelsarealsonotnecessarilymorearduoustointerpret,andcan bevaluable for discoveringhidden,unexpected patterns [40].Additionally,they could be usefulin exploringnonlinear anddynamic associations of specieswithindirectpredictors acrosslandscapes,seasons,oryears[40],tohelpbetteraccommodatenonstationarity.

Thatsaid,ascomplexitygrows,sodopotentialpredictorcombinationsandthelikelihoodof mismatchbetweenreferenceandtargetconditions,whichcanresultinincorrectinterpolation andextrapolation[42].Species’life-historytraits,physiology,orbehaviorcanalsoinfluence complexity,suchthat choosing an optimallycomplexmodel requiresidentifyingthe most sensiblepredictorsanddatasetsrelativetoagivenstudyobjective.Novelindicesofcom- plexitythat emphasizethestructuralpropertiesoftheinputdatamighthelp[43],ascould standardizedmetricsofpredictiveperformance.

AreThereSpatialandTemporalLimitstoExtrapolationinModelTransfers?

Whilepredictionerrorisexpectedtoincreasewith‘distance’(e.g.,km,days)fromreference conditions [1], model transferability appears little related to geographic (and temporal) separationbetweenreferencesystemsandtargetsystems(Figure1).Instead,environmen- tal dissimilarity is what matters most for successful transfers, for which spatio-temporal distances might only occasionally be good surrogates. The minimum level of similarity requiredtosupporttransferablemodels,however,remainsunknown.Someauthorscaution againstseekinginferencebeyondone-tenthofthesampledcovariaterange,yetthisruleof

thumb[44]doesnottranslateintopracticalandcomprehensibleguidelinesformodelend- users(e.g.,spatialplanners,resourcemanagers).Anothersolutioncouldlieinthe‘forecast horizon’,whichdefinesthepointbeyondwhichsufficientlyusefulpredictionscannolongerbe madeinanygivendimension(e.g.,space,time,phylogeny,environment)[45].Calculatingthis horizonrequireschoosing ameasureofpredictionquality(i.e.,afunctionofaccuracyand precision),andaproficiencythresholdfor‘acceptable’predictions[45].Bothchoicescanbe framed in decision theory and informed through stakeholder participation, making the forecasthorizonaflexibleandpolicy-relevantinstrumentforassessingandcommunicating ecologicalpredictability.

TechnicalChallenges

HowCanNon-analogConditionsBeAccountedforWhenTransferringModels?

Transferringmodelsintonon-analogousenvironmentsbringsnumerousandwell-documented perils[46],butthepredictiveperformanceofmodelstransferredintonovelconditionsisrarely testedexplicitly[47].Differenttechniquestoaccountfornon-analogconditionswilllikelybe requireddependingonthe degree of environmentaldissimilarity (i.e.,novel conditionsjust beyondthoseobservedversusthosethatareextremelydissimilar).Severaltoolsareavailable tovisualizeregionswhosecharacteristicsdepartfromtheinitialcovariaterange(e.g.,[42,48]), and these can help assess the potential impacts of non-analog conditions on predictive performance.However,thesetoolscannotpredictspecies’responsesto novelconditions, whichcanbeparticularlyunexpectedifenvironmentalchangeimposesselectionpressuresthat disruptbioticinteractionsandcausecommunitiestoevolve[49].Furtherdevelopmentofthese tools for future transfers, and their application in examiningof the outcomes of historical transfers,will improveourunderstandingonhownon-analogconditionscanbeaccounted forwhentransferringmodels.

HowCanNonstationarityandInteractionsBeIncorporatedinModelTransfers?

Successfultransfersrelyontheinherentpremisethatspecies–environmentrelationshipsare stationaryatthecalibrationsiteandremainsobeyondit.However,species’responsestothe environment are rarely static, and can vary nonlinearly with resource availability, species ontogeny,andpopulationdensity[50].Species–environmentrelationshipsarethereforecon- text-specific,andhabitat occupationultimatelydepends onrelativehabitat availability[33].

Moreover,anthropogenicactivitiescanstronglyinfluencespecies’distributionandabundance patterns,and arethemselvesvariable [51].Disentanglingtheir effectsfrom environmentally drivencovarianceisdifficult,especiallywhenhistoriesofhumanexposureareunknown,orthe magnitudeofimpactsunobservable.Recentstudieshavealsoreconciledtransferabilitywith strongevidencefortheroleofbioticinteractionsinshapingspecies’rangesatlargespatial scales[52],offeringablueprintfordeterminingwhenbioticinformationcansupportpredictions under unobserved conditions. Methods that incorporate functional responses have now progressedtocombinedatafromdifferentregionsandusenonstationarymodelcoefficients, enablingenhancedtransferability[8,53].Weexpectfurtherimprovementsinknowledgewillbe madebyencouragingthedevelopmentofmodelsgroundedinwell-describedmechanisms (Box3).

DoSpecificModelingApproachesResultinBetterTransferability?

Studieshavebenchmarkedthe predictivecapacity andtransferabilityofexistingalgorithms underarangeofparameterizationscenarios,withmixedresults(e.g.,[54,55]).Randomforests and boosted regression trees, two data-driven approaches that are relatively immune to overfitting and can handle predictor interactions, can demonstrate high performance in unsampledareas(e.g.,[56]).MaxEnt,another machinelearning method,hasbeen ranked

Glossary

Anticipatorypredictions:

predictionsarisingfromextrapolating thestateofasystem(ecologicalor otherwise)eitherintothefuture (forecasts),underuncertaintyaround modelparameters(projections),or withinsystemslikelytobeimpacted byhumanaction(scenarios).

Bioticinteractions:interactions betweenorganisms,suchas predation,competition,facilitation, parasitism,andsymbiosis.

Correlativemodel:modelfittedto dataandrelatingspeciesoccurrence orabundanceatknowntimesand locationstosetsofenvironmental (bioticandabiotic)factors.Theaim ofacorrelativemodelistodescribe theconditionsproscribingaspecies’

range,therebygeneratinga quantitativeestimateofits geographicaldistribution.

Cross-validation:processof partitioningadatasetinto complementarysubsets,developing themodelonone(i.e.,trainingset) andvalidatingitontheother(s)(i.e., thevalidationset).Cross-validationis mostcommonlyusedtoestimate predictiveperformance;asinglefinal modelisoftenfittedtothefull dataset.

Explanatorypredictions:testable expectationsaboutindividual systems,outcomes,orproperties, derivedfromscientifictheory.The aimofexplanatorypredictionsisto constructand/orcorroborate hypotheses,andestablish explanationsforthemechanisms underpinningthefunctioningof naturalsystems.

Extrapolation:processofmaking predictionstocovariatevaluesthat areoutsidetherange,correlation structure,orvaluecombinationsof thoseinthetrainingdata.Canbe spatial,temporal,environmental,or anycombinationsthereof.

Fundamentalniche:fullsetof conditionsandresourcesan organismiscapableofexploitingto maintainpopulationsintheabsence ofbioticinteractions,dispersal limitations,habitatdegradation,or immigrationsubsidy.

Mechanisticmodel:model representingcausalprocesses underlyingrelationshipsbetween componentsofthestudiedsystem.

Usuallydevelopedbasedona

combinationofexpertandempirical knowledgeofthedominantrange- limitingprocessesthatunderlie survivalandreproductionofthefocal species(e.g.,physiology,population dynamics,andcompetitive interactions).Inmechanisticmodels, parametershaveaclearbiologicalor ecologicalinterpretationthatis definedapriori,suchthattheycan bemeasuredindependentlyofthe inputdata.Synonym:process-based model.

Non-analogconditions:conditions differingfromthosecurrently experiencedbyaspecies,including thosethatdonotpresentlyexist.

Termoftenusedtodescribefuture climates,butalsocommunitiesthat arecompositionallyunlikeanyother foundtoday.

Nonstationarity:stateofasystem inwhichrelationshipsbetween variablesandbyextension,model parameters,donotremainconstant throughspaceandtime.Antonym:

stationarity.

Predict:anticipateanunknown quantityorvariablebeforeitis observed.

Realizedniche:portionofthe fundamentalnichethataspecies actuallyoccupies,becauseof constrainingeffectssuchas biologicalinteractionsordispersal limitations.

Referencesystem:systeminwhich amodeliscalibratedbeforetransfer.

Targetsystem:systemtowhicha modelistransferred.

Transferability:capacityofamodel toproduceaccurateandprecise predictionsforanewsetof predictorsthatdifferfromthoseon whichthemodelwastrained.For instance,spatiallydistinctfor projectionstonewareas,or temporallydistinctforprojectionsto pastorfuturetimes.Synonyms:

cross-applicability,generalizability, generality,transference.

themosttransferableinsomestudies(e.g.,[57]).Generalizedlinearandadditivemodelshave also been identified as robust choices for extrapolation (e.g., [37]), despite potential for generatingunrealisticpredictionsoutsidethetrainingscope.However,differentapproaches tomodeltuninganddatatreatmentcontributetoheterogeneityinperformance[58],makingthe suitabilityofanygiventechniquelargelycase-specific.A‘silverbullet’algorithmthatisbest under all circumstances is therefore highly unlikely, and other factors, such as species’ characteristics,can sometimesmatter morethan modelchoice [59]. Modelaveragingcan avoidoverrelianceonasingletechniquebyprovidingaweightedaverageofcompetingmodel predictions[60],and techniquesthat enable modelcoefficientsto fluctuate inresponseto changesinhabitatandresourceavailability[53] shouldimprovetransferability[8].Inrecent years,dynamicmodelscapableoftrackingthetemporalaspectsofaspecies’behaviorand distribution,andjointspeciesdistributionmodelsdesignedtosimultaneouslyaccountforthe co-occurrenceofmultiplespecies,havealsogainedtraction.Althoughstillinearlystagesof development,preliminaryfindingsindicatepotentialforimprovedpredictiveperformance[61].

Mechanisticmodelsthatharnesspriorbiologicalknowledgewithinagivensystem(Box3) couldalsoenhancetransferability,yetremainmostlyundertested[62,63].

HowShouldUncertaintyBeQuantified,Propagated,andCommunicatedWhen TransferringaModel?

Uncertainty arises from many sources [64], including: sampling methodology, species vagrancy,dataquality,environmentalstochasticity,initialconditions,speciesidentification, model specification, predictor choice, algorithm selection, and parameter estimation [7,45,57].Improvingpredictability,andthusdecision making(Box 1) [65],requires under- standingthe origins, propagationpathways,and ramifications ofuncertainty, includingits spatialandtemporalpatterns [64].Model uncertaintyis groundedinmodel assumptions, whichunderpinthechoiceofmodelalgorithm,structure,andparameterization[65].Uncer- tainty also varies spatially across a species’ predicted habitat [66], spreads through the multiple phases ofmodel development (e.g., inhierarchical, multistage models), and has multiplicativeeffects,suchthatitsmagnituderemainsgenerallyunderappreciated[64].These aresignificantchallenges,whichpossiblyexplainthescarcityofattemptstoaccountjointlyfor multipletypesofvariation(but see[29,66]).Forthisreason,clearprotocolsfor measuring, accountingfor,andreportingonuncertaintyremainlargelylacking.Thelatteroftenrelatesto themodel’sintendedpurpose,suchthatquantifyingparameteruncertaintymightbeapriority whenseekinginferenceaboutagivenpredictor,butpredictionuncertaintywillgainimpor- tancewhentheprimaryobjectiveismodeltransfer.Modelaveragingcanhelp,thoughitis importanttochooseamodelaveragingmethodthatadequatelypreservestheuncertaintyof thecombinedprediction[64].Recentadvancesinhierarchicalmodelingallowerrorestimates topropagatethroughvarioussubmodelswithinone‘integratedstatisticalpipeline’,andcould offerasolutioninsomecases.

HowCanWeBestTransferModelsthroughTimeandEvaluateTheminTemporally DynamicSystems?

Allecologicalsystemsexhibittemporalvariability,whetherpredictable(e.g.,tides),systematic (e.g.,gradualclimatewarming),orrandom(e.g.,cyclones).Constructingmodelsusingthefull span(diurnal,seasonal,phenological,andannual)ofconditionsunderwhichtheywilllikelybe appliedcanaddressthisvariation,althoughdistinguishingerroneouspredictionsfromtempo- rallystochasticeventsinmodelvalidationsremainsachallenge.Timeseriesofenvironmental variationcouldhelpdiagnoseanomalousconditionsfallingoutsidethebaselinecharacteristics ofreferenceandtargetsystems.Studiessuggestthatsomemodelscanprojectmorereliably over centuries [67] than shorter [68] or longer [69] time scales. A fundamental issue for

forecastingisthattemporaltransfersareoftenimpossibletovalidatebecausefutureeventsare unknown.Onesolutionistoevaluatepredictionsofpastevents(i.e.,hindcasting)basedon independent historical (e.g., harvest and museum records) or paleoecological datasets, althoughspatio-temporal,collector’s,andtaphonomicbiaseswillcomplicatemodelcalibration andvalidation[70].However,formanyspeciesofmanagementinterest,suchrecordsremain unavailable orundermined by issues of spatial or temporal bias, mismatching resolutions betweenpastandpresentdata,anderrorpropagation[71].Samplingtheresponsevariable acrossitsrangeofhabitatvariabilityoffersanalternative.Thisstrategyembodiestheprincipleof

‘space-for-time substitution’, which assumes that spatial heterogeneity across multiple

Transferability Good Mixed Limited/poor

10

3 9

1 20

4 14

17 8

5

5

16 6

6 1

11 12 13 14 15 16 17 18 19 20

2 3 4 7 8 9 10

2

13

18 7

15 11 19 12

Figure1.SnapshotofSomePredictiveModelTransfersPublishedintheEcologicalLiterature.Itiscommonly assumedthattransferabilitydegradesawayfromtheareaofcalibration.However,geographicseparationisoftenapoor predictorofenvironmentalsimilarity,suchthat even modelstransferredovershort distancescan yielderroneous predictionsifconditionsbetweenreferenceandtargetsitessubstantiallydiffer.Thisparadoxisreflectedinthecontrasting performanceofmodelsprojectedoverarangeofdistances.Here,weshowcasestudiesselectedtocaptureabroadrange oftaxa(e.g., birds,mammals,plants),ecosystems(terrestrial,freshwater,marine),locations(e.g., Australia,North America,Eurasia),andtransferdistances(tenstothousandsofkm).Colorsindicatewhethertransferswereconsidered successful(green,unbrokenlines;1–9)ornot(darkred,dashedlines;16–20),asreportedbytheauthorsandirrespective ofthestatisticalmethodschosentobuildthemodelsorthemetricsusedtoevaluatethem.Dualcolorsindicatescenariosin whichthequalityoftransfersvariedasafunctionofmodelingalgorithms(10–11),space(12),orspecies(13–15).Line thicknessisproportionaltothenumberofmodeledspecies.Referenceandtargetsystemsareshownasfilledandopen circles,respectively.Notethat,forclarity,notallindividualmodeltransfersareportrayedforeachstudy.Photographs depictmodelorganismsandinclude:(1)Eurasianbadger,Melesmeles;(2)smoothcrotalaria,Crotalariapallida;(3) Norwegianlobster,Nephropsnorvegicus;(4)garlicmustard,Alliariapetiolata;(5)invasiveseaweed,Caulerpacylindracea;

(6)bluestripesnapper,Lutjanuskasmira;(7)spinywaterflea,Bythotrepheslongimanus;(8)Bengalflorican,Houbaropsis bengalensis;(9)northernpike,Esoxlucius;(10)marbledmurrelet,Brachyramphusmarmoratus;(11)yellow-billedcuckoo, Coccyzusamericanus;(12)bluewhale,Balaenopteramusculus;(13)bronzedungbeetle,Onitisalexis;(14)daisyfleabane, Erigeronannuus;(15)rainbowdarter, Etheostomacaeruleum;(16)koala,Phascolarctoscinereus;(17)Asiantiger mosquito,Aedesalbopictus;(18)greypetrel,Procellariacinerea;(19)black-backedwoodpecker,Picoidesarcticus;

and(20)commontoad,Bufobufo.Referencesandadditionaldetailsaregiveninthesupplementarymaterialonline.

contemporaneoussitesatdifferentpositionsalonganenvironmentalgradientcanapproximate temporalvariability[72].Suchwouldbethecase,forexample,forareassubjecttotemperature regimessimilartothoseanticipatedinthefuture,notingitwillnotbeappropriateforspecies occupyingsmallrangesorthosenotwell-representedinthefossilrecord.

HowShouldTransferabilityBeAssessed?

Assessmentsof transferabilitydemandappropriatediagnosticsof predictionaccuracy and precision[73],yetthereisstilllittleconsensusonwhichmetricsaremostappropriate[6,74].All else being equal, true validation is possible only with independent data, which are often

Fundamental challenges Modelling workflow Technical challenges

Is model transferability trait- or taxon-specific?

Which response variables make models more or

less transferable?

How can non-analog condions be accounted for when transferring models?

How can nonstaonarity and interacons be incorporated in model

transfers?

Do specific modeling approaches result in beer transferability?

How should uncertainty be quanfied, propagated,

and communicated when transferring a model?

How can we best transfer models through me and evaluate them in temporally dynamic

systems?

How should transferability be assessed?

To what extent does data quality affect model transferability?

How can sampling be opmized to maximize model transferability?

How does model complexity influence model transferability?

Are there spaal and temporal limits to extrapolaon in model

transfers?

Stage 1

DataModelsPredicons

Focal taxa and study objecves

Predictor variable(s)

Model choice and implementaon

Model calibraon and validaon

Model applicaon Response variable(s)

Organisms: terrestrial, marine, freshwater ...

Preparaon: manipulaon of environmental layers, e.g., standardisaon, geographic projecon ...

Exploraon: collinearity, spaal & temporal coverage, quality and resoluon, outliers, transformaons ...

Class: correlave, mechanisc, hybrid ...

Scheme: internal cross-validaon versus external tesng on independent datasets ...

Predicon: project model across space, me, taxa (into novel condions) ...

Visualizaon: map (or plot) model outputs and associated measure(s) of confidence ...

Performance: correlaon score, coefficient of determinaon, specificity, sensivity, AUC ...

Algorithm: GLM/GAM, MaxEnt, CLIMEX, FATE-HD ...

Ensemble: single model, mulmodel average ...

Common goals: conservaon planning, impact assessment, theorecal ecology, niche evoluon ...

Type: presence, absence, abundance ...

Species traits: dispersal, physiological tolerance ...

Stage 2

Stage 3

Stage 4

Stage 5

Stage 6

Figure2.OutstandingChallengesintheTransferabilityofEcologicalModels,andtheirRelevancetotheModelingWorkflow(AdaptedfromFigure7 inRobinsonetal.[12]).Challengeswereidentifiedbyaconsortiumof50experts.Eachdirectlyinfluences,orisinfluencedby,oneormorestagesinthemodel constructionprocess,fromthecollectionandpreparationofdata,tothechoiceofmodelalgorithmsandtheircalibration,validation,andapplication.Linkagesarenot intendedtobecomprehensive,butrathertocapturetheintegralrolethattransferabilityplaysasanelementofecologicalmodelingpractice.Ultimately,fundamentaland technicalchallengesareinterrelatedincomplexways,suchthataddressingonemaybenecessaryfor,and/orhaveknock-oneffectson,ourabilitytoaddressany others.Thebestwaytoimproveourunderstandingofchallengesaroundmodeltransferabilityandenhancepredictiveperformanceistousepredictionsastoolsfor learning,hencethemodelingworkflowisalooprepresentingtheongoingprocessoflearningbydoing.AUC,Areaunderthecurveofthereceiveroperating characteristic;CLIMEX,amechanisticmodelofspeciesresponsestoclimatechange;FATE-HD,adynamiclandscapevegetationmodelthatsimulatesinteractions betweenplantspecies,whilstaccountingforexternaldriverssuchasdisturbanceregimesandenvironmentalvariations;GAM,generalizedadditivemodel;GLM, generalizedlinearmodel.