• Keine Ergebnisse gefunden

A bibliometric analysis of the model validation literature

N/A
N/A
Protected

Academic year: 2022

Aktie "A bibliometric analysis of the model validation literature"

Copied!
1
0
0

Wird geladen.... (Jetzt Volltext ansehen)

Volltext

(1)

Model validation

• examines to what extent validation (and specific validation approaches) is acknowledged and adopted,

• investigates how the validation practices in different modelling fields are related,

• employs a combination of citation and text-mining analyses on a dataset of 10688 academic publications.*

This study…

References

1.Oreskes, N., Shrader-Frechette, K., Belitz, K., 1994. Verification, validation, and confirmation of numerical models in the earth sciences. Science

263(5147) 641-646.Jakeman

2.Jakeman, A.J., Letcher, R.A., Norton, J.P., 2006. Ten iterative steps in development and evaluation of environmental models. Environmental Modelling & Software 21(5) 602-614.

3.Bennett, N.D., Croke, B.F.W., Guariso, G., Guillaume, J.H.A., Hamilton, S.H., Jakeman, A.J., Marsili-Libelli, S., Newham, L.T.H., Norton, J.P., Perrin, C.,

Pierce, S.A., Robson, B., Seppelt, R., Voinov, A.A., Fath, B.D., Andreassian, V., 2013. Characterising performance of environmental models. Environmental Modelling & Software 40(Supplement C) 1-20.

4.Maaten, L.v.d., Hinton, G., 2008. Visualizing data using t-SNE. Journal of Machine Learning Research 9(Nov) 2579-2605.

5.Blei, D.M., Ng, A.Y., Jordan, M.I., 2003. Latent dirichlet allocation. Journal of Machine Learning Research 3(Jan) 993-1022.

A bibliometric analysis of the model validation literature

Sibel Eker, Elena Rovenskaya, Simon Langan, Michael Obersteiner

International Institute for Applied Systems Analysis (IIASA) Laxenburg, Austria

eker@iiasa.ac.at

@sibel_eker_

• Validation is a crucial step in quantitative modelling to establish confidence and reliability.

• However, it is often said that validation approaches proposed in the literature are not widely adopted by practitioners,

• and the validation approaches in different modeling fields do not benefit from each other.

METHODS

* The dataset is retrieved from Scopus with keywords “model validation, evaluation,

assessment and testing” and limited to the

disciplines such as environmental science, decision sciences, economics, energy, computer and social sciences.

** Using the t-SNE algorithm [4], which reduces the dimensions of multi-dimensional data points

(articles) and builds a 2D map where the distances between the points depend on the word similarities in their abstract.

*** Topic modelling algorithm Latent Dirichlet Allocation (LDA) [5] allocates each document to one of the predefined number of bags to a certain extent, forming document-topic and topic-word

pairs.

How related are these publications in terms of their

content? How does this relatedness reflect on citation scores

as an indicator of uptake?

Can this content relatedness be explained by

different topics? Do the publications from different fields cite each

other?

The word content of the four topics

Based on nonlinear mapping** a large dense (dark) region contains many

similar articles. Well- known articles are in the periphery, not

very similar to the

others, implying their content might not be widely adopted.

The most-cited articles are not necessarily in the densest regions.

Instead, they are rather in the periphery of clusters, which can be considered different and more innovative.

Among the four topics

identified***,

Ecosystems

is relatively distinctive,

implying a different content of validation

articles.

Emissions and Energy

is dispersed,

most similar to the

Methods

topic.

The articles in each topic cite the articles in the same topic most, indicating that the validation literatures of these modeling areas are closed to each other.

The

Methods

topic is the most-cited.

KEY FINDINGS

Well-known articles proposing different validation approaches have a different content than most publications.

The most-cited publications are not similar to the rest in terms of their content.

Different modeling fields are closed to each other’s validation practice.

Referenzen

ÄHNLICHE DOKUMENTE

The aim of the present study is to build a simplified model for active vibration control with sampling delay of a suspended cable exposed to periodic excitation, and to investigate

After having presented the simulation algorithm for DEPFET pixel modules and the data reconstruction for charged particles in the EUDET beam telescope, the final part of this

Box plot, PCA and density plot are different ways to visualize the distribution of data points in the individual samples, see also lecture #2 slide 21.. In the case shown here,

To quantify the eye volume, a package of commercial image processing software (Disect Systems Ltd., UK) was applied to analyse the data received by the detected density

Figure 4: Map of the validation publications and their citation scores: (a) According to the total number of

( 1993) concluded that the recov- ery of core from Site 806 was incomplete and that there were missing segments. In particular, by comparing the data with data from other sites,

They proposed a method of discovering missing links in Wikipedia pages via a clustering approach.The clustering process is performed by grouping topically related pages using LTRank

™ Alternatives to oil as an energy source, keep oil price tied to other energy prices. ™ Supply shortages do not justify enforced and rapid switch to renewables,