Errors and uncertainties

In the scientific domain, according to the Joint Committee for Guides in Metrology (JCGM, 2010), the uncertainty is a parameter related to the result of a measurement that describes the spread of values that can be reasonably assigned to the measurement. Other authors complement this

definition as the range in which results are expected, including the probability with which this range was obtained (Kallner, 2001; Taylor, 1997).

The method to analyze the uncertainties associated with estimation is the propagation of

uncertainties (JCGM, 2010; Taylor, 1997). This method divides the problem into stages, quantifies the uncertainties separately, and then combines them to get the total uncertainty (Kallner, 2001).

The division into stages is due to the identification of the sources of uncertainty that, presumably or with information from previous studies, contribute most to the total uncertainty (Schmid & Lazos Martínez, 2000). The quantification of uncertainty usually embraces the assignment of value plus its distribution (Schmid & Lazos Martínez, 2000).

I.3.2 Reporting uncertainty

Ascough et al. (2008) point out the importance of reporting uncertainty in all types of empirical studies in the environmental and ecological context, emphasizing the relationship between understanding the uncertainty with the quality of decision-making. In the forestry context, Kauffman et al. (2013) mentioned that by including uncertainty analysis in aboveground biomass (AGB) estimation, reference was made to the precision of the reported information. According to the Global Terrestrial Observing System and Food and Agriculture Organization (GTOS & FAO, 2009), this refers then also to the reliability of the information.

Research on forest biomass, including uncertainty analysis has increased, given the development of government policies and international negotiations about forest response and climate change (Shi &

Liu, 2017). As a national strategy, the Mexican government has promoted the development of public policy instruments that consider strategies in the economic/climate sectors (SEMARNAT &

SHCP, 2009), and their relationship with the technical parameters in forest emissions (CONAFOR, 2017b). These policy instruments contain clear methodologies of uncertainty analysis and include uncertainty estimates in the results to be obtained (CONAFOR, 2014b).

I.3.3 Errors in National forest inventories

Different measurement methods are used in NFIs to get observations to record the current state of

accuracy of the measurements while reducing the acquisition time (Diéguez Aranda et al., 2005;

Kershaw Jr., Ducey, Beers, & Husch, 2017). Therefore, data collected from NFIs is objectively prone to error. Across this thesis work, the word "error" will not be used as a synonym for

“mistake” or “carelessness” (Gil & Rodríguez, 2001) rather as the uncertainty of

measurement/estimation (Taylor, 1997). As before defined Section I.3.1, an error will be considered as the residual variability associated with the measurement or estimation, thus describing the dispersion values logically attributed to AGB measurement (JCGM, 2010).

The total error of estimation in NFIs involves different components, including sampling and non-sampling error (FAO, 1981; Kleinn et al., 2015; United Nations, 2008). The final report of the NFI in Mexico, for example, included sampling error as the source of all observed variations in the variables considered (number of trees, basal area, volume, biomass, etc.) without reporting non-sampling errors (CONAFOR, 2012b, 2017a).

I.3.4 Sampling error

NFI plots sample the landscape to measure variables of interest, from which the parameters of the target population are estimated (Köhl et al., 2006). However, the estimators are subject to error due to the sampling design applied (Kershaw Jr. et al., 2017), meaning that the error would not be present if the entire population was included in the observations (Gormanson et al., 2017;

McRoberts et al., 2015). This error is referred to as sampling error. The sampling error of

probability samples is reported as the standard error of the mean (SE), coefficient of variation (CV) or the confidence interval (Köhl et al., 2006; United Nations, 2008), of a given variable e.g. volume, AGB, etc. (FAO, 1981; Köhl et al., 2006; McRoberts, Næsset, et al., 2015). Considering that SE measures the precision of the estimate, sampling error is related to the sample size and is therefore intrinsically associated with the time spent doing fieldwork and budgets allocated to the inventory (Kershaw Jr. et al., 2017; United Nations, 2008).

McRoberts et al. (2015) show the use of the CV to be effective when comparing across sampling designs; whereby the differences between sampling designs were given by sample size and inter-plot distances. Similarly, an optimal sampling error is defined by the smallest SE per sampling design given the costs assigned to the inventory (United Nations, 2008). Tomppo et al. (2010) compared 31 European countries, 3 Asian (China, Japan, and Republic of Korea), 3 on the

American continent (Brazil, Canada, and USA) and New Zealand from 1992 to 2009 and reported NFI sampling errors for wood volume by SE ranging from 0.46% (USA) to 7.14% (Ireland).

However, in this comparison, there were inconsistencies in the definition of wood volume as the height of DBH (1.3 - 1.5 m), minimum DBH (0 - 12.5 cm), elements sampled other than standing trees (stumps, branches and/or dead wood), among others. More recent results to those reported by Tomppo et al. (2010) can be accessed online in most countries, and show an improvement in the estimation precision of wood volume with SE of 0.31% in the USA and 2.17% Ireland

(https://www.fia.fs.fed.us/; https://www.agriculture.gov.ie/nfi/). In Mexico, the first repeat survey of the NFI (2009-2014), reported for volume a SE of 3.2 - 4 % (CONAFOR, 2017a). These results are consistent with those obtained in the first NFI (2004-2009) where the volume was estimated with 3.2 - 4 % of SE (CONAFOR, 2012c). The AGB was reported only for the temperate forest in NFI (2009-2014) with 2.6 - 3.3 % of SE.

I.3.5 Non-sampling errors: measurement errors

The goals established in an NFI, such as timber supply, biodiversity, REDD+, etc., determine the variables measured during fieldwork (Kleinn, 2017; Kleinn et al., 2015). Trees are the object, where the measurements are made and the values of the variables of interest registered. Tree

measurements are made assuming geometric forms like the tree cross-section (circle, oval), tree form (cone, frustum cone), or tree crown (circle, oval) (Kershaw Jr. et al., 2017; Matérn, 1956) and thus inherently carry an error in their magnitudes. Thus, it is important to estimate measurement uncertainty, so as to determine the quality of the measurement. Such a result can be the source of information for another project or for a decision-making process (Pérez-Hernández, 2012).

There are two important components of measurement error, systematic and random errors (Taylor, 1997; United Nations, 2008). Both systematic and random measurement errors are independent of each other and hence should be quantified independently. Figure I-3 shows that the total error can be quantified as the hypotenuse, of the Pythagoras' theorem, joining both error axes. Considering that it is not possible to avoid random error in any measurement (Taylor, 1997), one can posit that smaller total error can be achieved, when systematic error reduces and as systematic error tends to zero, the total error equals to the random error.

Systematic errors

Total error

Theorem of Pythagoras a² + b² = c² Random errors

Figure I–3. Total error in the measurement as a product of systematic errors and random errors.

Modified from source (United Nations, 2008).

Practically, measurement errors arise from the faulty or incorrect use of measurement devices e.g.

from device calibration (Diéguez Aranda et al., 2005) or dependent on the accuracy of the measurement devices (Gil & Rodríguez, 2001). Incorrect use of measurement devices by staff is often related to either measurement criteria or staff capacity use the availed forest inventory devices (Canavan & Hann, 2014; Diéguez Aranda et al., 2005).

I.3.6 Non-sampling errors: prediction errors due to allometric models

The allometric model to estimate the AGB is obtained from regression analysis (Picard et al., 2012).

The AGB is the result of statistical relationship with tree variables such as DBH (Avendaño

Hernandez et al., 2009; Návar, 2009), TH (Foroughbakhch et al., 2006; Vargas-Larreta et al., 2017), crown diameter (Návar et al., 2004), wood density (Martinez-Yrizar et al., 1992; Wiemann &

Williamson, 2013). This model can be applied to other standing trees located in the same site, where the model was obtained, or in areas with similar site-characteristics (GTOS & FAO, 2009); or applying a scientific approach as a suitability check of the model validating the prediction

uncertainty precision to select a AGB model (Pérez-Cruzado et al., 2015). Then, site-characteristics

because it is related to climatic conditions or soil types and these are associated with tree growth and with accumulation of AGB (GTOS & FAO, 2009; Picard et al., 2012; Shi & Liu, 2017).

As a result of the regression model, statistics that represent the adjustment of the equation for the AGB and the uncertainty of the model, based on the tree measurement variable(s) are obtained (GTOS & FAO, 2009). The most commonly used indicator is the coefficient of determination (R²), representing the quality of the model to be replicated and the proportion of variation of the results that can be explained by the model (Ayala Gallego, 2015; Mehtätalo, 2013). Mean square error (MSE) is a measure of the difference between the estimator and what is estimated. The square root of the MSE (RMSE), is the parameter of precision associated with the model assuming a constant variance of the error (Cochran, 1977). Two more indicators are obtained from the regression analysis, the error in prediction of the mean used to estimate the confidence interval of the

regression, and the error in prediction of an individual used to estimate the prediction interval of the regression (Draper & Smith, 1998). Only 40 of 478 studies in AGB allometric equations for

Mexican trees forest, reported the parameters related to the uncertainty: RMSE or SE (Rojas-García et al., 2015a).

The allometric models of AGB are generated under different criteria related to the stand

characteristics, geographic area and the delimitation of classes according to the size range of the trees to be characterized. Based on stand characteristics, the allometric model could be made for species specific (Vargas-Larreta et al., 2017), genus (Méndez González et al., 2012) or group of species (Búrquez et al., 2010). The geographic area criterion is related to allometric models

generated with information of one stand (Shi & Liu, 2017), a group of stands (Méndez González et al., 2012), a location (Návar et al., 2004) or a region (Shi & Liu, 2017; Vargas-Larreta et al., 2017);

considering those areas are referred to the physiographic conditions of the area (Shi & Liu, 2017).

The models have a range of validity whose extreme values are the minimum and maximum values of the variables used when calculating the model; if we use the model to predict AGB outside this range of values, estimation biases may occur (Picard et al., 2012).

I.3.7 Quality control of data

The quality control of data is due to the handling of data records in the field (United Nations, 2008), transfer of field forms observations to electronic media (Kershaw Jr. et al., 2017), and the statistical process of data in computer programs (Canavan & Hann, 2014). Therefore, it is important to implement verification mechanisms at each step involving data transfer, to have a reliable database for information analysis (United Nations, 2008). With an emphasis on the proper training of the work teams in the process of collection and storage of field information (FAO, 1981; United Nations, 2008).

To ensure data quality, NFI's have been implementing protocols to assess the quality of information recorded (Tomppo et al., 2010). The protocols include data electronic storage, double review in fieldwork, plausibility checking (included in the storage equipment), automatic verification on the central server (logical check) and verification of 5-10% of the sampled plots.