The Global Innovation Index (GII) is a cross-country performance assess-ment, compiled on an annual basis, which continuously seeks to update and improve the way innovation is measured. The GII report pays special attention to making accessible the sta-tistics used in the Country/Economy Profiles and Data Tables, providing data sources and definitions, and detailing the computation method-ology (Appendices I, II, III, and IV, respectively). This annex summarizes the changes made this year and pro-vides an assessment of the impact of these changes on the comparability of rankings.
Adjustments to the Global Innovation Index framework
The GII model is revised every year in a transparent exercise. This year, no change was made at either the pillar or the sub-pillar level.
Beyond the use of World Intellectual Property Organization (WIPO) data, we collaborate with both public international bodies such as the International Energy Agency;
the United Nations Educational, Scientific and Cultural Organization (UNESCO); the United Nations Industrial Development Organization (UNIDO); the International Telecommunication Union (ITU);
and the Joint Research Centre of the European Commission (JRC) as well as with private organizations such as
the International Organization for Standardization (ISO); IHS Global Insight; QS Quacquarelli Symonds Ltd; Bureau van Dijk (BvD);
ZookNIC Inc; and Google to obtain the best available data on innovation measurement globally.
Table 1 provides a summary of adjustments to the GII 2017 frame-work for quick reference. A total of five indicators were modified this year: one indicator was removed, one indicator changed its number as a result, and three indicators under-went methodological and name changes. Indicators that retained the same name as last year but are derived from a source that changed
its methodology are not identified in Table 1.
The statistical audit performed by the JRC (see Annex 3) pro-vides a confidence interval for each ranking following a robustness and uncertainty analysis of the modelling assumptions.
Sources of changes in the rankings The GII compares the performance of national innovation systems across economies, and it also presents changes in economy rankings over time.
Importantly, scores and rankings from one year to the next are not Table 1: Changes to the Global Innovation Index framework
GII 2016 Adjustment GII 2017
4.2.3 Total value of stocks traded,
% GDP Removed
4.2.4 Venture capital deals/bn
PPP$ GDP Number changed 4.2.3 Venture capital deals/bn
PPP$ GDP 5.3.4 Foreign direct investment net
inflows Name and
methodology changed 5.3.4 Foreign direct investment net inflows (3-year avg.) 6.3.4 Foreign direct investment net
outflows Name and
methodology changed 6.3.4 Foreign direct investment net outflows (3-year avg.) 7.3.3 Wikipedia monthly edits Name and
methodology changed 7.3.3 Wikipedia yearly edits Note: Refer to Annex 1 and Appendix III for a detailed explanation of terminologies. Indicators whose name did not change but methodology at the source did
are not part of this list. Refer to Appendix III for a detailed explanation of methodological changes at the source.
Annex 2: Adjustments and Year-on-Year Comparabilty direct investment are now being mea-sured as an average of the most recent three years to produce a more stable reflection of these indicators’ datasets.
The underlying methodology for indicator 7.3.3 has also changed; it now measures edits within each econ-omy by year rather than by month.
Missing values
Since its inception, the GII has had a positive inf luence on data avail-ability, increasing awareness of the importance of submitting timely data.
The number of data points submitted by economies to international data agencies has substantially increased in recent years. In the GII 2016, 12.8%
of data points were missing; this year, in the GII 2017, coverage improved again, with only 10.3% of data points missing.
When it comes to country cover-age, the objective is to include as many economies as possible. However, it is also important to maintain a good level of data coverage within each of these economies. Because the GII results are linked to data availability (see the JRC Statistical Audit pre-sented in Annex 3 for more details), which affects the overall GII ranks, this year the minimum data coverage threshold rule was strengthened—on the recommendation of the JRC—to maintain the significance of both the GII results and the country sample.
To be included in the GII 2017, an economy must have a minimum symmetric data coverage of 36 indi-cators in the Innovation Input Sub-Index (66%) and 18 indicators in the Innovation Output Sub-Index (66%), and it must have scores for at least two sub-pillars per pillar. Missing values are indicated with ‘n/a’ and are not considered in the sub-pillar score.
This adjustment derives from a sensitivity that is the result of the data availability, which is less satisfactory directly comparable (see Annex 2 of
the GII 2013 for a full explanation).
Making inferences about absolute or relative performance on the basis of year-on-year differences in rankings can be misleading. Each ranking reflects the relative positioning of that particular country/economy on the basis of the conceptual framework, the data coverage, and the sample of economies—elements that change from one year to another.
A few particular factors inf lu-ence the year-on-year ranking of a country/economy:
• the actual performance of the economy in question;
• adjustments made to the GII framework;
• data updates, the treatment of outliers, and missing values; and
• the inclusion or exclusion of countr ies/econom ies in the sample.
Additionally, the following char-acteristics complicate the time-series analysis based on simple GII scores or rankings:
• Missing values. The GII pro-duces relative index scores, which means that a missing value for one economy affects the index score of other economies.
Because the number of missing values decreases every year, this problem is reduced over time.
• R eference yea r. The d at a underlying the GII do not refer to a single year but to several years, depending on the latest available year for any given vari-able. In addition, the reference years for different variables are not the same for each economy.
The motivation for this approach
is that it widens the set of data points for cross-economy com-parability.
• Normalization factor. Most GII variables are normalized using either GDP or population.
This approach is also intended to enable cross-economy com-parability. Yet, again, year-on-year changes in individual vari-ables may be driven either by the variable’s numerator or by its denominator.
• Consistent data collection.
Finally, measuring year-on-year performance changes relies on the consistent collection of data over time. Changes in the defi-nition of variables or in the data collection process could create movements in the rankings that are unrelated to true perfor-mance.
A detailed economy study based on the GII database and the country/
economy profile over time, coupled with analytical work on grounds that include innovation actors and deci-sion makers, yields the best results in terms of grasping an economy’s inno-vation performance over time as well as possible avenues for improvement.
Methodology and data
The revision of the computation methodology for certain individual indicators has caused shifts in the results for several countries.
For indicator 3.3.1, which mea-sures energy use, the constant PPP$
per kg of oil equivalent was updated from 2005 PPP$ to 2010 PPP$.
The methodology underpinning indicators 4.2.3 and 5.2.4 expanded to use datasets from previous years to improve data coverage.
For indicators 5.3.4 and 6.3.4, the net inf lows and outf lows of foreign
Annex 2: Adjustments and Year-on-Year Comparabilty
Economy Number of missing values
Trinidad and Tobago 25
Togo 23
Burundi 22
Niger 22
Benin 21
Economy Number of missing values
Brunei Darussalam 21
Burkina Faso 20
Guinea 20
Nepal 20
in the case of the Output Sub-Index:
four countries that were part of the GII 2016 have data coverage below the 66% threshold in the 27 variables in the Output Sub-Index. In contrast, data coverage is satisfactory in all of these cases in the Input Sub-Index (all of these economies have indicator coverage of more than 66% over the 54 input variables). As a result, the fol-lowing countries included in the GII 2016 dropped out this year: Bhutan, Ghana, Nicaragua, and the Bolivarian Republic of Venezuela.1 The rules on missing data and the minimum cover-age necessary per sub-pillar will be progressively tightened, leading to the exclusion of countries that fail to meet the desired minimum coverage in any sub-pillar (see Appendix I for more details).
Despite requiring minimum lev-els of coverage, for several economies the number of missing data points remains very high. Table 2 lists the countries that have the highest num-ber of missing data points (20 or more), ranking them according to how many data points are missing.
Conversely, Table 3 lists those economies with the best data cover-age, ranking them according to the least number of missed data points.
These economies are missing at most only five data points; some are miss-ing none at all.
Note
Conversely, Brunei Darussalam, Trinidad and Tobago, and Zimbabwe—which were not included in the GII 2016—enter the GII this year with the required coverage in both sub-indices and sufficient data availability per pillar.
Table 2: GII economies with the most missing values
Economy Number of missing values
Colombia 0
Hungary 0
Mexico 0
Romania 0
Bulgaria 1
Chile 1
Czech Republic 1
Malaysia 1
Poland 1
Russian Federation 1
Turkey 1
Austria 2
Brazil 2
France 2
Italy 2
Japan 2
Korea, Rep. 2
Portugal 2
Slovakia 2
South Africa 2
Thailand 2
Ukraine 2
Australia 3
Belgium 3
Costa Rica 3
Denmark 3
Estonia 3
Finland 3
Germany 3
Indonesia 3
Economy Number of missing values
Israel 3
Kazakhstan 3
Netherlands 3
Serbia 3
Slovenia 3
Spain 3
Sweden 3
Argentina 4
Croatia 4
Egypt 4
Latvia 4
Lithuania 4
Malta 4
Morocco 4
New Zealand 4
Norway 4
Philippines 4
Switzerland 4
Tunisia 4
United Kingdom 4
Cyprus 5
Georgia 5
Greece 5
India 5
Ireland 5
Luxembourg 5
Moldova, Rep. 5
Panama 5
Singapore 5
United States of America 5
Table 3: GII economies with the fewest missing values
Annex 3: JRC Statistical Audit of the GII
Conceptual and practical challenges are inevitable when trying to under-stand and model the fundamentals of innovation at the national level worldwide. In its 10th edition, the 2017 Global Innovation Index (GII) considers these conceptual challenges in Chapter 1 and deals with practical challenges—related to data quality and methodological choices—by grouping country-level data over 127 countries and across 81 indicators into 21 sub-pillars, 7 pillars, 2 sub-indices and, finally, an overall index. This annex offers detailed insights into the practical issues related to the construction of the GII, analysing in depth the sta-tistical soundness of the calculations and assumptions made to arrive at the final index rankings. Statistical soundness should be regarded as a necessary but not sufficient condi-tion for a sound GII, since the cor-relations underpinning the majority of the statistical analyses carried out herein ‘need not necessarily represent the real inf luence of the individual indicators on the phenomenon being measured’.1 Consequently, the devel-opment of the GII must be nurtured by a dynamic iterative dialogue between the principles of statistical and conceptual soundness or, to put it another way, between the theoreti-cal understanding of innovation and the empirical observations of the data underlying the variables.
The European Commission’s Competence Centre on Composite
Indicators and Scoreboards at the Joint Research Centre ( JRC) in Ispra has been invited for the seventh consecutive year to audit the GII.
As in previous editions, the present JRC audit focuses on the statistical soundness of the multi-level structure of the index as well as on the impact of key modelling assumptions on the results.2 The independent statistical assessment of the GII provided by the JRC guarantees the transparency and reliability of the index for both policy makers and other stakeholders, thus facilitating more accurate prior-ity setting and policy formulation in this particular field.
As in past GII reports, the JRC analysis complements the country rankings with confidence intervals for the GII, the Innovation Input Sub-Index, and the Innovation Output Sub-Index in order to better appre-ciate the robustness of these ranks to the computation methodology. In addition, the JRC analysis includes an assessment of the added value of the GII and a measure of distance to the efficient frontier of innovation by using data envelopment analysis.
Conceptual and statistical coherence in the GII framework
An earlier version of the GII model was assessed by the JRC in April–
May 2017. Fine-tuning suggestions were taken into account in the final computation of the rankings in an iterative process with the JRC aimed
at setting the foundation for a bal-anced index. The entire process fol-lowed four steps (see Figure 1).
Step 1: Conceptual consistency
Eighty-one indicators were selected for their relevance to a specific innovation pillar on the basis of the literature review, expert opinion, country coverage, and timeliness. To represent a fair picture of country dif-ferences, indicators were scaled either at the source or by the GII team as appropriate and where needed.
Step 2: Data checks
The most recently released data within the period 2006–16 were used for each economy: 77% of the available data refer to 2015 or more recent years. In past editions, coun-tries were included if data availability was at least 60% across all variables in the GII framework. A more strin-gent criterion was adopted this year, following the JRC recommendation of past GII audits. That is, countries were included if data availability was at least 66% within each of the two sub-indices (i.e., 36 out of 54 vari-ables within the Input Sub-Index and 18 out of the 27 variables in the Output Sub-Index) and at least two of the three sub-pillars in each pillar could be computed. This more strin-gent criterion for a country’s inclu-sion in the GII was introduced this year in order to ensure that country scores for the GII and for the two Input and Output Sub-Indices are