Absence of Quantitative Work

many variables. This inability, both as a result the author’s limitations and the project’s nature, to empirically test the theories developed here might reduce this dissertation’s academic value.

Apart from the hindrance that empirical testing is difficult in this case, the author would have encountered a more serious problem had it been possible: acquiring data. In order to measure the effects of treaties on the level of human rights protection a state affords its citizens, econometric research relies heavily on

‘human rights indicators’.⁸⁵ Such indicators are used to assist in the analysis of changes over time, but are problematic because they often ‘have a striking resemblance to the standard indicators of socio-economic progress’,⁸⁶ which makes it difficult to determine whether it is a change in socioeconomic or human rights conditions that is being analysed.

Aside from this uncertainty as to the identity of the dependent variable, the indicators used in such approaches are expected to be ‘SMART’: specific, measurable, attainable, relevant, and time-framed.⁸⁷ In attempting to fulfil all of these criteria, and in attempting to achieve progress toward more empirical work on the impact of international law, the crucial difficulty is that such analysis is burdened ‘by a shortage of good reliable data’.⁸⁸ Acquiring data in relation to human rights issues is difficult for countless reasons: governments may be cautious about handing over data to international bodies, NGOs, or academic institutions for fear of being ‘outed’, while a further limitation will be the fact that many abuses take place behind closed doors.⁸⁹ In addition, the difficulty in                                                                                                                

85 ‘Technically speaking, an indicator refers to a set of statistics that can serve as a proxy or a metaphor for phenomena that are not directly measurable. However, the term is often used less precisely to mean any data pertaining to the social condition’, in Maria Green, ‘What We Talk About When We Talk About Indicators: Current Approaches to Human Rights Measurement’, [2001] 23 Human Rights Quarterly 1062, 1076.

86 UN Office of the High Commissioner for Human Rights, ‘Draft Guidelines: A Human Rights Approach to Poverty Reduction Strategies’, (10 September 2002)

<http://www.refworld.org/docid/3f8298544.html> accessed 9 September 2013.

87 This is discussed in Erik Andre Andersen and Hans-Otto Sano, ‘Human Rights Indicators At Programme and Project Level: Guidelines for Defining Indicators Monitoring and Evaluation’, (Danish Institute for Human Rights, 2006), 11-13.

88 Oona A Hathaway and Ariel N Lavinbuk, ‘Book Review: Rationalism and Revisionism in International Law’ [2006] 119 Harvard Law Review 1404, 1441.

89 These various influences are broadly discussed in Herbert F Spirer, ‘Violations of Human Rights. How Many? The Statistical Problems of Measuring Such Infractions Are Tough, but

acquiring data means that quantitatively assessing human rights violations and standards is likely to be troubled by ethical issues in its methodology.⁹⁰

These very practical output-oriented hindrances should also be read in light of the input-oriented SMART criteria. Firstly, achieving specificity as to the level of human rights violations is problematic for numerous reasons: victims of violations may not report incidents of rights violation to specific state institutions out of fear of reprisals or a lack of faith in the national system; those incidents that are reported to one state institution and which relate to another state institution may not be recorded in national statistics; in some cases, victims of violations may die, making self-reporting impossible; victims may be ignorant of a violation having taken place, making it evidently implausible that it will be reported; victims of violations may be geographically removed from the state institutions to which reports of violations ought to be submitted, making reporting difficult.

Equally, measuring violations and acquiring the requisite data from states is also difficult, as those institutions or non-governmental organisations that might be interested in measuring rights violations might suffer from a lack of expertise in collecting data or might be subject to resource – or bureaucratic⁹¹ – constraints, while the rights subject to measurement might simply be immeasurable.

In addition, and stemming from the aforementioned factors, the requirement that human rights indicators be time-framed also runs into difficulties. Often, it might be the case that data is acquired long after a violation has taken place, meaning that analysing rights violations or changes in a state’s adherence to its legal obligations will require time-lagging the variables. Further, if the relevant data                                                                                                                                                                                                                                                                                                                                            

Statistical Science Is Equal to It’ [1990] 49(2) American Journal of Economics and Sociology 199;

James M McCormick and Neil J Mitchell, ‘Human Rights Violations, Umbrella Concepts, and Empirical Analysis’ [1997] 49(4) World Politics, 510.

90 For a good overview of the ethical issues involved in such work, see Craig K Enders and Amanda C Gottschall, ‘The Impact of Missing Data on the Ethical Quality of a Research Study’ in AT Panter and Sonya K Sterba (eds), Handbook of Ethics in Quantitative Methodology (1^st edn, Routledge 2011); Michael Stohl, David Carleton, George Lopez, and Stephen Samuels, ‘States Violations of Human Rights: Issues and Problems of Measurement’ [1986] 8 Human Rights Quarterly 592.

91 For example, hierarchies within these institutions may have determined that the collection of data on rights violations is not a priority.

can only be acquired a significant period after the violation has taken place, then any analysis that is carried out in the period between the violations themselves and the acquisition of the data relating to the violations may undervalue the level of violations present.

Resolving these issues requires a systematic shift in the collection of data relating to human rights issues. As mentioned, distinguishing between strictly human rights indicators and measurements of socio-economic change can be difficult. Both allude to changes in the examined society but whereas the former might be indicative of a breach of a legally mandated obligation, the latter might not. Furthermore, an additional challenge will be processing the raw data into quantitative ‘scores’ for inter-state comparison and for measurement against legal obligations. Human rights law has not been structured in a way that easily facilitates the quantification of violations, an argument best exemplified by the ambiguity of the United Nations Convention against Torture’s (UNCAT) definition of torture as:

any act by which severe pain or suffering, whether physical or mental, is intentionally inflicted on a person for such purposes as obtaining from him or a third person information or a confession, punishing him for an act he or a third person has committed or is suspected of having committed, or intimidating or coercing him or a third person, or for any reason based on discrimination of any kind, when such pain or suffering is inflicted by or at the instigation of or with the consent or acquiescence of a public official or other person acting in an official capacity. It does not include pain or suffering arising only from, inherent in or incidental to lawful sanctions.⁹² The intricacies of this description would require tediously analysing each and every alleged and actual incident falling under the Article’s remit, and meticulously ascribing a score to each incident. On top of this, accounting for the psychological effects of torture on victims ‘makes this concept largely subjective’,⁹³ and accordingly results in greater difficulties when quantifying. The

92 United Nations Convention against Torture and Other Cruel, Inhuman or Degrading Treatment or Punishment, (adopted 10 December 1984, entered into force 26 June 1984) 1465 UNTS 85 (UNCAT) art 1.1.

93 Robert J Goldstein, ‘The Limitations of Using Quantitative Data in Studying Human Rights Abuses’ [1986] 8 Human Rights Quarterly 607, 610.

unmeasurable nature of psychological harm stemming from torture⁹⁴ is likely to mean that scores based on these qualitative sources, and which measure physical harm in a numerical sense,⁹⁵ will be uncertain.

The quantitative findings of other authors were used in the early stage of this research project to better understand the context in which the research would be conducted. Quite a lot of the data used in the statistical and econometric work cited in this dissertation is based on a database of human rights scores held by Cingranelli and Richards.⁹⁶ This, in turn, is a composite rating of annual human rights standards compiled by Amnesty International and the US State Department. While this database is an invaluable resource for scholars employing econometric approaches, the origins of the data in US State Department country scores may weaken the database’s reliability, as those scores are likely to reflect the United States’ particular global outlook. During the Cold War, for example, the data is likely to have been biased against communist states, irrespective of the actual human rights standards that may have existed in those states; accordingly, using these scores as a means of constructing a composite score is likely to have affected the reported statistical findings.⁹⁷ Readers are therefore discouraged from assuming that the econometric work cited in this dissertation is absolutely reliable in either its input or output.

94 Derrick Silove, ‘The Psychosocial Effects of Torture, Mass Human Rights Violations, and Refugee Trauma: Toward an Integrated Conceptual Framework’ [1999] 187(4) Journal of Nervous

& Mental Disease 200.

95 See, for example, David L Cingranelli and David L Richards, ‘Measuring the Level, Pattern, and Sequence of Government Respect for Physical Integrity Rights’ [1999] 43(2) International Studies Quarterly 407.

96 David L Cingranelli and David L Richards, ‘The Cingranelli and Richards (CIRI) Human Rights Data Project’ [2010] 32(2) Human Rights Quarterly 401; the project and the data can be found at:

The CIRI Human Rights Data Project; available at <http://ciri.binghamton.edu/> accessed 23 May 2011

97 Ryan Goodman and Derek Jinks, ‘Measuring the Effects of Human Rights Treaties’ [2003] 14(1) European Journal of International Law 171, 179.