Tis appendix discusses the methods and procedures we used to collect the data and to deal with several challenges of data collection and data management.
1. Detailed Questionnaire Responses
As mentioned in the chapter, we submitted the criteria for identifying major religions and religious families, as well as the candidate religions/religious families, to a panel of experts on global and regional religions. Te results of the survey are given below (Table A4.1.)
Table A.4.2 provides the fnal religion tree used in the WRP.
2. Data Collection and Data Aggregation
We started the data collection project by forming a religion data bibliogra-phy.11 Tese sources varied from census-based data to various estimates of religious groups, or sources that focused on a given religion in a longitudi-nal manner (either within a given country or for several countries). Some of the sources contained multiple data points on global or regional levels, but most contained scattered data on specifc countries at discrete points in time.
Our coding manual refected the kind of issues that we anticipated, given the literature review and a general review of source content and quality.
Te coding instructions presented several dilemmas. First, we had to ensure that denominational level data would be aggregated into the appropriate religious families. Second, in many cases or sources—even census data—
mentioned only the number (or percent) of adherents of major religions but provided no reliable religion family data. In other cases, religion family data existed for only some of the families but not of others. Te coding instructions were not always sufciently specifc to handle the diversity of categories provided by diferent sources; hence, we had to resolve multiple ambiguities in these sources.
Our initial strategy was to collect data from each source on a difer-ent record. We did that even if a given source listed only the number (or percent) of adherents for a single religion. Each data point (or a set of data
points) was identifed by the source from which it was taken, the date of the data, and the date these data were coded within the given source. We ran a number of tests on the data collected from each of the sources (such as consistency over time; source of the data coded in each source, e.g., census, secondary data, etc.; and comprehensiveness of coverage of diferent reli-gions). We distributed a questionnaire among Association of Religion Data Archives (ARDA) members to solicit reliability estimates for each of the sources used. We then ranked sources according to an estimate of reliability.
Before aggregating the data, we had to deal with a number of problems:
1. Adjusting source data to the categories of the religion tree. In quite a few cases, the religious categories reported in a given source were not consistent with our religion tree. Here we had to make deci-sions about which specifcally labeled religious group matched which of the religions or religious families in our religion tree. Tis proved to be a major challenge, especially within the Christian Protestant family. For example, some sources coded Anglicans as Protestants. Other included multiple Protestant denominations, sometimes under diferent labels. A related problem concerned aggregating the various Christian Orthodox denominations under the Eastern Orthodox family. Islamic denominations also pre-sented a signifcant challenge.
2. Eliminating the double counting of religious categories. Several sources double counted religious groups. For example, some sources with data down to the denominational level counted Protestant adherents frst by the number of Protestants and then by Protestant denomination. We had to make sure that the sum of the denominations matched the total number of Protestants.
When discrepancies were observed, specifc adjustments had to be made.
3. Resolving categories such as “doubly afliated” Christian groups (e.g., Protestants and Roman Catholics). Since we view these religious families as mutually exclusive, we had to decide how to allocate doubly afliated adherents into religious families.
4. Addressing adherents of religions not under our religion tree categories.
We had to distinguish between religious groups that were labeled diferently but in practice were within our religion tree, and those
Table A4.1 Survey of major world religions—validity scores N = 67
Question Category Response Response Comments
range (SD)
Importance Scriptures 1-6 5.13 (0.93) Categories eliminated:
of criteria for Institutions 1-6 4.83 (1.12) Holidays identifying major Historical Evolution 1-6 5.06 (0.91)
world religions Beliefs, Practices, Rituals 1-6 5.06 (0.99)
Importance Scriptures 1-6 4.95 (0.94)
of criteria for Institutions 1-6 5.08 (0.86) identifying Historical Evolution 1-6 5.27 (1.02) religious families Beliefs, Practices, Rituals 1-6 5.10 (0.93)
Rank order Scriptures 1-8 2.91 (1.61) 1 = highest rank
of criteria for Institutions 1-8 3.40 (1.55) 5 = lowest rank identifying major Historical Evolution 1-8 3.55 (1.75)
world religions Beliefs, Practices, Rituals 1-8 2.63 (1.80)
Validity of Religion Judaism 1-10 9.45 (1.79) 1 = completely
Christianity 1-10 9.51 (1.82) disagree;
Islam
Confucianism 1-10 8.27 (2.43)
Jainism 1-10 8.14 (2.14)
Syncretism Afro/Christian 1-10 6.08 (3.02) Religions of Latin
America—Santeria
Animism 1-10 6.00 (3.60)
Non-religious13 1-10 4.68 (3.68)
Validity of religious Judaism—Orthodox 1-10 8.93 (2.43) Categories with families Judaism—Conservative 1-10 8.93 (2.32) validity rank below
Judaism—Reform
Christianity—Orthodox 1-10 8.78 (2.70) Christianity—Protestant 1-10 8.88 (2.38) Christianity—Anglican 1-10 8.00 (3.02)
Islam—Sunni 1-10 9.56 (1.39)
Islam—Shi’a/Shiite 1-10 9.54 (1.39) Islam—Ibadhi (Abāḍiyya) 1-10 7.91 (2.56) Islam—Nation of Islam 1-10 7.24 (2.97) Islam—Alawites (Nusayris) 1-10 7.00 (3.09)
Islam—Ahmadiyya 1-10 7.31 (2.93)
Buddhism—Mahayana 1-10 9.08 (2.16)
Question Category Response Response Comments
range (SD) Buddhism—Teravada 1-10 9.20 (2.07)
Discipline of Humanities 77.6 percent
respondent14 Social Sciences 40.3
Physical Sciences 1.5
Biological Sciences 1.5
Other Academic 14.9
Non-academic Religion 19.4
Non-academic Other 7.46
World Religions 1-5 1.92 (1.15) 1= defnitely;
Scholar? 5 = not at all
religions that had been candidate religions but that we had delib-erately grouped into other categories (e.g., a variety of animist or syncretic religions).
5. Dealing with religious practice of noncitizens. In several nations, a large number of noncitizens exist who might be adherents of diferent religions than the citizens of the state, for example, the population of noncitizens in some of the Persian Gulf states (e.g., the population of the United Arab Emirates consists of 20 percent citizens and 80 percent noncitizens). In some cases, especially when using secondary sources, there is no distinction between citizens and noncitizens. In general, we attempted to collect data only for citizens and, if available, for permanent residents. However, this was not always possible. In such cases we downgraded the estimate of data quality.
Tis required us to review multiple sources and specifc data points, and make decisions about how to deal with these problems. We documented our deci-sions in the raw data fles, with specifc comments. Tese are available upon request.
Reconciling data points from multiple sources. We sorted the data by state, year, and source. Tere were two types of cases in which data existed in two or more sources for the same state and year. One was a case where two or more sources contained complete or near-complete data for all major religions or religious groups within the state. Te second consisted of cases where one source provided partial data on some of the religions and another source pro-vided partial data that covered other religions, which were not documented in the previous source.
Table A4.2 Major world religions and religious families in the WRP
Major Religion Religious Family Pct. Agreement in Comments Expert Poll (Std. Dev.)
Christianity 9.51 (1.82)
Protestants 8.88 (2.38) Roman Catholics 8.95 (2.62)
Orthodox 8.78 (2.70) Tis includes all the Orthodox families (Greek, Russian, etc.)
Anglican 8.00 (3.02)
Other Christians Residual category
Judaism 9.45 (1.79)
Other Muslims Residual category
Buddhism 9.21 (2.26)
Mahayana 9.08 (2.16) Teravada 9.20 (2.07)
Other Buddhists Residual category
Zoroastrianism 7.83 (2.82)
Syncretism 6.08 (3.02) Afro/Christian Religions of
Latin America—Santeria
Animism 6.00 (3.60)
Non-religious 4.68 (3.68)
When two sources provided relatively comprehensive coverage of all or most religions and these data were very similar or identical, this did not present a major problem. Te problem emerged when there were sub-stantial diferences across sources in terms of the number (or percentages) of adherents of certain religions. Te strategy we applied for reconcilia-tion was threefold. First, we checked for within-source consistency over
time. Te assumption here is that—unless a dramatic political or natural event occurred between two time points (e.g., a major population transfer, a genocide that eliminated a signifcant proportion of a religious group)—
the percentage of a state’s population that practiced a given religion did not change dramatically within a given fve-year interval. If a given source indicated dramatic shifts in the distribution of religious adherents without evidence of an event that would have caused such a shift, we concluded that there was a reliability problem. Second, if we had information about the source for this specifc data point (e.g., census, survey, estimate), we assigned a specifc reliability score to this data point. Tis enabled us to make aggregation decisions later on.
When a given source contained only partial data on one or more religious groups, we compared these data to data on those religious groups from other sources. We checked for consistency over time as well as the origin of the data for this source. Source reliability information is given in the dataset.
Te general strategy for all cases that were covered by multiple sources was to generate single records of religious groups via a reliability-weighted mean of all sources.
Interpolating missing data. Missing data were a more serious and more common problem. We confronted four types of missing data issues:
1. missing data on the frst data point12
2. missing data on a specifc fve-year point, but with data existing for adjacent years (e.g., no data on 1 55 but existing data on 1 56 and 1 57)
3. missing data on a specifc fve-year point but with data existing for previous and subsequent fve-year points (e.g., no data for 1 55 but data available for 1 50 and 1 60)
4. missing data for 2010
In the case of missing data on the frst or last time point, we applied trend interpolation. We calculated a moving average rate of change coefcient for the series of that particular state and applied it to the frst or last data point.
In the case of missing data with adjacent data points available, we applied a two-step process. First, if we had data for more than one adjacent year before or after the data point for which data were missing, we calculated an expected distribution of religious groups based on the trend for these
two or more years for which data were available. Second, we calculated an expected trend between the two time points for which data were available before and after the date for which we needed data. Tird, we calculated the average between the expected distribution of religious groups and the trend distribution. For cases included in #3, we interpolated a yearly distribution from the two time points in which data were available, and applied it to the year where data were needed.
In order to ensure that users know how we obtained the data for a specifc record, we created a variable labeled “DATA TYPE” that specifes whether the data for that country-year are from a single source or multiple sources, and whether they are interpolated or trend based.
Dual Religions. In general, religious adherence forms mutually exclusive groups. People typically practice one type of religion or do not practice any religion at all. Tis means that when summing across all religious groups in a given state (including “non-religious” and “other religion” categories), the total should equal the state’s population—and the percentages of religious groups should sum up to 100 percent. Tere are, however, a few states in which dual religion is a common practice. In such cases, the sum of reli-gious adherents exceeds the population, sometime by a wide margin. We therefore introduced a code for dual religions.
Final Cleaning of Data. We applied two additional tests in the process of fnal data cleaning: population adjustment and trend adjustment.
Te frst test was meant to ensure that—with the exception of states with dual religions—the sum of the religious groups equaled the state’s population. We used the COW (2008) total population data as the bench-mark. Tere were, however, a few cases where population adjustments had to be made. In some cases, the sources we used included data on total population that were dramatically diferent from those of COW (e.g., one of the sources, Barrett et al. 2011, lists Afghanistan’s population in 2005 at 27 million, whereas COW’s total population for Afghanistan is only 24.86 million). In that case, we adjusted the number of religious adherents in that state to ft COW’s total population, by frst calculating the percent-age adherents for each group based on the original source’s population, and then remultiplying the percentages by the COW total population to get the adjusted raw fgures of adherents for each religious group.
Tird, there were a few cases where the COW population fgures refected some break points over time. Specifcally, there were some cases
of signifcant dips in population fgures for the same state over a single year period (for example, COW’s data indicate that Jordan’s population dipped by over 33 percent—from 6.669 million in 2000 to 4.978 million in 2001). Some of these cases are due to political changes. For example, the dissolution of Yugoslavia between 1991 and 1999, or of Pakistan in 1971 explains dramatic declines in population. Tus, for each case of population decline, we investigated frst whether any political shifts explain this trend.
If we could not fnd such an explanation, we informed the COW data host of these changes and adjusted religion groups accordingly, while maintain-ing consistency with the COW data. Tese caused some breaks in the dis-tribution of religious groups over time within a given country, as is evident in the signifcant downward trend in the Bulgarian population since 1991 and its efect on the distribution of Christians in the country.
Te trend adjustment was designed to ensure that—barring major events that caused dramatic population changes in a given state—the rates of change in the relative size of any given religious group in a state would not exhibit dramatic changes from one fve-year point to another. Tis proved to be difcult to ensure, as data for specifc fve-year points were derived from diferent sources. However, whenever necessary, we applied an adjustment rule to ensure that rates of change in the relative sizes of various religious groups are fairly smooth. Tis was the case especially if the data for a given fve-year time point exhibited a dramatic diference between a preceding set of fve-year points and a subsequent set of fve-year points.
However, in quite a few cases, such a smoothing operation was not possible because we lacked sufcient information enabling us to carry out a smooth-ing operation. In particular, this was the case when the data for that specifc time point were drawn from a high-reliability source. Tis implies that in several cases, there are signifcant changes in percent adherents of a given religious group’s across fve-year time points. Tis is the case in particular with respect to two groups: “non-religious” and “other religion.” Both of these groups represent residual categories in many of the sources. We used the latter as an adjustment category to ensure that the total number of adherents matched that of the total population.
146
Chapter 5