3. Stock-taking of e-Infrastructures in the social sciences and humanities
3.3 Background information on projects
3.3.4 Project outcomes and user constituency
The respondents were also asked about the main outcomes of the project (QC5).
Most identified publications (148), new methods (129), new data (114), follow-on collaborations (143) and new tools (143) as key outcomes. In response to the open part of the question (the “other” category), many more outcomes were
identified (see Annex I.4, p. 194). The questionnaire also probed for a discussion of what type of data had been produced: 80 respondents identified numerical data, 75 verbal/textual data, 67 visual data, and 22 identified other data types (see Annex I.4, p. 195).
We have asked in more detail about new methods and tools developed in the projects. Unfortunately it turned out, that it was not possible to differentiate between methods and tools. Both are mutually dependent. To categorize the methods and tools respectively we have looked at their purpose. Obviously the categorization corresponds to two further questions: the technological features used (QB4, see chapter 3.3.3) and the type of data produced (QC6b). We could differentiate
between eight different functions of new methods and tools which are distributed as shown in Table 3.10.
Table 3.10: Function of new methods and tools
Function Frequency In %
Generation or analysis of qualitative data 83 57.2%
Generation or analysis of quantitative data 83 57.2%
Visualisations 73 50.3%
Building a database (including data grids, data management systems,
ontologies, digital libraries, data curation, data repositories, etc.) 38 26.2%
Simulation 14 9.7%
GIS 9 6.2%
Expert-knowledge systems 6 4.1%
No category/other/unclear (e.g. specialized search engine, e-learning
tools) 5 3.4%
Communication 3 2.1%
Total responses 145* 100%
* Multiple functions per response are possible.
Source AVROSS WP2 survey
The assignment of the method to one or more of the categories has not been clear in some cases. Hence the figures have to be treated cautiously.
The generation or analysis of data is the most important purpose of the newly developed methods. Many of the methods are designed for both, quantitative and qualitative data. This holds for 51 (35.2%) of the projects. Fairly common are also visualisations which were included in around half of the responses that answered the questions on new tools or new methods.
As different disciplines have different demands on their methodological toolboxes we expect some differences between the humanities, social sciences and sciences.
Percentages in Table 3.11 correspond to all projects in the particular discipline having developed new methods or tools. The number of cases in each cell is relatively small. Hence differences between percentages may be stochastic.
However, there are a few obvious things to claim. Researchers of the different fields struggle with different problems. Particularly they treat different kinds of data and have different necessities to represent them. The need for tools or methods to analyze or generate quantitative data is less frequent in the humanities compared to other disciplines. However, researchers from the humanities prefer visualisations more than their colleagues from other disciplines.
Table 3.11: Function of new methods and tools by discipline included in the projecta Function Humanities Social Sciences Natural Sciences
Freq. In % Freq. In % Freq. In % Generation or analysis of qualitative data 21 63.6% 35 49.2% 12 52.2%
Generation or analysis of quantitative data 13 39.4% 37 62.7% 11 47.8%
Visualisations 21 63.6% 29 49.2% 8 34.8%
Building a database 13 39.4% 12 20.3% 5 21.7%
Simulation 2 6.1% 6 10.2% 1 4.3%
GIS 2 6.1% 6 10.2% 1 4.3%
Expert-knowledge systems 1 3% 2 3.4% 0 0.0%
no category / other / unclear 1 3% 2 3.4% 1 4.3%
Communication 1 3% 0 0.0% 2 8.7%
Outcomes, user constituencies and country of the project
The different output categories do not vary too much by country/region of the respondent. Publications and new methods resulted less often from the projects in the other countries (Canada, Australia, New Zealand etc.) and new data and collaborations less often in the UK (see Table 3.12).
Table 3.12: Project outcomes by country of the project
UK Continental Europe USA Other countries Outcomes
N % of
valid N N % of
valid N N % of
valid N N % of valid N
Publications 47 84% 43 92% 48 85% 10 77%
Patent applications 1 0% 0 4% 1 3% 0 0%
New methods 47 82% 37 82% 36 89% 9 75%
New data 41 71% 32 81% 30 82% 11 92%
New tools 47 91% 41 85% 39 82% 16 94%
Follow-on
collaborations 51 81% 39 88% 37 91% 16 94%
Others 10 44% 4 58% 7 71% 1 100%
Question C5 by country of the respondent.
Source: AVROSS WP2 survey.
We also attempted to use the response to this question to approximate the
outcome of a project more generally by counting how many items were identified as outputs. Although this is a relatively weak indicator of depth, since, for example, one publication is valued as much as many, it is an indicator of the breadth, and hence possibly the maturity, of the project. Overall the 220 respondents which provided information on projects listed an average output of 4.2 out of the 7 different types provided in question C7. Our analysis suggested that projects from the other countries are the ones with the broadest array of outcomes, averaging 4.7 per project. This is followed by the US (4.5), continental Europe (4.1) and the UK (3.7).
About 180 respondents answered the questions dealing with their user constituen-cy: 129 said there was a constituency for their work, 58 did not. The list of the domains of their constituency is provided in Table 3.13 (see also question QC8) – again, the four fields of interest to the project appear to be well represented. It is worth noting, however, that a number of additional constituencies were identified, including statistics, geospatial analysis, tourism classics, law enforcement
institutions, anthropology, government departments and agencies, art history, government and industrial planners, ethnography anthropology, indigenous users, general public teaching, community non-profit groups, people with disabilities, government policy analysts, public media studies, natural resource management, policy-making, and decision support.
Table 3.13: User constituency Domain areas for constituency of users
Constituency applies
Proportion of projects with this domain as a constituency
Agricultural Sciences 9 7.0%
Engineering & technology 16 12.4%
Electrical engineering, electronic engineering, information engineering
(hardware) 8 6.2%
Engineering & technology (civil, mechanical, chemical, materials, environmental or medical
engineering, bio- or nanotechnology,
others) 14 10.9%
Humanities 69 53.5%
Archaeology 18 14.0%
Art (arts, history of arts, performing
arts, music) 34 26.4%
History 33 25.6%
Languages and literature (excluding
linguistics) 27 20.9%
Linguistics (including computational
linguistics) 27 20.9%
Other Humanities 29 22.5%
Philosophy, ethics, religion 9 7.0%
Medical and Health sciences 22 17.1%
Natural sciences 55 42.6%
Natural sciences (mathematics, physical, chemical, biological sciences, earth & environmental
sciences, other natural sciences 31 24.0%
Computer and information sciences
(software) 38 29.5%
Social sciences 92 71.3%
Economics and business 20 15.5%
Educational sciences 45 34.9%
Law 15 11.6%
Political science 25 19.4%
Psychology 26 20.2%
Social and economic geography,
regional science 43 33.3%
Sociology 47 36.4%
Others 21 16.3%
Source: AVROSS WP2 survey.
The breadth of this user constituency, i.e. the number of different fields listed among it, shows again substantial variation by region of the project. The average US project has users from 4.8 academic domains. In contrast, the average continental European and UK project has users from 3.8 academic domains.
Surprisingly the breadth of the user constituency, as measured by the number of disciplines represented, decreases with the length of the project duration. Short-term projects have users from 4 fields, medium-Short-term projects from 3.4 and
long-Outcomes, user constituencies and discipline of the project
Although one might expect there to be substantial variation in outcomes across discipline, this is not the case. As Table 3.14 shows, projects that had a user constituency in the social sciences were more likely to mention tools as an important outcome; this result holds even when weighted by the number of times an outcome was mentioned.
Table 3.14: Outcomes by major discipline of the user constituency (% of all responses in the discipline listing an output for a project)
Humanities Social
Sciences
Neither humanities nor social sciences
Publications 85.2% 89.4% 86.5%
Patent applications 6.3% 3.8% 2.1%
New methods 88.9% 88.4% 83.8%
New data 75.0% 77.5% 79.2%
New tools 84.6% 94.1% 86.7%
Follow-on collaborations 99.9% 81.3% 87.7%
Others 42.9% 84.6% 61.1%
Source: AVROSS WP2 survey.
There are other measures of project depth and breadth. One measure is to calculate, for each project, whether a discipline represented within a project has developed a user constituency within that same discipline. The proportion of such projects is reported in the middle column in Table 3.15, and ranges from about half (in education, languages and natural sciences) to under a quarter (in computer and information sciences). The last statistic is to be expected, given the fact that computer and information sciences are typically engaged in providing e-Infrastructure to other disciplines rather than their own. Turning the question around, we also calculated, for each user constituency that was identified, whether or not that discipline was represented in the project. This set of results is reported in the second column of the table, and the range is much higher. Almost all
disciplinary constituencies that are reached are reached by a project that includes a researcher with the same discipline as the user constituency. There are a number of possible interpretations of this intriguing result. It could be that projects are developed by researchers in given disciplines because they have specific disciplinary needs in mind. It could also be that researchers in a project already have a dissemination network in place that is discipline specific, and that
knowledge about the project is transmitted through such disciplinary networks.
These different possibilities have useful, but differing, implications for the structure of funding and should be explored in a broader scientific study.
Table 3.15: The interaction between project disciplines and the disciplines of user constituencies
Proportion of identified project disciplines with constituency in same
disciplinea
Proportion of constituencies identified with the same discipline
as the projectb
Agricultural Sciences 58.3% 77.8%
Engineering and Technology 46.4% 81.3%
Electrical engineering, electronic engineering, information
engineering (hardware) 35.3% 75.0%
Engineering & technology (civil, mechanical, chemical, materials, environmental or medical engineering, bio- or
nanotechnology, others) 47.4% 64.3%
Humanities 50.5% 79.7%
Archaeology 50.0% 72.2%
Art (arts, history of arts, performing
arts, music) 57.1% 70.6%
History 47.8% 66.7%
Languages and literature
(excluding linguistics) 54.3% 70.4%
Linguistics (including computational
linguistics) 44.4% 74.1%
Other Humanities 38.5% 51.7%
Philosophy, ethics, religion 31.3% 55.6%
Medical and Health sciences 27.6% 36.4%
Natural sciences 35.2% 90.9%
Natural sciences (mathematics, physical, chemical, biological sciences, earth & environmental
sciences, other natural sciences) 51.1% 74.2%
Computer and information sciences
(software) 24.4% 86.8%
Social sciences 50.3% 83.7%
Economics and business 31.1% 70.0%
Educational sciences 50.0% 60.0%
Law 33.3% 40.0%
Political science 35.1% 52.0%
Psychology 40.0% 46.2%
Social and economic geography,
regional science 48.4% 72.1%
Sociology 45.8% 70.2%
Others 28.9% 61.9%
a Read as follows: 58.3% of the projects with agricultural scientists on the team had also agricultural science as user constituency.
b Read as follows: 77.8% of the projects with agricultural science as the user constituency also had agricultural scientists on the team.
Source: AVROSS WP2 survey.
Looking again at the fields highlighted in this work-package we see only little differences in the extent to which they produce the most frequent outcome, publications (see Table 3.16). Some differences appear for new methods which result less often in any of the five fields, and clearly less often in projects with economics & business participation. New tools and follow-on collaborations, on the other hand, result less often from archaeology projects.
Table 3.16: Outcomes of e-Infrastructure projects by fields targeted by the project
Archaeo-logy
Economics and business
S o c i o l o g y
Social geogra-phy, regional
science
Linguistics All projects Publications 82.4% 81.1% 84.7% 80.0% 81.1% 86.5%
Patent applications 6.7% 4.8% 2.8% 2.9% 0.0% 2.1%
New methods 75.0% 66.7% 72.2% 82.6% 77.1% 83.8%
New data 78.9% 79.3% 80.0% 73.5% 84.8% 79.2%
New tools 73.7% 88.6% 80.0% 87.8% 86.5% 86.7%
Follow-on
collaborations 78.9% 84.8% 86.2% 83.7% 86.1% 87.7%
Others 40.0% 69.2% 50.0% 66.7% 72.7% 61.1%
Source: AVROSS WP2 survey.
Roundabout three quarters of projects with archaeologists and linguists had a user constituency; in sociology and geography/regional science this percentage went down to two-thirds and in economics to only 55%.
Outcomes, user constituencies and duration of the project
As might be expected, the number of results reported for a project increases the longer the project lasts, as is evident from an examination of Table 3.17. There are, however, two exceptions: some long-term projects of more than three years have not produced any publications and new data is even less often an outcome in mid-term and long-mid-term projects than in short-mid-term projects. Hence, the generation of new data obviously does not need a long-term arrangement.
Table 3.17: Outcome by project duration Short-term (up to 18 months)
Medium-term (19-36 months)
Long-term (more
than 36 months) Valid N
Publications 69.7% 94.5% 83.9% 137
Patent applications 0.0% 2.4% 5.6% 83
New methods 78.1% 83.3% 90.0% 128
New data 81.3% 77.8% 68.0% 120
New tools 81.8% 87.5% 93.9% 138
Follow-on collaborations 81.8% 88.4% 93.3% 132
Source: AVROSS WP2 survey.
The relationship between project duration and the existence of a user constituency is difficult to interpret: seven out of ten short-term projects reported such a
constituency, compared with six out of ten for medium-term projects and eight out of ten for long-term projects. It would be interesting to probe the reasons for this non-monotonicity in a broader reaching study.
Outcomes, user constituencies and activity profile of the respondent Respondents with different activity profiles reported working on projects with very different outcomes (see Section 3.2.2). Those respondents whose time allocation fit a professional’s activity profiles were engaged in projects that produced fewer results than projects of the other respondent categories. In particular, these projects produced less often publications (only 70% of the projects compared to 90% for the other respondents) and new data (50% compared to 80% for the other respondents, see Figure 3.20).
Figure 3.20: Percentages of projects producing publications and new data by activity profiles
0.0%
20.0%
40.0%
60.0%
80.0%
100.0%
Researchers Professionals Administrators Scholars Publications New data
Data for this figure in Annex I.3, Table A.6.
Source: AVROSS WP2 survey.
There are also substantial differences in whether the respondent’s project has developed a user constituency. Indeed, “only” two thirds of the scholars and researchers were working on projects that had developed such a constituency, compared with three quarters of the professionals and administrators. This may, of course, reflect a project’s life cycle, where young projects are more likely to engage researchers, and more mature projects, which have developed a constituency, need administrators and professionals
Table 3.18: Percentages of projects with a user constituency by activity profiles
User constituency
Researchers (n=51) 66.7%
Professionals (n=21) 76.2%
Administrators (n=35) 74.3%
Scholars (n=78) 65.4%
All respondents (n=158) 68.6%
Source: AVROSS WP2 survey.
In sum, the projects which were described by the professionals are more likely to be application-oriented, whereas projects described by researchers and scholars are stronger in the science dimension. The administrators’ projects seem to incorporate both a scientific orientation and user focus.