CONCEPTUAL AND METHODOLOGICAL FRAMEWORK
1.3 METHODOLOGY: METHODS AND CONSTRUCTS
1.3.1 The Appropriateness of Quantitative Research for Cross-Cultural and Cross-Linguistic Studies 1.3.1.1 Introduction
In the two previous sections we presented an extended overview of the thesis’ two main research subjects. As illustrated, methodological approaches for investigating cultures are fervently discussed. Quantitative cultural studies have especially been more and more strongly contested in
s content, different research methods are applied. Whereas cognition-related analyses are almost exclusively based on quantitative studies, the social role of
d are presented in the two following sections. It is deemed beyond the purpose of this thesis to provide a detailed picture of epistemological and ontological discussions in IS t reasons and underlying
and multidisciplinarity. Its multidisciplinarity and application orientation give rise to a variety of research methods and
oing discussions presented next, and also our own approach, therefore have to be interpreted and evaluated taking this context into account. In particular, the fundamental
not arrived at by statistical or other quantitative procedures”) or by comparing their research goals and
ch of them. It puts qualitative and quantitative research methods into recent years. With respect to language, methodological debates seem to be less relevant.
Nevertheless, depending on the study’
language is investigated by both quantitative and qualitative methods.
Our research designs are predominantly based on quantitative methods. The reasons behind this choice are varied, an
research. Yet, an extended introduction to the subject helps depic
assumptions regarding the dominance of quantitative approaches adopted in our research. In the following paragraph we therefore want to give a simplified picture of what distinguishes qualitative approaches from quantitative approaches. In this context, advantages and drawbacks of the research approach for cross-cultural and cross-linguistic IS research are discussed.
Methodological debates about the appropriateness of qualitative and quantitative approaches in IS research have always been influenced by this application orientation
theoretical foundations borrowed from other disciplines, each of them characterized by their own particularities and research traditions.
The ong
relationship between research objectives and methodological appropriateness is essential to our argumentation line.
1.3.1.2 Epistemological Assumptions of Quantitative and Qualitative Research
One possible distinction of research methods is between quantitative and qualitative research.
However, giving a precise definition of each term is not an easy task. Quantitative and qualitative research are usually defined in a negative way (such as “quantitative research’s findings are the applied research methods. For example, qualitative research is usually described as less structured than quantitative research. As such it employs research methods such as observation, content analysis, (unstructured) interview, case studies, or focus groups.
This section aims to provide more information about the distinction of these two research methods and the characteristics of ea
y conventional scientific and interpretivist research. This lexical choice first of all insinuates the traditional role of quantitative methodological approaches in scientific research, and is at the
natural sciences and were increasingly adapted to social sciences
ich “… commences with observation of specific instances, and seeks to establish generalizations” (Hyde, 2000; Lienert,
search has traditionally been playing an important role in social sciences (Hoepfl, 1997). Observations are consequently conducted in a rather exploratory way. Deductive the context of epistemological objectives and the research disciplines where they were traditionally applied.
Clarke (2000) distinguishes between three research approaches appropriate for IS research:
quantitative research, qualitative research, and engineering research. The author classifies the first two approaches as business disciplines, as opposed to engineering disciplines and computer science (Clarke, 2000). Clarke himself avoids the terms of quantitative and qualitative research and replaces them b
same time an indicator of the ongoing debate about a distinction between the two approaches.
Despite many years of fervent discussions, debates about qualitative and quantitative methods cannot provide any general insight about the usefulness of one of these methods. Their appropriateness can only be evaluated taking the study’s goal into account. Judgements about the suitability of qualitative and quantitative research are based on underlying epistemological assumptions (Trochim, 2000) which simply can differ from researcher to researcher without one being superior to the other.
Quantitative studies emanate from
only in the last few decades (Stangl, 1989). Their epistemological objective consists in obtaining
“objective knowledge” and “true facts” about the world (Lienert, 1989). Research is based on a large number of single observations, conducted to confirm theories and hypotheses that were postulated beforehand. Thus, insight in quantitative studies is gained through confirmatory, inductive research processes (Clarke, 2000; Trochim, 2000) wh
1989).
In contrast, assumptions underlying qualitative research reject the existence of objective facts and truth. Epistemologically, the goal is to understand a phenomenon, a research objective requiring analyses that take a large variety of context into account. Its comprehensivity is the reason why qualitative re
conclusions are more often applied in qualitative research than in quantitative research, although also qualitative research most often follows an inductive process (Hyde, 2000). “Deductive reasoning commences with generalizations, and seeks to see if these generalizations apply to specific instances.” (Hyde, 2000)
tion of constructs and variables inevitably inheres a reduction of complex processes and coherences. This reduction can however be interpreted as a
rough quantification are aimed at, or whether revealing the processes of a complex phenomena is
z
usually coded quantitatively. Conversely, numbers as used in quantitative research are based on qualitative judgement and several underlying
ive and Qualitative Research in Cross-Cultural and Cross-Linguistic
psychological studies. It often borrows its theoretical basis and methodological approaches from its Adherents of the qualitative approach often criticize the narrowness of assumptions in quantitative research. Indeed, quantitative research is limited in nature and can only illustrate a small portion of reality (Trochim, 2000). The necessary quantifica
type of standardization allowing the comparability of research results.
In contrast, research results from qualitative approaches have little validity as defined by quantitative research approaches. Due to the specific contextuality of each research situation, replications and comparability can hardly be achieved. Hence, in its simplest form, the question of quantitative or qualitative research is the question of whether simplifying and standardising effects th
emphasi ed.
Besides, it is often argued that each qualitative approach contains quantitative elements and each quantitative approach embodies qualitative parts. One example is that analyses of words - as they often occur in qualitative research - can and are
assumptions (e.g. Did the respondent understand the meaning of the numbers in the survey?) (Trochim, 2000). Also, in most instances quantitative theory developed from qualitative investigation of untested theories (Hyde, 2000).
Finally, given the topic of this thesis, the extent to which favouring either quantitative or qualitative research approaches is biased by the scientist’s cultural background should be questioned. Although speculative at this point, the dominance of quantitative studies in sciences shows indeed a relationship with the dominance of Western culture in science, and its preference for linear rather than comprehensive thinking inherited from the Greeks. “Studying the model designed to explain nature tells more about their creators than about the part of nature being studied”. (Missana)
1.3.1.3 Appropriateness of Quantitat Studies
After having situated qualitative and quantitative research in the context of epistemological objectives, one needs to answer the question of how appropriate these methods are for the purpose of cross-linguistic and cross-cultural IS research.
Cross-linguistic and cross-cultural IS research is close to various social science disciplines and
on of culture’s idiosyncratic complexity, and also requires a number of assumptions. Finally, the lack of quantitative data alternatives limits the variety of quantitative
ich finds its justification if the environment of decision processes and product development cycles is considered. Cross-linguistic and
cross-Information regarding cross-linguistic and cross-cultural adaptations therefore the decision maker. Also, information needs to be understood within a short amount of time. Quantified results help rationalise the process of decision making by lity
ansfer between ethnologists and programmers (Kirah, 2005). Thus, quantitative approaches in cross-cultural (and cross-linguistic) IS studies are predominantly justified reference disciplines. In particular, cultural studies are often conducted employing qualitative approaches such as ethnographic research (Honold, 2000), repertory grid technique (Agourram and Saucier, 2004), or in-depth interviews, etc. (Gillham, 2005). Indeed, examinations of culture’s impact appear to be prone to be conducted with qualitative techniques for a number of reasons.
First, as can be inferred from our discussion in section 1.1.3, culture is a particularly complex and context-dependent construct. Additionally, the quantification of cultural data embodies a considerable reducti
cultural research.
We argue that cross-cultural studies and cross-linguistic studies in the context of IS research differ considerably in their objectives from classic social studies due to their application orientation and integration into a business and engineering environment. Business and engineering areas have a rather strong tradition of quantitative research, wh
cultural studies are the basis for investment decisions regarding product or service adaptation in a business context. Investment decisions are not usually made by experts of cultural and cross-linguistic differences.
need to be acquired by
presenting a reduced picture of reality in the form of “numbers” and “facts”. The question of how useful simplified pictures of reality are for this kind of decision remains, and to which degree rea should be reduced.
Also, the implementation of language and culture related to the product design is more challenged if insight is obtained from qualitative research. Various field reports describe, for instance, the difficult information tr
through the studies’ integration into investment and product development cycles, requiring matching approaches between the two disciplines.
Finally, it should be noted that the quantitative approach also resulted from the proximity of our studies with psychological research, where inductive research has a long scientific tradition and high level of recognition.
temological
Action (Ajzen and Fishbein, 1977; Ajzen and Fishbein, 1980)) were concerned. In contrast, the exploratory identification of and regularities and the understanding of single phenomena received less
y
Logfile-Analyses
In our research we mostly relied on logfile-analyses. Logfile analysis examines the information logged on a server by users’ website visits. We analysed in the relevant studies data concerning one single website. In study 8 we ad ler in order to obtain additional
1.3.2 Applied Methods
The dominance of quantitative approaches in our research implies an important epis premise of this work: we emphasize testing well-known theories from other disciplines, applied within the context of cross-cultural and cross-linguistic Internet communication. In particular, theories from psychological research (such as the Theory of Reasoned
undetected relationships attention in our studies.
In order to obtain data for answering the research questions outlined, we gathered data through different methods. These methods allow the collection of data that is suitable (after transformation) for statisctical procedures. Nevertheless, the methods differ in their appropriateness for investigating specific theories. In order to compensate for the limits of one particular method, in the majority of our studies we combined two or more methods. In particular, high external validit was assured through employing logfile analyses, whereas higher internal validity is provided (to a certaint degree) through questionnaires or laboratory studies.27 The particular characteristics of each data gathering method employed are illustrated in subsequent sections of this chapter.
A large part of our findings are based on logfile analyses. The discussion of this methods’ suitability is preceded by brief introductions due to the fact that the application of logfile analyses is rather unique and little known in cross-cultural and cross-linguistic research.
1.3.2.1
ded information from a web craw
information that allowed us to put the concerned website into the context of the World Wide Web.
The web crawler provided data about how other websites link to our website, the target website.
Further details are presented after the discussion of aspects of logfile analysis, in section 1.3.2.1.3.
27 Internal and external validity, in addition to construct validity and statistical validity, are the main forms of validity distinguished by Shadish, Cook, and Campbell Shadish, W.R.; Cook, T.D. and Campbell, D.T. (2002): Experimental and Quasi-Experimental Designs for Generalized Causal Inference, Houghton Mifflin, Boston, MA.. Internal validity regards the question of whether the research design is appropriate to provide evidence for the correlation between the independent and the dependent variables.
External validity relates to a possible generalization of the research results to other groups than the tested sample group.
bsite was visited (lang=…). Figure 4 gives an example of a logfile as employed in our studies. Each requested page (click) is recorded as one entry (line) in the Data from logfile analyses provide insight into the behaviour of the website’s users, and allows, to a certain extent, conclusions about the user’s preferences and difficulties when navigating on the website. The logfile data contains various information. In our studies we mainly refer to the user’s IP address, the requested page, the page that was requested before (referrer page), and the time of access. Additionally, a particularity of the specific logfile we used allowed us to obtain information about the language in which the we
logfile.
Figure 4. Example of Logfile Data
Logfile analyses constitute a data gathering method that is appropriate for obtaining highly structured data that allows for statistical analyses. We primordially obtained the number of
e icular page (set)) within one session or within an
(see study 1 – section 2.3.2), preprocessing data analyses and data structuring were required (e.g., application of taxonomies). In order to make
ral, and logfile analysis in particular, can be conducted either in an exploratory or in a confirmatory way. As already suggested by the definition
our studies, logfiles analyses were employed for confirmatory approaches.
occurrenc s of certain events (e.g. request of a part
aggregated group of individuals (e.g. group of native speakers) from logfile data. Time consumption was a further often-collected piece of information. In order to obtain structural information, i.e.
information about the form of navigation sequences
logfile data available for statistical procedures, we transferred the data set into a MySQL or Oracle database. Data was then analysed or further processed by means of queries. Final statistical analyses were done with SPSS 11.0 and 12.0.
Finally, it should be noted that data Mining in gene
above, exploratory knowledge discovery is a traditional approach in data mining. A classic example is the detection of frequent patterns within the data set. However, confirmatory models based on the knowledge gained from data mining, tested for validity in another sample set, often complement this exploratory approach. Examples of data mining applications are the fraudulent use of services (e.g. for credit card use, insurance companies) or for marketing purposes in business areas, including customer profiling and cross-selling (Berry and Linoff, 1997). Within
studied. Necessary assumptions about the equal distribution of specific variables are more justified if the sample group is large. Hence, logfile analyses usually ensure high external
of a logfile analysis is in consequence usually low and can only be assured in combination with other methods.
ultural or linguistic variable exist.
As such, the correlations found are much more likely to be produced by the impact of culture
the set of environmental conditions (in laboratory studies created by the researcher) to other environmental conditions (settings and conditions)” (Bracht and Glass, 1968).
1.3.2.1.1 Advantages and Limits of Logfile Analyses
In our evaluation of the suitability of logfile analyses for cross-cultural and cross-linguistic IS research, we refer to two key quality criteria: the control of variables (internal validity) and the generalizability of results obtained (external validity).
One of the advantages of logfile analyses is its accommodation of the large size of the data set.
Such a large sample set offers unprecedented opportunities in terms of the amount and variability of data and users
validity and therefore the generalizibility of the study.
Yet, despite the large data set provided by a server’s log, logfile analyses do not allow to control for specific variables. Logfile data assesses behavioural variables, but data about the users’ attitudes and reasons for their behaviour cannot be obtained. Splitting the user-group into two groups with different conditions (treatments) and gathering data about these two user groups is difficult if the analysed logfile comes from a real website. The internal validity
Nonetheless, external and internal validity cannot be considered independently of each other. First, practical reasons usually require the researcher to focus on one form of validity. Second, an increase of external validity may also increase the internal validity and vice versa. In our case, a large variance of non-cultural and non-linguistic variables within each cultural or linguistic group increases the chances that the cultural and linguistic variables investigated are not confounded with other (e.g., socio-economic) variables, unless systematic relations with the c
and/or language.
Ecological validity is one specific aspect of external validity, regarding “the extent to which results can be generalized from
Logfile analyses belong to the group of non-reactive data gathering methods, i.e. analysed test-subjects do not notice the test-situation. Non-reactivity is one way to ensure that the behaviour analysed behaviour does not differ considerably from the user’s usual behaviour (Albrecht, 1982;
see also Albrecht, 1985; Bungard and Lück, 1974; Webb, et al., 1981).
idity (Bracht and Glass, 1968), which is consequently large in logfile analyses.
Nevertheless, a server-centric design that only allows an examination of the website users (vs.
non-r ological approaches. Firstly, it does not allow
many users who
e an easy means of data collection but require
impact of applied heuristics on the data set is documented in the following paragraph.
lyses in our Research
Despite the number of positive aspects, the use of logfile analyses for the purpose of cross-linguistic and cross-cultural studies is still rare. Exceptions are, for example, Stander et al. (2004) A further reason why logfile analyses are suitable for assessing genuine behaviour consists of the fact that data from logfiles usually record behaviour in the user’s authentic environment. As a result, it can be furthermore assumed that the users analysed are typical representatives of their cultural or linguistic groups, in contrast to recruited international students. “The extent to which the results of a study can be generalized from the specific sample that was studied to a larger group of subjects” -with potentially diverging characteristics is referred to as population val
Finally, since logfiles assess data about every single user, the bias of self-selection is considerably reduced and solely regards the exclusion of non-users. Thus, logfile analyses are characterized by high ecological but also high population validity that may not be reproduced in a laboratory setting (see also Reips, 1999).
users) rep esent some of the limitations of our method
the gathering of important data about people who do not visit the website for some reason.
Secondly, it is not clear to which extent insight is limited to the specific features of the investigated website.
Furthermore, unless persistent cookies are used, analyses can only be carried out on the session level. However, while employing persistent cookies would make it easier to identify unique website visitors, it might also bias the results due to the effects of the privacy concerns of
disable cookies. The extent to which such a data aggregation biases results depends on the study’s objective. In some cases, an analysis on the session level can be justified by the fact that frequent websites visits by the same user reflect a low access barrier or a high usefulness.
disable cookies. The extent to which such a data aggregation biases results depends on the study’s objective. In some cases, an analysis on the session level can be justified by the fact that frequent websites visits by the same user reflect a low access barrier or a high usefulness.