Adel Daoud and Sebastian Kohl Is there a New Economic Sociology Effect? A Topic Model on the Economic Orientation of Sociology, 1890 to 2014

Adel Daoud and Sebastian Kohl

Is there a New Economic Sociology Effect? A Topic Model on the Economic Orientation of Sociology, 1890 to 2014

September 2015

Working Paper 20/2015 Department of Economics

The New School for Social Research

The views expressed herein are those of the author(s) and do not necessarily reflect the views of the New School

for Social Research. © 2015 by Adel Daoud and Sebastian Kohl. All rights reserved. Short sections of text may be

quoted without explicit permission provided that full credit is given to the source.

Is there a New Economic Sociology Effect?? A Topic Model on the Economic Orientation of Sociology, 1890 to 2014 ¹

Adel Daoud

Department of Sociology and Work Science University of Gothenburg

&

The Economics Department The New School for Social Research

Adel.daoud@sociology.gu.se

Sebastian Kohl

Max Planck Institute for the Study of Societies ko@mpifg.de

Abstract

The conventional story tells us that since the birth of the discipline of sociology, the economic orientation of the discipline has peaked twice: the first peak was during the classical era between 1890

and 1920; the second peak was sometime after 1985, marking Granovetter´s Economic Action and Social Structure paper. We have tested this story by using all full-text articles provided by JSTOR

between the periods 1890 to 2014: this contains 142 040 articles and 157 journals. We used a combination of topic modelling (machine learning applied to text) and multilevel modelling (regression) to accomplish this. We have found the following. (1) there is strong evidence for the first

peak, but contrary to this narrative, we also find a decreasing proportion of economic topics over the last century. (2) The rise of the new economic sociology as a sub-discipline of sociology, comes not in the form of an increasing focus on general economic issues, but rather in the form of a particular topic

mix of organization and social-theory research. (3) We show, accordingly, that this particular topic mix reached its bottom and started to rise by the 1929; it peaked by 1989. (4) We suggest, therefore, that Granovetter´s article (and the new economic sociology) does not mark the beginning of a second peak – as the conventional story has it – but it is rather a product of a preceding sociological interests, innovations, and orientation towards socio-economic theory development. (5) Moreover, we discover that neither the classics nor the new economic sociologists contribute much to an empirical (applied) type of economic topic found in industrial relations and political economy research. In conclusion, the

future impact that the discipline of sociology might have on economic oriented research in the social sciences, will most likely require (a) less of a within- and between disciplinary fragmentation that is most likely hampering the potential contributions sociologists can make; (b) more of engaging with applied economic affairs and thus bridge current sub-disciplinary divides. This is crucial in the age of austerity and if we seek to conceive of better socio-economic theories than existing economic theories.

1 We thank Pascal Braun, and Hans Ekbrand for their support. The article has profited from its presentation at the

Gesis Text Mining Workshop and the Max Planck Institute for the Study of Societies in 2015. We thank JSTOR

for providing us with the necessary data and technical support.

2

Contents

1 Introduction ... 3

2 Literature background ... 5

3 Data and Methodology ... 7

3.1 Data – the full JSTOR sociology data between 1890 to 2014 ... 7

3.2 Limitations of the data ... 9

3.3 Topic modeling ... 10

3.4 Multilevel modeling ... 11

4 Analysis and Result ... 14

4.1 The topic model: validation, interpretation, and analysis ... 14

4.1.1 The first validation step: The word-over-topic distribution ... 15

4.1.2 The second validation step: The topics-over-articles distribution... 18

4.1.3 The third step: Leading economic sociologists and their topic distribution... 20

4.2 Hypotheses and questions derived from the topic model results ... 24

4.3 The multilevel modeling: analyzing the time-trend of the economic orientation of sociology over the last 124 years. ... 26

4.3.1 Journal rankings ... 31

5 Discussion and Conclusions ... 32

6 References ... 35

7 Appendix ... 39

3

1 Introduction

The story of the new economic sociology is usually presented as a story of success: after a dearth of economic topics in the Parsonian era, the 1980s, and particularly Mark Granovetter’s much cited 1985 article publication, the sub-discipline saw an unprecedented Renaissance of economic topics in sociology much as in its classical founding period. In one of the latest reflections on the state of economic sociology, the entry in the new International Encyclopedia of the Social & Behavioral Sciences, it reads: "Over the past 30 years, economic sociology has erupted into a vibrant and visible subfield as sociologists increasingly apply social theories to study the economy" (Fligstein and Dioun 2015: 4128).

By analyzing all full-text articles in JSTOR between 1890 and 2014 (n=142040), this article puts these claims under closer scrutiny: does the development of economic topics in sociology really follow a U- shaped trend and did Granovetter’s article make in fact a difference? Our findings challenge indeed the established narratives about the discipline: economic topics have seen a more or less constant decline over the last century, while organizational sociology, social theory and labor-related topics, as realized particularly in articles of New Economic Sociologists, have grown. Rather than narrowly writing about the economy, economic sociologists seem to have moved from their original disciplines such as network analysis or organizational sociology into describing more economic phenomena. We thus confirm claims about the fragmented status of economic sociology (Beamish 2007: 1000, DiMaggio and Zukin 1990). A positive effect of Granovetter's most-cited article on increasing economic topics cannot be found.

The article contributes to the sociology of sociology by making use of the largest text data set used so far, to the best of our knowledge, while applying for the first time advanced methods in automated content analysis. On the one hand, it thus makes empirical accounts of the sociological discipline and its sub-disciplines more replicable, data-driven and rigorous.

While many claims are made about the empirical development, state of the art and possible future of economic sociology, those are almost never accompanied by an empirically grounded sociology of science but rather by anecdotic evidence and personal impressions of involved authors (see e.g. Beamish 2007). Our article enriches these self-interpretations of the discipline. On the other hand, it makes use of topic modeling techniques not for its own methodological sake, but to address existing claims and questions concerning a substantive topic.

Sociology of any sociological sub-discipline is important because implicitly it guides not only

researchers ’ self-interpretation and academic identity, but it is often part of the legitimation

for the existence and support of certain disciplines. Disciplines often emerge in opposition to

existing ones supposed faults and gaps they intend to cure. Growth studies about disciplines'

past successes are used to attract research money. Disciplinary history is therefore no political

neutral field, is often undertaken by insiders and does not always necessarily live up to

standards of scientific rigor.

4 Furthermore, the extreme and ongoing division of labor in scientific disciplines such as sociology makes it impossible for individual authors to read the entire output of a discipline, even of some sub-disciplines. To read the already somewhat selective JSTOR-corpus of American journal articles a researcher devoting her entire lifetime on reading would hardly come through. Meanwhile, a similar corpus would have been produced without the researcher having herself written anything. This suggests that some more powerful tools of automated text analyses might be of use which by losing information increase the time and number of documents covered.

Our findings depend, of course, on what one understands by "economic sociology" and we distinguish three different meanings. First, it could refer to the corpus of texts whose authors self-declare to write in the current of economic sociology. This would result in quite a narrow set of texts as these self-declarations are of rather recent age and moreover not often explicitly done (Trigilia 2006: 198).

Second, it can refer to the status that external observers, often ex-post, attribute to a certain corpus of texts. Thus, many sociological classics have been reclaimed by new economic sociologists as precursors of their discipline avant la lettre. This meaning, usually assumed in the standard narrative, widens the set of included texts, but hinges upon an external ascription, possibly difficult to justify or even to replicate in empirical studies.

A third meaning – the one adopted here – refers to the corpus of texts which, by using specific economic vocabulary, deal with economic things as subject matter. While the first two meanings rather concern what the author herself or others think she is doing, the third meaning is mostly defined by what the author is actually doing, i.e. writing about. The author could mistakenly consider herself a family sociologist, but when writing about economic phenomena, by the very third meaning adopted here, her text is singled out as an "economic sociology" text. Thus, by adopting the third meaning of "economic sociology" as representing economic objects talked about in economic vocabulary, we avoid possible distortions by how people interpret themselves and go back to what they actually do. ²

To back our claims aiming at a revision of economic sociology's standard narrative, the article proceeds as follows. In the next section, we summarize empirical findings about the development of economic sociology, on the one hand, and uses of topic-modeling techniques in the sociology of science, on the other hand. The gap we are filling with regard to these literatures is to transfer a new method to an existing domain of questions. The third section is dedicated to the presentation of our data and the methods through which we analyzed them to assure reliability. Section four then starts with a validity check of these methods and presents

2 In that, we actually follow many economic sociologists (see Sparsam 2015: 69), e.g. Fligstein (2003: 63):

"Economic sociology should be concerned with all aspects of material production, including the organization of

production and the organization of consumption. Thus, households, labor markets, firms and product markets are

all legitimate objects of study."

5 our main descriptive results as well as multi-level models which closer analyze the time trends. The discussion section touches on implications of our findings and suggests further research.

2 Literature background

While there are some global studies on the development of American sociology across the last century (Turner and Turner 1990, Calhoun (ed.) 2007), the development of economic sociology more narrowly has only been subject to few studies, all undertaken by researchers themselves participating in the field under study. Thus, working on a sample of manually coded American and German sociology selective articles between 1974 and 2005, Beckert and Besedowsky (2009) show that the portion of economic topics increases continuously from the 1970s onwards, with the turn from the 1970s to 1980s displaying the largest increase; the portion of articles with dependent economic variables increases; companies and markets as topics become increasingly important from the 1980s onwards, while the portion of economic-sociology theories (network, institutionalism, cultural sociology) increases from then on. In Germany, most of these developments are less pronounced and roughly lag by a decade.

Using institutional and biographical data on 31 key contributors of the new economic sociology, Convert and Heilbron (2007) explain the rise of economic sociology by several factors: the end of existing dominant paradigms such as functionalism, the increasing number of sociologists in general and those working at business schools, the support of the Russell Sage Foundation. They also note the prevalence of male researchers in economic sociology.

A content analysis of economic topics in France's sociology reveals a similar sociological retreat from economic topics after Durkheim's legacy expired. Between 1960 and 1980 less than 2% of articles and book reviews in the Revue française de sociologie were distinctly economic and their share increased only in the 1980s (Heilbron 1999). Even in the traditionally Durkheimian Année sociologique economic-sociology topics appear in a very volatile way between 1949 and 1980 and on average far less than work-related themes, based on mentions in the journal's index; moreover, economic topics are rather dealt with by non- sociologists (Steiner 2005). Findings for the other major French sociology journals for the pre-1980 period are similar: though economic topics appear, they are far from being central or regular, whereas economics with the Revue économique dominates the field (ibid.).

Besides these quantitatively oriented accounts, the new economic sociology has explicitly and implicitly undertaken hermeneutic work on its own history. The resulting and generally shared standard narrative concerning the place of economic sociology within the overall discipline is roughly this (Swedberg 1987: 17, Beckert 2002: 1f, Steiner 2007: 3, Gislain and Steiner 1995: 198): while the 19 ^th -century classics worked centrally on economic topics, by the 1920s, "sociology became what early Chicago sociologist Albion Small called the

"science of leftovers," backing off of the economic and political spheres and focusing on such

6 unclaimed subjects as the family, deviance, crime, and urban pathology. […] Thus, by 1920, both European and American sociologists were occupied with subjects far removed from the core concerns of economics. The separation of the disciplines was well underway before Talcott Parsons came on the scene, but Parsons' influence reinforced and solidified that separation" (Granovetter 1990: 89, 90). While only industrial sociology, Marxism and Third- World studies kept some economic topics selectively alive between 1940 and 1970 (Guillén et al. 2003:4f), the end of the Parsonian era, the end of the Keynesian consensus, the defense against economic imperialism and the economic crisis of the 1970s brought economic topics back on the sociological agenda (Beckert 2007: 5f, Fourcade 2007: 1015, Beckert, Diaz-Bone, and Ganßmann 2007: 21ff), making the sub-discipline supposedly one of the most vibrant fields within sociology (Beckert and Deutschmann 2010: 7, Sparsam 2015: 53). Precise dating of this rebirth is difficult, but [i]f one nonetheless were to choose one single year as the birthdate for New Economic Sociology, it would be 1985 since this was the year when the term 'New Economic Sociology' was born and also the year when Mark Granovetter’s article appeared that was soon to become the most popular article of all in contemporary economic sociology" (Swedberg 1997: 162, Convert and Heilbron 2007: 38, Granovetter 1985), moved to one of the most cited articles in the discipline (Healy 2014) and ranks first in a syllabus analysis of economic sociology (Wang 2012).

Besides other content analyses, our study also addresses the wider field of bibliometrics.

Citation analyses also allow finding out about clusters of topics through joint citations or authorships and historiographs allow tracing concepts over time through citations (Garfield, Sher, and Torpie 1964, Garfield, Pudovkin, and Istomin 2003). Main findings, however, usually concern the language, age or format (monograph, article) of citations, degree of interdisciplinarity, authorship networks, gender effects or other formal patterns of citations (e.g. Rosenberg 2015) . Sometimes these also concern citation’s content, such as the qualitative-quantitative divide in sociology (Swygart-Hobaugh 2004). Historiographs are able to trace topics through citation links based on the keyword searches offered in databases such as the Web of Science. To our knowledge, with regard to economic sociology specifically, however, only a syllabus citation analysis has been undertaken, where a division between classical and more modern topics and authors was found (Wang 2012). Our topicmodel approach complements much of this bibliometric literature thanks to its richer empirical material regarding the entire content of article publications. It rather creates a link between articles through common semantic structures than through self-declared or declared links between citations, authors, title or keywords. Thus, it is much richer in terms of what semantic content actually ties together certain texts over time, but could be fruitfully linked to the existing literature.

Thus, while the existing studies on economic sociology have relied on traditional methods of

content analysis, less so on bibliometric work, economic sociologists themselves have already

made use of automated text analysis such as topic-modeling in their own empirical studies

(Fligstein, Brundage, and Schultz 2014, DiMaggio, Nag, and Blei 2013), while its general use

in the social sciences remains debated (Ramage et al. 2009). Other scholars have employed

automated text analyses in the study of other scientific and non-scientific texts: It has been

7 applied to various scientific disciplines (McFarland et al. 2013, Teich et al. 2015, Argamon, Dodick, and Chase 2008), but also to newspaper or historical-document corpora (McCallum and Corrada-Emmanuel 2007, Block 2006), social media (Zhao et al. 2011) and fictional texts (Blevins 2010). Closest in content is a method article which investigates 100 years of German sociology through the yearly proceedings of the German Sociological Society (Bleier and Strotmann 2013). The purpose of many of these analyses, however, rather lies in introducing and probing a new method than in making a contribution to existing debates in the sociology of science.

Our contribution is at the very intersection of these two cited strands of literature. On the one hand, we rely on the most recent developments in automated content analysis, namely topic modeling and related techniques. Contrary to much of the technical literature, we do not use these techniques as l'art pour l'art to make methodological claims or to create topics inductively without further questioning. Rather, on the other hand, we address concrete claims made about the development of one of sociology's main sub-disciplines, namely economic sociology. The self-description of this discipline has up-to-now not been subject to quantitative text analyses, but has rather relied on classical content-analysis techniques or the treatment of individual texts thought to be hermeneutically crucial for the discipline. Our contribution thus lies in use of a new technique for answering existing questions of a sociological sub-discipline using unique and extensive data.

3 Data and Methodology

3.1 Data – the full JSTOR sociology data between 1890 to 2014

Our original data consist of 142 040 full-text articles from 157 journals, all written in English language. The time range is 1890 to 2014. These articles are the full sociology ³ journal coverage of JSTOR (see pp. 1-3 in appendix), provided by their service Data for Research (http://dfr.jstor.org/). This data were accessed by agreement with JSTOR on 12 ^th December, 2014.

The full-text articles were cleaned, organized, and analyzed using the R programming language accompanied with a variety of packages, most notably: the tm (text mining) package for management of the text corpus (Meyer, Hornik, and Feinerer 2008); the dplyr package for general data management (Wickham and François 2005); the topicmodels package for estimating latent topics (Hornik and Grün 2011); ggplot2 for graphical outputs; and R2MLwiN accompanied by the MLwiN software to fit multilevel growth models (Rasbash et al. 2015).

We cleaned and organized the data in the following steps. First, data was downloaded as string objects from JSTOR's HTML provided data. Second, we extracted meta-data about the

3 What counts as “sociology” journal is in fact also determined by automated content analysis procedures

(personal communication from JSTOR, 21 ^st August 2015).

8 articles using regular expressions and stored everything as a large matrix (one row per article, and as many columns as needed to store meta-data). Third, we created another large matrix called document-term matrix, which consists of all the words (terms) contained in the articles.

We went then on to remove common words (called stop-words, such as: and, is, or,), numbers, white space and punctuations from this matrix. We also stemmed all the terms. This means that different grammatical forms of a term are reduced to the common root component:

for example, the words “ industry, industrial, industrious, industrialist ” thus all become reduced to “industri” . These are all common procedures in text mining. After applying these procedures, we still had about 3.7 million common terms. We went a step further and removed terms that are sparse, i.e. not shared by many documents – we removed terms with less than 0,001% prevalence rate, which resulted in 216.406 remaining terms. ⁴

Figure 1: Flow-chart of how we transformed the data

4 We experimented with different thresholds and found 10 % prevalence rate to provide enough words to distinguish variety between articles. When we applied the topic modelling algorithm on the 3.7 million terms, we never managed to get the model to converge – even after running it for about six weeks.

Corpus

• Downloading 142 040 full- text articles from 157 journals listed in Jstor

• 3,7 million terms

Dropping cases

• Extracting meta-data

• Removing incomplete records, journals with less than 4 articles

• Result:

136.843 articles in 143 journals

Creating an article-by-term matrix

• Stemming of words

• Removing punctuation and numbers

• Case-lowering

• Elimination of most common stopwords and white spaces

• Removing the sparsest (least shared terms (0.999)

• Removing words with less than 3 letters

Topic modeling

• Using 136.843 artiicles on 276.012 terms

• Alpha- parameter of 0.01

• Using the LDA, 15 topics, 150 iterations

• Output: 15 topics values for each articles, 276.012 terms per topic

Analysis

• Identifying topics and localizing economic topics by most important terms

• Time series statistics of topics

• Multi-level

regressions on

economic

topics

9 3.2 Limitations of the data

There are several data limitations that need to be mentioned. First, we have analyzed only articles written in English which means that we do not cover developments in some important journals written in other languages, such as: in French, Actes de la recherche en sciences sociales, Revue française de socio-économie, and Revue française de sociologie; in German, Kölner Zeitschrift für Soziologie und Sozialpsychologie, and Zeitschrift für Soziologie; in Italian, Stato e mercato. The reason to focus on English only is that the quantitative text mining technique we use is most effective when dealing with one language at a time.

Second, even in the English-speaking world, we do not cover all journals and all years. The problems striking an account of all sociology journals have been well-known: no index seems to cover all self-declared sociology journals, instead they include many self-declared non- sociological journals, the journal population is constantly changing and sociology is also published outside of sociology journals proper (Bell 1967, Hardin 1977: 32f). Our corpus essentially covers the major journals of the discipline which have been used as representative of American sociology for various periods in preceding studies (Sieg 2002: 111f, Abend, Petre, and Sauder 2013, Abend 2006). Beyond this common basis, there are divergences from article collections other than JSTOR: SocIndex is probably the most encompassing sociological research database with almost 900 full-text journals with 700,208 English articles for the period 1895-2015 (SocIndex, 07.13.2015). A closer look reveals that this larger number of journals and articles is mostly achieved by extending sociology in the neighboring fields of psychology, criminology, regional studies, etc. The well-known Web of Science Core Collections, in turn, lists 139,773 articles for "sociology" from the 19th-century to 2015 which closely resembles our data volume. The Social Science Citation Index (SCCI) lists 142 journals in sociology (2015), 31 of which intersect with our corpus because the SCCI also includes many non-English journals. The intersection set includes well-known, highly ranked sociology journals, while no clear topic-related pattern can explain the coverage by one, but not the other listing. A shortcoming of our corpus is the absence of some newer journals associated with economic sociology ⁵ such as Socio-economic Review. As also other non- economic-sociology journals were not recently included, however, and as this concerns only the most recent period, one cannot speak of a systematic distortion of the entire corpus.

Finally, the Scopus database lists 1,7 million articles in "social sciences" since 1960, while no further discipline-refinement is possible.

Third, while we go beyond bibliometric analyses that do not use the textual body of articles, we do not analyze books, which means that any conclusions reached in this paper could be distorted by developments in the book area. Most of the sociology classics, for instance, wrote in book form. Since their influence had a certain time-lag, it might somewhat affect negatively our hypothesis about the U-shaped curve of economic orientation of sociology articles. Still, it would be somewhat unlikely that the language, the vocabulary, and the

5 See: http://econsoc.mpifg.de/links.asp

10 research interests in sociology books are disjunctive to developments in sociology articles.

Accordingly, analyzing journal-articles only will give a reasonably good picture about developments in sociology.

These three limitations – English centeredness, selective journals, no books – should be kept in mind when interpreting the results. They are not in principle unsolvable restrictions, but face any data project of this size. In the Discussion-section we will point to some ways in which future research will be able to deal with these limitations.

3.3 Topic modeling

As pointed out by DiMaggio et al. (2013) sociologists analyze text in one of three approaches:

qualitative reading of text, semi-structured qualitative reading with a coding sheet, or fully automated algorithmic analyses. One of the main limitations of the first approach comes mainly in terms of producing reproducible results. Two of the limitation of the second approach are that it is impractical to use for large corpora: we have about 140.000 articles, if one would spend two hours to read each article without doing anything else (eating, sleeping, publishing etc.), it would take about 32 years to get through our corpora – without any analyses of the articles. Moreover, it would also be difficult to achieve a reasonable degree of inter-coder reliability, would one employ several coders instead. The main limitation of the third approach is that the meaning of a text (an article) is reduced to its constitutive words (keywords), without necessarily looking at the discursive, contextual, and linguistics relations between these words. Frequency based content analysis is an example of such an approach (Jockers, 2014, p. 73 ff. ; Stone, 1966). What we need, as DiMaggio et al. (2013, p. 577) argues, are approaches that must satisfy four desiderata: first, they must be explicit which means that data and estimation methods are reproducible and transparent; second, it needs to be automated in order to allow for analysis of large corpora; it must be inductive to allow for discovery of underlying structures and so for (qualitative or quantitative) hypothesis testing;

fourth, the approach must account for the relationality of meaning across varying discursive and linguistic contexts. Topic modeling fulfills all these conditions (Blei et al., 2003; Blei and Lafferty, 2007).

The basic idea behind topic modeling is that of a bag of words. ⁶ The main assumption is that there are certain given latent topics that inform a given field (e.g. sociology) and which condition the writing of documents (e.g. articles). Each topic (i.e. the bag) has then a list of all terms that exist in that field (i.e. words) which they load on with a certain probability; each document in turn, loads on each topic with a certain probability. The “writing process of a document” can then be des cribed in the following steps: assume that we have 20 topics: first,

6 There are several pedagogical or technical introductions to how topic models work and examples of its

application (DiMaggio et al., 2013; Fligstein et al., 2014; Newman et al., 2006), we will here only give a brief

primer.

11 take a random document in the field of sociology and roll a dice with 20-sides (20 topics) with a likelihood for each side equal to the documents topic probabilities – in other words, it is a weighted dice. Imagine that our dice showed topic 5. Second, go to topic 5 and roll another dice but now one with sides equal to terms (assume that we have 3000 terms) and weighted according to the probability distribution of words across that topic. Picture that we roll the 3000- sided dice and we get the word “market”. Third, we assign the term “market” to our document and re-do the whole process again until we fill up all the so-called tokens of that document. Tokens are the sum of all the randomly assigned words for each document: we may re-use a word in the process described above. Accordingly, a document score to all the topics (document-topic matrix) with a certain probability – summed to 1 for each document;

all topics score to all words (topic-words matrix) with a certain probability – summed to 1 for each topic.

The essential task of topic modeling is to estimate these probabilities: in the example above, all parameters were assumed. It does so by back-tracking the whole process. There are several estimation algorithms, but the most common and the one we use is called Latent Dirichlet Allocation (LDA), which is underpinned by Bayesian statistical theory. LDA has a relational and machine learning approach to modeling language. The algorithm will seek to find structure in the corpus by co-occurrence between words with respect to how they cluster in documents. The only observed data are words and documents whereas topics are estimated.

As DiMaggio et al. (578) describe:

“LDA trades off two goals: first, for each document, allocate its observed words to few topics; second, for each topic, assign high probability to few words from the vocabulary. Notice that these goals are at odds.

Consider a document that exhibits one topic. Its observed words must all have probability under that topic, making it harder to give few words high probability. Now consider a set of topics, each of which has very few words with high probability; documents must be allocated to several topics to explain those observations, making it harder to assign documents to few topics. LDA finds good topics by trading off these goals.”

An important premise to bear in mind is that the number of topics has to be specified by the researchers manually, which some have suggested to be problematic (Schmidt 2013).

However, we argue, that this manual specification does not pose a problem per se. We regard topic modeling as a way of solving a jigsaw puzzle: whether the puzzle consists of 20 pieces or 2000 pieces, it will always reconstruct the exact same picture. To ensure interpretability, and in similarity to DiMaggio et al. (2013) define 12 topics, and Fligstein et al. (2014) specifies 15 topics, we kept our topics to 15.

3.4 Multilevel modeling

While topic modeling measures the topical orientation in sociology, it lacks standardized mechanisms to test various hypotheses. We use, therefore, multilevel modeling (also known as, random effect models, mixed models, or hierarchical models) as supplementary method.

The main advantage of multilevel modeling is that it allows us to capture the time-trend of the

12 economic orientation of sociology as well as control for the journal clustering of articles (Singer and Willett 2003, Steele 2014). We chose multilevel models over fixed effect models because we do want to estimate both the within and between journal variation to determine their relative importance. When the model is defined correctly, it can properly estimate what a fixed effect model can do (capture time-varying effects) and other more useful estimations (time-invariant effects, partition lower and higher level variance, clustering effects, etc.) (Bell and Jones 2014)

We will define the following baseline multilevel model, and vary it according to the hypotheses we will test (formulated in the following sections) and dependent variables we will define. It has the following fixed part ⁷ :

�� �� �� = � �� + � � �� + � � ��

+ � _{�� ���} + � _{� �� ���}

+ � � �� ��� + � ���� �� + � ��

With the following random part ⁸ :

� � = � + �

� _� = � + _�

� � = � + �

� _� = � + _�

And we assume that both the higher level random terms ( � _� , � _� , etc.) and lower level variance ( � _� ) are normally distributed with mean zero and are uncorrelated with the fixed effect parameters. The random terms are allowed to have covariance among themselves (captured by the covariance parameters � _{� �} , � _{� �} etc.), such as:

[

�

�

�

�

] ~ � (

, || � _�

� _{� �}

� _{� �}

� � �

� _�

� _{� �}

� � �

� _�

� � � � �

||

)

[� _�� ] ~ � , � _�

The dependent variables are captured by the term �� �� _�� which measures the economic orientation of a particular article and will be derived from the topic model output: in this paper we will device two different dependent variables that follow directly from the topic

7 Observe, the terminological difference between a fixed effects models and the fixed part of multilevel modelling. Fixed effects model is a type of regression, whereas the fixed part of a multilevel model captures the average effect of the specified variables.

8 Whereas the fixed part captures the average estimated effect, the random part captures how the effect is

distributed (deviates) for each and every case (articles, and journals).

13 model analysis. The index a is the article identifier: it runs from [1, 2, …136843] – the total valid articles ⁹ in our sample. The index j is t he journal identifier: it runs from [1, 2, … 143] – covering all the journals in our sample. This also means that all the articles (136.843) are hierarchically nested in 143 journals. Moreover, the values of �� �� _�� runs from 0 to 1 since it is a proportion variable. Higher values indicate that article a in journal j has a larger economic orientation than lower values.

The focal independent variables are the era-variables which are defined as the mean (intercepts) economic orientation during the eras 1890to1920,1921to1984, 1985to2014, respectively; the other three are defined as the trend (slopes) of the economic orientation of articles during the 1890to1920 slope ,1921to1984 slope ,, 1985to2014 slope ,. The variables estimating the mean are all defined as dummy variables; the variables estimating the slope are timer- variables counting in decimal years when the article was published in the relevant era. For example, an article published mid-1986 will have a value of 1.5 counter (one and a half year):

it counts the difference of when the era starts and the date when the article was published in that relevant era.

If the general story described by the standard narrative of economic sociology is correct, the classical era should have a high 1890to1920 value with an increasing 1890to1920 _slope , the intermediary era should have a lower 1921to1984 with a decreasing 1921to1984 _slope , the New Sociological-era should have again a high 1985to2014 and an increasing 1985to2014 _slope . Besides the era-variables, which are all based on time, we controlled for the page length of an article: whether an increasing page length also generates an article with more economic orientation.

All our main results will be based on linear multilevel models. The ideal case would be to use a Tobit model since the dependent variable is censored; or Dirichlet regression models since we are modeling proportions. However, we prefer to not add additional complexity to an already complicated regression model. A multilevel Tobit model or Dirichlet regression will force us to do just that. Still, using single level Tobit regressions models shows that the coefficients are relatively robust compared to a multilevel model. For a Dirichlet model, one needs to assume a negative correlation between all topics. This does not hold for our model.

Another approach would be to use logit or probit models. This approach has its own limitation: we would have to come up with, mostly arbitrary, thresholds to define a dichotomous dependent variable. Since we are primarily interested in the general trend of the coefficients (positive, negative or zero) and only secondarily in the magnitude of this trend, we chose to work with linear models.

9 Several articles lack page number and have to be dropped from the multilevel analysis. These articles are still

used in the topic modelling.

14

4 Analysis and Result

The main aim of this section is to analyze the functional shape (trend) of the economic topics in sociology with the help of the JSTOR-data. This analysis involves the following elements:

(1) implementation of a topic model in order to measure the development of 15 topics in sociology over the last 124 years in the data sample described above; (2) the topic modeling output is then used as input for the multilevel regression, which then was used as a formal test of our hypotheses; and (3) further sensitivity analysis of mainly the topic model but also the multilevel model.

4.1 The topic model: validation, interpretation, and analysis

As described in the methodology section, we applied an unsupervised machine learning algorithm named Latent Dirichlet Allocation. Our model used a predefined number of topics:

15. This number merely defines the number of clusters that we want the algorithm to order the terms (words) of the articles into and then the topic (viz. 15) distribution across each article.

We also set the α -parameter to 0.01: this defines the prior Dirichlet distribution the model should assume. The lower the α -parameter, the more concentrated topic distribution the model will assume and thus generate: this means that the model will try to assign the most probable topics with even higher probability, the lower the α -parameter is. Conversely, the higher the α -parameter is, the more uniform the topic distribution will be across each article. We have experimented with various topic numbers and α -parameters; we still found the results to be robust. ¹⁰

Before presenting the results, we will do some validity check (DiMaggio, Nag, and Blei 2013, Grimmer 2010) to show that the model measures what knowledgeable readers would expect it to measure. In the first step and section, we ask: does the term by topic distribution intelligible? Do the words cluster in a way that they would describe a substantive sociology topic? From this we will also stipulate the names for the 15 topics since the topic model will only number the topics one through fifteen. In this step, we also check the topic distribution of the term “embeddedness” – the hallmark concept of the new economic sociology – to properly identify the correct economic topic(s).

In the second step, we ask: do the top articles assigned to each topic make sense? This step involves qualitative reading of the actual articles and ensures both that the economic topics and the non-economic topics are being measured properly.

In the third step, we ask: what are the topic distributions for the articles of the leading economic sociologists? This, again, ensures that the economic topic(s) itself is robust across

10 A model with arbitrarily high number α-parameter will force the articles to have a uniform topic distribution

with 1/15 probability in each topic. This because the α-parameter regulates how much prior distribution should

an article be assigned to each topic. Therefore, having a low α-parameter will let data (text) have more influence

rather than this prior information.

15 various authors that explicitly define themselves as belonging to the (new) economic sociology.

Lastly, we put everything together to paint how the general economic orientation of sociology has been trending for the last 124 years. We will then regard this piece of results as descriptive or indicative until we have carried out a formal statistical test in the regression section.

4.1.1 The first validation step: The word-over-topic distribution

Table 1: The 15 Topics and their Top 50 terms

Table 1 depicts all the topics and their top 50 defining terms. All words score with a certain

probability on all topics; however, they tend to concentrate on one or several topics. This

means that some terms can have a high ranking on more than one topic. See, for example, the

term “ social ” : it is the top term for the Social Theory and for the Micro-Individual topics; it is

the third highest term ranked on the Law-Crime topic; the fourth ranked on both the Ethnicity-

Race and Politics-State topics; the seventh on the Education topic; sixteenth on the Gender-

Family topic; the nineteenth on the Analytics-Quant topic; twenty-sixth on the Culture-

Generic topic; the algorithm allocates the term “ social ” to the top 50-terms for all topics,

except for religion where it ranks 7328 ^th . That it ranks low on the Religion topic is most likely

a reflection of the fact that other terms are used to describe social relationships by

membership to various groups: using terms as israel, palestinian, cathol (viz. catholic),

christian, jewish, islam, movements, member. For a discipline being mainly about the social

world, it is reassuring that this term is important for all most all topics of sociology.

16 However, the removal of omnipresent or sparse terms might improve topic mixture, but does not usually distort the topic model outcome. We, therefore, tested the topic mixture sensitivity by removing both omnipresent and very sparse terms. The distributions for topics across articles and terms across topics were very similar to this final output presented in Table 1.

Most topics are straightforward to interpret qualitatively and we have therefore stipulated the topic title accordingly: the topic model output merely numbers the topic titles. The following topics do not need much explanation as to why they are named as they are: Education, Law- Crime, Gender-Family, Work-Labor, Economic, Organization, Religion, Ethnicity-Race, Politics-State, Analytic-Quant. Topics that are less obvious, needed some further validation:

Social Theory, Micro-Individual, Global Issues. For these, we read some topic authors and articles to determine their appropriateness (see the two next sections). The topics that are the least obvious are Public and Culture-Generic: in that order. Public is the least obvious since it contain several general terms: will, one, may, can, unit, new, must, etcetera. However, after further analysis including the articles and scholars clustering in this topic, we settled to call it Public. This topic has terms about: public, state, govern, interest. The same logic applies to Culture-Generic topic. We added the suffix Generic to both these topics to indicate this uncertainty.

In this instance, we should flag for the fact that topic model algorithms can sometimes create so-called residual topics; meaning that the algorithm allocates terms to one or more terms that it has difficulty to allocate to more meaningful topics. As the large JSTOR dataset covers diverse time periods, we expect such topic to arise. That the topic Public-Generic is capturing both a meaningful but also a less meaningful topic (residual) is further fuelled by observing the size of the topic proportion during the time period 1890 to at least 1950. It is not ideal to have this large topic proportion, but our sensitivity analysis has not managed to decrease the size of this topic. Additionally, this topic has a relatively high correlation with so called garbage-terms, i.e. terms that do not add any substantive meaning. Even if we have gotten rid of many stop-words, very sparse terms, and other common terms, some terms are difficult to completely get rid of because they can be created by transformation of more substantive terms. Nevertheless, we are confident that given the assumptions of this paper and our sensitivity checks, the issues of residual topics and garbage-terms do not affect the main conclusions of the paper.

One of the most important terms of the new economic sociology is the term embeddedness.

Understanding this term ’ s topic distribution is a prerequisite to determine the meaningfulness of the economic topics that we need to focus on. Figure 2 depicts this distribution. The x-axis represents the 15 topics and the y-axis the topic belongingness probability. Here we see that this term scores highest on Organization followed by Social Theory and Culture-Generic. This is a reassuring result because we know from theory that Granovetter´s work has had a major impact on organization research. ¹¹ Observe that this term has a very low probability score on

11 It also corresponds well to Granovetter’s topic-mix as represented in the heat-map (figure 4).

17 the Economic and on the Work-Labor topics. One of the reasons is, most likely, is that the Economic topic represents work more of the classical type; the Work-Labor topic captures the general field of industrial relations and political economy. Both these are certainly important in their own right to determine the economic orientation of sociology. Hence, we will include all three of them in our further analysis.

Figure 2: The topic distribution of the term “embeddedness”

In summary, we are confident that the algorithm is capturing at least 13 meaningful topics;

three of them having direct relationship to economic orientation of sociology: Economic, Work-Labor, and Organization. It might appear to be remarkable for a social scientist who is not familiar with machine learning to see that a computer algorithm is able to order and allocate terms in the manner shown Table 1 – and the following tables (see next sections).

However, there is as little, or as much, mysteriousness about this algorithm as there is for the

Google Search algorithm. They are of the same algorithmic-family – and who does not use

Google?

18 Nevertheless, even if you use and trust the Google Search algorithm, you might not trust the outcome of our particular model: in the following two sections we therefore conduct further robustness analyses of the topic model output to ensure that it is validly and reliably measuring what it is intended to measure.

4.1.2 The second validation step: The topics-over-articles distribution

Table 2: Top 60 articles per topic in the American Journal of Sociology and the American Review of Sociology

Table 2 lists a series of tables containing the top 60 articles for each topic, restricted to the American Journal of Sociology and the American Review of Sociology. ¹² The tables are sorted in descending rank for each particular topic. Most of these articles have a topic membership above ninety percent. This indicates how likely a particular article is about a given topic. To reiterate, the article membership to a given topic is not dichotomous (either, or), but rather continuous: assigned with a certain probability.

After that we have analyzed the articles (titles, authors, journals) and their correspondence to the topics. We can conclude that the topic modeling has accomplished its task well. For example, as expected from the first analysis step above, all the titles in the Public topic are about terms such as government, legislation, municipal, city, law. Similarly, with the Education topic is clearly capturing issues with regards to schools, stratification, educational achievement, occupation; with articles from notable authors such as James Coleman article from 1960 “The Adolescent Subculture and Academic Achievement” or else John W. Meyer's work on education. Articles listed under the topic Law/Crime have titles from prominent criminologist Lawrence W. Sherman. His “Reply: Implications of a Failure to Read the Literature” in the American Sociolog ical Review ranks highest on this topic; his article from

12 The appendix lists the top 60 articles, for all topics, and for the whole JSTOR sample.

19 1993 “Attacking Crime” in the journal Crime and Justice ranks 36 ^th in all of JSTOR. The renowned Walter R. Gove has six articles on this list.

The Organization topic is capturing the work of the leading scholars in both organization studies and more particularly scholars from the new economic sociology. Neil Fligstein has five articles in the American Journal of Sociology or the American Sociological Review that takes a high rank in this topics: 2 ^nd rank and co-authored with Peter Brantley, (1992) Bank Control, Owner Control, or Organizational Dynamics; 7 ^th rank, (1995) Networks of Power or the Finance Conception of Control?, 21 ^st rank, (1985) The spread of the multidivisional form among large firms, 1919-1979; 30 ^th rank, (1987), The Intraorganizational Power Struggle;

37 ^th rank, (1996) Markets as Politics. These five articles have an Organization topic probability of over sixty-three percent. Paul DiMaggio has one article on this list co-authored with Walter Powell; The Iron Cage Revisited (1983) ranks 38 ^th , with sixty-three percent topic probability. Brian Uzzi has three articles ranking among the top 60: 11 ^th rank (1999) Embeddedness in the Making of Financial Capital; 17 ^th rank (1996) The Sources and Consequences of Embeddedness for the Economic Performance of Organizations; 59 ^th rank (2004) Embeddedness and price formation in the corporate law market. These three articles have an Organization topic probability of over fifty-eight percent.

As to the other topics, the ethnicity/race-topic includes such important authors as Douglas S.

Massey (seven times), known for his work on racial segregation (Massey and Denton 1993), or else Stanley Lieberson (27th rank), known for his work on ethnic relations. The politics- state topic includes classical authors in comparative politics such as Theda Skocpol or else Seymour M. Lipset. The Work-Labor topic, in turn, is capturing traditional issues in stratification, wage, inequality, and industrial relations research. Rachel Rosenfeld's important work on occupational inequalities is represented (Moller 2007).

Similarly, the economic topic is also capturing what we intend to measure: core economic issues. By a closer examination we can see that these authors and their articles resonates more with empirical research in political economy. However, few or none of the new economic sociologists rank on this topic; indicating that these types of issues are ignored by these scholars. Moreover, most of these articles are published in the first half of the twentieths century indicating that they resonate with the classics and their work. At least, they resonate with the scholarship of the classical era. Both these observations indicate, as the conventional story has it that there is indeed a difference between the classical period and the new economic sociology period.

However, on a closer examination, as we will see in the next section, the topic model is

suggesting that neither the classics nor the new economic sociologists focus much on issues in

the Economic topic nor in Work-Labor. Is this a bad calibration due to the topic model or an

actual feature of the field of economic sociology?

20 4.1.3 The third step: Leading economic sociologists and their topic distribution

An even more substantive way to determine the robustness as well as meaningfulness of the topic modeling output is to analyze the topic-over-article distribution for scholars that explicitly regard themselves as economic sociologists. By doing that, we will also reason about why most economic sociologists fall into the Organization topic and few fall into the Economic as well as the Work-Labor topic.

The panel below depicts the topic distribution for four leading scholars, their articles, and

topic distribution. This panel gives a detailed picture over all the articles these scholars have

in our JSTOR sample and their respective topic distribution. Several insights emerge.

21

Figure 3: A panel of four economic sociologists and their topic distribution across articles in JSTOR. The heat- maps are, for each scholar, ordered from the highest concentrated article to the most diffuse.

Not coming as a major surprise, the topic model suggests that Richard Swedberg´s work tends to be more theoretically oriented. This fits well with Swedberg´s own research profile, which is reassuring. ¹³ Of his 19 articles in our JSTOR data, Can There Be a Sociological Concept of Interest is the most theoretical of his work, with a topic proportion of 88 percent, which makes it the most topic concentrated paper. As can be seen by the heat map, Civil courage (Zivilcourage): The case of Knut Wicksell, is his most topic diffuse paper. For Swedberg´s 19 JSTOR articles the average topic proportion is: 60 % in Social Theory, 10 % in Politics-State, 8 % in Economic. He has only 5 % in Organization. Interestingly, Patrik Aspers, one of the leading economic sociologists in Sweden, which was the Ph.D. student of Swedberg, scores

13 http://www.economyandsociety.org/people/richard-swedberg/

22 exactly 60 % in the Social Theory topic. He scores higher in the Economic and Organization topic than Swedberg does: 17 % and 11 %, respectively. This makes him fit somewhat better with the overall tendency in the new economic sociology than Swedberg does. Judging from our heat maps, the most diverse leading economic sociologists, defined as having most articles and most varied topic distribution, are Neil Fligstein, Paul DiMaggio, and Frank Dobbin. Having studied (regular reading) their articles closer, the topic model is sensible.

Mark Granovetter´s celebrated article Economic Action and Social Structure has a topic mix of: Social Theory 40 %, Organization 36 %, Public 11 % , Economic 5 %, less than 0.01 % in Work-Labor. His own averaged mixture of the 11 JSTOR papers is similar, but has less probability in the Social Theory and Organization topics, and more in the Analytics-Quant topic (which partial derives from Harrison White ’s influence and Granovetter´s own application of social network theory). Nevertheless, since this article is regarded as the model of the new economic sociology, its topic distribution can be perceived to be the genome or the DNA of this field.

The balance between Organization and Social Theory seems to be the defining genetic feature of the current state of the new economic sociology. The average leading economic sociologist has a topic distribution of: 28 % in Social Theory, 21 % in Organization, 10 % in Economic, 7

% in Work-Labor, 7 % in Culture. This is the topic mixture that any junior scholars should aim at replicating to fit the new economic sociology.

This picture is reinforced by making the following comparison. If we compare the average topic distribution for the leading economic sociologist with the topic distribution for the average sociologists restricted for the New Economic Sociological era (the year 1985 to 2014) we observe the following. What produces a new economic sociologist is – somewhat surprisingly – not so much the focus on the Economic topic (economic sociologists have only 2% more than the average sociologist), nor on the Work-Labor topic (economic sociologists have 1% less than the average sociologist); it is rather the topical mixture of Organization and Social Theory. The average new economic sociologist has 21 % in Organization, whereas the average sociologist has 7 %; likewise, the average new economic sociologist has 28 % in Social Theory, whereas the average sociologist has 10 %.

There are certainly several unique variations that deviate from these observations. For example, Viviana Zelizer´s work has an average of 11 % in the Gender-Family topic, which is higher than the average new economic sociologists which has only 2 % in that topic. The fact that the new economic sociologists (and the classics) have ignored gender and family issues is well-documented. It is encouraging that the model is capturing this fact too. Zelizer´s work has the lowest proportion in the Organization topic (4%) along with Swedberg (5 %).

Granovetter´s work tends to be more in the Analytics-Quant topic (25 % compared to the average of 6 %).

All comes second to Talcott Parsons in terms of the bridging the work of the classics and

assisting in establishing the discipline of modern sociology; but those who know Swedberg´s

23 work, also know that he is one of the key contemporary bridges between the classical period, including Parsons, and the new economic sociology period. Swedberg´s work tends to be less empirical, more theory driven, and highly focused on the classics. ¹⁴ If Granovetter´s article is regarded to be the marking of a new species, Swedberg´s work should be regarded as the genetic link between the classical period and the contemporary one. As a consequence, the genome of his work, its topic distribution, fits more with the classics, rather than the new economic sociologists. Table 3 depicts the topic distribution of the writings about the classics. ¹⁵ These two tables are similar. Comparing Swedberg´s topic distribution with the average classics, we observe that they are both highly theoretically driven: both rank 1 ^st on Social Theory (Swedberg 60 % vs. the average classic 54 %), with only second-order on the Economic and Work-Labor. Even the articles on Marx tend to have little score on Work- Labor (3%), but this is most likely due to the fact that Marx (as the other scholars) wrote on diverse things. Accordingly, one of the things that made the classics so classical is that they were programmatic, driven by theory and theorizing (Swedberg 2003), with merely a second order focus on empirical matters which the Economic and Work-Labor topic seems to be capturing.

Table 3: Classical and New Economic Sociologist (Convert and Heilbron 2007) and their topic distributions

In summary, we want to make the following points. Firstly, the three steps above were all mainly meant to assess the validity of our model, namely, whether it is measuring what we intend it to measure, if it is sound and its implications are plausible. We have argued by using a series of tables capturing topics across the whole discipline of sociology, accompanied with our interpretation, that the model output is indeed sound and plausible. Secondly, by examining closer the actual articles realizing most strongly certain topics, we found typical themes and authors that knowledgeable readers would attribute to the respective topics.

Thirdly, re-affirming the point of validity further but now focusing on economic sociology, we have pointed out that the topic model is also capturing what we already know about new

14 http://www.soc.cornell.edu/faculty/swedberg/

15 The second table of Table 3 is produced on the basis of the classic being named in the JSTOR authorship slot.

The first of the tables in Table 3 is produced on the basis that the classics name is mentioned in the title of the

paper.

24 and old economic sociologists: That the new type of economic sociologists focuses on organization research (the Organization topic) and is social theory oriented (Social Theory topic); the classical type of economic sociology is programmatic and thus solely social theory oriented. Given that we now accept that our results are valid, and maybe somewhat surprisingly, on the aggregate we find that neither the new nor the classical economic sociologists seem to focus much on traditional empirical economic issues, that is: Economic and Work-Labor issues. This is indicative of the fact that there might be third discipline, which is also interested in economic issues but is less explicitly and systematic about its nature as an organized discipline. What we are capturing then, might be the direction of political economy (Beckert and Streeck 2008).

4.2 Hypotheses and questions derived from the topic model results

What does all this mean for the purpose of this paper, namely, testing the U-shaped economic orientation of the standard narrative on economic sociology? At least three things:

(1) We need to keep distinct what people in the new economic sociology believe to be an economic issue (call this economic-as-in-new-economic-sociology, or economic-nes) from what the topic model measures, and we interpret, to be an economic issue(s), (call it, economic-as-in-topic-model, or shorthand, economic-tp, work-labor-tp, and organization-tp).

The topic model does not capture all the nuances of a language, a theory, or a discipline. It is a tool to aid us order and quantify a certain corpus. This means that even if two different research disciplines define themselves in opposition to each other but work on exactly the same topics, the topic model we are employing ¹⁶ will find them to belong to the same topic- mix. However, this is also the advantage of our topic model, namely that it will capture what people actually write about regardless of disciplinary membership. This allows us to discover economic topics and orientation inductively, which is our focus since we are agnostic about disciplinary memberships. In that we follow, “[…] Jacob Viner’s little phrase about economics, we should perhaps simply and modestly say that today economic sociology is what economic sociologists do" (Fourcade 2007: 1018).

(2) We need to construct a proxy(s) in order to be able to measure and to conduct a formal statistical test of the trending of the classical and the new economic sociology for the period 1890 to 2014. What we have shown in the three analysis steps above, that economic-nes and organization-tp fits strongly; moreover, that the average new economic sociologists have a topic-mix of both a high portion of the organization-tp and another portion of the social- theory-tp. The classics have a topic-mix concentrated to social-theory-tp. Consequently, we will construct a proxy that calculates the sum of the organization-tp and social-theory-tp for each article, for the whole sample; we will then use this proxy to analyze the trending of both the classical and the economic sociology.

16 See supervised machine learning or sentiment analysis for a wider family of algorithms.