4.1 The topic model: validation, interpretation, and analysis
4.1.2 The second validation step: The topics-over-articles distribution
Table 2: Top 60 articles per topic in the American Journal of Sociology and the American Review of Sociology
Table 2 lists a series of tables containing the top 60 articles for each topic, restricted to the American Journal of Sociology and the American Review of Sociology. 12 The tables are sorted in descending rank for each particular topic. Most of these articles have a topic membership above ninety percent. This indicates how likely a particular article is about a given topic. To reiterate, the article membership to a given topic is not dichotomous (either, or), but rather continuous: assigned with a certain probability.
After that we have analyzed the articles (titles, authors, journals) and their correspondence to the topics. We can conclude that the topic modeling has accomplished its task well. For example, as expected from the first analysis step above, all the titles in the Public topic are about terms such as government, legislation, municipal, city, law. Similarly, with the Education topic is clearly capturing issues with regards to schools, stratification, educational achievement, occupation; with articles from notable authors such as James Coleman article from 1960 “The Adolescent Subculture and Academic Achievement” or else John W. Meyer's work on education. Articles listed under the topic Law/Crime have titles from prominent criminologist Lawrence W. Sherman. His “Reply: Implications of a Failure to Read the Literature” in the American Sociolog ical Review ranks highest on this topic; his article from
12 The appendix lists the top 60 articles, for all topics, and for the whole JSTOR sample.
19 1993 “Attacking Crime” in the journal Crime and Justice ranks 36 th in all of JSTOR. The renowned Walter R. Gove has six articles on this list.
The Organization topic is capturing the work of the leading scholars in both organization studies and more particularly scholars from the new economic sociology. Neil Fligstein has five articles in the American Journal of Sociology or the American Sociological Review that takes a high rank in this topics: 2 nd rank and co-authored with Peter Brantley, (1992) Bank Control, Owner Control, or Organizational Dynamics; 7th rank, (1995) Networks of Power or the Finance Conception of Control?, 21st rank, (1985) The spread of the multidivisional form among large firms, 1919-1979; 30th rank, (1987), The Intraorganizational Power Struggle;
37 th rank, (1996) Markets as Politics. These five articles have an Organization topic probability of over sixty-three percent. Paul DiMaggio has one article on this list co-authored with Walter Powell; The Iron Cage Revisited (1983) ranks 38 th , with sixty-three percent topic probability. Brian Uzzi has three articles ranking among the top 60: 11 th rank (1999) Embeddedness in the Making of Financial Capital; 17th rank (1996) The Sources and Consequences of Embeddedness for the Economic Performance of Organizations; 59th rank (2004) Embeddedness and price formation in the corporate law market. These three articles have an Organization topic probability of over fifty-eight percent.
As to the other topics, the ethnicity/race-topic includes such important authors as Douglas S.
Massey (seven times), known for his work on racial segregation (Massey and Denton 1993), or else Stanley Lieberson (27th rank), known for his work on ethnic relations. The politics-state topic includes classical authors in comparative politics such as Theda Skocpol or else Seymour M. Lipset. The Work-Labor topic, in turn, is capturing traditional issues in stratification, wage, inequality, and industrial relations research. Rachel Rosenfeld's important work on occupational inequalities is represented (Moller 2007).
Similarly, the economic topic is also capturing what we intend to measure: core economic issues. By a closer examination we can see that these authors and their articles resonates more with empirical research in political economy. However, few or none of the new economic sociologists rank on this topic; indicating that these types of issues are ignored by these scholars. Moreover, most of these articles are published in the first half of the twentieths century indicating that they resonate with the classics and their work. At least, they resonate with the scholarship of the classical era. Both these observations indicate, as the conventional story has it that there is indeed a difference between the classical period and the new economic sociology period.
However, on a closer examination, as we will see in the next section, the topic model is
suggesting that neither the classics nor the new economic sociologists focus much on issues in
the Economic topic nor in Work-Labor. Is this a bad calibration due to the topic model or an
actual feature of the field of economic sociology?
20 4.1.3The third step: Leading economic sociologists and their topic distribution
An even more substantive way to determine the robustness as well as meaningfulness of the topic modeling output is to analyze the topic-over-article distribution for scholars that explicitly regard themselves as economic sociologists. By doing that, we will also reason about why most economic sociologists fall into the Organization topic and few fall into the Economic as well as the Work-Labor topic.
The panel below depicts the topic distribution for four leading scholars, their articles, and
topic distribution. This panel gives a detailed picture over all the articles these scholars have
in our JSTOR sample and their respective topic distribution. Several insights emerge.
21
Figure 3: A panel of four economic sociologists and their topic distribution across articles in JSTOR. The heat-maps are, for each scholar, ordered from the highest concentrated article to the most diffuse.
Not coming as a major surprise, the topic model suggests that Richard Swedberg´s work tends to be more theoretically oriented. This fits well with Swedberg´s own research profile, which is reassuring. 13 Of his 19 articles in our JSTOR data, Can There Be a Sociological Concept of Interest is the most theoretical of his work, with a topic proportion of 88 percent, which makes it the most topic concentrated paper. As can be seen by the heat map, Civil courage (Zivilcourage): The case of Knut Wicksell, is his most topic diffuse paper. For Swedberg´s 19 JSTOR articles the average topic proportion is: 60 % in Social Theory, 10 % in Politics-State, 8 % in Economic. He has only 5 % in Organization. Interestingly, Patrik Aspers, one of the leading economic sociologists in Sweden, which was the Ph.D. student of Swedberg, scores
22 exactly 60 % in the Social Theory topic. He scores higher in the Economic and Organization topic than Swedberg does: 17 % and 11 %, respectively. This makes him fit somewhat better with the overall tendency in the new economic sociology than Swedberg does. Judging from our heat maps, the most diverse leading economic sociologists, defined as having most articles and most varied topic distribution, are Neil Fligstein, Paul DiMaggio, and Frank Dobbin. Having studied (regular reading) their articles closer, the topic model is sensible.
Mark Granovetter´s celebrated article Economic Action and Social Structure has a topic mix of: Social Theory 40 %, Organization 36 %, Public 11 % , Economic 5 %, less than 0.01 % in Work-Labor. His own averaged mixture of the 11 JSTOR papers is similar, but has less probability in the Social Theory and Organization topics, and more in the Analytics-Quant topic (which partial derives from Harrison White ’s influence and Granovetter´s own application of social network theory). Nevertheless, since this article is regarded as the model of the new economic sociology, its topic distribution can be perceived to be the genome or the DNA of this field.
The balance between Organization and Social Theory seems to be the defining genetic feature of the current state of the new economic sociology. The average leading economic sociologist has a topic distribution of: 28 % in Social Theory, 21 % in Organization, 10 % in Economic, 7
% in Work-Labor, 7 % in Culture. This is the topic mixture that any junior scholars should aim at replicating to fit the new economic sociology.
This picture is reinforced by making the following comparison. If we compare the average topic distribution for the leading economic sociologist with the topic distribution for the average sociologists restricted for the New Economic Sociological era (the year 1985 to 2014) we observe the following. What produces a new economic sociologist is – somewhat surprisingly – not so much the focus on the Economic topic (economic sociologists have only 2% more than the average sociologist), nor on the Work-Labor topic (economic sociologists have 1% less than the average sociologist); it is rather the topical mixture of Organization and Social Theory. The average new economic sociologist has 21 % in Organization, whereas the average sociologist has 7 %; likewise, the average new economic sociologist has 28 % in Social Theory, whereas the average sociologist has 10 %.
There are certainly several unique variations that deviate from these observations. For example, Viviana Zelizer´s work has an average of 11 % in the Gender-Family topic, which is higher than the average new economic sociologists which has only 2 % in that topic. The fact that the new economic sociologists (and the classics) have ignored gender and family issues is well-documented. It is encouraging that the model is capturing this fact too. Zelizer´s work has the lowest proportion in the Organization topic (4%) along with Swedberg (5 %).
Granovetter´s work tends to be more in the Analytics-Quant topic (25 % compared to the average of 6 %).
All comes second to Talcott Parsons in terms of the bridging the work of the classics and
assisting in establishing the discipline of modern sociology; but those who know Swedberg´s
23 work, also know that he is one of the key contemporary bridges between the classical period, including Parsons, and the new economic sociology period. Swedberg´s work tends to be less empirical, more theory driven, and highly focused on the classics. 14 If Granovetter´s article is regarded to be the marking of a new species, Swedberg´s work should be regarded as the genetic link between the classical period and the contemporary one. As a consequence, the genome of his work, its topic distribution, fits more with the classics, rather than the new economic sociologists. Table 3 depicts the topic distribution of the writings about the classics. 15 These two tables are similar. Comparing Swedberg´s topic distribution with the average classics, we observe that they are both highly theoretically driven: both rank 1 st on Social Theory (Swedberg 60 % vs. the average classic 54 %), with only second-order on the Economic and Labor. Even the articles on Marx tend to have little score on Work-Labor (3%), but this is most likely due to the fact that Marx (as the other scholars) wrote on diverse things. Accordingly, one of the things that made the classics so classical is that they were programmatic, driven by theory and theorizing (Swedberg 2003), with merely a second order focus on empirical matters which the Economic and Work-Labor topic seems to be capturing.
Table 3: Classical and New Economic Sociologist (Convert and Heilbron 2007) and their topic distributions
In summary, we want to make the following points. Firstly, the three steps above were all mainly meant to assess the validity of our model, namely, whether it is measuring what we intend it to measure, if it is sound and its implications are plausible. We have argued by using a series of tables capturing topics across the whole discipline of sociology, accompanied with our interpretation, that the model output is indeed sound and plausible. Secondly, by examining closer the actual articles realizing most strongly certain topics, we found typical themes and authors that knowledgeable readers would attribute to the respective topics.
Thirdly, re-affirming the point of validity further but now focusing on economic sociology, we have pointed out that the topic model is also capturing what we already know about new
14 http://www.soc.cornell.edu/faculty/swedberg/
15 The second table of Table 3 is produced on the basis of the classic being named in the JSTOR authorship slot.
The first of the tables in Table 3 is produced on the basis that the classics name is mentioned in the title of the
paper.
24 and old economic sociologists: That the new type of economic sociologists focuses on organization research (the Organization topic) and is social theory oriented (Social Theory topic); the classical type of economic sociology is programmatic and thus solely social theory oriented. Given that we now accept that our results are valid, and maybe somewhat surprisingly, on the aggregate we find that neither the new nor the classical economic sociologists seem to focus much on traditional empirical economic issues, that is: Economic and Work-Labor issues. This is indicative of the fact that there might be third discipline, which is also interested in economic issues but is less explicitly and systematic about its nature as an organized discipline. What we are capturing then, might be the direction of political economy (Beckert and Streeck 2008).
4.2Hypotheses and questions derived from the topic model results
What does all this mean for the purpose of this paper, namely, testing the U-shaped economic orientation of the standard narrative on economic sociology? At least three things:
(1) We need to keep distinct what people in the new economic sociology believe to be an economic issue (call this economic-as-in-new-economic-sociology, or economic-nes) from what the topic model measures, and we interpret, to be an economic issue(s), (call it, economic-as-in-topic-model, or shorthand, economic-tp, work-labor-tp, and organization-tp).
The topic model does not capture all the nuances of a language, a theory, or a discipline. It is a tool to aid us order and quantify a certain corpus. This means that even if two different research disciplines define themselves in opposition to each other but work on exactly the same topics, the topic model we are employing 16 will find them to belong to the same topic-mix. However, this is also the advantage of our topic model, namely that it will capture what people actually write about regardless of disciplinary membership. This allows us to discover economic topics and orientation inductively, which is our focus since we are agnostic about disciplinary memberships. In that we follow, “[…] Jacob Viner’s little phrase about economics, we should perhaps simply and modestly say that today economic sociology is what economic sociologists do" (Fourcade 2007: 1018).
(2) We need to construct a proxy(s) in order to be able to measure and to conduct a formal statistical test of the trending of the classical and the new economic sociology for the period 1890 to 2014. What we have shown in the three analysis steps above, that economic-nes and organization-tp fits strongly; moreover, that the average new economic sociologists have a topic-mix of both a high portion of the organization-tp and another portion of the social-theory-tp. The classics have a topic-mix concentrated to social-social-theory-tp. Consequently, we will construct a proxy that calculates the sum of the organization-tp and social-theory-tp for each article, for the whole sample; we will then use this proxy to analyze the trending of both the classical and the economic sociology.
16 See supervised machine learning or sentiment analysis for a wider family of algorithms.
25 (3) We have in the validation step discovered that there is merely a weak fit between economic-nes, on the one hand, and the economic-tp as well as the work-labor-tp on the other hand. We have demonstrated that this is an actual feature of the population and not a validity issue. Neither the classics nor the new economic sociologists contribute much to the economic-tp and work-labor-tp. Nevertheless, since economic-tp might prove to be a third type of sociologists being interested in economic issues but with no explicit disciplinary membership in the new economic sociology (viz. the political economists), we will depict the trending of this proxy as well.
From the account above and in order to test the U-shaped economic orientation of sociology of the standard narrative we articulated the following hypotheses:
1. During the era of 1890 to 1920 there is a high level of economic orientation in sociology (intercept) with an increasing trend (positive slope). This is the Era of the Classics.
2. During the period 1921 to 1985 there is a lower level of economic orientation in sociology (intercept; lower than the classical era) with a decreasing trend (negative slope). This is the Dormant Era.
3. During the period 1986 to 2014 there is a higher level of economic orientation (intercept; higher than the Dormant Era) with an increasing trend (positive slope). This cut point marks the seminal publication of Mark Granovetter´s (1985) Economic Action and Social Structure – the problem of embeddedness.
The standard narrative gives us merely some anecdotic cut-points regarding when we should expect a rise and fall in the economic orientation of sociology. In order to depict the exact historical trending of these various topics we also ask:
4. What year(s) does the organization-tp + social-theory-tp topic reach its peak (maximum) and bottom (minimum)?
5. What year(s) do the economic-tp and work-labor-tp topics reach their peaks (maximum) and bottoms (minimum), respectively?
Lastly, several sociological journals are explicitly oriented towards economic issues. Some are not in our sample (e.g. Socio-Economic Review), others are (e.g. the American Journal of Economics and Sociology). Therefore, it is relevant to know which of the journals in our sample contribute more or less than the average to the overall economic orientation of sociology:
6. What are the journal contributions to the overall economic orientation of sociology?
26 4.3The multilevel modeling: analyzing the time-trend of the economic
orientation of sociology over the last 124 years.
The multilevel models yield the following results for the economic and the organization/social-theory topics as dependent variable:
Table 4: Multilevel Regressions for the Economic and the Organization/Social-Theory topics
27 Table 4 outlines the multilevel regression analysis. In table2 in Table 4, we conduct all the tests regarding the trending of the combined measure of the organization and social theory topics. In model1 we decompose the variance into the between-journal (RP2.var.Intercept) and the within-journal variation (RP1.var.Intercept). This model shows where we have 36.6%
of the variation between journals and the rest within them (between-articles in the same journal). This model has a superior log-Likelihood 587256.19) over a single level model (-618039) (not shown in the table). This variation defines how much of the differences with regard to the combined organization and social theory measure is due to differences between the journal-level versus the article-level. Any journal variable can thus at a maximum explain 36.6 %, whereas any article variable can explain that as well as the remaining 73.4%. Journal variable examples are: the editors, the editorial board, the publisher, the journal affiliation.
Journal variables are characteristics of the journal that apply to every article within the journal. Article level variables are: publication date, authors´ characteristics (e.g. gender, age, affiliation, funding), page length, citation, and other bibliometric characteristics. In this study, besides the time (publication date) variables, we will only use the page length of an article as a covariate since we lack other readily available features in the JSTOR data (such as the gender of the author, institution, funding, title of the author etc.).
In model2 and model3 we test the conventional story of the U-shaped economic orientation of
the discipline of sociology. In model2 we estimate the average level of organization and social
theory orientation. We can already here see that for this combined topic-mix, sociologists
were as economically (viz. economic-as-in-new-economic-sociology) oriented today as they
were about 70 years ago, but more so as they were 100 years ago. On average, during the
classical era, between the period 1890 to 1920, a random sociology article had 5.64 % of
In model2 and model3 we test the conventional story of the U-shaped economic orientation of
the discipline of sociology. In model2 we estimate the average level of organization and social
theory orientation. We can already here see that for this combined topic-mix, sociologists
were as economically (viz. economic-as-in-new-economic-sociology) oriented today as they
were about 70 years ago, but more so as they were 100 years ago. On average, during the
classical era, between the period 1890 to 1920, a random sociology article had 5.64 % of
