Yesterday, I picked up from the library a copy of Salsa Dancing into the Social Sciences, a newish (2008) methods book by UC Berkeley sociologist Kristin Luker. I just finished reading -or perhaps more accurately “devouring” – it. Luker does something (really several things) I’ve been looking for ever since I started graduate school but never seen done well and manages to produce an incredibly fun, readable, and useful text, at least for someone like me. Salsa Dancing is not for everyone (much as salsa dancing is not for everyone, including me!). Luker’s book is specifically aimed at scholars doing “non-canonical”, usually qualitative research. And indeed, I think her division of the world of social science (really, sociology) research into two camps is one of the single most useful contributions of the book. This division is not based on methodology, in the old sense of “kind of data”, but rather in the kind of questions we are asking – and Luker argues that the non-canonicals have a lot of learn from the canonicals, and a lot to learn about how to talk to them and convince them of the value of our work.
So, what are canonical and non-canonical social science? Here’s how Luker describes the difference:
[C]anonicals want to know the distribution of a population among known categories, as estimated from a properly drawn sample with a known error factor. We salsa-dancing researchers, on the other hand, want to discover the relevant categories at work, not the distribution of some larger population across categories that we have a priori chosen. We have turned to our kind of research because we have a question that canonical social science can’t take on, or can’t take on very well. (p. 102)
Note here that the difference is not simply in what kinds of data the researcher draws on – surveys vs. interviews vs. fieldwork vs. archives etc. – but rather in the kind of questions that puzzle us. Luker emphasizes how canonical research is grounded heavily in a logic of verification, while salsa-dancers are (or ought to be) grounded in a logic of discovery. The problem is that until now, there have been relatively few “good” places for non-canonicals to go for rigorous advice on methodology.
What do I mean by good? One of the most compelling parts of Luker’s book is the way she seamlessly weaves together insights from the history of social science and methodology, especially Foucauldian and science studies perspectives, along with practical advice. Luker quickly traces the history of the large scale survey, and how before WWII, quantitative work in Sociology was gendered feminine and associated with activists like Jane Addams (who was in fact on faculty at Chicago, and published in AJS frequently). With the rise of the large-scale survey, and the infusion of government and foundation money into the practice (“governmentality” much?), the gender of quantitative work switched rapidly, and the older model of verstehen and all that fell out of favor. Luker tells this story to get across a key finding from the Foucauldian/post-y perspective – all methods are problematic, all methods are tied to power dynamics, and so on. But, that doesn’t mean that the folks doing quantitative, canonical work are mindless cultural dopes fulfilling their role in the state apparatus. And in many ways, because there has been so much work put into canonical sociology, salsa-dancers have a lot of catching up to do.
Luker focuses on three interrelated aspects of canonical methodology that non-canonicals have to pay attention to, and have to do figure out how to do differently: sampling, operationalization and generalizability. Luker’s point is that canonical methodology works well to answer these three concerns for canonical problems – the relationships between pre-given categories. For research that aims to generate theories – and to generate lists of relevant categories rather than conditional correlations between variables – these problems must be solved differently.
So, rather than producing a random sample from an exhaustive (or nearly) frame of everyone you wish to generalize about, salsa-dancers should seek out “data outcroppings”, rich sites where the dynamics of interest should be easiest to see. Unfortunately, such sampling can (just as canonicals always allege) create a problem of bias that weakens generalizability. The non-canonical researcher relies on a logical generalizability rather than a statistical one – e.g. I have no reason to think that the public school I studied in my fieldwork is different from most other schools in ways that would affect my findings, and I put the burden on your to show me what I have logically missed. Having a few cases, or at least a shadow comparison, can help with this sort of argument. Luker also pushes us to generalize up, rather than across – not simply generalizing to all sites similar to where we do our fieldwork, but to broader kinds of problems (“how classes reproduce themselves without seeming to” p. 126).
So, Luker does a great job on the theoretical (establishing a framework for understanding the discipline of sociology and how it got to be arrayed the way it is, and then for figuring out which kind of research you want to be doing) and the broadly methodological (thinking about case selection, constantly asking “what is this a case of?”, considering sampling, operationalization and generalizability on our own terms). She also has tons of nuts and bolts practical tricks – ways to think about coding interviews, or gaining entree to a research site, or organizing an interview to elicit (including, for example, embracing leading questions as useful ways to elicit responses and push-back from interviewees, rather than eschewing them as is done in canonical survey research, p. 177). She also writes like someone who actually cares about how easy and fun it is to read her book, and it makes a huge difference. From the salsa-dancing metaphor running throughout the book (which is also practical advice about the virtues of unfamiliar physical activity as a way to relieve anxiety and open up your mind), to hilarious asides*, the book is just eminently enjoyable. It’s also very tight – just 225 pages plus appendices and bibliography. If I were teaching a research methods class**, I’d probably assign the whole thing or almost all of it.
A few parting thoughts. One, I’m not wholly convinced by the discussion of Ragin’s Qualitative Comparative Analysis (QCA), which Luker discusses in glowing terms. But I haven’t read the original, and I’m now at least interested enough to go check it out. Two, for someone with some background in historical research, a lot of what Luker discusses feels familiar. I think this is actually a strength, not a weakness. Luker comes out and admits in a chapter on comparative and historical work that many of her ideas come from comparative and historical sociology, and she argues that most comp/hist sociologists (if not all) are already salsa-dancers. But she goes on to say that they learned the tricks of their trade through trial-and-error and apprenticeship; there is no good go-to text. I think she’s right about that, and I think her book will be an excellent tool for bridging the gap between what comp/hist folks know they want to do and helping them figure out how to do it. More importantly, I think this book will help qualitative/fieldwork/interview-based scholars who are non-canonical find historical allies and build a common methodological language – rather than attempting to shoehorn their work into the independent/dependent variable model inherited from the canonical folks.
In summary, Salsa Dancing is a fantastic book for anyone who wants to do non-canonical social science, or teach others how to do it.
* Two favorite examples: On librarians: “Librarians, along with pediatricians, are among the greatest human beings in the universe.” (85) On content analysis and its history: “Paul Lazarsfeld (who seems to have invented almost every social science method known to humans in his spare time) and Harold Lasswell laid out the framework for doing content analysis as long ago as the 1920s and 1930s.” (187)
** More specifically, if I were teaching a graduate research methods class. I am actually GSI-ing (TAing) an undergrad methods class in the Fall.
A (Budding) Sociologist's Commonplace Book 
JeffL
/ August 27, 2010Dan,
I haven’t read the book. Also, I’m of the opinion that we need more canonical work, not less of it. STILL, I have to say that I appreciate someone elaborating on what I see as a fundamental divide in sociology methods.
It’s amazing how frequently I’ve had to explain (/defend) what statistical methods are good for. I always bring it back to exactly what Luker argues; they’re good for knowing the distribution of stuff in the world. Specifically, the statistical enterprise is one where we hold for a moment the question of “what kinds of things are there in the world?” and instead ask: “assuming that certain kinds of things exist, and are crudely measurable in the following ways, then how many of these things are there, and how are they distributed within space, time, and human groups?”
Of course, the opposite tack is good too (and is complementary). It says “holding for one second the question of distribution, what kinds of things can we identify in the world?” Obviously, the canonical approach is good for “reality checking.” E.g. maybe distracted driving does strongly lead to automobile death — BUT does it lead to a large proportion of total auto deaths, or of deaths in general? This is an important question if we want to maximize our governmental policies that regulate the world.
Of course, we need the other approach too. Frequently we need to question the “solidity” of our canonical concepts. Is “distracted driving” actually a thing? Is it a bunch of different things, masquerading as a single thing? What aren’t we measuring when we “operationalize” distracted driving?
To sum all this up, I still maintain that the field of sociology if filled with more non-canonicals than canonicals. Despite this, all of the non-canonicals go around lamenting how canonical the field is. Also, the distinction is one based on emphasis. At any point in time the distinction boils down to what you happen to be de-emphasizing at the moment (i.e. either definition or distribution). And, of course, learning the distribution of things often questions the definition of things; as much as changing the definition of things questions prior understandings of distribution.
Also, I wonder how one is going to go about canonizing a non-canonical position. Counting things is essentially about standardization. However, discovering things is always about breaking molds of thought. Is there such a thing as a standardized method for breaking apart old molds of thought?
Dan Hirschman
/ August 27, 2010Jeff,
A couple thoughts. First, I think one of the strengths of taking Luker’s argument seriously would be releasing “canonicals” from the burden of satisfying their intro theory professors. That is, just as salsa-dancers feel constrained by the need to identify independent and dependent variables and to map causal relationships between them (as we were trained to do in our stats courses, and most “methods” courses), canonicals feel like they have to make a revolutionary argument and “be theoretical”, because that’s what it means to be a sociologist (the “Mark Mizruchi” question). So maybe if the two camps, or schisms in the collective brain, were better at talking to one another, we’d also be better at letting each other off the hook. I think we have enough canonical sociology (well, in proportion – of course I think there should be more sociologists!) – but a lot of it has to dress itself up as more than that, to be constantly trying to make a big theoretical argument out of what is really a good empirical description of how the world falls out into established categories. Does that make sense?
Second, I agree with you that it is a bit paradoxical to try to routinize the process of getting out of the routine. That being said, I think that there are patterns to discoveries, so to speak, and helpful tricks. Helping someone come up with a good system for analyzing interviews is pretty independent of the specific question you are asking, and can definitely be taught. Same for how to talk to reference librarians, how to find archives, etc. There are methods to the madness!
Austen
/ August 31, 2010You write:
“Note here that the difference is not simply in what kinds of data the researcher draws on – surveys vs. interviews vs. fieldwork vs. archives etc. – but rather in the kind of questions that puzzle us.”
I don’t think this is a very good point. But maybe I don’t understand. Are you saying that a qualitative researcher could not find a reasonable answer to a question such as, is the US economy growing or contracting?
Dan Hirschman
/ August 31, 2010Austen – Good question. Let me explain a bit. Luker is not talking about qualitative vs. quantitative work. That’s not the distinction she’s emphasizing. Rather, she’s focused on canonical vs. non-canonical work. Canonical work asks about relationships between established categories. Your example “Is the US economy growing or contraction?” would be a great example of a canonical research question. We have established definitions of an economy and what it means for an economy to be growing. Lots of economists and statisticians spend time asking those questions. A canonical, qualitative researcher could try to answer it as well, but it would be hard because our definitions of the economy and growth tend to be in terms of large, statistical aggregates. So in this case, it’s likely that one method would be less appropriate for the question. But researchers with different methods could try to answer this questions – say, consumer confidence surveys vs. folks looking at bank loans.
On the other hand, a non-canonical researcher would ask (if they were me working on my dissertation), what is an economy? What does it mean for it to grow? Why do we divide up the world in that particular way? Why do we define growth in particular terms (e.g. increase in GDP)? Again, that question could be tackled from a variety of methods – I use historical and archival data, I plan to do some fieldwork working with economists who study the US economy, and I might even do some quantitative analysis of a large set of newspaper documents to look at how the economy is discussed over time. But the key thing is that I’m asking a different question – not, is the US economy growing? But, how did we get to the point where “is the US economy growing” is a meaningful question and why does it mean what it means? What else could we be asking? Etc.
Does that make sense?
Austen
/ September 2, 2010Yeah, it makes sense, thanks for the response.
Re: Luker’s categories of ‘canonical’ vs ‘non-canonical’: I think a similar, more precise distinction is between (a) sociologists who practice variable-analysis; (b) sociologists who analyze parts, or qualities, and how they fit together; and (c) sociologists who study the ‘social construction’ of objects.
I suspect your dissertation falls under (c), if I may.
Mike
/ September 17, 2010“The non-canonical researcher relies on a logical generalizability rather than a statistical one – e.g. I have no reason to think that the public school I studied in my fieldwork is different from most other schools in ways that would affect my findings, and I put the burden on your to show me what I have logically missed.”
Quants have to do this as well when seeking to generalize outside of their sample. And how do we feel when people try to do that? I’m often a little nervous.
Dan Hirschman
/ September 17, 2010Yeah.. we should be nervous. But that doesn’t mean we can’t even attempt it. It just means we need a bit more humility, I think, and a willingness to go test things in different contexts or after some time has passed. Right?