As part of reframing a paper on the history of swaps and Glass-Steagall,* I’ve been reading a lot of papers in economics journals about financial innovation. In 2004, Frame and White published a very useful survey in Journal of Economic Literature of empirical studies of financial innovation. F&W argue that despite a massive wave of innovation starting in the 1970s-1980s, there has been very little empirical research on the topic (unlike other forms of innovation). What’s interesting to me is how they define empirical for purposes of their survey:
Empirical: The article must have formally presented data and tested hypotheses. As a result, a necessary condition for inclusion in the survey was that a standard error (and/or a t-test) appeared some where in the article. (124)
Ok, I know that F&W are mostly trying to distinguish empirical articles from theory articles, and that in economics itself such a rule-of-thumb (does it have a t-test?) probably comes very close to catching all relevant papers, but I still find the collapsing of “empirical” to “amenable to a t-test” to be quite confining. Do case studies never add to our knowledge? If they do, aren’t they “empirical”? This definition excludes all of the work from the social studies of finance tradition, for example. Even if you only believe such case studies are useful for generating, and not testing, hypotheses, they would still seem of interest to scholars of financial innovation.
I’m guessing this definition of empirical is reasonably widely held in economics, but I wonder how pervasive it is (if less audibly so) among quantitative sociologists.
*Earlier version here, but note that the next version will look very different.
A (Budding) Sociologist's Commonplace Book 
Kevin Donovan
/ April 29, 2013Deaton makes a similar point here with regard to “rigorous” in development economics: nyudri.org/events/past-events/annual-conference-2012-debates-in-development/
Rigorous is now a code word for RCT even though there are plenty of (a) ways to be rigorous without RCTs and (b) shoddy RCTs.
Jason Kerwin
/ April 29, 2013I don’t think the use of “empirical” to mean “quantitative” is an economics-specific phenomenon. Here’s an example from Wikipedia: http://en.wikipedia.org/wiki/Empirical_research
If we reason that non-quantitative studies are useful only for generating and not testing hypotheses, then “empirical” is bring used as a shorthand for the testing phase of the overall empirical cycle. In that case this is more of a linguistic annoyance (along the lines of “overseas” meaning any foreign country, including Mexico) than a problem with how people think about science.
Jason Kerwin
/ April 29, 2013*being used, not bring
Dan Hirschman
/ April 29, 2013Agreed. Though, (1) standard errors and t-tests are one very specific implementation of quantitative (and are inappropriate when you have full populations, for example, and may be misleading in other contexts, or are just not the style of Bayesian research), (2) many qualitative researchers would argue that case-studies *are* useful for testing hypotheses, especially of the “necessary” and “sufficient” (as opposed to more X –> more Y) conditions. See, for example, Goertz and Mahoney’s work (reviewed by me here). That’s the sort of analysis ruled out by this definition of empirical, I think.