Sociology and the Technopolitical Two-Step: The Case of the Regnerus Study

The July 2012 publication in this journal of my study on the young-adult children of parents who have had a same-sex relationship created more criticism and scrutiny than have most sociological studies. … Some perceive it as a tool for this or that political project, a role it was never designed to fill. It cannot answer political or legal questions, and is by definition a retrospective look at household composition and dynamics. – Mark Regnerus

In July of this year, Mark Regnerus, a family demographer and Associate Professor of Sociology at UT-Austin, published a study comparing various outcomes for “young-adult children of parents who have had a same-sex relationship,” also referred to as children of “gay fathers” and “lesbian mothers.” This study, in contrast with most prior research, found that these children had worse outcomes on a variety of measures (educational attainment, current employment, etc.). The study elicited immediate interest from conservative political groups, the media, and a wide range of scholars. Virtually every aspect of the study has since been criticized, from the representativeness of the survey sample to the seemingly sloppy shorthand that transformed “young-adult children of parents who have had a same-sex relationship” into children of gay fathers and lesbian mothers to the publication process (especially the shockingly fast timeline, and the conflicts of interest of reviewers and commenters). The already available November issue of Social Science Research (SSR, the journal in which the original study was published) explores these criticisms and perceived irregularities in extensive detail. Sherkat, an editorial board member at SSR, reviewed the internal records and reports thoroughly on the review process. The issue also includes two open letters commenting on the paper, a summary by SSR’s editor, and commentary by several other authors, as well as a response from Regnerus himself (quoted above).

I’m not going to comment much on the substance of the dispute here, as it’s not especially my area of expertise, and the critics in SSR and elsewhere have done an admirable job on that score already (I especially recommend this early criticism by Andrew Perrin aptly summarized by its title “Bad Science Not About Same-Sex Parenting.”) I do want to comment on the role of politics in the research process and especially the quote with which I opened this post. Regnerus’s study, flawed as it may be, attempts to address a politically salient question: do the children of same-sex parents experience worse outcomes than children from various other family structures? Anyone paying the slightest bit of attention to debates over the legalization of same-sex marriage (or, perhaps, the end of the prohibition on same-sex marriage), and especially the debates over adoption by same-sex parents, knows that this question is politically salient. We also can tell that this issue is politically salient because of the coverage afforded to the prior research on the topic (the same research Regnerus critiques). For example, one study found that children raised by lesbian couples experience better outcomes than their peers. Finally, Regnerus’s study was funded by the politically active, and very conservative, Witherspoon Institute, which suggests that someone somewhere thought it was going to be politically relevant. Were it not politically relevant, odds are that Regnerus would never have been able to conduct the study.*

So, how is it that Regnerus can open his response in the November issue of SSR with the blanket statement that his study “cannot answer political or legal questions”? Here, I’m going to argue that Regnerus engages in a beautiful example of what I am going to call the “technopolitical two-step.” Technopolitics is a wonderful term that comes from the field of science, technology, and society (STS), especially the work of Gabrielle Hecht and Timothy Mitchell (although their own uses of the term differ), both of whose work is rooted in the tradition of Bruno Latour. This tradition argues that technical and scientific projects have incredibly important political consequences and that technical and scientific decisions are often made with political projects (broadly conceived) in mind. In other words, the technical and political are deeply intermingled. One goal of this tradition in STS is to expose this intermingling and thus refute, in some sense, deep claims about impartiality often put forward by scientists and engineers.

Economists have perhaps the most famous version of this claim to neutrality in a classic essay by Milton Friedman, “The Methodology of Positive Economics.” Friedman argues, among other things, that economics must be a positive science, not a normative one, and that, while economists can help governments efficiently achieve a pre-specified end, they cannot offer any guidance about how much weight to give the various possible ends. Friedman writes, “Positive economics is in principle independent of any particular ethical position or normative judgments.” For example, following Friedman, economics might be able to show that there are tradeoffs between efficiency and equality, but economists have nothing to say about the moral, political, normative question of how much weight we want to place on maximizing our total output vs. distributing that output more evenly.

The “technopolitical two-step” then is a dance commonly performed by researchers engaged in policy-relevant work. Step one, claim privileged knowledge or skills for answering questions of deep political significance. Step two, reject the implication that the knowledge you produced can actually answer any political question. This dance is, I think, especially pervasive among social scientists. On the one hand, claiming relevance is absolutely essential to securing funding and attention. And, most social scientists, I think, actively want to influence political outcomes (at a minimum in some sort of generic “making the world a better place” fashion). But if the connection between a finding and a particular political debate is too transparent, the objectivity of the research is more likely to be called into question, threatening to transform the findings from “facts of the matter” into mere political rhetoric.

Regnerus’s version of the technopolitical two-step is instructive, though not particularly unique. Regnerus emphasizes that his findings are “by definition a retrospective look at household composition and dynamics.” In other words, Regnerus has only studied what has happened in the past, not what will happen next. But, this is true of virtually all scientific research. Without some assumptions that the future will be like the past, that the past provides a useful guide to the future, science is politically worthless.** Frustratingly, the researcher is often in one of the best possible positions to assess the likelihood that the future will be like the past and to outline the circumstances that might affect the reliability of estimates from the recent past as guides to the near future. Regnerus hints at this issue in the discussion section of the original article:

While the NFSS [Regnerus’s survey] may best capture what might be called an “earlier generation” of children of same-sex parents, and includes among them many who witnessed a failed heterosexual union, the basic statistical comparisons between this group and those of others, especially biologically-intact, mother/father families, suggests that notable differences on many outcomes do in fact exist.

What does it mean for the survey to capture an “earlier generation”? I think Regnerus is suggesting that the causal forces at play during the childhoods of the adults he surveyed are quite different from the causal forces that might be at play for similarly categorized individuals today. But Regnerus does not especially expand on this point and certainly does not add any kind of deep history to his finding. Again, I don’t think anything about this is atypical – it’s a short article, and it states clearly its own limitations in a way that other social scientists would be unlikely to mistake. But nothing more.

Another variation on the technopolitical two-step involves the classification of individuals (or other units of analysis) and the mapping of those classifications onto commonly used, politically and culturally salient, terms. Here is another common moment where researchers have significant influence, albeit not necessarily individual influence. Regnerus compares eight statistical groups (including “intact bio family”, “lesbian mother”, and “gay father”) on outcomes like educational attainment, relationship quality, and depression. Regnerus summarizes the main result of his study as one of simply noting differences between these groups: “The between-group comparisons described above also suggest that those respondents with a lesbian mother and those with a gay father do not always exhibit comparable outcomes in young adulthood.” Regnerus in particular has received a lot of criticism for this part of the dance, but again I think it’s a pervasive problem for social science. For sexuality research in particular, Barrett’s comment in SSR does an excellent job of discussing the problems of constructing categories in research and mapping those categories onto politically salient ones:

Coming from a research focus on the experiences of working-class men who are sexually active with other men but may not identify as ‘gay,’ I have found that representation and presentation of these men often taps into public meanings that misrepresent them. … When I have taken the shortcut of labeling the men ‘gay’ but noting that they do not identify as gay, I have found that both reviewers and other readers conflate the men’s experiences with the experiences of the more well-known middle-class, publicly self-identified, gay male; readers thus have difficulty understanding my points about class discrimination within gay culture, or about the class limits that affect the ability to access the resources available to self-identified gays (e.g., supportive work environments). Similarly, it is very common in manuscript reviews to encounter situations where authors (and some journal editors) do not grasp the hazards of using the term ‘gay’ to describe populations where many are same-sex involved but may not publicly self-identify as gay.

Here is a fantastic place to show the deep problems with trying to cleanly separate out technical “findings” and political “consequences.” Regnerus, constrained somewhat by his reviewers and editor, had control over the terminology of the findings. Without some specialized skills, outside readers would not be able to assess the quality of the link between the political commonplace “gays and lesbians” or “gay and lesbian parents” or “gay and lesbian families” and the family structures specified in Regnerus’s study. Regnerus’s study actually investigates a group that is not socially or politically meaningful “young adult children of parents who have had a same-sex relationship.” But Regnerus reports his findings in terms that are filled with political salience: children of same-sex parents, or children of lesbian mothers and gay fathers. An attentive reader with access to the full article and some understanding of how to read a methods section can decide on their own whether or not the statistical category in the data corresponds to the salient, socially recognized, politically meaningful category. But in attempting to forge that link, Regnerus converts his findings into ones more directly relevant to the political debate, and thus more capable of answering “political or legal questions”, like should same-sex couples be allowed to marry or adopt. The conclusion is especially telling:

While previous studies suggest that children in planned GLB families seem to fare comparatively well, their actual representativeness among all GLB families in the US may be more modest than research based on convenience samples has presumed.

Note here that Regnerus himself is offering a redefinition of the politically salient category of gay and lesbian families. It’s quite possible that advocates for GLB families would not draw the boundaries of the category in the same fashion of Regnerus. And, significantly, advocates of same-sex marriage (who Regnerus references elsewhere in the conclusion, noting that courts are increasingly legalizing same-sex marriage) may explicitly have in mind stable, intact, two-parent units who plan their families, not Regnerus’s definitions, which count as a GLB family any case where either parent had any same-sex relationship. Neither definition is wrong – definitions, in some sense, are incapable of being wrong in their own context. But findings do not stay within the confines of a single paper or study. Facts “travel”, in Mary Morgan’s delightful phrase. And when social scientists subtly redefine a category (and in some sense, they are incapable of doing otherwise, as the socially recognized category rarely maps perfectly onto the categories available in a dataset, and is likely too fuzzy and contested for any single definition to accurately characterize everyone’s understanding), and then make claims about that category, they are engaging in an important kind of politics.

Similarly, Regnerus explicitly disavows any causal claims in the conclusion of the article:

Although the NFSS offers strong support for the notion that there are significant differences among young adults that correspond closely to the parental behavior, family structures, and household experiences during their youth, I have not and will not speculate here on causality, in part because the data are not optimally designed to do so, and because the causal reckoning for so many different types of outcomes is well beyond what an overview manuscript like this one could ever purport to accomplish. … I am thus not suggesting that growing up with a lesbian mother or gay father causes suboptimal outcomes because of the sexual orientation or sexual behavior of the parent; rather, my point is more modest: the groups display numerous, notable distinctions, especially when compared with young adults whose biological mother and father remain married.

Here is a beautifully executed dance move. Regnerus makes a technical point: this study looks at correlation, not causation. This point is flawless. The statement satisfied reviewers, as well it should – within the context of the article, it’s very clear and appropriate. And yet, social science research that explicitly disavows causality is still mobilized as causal for political purposes. We all know this. How responsible should authors be for not only accurately stating the technical limitations of their findings, but going out of their way to explain how those limitations affect the political usefulness of their findings? If the author disavows any capacity to answer “political or legal questions”, then of course they do not need to worry about it. If instead of continuing to dance the technopolitical two-step, the author were to engage frankly in a discussion of why and how their findings might or might not be useful for particular political claims (about the causal connection between having gay parents and having good outcomes), the veil would be cast aside, and the separation between the two realms would be in danger.

My point in writing this post is not (just) to criticize Regnerus for his dance moves. I want to argue that these problems are endemic to policy-relevant social science. The case of Regnerus is especially useful because the intense scrutiny focused on the study brought together communities of experts who laid bare the tight connections between technical decisions (how variables were collapsed together and named) and political consequences (claims by conservatives that Regnerus found that children of GLB parents had worse outcomes and thus refutes the claims made by proponents of legalizing same-sex marriage and adoption). Sociologists can continue to dance the technopolitical two-step in our journals, but the collapse of the fact/value dichotomy is real, and claims to objective knowledge are inherently political acts. We need more systematic ways of thinking about the political implications – both obvious, as in the Regnerus case, and more subtle – and how those should affect the way we conduct our research and frame our findings.

* Not everyone agrees that the question Regnerus attempts to answer is, in fact, relevant to the political question of legalizing same-sex marriage or adoption. We don’t deny anyone else the right to parent just because their background might predict that their children would be less likely to succeed in school, etc. And, even those who believe the findings are relevant mobilize them in different ways: on the right as evidence that GLB parents are unfit, and on the left as evidence that banning same-sex marriage (and other anti-GLB policies) harm the children of GLB parents.

** Lightman’s Einstein’s Dreams provides a beautiful, and haunting, description of such a world.

“Consider a world in which cause and effect are erratic. Sometimes the first precedes the second, sometimes the second the first….
Each act is an island in time, to be judged on its own. It is a world of impulse. It is a world of sincerity. It is a world in which every word spoken speaks just to that moment, every glance given has only one meaning, each touch has no past or future, each kiss becomes a kiss of immediacy.”

Acknowledgments: I would like to thank Sasha K and Jamie B for comments on a previous draft of this post.

5 Comments

  1. Dan, I see a lot of similar issues when I read studies inequality in the news. One of the most common research questions is to describe differences between two theoretically interesting cases of news coverage. As a simple example, does the Ronmey campaign receive different kinds of coverage than Obama’s campaign? Everyone who gets to publication finds some sort of a difference.

    It’s an explicit political position to assume potential news stories should not be equal, because some things are less inherently interesting or some newsmakers signal their upcoming event will probably be a snoozefest. Large N studies allow scholars to test these possibilities. By owning political implications as an explicit part of the research design, scholars could gain much greater leverage on how patterns in media content emerge. But most media scholars run away from this potential, disavowing any agency of people seeking media attention.

    When scholars compare coverage across two different cases, like Romney vs. Obama, it’s easy to explain differences through journalistic bias or structural influence. However, this study design takes its own explicit political position: two potential news stories should be equal and any difference between them is a sign of journalistic bias. This kind of relativism is typical in published research. However, the low N makes it impossible to consider more nuanced theoretical explanations of patterns in the news. Many news scholars make an implicit assumption that journalists are biased, then design their study so findings could only reinforce the assumption or be statistically inconclusive.

    A lot of what I try to do in my research is construct an alternative model to understand how patterns in the news emerge and why some people/ideas are treated differently than others.

  2. Some serious sense. Thanks!

  3. Just ran across this interesting post, which is now somewhat old. However consider this: You argue that the problems of the Regnerus study “are endemic to policy relevant social science.” I couldn’t agree with you more, and I have made that same point myself. Which is exactly why the indignant outcry against Regnerus is so wrong-headed: it is a clear example of a double standard, a selective targeting of this researcher. I’m not quite as sure as you are that one cannot attempt to distinguish between value and fact, but I would certainly agree that values shape the questions we ask and how we ask them. I think that political values do probably influence the Regnerus study, but you only need to read the presidential addresses of some of our past ASA presidents to see that much of contemporary sociology is completely driven by political values and political programs. So, it is obvious that the reason that the Regnerus study has received such a vehement response from social scientists (as well as activists), rather than the kind of critical analysis that should be applied to all research, is that it does not conform strictly to the political values that dominate modern social science. This is exactly the point Christian Smith made in his Chronicle of Higher Ed piece, and it is a serious indictment of the openness of the academic social sciences to non-conforming values and perspectives.

  1. [LINK] “Sociology and the Technopolitical Two-Step: The Case of the Regnerus Study” « A Bit More Detail
  2. Yes, that is a causal claim. « A (Budding) Sociologist's Commonplace Book