Yes, it really is all about culture.

The Freakonomics Blog has a shockingly humble post today, given their reputation and universalizing tendencies, about cultural variation in response to various psychological experiments. I should probably read the underlying paper first, but I wanted to get a thought out before doing so. The paper, as reported by Ayres on the blog, examines cross-cultural variation in responses to various deviations from rationality, broadly-construed. For example, one classic experiment uses a visual illusion to trick respondents into thinking two lines are not of equal length, and then varies the lengths until respondents believe them to be equal. It turns out that for respondents from some cultures, the illusion basically fails – they start off seeing the two lines as equal. Ayres also reports on variation in how participants play the “Dictator Game” and similar game-theoretic experiments, where participants have to share resources to maximize social (but not individual) benefits.

The conclusion drawn from the study, according to Ayres, is that we moderns are “WEIRD”:

The article, “The Weirdest People in the World” (ungated working paper), has the startling thesis that social scientists in trying to investigate basic psychology may have erred by oversampling outlier populations. The “Weirdest People” of the title are Western, Educated, Industrialized, Rich, and Democratic. (The cuteness of the title is not one of the article’s strengths.) But the idea that “we” are the exotics usefully jars one from complacency.

Ok, nice, clever title and backronym. But I think it might be misleading, and might unintentionally reinforce the exact tendency the paper attempts to dispel. To argue from a set of responses to a certain set of experiments is to privilege those experimental results over all the experiments we did not run, and thus universalize them as the standard against which to measure deviation. In other words, this paper shows that the educated rich West is full of people who answer the most extreme on a particular set of measures. But why are those measures interesting? Only because we, until recently, thought they described universal problems in perception or rationality! Are we actually outliers because we have the most extreme responses to these games? Or do we look to these experiments because they are among the set of experiments for which Westerners are prone to give the most extreme answers?

One last way of saying it: assuming for a second that the results hold, and that people from the Kalahari desert see two lines as equal that we see as unequal, then we can guess that someone from the Kalahari trying to understand errors in perception would never have bothered asking anyone about those two lines – there would literally have been nothing interesting to see. Our methods are just as social, cultural, whatever you want to call it, as the variations in the findings from those methods. Turtles, all the way down.

Now I suppose I should read the paper and see if they make the same point.

EDIT: I read through some of the results and conclusions. I think Ayres’ summary is decent, but the authors both do and do not address the issues raised above. For example, from the discussion section:

The empirical foundation of the behavioral sciences comes principally from experiments with American undergraduates. The patterns we have identified in the available (albeit limited) data indicate that this sub-subpopulation is highly unusual along many important psychological and behavioral dimensions. It is not merely that researchers frequently make generalizations from a narrow subpopulation. The concern is that this particular subpopulation is highly unrepresentative of the species. The fact that WEIRD people are the outliers in so many key domains of the behavioral sciences may render them one of the worst subpopulations one could study for generalizing about Homo sapiens. (Henrich et al. 2010: 79)

I really like the overall tone of the study, especially the critique of psychology for ignoring or downplaying cross-cultural variation and for jumping to evolutionary origins for results that don’t hold up across groups. While I readily agree with the claim that we shouldn’t jump to cross-cultural or species-wide conclusions based on data from one group, I think jumping to the conclusion that rich Westerners are especially WEIRD falls into the trap of privileging our measures for weirdness, which are in turn based on precisely those deviations that we previously identified studying the people we now think are weird. Henrich et al. come close to this point:

Relying on WEIRD populations may cause researchers to miss important dimensions of variation, and devote undue attention to behavioral tendencies that are unusual in a global context. There are good arguments for choosing topics that are of primary interest to the readers of the literature (i.e., largely WEIRD people); however, if the goal of the research program is to shed light on the human condition, then this narrow, unrepresentative sample may lead to an uneven and incomplete understanding. We suspect that some topics such as self-enhancement, cognitive dissonance, fairness, and analytic reasoning might not have been sufficiently interesting to justify in-depth investigation for most humans at most times throughout history. Alternatively, the behavioral sciences have shown a rather limited interest in such topics as kinship, food, ethnicity (not race), religion, sacred values, polygamy, animal behavior, and rituals (for further critiques on this point, see Rozin 2001; Rozin et al. 2006). Had the behavioral sciences developed elsewhere, important theoretical foci and central lines of research might likely look very different (Medin & Bang 2008). (2010: 80)

That discussion comes close to (and is fair more coherent than) my discussion above. It’s an excellent start, but I think it still casts puzzles in terms derived from WEIRD populations, and I’m not sure I see a way past that. And given that it’s only one paragraph out of a paper whose overall theme is more similar to the first quote (from p79), I can see how it would easily get lost in translation and simplification.

In conclusion, a really interesting paper that raises lots of troubling issues about human nature and how we jump to conclusions whenever we think we’ve measured it.

Advertisements

2 Comments

  1. Alex

     /  January 5, 2011

    Excellent post. I came across a news article about the WEIRD paper a few months ago and downloaded it. (Though I haven’t read it yet–something about the cheesy backronym makes it hard for me to get motivated.) There’s always a big debate during the holidays in my family between, on one side, my sister and brother in law (both psychology professors) and, on the other, myself and my girlfriend (a [budding] sociologist and anthropologist, respectively) about the validity of many psychological findings. It’s heartening to see people asking the same questions from within their discipline. Reflexivity is an important characteristic of any intellectual activity.

    To introduce myself, I came across your blog a few weeks ago and find it very interesting. I am a (prospective) budding sociologist myself, hopefully matriculating in the fall. I share several of your interests, and am looking forward to plumbing through the past entries. Have you decided, by the way, what you want to write your dissertation on?
    Best,
    Alex

    • Alex,

      Thanks for the comment and for the introduction!
      I have indeed decided what my dissertation topic will be, and I’m currently working on my prospectus. The short version is that I am interested in the history of national income statistics (GDP, GNP, etc.) and how they shaped our understanding of “the economy as a whole”. I am going to trace out some of the debates about what was included and excluded, and how we determined how to measure the value of hard to price things (like unpaid housework, the environment, etc.). I could get a lot more specific, but that should give you the general idea. If you want to email me, I’d be happy to chat more about it and I’d love to hear what you are thinking of working on!

      Dan