About a month ago, the New York Times ran an interesting article on evaluations of a federal program that gave subsidies to poor families to move into richer neighborhoods.* The article is titled Intangible Dividend of Antipoverty Effort: Happiness, which suggests that the only measurable benefit from the antipoverty program was increased happiness. And that’s how the article begins:
When thousands of poor families were given federal housing subsidies in the early 1990s to move out of impoverished neighborhoods, social scientists expected the experience of living in more prosperous communities would pay off in better jobs, higher incomes and more education.
That did not happen. But more than 10 years later, the families’ lives had improved in another way: They reported being much happier than a comparison group of poor families who were not offered subsidies to move, a finding that was published on Thursday in the journal Science.
That alone is an interesting finding, as it leads to all sorts of tough, reflective policy questions. Is the purpose of government to promote economic well-being and equality? Or more intangible gains like increased happiness? Of course, social science have a solution to this problem: commensurate the intangible with the real:
The improvement was equal to the level of life satisfaction of someone whose annual income was $13,000 more a year, said Jens Ludwig, a professor of public policy at the University of Chicago and the lead author of the study.
So, even though the poor didn’t actually get $13,000 more per year in income, they’re as happy as someone who did. Hooray? This bit alone would merit a blog post in my “quantification of everything” series. But what really pushes this article over the top in terms of its reification of the economic comes next:
[T]here was little evidence that the new neighborhoods made much of a difference in either income or education, a disappointment for social scientists, who had hoped that the experiment would lead to new ways of combating poverty.
What researchers did find were substantial improvements in the physical and mental health of the people who moved. Researchers reported last year in The New England Journal of Medicine that the participants who moved to new neighborhoods had lower rates of obesity and diabetes than those not offered the chance to move.
So let’s walk through this one more time. Researchers were disappointed to learn that these intervention efforts had only “intangible” effects, like increased happiness, and not “tangible” effects like increased income. But, oh by the way, participants also had lower rates of diabetes.
You see my confusion, right? I, perhaps naively, would have thought that better physical and mental health outcomes would be one notch more important than higher income.** I mean, income itself is an intangible – it’s a stream of promises. What we care about, what economists have told us to care about since as far back as Adam Smith and before, are not the promises, but what we can get from them. Like, I don’t know, more health and happiness.*** And yet, somehow, finding out that the program increases both health and happiness leads to a headline emphasizing “intangible” dividends because there was no direct effect on income?
* Hat tip to Beth Berman for sending the article along and sharing my confusion over the definition of intangible used in the story.
** I want to emphasize here that my problem is with the framing of the NYT article, and not the study itself (which I have not read). We all know that media reportage of complex (or even relatively simple) social science findings is always through a scanner darkly, so to speak.
*** Also, naively, I would have guessed that most of the benefits would accrue to the next generation, who presumably grew up in neighborhoods with better schools, etc.
A (Budding) Sociologist's Commonplace Book 
Jason Kerwin
/ October 17, 2012I completely agree that health and well-being are the real outcomes of interest that we should be concerned with. I would also agree that SWB is fairly intangible, but there are plenty of objective well-being measures we can use to supplement and validate it. I dug up the underlying paper to try to learn more about what might be driving their results. Here’s a UM library proxy link you should be able to access, Dan: http://www.sciencemag.org.proxy.lib.umich.edu/content/337/6101/1505.full
What they find “no effect” on is a synthetic index of economic self-sufficiency, not of income itself. but they *also* find no effect on an equivalent index of physical health, as well as mental health. Here’s the graph of those results, which I’ve copied to imgur from the gated paper since I believe this counts as fair use:http://i.imgur.com/7npz7.gif Here “no effect” means that the point estimate and virtually all of the 95% CI lie below zero for the economic index, and above zero for the physical and mental health indices. This is the kind of thing that sets Andrew Gelman off: looking seriously at the data and the context, we can be pretty sure there are negative effects on economic well-being and positive effects on actual mental and physical health. Given the journal in question, I’m guessing describing things this way might have been requested by the editors.
They don’t provide lists of the components of each index, and also don’t report the baseline values of the indices. I’m guessing they’re another paper the authors cite. The reason for using this method is to increase power and decrease issues of multiple comparisons, but I can see a potential problem with that. Depending on what’s in each indicator, the intervention could impact different components in different ways. For example, living in a richer neighborhood could decrease job-finding and increase average incomes because the jobs that people do find are much more lucrative. Moreover, it’s unfair to make comparisons between effects on individual health factors and on the aggregated economic sufficiency index – they should compare apples to apples (or normalized indices of fruits to normalized indices of fruits, I guess).
Taking their indices at face value, it’s impossible to support the abstract’s claim that “we found that moving from a high-poverty to lower-poverty neighborhood leads to long-term (10- to 15-year) improvements in adult physical and mental health and subjective well-being, despite not affecting economic self-sufficiency”. Inasmuch as we consider the changes in physical and mental health to be improvements, which I do think is fair, we must also consider that economic self-sufficiency has declined, not stayed flat. That’s a far more interesting result, and, I think, totally plausible.
Dan Hirschman
/ October 17, 2012Thanks so much for digging through the underlying analysis!
Noah
/ October 17, 2012Good post, Dan. As a Sociology PhD at UChicago (and Booth), I also found the NYT article interesting and actually had a back-and-forth about it with a few (non-sociology) friends. Basically, I netted out here (though I, too, have not read the actual study):
I think the fact that the only “measurable” change thus far is in happiness is not that surprising, though the passing reference to health outcomes could be as much tied to improvements in happiness as anything else. As the authors of the NYT article note, most students stayed in the same school districts, meaning their social circles wouldn’t have changed dramatically (yes, their immediate neighbors would have changed, but if they already had friend networks at school, those would be unlikely to change).* If their social circles and schools didn’t change, their actual outcomes–even if living in safer and in nicer neighborhoods–would be unlikely to change dramatically. Even more so due to the fact that the families weren’t just gifted money or new/better jobs, only new locations. So, again, the main result isn’t all that surprising.
As you allude to, I think would be much more interesting to check in again in a generation or two and see where things stand then. Even though it’s been 20 years now, initial familial and friend ties are not likely to have changed super dramatically, given the proximity of new homes to old homes (among those who were moved by the program). But each generation that is born into the better neighborhood has a better chance of new affiliations and (depending on districting) new schools, to say nothing of improved health outcomes. That subsequent generations are where you would likely see more measurable differences in tangible outcomes (jobs, income, education levels, etc). That would be my hunch, in any case.
I would also be interested to see how that bump in happiness affects other things (including the health outcomes referenced): marital happiness, parental engagement in children’s education, community involvement, etc. I bet those things, which are unreported in the article, would all have improved. Which would reinforce my belief about the benefits to future generations.
* unless their old friends didn’t want to be associated with someone who had been moved into a nicer neighborhood, which is entirely possible. AND, the kids in the nicer neighborhoods–regardless of ethnicity–may not have wanted to associate with kids who were just moved out there from poorer neighborhoods. I would be really interested to see how the kids who were involved in this were affected socially. High probability of stigma.