One of the most interesting parts of the history of national income statistics is how rapidly and widely they diffused across the globe. Before World War II, hardly any country had official national income data, and none routinely relied on them to make policy. In the 1950s, the United Nations codified a global standard, the System of National Accounts, and official national income data became a requirement of modern nationhood – something every country had to produce, knowledge seen as indispensable for planning purposes, development aid, even assessing UN dues. And yet, while the UNSNA was a global standard, not all of the globe was equally suited to produce numbers according to its rules. In particular, the economies of Africa did not look like the economies of Western Europe where national income statistics were pioneered. Essential data that powered national income statistics, including censuses, income and payroll tax data, and more, simply did not exist in many poorer countries. African economies were just not as calculable, as those of Western Europe (at least not in the framework of the UNSNA). Thus, producers and users of African national income statistics have long known that such data were not perfect. But how bad were these numbers? Where are the problems? Are the uncertainties uniform (like the systematic undercounting of the informal sector) or more idiosyncratic? And at worst, users of such data hoped that even if the absolute GDP levels were off, the trends were still meaningful – that we might not know exactly how much poorer one nation was than another, but we could tell which countries were growing the fastest and thus assess the impact of economic development policies.

For the past five years, economic historian Morten Jerven has been arguing that African economic statistics are truly poor; so poor that we have misled ourselves into believing that we know much more about African economies than we really do. His new book, Poor Numbers: How We Are Misled by African Development Statistics and What to Do about It, summarizes this research and presents a coherent case for skepticism, and especially for criticism of the seemingly standardized world economic databases (such as the World Bank’s Development Indicators). These supposedly comparable data leave very little trace of the messy process that produced them, and thus leave end users incapable of assessing just how bad (or, less often, good) the statistics they are working with really are. As a consequence, we are collectively capable of being surprised when countries manage to update and improve their statistical output and produce seemingly fantastic outcomes, as when Ghana’s GDP shot up by 60% after a recent revision. The blog Democracy in Africa has a more detailed, chapter by chapter review of the book here. Below is a short Q&A with Professor Jerven about his new book, including his experience interviewing government statisticians across Africa, interacting with World Bank officials, and trying to convince the development community to better understand the available data.

Q1. Professor Jerven, thank you very much for agreeing to this Q&A. To start, I’m curious to know how you got interested in the topic of development statistics – and especially GDP and national accounts data. Was there a particular encounter with a “poor number” that set you off on this research trajectory?

MJ: As most students I started out accepting the GDP figures as facts. When I studied for my master’s degree in Economic History at London School of Economics, I was looking into some country evidence of GDP growth in African countries. I remembered having found a pattern of growth in one source of evidence, but then I could not recall which source I had used. Redoing the same work with a different data source this time, I found that what I now found did not match up with what I thought was true before.

It then dawned upon me that Penn World Tables, World Development Indicators, the Maddison datasets all report different data, taken from different national account files. Later, I requested the raw data that goes into the GDP time series data at the World Bank, they replied: “Raw data provided by the National Statistics Agencies are not available for external users and only handful of people at the World Bank have access to it. You may want to visit the National Statistics Offices website or contact them directly.” Thus, my research trajectory was launched.

Q2: As part of your research, you conducted interviews with statisticians working in statistical offices across anglophone Africa. What were some of the most surprising things you found in doing your fieldwork? How did statisticians react to your project?

MJ: There were many surprises, one of them was how hard it was to figure out how these GDP series were created, when and how methods and sources were revised. The lack of institutional memory and record keeping of the production of statistics was something I did expect, but at some places and times it was particularly striking. I did not expect that the contrast of the 1960s and 1970s with the 1980s and 1990s would be quite as stark.

My book is dedicated to the statisticians working at statistical offices across sub-Saharan Africa. Without their contribution and willingness to share information this book could not have been written. I think that the main reason I got their attention and that they shared their time with me is that they themselves were concerned about the state of affairs.

Q3: In chapter four, you talk a bit about the turn towards micro and experimental approaches in development economics. What do you think of the growing trend of relying on randomized control trials and other very local interventions to understand development? What are the limitations of this micro approach? Can we understand the turn to the micro as a reaction, in part, against the shoddiness of the statistical apparatus for macroeconomic analysis of African economies?

MJ: The turn to RCT and from macro to micro I think can in great part be explained by the lack of reliable official data. But I think it can be a circumvention that will be costly in the medium and long term. Data does not only have a knowledge function, it has a governance function. Thus without macro statistics it is hard to have a public debate about trends and magnitudes of the economy, employment… These issues cannot be randomized. You cannot ask Togo to devalue and Benin to revalue their currency or otherwise randomize countries agricultural policies. It does matter for a government whether one is running out of food and/or foreign currency – and arguably it matters more than knowing exactly the net welfare benefit of subsidised school lunches in one school district.

If we turn more towards micro and RCT there may be lasting detrimental governance effects from not demanding and interrogating macroeconomic statistics.

Q4: In the book’s conclusion, you discuss the continued need for GDP data. Given all of the problems with producing reliable GDP statistics for so many poor countries, why do you think more accurate GDP numbers are so crucial and worth investing in? If you could convince international aid organizations and development economists to switch to some other measure or collection of measures, would you and what measures would you emphasize?

MJ: My answer here is in the first two sentences in Question 5 (below):
“For better or worse, GDP is heavily entrenched as a measure of the size of an economy. GDP is invoked almost automatically in any discussion of growth or development. Thus despite its known weaknesses its functional importance will not diminish.” There is also a need for GDP in domestic institutions such as the central banks.

I think I would advise that we get better numbers on wages, prices and employment. Those numbers matter for analysts, states and – most certainly– citizens, voters and consumers in African countries. In fact, I would advise that one followed similar collection methods that are used for Consumer Price Indexes in other measures of activity in production and service sectors as well.

Q5: For better or worse, GDP is heavily entrenched as a measure of the size of an economy. GDP is invoked almost automatically in any discussion of growth or development, and yet sometimes its role seems merely symbolic. Part of your argument, I think, is that GDP is important for both symbolic reasons, like our false sense of security about the relative rankings of the economic welfare of African countries, but also for practical policy purposes. What are your favorite or go-to examples of policy decisions that were made poorly or could have been made better with more accurate GDP data?

MJ: Well. There are some reports written in 2009, 2010 and 2011 that advised how much investment was needed in Ghana in order for the country to reach middle low income status. It turned out that all that was needed was to recount the GDP statistics. The huge uncertainty in GDP levels for two thirds of African economies right now makes a mockery of any attempt to rank or classify African economies according to income levels as the World Bank routinely do.

Right now there is the whole ‘Rising Africa’ debate which again is superficially based on observing that GDP has been rising in ‘Africa’ over the past decade. Many if not most commentators are relying blindly on the reported figures, and without checking the data basis for their claims and may thus run the risk of reporting statistical fiction.

Q6: Another topic you briefly touch on is the emergence of “night light” data. Several economists have attempted to use satellite measurements of light emitted at night to create alternative measurements of economic output. These efforts are fascinating to me, but also potentially a bit frustrating as they attempt to jump straight to an estimate of the final aggregate (GDP) without providing much information on all of the components of GDP that seem important for policymaking (the relative sizes of different sectors, etc.). What do you make of these data? What role do you think they will play in national accounting and studies of growth in Africa going forward?

MJ: I think some of these efforts are mostly going to amount to intriguing academic past time, but not have much relevance for policy makers for some of the reasons you quote there. However, new technologies will and do already make it cheaper to report data, and to compile more regular and reliable data. I think there is promise in correcting sampling frames in survey data through satellite pictures. As you say, I think that while these proxies can be ways of getting some information, they cannot fully substitute for actual economic and social data.

One of the problems we are approaching very quickly is that the GDP measure is losing its analytical distinctiveness – by using assets instead of production, using data on consumption instead of real income etc I think there is a case for ‘improving the GDP measure’ by taking into account for leisure, resource depletion and so on. On the other hand maybe it is good to hold the line and say – this is GDP- it does not measure all aspects, and indeed misses many aspects of increases in living standards, but fundamentally it is supposed to capture the trend in capacity to produce and consume goods and services in a geographical area.

Q7: Finally, I’m curious about the reception of your research among development economists and international organizations. Your book follows a string of excellent papers on related topics, going back about 5 years. How have development economists responded? Have your calls for better metadata in the big macroeconomic databases (World Development Indicators, Penn World Tables) yielded any fruit?

MJ: The World Development Indicators are already improving their practices. That’s good news. Still some way to go, but I think they are trying their best, while carefully guarding their credibility of the data they disseminate. Thus they cannot say – here’s a number, it is meaningless – so the extent to which they can spread doubt about the information they themselves are disseminating is limited.

Many development economists have surprised me by saying that, yes, you are right, thanks for investigating this. There will always be some resistance. My hope is that the book can end this divide between some scholars who will not touch the numbers and those who will always argue that any numbers are better than none.