World Bank poverty data, methodology faulted

Two US academics have disputed the validity of the World Bank’s global poverty statistics, exposing serious conceptual and methodological shortcomings in its calculations of poverty levels worldwide. Use of these flawed figures, they contend, may have led to a substantial underestimation of the extent of world poverty and to mistaken inferences that global income poverty has fallen.

by Chakravarthi Raghavan

GENEVA: The World Bank’s global poverty statistics, based on income and purchasing power parity, are produced using wrong methodologies and are unreliable for estimating levels, distribution and trends of global poverty, according to two academics of Columbia University in New York.

The issue of poverty figured prominently in the run-up to and at the Copenhagen Social Summit of 1995, where, under the pressure of civil society groups, the Summit said that each country must define an absolute poverty line and aim to eradicate it by a target date to be set by each country.

The Washington-based international financial institutions and Washington itself (both pushing the Washington Consensus and, after the Mexican peso crisis, pushing the same policies under a different name, “globalization”) as well as the Organization for Economic Cooperation and Development (OECD) had not been keen on the Copenhagen Summit in the first place, and were not at all happy when that meeting came up with 10 commitments.

The World Bank then hijacked the poverty issue to argue that the “good policies” (of the Washington Consensus), with some more aid, reduced poverty in countries and that the Bank was best situated to deliver these.

With the International Monetary Fund (IMF), and the UN and UN Development Programme (UNDP), soon joining to have a piece of the action, a new combination emerged by the time of the Copenhagen+5 Summit in Geneva in 2000 to push and promote the Fund-Bank policies and provision of more aid by the OECD countries to countries which practised the “good policies.”

To show that their policies are working, the World Bank defined (a global level of) absolute poverty on the basis of incomes to satisfy the “most compelling of human desires”, and then refined it in statistical terms by defining absolute poverty as per capita income (in purchasing power parity, PPP, terms) of $1 a day in Asia and Africa, and $2 a day in Latin America and the Caribbean. It called for halving the poverty levels by 2015 and for aid to be provided and channelled based on good policies. On this basis, the Bank has been calculating and publishing data on poverty in countries to claim that global poverty is being reduced.

Civil society groups are not equipped to challenge these methodologies, but at Copenhagen+5 challenged the claims of poverty reduction on the basis that at country level there was not much indication thereof and the social goals set at Copenhagen were in fact slipping.

The poverty estimates had a utility other than in policy debate terms. They became the basis for providing aid (even when bilateral aid policies of the US, Europe and Japan had no relationships to these but were based on alliance, military and strategic considerations).

Methodology challenged

The Least Developed Countries Report 2002, recently published by the UN Conference on Trade and Development (UNCTAD) (see TWE #283), questioned the World Bank’s poverty data calculated from household surveys, and said these understated the extent of poverty in Africa, particularly sub-Saharan Africa (where most of the LDCs are). It suggested that a better method would be to use national income statistics and that aid should be allocated on that basis.

The report did not challenge the income approach to judging poverty, merely whether it should be calculated by using national income data or household surveys (or a mixture of both).

The two US academics, Sanjay G. Reddy and Thomas W. Pogge, have challenged the methodology used by the Bank and given a scathing account of the problems with the Bank’s poverty numbers (bringing out, incidentally, that poverty is more widespread in South Asia and some other Asian countries than the Bank has calculated).

In a paper published on the website, Reddy and Pogge say that the estimates of the extent, distribution and trend of global income poverty provided in the Bank’s World Development Reports for 1990 and 2000/01 “are neither meaningful nor reliable.”

The Bank, the two academics charge, uses “an arbitrary international poverty line unrelated to any clear conception of what poverty is, employs a misleading and inaccurate measure of purchasing power ‘equivalence’ that creates serious and irreparable difficulties for international and inter-temporal comparisons of income poverty, and extrapolates incorrectly from limited data and thereby creates an appearance of precision that masks the high probable error of its estimates.”

The academics are “surprised” that the Bank has been publishing regular poverty statistics for 12 years now, “precise” to six digits and very widely used in academic publications and popular media all over the world, without significant attention having been paid to the massive flaws in its procedures. It is hard not to see this fact as indicative of the low priority that has hitherto been attached to the global problem of persistent severe poverty.

As a result of these flaws, “there is a large understatement of the extent of global income poverty and ... an incorrect inference that it has declined,” Reddy and Pogge say, suggesting that “a new methodology of global poverty assessment is feasible and necessary.”

In the 1990 World Development Report (WDR), and subsequently in WDR 2000/01, the two academics say, the World Bank presented groundbreaking comprehensive estimates of the number of poor people in the world as a whole and in its regions, in different years. These estimates have been widely accepted and employed for a range of policy analyses and assessments.

A primary conclusion of WDR 2000/01 is that the world is indeed on the right track.

The two academics note that estimates of the number of poor people in the world also influence assessments of the seriousness of the problem of world poverty, the scale of resources that should be devoted to reducing it, and the regions to which these resources should be directed. WDR 2000/01 argues, for example, that the largest number of the world’s poor are now in Africa rather than South Asia.

The questions of how many poor people there are in the world, how poor they are, where they live, and how these facts are changing over time are clearly very important ones. The Bank’s estimates of global income poverty are influential not only because of their importance and usefulness, but because the Bank is currently the sole producer of such estimates.

Conceptual errors

Reddy and Pogge argue that the Bank’s estimates of global poverty should not be accepted, since “these estimates are flawed due to three related but distinct types of significant conceptual errors, which make it impossible to use them to identify with any reasonable accuracy the level, distribution or trend of global poverty.”

The first type of error, the two academics say, involves the failure to define a global poverty line that corresponds to a clear underlying concept of poverty, so as to allow identification of the commodities that must be commanded in order to avoid being poor. The second type of error involves the failure to employ purchasing-power-parity factors that permit meaningful and accurate identification of the national currency equivalents of the global poverty line, and of changes in their value through time. The third type of error involves the building into the methods used of false precision and mistaken inferences, in the face of data limitations.

“These three types of error together lead to the likelihood of substantial distortions in estimates of global poverty.” While it is impossible to identify precisely the extent and direction of the gap between the Bank’s estimates and those that would have been yielded by a sounder methodology, simple estimation procedures suggest however “that the biases may be very substantial, and may have led to a substantial underestimation of the extent of world poverty.” These errors may also have led to false claims about downward trends in global income poverty and “the mistaken inferences that global income poverty has fallen.”

The two academics say that it is not difficult to describe a methodology for assessing global poverty that would inspire substantially greater confidence, and the adoption of such a sound methodology may not require significantly greater effort or resources than that actually expended in producing current estimates. “The current estimates should no longer be employed, and new ones corresponding to a defensible methodology, covering past as well as current years, should be generated.”

Income poverty is only one aspect of poverty, and other poverty estimates, based on under-nutrition, infant mortality, access to health services, and other indicators, can continue to inform us even in the absence of usable figures concerning global income poverty. International development targets should appropriately continue to focus on these measures of deprivation in the world, while a new procedure for the global assessment of income poverty is developed and implemented.

The procedure normally used in national exercises of poverty assessment, Reddy and Pogge say, is to define a poverty line in terms of the cost of meeting a set of minimal criteria. These minimal criteria are most often defined in terms of access to some set of basic (for example, nutritional) capabilities or of commodities that generate such capabilities. The commodity requirements specified by their relation to a salient set of elementary capabilities in this way can in principle be allowed to vary across groups of persons (defined, for instance, by age, gender and other factors). Procedures of this general kind have the advantage that, once fixed, they offer a consistent basis for determining the level of the poverty line across time and space. Of course, they also face difficulties. When specified too rigidly, a commodity-specification approach may fail to take note of the substitution possibilities available to the poor when prices change, which may enable them to escape poverty at lower incomes than suggested by updating the cost of a fixed bundle.

Such approaches may also fail to account for the fact that escaping poverty may require additional commodity resources when a society becomes more prosperous. A capability approach seeks to take account not only of people’s command over commodities, but of what they can do with these commodities, and therefore of their diverse needs, handicaps and endowments. It should therefore ideally find a plausible way of gathering and integrating such information, although it may choose to limit the attention to such diversity as a matter of informational or resource constraints. The approach of defining a poverty line in terms of the resources necessary to achieve a salient set of elementary capabilities offers a feasible methodology for consistent poverty assessment across time and space, and for this reason has been widely employed.

The methodology pursued by the Bank in its landmark 1990 and 2000/01 global poverty assessment exercises does not correspond to such a procedure, say Reddy and Pogge in explaining the method adopted by the Bank (and outlined in the Bank staff papers, but merely footnoted in the Bank’s publications and to which few, including critics, pay attention).

In 1990, the Bank constructed a global poverty line from a set of official domestic poverty lines for 33 countries surveyed during the mid-1980s. These official domestic poverty lines were scaled upward or downward by changes in the national consumer price index to their “equivalent” in 1985 national currency units. These 1985 national currency units were then converted into a common unit of “real purchasing power” equivalence using the 1985 purchasing-power-parity conversion factors for consumption (expressed in US dollars) calculated by R. Summers and A. Heston in 1988. A global poverty line of $31 per month was chosen on the basis that the official domestic poverty lines of eight of the poorer countries in the sample, converted into dollars in this way, were very close to it. This “most typical” poverty line became the “$1 (PPP 1985) a day” (actually $1.02 PPP 1985) global poverty line applied in WDR 1990 (and subsequently revised downward, without justification, to $1 in Chen, Datt and Ravallion (1994) in the World Bank paper “Is poverty increasing in the developing world?”). This uniform poverty line was then converted to the national currency units of different countries using the Penn World Tables PPP conversion factors for 1985. The resulting poverty line was then inflated or deflated as appropriate by changes in the national consumer price index (as reported in the IMF International Financial Statistics) and applied to household survey data to create a measure of the number of poor persons in a country in a particular year.

However, for the 2000 poverty estimates exercise, the Bank established a new poverty line. For the same list of 33 countries used earlier, the Bank identified the 10 with the now lowest poverty line when converted to the 1993 $PPP, using consumption factors for conversion, and chose  the median of these poverty lines - $1.08 PPP 1993 a day - as its new  poverty line. The Bank has provided no justification for this change.

It is impossible to say whether the new poverty line is “higher” or “lower”, say Reddy and Pogge, since PPP dollars corresponding to different years are not in any way comparable. However, these methodologies result in some confusion. The methodological shift in global poverty assessment from WDR 1990 to WDR 2000/01, they say, entailed significant changes in the extent of poverty deemed to exist in many countries and regions. Though the Bank claims that the total number of poor persons under the two methodologies is roughly the same, in view of the magnitude of the shift in the regional composition of poverty caused by the change in methodology, there is no reason to expect that the total would continue to be similar for subsequent years.

Calling for the adoption of new procedures urgently to calculate the poverty levels, Reddy and Pogge say that there are “strong reasons to doubt the reliability and meaning of the estimates of the level, distribution and trend of global poverty” provided in WDR 1990 and WDR 2000/01.

The reasons for doubt, they say, revolve around the lack of a well-defined poverty line that permits meaningful and reliable inter-temporal and inter-spatial comparisons, the use of misleading and inaccurate measures of purchasing power equivalence and building into the methods use of false precision and mistaken inferences in the face of data limitations. All these flaws are likely to “systematically distort” the trend of global income poverty, and much of the distortion is in the direction of understating the extent of poverty in the world.

Moreover, say Reddy and Pogge, statements that global poverty is decreasing have no evidential justification in the light of these distortions. “The problems are readily avoidable,” although their avoidance would require a fundamental change in the methodology of global poverty assessment.

The rejection of the Bank’s procedure does not, however, support the sceptical conclusion that the attempt to provide a standard of income poverty comparable across time and space is doomed to fail. Much better procedures are available and, given the modest informational and institutional requirements, can be easily implemented.

As an alternative procedure, Reddy and Pogge suggest the construction of a reference basket of commodities containing relevant characteristics (for e.g., calorific content) that enable them to meet the elementary consumption needs of individuals. The international poverty line is then defined as the amount of national currency minimally necessary to each country or more specific locality to purchase this reference basket.

This procedure would focus on whether the incomes of poor people are sufficient, not in relation to all prices everywhere but rather in relation to the local prices of goods that are relevant to meeting their elementary requirements. The reference basket should be composed of commodities that are defined in a suitably abstract way so as to take reasonable account of local variation in tastes, while also possessing the characteristics that enable elementary requirements to be met. (SUNS5180)                                  

From Third World Economics No. 287 (16-31 August 2002)