After a two day-meeting in Bangkok, at the end of October, the statistical experts of the United Nations agencies and 28 countries have come with a controversial list of 159 “generally agreed” indicators to measure the Sustainable Development Goals approved last September by the UN General Assembly attended by Heads of State and Government.
Traditionally, development agencies have tried to summarize in a single indicator or index complex development goals. Thus, UNICEF emphasized infant mortality as a proxy indicator of its mandate to protect children, the World Bank has traditionally equated development with per capita income, and UNDP enriched that idea with its Human Development Index based on a larger but still reduced set of indicators. Those single figures were considered to correlate so close with other relevant indicators (for example infant mortality correlates with child underweight) that they were considered as useful to summarize broad trends, progress or regression, in realities that everybody understood as being complex.
Now, instead of attempting to find summary indicators for each of the 17 Sustainable Development Goals (SDGs), the experts are trying to find at least one (frequently more) indicator for each of the 169 targets that the governments agreed should describe the Goals or lead to their achievement. Of those targets, 121 have now at least one indicator marked as “green” (agreed) in the list (which add up to 159 indicators because many targets have more than one indicator), leaving 48 targets with "grey indicators (to be determined because more in-depth discussion and/or methodological development are still needed).
This approach seems to imply that each of the 17 Goals is equal to the sum of the targets agreed under it, that they are all equally relevant and exhaustive. But counting the trees can hide the forest instead of understanding it, as the popular saying wisely warns us. The analysis of the “green” goals shows this to be the case for many of the goals.
For example, Goal 10 (Reduce inequality within and among countries) proposes quite clearly and needed in its title to “reduce inequality within and among countries”. This is a very straightforward commitment. The governments did not agree on a benchmark on where to stop. At what point would inequalities be reduced so much that they acceptable? But if there was a worldwide political agreement that current inequalities are to be reduced, because they have reached intolerable levels, the indicators should at least help establish if reduction is indeed happening.
However, the list proposed does not include anywhere the Gini index of inequality, which is the well established scientific standard, nor the “Palma ratio” (comparing the top 10 per cent and the bottom 50 percent) that is easy to understand and popular in academia. Both these indicators already exist, are available at global and country levels for most countries and backed by growing literature.
Instead, the key indicators proposed as “green” for Goal 10 such as “people living below 50% of median income” or “ income per capita among the bottom 40” seem designed to carefully avoid looking at the high income end of the scale (the upper 10 or 1%) and completely ignore inequalities among countries.
Many of the targets are very precise and include the indicator in their formulation. This is the case of the first target of Goal 3 (Ensure healthy lives and promote well-being for all at all ages): “By 2030, reduce the global maternal mortality ratio to less than 70 per 100,000 live births”. Targets like these do not generate any major controversy.
Targets that require specific policies can be assessed by a binary indicator (‘yes’ or ‘no’). For example, the first target of Goal 5 (Achieve gender equality and empower all women and girls) demands to “End all forms of discrimination against all women and girls everywhere”. And the proposed “green” indicator is a binary one: “Whether or not legal frameworks are in place to promote equality and non-discrimination on the basis of sex.” The implementation of those policies and their impact will still have to be assessed but their very existence is a starting point that needs to be monitored.
But other targets are programatic statements, that imply a multiplicity of policies and potential results that cannot be captured within the self-imposed restriction of one or two indicators per target. See, for example, the 4th target of Goal 1 (End poverty in all its forms everywhere): “By 2030, ensure that all men and women, in particular the poor and the vulnerable, have equal rights to economic resources, as well as access to basic services, ownership and control over land and other forms of property, inheritance, natural resources, appropriate new technology and financial services, including microfinance”.
This is one of “grey” cases. The experts asked UNEP (the UN Environment Program) “to review the ontology about basic services.” But even if this review comes out with an acceptable assesment tool, basic services are only one of the many relevant issues spelled out of this target and key issues like access to property, technology and finances will be left out.
Under Goal 12, which addresses “sustainable production and consumption patterns” the indicators agreed upon refer to the existence of policies in place but the key indicator of “footprint” which measures actual impact was left as “grey”.
Finally, the indicators for Goal 17, on Means of Implementation (MoI) and for the targets around MoI in the other 16 SDGs are those that proved most controversial in intergovernmental discussions so far. Not only do they have the greatest number of greys and yellows, but when indicators are listed as “green” they frequently miss the point or completely distort the intention of the target.
As if the name of “means of implementation” was not enough, the very first MoI target (1.a.) requests explicitly to “ensure significant mobilization of resources from a variety of sources, including through enhanced development cooperation, in order to provide adequate and predictable means for developing countries, in particular least developed countries, to implement programmes and policies to end poverty in all its dimensions”.
The proposed indicators, labeled as green, measure the “percentage of resources allocated by the government directly to poverty reduction programmes” and the “spending on essential services (education, health and social protection) as % of total government spending”. Thus, developing country governments will be judged negatively if they are unable to spend more on the poor, but the promise of “enhanced development cooperation” will be left unmonitored.
Target 17.1 is to “Strengthen domestic resource mobilization, including through international support to developing countries, to improve domestic capacity for tax and other revenue collection”. But the proposed indicator is “total tax revenue/GDP”, which again measures the ultimate expected result, but not the promised international support.
Target 17.3 is even more problematic. The target is to “Mobilize additional financial resources for developing countries from multiple sources” but the indicators proposed are "Foreign Direct Investments as % of total FDI + ODA" and “additional Volume of remittances (USD)/GDP.” This indicator might artificially inflate the accounted contribution of developed countries.
On the one hand the OECD itself recognizes that "microeconomic or macroeconomic impacts of remittances are controversial and the extent to which these flows contribute to development is still not clear". On the other hand, not every FDI contributes to development and if capital inflow weighs positively in the balance of payments, it is the net balance what counts and thus outflows (profits, royalties, etc.) should not be ignored.
Target 17.4 to “Assist developing countries in attaining long-term debt sustainability” has as indicator “Debt service as a percentage of exports of goods and services”, ignoring again to look at the promised assistance.
The main point of Target 17.6 is to “Enhance (...) cooperation on and access to science, technology and innovation and enhance knowledge sharing (...) including through improved coordination among existing mechanisms (...) and through a global technology facilitation mechanism.”
Instead of a simple yes or no to the question of whether such a mechanism or an assessment of cooperation was enhanced, the proposed indicator is the “access to the WIPO Patent
Database and use of the international IP system”. This is adding insult to injuty, because the current global intellectual property system is precisely the obstacle that this goal needs to overcome.
Target 17.8 that wants to “Fully operationalize the technology bank” is measured by the proportion of individuals using the Internet.
Key demands like enhancing global macroeconomic stability (which is important everywhere and not just for developing countries), or “enhance the global partnership (among countries) for sustainable development” still lack agreed indicators, while the promotion of “effective public, public-private and civil society partnerships” will be measured by the money “committed on public-private partnerships”, not assessing whether those funds were actually disbursed and the impact of those funds on the achievement of Goal 17. In addition, while three different categories of associations are identified, there is nothing on civil society and exclusively public initiatives.
The commitment to “respect each country's policy space and leadership to establish and implement policies for poverty eradication and sustainable development ” is left without an agreed indicator, even when developing countries suggested to use a simple count of the “numbers of constraints” (conditionalities) that are embodied in ODA or loan agreements as well as investment and trade agreements.
This work on indicators is undertaken by an Inter-agency and Expert Group on SDG Indicators (IAEG), composed of 28 UN Member States (based on geographical representation) with regional and international agencies as observers. The IAEG will continue work on the “greys” until 15 February 2015 and provide a proposal of a global indicator framework (and associated global and universal indicators) to the UN Statistical Commission at its forty-seventh session in March 2016.