October 20 is World Statistics Day and a good occasion to reflect on the ever growing attention to statistics. Not only does the old adage “if you cannot measure it you cannot manage it” hold true, but increasingly we suspect that if “you do not measure it you do not pay attention to it.” However, reliable statistics remain expensive to produce and much of the information base that the richest countries take for granted remains out of reach for poorer countries due to limited financial and human capacity. It is, therefore, important that data already collected is put to good use. This means turning available data into accessible information, presented in ways that enhance understanding. Indeed, the Human Development Report Office (HDRO) has recently introduced new data visualization tools on our website to promote and stimulate informative use of internationally comparable national data.
In this article we consider some principles for developing measurements, starting with the case for good statistics.
In the 18th century, the German word Statistik meant ‘analysis of data about the state.’ Statistics is now used to describe numerical information, its collection, analysis and presentation. The latter can include both statistical indicators (a series of data points used to draw comparisons over time, between places, populations etc.) and statistical indices (measures combining two or more statistics into one number, such as the body mass index).
Construction of indicators and composite indices is part science, part art and considerable innovation. When designing a new indicator or an index, such as the Human Development Index (HDI), three fundamental aspects apply: the concept we seek to quantify; the purpose for which the number will be used; and the audience we want to reach.
1. Quantification
Some indicators should be quite simple to construct because they are well defined or lend themselves to straightforward statistical measurement, e.g., the proportion of people in a country who are over 65 years old. But not everything that counts is counted, and many important aspects of life can be challenging to measure. We’ll mention three reasons why.
It can be:
- Conceptually challenging to express complex concepts in a way that lends them to straightforward quantification e.g., social capital or human development.
- Statistically challenging to measure aspects of life that are precise but complex –e.g., biodiversity is well defined – representing the genetic variety of life on earth - but almost impossible to measure precisely given current levels of understanding.
- Practically challenging because decent data does not exist in many countries (e.g., measuring income poverty requires detailed household surveys that are unavailable in many places because statistics are expensive to produce).
2. Purpose
Indicators can be used for different purposes by different people. One popular classification describes three broad purposes for which they are used . Some indicators are primarily instrumental, that is the value or direction of movement of an indicator can lead to action or change in a linear and deterministic way. Other indicators are primarily conceptual – they seek to quantify an aspect of life to inform debate and discussion. For example, the HDI presents an alternative view of national progress to that presented by Gross Domestic Product (GDP) and captures elements of “human development”. While other indicators are primarily political – they seek to justify and legitimize actions (e.g., the crime rate in a country can frame debates on law and order)
At this point it is important to recognise that, no matter what their primary purpose is, indicators are inherently political. While many statisticians see themselves as impartial, it is also the case that the choice of what goes in to an indicator can have a profound effect on debate and decisions. Consider for example an indicator of national CO2 emissions. One might measure these as emissions produced within national borders, or as the emissions associated with the production and consumption of goods and services within a country. The former indicator would show national progress if countries imported electricity for example from abroad, even if the production of the power increased global emissions. The latter indicator would do more to measure domestic contribution to global warming, though would also exclude emissions associated with exports. Each measure has strengths and weaknesses, but the essential point to recognise is that each indicator would encourage different policy responses.
3. Audience
Finally we must consider the audience we seek to reach. This is closely linked to how an indicator would be used. The sort of instrumental indicator used for detailed policy making by experts might need a lot of complexity to be useful. The opposite might be true for a political indicator which needs to resonate – and so be readily interpreted and (hopefully) understood – by the population as a whole. This can often represent a challenge to statisticians who might be uncomfortable with measures that are less than perfect. But “the perfect is the enemy of the good”, or it can be when it comes to communicating information.
Chart 1: Audience versus complexity of data (From Scrivens and Iasiello (2010)
This has been the story of the HDI. Mahbub ul-Haq, the author of the first human development reports, argued that the single-number approach was necessary to get people interested in the underlying data. Amartya Sen later wrote “Mahbub got this exactly right, I have to admit, and I am very glad that we did not manage to deflect him from seeking a crude measure….The crude index spoke loud and clear and received intelligent attention and through that vehicle the complex reality contained in the rest of the Report also found an interested audience.”
When it was introduced in 1990 it was an innovation in measuring development, which was almost exclusively assessed by economic aggregates such as GDP, which Robert Kennedy noted, “measures everything in short, except that which makes life worthwhile.” Twenty five years later the index is arguably still the best-known complement to GDP as a measure of national progress, though HDRO is the first to recognise its imperfections.
Yet, while one can criticise the index, it is less easy to not recognise the positive outcomes it has helped generate. The HDI has led to greater attention in development debates on aspects of life beyond economic growth, and has challenged the popular belief that economic growth is the only path to development. And, as an important aside, the importance of the HDI rankings has also encouraged NSOs and governments to place more attention on the quality of the underlying data.
Chart 2 – Human Development Trends 1980-2014, selected countries
To stimulate a broader discussion on barriers to human development HDRO now publishes a family of human development indices, including the Inequality-adjusted HDI, which accounts for inequality in the three HDI dimensions (health, education and income); the Gender Inequality Index, which reflects inequality in basic achievements between women and men; the Gender Development Index which measures the human development gender gap, and the Multidimensional Poverty Index which quantifies poverty in ways that go beyond income.
The new data tools on the HDRO website, examples of which are used in this article, bring data for assessing human development to users in new ways. We suggest you use them to paint your own picture of global human development on World Statistics Day.
Chart 3 - Index comparison for a selected country
Photo credit: Creative Commons
The HDIalogue blog is a platform for debate and discussion. Posts reflect the views of respective authors in their individual capacities and not the views of UNDP/HDRO.
HDRO encourages reflections on the HDIalogue contributions. The office posts comments that supports a constructive dialogue on policy options for advancing human development and are formulated respectful of other, potentially differing views. The office reserves the right to contain contributions that appear divisive.
i Stiglitz, Sen, Fitoussi (1999) Report on The Measurement of Economic Performance and Social Progress. http://www.insee.fr/fr/publications-et-services/dossiers_web/stiglitz/do...
ii http://academlib.com/13661/political_science/indicators_policy_formulation
iii Scrivens, K. and Iasiello, B. (2010) Indicators of "Societal Progress" Lessons from International Experiences. OECD Statistics Working Papers. DOI: 10.1787/18152031
iv Amartya Sen (1999). Human Development Report 1999. Special contributions: Assessing human development (p.23). http://hdr.undp.org/sites/default/files/reports/260/hdr_1999_en_nostats.pdf
v Robert F. Kennedy (1968). Speech at the University of Kansas. https://www.jfklibrary.org/Research/Research-Aids/Ready-Reference/RFK-Sp...
