‘A composite index’ is a common approach for tracking and/or measuring a particular concept or idea by combining multiple data series into a single measure. Global indices, that attempt to compare and rank a nation’s performance, typically try and measure something that can’t be observed. You’ll likely have seen these before in the form of Transparency International’s Corruption Perception Index, the Fund for Peace’s Fragile States Index and the UNDP’s Human Development Index (HDI). With each of these composite indices attempting to measure a broad idea or concept that is difficult to measure directly, such as the level of corruption, the stability of a country or the level of human development.
Often because the concept itself is complicated and ‘multi-dimensional’ the process for creating an index requires as much art as it does science: the analyst sets a definition for what is being measured, picks data relevant to this and combines them to create single score relevant to the chosen definition. For example the UNDP’s Human Development Index is guided by the idea that human development is guided by the reasonably idea that development is about having “a long and healthy life, being knowledgeable and having a decent standard of living“. The index is therefore calculated by combining national measures of life expectancy at birth, average years of education (or anticipated education for young children) and GDP per capita (I’m paraphrasing here: see link for details).
The UNDP’s final ‘human development index’ is then a combination of each of these measures; with high development being countries whose population has a higher average life expectancy, higher years of schooling and higher per-capita income. And while I’d say the UNDP’s HDI is one of the better indices out there, it’s not clear to me how informative they are from a practical standpoint. For instance, what constitutes progress will vary over time depending on the nation, community and individual: meaning that what constitutes ‘high development’ isn’t clear or set in stone: making it difficult to use a single approach across nations and over time. At the same time, even if there was agreement on the UNDP’s definition of development and its choice of indicators, dimensions need to be properly weighted to realistically account for tradeoffs between indicators. For instance, how much extra GDP per-capita would be needed to substitute for a year less of average life expectancy without changing a country’s score. Finally, as many composite indices have been created to measure something important that can’t be directly observed, once an index is created it’s not entirely clear to know how to evaluate its accuracy, making it difficult to know how serious to take each index, after all:
…Composite indicators are much like mathematical or computational models. As such, their construction owes more to the craftsmanship of the modeller than to universally accepted scientific rules for encoding.…
OECD’s index design handbook
I intend to write more on this topic in the future, but my suggestion is to view them as a communication tool, rather than evidence (not unlike a press-release). As an index is typically created using a framework that mirrors the belief systems and aims of the organization that publishes them.
For those interested in reading more I’d recommend checking out this paper.
Leave a Reply