11 Feb History Is Written By The Victors
Photo Pixabay
When we think about the heroes of WWII, our inclination is naturally to consider first the service men and women who gave their lives, the resistance fighters, and the leaders of our nation, such as Winston Churchill.
Beyond that, we might consider those who risked their lives who had no reason to do so, such as Oskar Schindler, or those responsible for technological advances which helped us to win the war, such as Alan Turing.
Sitting firmly in that last category is Abraham Wald. Wald graduated from the University of Vienna in 1931, with a PhD in mathematics, but was unable to find a suitable position at a university in his home country because he was Jewish.
Wald was able to immigrate to the United States, joining the Cowles Commission for Research in Economics, to work on econometrics research.
During World War II, Wald became a member of the Statistical Research Group and one of the problems that he worked on was how to minimize bomber losses to enemy fire. The US Navy’s view was to provide greater protection to the parts of the aircraft that received more damage. However, Wald noted that their studies only considered the aircraft that had survived their missions—the bombers that had been shot down were (obviously) not present for damage assessment. The holes in the returning aircraft represented areas where a bomber could take damage and still return home safely.
The armour, said Wald, doesn’t go where the bullet holes are. It goes where the bullet holes aren’t.
Wald proposed that the Navy reinforce the areas where the returning aircraft were not damaged, since those were the areas that, if hit, would cause the plane to be lost.
What Wald was trying to prevent was for a plan of action to be executed based on the data provided by the planes which made it home – the ‘survivors’. Instead, he advised the Navy to focus on those planes which did not survive. As a result of Wald’s work, countless lives were saved.
Mathematicians call the method of analysing data which the Navy was using ‘survivorship bias’.
David McRaney, in his TedX talk, calls this ‘Missing what’s missing’. Survivorship bias affects us in multiple ways as we go about our daily lives but can also affect our investment decisions.
The most obvious example of survivorship bias from an investment perspective occurs when looking at past performance data.
For example, imagine you are trying to identify the proportion of active funds which outperform their benchmark index in a given year. If, for example, 10 of the funds available at the start of the year close during the year due to poor performance, the number of available funds would have reduced to only 90 funds. Because the funds that dropped out had worse than average performance, omitting them from the results will overstate the average performance of the remaining 90 funds.
Why does it matter?
It matters because ignoring the funds which were closed (and the number one reason for a fund closing is underperformance) skews the data. If these underperforming funds are excluded from the data set, this will result in an exaggeration of the degree to which a given sector of active managers can outperform their benchmark.
Vanguard has carried out several analyses of this issue on the various investment sectors available in the UK. In their latest analysis (‘The case for low-cost index fund investing’ April 2018) covering the period from January 2003 to the end of 2017, they confirmed that this is exactly what happens.
Vanguard also looked at whether it was fair to assume that the funds which were closed during the period analysed were closed due to underperformance. When funds are closed or merged, can you really assume it is about performance?
To test this, Vanguard assessed the performance of all the funds identified by Morningstar as either being liquidated or merged into another fund. They then calculated the closed funds’ excess returns versus a broad market benchmark from January 2003 up until the month-end prior to the fund’s date of closure. *
Whilst it’s fair to say that not all the funds were closed or merged due to underperformance, given the negative median return across all sectors, it would seem safe to assume that on average it appears that they were:
Chart: Vanguard
They then looked at the performance figures once you add back the funds which were closed during the period, and account for their performance. As the chart below illustrates, the number of funds underperforming their benchmark is greater than the number of outperforming funds in all sectors.
Chart: Vanguard
How can you avoid succumbing to survivorship bias?
Firstly, when looking at performance data, make sure that the data accounts for funds which were closed during the period so that you can be certain that the historical data is not being overstated. Think about whether you are ‘missing what’s missing’.
If the numbers haven’t been adjusted for survivorship bias, they will probably look better than they really are.
Or as Winston Churchill put it, “History is written by the victors.”
And always remember that past performance is no guarantee of future results.
Carolyn
*Benchmarks used in Vanguard’s study:
UK equity FTSE All-Share Index
US equity MSCI USA Investable Market Index
European equity MSCI Europe Investable Market Index
Global equity MSCI World Investable Market Index
Emerging market equity MSCI Emerging Markets Investable Market Index
Eurozone equity MSCI EMU IMI
Global bonds Bloomberg Barclays Global Agg (GBP)
GBP diversified bonds Bloomberg Barclays Sterling Agg (GBP)
GBP government bonds Bloomberg Barclays Gilts (GBP)
EUR diversified bonds Bloomberg Barclays Euro-Agg (GBP)
USD diversified bonds Bloomberg Barclays US Agg (GBP)