Optimize with Caution
Resisting the allure of in-sample backtesting
Summary: Modern Portfolio Theory (MPT) coupled with the Sharpe Ratio provide a simple way to build optimal portfolios. However, as discussed in Part 1, MPT's inputs are typically based on past market data, which can yield optimization results that would be foolish to implement in actual endowment portfolios. Therefore, avoid optimizing as it can be a siren song, or do so using a disciplined, best-practices process.
This post doesn't go into detail about robust back-testing methodologies, but many resources for that can be found online. Instead, it highlights how optimization results can change dramatically with small changes to the input data's date range, hurdle rates, etc., which can fool investors into constructing portfolios that perform poorly in the future.
Index Data Utilized
The data used in this analysis is not comprised of individual stocks. Instead, MPT and Sharpe Ratio optimization is applied to a portfolio of a dozen indices of major asset classes (monthly return data from 2/2002 thru 3/2018). Below are the compounded returns of the indices used.
Index Data Utilized
The data used in this analysis is not comprised of individual stocks. Instead, MPT and Sharpe Ratio optimization is applied to a portfolio of a dozen indices of major asset classes (monthly return data from 2/2002 thru 3/2018). Below are the compounded returns of the indices used.
A Caution on Backtesting Strategies using Market Data that Spans Oct 2008
Avoiding the Oct 2008 plunge via market-timing strategies is very easy to do while backtesting. As you can test for yourself, the optimized parameters will usually avoid being invested in the market during Oct 2008. But, the details of how each bear market plays out are unique, and a strategy that was optimized to miss the Oct 2008 plunge while still capturing most of the subsequent upside may fail to avoid the next bear market, and may miss significant upside moves in the future. Statistically, the sample size of bear markets is generally too small to have confidence in market-timing models optimized to avoid them. Plus, the markets continually evolve, rendering the details of past market events less useful.
MPT Plots and Observations
Methodology
1. 186,624 portfolios are analyzed for each scenario
2. All indices are assigned the same initial weight in the portfolio, and each index is assigned its own max weight. More liquid indices like the SPX or US Bonds are given higher max weights than less liquid indices like MLPs or Commodities
3. The resulting risk/reward of the portfolios are plotted in MPT style, with the Efficient Frontier being the top perimeter of the plotted portfolios.
4. Each portfolio's Sharpe Ratio is indicated by color, with the highest Sharpes in red. The portfolio with the max Sharpe Ratio is identified with a blue dot. Sharpe Ratio = (Annualized Portfolio Return minus Risk Free Interest Rate) / Std Dev of Returns
5. The risk-free rate used in the Sharpe Ratio calculation is the 3 month T-bill rate.
6. The asset allocation that generated the maximum Sharpe Ratio is shown in the accompanying bar graph.
Plot 1: The Optimal Sharpe Ratio Portfolio for 2003-2018
Date range: Jan 2003 - Mar 2018
0% annual draw (spending policy)
0% leverage
As you can see in the Asset Allocation bar graph, US Bonds, Leveraged Loans, and Catastrophe Bonds (Cat Bonds) were most heavily weighted, with the remaining indices weighted at their initial allocation. This heavy weighting of bonds is partially due to this scenario's lack of a minimum return constraint.
1. 186,624 portfolios are analyzed for each scenario
2. All indices are assigned the same initial weight in the portfolio, and each index is assigned its own max weight. More liquid indices like the SPX or US Bonds are given higher max weights than less liquid indices like MLPs or Commodities
3. The resulting risk/reward of the portfolios are plotted in MPT style, with the Efficient Frontier being the top perimeter of the plotted portfolios.
4. Each portfolio's Sharpe Ratio is indicated by color, with the highest Sharpes in red. The portfolio with the max Sharpe Ratio is identified with a blue dot. Sharpe Ratio = (Annualized Portfolio Return minus Risk Free Interest Rate) / Std Dev of Returns
5. The risk-free rate used in the Sharpe Ratio calculation is the 3 month T-bill rate.
6. The asset allocation that generated the maximum Sharpe Ratio is shown in the accompanying bar graph.
Plot 1: The Optimal Sharpe Ratio Portfolio for 2003-2018
Date range: Jan 2003 - Mar 2018
0% annual draw (spending policy)
0% leverage
As you can see in the Asset Allocation bar graph, US Bonds, Leveraged Loans, and Catastrophe Bonds (Cat Bonds) were most heavily weighted, with the remaining indices weighted at their initial allocation. This heavy weighting of bonds is partially due to this scenario's lack of a minimum return constraint.
Plot 1: The Optimal Sharpe Ratio Portfolio for Jan 2003 - Mar 2018
Plot 2: Higher Return Requires Higher Risk
Date range: Jan 2003 - Mar 2018
4% annual draw (spending policy)
0% leverage
The only difference between Plot 2 and Plot 1 is the 4% spending rule's draw. Rather than select the Efficient Portfolio than earned 4% more than Plot 1, the 4% draw was used as a hurdle rate in the Sharpe Ratio optimization (by subtracting it from the portfolio's return). Notice the increase in the max Sharpe Ratio portfolio's return and risk. Also, notice the allocation changes, notably that bonds decreased and equities increased, with the S&P 500 now having the largest allocation.
Date range: Jan 2003 - Mar 2018
4% annual draw (spending policy)
0% leverage
The only difference between Plot 2 and Plot 1 is the 4% spending rule's draw. Rather than select the Efficient Portfolio than earned 4% more than Plot 1, the 4% draw was used as a hurdle rate in the Sharpe Ratio optimization (by subtracting it from the portfolio's return). Notice the increase in the max Sharpe Ratio portfolio's return and risk. Also, notice the allocation changes, notably that bonds decreased and equities increased, with the S&P 500 now having the largest allocation.
Plot 2: Higher Return Requires Higher Risk
Plot 3: Change the Dates, Change the Results
Date range: Feb 2002 - Mar 2018
4% annual draw (spending policy)
0% leverage
In this plot, Feb-Dec 2002 is added to Plot 2's data. Because 2002 included the equity bear market related to Enron's collapse, most of the equity indices' Sharpe Ratios declined enough that the optimizer gave them a minimum allocation (except for emerging markets).
Date range: Feb 2002 - Mar 2018
4% annual draw (spending policy)
0% leverage
In this plot, Feb-Dec 2002 is added to Plot 2's data. Because 2002 included the equity bear market related to Enron's collapse, most of the equity indices' Sharpe Ratios declined enough that the optimizer gave them a minimum allocation (except for emerging markets).
Plot 3: Change the Date Range of the Data, Change the Results
Plot 4: Add Leverage - which indices get higher allocations?
Date range: Feb 2002 - Mar 2018
4% annual draw (spending policy)
50% leverage
Similar to levered Risk Parity portfolios, when 50% leverage was added to Plot 3, most of the allocation boost went to bonds. Note: the index weights were normalized by multiplying by (1 + Leverage)/(sum of index weights), so that the weights sum to 100% plus the leverage added.
Date range: Feb 2002 - Mar 2018
4% annual draw (spending policy)
50% leverage
Similar to levered Risk Parity portfolios, when 50% leverage was added to Plot 3, most of the allocation boost went to bonds. Note: the index weights were normalized by multiplying by (1 + Leverage)/(sum of index weights), so that the weights sum to 100% plus the leverage added.
Plot 4: Adding leverage resulted in a higher allocation to bonds - for this time period at least
Note: I'm not advocating levered Risk Parity. Bonds have been in a long bull market, and with estimated investment in the strategy as high as $400 billion, doesn't it seem likely that bonds will eventually enter a period of positive correlation with equities? With the potential for both asset classes to simultaneously decline in a global, risk-off event?
Plots 5 thru 8: In-Sample vs. Out-of-Sample
Plot 5: The Optimal Sharpe Ratio Portfolio for Feb 2002 - Feb 2009
4% annual draw (spending policy)
0% leverage
Plot 5: The Optimal Sharpe Ratio Portfolio for Feb 2002 - Feb 2009
Plot 6: The Optimal Sharpe Ratio Portfolio for Mar 2009 - Mar 2018
4% annual draw (spending policy)
0% leverage
4% annual draw (spending policy)
0% leverage
Plot 6: The Optimal Sharpe Ratio Portfolio for Mar 2009 - Mar 2018
In-Sample vs. Out-of-Sample
Plots 7 & 8: the compounded return for both periods, using both of the optimized allocations.
Plots 7 & 8: the compounded return for both periods, using both of the optimized allocations.
Conclusions/Recommendations?
- Follow your advisor's counsel
- Avoid switching advisors frequently. Switching to advisors with the best recent track record, and then to the next hot hand, and then to the next is akin to market-timing, which generally results in sub-optimal performance over time.
- Spend some time researching robust back-testing methods on sites like Quantopian.
- For long-term endowment portfolios, optimize on multi-decade datasets that span several business cycles.
- Be disciplined about out-of-sample testing, and hold off on using it until you are convinced that your optimization is complete. If you test an optimized portfolio on your out-of-sample data, then that data will no longer be out-of-sample, and from that point on, any new changes made to your portfolio will essentially be in-sample changes.
And, always treat portfolio optimization with caution!
Treat snappers, and portfolio optimization with caution
Wade Vagle, CFA, CAIA
Get in touch at Wade@SchoolsThatLast.com