Notes from the Trenches,

Statistical & Machine Learning Approaches to Investment Management May 2019

# Agenda

1. Common pitfalls & how to avoid them
2. Instability & Interpretation
• A Principal Components Example
• A Kalman Filter Example
3. More than one way to scramble an egg
• Features and interpretation over model selection
4. A Focus on Implementation & Implied Assumptions

# Avoiding Common Pitfalls

### No magic formula, but there are some good guidelines

• Reproducing sub-results in different environments (Excel & R/MatLab)
• Interpret, decompose, interpret
• No replacement for time, experience, & care
• You will be rushed -- It is okay to delay
• Review your assumptions, then do it again
• Are there implicit assumptions?
• Can you realistically implement?
• Was all data available? revised? realistic lag time?

# Instability & Interpretation

### Stability & Interpretation normally go hand in hand

• As humans, we are natural story-tellers
• It is far too easy to append a narrative to data or output
• But if that output is unstable, how can our narrative be true?
• Ensuring our analysis is stable and consistent is one of the most important excercises we can carry out
• Let's look at two examples:
• Principal Component Analysis
• Kalman Filter & Neural Nets

# PCA Intro 1

### Principal Component Analysis -- A Quick Primer

• In contrast to Linear Regression which finds the best β to fit                 ,    PCA find the best combination of r so that                 and
• Like Linear Regression where betas are 1, moves the data instead
• What are the principal components?
1. The first principal component is the combination of data that explain the most variance and
2. Every following component is the same, but also with the covariance between it and the previous component equal to zero
\bold y = \pmb\beta*\bold r
\bold y = \pmb\beta*\bold r
\pmb\beta'*\pmb\beta = 1
\pmb\beta'*\pmb\beta = 1

# PCA Intro 2

### Principal Component Analysis -- What Is It Good For?

• Dimension Reduction & Clustering

• In practice
• Check autocorrelation of returns
• Can be used/compared with Factor Models to gain intuition

Explain All This

With Just This

# Kalman Filter Intro 1

### Kalman Filter -- A Quick Primer

• The Kalman Filter is a model with many names
• First consider Time-Series Model, the Vector Auto-Regression

• The prediction for      is a state, which helps prediction

• The state is sometimes considered a hidden regime or layer
x_{t} = \phi_{t-1} + w_{t}
x
y_{t} = \Alpha_{t}x_t + v_{t}

# Kalman Filter Intro 2

### The importance of the transition matrix

• When using the Kalman Filter for time-varying models and states, the transition matrix helps stability of the state
• First consider two states

\begin{matrix} S_{1,t} \\ S_{2,t} \end{matrix} = \pmb\phi_{t-1} + \bold w_{t}
 95% 5% 10% 90%
\begin{matrix} S_{1} \\ S_{2} \end{matrix}
S_{1}
S_{2}

From

State

To

State

Unstable

# Comparison With Neural Nets

### A simple visual comparison

Hidden Markov Model

Inputs

Hidden States Selected on Probability of Fit According to Features

States Effect Output

Neural Net

Select Features That Describe Hidden Layer

Hidden Layers Chosen

Hidden Layers Effect Output

# Implementation

### Let's take a step back and discuss implementation

• One of the most famous models in finance is the Shiller PE 10

*Data from EDS & Robert Shiller's website

# A Simple Implementation

### Implementation of this signal, on its face, should be easy

*Data from EDS & Robert Shiller's website

# Quick Test

### However, implementing a quick test, we don't see gains?

• This test implies free trading costs

*Data from EDS, Shiller & Bloomberg, courtesy of London Business School

# So What Went Wrong Part 1

### Implied Assumptions -- If You Sell Equities, What Do You Buy?

• Bonds share the same relationship with the measure

Interesting Side Note, These Are Driven By Inlfationary Periods

*Data from EDS, Shiller & Bloomberg, courtesy of London Business School

# So What Went Wrong Part 2

### Didn't Check Assumptions

• Strong autocorrelation, breaks an underlying assumption of OLS
• Interpretation: A new portfolio created each month?

*Data from EDS & Shiller

# Different Models Same Insight

### Both MVO & OLS Lead to the Same Output

• Let's go through the spreadsheet here

*Data from EDS & Bloomberg, courtesy of London Business School

# From Slack      L1      L2

### Generalized Additive Models - Mixing Elastic Net with Cross-Validation

• Similar to the Slack variable we have a clear intuition
• L1-Norm: Make sure outliers don't drive model
• L2-Norm: Which factors should we choose?
\small \boldsymbol L(\lambda_1,\lambda_2,\beta) = \underbrace{\text{\textbar} y - X\beta \text{\textbar}^2}_{OLS} + \underbrace{\lambda_2\text{\textbar} \beta \text{\textbar}^2}_{L2-Norm} + \underbrace{\lambda_1 \text{\textbar} \beta \text{\textbar}_1}_{L1-Norm}

# Using Cross-Validation

### Cross-Validation is used to choose parameters

• The intuition behind this approach is sacrificing the 'best fit' for stability
• A New Goal: The least wrong, the most often

1st Iteration

2nd Iteration

10th Iteration

...

\overbrace{\textcolor{#f9f9f9}{..............................}}^{\text Training Folds}
\overbrace{\textcolor{#f9f9f9}{}}^{\text Test Fold}
\rArr E_1
\rArr E_2
\rArr E_{10}
\Huge \rbrace \small Avg

Full Training Data

# Why Are GAMs Useful?

### Rather than focusing on a new model/approach that is harder to interpret, GAMs provide a framework for robust traditional modelling

• At the end of the day, GAMs are just a special case of OLS
• But then, so are most models
• GAMs are a machine learning framework to automate the model building process
• 'Features are King' -- but which ones to choose?
• These features should be interpretable, robust, and consistent
• Easier to understand the connection between theory and output

# Main Takeaways

### Focus on details & assumptions, interpretation, implementation, and consistency

• Avoid pitfalls through reproducibility, patience, and care
• Learn your tools and systems, check the details and data types
• By focusing on interpretation all other good habits follow
• What does this mean in terms of implementation?
• What are the underlying & implicit assumptions?
• Do I understand and can I explain the output?
• Is it consistent? Am I achieving my goals?

# Frameworks Not Models

### Thinking of these in terms of frameworks will gain more benefits

• Building a model is great, but it isn't really a solution
• Successful value add will come by combining many tools and insights into a robust framework for decision making
• These tools are about automation & enablement
• Focus more on finding implementable & interpretable insights
• Look more broadly, more deeply than before
• This is fundamental

# Why Bother With All This? Part 1

• Better outcomes, one example of many is in Hedge Funds

*Data from EDS & Bloomberg, courtesy of London Business School

# Why Bother With All This? Part 2

### Using A.I. in financial services is becoming a 'must-have'

• Aug 2017: 20% usage of A.I. or Machine Learning
• Aug 2018: 56% usage

### Exploratory programs on machine learning

• A.I. based risk management
• Dynamic factor analysis using A.I.
• A.I. reconciling investment decisions

# Concluding Remarks

• ML and AI aren't neccessarily a value-add
• When combined in a strong framework the value adds are fundamental:
• Enabling you to work with a more holistic perspective
• Increasing efficiency and therefore productivity
• Allows you to focus more on insights & implementation
• Know your systems, big problems require an understanding of tools
• Interpretation is the realm of the human
• These tools are powerful, but if you can't understand the 'why' then they are little use to anyone

# Disclaimers

Please remember that past performance may not be indicative of future results. Different types of investments involve varying degrees of risk, and there can be no assurance that the future performance of any specific investment, investment strategy, or product made reference to directly or indirectly in this presentation, will be profitable, equal any corresponding indicated historical performance level(s), or be suitable for an individual's portfolio.

Our projections are based on current market conditions which can vary over the coming months and weeks. Additionally, our projections are based on historical market behavior which may vary unexpectedly. Using Machine Learning, our tool should adjust to new market fluctuations but we might not be able to avoid short term volatility.