Three truths of data-science

qplum.co

1. We don't care about past performance of a model. We care about how it will perform in future.



2. Past data is all we have to learn from. We don't have data of what will happen in future.



3. Not all aspects of past data are equally likely to occur in future.

The process of making a trading strategy

qplum.co

What is meant by overfitting?

qplum.co

Real life experiments with overfitting and underfitting

qplum.co

• Trading group D would regenerate their model once in three months using as much data as they could get their hands on and with the train-test approach to splitting data. ( underfitting )
• Trading group J would regenerate the model every day using last five days data. ( walk-forward backtesting)

The problem with D's approach is that in trading there is a lot of seasonality. Ignoring the most recent data in model training will lead to under-fitting. Doing so, however, precludes that data from being used in model evaluation. 

J's approach takes care to use most recent data without using the same data in evaluating the model that was used to train it.

How do data-scientists avoid overfitting?

qplum.co

1. Take as much data as you can possibly get to verify your ideas.

2. Optimize metrics like Sharpe Ratio and not based on returns.

3. Labeled data is expensive. Maximize what you can learn from unlabeled data before you start learning from labeled data.

4. Split data into training and testing. However, this can be tough and impractical when trying to learn ideas that are short-lived.

5. Walk-forward optimization: Measure your strategy strictly in accordance with 2nd truth. 

6. Think in terms of future-testing and not back-testing.

Walk-forward = No cheating

qplum.co

Presentation Video

qplum.co

Let's learn together

Our aim here at FinTech - DataScience - NY/JC meetups is for people to come together and share with others what they have learned, towards building a better future.

If you want to want to present at our meetups please email at community@qplum.co

qplum.co/events