Introduction

There is a company "ABC Private Limited" who wants to understand the purchasing behavior of customers using the existing sales data.

Problem Statement

To build a model to predict the purchase amount of customer against various products

Problem Statement

Data Description

 Train Set 550k rows

 Test Set 233k rows

Data Preprocessing

Null Values in Dataset per column

Replaced null values by zero

Feature Engineering

and

Modelling

• Label Encoded Gender, Age, City_Category
• About 3600+ different Product IDs
• One-hot-encoded top 1000 most frequent Product IDs

Model: Multiple Linear                     Regression

Rank: 1495

RMSE: 3240.59

Baseline Model

Time for more Feature Engineering and a new model

• Reduced number of one-hot encoded features to 20

Model: Random Forest Regressor

Rank: 773

RMSE: 2732.16

Much more Feature Engineering and another model

• Performed Target mean encoding and frequency encoding for Product_ID and User_ID

Model: XGBoost ( Default parameters)

Rank: 310

RMSE: 2496.3621

Time for Hyperparameter Tuning

(Used Bayesian Optimization)

XGBoost Hyperparameters:

• n_estimators
• max_depth
• min_child_weight

• gamma

• colsample_bytree

• subsample

• reg_alpha

Rank: 279

RMSE: 2487.4789

Again Feature Engineering and LightGBM !!

• Performed Target min, max encoding for Product_ID and User_ID

Model: LightGBM ( Default parameters)

Rank: 310

RMSE: 2574.68 (lower than XGBoost(optimized))

Optimized LightGBM

LightGBM Hyperparameters:

• n_estimators
• num_leaves
• max_depth
• learning_rate

• reg_alpha

• reg_lambda

Finally!!

Rank: 182

RMSE: 2468.2523

Feature Importance (Top 10)

What's next?

• Try to engineer more features
• Create ensembles
• Try Catboost

What I learned?

• Feature Engineering is King
• Target Encoding is key to jumping up in leaderboard
• Ensembles are necessary
• How much LightGBM is fast?
• Don't waste much time in tuning hyperparameters

My 1st Online Data Science Competition

Thank You