Model Optimization

Hyperparameter Tuning (GridSearchCV, RandomizedSearchCV)

Learning Outcome

Implement hyperparameter tuning using Scikit-Learn

Describe how RandomizedSearchCV works

Describe how GridSearchCV works

Explain the purpose of hyperparameter tuning

Distinguish between model parameters and hyperparameters

Understand what hyperparameters are

Topic Name-Recall(Slide3)

Hook/Story/Analogy(Slide 4)

Transition from Analogy to Technical Concept(Slide 5)

Why Hyperparameter Tuning is Important?

Improves Model Performance

The right hyperparameter values allow the model to capture patterns more effectively.

Reduces Overfitting

Proper tuning prevents models from becoming overly complex.

Reduces Underfitting

Poor parameter choices may make models too simple.

Improves Generalization

Well-tuned models perform better on unseen data.

Hyperparameter Tuning Techniques: GridSearchCV

How it works?

It performs an exhaustive search across all possible combinations of hyperparameters

Example parameter grid:

max_depth

criterion

[2,4,6,8]

['gini','entrophy']

Limitations

Computationally expensive
Slow for many parameters

Advantages

Thorough search
High accuracy

Hyperparameter Tuning Techniques: RandomizedSearchCV

Implementation in Python (Scikit-Learn)

Step 1 — Import required libraries

from sklearn.datasets import load_iris
from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import GridSearchCV, RandomizedSearchCV

Step 2 — Load the dataset

X, y = load_iris(return_X_y=True)

Step 3 — Initialize the model

model = DecisionTreeClassifier()

Step 4 — Define the parameter grid

param_grid = {
    'max_depth': [2, 4, 6, 8],
    'criterion': ['gini', 'entropy']
}

Step 5 — Apply GridSearchCV

grid_search = GridSearchCV(model, param_grid, cv=5)

grid_search.fit(X, y)

print("Best parameters from GridSearchCV:", grid_search.best_params_)

GridSearchCV will:

Try all parameter combinations
Perform cross-validation
Select the best-performing configuration

Step 6 — Apply RandomizedSearchCV

from scipy.stats import randint

param_dist = {
    'max_depth': randint(2, 10),
    'min_samples_split': randint(2, 10)
}

random_search = RandomizedSearchCV(
    model,
    param_distributions=param_dist,
    n_iter=10,
    cv=5,
    random_state=0
)

random_search.fit(X, y)

print("Best parameters from RandomizedSearchCV:", random_search.best_params_)

RandomizedSearchCV randomly samples parameter combinations and evaluates them using cross-validation

Summary

Both use cross-validation for reliable evaluation

GridSearchCV tests all combinations; RandomizedSearchCV samples randomly

Proper tuning improves performance and generalization

Hyperparameters are set manually, not learned

Hyperparameter tuning finds the best model settings

Quiz

What does GridSearchCV do?

A. Tests random parameter combinations

B. Tests all possible parameter combinations

C. Removes irrelevant features

D. Splits data into folds

Quiz-Answer

What does GridSearchCV do?

A. Tests random parameter combinations

B. Tests all possible parameter combinations

C. Removes irrelevant features

D. Splits data into folds

Model Optimization

Hyperparameter Tuning (GridSearchCV, RandomizedSearchCV)

Hyperparameter Tuning (GridSearchCV, RandomizedSearchCV)

More from Content ITV