Optuna
Alan Barzilay
Grid Search
Impractical most of the time.
Wastes too much time in low performance regions.
Random Search
Mostly same issues as Grid Search, but a step in the right direction.
Partitioning the Search Space and Parzen Estimation
l(x) groups the top γ% performing hyperparameter combinations.
l(x) is "good" and g(x) is "bad".
2.
1.
Determining the Next “Best” Hyperparameters to Test
Test the maximum of g(x)/l(x), rinse repeat.
3.
Random Exploration
TPE Refs
2. Research into the project space, competitors and the market.
1. Uncovering project requirements.
3. Creating a Plan that sets the requirements for the design and build phases.
4. Design a number of iterations that capture the plans and requirements.
5. Review and Iterate on the designs with testing of ideas, client feedback and prototypes.
Bento
By barzilay
