Block 73: Classification: Decision Trees & Random Forests
Use tree-based models for classification.
Concepts
- DecisionTreeClassifier with max_depth
- Feature importance: .feature_importances_
- RandomForestClassifier as ensemble
- Overfitting in trees and how depth controls it
Code Examples
See exercise below.
Exercise
Train a Decision Tree (depth=3) and a Random Forest (100 trees) on a dataset. Compare accuracy. Plot feature importances from the Random Forest as a bar chart.
Homework
Why does a single deep decision tree overfit but a Random Forest doesn't?