How does gradient boosting differ from bagging?medium

Type
conceptual
Topic
how-does-gradient-boosting-differ-from-bagging
Frequency
common
Tags
machine-learning, how, does, gradient, boosting, differ
Answer

Bagging (Random Forest) trains trees in parallel on random subsets and averages — reduces variance.

Explanation

Bagging (Random Forest) trains trees in parallel on random subsets and averages — reduces variance. Boosting trains sequentially, each tree correcting previous errors — reduces bias. XGBoost adds L1/L2 regularization to the objective, handles missing values natively, and uses second-order gradients for faster convergence. Chosen for markdown because demand patterns have structured nonlinearities boosting captures well.

Follow-upWhen would you choose one approach over the other?