Updating belief with evidence — the foundation of probabilistic ML
Bayes' theorem is arguably the most important single equation in ML. It tells you how to rationally update your beliefs when you get new evidence. Naive Bayes, Bayesian optimisation (used in AutoML and hyperparameter tuning), variational inference, and MAP estimation are all direct applications.
Before seeing any evidence, you have a prior belief — your best guess before the data arrives. Then you observe something (evidence). Bayes' theorem tells you how to update your prior into a posterior — your new, refined belief after incorporating the evidence.
The update is multiplicative: you scale your prior by how likely the evidence is under each hypothesis, then renormalise. Hypotheses that predict the evidence well get their probability boosted; those that predict it poorly get shrunk.
Naive Bayes classifies by computing the posterior of each class given all features. The "naive" assumption: all features are independent given the class — which simplifies the product enormously:
Despite the independence assumption being almost always wrong, Naive Bayes works surprisingly well for text — because even with wrong probabilities, the ranking of classes is often correct.