Joint, conditional, marginal — the vocabulary of uncertainty
Probability is the language of uncertainty, and ML is fundamentally about making decisions under uncertainty. Every model output that is a "probability" — sigmoid, softmax, language model next-token scores — requires you to understand what a probability actually means and how probabilities relate to each other.
Without this foundation you cannot understand Naive Bayes, Bayesian networks, generative models, or even why cross-entropy loss is the right choice for classification.
Think of probability as a budget of belief — you have 1.0 to distribute among all possible outcomes. Joint probability asks: how much of that budget goes to two things happening together? Conditional asks: if I already know B happened, how do I redistribute the remaining budget over A? Marginal asks: ignoring everything else, what's the total budget on A?
The three are connected like different views of the same underlying table of counts.
If B can take several values (B₁, B₂, ..., Bₙ) that partition the sample space:
This lets you compute a marginal by summing over all cases of a conditioning variable. Example: P(spam) = P(spam|"win")·P("win") + P(spam|no "win")·P(no "win").