Labeled vs unlabeled data — and the spectrum in between
Framing a business problem correctly before reaching for an algorithm is the first real test of ML judgment. Treating a ranking problem as classification, or a segmentation problem as regression, leads to the wrong metric, the wrong loss function, and a model that never actually solves the real objective.
It also sets up every other ML topic: bias-variance, evaluation metrics, and regularization all mean something different depending on whether you're predicting a label or discovering structure.
Supervised learning is learning with an answer key — every training example comes with the correct output, and the model learns the mapping from input to that recorded answer. Unsupervised learning has no answer key — the model just looks for patterns: which points are similar, which directions carry the most information.
Semi-supervised and self-supervised sit in between: a little labeled data plus a lot of cheap unlabeled data, or a task where the "labels" are manufactured automatically from the data's own structure — no human annotation needed.
Both are supervised — the difference is the shape of the output.
Ordinal targets (star ratings, severity levels) are a common gray area — technically discrete but ordered, sometimes best modeled as regression then rounded, sometimes as ordinal classification.
No ground truth to check against, so evaluation is inherently harder — you're judging structure, not correctness.
Semi-supervised: a small labeled set plus a much larger unlabeled set. Pseudo-labeling trains on the labeled data, predicts on the unlabeled data, then retrains including the confident pseudo-labels. Label propagation spreads labels through a similarity graph.
Self-supervised: labels are generated automatically from the data's own structure, no human annotation required — mask a word and predict it (language models), predict a rotated image's rotation, or pull two augmented views of the same image together while pushing different images apart (contrastive learning). This is how modern LLMs and vision foundation models get pretrained before any task-specific fine-tuning.