What is softmax and what does temperature do to it?medium

Type
conceptual
Topic
is-softmax-and-what-does-temperature-do-to-it
Frequency
common
Tags
deep-learning, what, softmax, and, does
Answer

Softmax converts raw logits to a probability distribution: softmax(z_i) = e^z_i / Σe^z_j.

Explanation

Softmax converts raw logits to a probability distribution: softmax(z_i) = e^z_i / Σe^z_j. Temperature T scales logits before softmax: softmax(z_i/T). T<1 (e.g., 0.3): sharpens → more deterministic. T>1 (e.g., 1.5): flattens → more random/creative. T→0 → argmax (greedy). Use low temperature for extraction tasks (a document extraction pipeline), higher for creative generation.

Follow-upCan you give a production example?