At each step, the model outputs a probability distribution over the vocabulary, samples a token, appends it to context, and repeats. Temperature controls randomness globally by scaling logits before softmax, top-k limits sampling to a fixed number of top candidates, and top-p (nucleus sampling) dynamically limits candidates by cumulative probability mass.

Temperature near 0 makes output focused and predictable (approaching greedy/argmax selection), around 0.7 is balanced, and 1.2+ is more random and creative — technically it sharpens or flattens the probability distribution before softmax is applied. Top-k restricts the model to only the k most likely tokens, e.g. top_k=5 samples only from the 5 highest-probability tokens and ignores everything else, which prevents very unlikely tokens from ever being picked but uses a fixed candidate count regardless of how confident the model actually is. Top-p instead selects the smallest set of tokens whose cumulative probability reaches p — top_p=0.9 means the model only considers tokens that together make up 90% of the probability mass — and because it's dynamic, that set might be 5 tokens when the model is confident or 50 when it's uncertain, which is why top-p is generally preferred over a fixed top-k. In short: temperature reshapes the whole distribution, top-k caps the candidate count, top-p caps the candidate set by probability mass. For factual tasks use lower temperature; for creative tasks use higher temperature with a suitable top-p.