The main training objective for GPT-style LLMs is next-token prediction: given a sequence of tokens, the model predicts a probability distribution over the vocabulary for what comes next, that prediction is compared against the actual next token using cross-entropy loss, and the weights are updated to reduce that loss.

Concretely, given input "The capital of India is" the target is "New"; then given "The capital of India is New" the target is "Delhi." At massive scale, this simple objective — repeated across huge amounts of text — teaches grammar, facts, reasoning patterns, code patterns, style, translation, summarization, and instruction-following behavior, none of which were explicitly labeled as separate tasks. The important caveat: the model is not explicitly trained to know truth, only to predict likely continuations, which is one of the fundamental reasons hallucination exists — a fluent, high-probability continuation and a factually correct one are not the same target.