Decoder Only Block Anatomy

LLMs interview question · Medium difficulty

What does a decoder-only GPT-style block actually look like, layer by layer?medium

Status
✓
Type
conceptual
Topic
decoder-only-block-anatomy
Frequency
common
Tags
GPT, decoder-only, transformer-block, architecture
Answer

Input tokens become token embeddings plus positional information, then flow through a stack of decoder blocks, and finally output logits used for next-token prediction. Each block contains causal self-attention, a feed-forward network, residual connections, and layer normalization — there's no separate encoder, unlike translation-style architectures.

Explanation

The full path is: input tokens → token embeddings + positional information → stack of decoder transformer blocks → output logits → next-token prediction. Unlike encoder-decoder models used for translation (like the original Transformer or T5), GPT-style models drop the encoder entirely and use only a stack of decoder blocks with causal attention, which naturally models P(next token | previous tokens) — exactly what generation needs. This architecture is what powers chat, completion, summarization, coding, reasoning, and tool-calling models, since all of those tasks reduce to "generate the next token given everything so far."

Follow-upWhy doesn't a decoder-only model need a separate encoder the way translation models do?
Follow-upWhat role do residual connections and layer normalization play inside each block?
Follow-upWhy is P(next token | previous tokens) the right objective to model for generation tasks?