RNNs process tokens sequentially and compress all prior context into one fixed-size hidden state, so information from early tokens fades as the sequence gets longer.

Each step's hidden state has limited capacity, and gradients shrink as they propagate backward through many timesteps (the vanishing gradient problem), so distant dependencies are hard to learn. Sequential processing is also slow to train, since token N can't be computed before token N-1.