Interview SkillsDeep Learning

What is backpropagation? Why do neural networks need activation functions? What is dropout? What is a CNN? What is vanishing gradient? Walk me through backpropagation from scratch. What is vanishing gradient and how does LSTM solve it? Explain the Transformer architecture from scratch. How does BERT differ from GPT in architecture and pretraining? Pretraining vs fine-tuning vs prompt-based learning — differences? How do token embeddings combine with positional embeddings in a Transformer? Batch normalization vs layer normalization — which is preferred in Transformers? Explain dropout — training vs inference behavior. What is softmax and what does temperature do to it? Encoder-only vs decoder-only vs encoder-decoder Transformers? Why did RNNs struggle with long text? What problem did the encoder-decoder architecture solve? What is attention, and why was it a breakthrough? How did Transformers lead to modern LLMs? Can you walk through a self-attention computation step by step? What's the difference between attention, self-attention, and cross-attention? What does the feed-forward network inside a transformer block actually do? Why do transformers need residual connections and layer normalization to train well?

Deep Learning Interview Questions

Neural network fundamentals for modern ML and AI engineering interviews.

23 questionsskill

What is backpropagation? →