NexForay

Learn Interview Roadmap Resources

Interview SkillsDeep Learning

What is backpropagation? Why do neural networks need activation functions? What is dropout? What is a CNN? What is vanishing gradient? Walk me through backpropagation from scratch. What is vanishing gradient and how does LSTM solve it? Explain the Transformer architecture from scratch. How does BERT differ from GPT in architecture and pretraining? Pretraining vs fine-tuning vs prompt-based learning — differences? How do token embeddings combine with positional embeddings in a Transformer? Batch normalization vs layer normalization — which is preferred in Transformers? Explain dropout — training vs inference behavior. What is softmax and what does temperature do to it? Encoder-only vs decoder-only vs encoder-decoder Transformers? Why did RNNs struggle with long text? What are LSTM and GRU, and why were they introduced? Why did transformers replace RNNs for many NLP tasks? What is positional encoding and why is it needed? How did transformers lead to modern LLMs? What is attention? What does bidirectionality add in a Bi-LSTM text classifier? What is multi-head attention and why is it better than single-head? You fine-tuned BERT for sarcasm detection. What layers did you modify? How does Word2Vec differ from contextual embeddings like BERT? What problem did the encoder-decoder architecture solve? What is self-attention?

Deep Learning Interview Questions

Neural network fundamentals for modern ML and AI engineering interviews.

27 questionsskill

Next →

Prepare for your next career move.

Learn·Interview Prep·Roadmaps·Resources

© 2026 NexForay

Built by Amit Jsr