NLP Interview Questions

Tokenization splits text into units such as words, subwords, or characters.

Tokenization converts raw text into model-readable pieces. Modern models often use subword tokenization to handle rare words and multiple languages.

TF-IDF scores words by frequency in a document and rarity across documents.

It highlights terms that are important to a document but not common everywhere, making it useful for search and classic text classification.

Embeddings represent words or text as dense numeric vectors.

Similar meanings tend to have nearby vectors, enabling semantic similarity, clustering, retrieval, and downstream model features.

Attention lets a model weight relevant tokens when building representations.

Attention helps models connect distant words and focus on important context, which is central to transformer architectures.

Use task-specific metrics plus qualitative error analysis.

Classification may use F1, generation may use human evaluation or task success, and retrieval may use precision, recall, or MRR.

A unidirectional LSTM only has past context at each timestep.

A unidirectional LSTM only has past context at each timestep. Bi-LSTM runs two LSTMs — one forward, one backward — and concatenates hidden states. For sarcasm, the final word often recontextualizes earlier words ('Oh great, another Monday'). Bidirectionality lets the model use future context to reinterpret earlier tokens — key reason for 7%+ accuracy gain.

Single-head computes one attention distribution — one way of relating tokens.

Single-head computes one attention distribution — one way of relating tokens. Multi-head runs h parallel attention functions with different learned projections, then concatenates. Each head can attend to different relationship types (syntactic, semantic, positional). For sarcasm or contract parsing, some heads focus on negation, others on named entities, others on long-range dependencies.

Kept BERT's pretrained weights mostly frozen (or used a very small LR).

Kept BERT's pretrained weights mostly frozen (or used a very small LR). Added a classification head on top of the [CLS] token: linear → dropout → linear → softmax over 2 classes. Also tested adding a Bi-LSTM layer on top of BERT's token outputs before classification. Fine-tuned with AdamW, warmup scheduler, and early stopping on validation F1.

Word2Vec: one static embedding per word — 'bank' always has the same vector.

Word2Vec: one static embedding per word — 'bank' always has the same vector. BERT: contextual embeddings — 'bank' in 'river bank' vs 'bank account' gets different vectors based on the full sentence. For a domain corpus ranking, Word2Vec was sufficient (less ambiguous domain terms). For a document extraction pipeline contract extraction, contextual embeddings are critical.