Machine Learning Interview Questions

Overfitting happens when a model learns training noise and fails on new data.

It often appears as high training performance and weaker validation performance. Regularization, simpler models, better validation, and more data can help.

It balances underfitting from high bias and overfitting from high variance.

Simple models may miss patterns, while complex models may be too sensitive to training data. Good generalization manages both.

It evaluates a model across multiple train-validation splits.

K-fold cross-validation reduces dependence on one split and gives a more stable estimate of generalization.

Leakage happens when training data includes information unavailable at prediction time.

Leakage creates unrealistically strong offline metrics and weak production performance because the model learned future or target-derived signals.

Pick a metric that matches the task and business cost of errors.

Classification may use precision, recall, F1, ROC-AUC, or PR-AUC. Regression may use MAE, RMSE, or R squared depending on error sensitivity.

Precision measures how many predicted positives are correct; recall measures how many actual positives are found.

Precision matters when false positives are costly. Recall matters when missing positives is costly, such as fraud, disease detection, or safety alerts.

Regularization penalizes model complexity to improve generalization.

L1 can encourage sparse weights, while L2 discourages large weights. Both reduce overfitting by making the learned function less sensitive to noise.

Use appropriate metrics, class weighting, resampling, threshold tuning, and careful validation.

Accuracy can hide poor minority-class performance. PR-AUC, recall, precision, confusion matrix analysis, and cost-aware thresholds often give a better picture.

Bagging (Random Forest) trains trees in parallel on random subsets and averages — reduces variance.

Bagging (Random Forest) trains trees in parallel on random subsets and averages — reduces variance. Boosting trains sequentially, each tree correcting previous errors — reduces bias. XGBoost adds L1/L2 regularization to the objective, handles missing values natively, and uses second-order gradients for faster convergence. Chosen for markdown because demand patterns have structured nonlinearities boosting captures well.

Precision = TP/(TP+FP), recall = TP/(TP+FN), F1 = harmonic mean.

Precision = TP/(TP+FP), recall = TP/(TP+FN), F1 = harmonic mean. In a resume screening system, recall matters more — missing a good candidate (false negative) is costlier than surfacing a borderline one for human review. With downstream HITL review, balance with F1 and use configurable scoring weights to tune per job template.

A single split gives a high-variance estimate — you might get lucky or unlucky.

A single split gives a high-variance estimate — you might get lucky or unlucky. k-fold uses all data for both training and validation across k rounds, giving a more stable estimate. Stratified k-fold preserves class distribution. For time-series (demand forecasting), use TimeSeriesSplit to avoid data leakage.

By training many deep trees on random bootstrap samples (bagging) and using a random feature subset at each split.

By training many deep trees on random bootstrap samples (bagging) and using a random feature subset at each split. Averaging uncorrelated trees cancels out individual noise. The key is decorrelation — if all trees see the same features, they make correlated errors and averaging doesn't help much.

L1 (Lasso) adds sum of |weights| — drives some weights to exactly zero, giving sparse models.

L1 (Lasso) adds sum of |weights| — drives some weights to exactly zero, giving sparse models. Good for feature selection. L2 (Ridge) adds sum of weights² — shrinks all weights small but nonzero. Better when all features contribute. ElasticNet combines both. In LLM training, L2-like weight decay is standard.

The LP takes demand forecasts as input and optimizes markdown percentage per SKU-brand pair to minimize revenue loss while clearing inventory before end-of-life.

The LP takes demand forecasts as input and optimizes markdown percentage per SKU-brand pair to minimize revenue loss while clearing inventory before end-of-life. Constraints: minimum margin floors, maximum markdown caps, inventory depletion deadlines. Objective: maximize recovered revenue subject to sell-through constraints. Implemented with scipy.optimize or PuLP.

Options: oversampling minority class (SMOTE), undersampling majority, class_weight='balanced' in sklearn, adjusting decision threshold post-training, or using F1/AUC-ROC instead of accuracy.

Options: oversampling minority class (SMOTE), undersampling majority, class_weight='balanced' in sklearn, adjusting decision threshold post-training, or using F1/AUC-ROC instead of accuracy. In resume screening, threshold tuning + weighted scoring worked better than resampling since the minority examples were genuine top candidates, not noise.

Start with domain knowledge (price, seasonality, sell-through rate, days-to-expiry).

Start with domain knowledge (price, seasonality, sell-through rate, days-to-expiry). Use correlation analysis, feature importance from a baseline tree model, and VIF for multicollinearity. For time-series, add lag features and rolling statistics. Drop features with high missingness or near-zero variance. Validate with time-aware CV.

Used an ensemble of XGBoost, a statistical baseline (exponential smoothing or ARIMA), and a linear model for regularization.

Used an ensemble of XGBoost, a statistical baseline (exponential smoothing or ARIMA), and a linear model for regularization. Combined via weighted averaging or stacking where a meta-learner decides weights based on recent forecast error. Reduces the risk of any single model failing on edge SKUs.

Leakage is when future information bleeds into training, inflating metrics.

Leakage is when future information bleeds into training, inflating metrics. Prevention: strict temporal splits (TimeSeriesSplit, no shuffle), lag features only from t-n onwards, rolling stats computed on training window only, careful with target encoding. Always verify that your validation set strictly follows your training set in time.

Regression: MAE, MSE, RMSE, MAPE, R². Classification: accuracy, precision, recall, F1, AUC-ROC, log loss.

Regression: MAE, MSE, RMSE, MAPE, R². Classification: accuracy, precision, recall, F1, AUC-ROC, log loss. For imbalanced classification, AUC-ROC and F1 are more informative than accuracy. For business-critical regression (demand forecast), MAPE is intuitive but breaks near zero — use sMAPE or RMSE with domain thresholds.

SMOTE generates synthetic minority samples by interpolating between existing ones in feature space.

SMOTE generates synthetic minority samples by interpolating between existing ones in feature space. Appropriate for tabular numeric features with enough minority examples to interpolate. Not appropriate for text, very sparse features, or when minority samples represent true anomalies (don't synthesize fraud patterns). For tree models, class_weight often works just as well with less risk.

A Pipeline chains preprocessing steps and the final estimator.

A Pipeline chains preprocessing steps and the final estimator. Each step implements fit/transform; the last step implements fit/predict. Benefits: prevents leakage (fit on train, transform on test within CV), cleaner code, easy serialization with joblib, and direct deployment. Nest ColumnTransformer inside pipelines for mixed-type data.

Key params: max_depth (keep 3-6), min_child_weight (higher = less overfit), subsample and colsample_bytree (0.7-0.9), small learning_rate + more rounds, lambda/alpha for L2/L1 reg.

Key params: max_depth (keep 3-6), min_child_weight (higher = less overfit), subsample and colsample_bytree (0.7-0.9), small learning_rate + more rounds, lambda/alpha for L2/L1 reg. Use early stopping on a validation set. For scale: Bayesian optimization (Optuna) instead of grid search. Always monitor train vs val loss gap.