Machine Learning Interview Questions and Answers

Supervised learning trains on labeled data. Each input has a known target. The model learns to predict the target for new inputs. Unsupervised learning uses data with no labels. It finds patterns by itself.

• Supervised: classification (spam or not spam) and regression (house price). Algorithms: linear and logistic regression, decision trees, SVM, neural networks.
• Unsupervised: clustering (customer groups), dimensionality reduction (PCA), and anomaly detection. Algorithms: k-means, DBSCAN, PCA.
• Semi-supervised and self-supervised sit in between. Semi-supervised uses a small labeled set plus lots of unlabeled data. Self-supervised makes labels from the data itself. This is how language models are pretrained.

The real difference is the cost of labels and how you check results. Supervised models have a clear score against the true answers. Unsupervised results need domain judgement or indirect (proxy) metrics.

What they are checking: Clear definitions with one example of each. Also that labels drive cost and how you evaluate.

Bias is error from assumptions that are too simple. It causes underfitting. Variance is error from being too sensitive to the training data. It causes overfitting. Expected test error is roughly bias squared plus variance plus irreducible noise.

• High bias: the model is too simple. Error is high on both train and test data (a straight line on curved data).
• High variance: the model memorizes noise. Train error is low but test error is much higher (a deep tree with no pruning).
• Tradeoff: a more complex model lowers bias but raises variance. You look for the complexity where total error is lowest.

Ways to fix it: more data and regularization reduce variance. More features or a richer model reduce bias. Ensembles also help: bagging cuts variance and boosting cuts bias.

What they are checking: Whether you can link the idea to checking a model by its train and test error.

Overfitting is when a model learns the noise and odd details of the training set. It scores well on training data but poorly on new data.

Detect it with the gap between train and validation scores. A learning curve helps too. Validation error goes up while training error keeps going down.

Prevent it with:

• More (or more varied) training data, and data augmentation
• A simpler model, pruned trees, or fewer features
• Regularization (L1, L2, dropout, weight decay)
• Early stopping on a validation metric
• Cross validation for honest model selection
• Ensembling such as bagging

Fix the cause first. If there is data leakage or a tiny dataset, tuning hyperparameters will not help.

What they are checking: A way to find the problem first, then several fixes that match it. Not a memorized list.

Both add a penalty on the size of the weights to the loss. This discourages complex models.

Elastic Net mixes both. The penalty strength is a hyperparameter. It is alpha in scikit-learn. In logistic regression, C is its inverse. Scale the features first, because the penalty depends on the size of the weights.

What they are checking: Whether you know why L1 gives sparse results (the geometry) and why you must scale features.

Each set has its own job. This keeps the performance estimate honest.

• Training set: fit the model parameters.
• Validation set: choose hyperparameters and compare models. You look at it many times, so its score becomes a little too optimistic.
• Test set: use it once at the end. It tells you the expected real-world performance.

Common splits are 70/15/15 or 80/10/10. With small data, use cross validation instead of a fixed validation set. Keep the time order for time series. Stratify for imbalanced classes. Split by group (user, patient) when rows from one entity are related. Otherwise leakage makes the scores too high.

What they are checking: Understanding of leakage and why the test set must stay untouched.

K-fold cross validation splits the data into k equal folds. The model trains k times. Each time, it trains on k-1 folds and validates on the one left out. The k scores are then averaged.

Use it when data is limited. It gives a more stable estimate than one split. It is also good for comparing models and tuning hyperparameters. Use StratifiedKFold for imbalanced classification. Use GroupKFold when rows share an entity. Use TimeSeriesSplit for time data. Never mix future data into the past.

from sklearn.model_selection import cross_val_score
from sklearn.linear_model import LogisticRegression
scores = cross_val_score(LogisticRegression(max_iter=1000), X, y, cv=5, scoring='f1')
print(scores.mean(), scores.std())

The cost is k times the training time.

What they are checking: Knowing the variants for imbalance, groups and time. Not only the textbook definition.

For a binary classifier, we count TP (true positives), FP (false positives) and FN (false negatives).

• Precision = TP / (TP + FP): of the items predicted positive, how many were right.
• Recall = TP / (TP + FN): of the real positives, how many did we find.
• F1 = 2 * P * R / (P + R): the harmonic mean. It is low if either one is low.

Choose by the cost of each error. For a spam filter, precision matters (do not put real mail in spam). For cancer screening or fraud detection, recall matters (do not miss cases). Accuracy can mislead on imbalanced data. So report precision, recall and F1 per class. Change the decision threshold to trade one for the other.

What they are checking: Whether you link the metric to the business cost of false positives and false negatives.

Linear regression predicts a continuous value. It uses a weighted sum of the features and is fit by minimizing squared error. Logistic regression predicts the probability of a class. It applies the sigmoid function to a linear combination of features. It is fit by minimizing log loss (cross-entropy).

A decision tree splits the data again and again. At each step it picks the feature and threshold that best separate the target. For classification it uses a measure like Gini impurity or entropy. For regression it uses variance reduction. The leaves hold the prediction.

Gradient descent reduces a loss by moving the parameters in small steps against the gradient: w = w - lr * dL/dw.

K-means splits the data into k clusters:

1. Pick k starting centroids (k-means++ spreads them out).
2. Assign each point to its nearest centroid.
3. Recompute each centroid as the mean of its points.
4. Repeat until the assignments stop changing.

A neural network is made of layers. Each layer does a weighted sum and then a non-linear activation.

Put preprocessing and the model in one Pipeline. Then scaling is fit only on training data.

Missing values: first understand why they are missing. Your options:

• Drop rows or columns if only a few are missing.
• Fill in the mean or median (numeric) or the most frequent value (categorical).
• Add a flag column that marks missing values.
• Use models that handle NaN directly (histogram gradient boosting, XGBoost, LightGBM).

Principal Component Analysis (PCA) finds directions that are at right angles to each other and capture the most variance in the data. These are the principal components. You project onto the top components. This reduces the number of dimensions and keeps most of the information.

Use a cross validated search over a grid or a distribution of parameters.

Feature scaling puts features on similar ranges. Standardization subtracts the mean and divides by the standard deviation. Min-max scaling maps values to a fixed range.

Feature engineering means turning raw data into inputs that make patterns easier to learn. It often improves results more than switching algorithms.

A confusion matrix is a table of predicted classes against actual classes. For binary classification:

Parameters are learned from data during training. Examples are regression coefficients, neural network weights, and tree split points. Hyperparameters are set before training. They control how learning happens. Examples are learning rate, tree depth, number of trees, regularization strength, k in k-NN, and batch size.

Classification predicts a discrete label. Regression predicts a continuous number.

Transfer learning reuses a model that was pretrained on a large dataset. It is the starting point for a related task. You can freeze the pretrained layers and train a new head (feature extraction). Or you can fine-tune some or all layers with a small learning rate.

Minimize mean squared error for y = w * x + b.

• MAE: the average absolute error. It is in the units of the target, handles outliers well, and is easy to explain.
• MSE / RMSE: squares the errors, so large mistakes cost more. RMSE brings the value back to the target's units. Use it when big misses are especially costly.
• R squared: the share of variance explained, compared with predicting the mean. It has no units. It can hide large errors. It also never goes down when you add features (use adjusted R squared).

A baseline gives you a reference point. It shows whether a complex model adds real value. Without it, 85% accuracy could be worse than always predicting the majority class.

The ROC curve plots the true positive rate against the false positive rate at all thresholds. AUC is the area under it. It equals the chance that a random positive scores higher than a random negative. 0.5 is chance and 1.0 is perfect.

PR-AUC (average precision) plots precision against recall.

Prefer PR-AUC when classes are very imbalanced. ROC-AUC can look great because the false positive rate divides by a huge number of negatives. This hides poor precision. ROC-AUC is fine when classes are roughly balanced, or when you care about ranking quality overall.

Both ignore the threshold. You still need to pick an operating threshold from the business trade-off. Check calibration too if you need real probabilities.

What they are checking: Knowing that ROC-AUC can mislead when classes are imbalanced.

Start by choosing the right metrics (precision, recall, F1, PR-AUC), not accuracy. Then combine these options:

• Class weights: class_weight='balanced', or scale_pos_weight in boosting. This is the cheapest and often enough.
• Resampling: oversample the minority class (SMOTE, with care) or undersample the majority class. Do this only inside the cross validation folds, never before the split.
• Threshold tuning: move the decision threshold using the precision-recall curve.
• Better data or features: collect more minority examples. Try anomaly detection if positives are very rare.
• Stratified splits so every fold has positives.

Always evaluate on the untouched test set, which keeps its natural imbalance.

What they are checking: Resampling inside the CV loop, and the choice of metric. Not just 'use SMOTE'.

Data leakage is when information that you would not have at prediction time affects training. Validation scores look too good, then fall apart in production.

Common sources:

• Fitting scalers, imputers or encoders on the full dataset before the split
• A feature made from the target (for example, 'refund issued' when predicting churn)
• Future information in time series
• Duplicate or near duplicate rows across train and test, or the same user in both

Signs: a score that is suspiciously high, one feature with extreme importance, or a big gap between offline and live results.

To prevent it, use a Pipeline so preprocessing is fit only on training folds. Use group and time aware splits. Review every feature and ask: 'would I know this at prediction time?'

What they are checking: Real leakage examples and how pipelines prevent it.

Both build ensembles (groups) of weak or unstable models.

In deep or recurrent networks, gradients are multiplied through many layers. If the factors are below 1, the gradient shrinks toward zero (vanishing). Then early layers barely learn. If the factors are above 1, the gradient grows without limit (exploding). This causes unstable updates and NaN loss.

Dropout randomly sets a share of activations to zero during training (for example 0.2 to 0.5). This stops neurons from depending on each other. It works like averaging many smaller networks. It is switched off at inference (model.eval()).

Options depend on the scope:

• Global (whole model): use linear coefficients (on scaled features). Tree impurity importance favors features with many values. Permutation importance is better. It measures the drop in score when one feature is shuffled. It works with any model and is more reliable.
• **Local (one...

Save the whole fitted pipeline (preprocessing plus model). Then serving applies the same transforms.

As the number of features grows, the volume of the space grows very fast. Data becomes sparse. Distances between points mean less (the nearest and farthest neighbors look almost equally far).

The options depend on the situation:

• Check a sample of labels. Measure how much annotators agree. Fix systematic errors and write clearer guidelines.
• Use robust losses or label smoothing. Drop or down-weight samples that always have high loss (they may be mislabeled).
• Use active learning: label the examples the...

A discriminative model learns the boundary, or the conditional probability P(y | x). Examples: logistic regression, SVM, and most neural classifiers. A generative model learns how the data is produced. It learns either the joint P(x, y) or P(x). Examples: naive Bayes, Gaussian mixtures, VAEs, GANs, diffusion models and language models.

I would look for a mismatch between offline and live conditions.

1. Define the problem: what counts as churn, how far ahead you predict, and what action the business takes. This sets the cost of false positives (a wasted offer) and false negatives (a lost customer).
2. Split by time, not randomly. Build features only from data before the prediction date. This prevents...

I would explain the trade-off. Pushing recall toward 100% means flagging many more good transactions. This hurts customers and costs reviewer time. Instead, agree on a target based on cost.

This is overfitting from epoch 10 onward. What I would do:

Watch the inputs, the predictions and the outcomes.

Loan decisions are regulated and affect people. So interpretability and fairness matter as much as accuracy.

With far more features than samples, overfitting is the main risk.

Suppose the same user (or patient, or device) appears in both train and test rows. The model can memorize patterns of that user. Then the score is too high for new users. Use group-aware splits (GroupKFold) when you must predict for new entities. Use time-based splits when you must predict future behavior. For cold-start problems, hold out whole users or items in validation. Make the split match how the model will be used in production. A random split answers a different question than the real one.