Feature Engineering Interview Questions
Feature engineering questions test judgement about what a model may know and how it sees the data. Six questions are answered with measurements on 41,188 real bank calls: a leaky feature worth 0.15 AUC, when scaling matters, target encoding done wrong, informative unknowns, U-shaped effects and misleading importances.
GloryTecks Team
Training & Placement Faculty
Last updated: 8 October 2026
Feature engineering questions in interviews are not about clever transformations. They test judgement: what the model is allowed to know at prediction time, how different models see the same column, and whether you can tell a real improvement from a leak. These feature engineering interview questions are answered with measurements on real data, so each answer has a before and after.
The data is the UCI Bank Marketing set: 41,188 phone calls from a Portuguese bank's term deposit campaigns, with client details, previous contacts, economic indicators and whether the client subscribed (UCI: Bank Marketing). Every score is ROC AUC on the same held-out quarter of the data unless marked otherwise.
import numpy as np
import pandas as pd
from sklearn.compose import make_column_transformer
from sklearn.ensemble import HistGradientBoostingClassifier, RandomForestClassifier
from sklearn.inspection import permutation_importance
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import roc_auc_score
from sklearn.model_selection import KFold, train_test_split
from sklearn.neighbors import KNeighborsClassifier
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler, TargetEncoder
calls = pd.read_csv("bank-additional-full.csv", sep=";") # UCI Bank Marketing
y = calls.pop("y").eq("yes").astype(int)
categorical = list(calls.select_dtypes(include=["object", "str"]).columns)
numeric = [c for c in calls.columns if c not in categorical]
train, test, y_train, y_test = train_test_split(calls, y, test_size=0.25, stratify=y, random_state=0)
def auc(model, columns, fit_on=train, score_on=test, labels=(y_train, y_test)):
model.fit(fit_on[columns], labels[0])
return roc_auc_score(labels[1], model.predict_proba(score_on[columns])[:, 1])
def gbm(columns):
cats = [c for c in columns if c in categorical]
return make_pipeline(make_column_transformer((OneHotEncoder(handle_unknown="ignore"), cats), remainder="passthrough"),
HistGradientBoostingClassifier(random_state=0))
everything = numeric + categorical
honest = [c for c in everything if c != "duration"]
print(f"F1 leakage: test AUC with duration {auc(gbm(everything), everything):.3f} | without {auc(gbm(honest), honest):.3f}")
nums = [c for c in numeric if c != "duration"]
knn_raw = auc(KNeighborsClassifier(50), nums)
knn_scaled = auc(make_pipeline(StandardScaler(), KNeighborsClassifier(50)), nums)
tree_raw = auc(HistGradientBoostingClassifier(random_state=0), nums)
tree_scaled = auc(make_pipeline(StandardScaler(), HistGradientBoostingClassifier(random_state=0)), nums)
print(f"\nF2 scaling: k-NN AUC raw {knn_raw:.3f} -> scaled {knn_scaled:.3f} | gradient boosting raw {tree_raw:.3f} -> scaled {tree_scaled:.3f}")
print(f" column ranges: pdays {np.ptp(calls['pdays'])}, nr.employed {np.ptp(calls['nr.employed']):.0f}, campaign {np.ptp(calls['campaign'])}, emp.var.rate {np.ptp(calls['emp.var.rate']):.1f}")
print(f" k-NN on the two widest columns alone, unscaled: {auc(KNeighborsClassifier(50), ['pdays', 'nr.employed']):.3f}")
combo = lambda d: d["job"] + "|" + d["education"] + "|" + d["month"] + "|" + d["day_of_week"]
tr_c, te_c = combo(train).to_frame("combo"), combo(test).to_frame("combo")
print(f"\nF3 a {combo(calls).nunique():,}-level category, encoded by its subscription rate")
rates = y_train.groupby(tr_c["combo"]).mean()
naive_train = tr_c["combo"].map(rates)
naive_test = te_c["combo"].map(rates).fillna(y_train.mean())
print(f" naive mean encoding: train AUC {roc_auc_score(y_train, naive_train):.3f} | test AUC {roc_auc_score(y_test, naive_test):.3f}")
enc = TargetEncoder(cv=KFold(5, shuffle=True, random_state=0))
cross_fitted = enc.fit_transform(tr_c, y_train)[:, 0]
print(f" TargetEncoder (cross-fitted): train AUC {roc_auc_score(y_train, cross_fitted):.3f} | test AUC {roc_auc_score(y_test, enc.transform(te_c)[:, 0]):.3f}")
print("\nF4 'unknown' can be information: subscription rate by credit default status")
print(" ", y.groupby(calls["default"]).agg(["mean", "size"]).round(3).to_dict("index"))
print("\nF5 a U-shaped effect in a linear model: age")
bands = lambda d: pd.cut(d["age"], [0, 25, 35, 50, 65, 120], right=False).astype(str).to_frame()
logit = make_pipeline(OneHotEncoder(handle_unknown="ignore"), LogisticRegression(max_iter=1000))
raw_age = auc(make_pipeline(StandardScaler(), LogisticRegression(max_iter=1000)), ["age"])
logit.fit(bands(train), y_train)
print(f" logistic regression: age as a number {raw_age:.3f} | age as bands {roc_auc_score(y_test, logit.predict_proba(bands(test))[:, 1]):.3f}")
rng = np.random.default_rng(0)
noisy, noisy_test = train[nums].assign(random_id=rng.integers(0, 10_000, len(train))), test[nums].assign(random_id=rng.integers(0, 10_000, len(test)))
forest = RandomForestClassifier(200, min_samples_leaf=1, random_state=0, n_jobs=-1).fit(noisy, y_train)
impurity = pd.Series(forest.feature_importances_, noisy.columns).sort_values(ascending=False)
perm = permutation_importance(forest, noisy_test, y_test, scoring="roc_auc", n_repeats=5, random_state=0)
perm = pd.Series(perm.importances_mean, noisy.columns)
print(f"\nF6 a random ID column: impurity importance rank {list(impurity.index).index('random_id') + 1} of {len(impurity)} "
f"({impurity['random_id']:.3f}) | permutation importance on test {perm['random_id']:.4f}")F1 leakage: test AUC with duration 0.951 | without 0.800
F2 scaling: k-NN AUC raw 0.772 -> scaled 0.773 | gradient boosting raw 0.792 -> scaled 0.792
column ranges: pdays 999, nr.employed 264, campaign 55, emp.var.rate 4.8
k-NN on the two widest columns alone, unscaled: 0.755
F3 a 2,638-level category, encoded by its subscription rate
naive mean encoding: train AUC 0.806 | test AUC 0.636
TargetEncoder (cross-fitted): train AUC 0.634 | test AUC 0.637
F4 'unknown' can be information: subscription rate by credit default status
{'no': {'mean': 0.129, 'size': 32588}, 'unknown': {'mean': 0.052, 'size': 8597}, 'yes': {'mean': 0.0, 'size': 3}}
F5 a U-shaped effect in a linear model: age
logistic regression: age as a number 0.494 | age as bands 0.590
F6 a random ID column: impurity importance rank 1 of 10 (0.358) | permutation importance on test 0.0032F1. What is data leakage, and how do you spot it?
Leakage is using information that would not be available at prediction time, which makes a model look better than it can ever be in production (scikit-learn: common pitfalls). This dataset has a famous example: duration, the length of the call. With it, gradient boosting reached an AUC of 0.951; without it, 0.800. The dataset's own documentation says why. Duration is not known before the call, and once the call ends the outcome is known, so the column should be dropped for any realistic model.
How to spot it in an interview: for every feature, ask "when does this value become known?" A single feature that lifts performance dramatically deserves suspicion before celebration.
F2. Which models need feature scaling?
Models that use distances or penalties: k-nearest neighbours, SVMs, regularised linear models and neural networks. Tree models split on one feature at a time, so scaling cannot change them: gradient boosting scored 0.792 before and after.
The k-NN result needs more care. Scaling moved it only from 0.772 to 0.773, which seems to contradict the textbook. The ranges explain it: pdays spans 999 and nr.employed 264, so they dominate unscaled distances (scikit-learn: preprocessing). Those two columns happen to be predictive, scoring 0.755 on their own. Unscaled k-NN was effectively a two-feature model that got lucky. The right answer is not "always scale" but "scaling decides which features dominate; without it, the widest column wins, whether or not it matters".
F3. How do you encode a category with thousands of levels?
One-hot encoding creates thousands of sparse columns; target encoding replaces each level with the target rate for that level. Done naively, it leaks. Combining job, education, month and weekday gave 2,638 levels, many with only a few rows. Naive mean encoding scored 0.806 AUC on the training data but 0.636 on the test data: for rare levels, each row's own label went into its encoding. scikit-learn's TargetEncoder uses cross fitting, encoding each training row with rates computed on other folds. Its training AUC of 0.634 matched the test AUC of 0.637, so the training score could be trusted.
The phrase to use: "target encoding must be fitted out of fold, or the training score is fiction."
F4. Should you always impute or drop "unknown" values?
No: missingness can carry information. The credit default field is "unknown" for 8,597 clients, and their subscription rate was 5.2% against 12.9% for clients recorded as "no". Dropping those rows would discard a fifth of the data; imputing "no" would erase a strong signal. Keeping "unknown" as its own category, or adding a missing-value indicator, lets the model use it. Then ask why the value is missing, because the reason may change after deployment.
F5. How do you help a linear model with a non-linear effect?
Transform the feature so the relationship becomes something a line can fit. Subscription is U-shaped in age: high for the young and the old, low in middle age. Logistic regression on age as a number scored 0.494 AUC, worse than chance, because no single slope fits a U. The same model with five age bands scored 0.590. Splines or polynomial terms do the same job more smoothly. Tree models find such shapes on their own, which is one reason feature engineering matters more for linear models.
F6. Can you trust a random forest's feature importances?
Not the default ones. Adding a column of random IDs, with no information at all, made it the most important of ten features by impurity importance, at 0.358. Impurity importance is measured on the training data and favours features with many distinct values (scikit-learn: permutation importance), so a forest that memorises IDs rewards them. Permutation importance on the test set gave the same column 0.0032, close to nothing. Use permutation importance on held-out data, or SHAP values, before explaining a model to anyone.
How to answer feature engineering questions
- Ask when each value becomes known. That one question catches most leakage.
- Know how your model sees features: distances, splits or a weighted sum.
- Fit every transformation inside the training folds, including encoders and scalers, ideally in a pipeline.
- Measure before and after on the same held-out data.
The full walkthrough of building features step by step is in feature engineering: a practical guide, and handling the 11% positive rate here is covered in handling imbalanced datasets. For production features, see feature stores explained; for the full round, data science interview questions and answers.
What are common feature engineering interview questions?
What is target leakage?
When should I use target encoding instead of one-hot encoding?
Do tree-based models need feature engineering?
Feature engineering is part of the Data Science course.
Sources
- Bank Marketing dataset (UCI Machine Learning Repository) — UCI Machine Learning Repository
- Common pitfalls and recommended practices (scikit-learn documentation) — scikit-learn
- Preprocessing data (scikit-learn documentation) — scikit-learn
- Permutation feature importance (scikit-learn documentation) — scikit-learn
GloryTecks Team
Training & Placement Faculty
The GloryTecks faculty team mentors learners across Data Science, AI and Data Engineering, with a focus on real-time projects and placement readiness in Hyderabad.
Want to master Interview Questions?
Join GloryTecks’ expert-led Data Science training in Hyderabad with real-time projects and 100% placement support.
Time Series Interview Questions for Data Science
NextData Warehouse Interview Questions and Answers
Related articles
Data Engineering Interview Questions and Answers
Data engineering interviews test whether you can move data reliably at the right scale. This guide maps the rounds with model answers, then works a design-round sizing task from real New York taxi data, where the numbers show that the obvious big-data design is the wrong one.
SQL Interview Questions: Top 50 with Answers
Fifty SQL interview questions with answers checked on PostgreSQL, DuckDB and SQLite against one small practice schema: evaluation order, NULL logic, joins and fan-out, window functions, dates, and performance, including where the three engines disagree.
Python Interview Questions for Data Roles
The plain-Python coding tasks data-analyst and data-engineer interviews use, without pandas: counting, deduplicating, grouping, streaming large files, flattening JSON, joining, and the bugs interviewers plant. Each has a worked solution and the follow-up questions to expect.