Pandas Interview Questions Every Analyst Should Know

Pandas interview questions for analysts tend to look simple: group this, merge that, fill the gaps. What they test is whether you know what pandas does without telling you. It drops missing keys from groups, multiplies rows in a merge, ignores assignments that look valid, and treats a placeholder like 999 as a real number. Each question below is answered on real data with pandas 3, so you can see the behaviour rather than take it on trust.

The data is the UCI Bank Marketing set: 41,188 phone calls from a Portuguese bank's term deposit campaigns between May 2008 and November 2010, with the client's job, education, previous contacts and whether they subscribed (UCI: Bank Marketing).

import warnings

import pandas as pd

calls = pd.read_csv("bank-additional-full.csv", sep=";")      # UCI Bank Marketing: one row per client contact
print("P1 shape and the target:", calls.shape, calls["y"].value_counts().to_dict())

print("\nP2 hidden missing values")
print("  pdays == 999 (never contacted before):", f"{(calls['pdays'] == 999).mean():.1%}")
unknown = (calls == "unknown").sum()
print("  'unknown' strings per column:", unknown[unknown > 0].to_dict())
print("  isna() finds:", int(calls.isna().sum().sum()))
clean = calls.replace("unknown", pd.NA).assign(pdays=lambda d: d["pdays"].mask(d["pdays"] == 999))
print("  after replacing them, isna() finds:", clean.isna().sum()[lambda s: s > 0].to_dict())

print("\nP3 agg vs transform: subscription rate by job")
clean["subscribed"] = clean["y"].eq("yes")
by_job = clean.groupby("job")["subscribed"].agg(["mean", "size"]).sort_values("mean", ascending=False)
print(by_job.head(3).round(3).to_string())
clean["job_rate"] = clean.groupby("job")["subscribed"].transform("mean")
print("  transform keeps one value per row:", clean[["job", "job_rate"]].shape)

print("\nP4 groupby drops missing keys unless told otherwise")
print("  groups with default dropna=True :", clean.groupby("job").ngroups)
print("  groups with dropna=False        :", clean.groupby("job", dropna=False).ngroups)

print("\nP5 chained assignment under copy-on-write (pandas 3)")
with warnings.catch_warnings(record=True) as caught:
    warnings.simplefilter("always")
    clean[clean["age"] > 90]["age"] = 90                     # looks like an update
print("  rows still above 90 after chained assignment:", int((clean["age"] > 90).sum()),
      "| warning:", type(caught[0].message).__name__ if caught else None)
clean.loc[clean["age"] > 90, "age"] = 90                     # one .loc call does it
print("  rows above 90 after .loc:", int((clean["age"] > 90).sum()))

print("\nP6 merge with validate catches a duplicated lookup key")
sectors = pd.DataFrame({"job": ["admin.", "blue-collar", "technician", "admin."],
                        "sector": ["office", "manual", "technical", "office"]})
try:
    clean.merge(sectors, on="job", validate="many_to_one")
except pd.errors.MergeError as error:
    print("  MergeError:", str(error).splitlines()[0])
print("  rows if you merge anyway:", len(clean.merge(sectors, on="job", how="left")), "instead of", len(clean))

print("\nP7 pivot vs pivot_table")
try:
    clean.pivot(index="month", columns="day_of_week", values="subscribed")
except ValueError as error:
    print("  pivot:", error)
table = clean.pivot_table(index="month", columns="day_of_week", values="subscribed", aggfunc="mean")
counts = clean.pivot_table(index="month", columns="day_of_week", values="subscribed", aggfunc="size")
best = table.stack().idxmax()
print("  pivot_table:", table.shape, "| best cell:", best, f"{table.stack().max():.1%} of {counts.stack()[best]} calls",
      f"| median cell size {int(counts.stack().median())}")

print("\nP8 binning with pd.cut")
bands = pd.cut(clean["age"], bins=[0, 25, 35, 50, 65, 120], right=False)
print(clean.groupby(bands, observed=True)["subscribed"].mean().round(3).to_string())

P1 shape and the target: (41188, 21) {'no': 36548, 'yes': 4640}

P2 hidden missing values
  pdays == 999 (never contacted before): 96.3%
  'unknown' strings per column: {'job': 330, 'marital': 80, 'education': 1731, 'default': 8597, 'housing': 990, 'loan': 990}
  isna() finds: 0
  after replacing them, isna() finds: {'job': 330, 'marital': 80, 'education': 1731, 'default': 8597, 'housing': 990, 'loan': 990, 'pdays': 39673}

P3 agg vs transform: subscription rate by job
             mean  size
job                    
student     0.314   875
retired     0.252  1720
unemployed  0.142  1014
  transform keeps one value per row: (41188, 2)

P4 groupby drops missing keys unless told otherwise
  groups with default dropna=True : 11
  groups with dropna=False        : 12

P5 chained assignment under copy-on-write (pandas 3)
  rows still above 90 after chained assignment: 10 | warning: ChainedAssignmentError
  rows above 90 after .loc: 0

P6 merge with validate catches a duplicated lookup key
  MergeError: Merge keys are not unique in right dataset; not a many-to-one merge
  rows if you merge anyway: 51610 instead of 41188

P7 pivot vs pivot_table
  pivot: Index contains duplicate entries, cannot reshape
  pivot_table: (10, 5) | best cell: ('dec', 'tue') 60.0% of 25 calls | median cell size 760

P8 binning with pd.cut
age
[0, 25)      0.240
[25, 35)     0.122
[35, 50)     0.084
[50, 65)     0.116
[65, 120)    0.472

P1. What do you do first with a new DataFrame?

Check the shape, the column types and the target's balance. Here 4,640 of 41,188 calls ended in a subscription, about 11%. That one number changes later answers: a model that always predicts "no" is right 89% of the time, so accuracy is a poor yardstick for this data. Saying so early shows you look at the data before you analyse it.

P2. How do you find missing values that isna() does not see?

isna() found nothing, yet the data is full of gaps. Six text columns use the string "unknown", 8,597 times in default alone, and pdays uses 999 for "never contacted before", which is 96.3% of rows. Left as they are, "unknown" becomes a category of its own and 999 drags every average of pdays upwards. Look for placeholders with value_counts() on each column, then convert them with replace and mask to proper missing values. After that, isna() reported them all.

The follow-up: do not automatically drop or fill them. "Unknown" credit default status may itself tell you something about a client.

P3. What is the difference between agg and transform?

agg returns one row per group; transform returns one value per original row, aligned to the input. The subscription rate by job, computed with agg, showed students at 31.4% and retired clients at 25.2%. The same calculation with transform gave all 41,188 rows a job_rate column, which is what you want when comparing each client with their group, or when building features for a model.

P4. Why does my groupby have fewer groups than expected?

By default, groupby leaves out rows whose key is missing (pandas: group by). After P2 turned "unknown" jobs into missing values, groupby("job") returned 11 groups, and the 330 clients with an unknown job vanished from every total. dropna=False keeps them as a twelfth group. Totals that do not add up to the row count are the usual symptom.

P5. Why did my assignment not change the DataFrame?

Because of chained assignment. df[df.age > 90]["age"] = 90 first creates a filtered result, then assigns to that result, not to df. In pandas 3, Copy-on-Write is the only mode, and chained assignment never updates the original (pandas: copy-on-write). The run showed exactly that: ten clients were still over 90 afterwards, and pandas raised a ChainedAssignmentError warning. A single .loc[rows, column] = value call did the update.

Worth adding: in pandas 2 the same line sometimes worked and sometimes did not, with a SettingWithCopyWarning. Copy-on-Write made the behaviour consistent.

P6. How do you stop a merge from multiplying rows?

Pass validate. A lookup table that lists admin. twice looks harmless, but every admin client then matches two rows. validate="many_to_one" checks that the right-hand keys are unique and raised a MergeError (pandas: merge). Merging without it produced 51,610 rows from 41,188, silently, and every later sum or rate would have been wrong. Check len() before and after any merge as well.

P7. When does pivot fail, and what do you use instead?

pivot only reshapes; it cannot combine rows, so it fails when an index and column pair appears more than once, as month and weekday do here: "Index contains duplicate entries". pivot_table aggregates, with a mean by default. The answer then hides a second trap. The best-looking cell, Tuesdays in December at 60.0%, rests on 25 calls, while the median cell has 760. Show counts next to rates before anyone acts on the highest number.

P8. How do you turn a number into bands?

pd.cut with explicit edges, and right=False so that 25 falls in "25 to 34" rather than "up to 25". Grouping by the bands with observed=True returns only bands that occur. Subscription rates formed a U-shape: 24.0% under 25, 8.4% for ages 35 to 49, and 47.2% from 65. Use pd.qcut instead when you want bands of equal size rather than equal width.

How to handle pandas questions in the room

• Check sizes after every step: shape after merges and filters, group counts after groupby.
• Look for placeholders before trusting isna().
• Make conditional updates in one step, with a single .loc call.
• Name the version. Copy-on-Write and the new string type arrived in pandas 3, so say which version your answer assumes.

For broader analyst preparation, see data analyst interview questions and answers; for standard-library Python tasks, Python interview questions for data roles. The library itself is covered in the pandas guide, and when to reach for SQL instead in pandas vs SQL for data analysis.

Pandas for analysis is part of the Data Analytics course.

Sources

• Copy-on-Write (pandas documentation) — pandas
• pandas.merge (pandas documentation) — pandas
• Group by: split-apply-combine (pandas documentation) — pandas
• Bank Marketing dataset (UCI Machine Learning Repository) — UCI Machine Learning Repository