Statistics Interview Questions for Data Science

Statistics rounds in data science interviews are rarely about reciting formulas. The interviewer describes a result, "weekend visitors buy more", and wants to see whether you ask how sure we are, compared with what, and what else could explain it. These statistics interview questions are worked on a real dataset so that each answer comes with numbers, and each number comes with the caveat an interviewer is listening for.

The data is the UCI Online Shoppers Purchasing Intention set: 12,330 sessions on an online shop, each from a different user over one year, with time spent on pages, the month, whether it was a weekend and whether the session ended in a purchase (UCI: Online Shoppers). 15.5% of sessions ended in a purchase.

import numpy as np
import pandas as pd
from scipy import stats

s = pd.read_csv("online_shoppers_intention.csv")              # UCI Online Shoppers Purchasing Intention
s["bought"] = s["Revenue"].astype(int)
rng = np.random.default_rng(7)
print(f"{len(s):,} sessions, {s['bought'].mean():.1%} ended in a purchase")

d = s["ProductRelated_Duration"]
print(f"\nQ1 skew: time on product pages, mean {d.mean():.0f}s, median {d.median():.0f}s, max {d.max():,.0f}s")

sd = s["bought"].std()
print(f"\nQ2 SD vs SE of the purchase rate: SD {sd:.3f}; SE with n=100 {sd / 100 ** 0.5:.4f}, with n={len(s):,} {sd / len(s) ** 0.5:.4f}")

wk, wd = s.loc[s["Weekend"], "bought"], s.loc[~s["Weekend"], "bought"]
diff = wk.mean() - wd.mean()
print(f"\nQ3 weekend {wk.mean():.1%} vs weekday {wd.mean():.1%}: difference {diff * 100:.1f} points")
table = [[wk.sum(), len(wk) - wk.sum()], [wd.sum(), len(wd) - wd.sum()]]
print(f"   chi-square test p = {stats.chi2_contingency(table)[1]:.4f}")
labels, outcome = s["Weekend"].to_numpy(), s["bought"].to_numpy()
shuffled = [outcome[p].mean() - outcome[~p].mean() for p in (rng.permutation(labels) for _ in range(10_000))]
print(f"   permutation test p = {np.mean(np.abs(shuffled) >= abs(diff)):.4f} (10,000 shuffles)")

boot = [rng.choice(wk.to_numpy(), len(wk)).mean() - rng.choice(wd.to_numpy(), len(wd)).mean() for _ in range(5_000)]
lo, hi = np.percentile(boot, [2.5, 97.5])
print(f"\nQ4 bootstrap 95% interval for the difference: {lo * 100:.1f} to {hi * 100:.1f} points")

rates = s.pivot_table(index="Month", columns="Weekend", values="bought", aggfunc="mean")
weights = s["Month"].value_counts(normalize=True)
adjusted = ((rates[True] - rates[False]) * weights).sum()
share = s.groupby("Weekend")["Month"].value_counts(normalize=True).unstack()["Nov"]
print(f"\nQ5 confounding: November is {share[True]:.1%} of weekend sessions vs {share[False]:.1%} of weekday ones")
print(f"   difference within the same month, averaged: {adjusted * 100:.1f} points (raw {diff * 100:.1f})")

def region_pvalues(y):
    out = []
    for r in sorted(s["Region"].unique()):
        inside = s["Region"].to_numpy() == r
        t = [[y[inside].sum(), (1 - y[inside]).sum()], [y[~inside].sum(), (1 - y[~inside]).sum()]]
        out.append(stats.chi2_contingency(t)[1])
    return np.array(out)

real = region_pvalues(outcome)
print(f"\nQ6 nine regions each tested against the rest: {np.sum(real < 0.05)} with p < 0.05, {np.sum(real < 0.05 / 9)} after Bonferroni")
null_hits = np.mean([np.any(region_pvalues(rng.permutation(outcome)) < 0.05) for _ in range(300)])
print(f"   with purchases shuffled at random, at least one 'significant' region in {null_hits:.0%} of 300 runs")

x, y = s["ProductRelated_Duration"], s["PageValues"]
keep = x < x.quantile(0.99)
print(f"\nQ7 Pearson r {x.corr(y):.3f}, Spearman rho {x.corr(y, method='spearman'):.3f}; "
      f"Pearson without the top 1% of durations {x[keep].corr(y[keep]):.3f}")

p1, p2 = 0.155, 0.170
z = stats.norm.ppf(0.975) + stats.norm.ppf(0.80)
n = z ** 2 * (p1 * (1 - p1) + p2 * (1 - p2)) / (p2 - p1) ** 2
print(f"\nQ8 sessions per group to detect {p1:.1%} -> {p2:.1%} with 80% power at 5% significance: {np.ceil(n):,.0f}")

12,330 sessions, 15.5% ended in a purchase

Q1 skew: time on product pages, mean 1195s, median 599s, max 63,974s

Q2 SD vs SE of the purchase rate: SD 0.362; SE with n=100 0.0362, with n=12,330 0.0033

Q3 weekend 17.4% vs weekday 14.9%: difference 2.5 points
   chi-square test p = 0.0013
   permutation test p = 0.0011 (10,000 shuffles)

Q4 bootstrap 95% interval for the difference: 0.9 to 4.1 points

Q5 confounding: November is 27.6% of weekend sessions vs 23.3% of weekday ones
   difference within the same month, averaged: 1.8 points (raw 2.5)

Q6 nine regions each tested against the rest: 0 with p < 0.05, 0 after Bonferroni
   with purchases shuffled at random, at least one 'significant' region in 33% of 300 runs

Q7 Pearson r 0.053, Spearman rho 0.360; Pearson without the top 1% of durations 0.083

Q8 sessions per group to detect 15.5% -> 17.0% with 80% power at 5% significance: 9,492

Q1. When should you report the median instead of the mean?

When the data is skewed or has extreme values. Time on product pages had a mean of 1,195 seconds but a median of 599: half the sessions spent under ten minutes, while a long right tail, up to almost 64,000 seconds, pulled the mean up (NIST: skewness). A mean nearly twice the median is a quick sign of right skew. Report the median for "typical", and the mean when you need totals, such as total time or revenue.

Q2. What is the difference between standard deviation and standard error?

Standard deviation describes how much individual values vary; standard error describes how much an estimate, such as a mean, would vary from sample to sample. For the purchase indicator the standard deviation was 0.362 whatever the sample size, while the standard error of the purchase rate was 0.036 with 100 sessions and 0.0033 with all 12,330. The standard error shrinks with the square root of n, so quadrupling the data halves it.

Q3. What does a p-value mean, and can you get one without a formula?

The p-value is the probability of seeing a difference at least as large as the observed one if there were really no difference. Weekend sessions converted at 17.4% against 14.9% on weekdays. A chi-square test of independence gave p = 0.0013 (SciPy: chi2_contingency). A permutation test, which shuffles the weekend labels 10,000 times and counts how often the shuffled difference is as extreme, gave 0.0011 (SciPy: permutation test). Two methods agreeing is reassuring, and the permutation version is easy to explain to someone who distrusts formulas.

What not to say: that p = 0.0013 is the probability the null hypothesis is true, or that the effect is large. It is neither.

Q4. How do you put a confidence interval on a difference?

Resample. Drawing each group with replacement 5,000 times and taking the middle 95% of the differences gave 0.9 to 4.1 percentage points. The interval says more than the p-value: the weekend effect is probably real but could be as small as one point. Bootstrap intervals need no distribution formula, which is why interviewers like candidates who know them.

Q5. Could something else explain the weekend effect?

This is the question that separates good answers. Purchase rates differ a lot by month, and the months are not spread evenly across weekends and weekdays: November, a high-converting month, made up 27.6% of weekend sessions but 23.3% of weekday ones. Comparing weekends with weekdays within the same month and averaging gave 1.8 points instead of the raw 2.5. Part of the "weekend effect" was a November effect. Month is a confounder: it affects both the grouping and the outcome.

How to phrase it: "Before I believe the difference, I would check whether the groups differ in other ways that also drive the outcome, and compare within those groups."

Q6. Why is it a problem to run many tests?

Each test at the 5% level has a 5% chance of a false positive, so many tests make at least one false positive likely. Testing each of nine regions against the rest found no region with p below 0.05 in the real data. But when purchases were shuffled at random, removing any real effect, at least one region looked "significant" in 33% of 300 runs, close to the 37% that 1 − 0.95⁹ predicts. A Bonferroni correction divides the threshold by the number of tests, here 0.05 / 9; false discovery rate methods are less strict alternatives.

Q7. Pearson or Spearman correlation?

Pearson measures linear association and is sensitive to extreme values; Spearman works on ranks and measures monotonic association. Between product-page time and the page value score, Pearson gave 0.053 and Spearman 0.360. Removing the longest 1% of sessions only moved Pearson to 0.083, so outliers were not the whole story: the relationship is monotonic but far from a straight line. Plot before choosing, and remember that neither says anything about cause.

Q8. How many users do you need for an A/B test?

It depends on the baseline rate, the smallest lift worth detecting, the significance level and the power. Using the normal approximation for two proportions (NIST: sample sizes for proportions), detecting a rise from 15.5% to 17.0% with 80% power at 5% significance needs about 9,492 sessions per group. Halving the lift to detect roughly quadruples that number. Saying this before a test starts prevents the most common failure: stopping early on a noisy result.

How to answer statistics questions

• Give the number and its uncertainty together.
• Ask what else differs between the groups before claiming a cause.
• Count your tests.
• Prefer intervals and effect sizes to bare p-values.

The concepts behind these answers are taught in statistics for data science and statistical concepts every data analyst must know. For the experiment design side, see A/B testing for data scientists; for the companion set, probability interview questions; and for the full round, data science interview questions and answers.

Statistics for data work is part of the Data Science course.

Sources

• Online Shoppers Purchasing Intention dataset (UCI Machine Learning Repository) — UCI Machine Learning Repository
• scipy.stats.chi2_contingency (SciPy documentation) — SciPy
• scipy.stats.permutation_test (SciPy documentation) — SciPy
• Sample sizes required: testing proportions (NIST/SEMATECH e-Handbook of Statistical Methods) — NIST
• Measures of Skewness and Kurtosis (NIST/SEMATECH e-Handbook of Statistical Methods) — NIST