Correlation vs Causation: A Must-Know for Analysts

"Correlation does not imply causation" is one of the best-known phrases in statistics, and one of the most often ignored in practice. Two quantities that move together may do so because one causes the other, because the other causes the first, because a third factor drives both, or by coincidence (Wikipedia: correlation does not imply causation). For an analyst, telling these apart is not academic: a dashboard that implies a false cause leads directly to money spent on the wrong lever.

The examples here use real data, because real confounding is subtler than the classic joke examples.

A confounder at work: humidity and bike rentals

Hourly bike rentals in Washington, D.C., come with weather measurements (UCI: bike sharing dataset). Across all hours, humidity and rentals are clearly negatively correlated. Is humid air putting people off cycling?

import numpy as np
import pandas as pd
from sklearn.datasets import fetch_openml
from sklearn.linear_model import LinearRegression

bikes = fetch_openml("Bike_Sharing_Demand", version=2, as_frame=True).frame
print("correlation of humidity with rentals, all hours:", round(bikes["humidity"].corr(bikes["count"]), 3))
by_hour = bikes.groupby("hour")[["humidity", "count"]].mean()
print("hourly averages at 4 am and 5 pm:", by_hour.loc[[4, 17]].round(2).to_dict("index"))
print("correlation of the 24 hourly averages:", round(by_hour["humidity"].corr(by_hour["count"]), 3))

correlation of humidity with rentals, all hours: -0.323
hourly averages at 4 am and 5 pm: {4: {'humidity': 0.74, 'count': 6.35}, 17: {'humidity': 0.51, 'count': 461.45}}
correlation of the 24 hourly averages: -0.736

Look at the time of day. At 4 am, humidity averages 0.74 and almost nobody rents a bike; at 5 pm, humidity is 0.51 and rentals peak. Nights are humid because air cools, and nights are quiet because people are asleep. Hour of day drives both, which makes it a confounder: a variable that influences both the supposed cause and the effect (Wikipedia: confounding). The 24 hourly averages alone correlate at −0.74.

To see what is left of the relationship, remove the hour-of-day effect from both variables and correlate what remains, or look within a single hour:

hours = pd.get_dummies(bikes["hour"], dtype=float)
humidity_left = bikes["humidity"] - LinearRegression().fit(hours, bikes["humidity"]).predict(hours)
rentals_left = bikes["count"] - LinearRegression().fit(hours, bikes["count"]).predict(hours)
print("correlation after removing hour-of-day effects:", round(np.corrcoef(humidity_left, rentals_left)[0, 1], 3))
within = bikes.groupby("hour")[["humidity", "count"]].apply(lambda g: g["humidity"].corr(g["count"]))
print("median correlation within a single hour:", round(within.median(), 3))

correlation after removing hour-of-day effects: -0.131
median correlation within a single hour: -0.091

Comparing like with like, hour by hour, the correlation shrinks from −0.32 to about −0.1. Some negative relationship remains, plausibly because very humid hours often come with rain, but most of the original correlation was the daily rhythm, not the weather. An analyst who reported "humidity strongly reduces rentals" from the first number would have been mostly wrong. And even the remaining −0.13 is still only an association: other factors, such as season, could be at work.

Trends correlate with trends

Anything that grows over time correlates with anything else that grows over time. Two completely independent random series that drift, like many business metrics, show how often this happens:

rng = np.random.default_rng(0)
walks = [np.corrcoef(np.cumsum(rng.normal(size=100)), np.cumsum(rng.normal(size=100)))[0, 1] for _ in range(1000)]
print("pairs of unrelated trending series with |r| > 0.5:", f"{np.mean(np.abs(walks) > 0.5):.1%}")
changes = [np.corrcoef(rng.normal(size=100), rng.normal(size=100))[0, 1] for _ in range(1000)]
print("same, using period-to-period changes:", f"{np.mean(np.abs(changes) > 0.5):.1%}")

pairs of unrelated trending series with |r| > 0.5: 39.4%
same, using period-to-period changes: 0.0%

Nearly four in ten pairs of unrelated drifting series correlate strongly, positively or negatively. Correlating their period-to-period changes instead, which removes the drift, produces no strong correlations at all. This is the spurious regression problem known in time series analysis, where non-stationary series give misleading evidence of a relationship (Wikipedia: spurious relationship). Before claiming that marketing spend drives revenue because both rose over two years, compare changes, or better, test the claim.

Other ways a correlation misleads

• Reverse causation. Stores with more staff have more sales, but busy stores are given more staff.
• Selection. Customers who use a premium feature churn less, but engaged customers were both more likely to try it and more likely to stay.
• Aggregation. A relationship in totals can reverse inside groups, the pattern known as Simpson's paradox; statistics for data science works through a famous case.
• Chance. Test enough metric pairs and some will correlate by luck alone.

A business example, reasoned through

Suppose a marketing team reports that customers who opened last month's newsletter spent 40% more this month than those who did not, and proposes doubling the newsletter budget. Before agreeing, an analyst asks:

• Who opens newsletters? Already engaged customers, who would probably have spent more anyway. Engagement is a confounder, and selection is at work.
• What came first? If openers were also higher spenders before the newsletter, the gap existed before the email.
• What would a fair comparison look like? Compare customers with similar past spending, or better, send the newsletter to a random half of a customer group and compare the two halves' spending afterwards.

If the randomised comparison shows a real difference, the budget case is strong. If the difference shrinks to nothing, the newsletter was reaching good customers, not creating them. Either way, the decision rests on evidence rather than on a correlation that engagement explains.

How analysts move from correlation to cause

1. Ask what else could explain it. List plausible confounders, such as time, season, customer type or region, and compare within them, as with the hour of day above.
2. Check timing. A cause must come before its effect. If the effect moves first, the story is wrong.
3. Look for a mechanism that makes business sense, and for evidence of it in other data.
4. Run an experiment when you can. Randomly assigning who gets a change breaks the link to confounders, which is why A/B tests are the standard for product and marketing decisions. A/B testing for data scientists covers how.
5. Use natural experiments when you cannot randomise: a change rolled out in one region first, or a policy that started on a known date.
6. Word findings honestly. "Associated with" is not "drives". Report what was controlled for and what was not.

Business analytics is the wider discipline this skill belongs to; business analytics explained puts it in context. Correlation analysis and trend and pattern analysis are part of the exploratory data analysis module of the Data Analytics course.

Sources

• Correlation does not imply causation (Wikipedia) — Wikipedia
• Confounding (Wikipedia) — Wikipedia
• Spurious relationship (Wikipedia) — Wikipedia
• Bike Sharing (UCI Machine Learning Repository) — UCI Machine Learning Repository