Real-Time Feature Engineering for Online Models
Online models need features from events seconds old, and those features must match the ones the model was trained on. On a month of real New York taxi pickups, this guide builds a streaming window counter, shows a one-character boundary change breaking 3.7% of lookups, and measures what a two-minute lag costs.
GloryTecks Team
Training & Placement Faculty
Last updated: 8 October 2026
Online models predict on live events: a card payment, a ride request, a page view. Their most useful inputs often summarise the last few minutes, such as payments on this card in the past hour or ride requests in this area in the past 15 minutes. These real-time features are hard for two reasons. They must be computed incrementally as events arrive. And they must equal what the model saw in training, where the same features were computed in hindsight from complete data.
This guide builds a streaming feature on a month of real taxi data, checks it against the training definition, and measures what a short delay in the data costs a model.
Windows on a stream
Most real-time features are aggregates over a sliding window per entity: counts, sums, averages or time since the last event. Each needs explicit answers to a few questions. Is the window's edge included? Which clock defines "the last 15 minutes"? What happens to events that arrive late?
Apache Flink's documentation separates the two clocks. Processing time is the clock of the machine handling an event; event time is when the event actually happened. Its watermarks track progress in event time, and waiting for out-of-order events costs latency and can only last so long (Apache Flink: timely stream processing).
Two implementations, one definition
Training features are usually computed in batch, with SQL or pandas over stored history. Serving features come from a stream processor. Google's Rules of ML recommend reusing code between training and serving where possible, precisely because batch and online processing differ (Google: Rules of ML). When the code cannot be shared, the two implementations need to be tested against each other.
The experiment
The data are NYC Taxi and Limousine Commission yellow taxi trips for January 2024 (NYC TLC: trip record data). The feature is pickups in a zone over the last 15 minutes, defined as events after t minus 15 minutes and up to t, plus a 60-minute version. It is requested every 5 minutes for each of the 20 busiest pickup zones.
The batch side counts events by binary search on sorted pickup times. The serving side is a small streaming counter that keeps a queue of event times per zone and drops old ones as time moves on. A model then predicts pickups in the next 15 minutes, the kind of demand forecast a dispatch system uses.
from collections import defaultdict, deque
import numpy as np
import pandas as pd
from sklearn.ensemble import HistGradientBoostingRegressor
trips = pd.read_parquet("yellow_tripdata_2024-01.parquet", columns=["tpep_pickup_datetime", "PULocationID"])
trips = trips[trips.tpep_pickup_datetime.between("2024-01-01", "2024-01-31 23:59:59")]
busiest = trips.PULocationID.value_counts().head(20).index
events = trips[trips.PULocationID.isin(busiest)].sort_values("tpep_pickup_datetime", kind="stable")
seconds = (events.tpep_pickup_datetime - pd.Timestamp("2024-01-01")).dt.total_seconds().astype("int64").to_numpy()
zones = events.PULocationID.to_numpy()
ask_at = np.arange(3600, 31 * 86400 - 900 + 1, 300) # every 5 minutes, leaving room for the 15-minute target
print(f"January 2024: {len(trips):,} pickups, {len(events):,} in the 20 busiest zones; "
f"{len(ask_at) * len(busiest):,} feature lookups")
def batch_counts(times, ends, width, lag=0): # the training-side definition: events in (t - width, t - lag]
return np.searchsorted(times, ends - lag, "right") - np.searchsorted(times, ends - width, "right")
class SlidingCount: # the serving side: per-zone count, updated as events arrive
def __init__(self, width, keep_boundary=False):
self.width, self.keep_boundary, self.windows = width, keep_boundary, defaultdict(deque)
def add(self, key, when):
self.windows[key].append(when)
def count(self, key, now):
window = self.windows[key]
while window and (window[0] < now - self.width if self.keep_boundary else window[0] <= now - self.width):
window.popleft()
return len(window)
exact, sloppy, largest = SlidingCount(900), SlidingCount(900, keep_boundary=True), 0
streamed, position = [], 0
for now in ask_at:
while position < len(seconds) and seconds[position] <= now:
exact.add(zones[position], seconds[position])
sloppy.add(zones[position], seconds[position])
position += 1
for zone in busiest:
streamed.append((zone, now, exact.count(zone, now), sloppy.count(zone, now)))
largest = max(largest, len(exact.windows[zone]))
streamed = pd.DataFrame(streamed, columns=["zone", "t", "stream", "sloppy"])
table = []
for zone in busiest:
times = seconds[zones == zone]
table.append(pd.DataFrame({
"zone": zone, "t": ask_at,
"last15": batch_counts(times, ask_at, 900), "last60": batch_counts(times, ask_at, 3600),
"lag15": batch_counts(times, ask_at, 900, lag=120), "lag60": batch_counts(times, ask_at, 3600, lag=120),
"next15": batch_counts(times, ask_at + 900, 900)}))
table = pd.concat(table, ignore_index=True).merge(streamed, on=["zone", "t"])
table["hour"], table["weekday"] = (table.t // 3600) % 24, (table.t // 86400) % 7 # 1 January 2024 was a Monday
print("\n1. One definition, two implementations")
print(f" streaming counter vs batch query: {(table.stream != table.last15).sum():,} lookups differ")
print(f" streaming counter that keeps the boundary event: {(table.sloppy != table.last15).sum():,} differ "
f"({(table.sloppy != table.last15).mean():.1%})")
print(f" most events held in one 15-minute window: {largest}")
print("\n2. A two-minute ingestion lag")
print(f" 15-minute counts are {1 - table.lag15.sum() / table.last15.sum():.1%} lower than the complete-data values")
print(f"\n3. Predicting pickups in the next 15 minutes (average {table.next15.mean():.1f}); train 1-21 Jan, test 22-31 Jan")
train, test = table[table.t < 21 * 86400], table[table.t >= 21 * 86400]
complete, lagged = ["zone", "last15", "last60", "hour", "weekday"], ["zone", "lag15", "lag60", "hour", "weekday"]
on_complete = HistGradientBoostingRegressor(random_state=0).fit(train[complete], train.next15)
on_lagged = HistGradientBoostingRegressor(random_state=0).fit(train[lagged], train.next15)
served_lagged = test[lagged].set_axis(complete, axis=1)
for label, model, features in (("trained on complete counts, served complete counts", on_complete, test[complete]),
("trained on complete counts, served lagged counts", on_complete, served_lagged),
("trained on lagged counts, served lagged counts", on_lagged, test[lagged])):
print(f" {label:<52} MAE {np.abs(model.predict(features) - test.next15).mean():.2f}")January 2024: 2,964,606 pickups, 1,903,677 in the 20 busiest zones; 178,280 feature lookups
1. One definition, two implementations
streaming counter vs batch query: 0 lookups differ
streaming counter that keeps the boundary event: 6,547 differ (3.7%)
most events held in one 15-minute window: 284
2. A two-minute ingestion lag
15-minute counts are 13.3% lower than the complete-data values
3. Predicting pickups in the next 15 minutes (average 32.0); train 1-21 Jan, test 22-31 Jan
trained on complete counts, served complete counts MAE 6.30
trained on complete counts, served lagged counts MAE 6.71
trained on lagged counts, served lagged counts MAE 6.39What the output shows
- One definition, identical answers. The streaming counter matched the batch query on all 178,280 lookups.
- One character broke it. Keeping the event that sits exactly on the window's edge, with "less than" in place of "less than or equal", changed 6,547 lookups, 3.7%. Pickup times are recorded to the second, so events land on window edges more often than intuition suggests. Write the boundary rule down, and test the two implementations against each other on real data.
- State stayed small. The busiest zone held at most 284 events in a 15-minute window. Memory grows with entities times events per window, so long windows over millions of keys need pre-aggregated buckets, such as per-minute counts, instead of raw events.
- A two-minute lag cost 13.3% of every count. If events reach the feature pipeline two minutes late, the live 15-minute count is that much lower than the complete-data value used in training.
- Training on hindsight hurt. A model trained on complete counts and served lagged ones had a mean absolute error of 6.71 pickups, against 6.30 when served the counts it was trained on. Retraining on counts as they would have been available, with the same lag, brought the error back to 6.39.
The general rule: train on features as they will be served. Rule 31 of the same Google guide makes a related point about joined tables, which can change between training and serving; its simplest fix is to log features at serving time and train on those logs.
Design choices that matter
- Choose the clock deliberately. For live features, what matters is what was known when the prediction was made.
- Decide what late events do: drop them, wait for them, or correct the feature afterwards. Training data must follow the same rule.
- Pre-aggregate long windows into small time buckets to bound state.
- Monitor ingestion lag as a first-class metric, since it changes feature values silently.
- Pick the runtime to match the scale. Stream processors such as Flink, Spark Structured Streaming or Kafka Streams compute the windows, and an online store serves the results; see feature stores explained.
For the event streams themselves, see Apache Kafka explained and a streaming pipeline with Kafka and Spark. For whether a model needs real-time inference at all, see batch vs real-time model inference.
Common mistakes
- Different window boundaries in the batch and streaming code.
- Training on complete, hindsight data for features served with delay.
- Unbounded state from long windows over many keys.
- Mixed time zones or clocks between producers and the pipeline.
- No alert on ingestion lag.
What are real-time features in machine learning?
How do you keep online and offline features consistent?
What is the difference between event time and processing time?
Do real-time features need a feature store?
Feature engineering basics and feature stores are covered in the MLOps course, in its Machine Learning Fundamentals and Advanced MLOps Concepts modules.
Sources
GloryTecks Team
Training & Placement Faculty
The GloryTecks faculty team mentors learners across Data Science, AI and Data Engineering, with a focus on real-time projects and placement readiness in Hyderabad.
Want to master MLOps?
Join GloryTecks’ expert-led MLOps training in Hyderabad with real-time projects and 100% placement support.
Related articles
Feature Stores Explained: Why Teams Need Them
A feature store keeps the features a model trains on and the features it is served the same. This guide builds one with Feast on a year of real retail orders and shows the problem it solves: a naive join that looked better in validation, 0.798 against 0.727, then did worse in production.
MLOps Roadmap 2026: Skills, Tools and Career Path
MLOps sits between data science, software engineering and operations, which makes it hard to know where to start. This roadmap orders the skills from foundations to production operations, gives each stage a project that proves the skill, and explains what the MLOps engineer role actually involves.
MLflow Tutorial: Tracking Experiments and Models
MLflow records what produced each model so you never have to remember. This tutorial trains three models on real bike-sharing data, logs every run, compares them by querying the tracking store, registers the best one, points a champion alias at it and loads it back by that alias, all on a laptop.