Tumbling Window Triggers: Backfills and Dependencies

A schedule trigger answers "when should the pipeline start?". A tumbling window trigger answers a different question: "which slice of time is this run responsible for?". That shift, from clock time to data time, is what makes loads repeatable, and it is why the tumbling window trigger is the right tool for most incremental pipelines even though the schedule trigger is the one people reach for first.

The model

Microsoft defines tumbling windows as a series of fixed-sized, non-overlapping and contiguous time intervals, fired at a periodic interval from a specified start time while retaining state (Microsoft Learn: tumbling window trigger).

Each phrase carries weight:

• Fixed-sized, non-overlapping, contiguous. Every moment belongs to exactly one window. No gaps, no double coverage.
• From a specified start time. The start can be in the past.
• Retaining state. The trigger knows which windows have run and how each ended.

A run is launched for each window and told that window's start and end. The pipeline's job is to process the data belonging to that interval, whenever the run actually happens.

Backfill: what happens when the start is in the past

Set the start time a month back and the trigger does not skip to today. It runs every window since. The documentation gives the formula, the number of backfill runs is current time minus start time divided by window size, and states that windows execute from oldest to newest, honouring the trigger's concurrency.

The script below models that arithmetic for a daily window with a two-hour delay, started on 2 October with a start time of 1 September. It is a Python model of the documented behaviour, not Data Factory itself.

from datetime import datetime, timedelta, timezone
from math import ceil

UTC = timezone.utc
# The trigger's settings
start_time = datetime(2026, 9, 1, tzinfo=UTC)          # startTime: the first window begins here
window = timedelta(hours=24)                           # frequency Hour, interval 24
delay = timedelta(hours=2)                             # wait for late data before running a window
max_concurrency = 4

# The moment the trigger is started
now = datetime(2026, 10, 2, 9, 30, tzinfo=UTC)


def windows_ready(at):
    """Every window whose end, plus the delay, has passed."""
    ready, begin = [], start_time
    while begin + window + delay <= at:
        ready.append((begin, begin + window))
        begin += window
    return ready


ready = windows_ready(now)
fmt = "%Y-%m-%d %H:%M"
print(f"trigger started at {now:{fmt}} UTC with startTime {start_time:{fmt}}")
print(f"windows ready to run immediately (the backfill): {len(ready)}")
print(f"  first: {ready[0][0]:{fmt}} -> {ready[0][1]:{fmt}}")
print(f"  last : {ready[-1][0]:{fmt}} -> {ready[-1][1]:{fmt}}")
print(f"with maxConcurrency {max_concurrency}: at most {max_concurrency} run at once, "
      f"so at least {ceil(len(ready) / max_concurrency)} rounds")

next_begin = ready[-1][1]
print(f"next window {next_begin:{fmt}} -> {next_begin + window:{fmt}} becomes ready at "
      f"{next_begin + window + delay:{fmt}} (window end + delay)")

# The same moment seen by a schedule trigger that fires daily at 02:00
fires = datetime(2026, 10, 2, 2, 0, tzinfo=UTC)
print()
print(f"schedule trigger: next fire after start is {fires + timedelta(days=1):{fmt}}; past days are not run")

trigger started at 2026-10-02 09:30 UTC with startTime 2026-09-01 00:00
windows ready to run immediately (the backfill): 31
  first: 2026-09-01 00:00 -> 2026-09-02 00:00
  last : 2026-10-01 00:00 -> 2026-10-02 00:00
with maxConcurrency 4: at most 4 run at once, so at least 8 rounds
next window 2026-10-02 00:00 -> 2026-10-03 00:00 becomes ready at 2026-10-03 02:00 (window end + delay)

schedule trigger: next fire after start is 2026-10-03 02:00; past days are not run

Thirty-one windows are ready the moment the trigger starts: every day of September, plus 1 October. With a concurrency of 4 they run four at a time, oldest first, in at least eight rounds. The window for 2 October is not ready, because it has not ended; it becomes ready at 02:00 on 3 October, its end plus the delay.

The last line is the contrast. A schedule trigger started at the same moment runs tomorrow and never looks back.

Microsoft adds a practical caution: all past windows run before future ones, so for a long history, do an initial historical load another way instead of backfilling years of small windows.

The settings that matter

Delay is for late data. If yesterday's records keep arriving until 01:30, a two-hour delay runs the window at 02:00 with complete data. The window's own boundaries are unchanged.

Concurrency is for backfills and catch-up. A value of 1 forces strict order. A higher value clears a backlog faster, provided windows are independent and the source can take the load.

Retry is at the pipeline level. It reruns the whole pipeline for a window that failed. It complements, and does not replace, retries on individual activities.

Window size is permanent. Since frequency and interval cannot be edited after publishing, changing from hourly to daily means a new trigger. Choose with care.

Using the window in the pipeline

The window's boundaries are available in the trigger definition as trigger().outputs.windowStartTime and trigger().outputs.windowEndTime, in UTC. The trigger passes them to pipeline parameters, and the pipeline uses the parameters. Written from the documentation, not deployed:

{
  "name": "DailyOrdersWindow",
  "properties": {
    "type": "TumblingWindowTrigger",
    "typeProperties": {
      "frequency": "Hour",
      "interval": 24,
      "startTime": "2026-09-01T00:00:00Z",
      "delay": "02:00:00",
      "maxConcurrency": 4,
      "retryPolicy": { "count": 2, "intervalInSeconds": 300 },
      "dependsOn": [
        {
          "type": "SelfDependencyTumblingWindowTriggerReference",
          "offset": "-1.00:00:00",
          "size": "1.00:00:00"
        }
      ]
    },
    "pipeline": {
      "pipelineReference": { "type": "PipelineReference", "referenceName": "LoadOrdersForWindow" },
      "parameters": {
        "windowStart": { "type": "Expression", "value": "@trigger().outputs.windowStartTime" },
        "windowEnd": { "type": "Expression", "value": "@trigger().outputs.windowEndTime" }
      }
    }
  }
}

Inside the pipeline the source query filters on the two parameters: rows modified at or after windowStart and before windowEnd. Because the boundaries come from the window and not from the clock, a rerun of 14 September next month selects the same rows it would have on 15 September. Pair that with a sink that replaces the window's data, and the load is idempotent.

Use a half-open interval, greater than or equal to the start and strictly less than the end. Since windows are contiguous, each row then falls in exactly one.

Dependencies

Two kinds of dependency make chains of windows reliable (Microsoft Learn: tumbling window dependency).

On another trigger. A daily aggregation should not run until the 24 hourly loads it summarises have succeeded. The aggregation's trigger depends on the hourly trigger, with an offset and a size saying which of the other trigger's windows must be complete. A trigger can depend on up to five others.

On itself. A self-dependency makes each window wait for earlier windows of the same trigger to succeed. The example above uses one, with a negative offset of one day: 15 September will not run until 14 September has succeeded. Microsoft recommends this when a pipeline relies on the output of previous windows, such as a running balance. It also turns a parallel backfill into an ordered one, whatever the concurrency setting.

One operational detail: a run in the "waiting on dependency" state cannot be cancelled until the thing it depends on finishes.

Rerunning a window

Because the trigger keeps state, a failed window can be rerun from the monitoring view, and it runs with the same window boundaries as the original. This is the practical payoff of the model. With a schedule trigger, "rerun last Tuesday" means working out and typing Tuesday's parameters by hand, if the pipeline accepts them at all.

Tumbling window or schedule?

Use a tumbling window when each run owns a slice of data: incremental loads, periodic aggregations, anything that might need a backfill. Use a schedule when the pipeline is not about a time slice, a full reload of a small table, a housekeeping job, or when you need calendar rules. Pipeline dependencies and scheduling compares all the trigger types.

Mistakes to avoid

• Filtering on the current time inside a window-triggered pipeline. That discards everything the trigger gives you.
• A closed interval at both ends, which loads boundary rows twice.
• A start time years in the past on a five-minute window. The backfill is enormous.
• High concurrency with windows that depend on each other. Add a self-dependency.
• A sink that appends. A rerun then duplicates the window.
• Forgetting UTC. Window times are UTC; a business day in another time zone needs an offset in the query or a different start time.

For the watermark alternative when data has no reliable timestamp to window on, see incremental data loading.

Azure Data Factory and incremental pipeline design are covered in the Data Engineering course.

Sources

• Create a trigger that runs a pipeline on a tumbling window (Microsoft Learn) — Microsoft Learn
• Pipeline execution and triggers in Azure Data Factory or Azure Synapse Analytics (Microsoft Learn) — Microsoft Learn
• Create a tumbling window trigger dependency (Microsoft Learn) — Microsoft Learn
• System variables supported by Azure Data Factory and Azure Synapse Analytics (Microsoft Learn) — Microsoft Learn