Deferred tasks¶
Choose the test by the handoff you need to verify:
| Test | Use | What it proves |
|---|---|---|
| One trigger | run_trigger(...) |
The trigger emits the expected first event |
| One deferrable operator | dag_maker.run_ti(..., run_triggerer=True) |
Airflow persists and reconstructs the trigger, then passes its event to the resume method |
| A whole Dag | dag_maker.run(run_triggerer=True) or run_dag(..., run_triggerer=True) |
Resumed tasks and their downstream dependencies settle together |
Use the persisted path for any contract crossing from defer() to execute_complete().
Trigger-only tests cannot catch broken serialization or a mismatched resume payload.
The trigger alone, no database¶
run_trigger drives one trigger's async run() to its first TriggerEvent on a private event
loop. It needs no triggerer job, DagRun, or metadata database:
from piab.taskinstance import run_trigger
def test_trigger_fires():
event = run_trigger(MyTrigger(value=42), timeout=5.0)
assert event.payload == {"value": 42}
Use this when polling and the event payload are the subject. cleanup() always runs, even when
the trigger raises or times out. Completing without an event raises TriggerExecutionError
naming the trigger class. timeout must be positive and defaults to 10 seconds
(DEFAULT_TRIGGER_TIMEOUT).
The complete defer-and-resume handoff¶
Pass run_triggerer=True to persist the trigger, fire it, submit its first event, and resume the
task through execute_complete():
from airflow.utils.state import TaskInstanceState
def test_deferred_operator_resumes(dag_maker):
with dag_maker(dag_id="deferred_demo"):
MyDeferrableOperator(task_id="wait")
ti = dag_maker.run_ti("wait", run_triggerer=True, trigger_timeout=5.0)
assert ti.state == TaskInstanceState.SUCCESS
assert ti.trigger_id is None
assert ti.xcom_pull(task_ids="wait", session=dag_maker.session) == 42
The runner reconstructs the trigger from the persisted Trigger row's classpath and
kwargs; it does not reuse the object created by execute(). Dropped constructor arguments,
payload mismatches, and exceptions from the trigger or resume method therefore fail the test.
Without run_triggerer=True, a deferring task simply settles deferred. For a whole-Dag
assertion, pass the same keyword to dag_maker.run(...) or
run_dag(...); the runner resumes each deferring task
before settling downstream states.
What is not modeled¶
- One event and one resume. The trigger may poll internally until its first event, but later
events are ignored. If
execute_complete()defers again, the task remainsdeferred; the runner does not fire a second trigger. - A production triggerer. High-availability assignment, multiple triggers sharing an event
loop, task deferral deadlines, and
TriggerFailureReasonhandling remain Airflow's job.trigger_timeoutonly bounds how long this test waits for the first event. - An executor and triggerer together.
run_triggerer=Truecannot be combined withexecutor=. Use an executor-driven run to test the worker boundary, or this page's persisted path to test resumption.
For the broader boundary between testing your component and retesting Airflow itself, see Whose fail is it anyway?.