Over time, millions of XCom entries can bloat your database. Ensure you have a maintenance DAG to prune old XCom data. When to use XComs? Passing a dynamic filename generated in Task A to Task B. Passing a specific API token or "Last Modified" timestamp.
That’s where (short for "cross-communications") come in. Think of XComs as a shared message board where tasks can leave small notes for subsequent tasks to read. What are XComs? airflow xcom example
One of the first hurdles beginners face when learning Apache Airflow is data sharing. You write a task that processes a file, and you want to pass the result to the next task. But how? Over time, millions of XCom entries can bloat your database
def extract_order(**context): # Simulate fetching an order ID order_id = 12345 # Push to XCom context['task_instance'].xcom_push(key='order_id', value=order_id) Passing a dynamic filename generated in Task A to Task B