# creating 6 mapped task group instances of the task group group1 (2.5 feature) # will print out a list of results from map index 2 and 3 of the add_42 task # only pull Xcom from specific mapped task group instances (2.5 feature) # reference a task in a task group with task_group_id.task_id # a downstream task to print out resulting pull_xcom ( ** context ) : # creating a task group using the decorator with the dynamic input ( group_id = "group1" ) decorators import dag, task_group, task Task_group_example = task_group_example ( )įrom airflow. Load ( transform_values ( extract_data ( ) ) ) datetime ( 2021, 1, 1, tz = "UTC" ), catchup = False )ĭef task_group_example ( ) ( task_id = "extract", retries = 2 )ĭata_string = ' """ Import ( schedule = None, start_date = pendulum. decorators import dag, task, task_group To use the decorator, add before a Python function which calls the functions of tasks that should go in the task group.
#Airflow documentation code#
Using task group decorators doesn't change the functionality of task groups, but they can make your code formatting more consistent if you're already using them in your DAGs. The task group decorator functions like other Airflow decorators and allows you to define your task group with the TaskFlow API. The task group decorator is available in Airflow 2.1 and later. Use the task group decorator Īnother way of defining task groups in your DAGs is by using the task group decorator. It is important that you use this format when calling specific tasks with XCOM passing or branching operator decisions. This ensures the task_id is unique across the DAG. When your task is within a task group, your callable task_id is the task_id prefixed with the group_id. The task group dependencies are shown in the following animation: The task immediately to the right of the first blue circle ( t1) gets the group's upstream dependencies and the task immediately to the left ( t2) of the last blue circle gets the group's downstream dependencies. When you click and expand group1, blue circles identify the task group dependencies. In the Airflow UI, blue highlighting is used to identify tasks and task groups.
0 kommentar(er)
