This post will discuss triggering dag using rest API in Airflow version 2. Airflow released Airflow API (Stable) (1.0.0) to access its object or trigger dag utilizing a range of REST API endpoints. In this post, we will directly discuss how to trigger dag using rest API we are not explaining how to set up and run airflow. If you want to set up airflow first then refer here.
Step 1: Enable the Rest API in Airflow
You need to edit the airflow config file as by default airflow does not accept any request through REST API. Browse airflow.cfg file and set auth_backends as
You can also restart the webserver if this is not working and You can also check the REST API, You simply need to pass the Authorization in the form of username:password where the username of the airflow user and password is the password of that user.
I have set these parameters in the Postman API app like this every different platform or framework provides a different method to set Authorization.
Step 2: Enable CORS
Write these lines below the [API] section in airflow.cfg and also replace https://exampleclientapp1.com and https://exampleclientapp2.com with your website name or localhost. You can also refer here to know more about CORS.
Step 3: Test the API by Listing all your dags available
We can list all our dags by hitting GET request on /api/v1/dags endpoint. If you are running airflow on your localhost you can get all your dags by hitting GET request to http://localhost:8080/api/v1/dags also don't forget to set the Authorization.
Step 4: Setup your dag file
Here is my sample dag file code put this code in a file that is in the dags folder if there is no file then create one name sample_dag.py.
As you already know this code now we quickly see how to trigger this dag using REST API.
Note: Dag run id can be anything you can pass any text or string in dag_run_id.
Step 5: Access conf in your code or function
We can also access the variables which we have set in the conf passed in our request. You just need to write provide_context=True in your PythonOperator function and access it using kwargs['dag_run'].conf.get("here is your variable name").
Updated code:
I have shown you how you can trigger your dag from the rest API and I hope this will help you to write your code and setup up rest API.