Recent questions tagged Airflow

0 votes

1.1k views

1 answer

airflow - Trigger Cloud Composer DAG with a Pub/Sub message

I am trying to create a Cloud Composer DAG to be triggered via a Pub/Sub message. There is the following ... giving me some directions? Thanks! See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

1.3k views

1 answer

airflow - How to pull xcom value from other task instance in the same DAG run (not the most recent one)?

I have 3 DAG runs: DAGR 1 executed at 2019-02-13 16:00:00 DAGR 2 executed at 2019-02-13 17:00:00 DAGR 3 ... want to pull from the same DAG run? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

2.4k views

1 answer

airflow - Dag Seems to be missing

I have a dag which checks for new workflows to be generated (Dynamic DAG) at a regular interval and if ... DAG, "testDynDags" is displayed. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

467 views

1 answer

airflow - Dynamic dags not getting added by scheduler

I am trying to create Dynamic DAGs and then get them to the scheduler. I tried the reference from https:// ... and added to the scheduler. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

506 views

1 answer

airflow - Fusing operators together

I'm still in the process of deploying Airflow and I've already felt the need to merge operators together. The ... and had abandoned the idea. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

1.2k views

1 answer

airflow - AirflowException: Celery command failed - The recorded hostname does not match this instance's hostname

I'm running Airflow on a clustered environment running on two AWS EC2-Instances. One for master and one for ... are experiencing it as well. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

682 views

1 answer

airflow - Debugging Broken DAGs

When the airflow webserver shows up errors like Broken DAG: [<path/to/dag>] <error>, how and where can we find ... am using airflow v1.7.1.3 See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

595 views

1 answer

airflow - How to define a DAG that scheduler a monthly job together with a daily job?

I have to update a table Foo monthly and another table Bar daily and join these two tables daily and insert the ... they are in the same DAG? See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

543 views

1 answer

airflow - Getting the date of the most recent successful DAG execution

I am looking to create a transform in Airflow, and I want to ensure to get all data from my source since the ... meet this, please let me know See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

Categories

Just Browsing Browsing

Most popular tags