instance variables - How to obtain GCP project name within SQL run against BigQuery from Composer Airflow

Question

I would like to write a query to be executed by Composer/Airflow (python BigQueryOperator referring to an SQL file) along the lines of

SELECT col1, col2, ... FROM `{{ GCP_PROJECT }}.dataset.table` ...

I wish for the GCP project to be parametrised in the SQL so I can deploy the same SQL file in the productiond/development (prod/dev) environments and test in dev without attempting to query prod tables the dev environment does not have access to.

Is this something that would already be set up?

On this point I couldn't find any helpful examples in the Composer guide, apart from that GCP_PROJECT is already reserved, not sure how to pass an environment variable on to the templating anyway.. Thanks.

question from:https://stackoverflow.com/questions/66047349/how-to-obtain-gcp-project-name-within-sql-run-against-bigquery-from-composer-air

Answer 1

Waitting for answers