拒绝访问：BigQuery BigQuery：缺少必需的 OAuth 范围。需要 BigQuery 或 Cloud Platform 写入范围

Question

我正在尝试从 Cloud Storage 存储桶中选择一个 JSON 文件并使用 Apache Airflow 将其转储到 BigQuery 中，但是，我收到以下错误：

Access Denied: BigQuery BigQuery: Missing required OAuth scope. Need BigQuery or Cloud Platform write scope。

这是我的代码：

from datetime import timedelta, datetime
import json
from airflow.operators.bash_operator import BashOperator
from airflow.contrib.operators.mysql_to_gcs import MySqlToGoogleCloudStorageOperator
from airflow import DAG
from airflow.contrib.operators.bigquery_operator import BigQueryOperator
from airflow.contrib.operators.bigquery_check_operator import BigQueryCheckOperator
from airflow.providers.google.cloud.transfers.gcs_to_bigquery import GCSToBigQueryOperator

default_args = {
    'owner': 'airflow',
    'depends_on_past': True,
    #'start_date': seven_days_ago,
    'start_date': datetime(2022, 11, 1),
    'email': ['uzair.zafar@gmail.pk'],
    'email_on_failure': True,
    'email_on_retry': False,
    'retries': 5,
    'retry_delay': timedelta(minutes=5),
}


with DAG('checking_airflow',
    default_args=default_args,
    description='dag to start the logging of data in logging table',
    schedule_interval='@daily',
    start_date=datetime(2022, 11, 1),
) as dag:


    dump_csv_to_temp_table = GCSToBigQueryOperator(
        task_id='gcs_to_bq_load',
        google_cloud_storage_conn_id='gcp_connection',
        bucket='airflow-dags',
        #filename='users/users.csv',
        source_objects='users/users0.json',
        #schema_object='schemas/users.json',
        source_format='NEWLINE_DELIMITED_JSON',
        create_disposition='CREATE_IF_NEEDED',
        destination_project_dataset_table='project.supply_chain.temporary_users',
        write_disposition='WRITE_TRUNCATE',
        dag=dag,
    )
    


    
dump_csv_to_temp_table

请帮助我解决这个问题。

Answer 1

你在哪里执行代码。它是您机器上的独立气流实例还是 GCP 上的虚拟机？两者都需要具有适当的权限。