Calling external table from bigquery with python

Question

While I trying to reach the external table's data, I'm getting error like as below. I can not solve this issue. Here are the details about the situation;

google.api_core.exceptions.NotFound: 404 Not found: Files /gdrive/id/id123456id

PS: id123456id is a dummy id.

The file with ID id123456 id exists in my Google Drive. Bigquery table looking this id.

bq_test.json -> service account credential's JSON file. This service account has those roles;

BigQuery Data Editor
BigQuery Data Owner
BigQuery Data Viewer
BigQuery User
Owner

Here is my code block:

from google.cloud import bigquery
from google.oauth2.service_account import Credentials

scopes = (
        'https://www.googleapis.com/auth/bigquery',
        'https://www.googleapis.com/auth/cloud-platform',
        'https://www.googleapis.com/auth/drive'
)
credentials = Credentials.from_service_account_file('bq_test.json')
credentials = credentials.with_scopes(scopes)
client = bigquery.Client(credentials=credentials)

QUERY = (
        """SELECT * FROM
            `project_name.dataset_name.ext_table`
            LIMIT 5"""
        )
query_job = client.query(QUERY)
rows = query_job.result()

for row in rows:
    print(row.name)

@DaImTo How can I give access? I cannot find drive.readonly or something else in the role list. — Emre KABALCI
– Emre KABALCI, Commented Sep 10, 2020 at 11:12

Emre KABALCI · Accepted Answer · 2020-09-10 12:35:33Z

1

I solved the problem as follows;

Go to https://console.cloud.google.com/iam-admin/iam?project=PROJECT_ID
Take service account mail value. ( like bq_test@PROJECT_ID.iam.gserviceaccount.com )
Go to https://drive.google.com and find the related file. (id = id123456)
Right-click and choose Share
Paste the above mail value. ( bq_test@PROJECT_ID.iam.gserviceaccount.com )
Choose read-only or whatever you need.

This flow provides the solution in my case.

answered Sep 10, 2020 at 12:35

Emre KABALCI

314 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Calling external table from bigquery with python

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related