BigQuery client python get column based partitioning column name

Question

I am looking to include a where clause to an existing query to exclude data in the BigQuery streaming buffer.

To do this I would like to get the Partition column name so I can add

WHERE partition_column IS NOT NULL;

to my existing query.

I have been looking at the CLI and the get_table method however that just returns the value of the column not the column name. I get the same when searching on .INFORMATION_SCHEMA.PARTITIONS this returns a field for partition_id but I would prefer the column name itself, is there away to get this ?

Additionally the table is setup with column based partitioning.

Why don't you use _PARTITIONTIME pseudo column to apply your filter? — guillaume blaquiere
– guillaume blaquiere, Commented Nov 10, 2021 at 20:18
@guillaumeblaquiere The _PARTITIONTIME column doesn't exist when using column based partitioning, you get the pseudo columns when using Partition by ingestion time at table creation. — Hutch
– Hutch, Commented Nov 11, 2021 at 9:37

user13237611 · Accepted Answer · 2021-11-11 09:56:07Z

2

Based on python BigQuery client documentation, use the attribute time_partitioning:

from google.cloud import bigquery
bq_table = client.get_table('my_partioned_table')
bq_table.time_partitioning # TimePartitioning(field='column_name',type_='DAY')
bq_table.time_partitioning.field # column_name

Small tips, if you don't know where to search, print the API repr:

bq_table.to_api_repr()

answered Nov 11, 2021 at 9:56

user13237611

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

BigQuery client python get column based partitioning column name

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related