Thanks, Bill. Is there some documentation on how to use the IDC API? The example response you posted is the type of data I’m looking for. I’m hoping to simply post a request using the Python “requests” package to avoid using the google packages, as I’ve had some difficulty getting the google packages to work; “google.colab” in particular I couldn’t install without errors.
However, I was able to access some of the collections metadata in the BigQuery tables using the bigquery client via the following code:
from google.cloud import bigquery
from google.oauth2 import service_account
cred_file = 'credentials.json'
cred = service_account.Credentials.from_service_account_file(cred_file)
pid = 'cgmeyer-001'
client = bigquery.Client(credentials=cred, project=pid)
collections_query = client.query("""
LIMIT 1000 """)
results = collections_query.result()
df = results.to_dataframe()
When I run this query, however, I only get 25 collections, whereas the table on the web at this URL mentioned in the documentation lists 142 collections:
Are only a subset of the collections available via the API? Or have I malformed my request?
Finally, just as some feedback: to achieve the above query, I needed to create a service account in the google console, grant myself certain permissions, and generate keys to use with the bigquery client, which, at least for me, took some time and came with a very high learning curve for simply accessing a small table of public data. I would have expected to be able to grab that data with a simple, unauthenticated request or cURL, etc.
Thanks for all your help!