# Question: IDC API Vs Big Query

**URL:** <https://discourse.canceridc.dev/t/question-idc-api-vs-big-query/363>\
**Category:** Support\
**Tags:** api, bigquery\
**Created:** [December 4, 2022, 4:30am UTC](https://discourse.canceridc.dev/t/question-idc-api-vs-big-query/363 "2022-12-04T04:30:41Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![nss10](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.canceridc.dev/nss10/32/233_2.png) [@nss10](https://discourse.canceridc.dev/u/nss10)\
**Post date:** [December 4, 2022, 4:30am UTC](https://discourse.canceridc.dev/t/question-idc-api-vs-big-query/363/1 "2022-12-04T04:30:41Z")

</div>

Hi,  
I’m a developer building [Biomedical Research Hub Data Commons](https://brh.data-commons.org). I was looking into a way to fetch data related to IDC studies, in such a way that, the information available in BRH will always be up to date with respect to the data available in IDC. I got a few questions related to that.

1. I see this endpoint – [https://api.imaging.datacommons.cancer.gov/v1/collections](https://api.imaging.datacommons.cancer.gov/v1/collections) which has the information I need, but wanted to make sure if the api is still being maintained?

2. I also see a lot of discussions talking about Big Query usage, wanted to make sure what would be the best way to get the info I need. API or BigQuery?

In order to fetch the info I need, I’ve been told to use this query in the BigQuery console

Query

```auto
SELECT *
   FROM canceridc-data.idc.data_collections_metadata
   LIMIT 1000

```

But this query only yields 25 results, despite of the API returning 128 results. Wasn’t sure if I was using the right query.

1. The `subject` value in the BigQuery result set is different from the `subject_count` value in the API, are the fields different? If not, which one of those is stale?

Your help would be sincerely appreciated.

Thank you

---

<div class="post-metadata">

**Author:** ![fedorov](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.canceridc.dev/fedorov/32/3_2.png) [@fedorov](https://discourse.canceridc.dev/u/fedorov)\
**Post date:** [December 4, 2022, 6:10pm UTC](https://discourse.canceridc.dev/t/question-idc-api-vs-big-query/363/2 "2022-12-04T18:10:50Z")

</div>

> [@nss10](#):
>
> I see this endpoint – [https://api.imaging.datacommons.cancer.gov/v1/collections](https://api.imaging.datacommons.cancer.gov/v1/collections) which has the information I need, but wanted to make sure if the api is still being maintained?

The API should be functional. If you see problems, we will need to look into this. @bill.clifford should be able to help with this.

> [@nss10](#):
>
> I also see a lot of discussions talking about Big Query usage, wanted to make sure what would be the best way to get the info I need. API or BigQuery?

BigQuery can be used to access all of the metadata, you can access it using standard SQL. API exposes a very tiny bit of metadata, mostly what is available in the IDC portal. I personally use BigQuery for all the data querying needs.

> [@nss10](#):
>
> In order to fetch the info I need, I’ve been told to use this query in the BigQuery console
> 
> Query
> 
> ```auto
> SELECT *
> FROM canceridc-data.idc.data_collections_metadata
> LIMIT 1000
> 
> ```

Can you please let me know who told you to use that query? That query is using the wrong table, and if it is mentioned anywhere in public documentation or examples we should fix this.

You can take a look at this notebook to get started with IDC BigQuery tables: [https://github.com/ImagingDataCommons/IDC-Examples/blob/master/notebooks/getting\_started/part2\_searching\_basics.ipynb](https://github.com/ImagingDataCommons/IDC-Examples/blob/master/notebooks/getting_started/part2_searching_basics.ipynb). If you let me know what exactly you want to query, I am happy to help you put together the queries and confirm you are searching the right tables.

---

<div class="post-metadata">

**Author:** ![wlongabaugh](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.canceridc.dev/wlongabaugh/32/34_2.png) [@wlongabaugh](https://discourse.canceridc.dev/u/wlongabaugh)\
**Post date:** [December 5, 2022, 9:18pm UTC](https://discourse.canceridc.dev/t/question-idc-api-vs-big-query/363/3 "2022-12-05T21:18:07Z")

</div>

The API at [https://api.imaging.datacommons.cancer.gov/v1/collections](https://api.imaging.datacommons.cancer.gov/v1/collections) is functional and maintained. It is primarily intended to allow programmatic interaction for cohort creation, as opposed to interactive use of the portal. But for the definitive read-only source of what is available in IDC, using the BigQuery tables now hosted in the Google Public Data program is probably the way to go. As Andrey says, the BQ dataset and table you are asking about are out-of-date.