How to get output of a bigquery in a specific json format - json

Question

I have a bigquery table in this format :
DataProvider,Id,Name,Time
ABC,f8453e99-516f-4f15-a3bd-8749089b6934,"xyz",43200
ABC,f8453e99-516f-4f15-a3bd-8749089b6934,"123",43200
ABC,00453e99-516f-4f15-a3bd-8749089b6934,"xyz",43200
I want to generate the output in this format (json) :
{"dataProviderId":"ABC","items":[{"Id":"f8453e99-516f-4f15-a3bd-8749089b6934","data":[{"Name":"xyz","Time":43200},{"Name":"xyz","Time":43200}],
{"Id":"00453e99-516f-4f15-a3bd-8749089b6934","data":[{"Name":"xyz","Time":43200}]}

Answer 1

In your CLI, you can use bq command with --format flag, where you can pass prettyjson format (easy-to-read JSON format).
bq query --format=prettyjson --use_legacy_sql=false 'SELECT * FROM `project_id`:dataset.table' > output.json
By using > at the end of the command, it is possible to save the output of a command to a new file. You will be able to see the output of query in output.json file.
I hope it helps.

Answer 2

Below is for BigQuery Standard SQL
#standardSQL
SELECT TO_JSON_STRING(t) json
FROM (
SELECT dataProvider, ARRAY_AGG(STRUCT(id, data)) items
FROM (
SELECT dataProvider, id, ARRAY_AGG(STRUCT(name, time)) data
FROM `project.dataset.table` t
GROUP BY dataProvider, id
)
GROUP BY dataProvider
) t
If to apply to sample data in your question - output is
Row json
1 {"dataProvider":"ABC","items":[{"id":"f8453e99-516f-4f15-a3bd-8749089b6934","data":[{"name":"xyz","time":43200},{"name":"123","time":43200}]},{"id":"00453e99-516f-4f15-a3bd-8749089b6934","data":[{"name":"xyz","time":43200}]}]}