I have scenario to parse the json data which is of one column in table.
Issue is that below Response column as json generated by Datastore backup to BigQuery. It has '\'attached to every data.
Reponse": "[
{
\"questionId\":5121566669012992,
\"answereId\":0,
\"answeredText\":\"Summer\"
},{
\"questionId\":5166851730440192,
\"answereId\":0,
\"answeredText\":\"Barcelona\"
},{
\"questionId\":6304057064947712,
\"answereId\":0,
\"answeredText\":\"Kitesurf\"
}
]"
How do I parse the below to get value for questionId using BigQuery?
JSON_EXTRACT cannot return REPEATED field, it can only do one match - hence no support for *
you can get the first position using hardcoded indexes as
SELECT JSON_EXTRACT_SCALAR('[
{
\"questionId\":5121566669012992,
\"answereId\":0,
\"answeredText\":\"Summer\"
},{
\"questionId\":5166851730440192,
\"answereId\":0,
\"answeredText\":\"Barcelona\"
},{
\"questionId\":6304057064947712,
\"answereId\":0,
\"answeredText\":\"Kitesurf\"
}
]', '$[0].questionId') AS str;
This returns:
+-----+------------------+---+
| Row | str | |
+-----+------------------+---+
| 1 | 5121566669012992 | |
+-----+------------------+---+
Related
I am attempting to extract from my json object
hits = [{“title”: “Facebook”,
“domain”: “facebook.com”},
{“title”: “Linkedin”,
“domain”: “linkedin.com”}]
When I use:
json_extract(hits,'$.title') as title,
nothing is returned. I would like the result to be: [Facebook, Linkedin].
However, when I extract by a scalar value, ex.:
json_extract_scalar(hits,'$[0].title') as title,
it works and Facebook is returned.
hits contains a lot of values, so I need to use json_extract in order to get all of them, so I can't do each scalar individually. Any suggestions to fix this would be greatly appreciated.
I get INVALID_FUNCTION_ARGUMENT: Invalid JSON path: '$.title' as an error for $.title (double stars). When I try unnest I get INVALID_FUNCTION_ARGUMENT: Cannot unnest type: varchar as an error and INVALID_FUNCTION_ARGUMENT: Cannot unnest type: json. I get SYNTAX_ERROR: line 26:19: Column '$.title' cannot be resolved when I try double quotes
Correct json path to exract all titles is $.[*].title (or $.*.title), though it is not supported by athena. One option is to cast your json to array of json and use transform on it:
WITH dataset AS (
SELECT * FROM (VALUES
(JSON '[{"title": "Facebook",
"domain": "facebook.com"},
{"title": "Linkedin",
"domain": "linkedin.com"}]')
) AS t (json_string))
SELECT transform(cast(json_string as ARRAY(JSON)), js -> json_extract_scalar(js, '$.title'))
FROM dataset
Output:
_col0
[Facebook, Linkedin]
Fits you have an array. So $.title doesn't exist see below
Second, you have not a valid json, is must have double quotes " like the example shows
SET #a := '[{
"title": "Facebook",
"domain": "facebook.com"
},
{
"title": "Linkedin",
"domain": "linkedin.com"
}
]'
SELECT json_extract(#a,'$[0]') as title
| title |
| :---------------------------------------------- |
| {"title": "Facebook", "domain": "facebook.com"} |
SELECT JSON_EXTRACT(#a, "$[0].title") AS 'from'
| from |
| :--------- |
| "Facebook" |
SELECT #a
| #a |
| :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [{<br> "title": "Facebook",<br> "domain": "facebook.com"<br> },<br> {<br><br> "title": "Linkedin",<br> "domain": "linkedin.com"<br> }<br>] |
db<>fiddle here
I am working with a nested json file. The issue is that the keys of the nested json are dates and their value is not known beforehand. Therefore I am unable to apply expandRecordColumn method on it.
Each row has a unique refId and looks like this
{
"refId" : "XYZ",
"snapshotIndexes" : {
"19-07-2021" : {
url: "abc1",
value: "123"
},
"20-07-2021" : {
url: "abc2",
value: "567"
}
}
}
I finally want a table with these columns,
refid | date | url | value
XYZ | 19-7-2021 | abc1 | 123
XYZ | 20-7-2021 | abc2 | 567
PQR | 7-5-2021 | srt | 999
In the new table, refId and date will together make a unique entry.
This is powerBi snapshot
Records
I was able to solve it using Record.ToTable on each row to convert from record to table and then applying ExpandTableColumn
let
Source = DocumentDB.Contents("sourceurl"),
Source = Source{[id="dbid"]}[Collections],
SourceTable= Source{[db_id="dbid",id="PartnerOfferSnapshots"]}[Documents],
ExpandedDocument = Table.ExpandRecordColumn(SourceTable, "Document", {"refId", "snapshotIndexes"}, {"Document.refId", "Document.snapshotIndexes"}),
TransformColumns = Table.TransformColumns(ExpandedDocument,{"Document.snapshotIndexes", each Table.ExpandRecordColumn(Record.ToTable(_), "Value", {"url","id","images"}, {"url","id","images"})}),
ExpandedTable = Table.ExpandTableColumn(TransformColumns, "Document.snapshotIndexes", {"Name","url","id","images"}, {"Document.dates","Document.url","Document.id","Document.images"})
in
ExpandedTable
I have below API response sample
{
"items": [
{
"id":11,
"name": "SMITH",
"prefix": "SAM",
"code": "SSO"
},
{
"id":10,
"name": "James",
"prefix": "JAM",
"code": "BBC"
}
]
}
As per above response, my tests says that whenever I hit the API request the 11th ID would be of SMITH and 10th id would be JAMES
So what I thought to store this in a table and assert against the actual response
* table person
| id | name |
| 11 | SMITH |
| 10 | James |
| 9 | RIO |
Now how would I match one by one ? like first it parse the first ID and first name from the API response and match with the Tables first ID and tables first name
Please share any convenient way of doing it from KARATE
There are a few possible ways, here is one:
* def lookup = { 11: 'SMITH', 10: 'James' }
* def items =
"""
[
{
"id":11,
"name":"SMITH",
"prefix":"SAM",
"code":"SSO"
},
{
"id":10,
"name":"James",
"prefix":"JAM",
"code":"BBC"
}
]
"""
* match each items contains { name: "#(lookup[_$.id+''])" }
And you already know how to use table instead of JSON.
Please read the docs and other stack-overflow answers to get more ideas.
I got this data from my SQL query:
addon_id | addon_name | addon_category_id
---------+------------+------------------
1 | abc | 10
2 | def | 20
3 | ghi | 10
Now I have to send this in the following JSON format and group the addons based on addon_category_id:
[
{
addon_category_id: 10,
addons:
[
{
addon_id: 1,
addon_name: abc
},
{
addon_id: 3,
addon_name: ghi
}
]
},
{
addon_category_id: 20
addons:
[
{
addon_id: 2,
addon_name: def
}
]
}
]
How can I do this? What is the logic behind that? Do I have to do it programmatically using a for loop or is there any other way?
As mentioned in the comments it depends on the programming language you use. In SQL Server 2016 you can use FOR JSON AUTO
SELECT b.addon_category_id ,addons.addon_id , addons.addon_name
FROM addon a
JOIN addon addons
ON a.addon_category_id = b.addon_category_id
FOR JSON AUTO;
I created a SQL Server Table with 25 columns. One of my columns is actually JSON text, stored as nvarchar(max).
Now I need to able to query this JSON column and parse out the various attributes. I have tried applying JSON_VALUE to my column but am doing something wrong; my query runs but returns NULL for all the values.
The JSON itself looks like:
[
{
"lineName":"GHjr",
"pipeDiameter":"12",
"pipeLength":"52000",
"pressure":"15",
"volume":"107"
},
{
"lineName":"Ks3R",
"pipeDiameter":"9",
"pipeLength":"40000",
"pressure":"15",
"volume":"80"
}
]
The SQL I am using is:
select
DOC_ID, LINE_SPECS,
JSON_VALUE(LINE_SPECS, '$.lineName') as line_name,
JSON_VALUE(LINE_SPECS, '$.pipe_Diameter') as diameter
from dbo.MY_TEST_DOCS
where ISJSON(LINE_SPECS) > 0
and len(LINE_SPECS) > 3
However, my 2 "parsed" columns are returning all NULL. How do I parse the five attributes from this column?
Without the [] ISJSON is returning false
With [] ISJSON retuns true
Without the [] JSON_VALUE returns NULLs
With [] JSON_VALUE returns values
dbfddle.uk has sql server 2016 available....
create table test (LINE_SPECS nvarchar(max));
insert into test values (N'
{
"lineName":"GHjr",
"pipeDiameter":"12",
"pipeLength":"52000",
"pressure":"15",
"volume":"107"
},
{
"lineName":"Ks3R",
"pipeDiameter":"9",
"pipeLength":"40000",
"pressure":"15",
"volume":"80"
}
');
select *
from test
where ISJSON(LINE_SPECS) > 0
;
GO
| LINE_SPECS |
| :--------- |
select
JSON_VALUE(LINE_SPECS, '$.lineName') as line_name
, JSON_VALUE(LINE_SPECS, '$.pipeDiameter') as diameter
from test
;
GO
line_name | diameter
:-------- | :-------
GHjr | 12
dbfiddle here