I was hoping to take advantage of $.. in json_value() function within TSQL queries using stored json documents. Unfortunately it doesn't work:
JSON path is not properly formatted. Unexpected character '.' is found at position 2.
and according to documentation, there was even no intention to:
Path steps can contain the following elements and operators.
Key names. For example, $.name and $."first name". If the key name starts with a dollar sign or contains special characters such as spaces, surround it with quotes.
Array elements. For example, $.product[3]. Arrays are zero-based.
The dot operator (.) indicates a member of an object. For example, in $.people1.surname, surname is a child of people.
Is there any other method, how to find the attribute a at any level(s) in structured json stored in a TSQL table column?
For example, let's have a on arbitrary level in the json document:
select json_value(json, '$..a') from data would return both values (thus 1, 2) for following values in data.json column:
first:
{
"a": 1
}
second:
{
"b": {
"a": 2
}
}
SQL Server has indeed limited support for JSON expressions.
You could mix a recursive common table expression with the openjson() function.
Sample data
create table data
(
json nvarchar(max)
);
insert into data (json) values
('{ "a": 1 }'),
('{ "b": { "a": 2 } }');
Solution
with rcte as
(
select x.[key] as path,
x.[key],
x.[value],
x.[type]
from data d
cross apply openjson(d.json, '$') x
union all
select r.[path] + '.' + rr.[key],
rr.[key],
rr.[value],
rr.[type]
from rcte r
cross apply openjson(r.[value]) rr
where r.[type] = 5 -- 5=object
)
select r.[key],
r.[value],
r.[path]
from rcte r
where r.[key] = 'a';
Result
key value path
--- ----- ----
a 1 a
a 2 b.a
Fiddle (with intermediate recursive common table expression results).
(This is an extension to this question, but my reputation is too low to comment or ask more questions on that topic...)
We work on bigquery, hence limited in importing packages or using other languages. And, as per the link above, js is a solution, but not what I'm looking for here. I implemented it in js, and it was too slow for our needs.
Suppose one of our columns is a string that look like this (array of json):
[{"location":[22.99902,66.000],"t":1},{"location":[55.32168,140.556],"t":2},{"location":[85.0002,20.0055],"t":3}]
I want to extract from the column the json for which "t":2
Where:
some columns don't have elements "t":2
Some columns have several elements "t":2
The number of json elements in each string can change
element "t":2 is not always in second position.
I don't know regexp well enough for this. We tried regexp_extract with this pattern: r'(\{.*?\"t\":2.*?\})')), but that doesn't work. It extracts everything that precedes "t":2, including the json for "t":2. We only want the json of element "t":2.
Could you advise a regexp pattern that would work?
EDIT:
I have a preference for a solution that gives me 1 match. Suppose I have this string:
[{"location":[22.99902,66.000],"t":1},{"location":[55.32168,140.556],"t":2},{"location":[55.33,141.785],"t":2}],
I would prefer receiving only 1 answer, the first one.
In that case perhaps regexp is less appropriate, but I'm really not sure?
How about this:
(?<=\{)(?=.*?\"t\"\s*:\s*2).*?(?=\})
As seen here
There is another solution but it is not regexp based (as I had originally asked). So this should not count as the final answer to my own question, nonetheless could be useful.
It is based on a split of the string in array and then chosing the element in the array that satisfies my needs.
Steps:
transform the string into something better for splits (using '|' as seperator):
replace(replace(replace(my_field,'},{','}|{'),'[{','{'),'}]','}')
split it using split(), which yields an array of strings (each one a json element)
find the relevant element ("t":2) - in my case, the first one is good enough, so I limit the query to 1: array( select data from unnest(split(replace(replace(replace(my_field,'},{','}|{'),'[{','{'),'}]','}'),'|')) as data where data like '%"t":2%' limit 1)
Convert that into a useable string with array_to_string() and use json_extract on that string to extract the relevant info from the element that I need (say for example, location coordinate x).
So putting it all together:
round(safe_cast(json_extract(array_to_string(array( select data from unnest(split(replace(replace(replace(my_field,'},{','}|{'),'[{','{'),'}]','}'),'|')) as data where data like '%"t":2%' limit 1),''),'$.location[0]') as float64),3) loc_x
May 1st, 2020 Update
A new function, JSON_EXTRACT_ARRAY, has been just added to the list of JSON
functions. This function allows you to extract the contents of a JSON document as
a string array.
so in below you can replace use of json2array UDF with just in-built function JSON_EXTRACT_ARRAY as in below example
#standardSQL
SELECT id,
(
SELECT x
FROM UNNEST(JSON_EXTRACT_ARRAY(json, '$')) x
WHERE JSON_EXTRACT_SCALAR(x, '$.t') = '2'
) extracted
FROM `project.dataset.table`
==============
Below is for BigQuery Standard SQL
#standardSQL
CREATE TEMP FUNCTION json2array(json STRING)
RETURNS ARRAY<STRING>
LANGUAGE js AS """
return JSON.parse(json).map(x=>JSON.stringify(x));
""";
SELECT id,
(
SELECT x
FROM UNNEST(json2array(JSON_EXTRACT(json, '$'))) x
WHERE JSON_EXTRACT_SCALAR(x, '$.t') = '2'
) extracted
FROM `project.dataset.table`
You can test, play with above using dummy data as in below example
#standardSQL
CREATE TEMP FUNCTION json2array(json STRING)
RETURNS ARRAY<STRING>
LANGUAGE js AS """
return JSON.parse(json).map(x=>JSON.stringify(x));
""";
WITH `project.dataset.table` AS (
SELECT 1 id, '[{"location":[22.99902,66.000],"t":1},{"location":[55.32168,140.556],"t":2},{"location":[85.0002,20.0055],"t":3}]' json UNION ALL
SELECT 2, '[{"location":[22.99902,66.000],"t":11},{"location":[85.0002,20.0055],"t":13}]'
)
SELECT id,
(
SELECT x
FROM UNNEST(json2array(JSON_EXTRACT(json, '$'))) x
WHERE JSON_EXTRACT_SCALAR(x, '$.t') = '2'
) extracted
FROM `project.dataset.table`
with output
Row id extracted
1 1 {"location":[55.32168,140.556],"t":2}
2 2 null
Above assumes that there is no more than one element with "t":2 in json column. In case if there can be more than one - you should add ARRAY as below
SELECT id,
ARRAY(
SELECT x
FROM UNNEST(json2array(JSON_EXTRACT(json, '$'))) x
WHERE JSON_EXTRACT_SCALAR(x, '$.t') = '2'
) extracted
FROM `project.dataset.table`
Even though, you have posted a work around your issue. I believe this answer will be informative. You mentioned that one of the answer selected more than what you needed, I wrote the query below to reproduce your case and achieve aimed output.
WITH
data AS (
SELECT
" [{ \"location\":[22.99902,66.000]\"t\":1},{\"location\":[55.32168,140.556],\"t\":2},{\"location\":[85.0002,20.0055],\"t\":3}] " AS string_j
UNION ALL
SELECT
" [{ \"location\":[22.99902,66.000]\"t\":1},{\"location\":[55.32168,140.556],\"t\":3},{\"location\":[85.0002,20.0055],\"t\":3}] " AS string_j
UNION ALL
SELECT
" [{ \"location\":[22.99902,66.000]\"t\":1},{\"location\":[55.32168,140.556],\"t\":3},{\"location\":[85.0002,20.0055],\"t\":3}] " AS string_j
UNION ALL
SELECT
" [{ \"location\":[22.99902,66.000]\"t\":1},{\"location\":[55.32168,140.556],\"t\":3},{\"location\":[85.0002,20.0055],\"t\":3}] " AS string_j ),
refined_data AS (
SELECT
REGEXP_EXTRACT(string_j, r"\{\"\w*\"\:\[\d*\.\d*\,\d*\.\d*\]\,\"t\"\:2\}") AS desired_field
FROM
data )
SELECT
*
FROM
refined_data
WHERE
desired_field IS NOT NULL
Notice that I have used the dummy described in the temp table, populated inside the WITH method. As below:
Afterwords, in the table refined_data, I used the REGEXP_EXTRACT to extract the desired string from the column. Observe that for the rows which there is not a match expression, the output is null. Thus, the table refined_data is as follows :
As you can see, now it is just needed a simple WHERE filter to obtain the desired output, which was done in the last select.
In addition you can see the information about the regex expression I provided here.
I have a JSON in my MYSQL like [1,2,3].
And how can i get where 1 is.
I tried to use this:
SELECT JSON_SEARCH('[1,2,3]','one',1);
But the result of this is NULL
I expect the output to be $[0]
Your JSON array elements should be in double quotes:
WITH yourTable AS (
SELECT '["1","2","3"]' AS json
)
SELECT
json,
JSON_SEARCH(json, 'one', '1')
FROM yourTable;
This returns "$[0]" for the match of '1' against the JSON text.
Demo
As to why it doesn't work with number literals (which should in fact be valid literal JSON values), it seems that the third parameter to JSON_SEARCH is a search string, and it only works against actual text, not numbers.
My JSON is like below. This JSON is read by ADF without problem and imports all rows.
{"Row":{"Col1":"Val1", "Col2":"Val2"}}
{"Row":{"Col1":"Val1", "Col2":"Val2"}}
{"Row":{"Col1":"Val1", "Col2":"Val2"}}
However when I read this from SQL using below query, it returns only first record.
SELECT * FROM OPENJSON(#JSONDATA, '$')
with (
Col1 varchar(25) '$.Row.Col1'
);
Do you know why?
I think you have to change your JSON data structure like below. Yours is also not valid JSON. Try any online JSON validators to see the difference.
Correct one would be:
[
{"Col1":"Val1", "Col2":"Val2"},
{"Col1":"Val1", "Col2":"Val2"},
{"Col1":"Val1", "Col2":"Val2"}
]
As yours is simply an object it will probably get just one row. MSDN uses OPENJSON like this too.
After you changed it to an array you have to edit your query like this:
SELECT * FROM OPENJSON(#JSONDATA, '$')
with (
Col1 varchar(25) '$.Col1',
Col2 varchar(25) '$.Col2'
);
It gets all of the rows now as in the screenshot.
If you can't change the input format to use an array, and you know that the objects are delimited by newlines, you can use STRING_SPLIT to get each into its own row:
SELECT JSON_VALUE(value, '$.Row.Col1')
FROM STRING_SPLIT(#JSONDATA, CHAR(10))
Or you can preprocess the input so it is parsable by OPENJSON:
SELECT *
FROM OPENJSON(N'[' + REPLACE(#JSONDATA, CHAR(13) + CHAR(10), ',') + N']')
WITH (
Col1 VARCHAR(25) '$.Row.Col1'
);
If you have no row delimiters, I'm not sure this can be done cleanly in T-SQL.
I am accessing an array (a json object called 'choice_values') in a jsonb field, and would like to parse its contents into a comma-separated text field.
SELECT
jsonb_array_elements(doc -> 'form_values' -> '8189' -> 'choice_values')
FROM
field_data.exports;
That jsonb_array_elements function returns a "setof text", which I would like converted to a comma separated list of the array values, contained within a single field.
Thank you.
Set returning functions (like jsonb_array_elements_text()) can be called in SELECT list, but then they cannot be used in aggregate functions.
This is a good practice to call set returning functions in FROM clause, often in a lateral join like in this example:
with the_data as (
select '["alfa", "beta", "gamma"]'::jsonb as js
)
select string_agg(elem, ',')
from
the_data,
jsonb_array_elements_text(js) elem;
string_agg
-----------------
alfa,beta,gamma
(1 row)
So your query should look like this:
select string_agg(elem, ',')
from
field_data.exports,
jsonb_array_elements_text(doc -> 'form_values' -> '8189' -> 'choice_values') elem;
Using the string_agg aggregate function with a sub-select from jsonb_array_elements_text seems to work (tested on PG 9.5). Note the use of jsonb_array_elements_text, added in PostgreSQL 9.4, rather than jsonb_array_elements, from PostgreSQL 9.3.
with exports as (
select $${"form_values": {"8189": {"choice_values": ["a","b","c"]}}}$$::jsonb as doc
)
SELECT
string_agg(values, ', ')
FROM
exports, jsonb_array_elements_text(doc -> 'form_values' -> '8189' -> 'choice_values') values
GROUP BY
exports.doc;
Output:
'a, b, c'
Also see this question and its answers.
Maybe not best practice: convert the json array to text, then remove the brackets.
WITH input AS (
SELECT '["text1","text2","text3"]'::jsonb as data
)
SELECT substring(data::text,2,length(data::text)-2) FROM input
It has the advantage that it converts "in-place", not by aggregating. This could be handy if you can only access part of the query, e.g. for some synchronization tool where there's field-based conversion rules, or something like the following:
CREATE TEMP TABLE example AS (SELECT '["text1","text2","text3"]'::jsonb as data);
ALTER TABLE example ALTER COLUMN data TYPE text USING substring(data::text,2,length(data::text)-2);