I have a table with the following schema in a SQL Azure DB (2019 compat level):
CREATE TABLE dbo.Properties
(
PropertyId int,
PropertyName nvarchar(100),
PropertyValue nvarchar(1000)
)
I'd like to take the data within this table and turn it into JSON using the value within the PropertyName column as the name of the JSON property, and obviously the PropertyValue value as the JSON property value.
EDIT 12/10/2021:
Importantly, the values within the PropertyName column will not be predictable ahead of time.
For example, consider this data in the table (3 rows):
1, "Color", "Blue"
1, "Name", "John"
1, "Cost", 5
The above would be turned into the following JSON:
{"Color":"Blue", "Name":"John", "Cost":5}
I'm obviously able to do this with a STRING_AGG function like the following:
SELECT '{' + STRING_AGG( '"' + p.PropertyName + '": ''' + p.PropertyValue,''',')
WITHIN GROUP (ORDER BY p.PropertyName) + '}' AS MyJson
FROM dbo.Properties p
GROUP BY p.Id
But I was hoping to use one of the build in JSON functions rather than hack together a big string.
FOR JSON AUTO works from the column names, so one method to get your desired result would be to PIVOT the property names into columns. For example:
SELECT Color, [Name], Cost
FROM dbo.Properties
PIVOT ( MAX( PropertyValue ) For PropertyName In ( [Color], [Name], Cost ) ) pvt
FOR JSON AUTO;
My results:
Of course this is only convenient if your JSON attributes / column names are always known and it's a simple example. For more complex examples, you are probably looking at dynamic pivot, or dynamic SQL and your STRING_AGG example isn't so bad.
(This is an extension to this question, but my reputation is too low to comment or ask more questions on that topic...)
We work on bigquery, hence limited in importing packages or using other languages. And, as per the link above, js is a solution, but not what I'm looking for here. I implemented it in js, and it was too slow for our needs.
Suppose one of our columns is a string that look like this (array of json):
[{"location":[22.99902,66.000],"t":1},{"location":[55.32168,140.556],"t":2},{"location":[85.0002,20.0055],"t":3}]
I want to extract from the column the json for which "t":2
Where:
some columns don't have elements "t":2
Some columns have several elements "t":2
The number of json elements in each string can change
element "t":2 is not always in second position.
I don't know regexp well enough for this. We tried regexp_extract with this pattern: r'(\{.*?\"t\":2.*?\})')), but that doesn't work. It extracts everything that precedes "t":2, including the json for "t":2. We only want the json of element "t":2.
Could you advise a regexp pattern that would work?
EDIT:
I have a preference for a solution that gives me 1 match. Suppose I have this string:
[{"location":[22.99902,66.000],"t":1},{"location":[55.32168,140.556],"t":2},{"location":[55.33,141.785],"t":2}],
I would prefer receiving only 1 answer, the first one.
In that case perhaps regexp is less appropriate, but I'm really not sure?
How about this:
(?<=\{)(?=.*?\"t\"\s*:\s*2).*?(?=\})
As seen here
There is another solution but it is not regexp based (as I had originally asked). So this should not count as the final answer to my own question, nonetheless could be useful.
It is based on a split of the string in array and then chosing the element in the array that satisfies my needs.
Steps:
transform the string into something better for splits (using '|' as seperator):
replace(replace(replace(my_field,'},{','}|{'),'[{','{'),'}]','}')
split it using split(), which yields an array of strings (each one a json element)
find the relevant element ("t":2) - in my case, the first one is good enough, so I limit the query to 1: array( select data from unnest(split(replace(replace(replace(my_field,'},{','}|{'),'[{','{'),'}]','}'),'|')) as data where data like '%"t":2%' limit 1)
Convert that into a useable string with array_to_string() and use json_extract on that string to extract the relevant info from the element that I need (say for example, location coordinate x).
So putting it all together:
round(safe_cast(json_extract(array_to_string(array( select data from unnest(split(replace(replace(replace(my_field,'},{','}|{'),'[{','{'),'}]','}'),'|')) as data where data like '%"t":2%' limit 1),''),'$.location[0]') as float64),3) loc_x
May 1st, 2020 Update
A new function, JSON_EXTRACT_ARRAY, has been just added to the list of JSON
functions. This function allows you to extract the contents of a JSON document as
a string array.
so in below you can replace use of json2array UDF with just in-built function JSON_EXTRACT_ARRAY as in below example
#standardSQL
SELECT id,
(
SELECT x
FROM UNNEST(JSON_EXTRACT_ARRAY(json, '$')) x
WHERE JSON_EXTRACT_SCALAR(x, '$.t') = '2'
) extracted
FROM `project.dataset.table`
==============
Below is for BigQuery Standard SQL
#standardSQL
CREATE TEMP FUNCTION json2array(json STRING)
RETURNS ARRAY<STRING>
LANGUAGE js AS """
return JSON.parse(json).map(x=>JSON.stringify(x));
""";
SELECT id,
(
SELECT x
FROM UNNEST(json2array(JSON_EXTRACT(json, '$'))) x
WHERE JSON_EXTRACT_SCALAR(x, '$.t') = '2'
) extracted
FROM `project.dataset.table`
You can test, play with above using dummy data as in below example
#standardSQL
CREATE TEMP FUNCTION json2array(json STRING)
RETURNS ARRAY<STRING>
LANGUAGE js AS """
return JSON.parse(json).map(x=>JSON.stringify(x));
""";
WITH `project.dataset.table` AS (
SELECT 1 id, '[{"location":[22.99902,66.000],"t":1},{"location":[55.32168,140.556],"t":2},{"location":[85.0002,20.0055],"t":3}]' json UNION ALL
SELECT 2, '[{"location":[22.99902,66.000],"t":11},{"location":[85.0002,20.0055],"t":13}]'
)
SELECT id,
(
SELECT x
FROM UNNEST(json2array(JSON_EXTRACT(json, '$'))) x
WHERE JSON_EXTRACT_SCALAR(x, '$.t') = '2'
) extracted
FROM `project.dataset.table`
with output
Row id extracted
1 1 {"location":[55.32168,140.556],"t":2}
2 2 null
Above assumes that there is no more than one element with "t":2 in json column. In case if there can be more than one - you should add ARRAY as below
SELECT id,
ARRAY(
SELECT x
FROM UNNEST(json2array(JSON_EXTRACT(json, '$'))) x
WHERE JSON_EXTRACT_SCALAR(x, '$.t') = '2'
) extracted
FROM `project.dataset.table`
Even though, you have posted a work around your issue. I believe this answer will be informative. You mentioned that one of the answer selected more than what you needed, I wrote the query below to reproduce your case and achieve aimed output.
WITH
data AS (
SELECT
" [{ \"location\":[22.99902,66.000]\"t\":1},{\"location\":[55.32168,140.556],\"t\":2},{\"location\":[85.0002,20.0055],\"t\":3}] " AS string_j
UNION ALL
SELECT
" [{ \"location\":[22.99902,66.000]\"t\":1},{\"location\":[55.32168,140.556],\"t\":3},{\"location\":[85.0002,20.0055],\"t\":3}] " AS string_j
UNION ALL
SELECT
" [{ \"location\":[22.99902,66.000]\"t\":1},{\"location\":[55.32168,140.556],\"t\":3},{\"location\":[85.0002,20.0055],\"t\":3}] " AS string_j
UNION ALL
SELECT
" [{ \"location\":[22.99902,66.000]\"t\":1},{\"location\":[55.32168,140.556],\"t\":3},{\"location\":[85.0002,20.0055],\"t\":3}] " AS string_j ),
refined_data AS (
SELECT
REGEXP_EXTRACT(string_j, r"\{\"\w*\"\:\[\d*\.\d*\,\d*\.\d*\]\,\"t\"\:2\}") AS desired_field
FROM
data )
SELECT
*
FROM
refined_data
WHERE
desired_field IS NOT NULL
Notice that I have used the dummy described in the temp table, populated inside the WITH method. As below:
Afterwords, in the table refined_data, I used the REGEXP_EXTRACT to extract the desired string from the column. Observe that for the rows which there is not a match expression, the output is null. Thus, the table refined_data is as follows :
As you can see, now it is just needed a simple WHERE filter to obtain the desired output, which was done in the last select.
In addition you can see the information about the regex expression I provided here.
I have some problems converting tabular data to JSON using the FOR JSON PATH syntax:
If i do a standard query:
SELECT b.Name FROM dbo
I get results of the form: 12/5-A-1. I need this converted to JSON data without escaping the backslash character. However, when i convert it to JSON:
SELECT b.Name FROM dbo FOR JSON PATH, WITHOUT ARRAY_WRAPPER
the result is of the form: {"Name": "12\/5-A-1"}
How can i do this transformation without escaping the backslash character and get the result {"Name": "12/5-A-1"}?
One option is to use a common table expression to generate the json, and then simply use replace when selecting from the common table expression.
First, create and populate sample data (Please save us this step in your future questions):
DECLARE #T AS TABLE
(
[Name] nvarchar(10)
)
INSERT INTO #T ([Name]) VALUES ('12/5-A-1');
The cte:
WITH CTE(Escaped) AS
(
SELECT [Name]
FROM #T
FOR JSON PATH, WITHOUT_ARRAY_WRAPPER
)
The final select:
SELECT REPLACE(Escaped, '\/','/') As Result
FROM CTE
Result:
{"Name":"12/5-A-1"}
I have a column in table which is stored in format:
{"field1":"val1","field2":"val4"}
{"field1":"val2","field2":"val5"}
{"field1":"val3","field2":"val6"}
I need to remove all field1 with values(e.g "field1":"val1","field1":"val2","field1":"val3" ) and result should be
{"field2":"val4"}
{"field2":"val5"}
{"field2":"val6"}
I am trying to acheive this via replace but stuck as in '"field1":"val1"' string val1 could be any value like null, some integer.
UPDATE emp SET col = REPLACE(col, '"field1":"val1"', '')
I am stuck due to this dynamic value of val1.
I would prefer to use the JSON_REMOVE function (MySQL) :
UPDATE emp
SET emp.col = JSON_REMOVE(emp.col, '$.field1');
You can also add a WHERE clause :
WHERE emp.col LIKE '%val6%';
References: MySQL JSON_REMOVE and MySQL JSON path
A blog post with examples: MySQL for your JSON
And a note about json path in MySQL:
Propery names in path must be double quoted if the property identifier contains interpunction (spaces, special characters, meta characters) bugs.mysql.com
You can do it like this:
SELECT SUBSTRING(Field, 1, INSTR(Field, '"field1"')) + SUBSTRING(Field, INSTR(Field, '"field2"'), LENGTH(Field)) FROM #Temp
I don't know if this works but this is the idea. (Can't test ATM)
Here is the MsSQL equivalent (works, just tested!):
SELECT SUBSTRING(Field, 0, CHARINDEX('"field1"', Field)) + SUBSTRING(Field, CHARINDEX('"field2"', Field), LEN(Field)) FROM #Temp
I have a MySQL statement that is trying to find if the ID of a row (CHAR(36)) is inside of a string.
My statement is :
select * from `options`
where concat('%',`OptionID`,'%')
like '{\"4d519a3d-c99b-11e4-8eda-b8ca3a83b4c8\":\"62.50\"}';
But this fails and doesn't return anything, while this
select * from `options` where
`OptionID`='4d519a3d-c99b-11e4-8eda-b8ca3a83b4c8';
works and returns my row. I thought this was because the ID field was a CHAR field instead of a VARCHAR field so I tried this instead
select * from `options` where
concat('%',cast(`OptionID` as char(38)) ,'%')
like '{\"4d519a3d-c99b-11e4-8eda-b8ca3a83b4c8\":\"62.50\"}';
But that still didn't work, and now I'm lost. I would like to avoid iterating through the json string and creating a set of conditions like
(`OptionID`={$JsonKey[0]} OR `OptionID`={$JsonKey[1]} OR ... )
Since these json strings can get very large, but if that it is the best option because of performance or what I'm trying to do can't be done any other way, then I'll just do it that way instead.
The concept will work, but you're doing it backwards. You need to reverse the fields.
Something like this:
select *
from `options`
where '{\"4d519a3d-c99b-11e4-8eda-b8ca3a83b4c8\":\"62.50\"}'
like concat('%',`OptionID`,'%')
Condensed Fiddle Example - this demonstrates both approaches