I am trying to search for a string in a simple JSON Array e.g.
'["Accounting","Administration & Office Support","Advertising, Arts & Media"]'
SELECT classification
FROM Skills.skills
WHERE JSON_EXTRACT(classification, '$[0]') = "Accounting";
This work and returns all rows with "Accounting" in position 0.
My understanding is if I were to use as wildcard '$[*]' then this would search all positions. However, this does not return any rows.
What JSON function should I use in this case?
You can use JSON_SEARCH to search the array for the value:
SELECT *
FROM skills
WHERE JSON_SEARCH(classification, 'one', 'Accounting') IS NOT NULL
You can also use JSON_CONTAINS, but you need to be careful to add double quotes around the string to make it a valid JSON scalar:
SELECT *
FROM skills
WHERE JSON_CONTAINS(classification, '"Accounting"')
Demo on db-fiddle
Related
I need to clean JSON data that could look like:
{
"reference":"0000010-CAJ",
"product_code":"00000-10",
"var_name":"CAJ-1",
"doc_date":"2020-02-09T21:01:01-05:00",
"due_date":"2020-03-10T21:01:01-05:00",
}
However, this is just one of many other possibilities (is for a log aggregation that gets data from many sources).
I need to replace "-" with "_", but without break the dates like "2020-03-10T21:01:01-05:00", so can't simply cast to string and do a replace. I wonder if exist an equivalent of:
for (k,v) in json:
if is_text(v):
v = replace(...)
You can check with a regex if the value looks like a timestamp:
update the_table
set the_column = (select
jsonb_object_agg(
key,
case
when value ~ '^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}.*' then value
else replace(value, '-', '_')
end)
from jsonb_each_text(the_column) as t(key, value))
This iterates over all key/value pairs in the JSON column (using jsonb_each_text()) and assembles all of them back into a JSON again (using jsonb_object_agg()). Values that look like a timestamp are left unchanged, for all others, the - is replaced with a _.
First week having to deal with a MYSQL database and JSON field types and I cannot seem to figure out why values are encoded automatically and then returned in encoded format.
Given the following SQL
-- create a multiline string with a tab example
SET #str ="Line One
Line 2 Tabbed out
Line 3";
-- encode it
SET #j = JSON_OBJECT("str", #str);
-- extract the value by name
SET #strOut = JSON_EXTRACT(#J, "$.str");
-- show the object and attribute value.
SELECT #j, #strOut;
You end up with what appears to be a full formed JSON object with a single attribute encoded.
#j = {"str": "Line One\n\tLine 2\tTabbed out\n\tLine 3"}
but using JSON_EXTRACT to get the attribute value I get the encoded version including outer quotes.
#strOut = "Line One\n\tLine 2\tTabbed out\n\tLine 3"
I would expect to get my original string with the \n \t all unescaped to the original values and no outer quotes. as such
Line One
Line 2 Tabbed out
Line 3
I can't seem to find any JSON_DECODE or JSON_UNESCAPE or similar functions.
I did find a JSON_ESCAPE() function but that appears to be used to manually build a JSON object structure in a string.
What am I missing to extract the values to the original format?
I like to use handy operator ->> for this.
It was introduced in MySQL 5.7.13, and basically combines JSON_EXTRACT() and JSON_UNQUOTE():
SET #strOut = #J ->> '$.str';
You are looking for the JSON_UNQUOTE function
SET #strOut = JSON_UNQUOTE( JSON_EXTRACT(#J, "$.str") );
The result of JSON_EXTRACT() is intentionally a JSON document, not a string.
A JSON document may be:
An object enclosed in { }
An array enclosed in [ ]
A scalar string value enclosed in " "
A scalar number or boolean value
A null — but this is not an SQL NULL, it's a JSON null. This leads to confusing cases because you can extract a JSON field whose JSON value is null, and yet in an SQL expression, this fails IS NULL tests, and it also fails to be equal to an SQL string 'null'. Because it's a JSON type, not a scalar type.
I have a server log, it continuously records json values without any delimiter, such as:
{"a":1}{"b",2}{"a":2}{"c":{\"qwe\":\"asd\"},"d":"ert"}{"e":12}....
I want to extract each element and put them into rows like:
{"a":1}
{"b",2}
{"a":2}
{"c":{\"qwe\":\"asd\"},"d":"ert"}
{"e":12}..
The log lacks delimiter and comprises nested json, so I cannot use
split function...How to achieve this...
One option would be to split on }{ character and get the elements using posexplode. Positions are only needed to concatenate properly for the first and last elements.
select case when pos = 0 then concat(split_str,'}')
when pos = max(pos) over(partition by str) then concat('{',split_str)
else concat('{',split_str,'}') end as res
from tbl
lateral view posexplode(split(str,'\\}\\{')) t as pos,split_str
Note the result will be a string.
I have the below string in a column in hive table which i am trying to query using apache drill:
{"cdrreasun":"52","cdxscarc":"20150407161405","cdrend":"20150407155201","cdrdnrar.1un":"24321.70","servlnqlp":"54.201.25.50","men":"42403","xa:lnqruup":"3","cemcau":"120","accuuncl":"21","cdrc:
5","volcuca":"1.7"}
Want to retrieve all values for key cdrreasun using apache drill SQL.
Can't use FLATTEN on the column as it says Flatten does not work with inputs of non-list types.
Can't use KVGEN as well as it works only with MAP datatype.
Drill has function convert_fromJSON which allows converting from String to JSON object. For more details about this function and examples of usage please see https://drill.apache.org/docs/data-type-conversion/#convert_to-and-convert_from
For the example you specified, you can run
convert_fromJSON(colWithJsonText)['cdrreasun']
I figured it out, hope it will be helpful for others.
We have to do it in 3 steps if the datatype is of type MAP:
KVGEN() -> FLATTEN() -> convert_from()
If it's of type STRING then KVGEN() function is not needed.
SELECT ratinggrouplist
,t3.cdrlist3.cdrreason AS cdrreason
,t3.cdrlist3.cdrstart AS cdrstart
,t3.cdrlist3.cdrend AS cdrend
,t3.cdrlist3.cdrduration AS cdrduration
FROM (
SELECT ratinggrouplist, convert_from(t2.cdrlist2.`element`, 'JSON') AS cdrlist3
FROM (
SELECT ratinggrouplist ,flatten(t1.cdrlist1.`value`) AS cdrlist2
FROM (
SELECT ratinggrouplist, kvgen(cdrlist) AS cdrlist1
FROM dfs.tmp.SOME_TABLE
) AS t1
) AS t2
) AS t3;
I have to extract data from a json file who contains spatial information. The content of this file is
{"vertices":[{"lat":46.744628268759314,"lon":6.569952920654968},
{"lat":46.74441692818192,"lon":6.570487107359068},
{"lat":46.74426116111054,"lon":6.570355867853787},
{"lat":46.74447250168793,"lon":6.569821681149689}],
"name":"demo-field",
"cropType":"sugarbeet",
"cropPlantDistance":0.18000000715255737,
"rowDistance":0.5,"numberOfRows":[28,12,12],"seedingDate":"2016-08-17T07:39+00:00"}
I've created a table then copied the content of this file into it
create table field(data json);
COPY field(data) FROM '/home/guest-pc5/field.json';
I now I can query my data
SELECT json_array_elements(data->'vertices') from field;
{"lat":46.744628268759314,"lon":6.569952920654968}
{"lat":46.74441692818192,"lon":6.570487107359068}
{"lat":46.74426116111054,"lon":6.570355867853787}
{"lat":46.74447250168793,"lon":6.569821681149689}
(4 rows)
The problem is that I can't use it like that. I would like to catch only values of "lat" and "lon" to put them in the field table
I've tried to use the function json_to_recordset without success
select * from json_to_recordset('[{"lat":46.744628268759314,"lon":6.569952920654968},{"lat":46.74441692818192,"lon":6.570487107359068},{"lat":46.74426116111054,"lon":6.570355867853787},{"lat":46.74447250168793,"lon":6.569821681149689}]') as (lat numeric, lon numeric);
ERROR: function json_to_recordset(unknown) does not exist
LINE 1: select * from json_to_recordset('[{"lat":46.744628268759314,...
^
HINT: No function matches the given name and argument types. You might need to add explicit type casts.
You can use json manipulator operator ->> to get the value you want as text out of json_array_elements output. To make it easier, you can call json_array_elements in FROM clause (which is a lateral call to a set-returning function):
SELECT
f.data AS original_json,
CAST((e.element->>'lat') AS numeric) AS lat,
CAST((e.element->>'lon') AS numeric) AS lon
FROM
field AS f,
json_array_elements(f.data->'vertices') AS e(element);
With that, you can simple create a table (or use INSERT into an existent one):
CREATE TABLE coordinates AS
SELECT
f.data AS original_json,
CAST((e.element->>'lat') AS numeric) AS lat,
CAST((e.element->>'lon') AS numeric) AS lon
FROM
field AS f,
json_array_elements(f.data->'vertices') AS e(element);
OBS: The LATERAL there is implicit, as the LATERAL keyword is optional for set-returning function calls, but you could make it really explicit, as:
FROM
field f
CROSS JOIN LATERAL json_array_elements(f.data->'vertices') AS e(element);
Also, LATERAL is 9.3+ only, although you are certainly above that as you are using json_array_elements (also 9.3+ only).