I have to extract data from a json file who contains spatial information. The content of this file is
{"vertices":[{"lat":46.744628268759314,"lon":6.569952920654968},
{"lat":46.74441692818192,"lon":6.570487107359068},
{"lat":46.74426116111054,"lon":6.570355867853787},
{"lat":46.74447250168793,"lon":6.569821681149689}],
"name":"demo-field",
"cropType":"sugarbeet",
"cropPlantDistance":0.18000000715255737,
"rowDistance":0.5,"numberOfRows":[28,12,12],"seedingDate":"2016-08-17T07:39+00:00"}
I've created a table then copied the content of this file into it
create table field(data json);
COPY field(data) FROM '/home/guest-pc5/field.json';
I now I can query my data
SELECT json_array_elements(data->'vertices') from field;
{"lat":46.744628268759314,"lon":6.569952920654968}
{"lat":46.74441692818192,"lon":6.570487107359068}
{"lat":46.74426116111054,"lon":6.570355867853787}
{"lat":46.74447250168793,"lon":6.569821681149689}
(4 rows)
The problem is that I can't use it like that. I would like to catch only values of "lat" and "lon" to put them in the field table
I've tried to use the function json_to_recordset without success
select * from json_to_recordset('[{"lat":46.744628268759314,"lon":6.569952920654968},{"lat":46.74441692818192,"lon":6.570487107359068},{"lat":46.74426116111054,"lon":6.570355867853787},{"lat":46.74447250168793,"lon":6.569821681149689}]') as (lat numeric, lon numeric);
ERROR: function json_to_recordset(unknown) does not exist
LINE 1: select * from json_to_recordset('[{"lat":46.744628268759314,...
^
HINT: No function matches the given name and argument types. You might need to add explicit type casts.
You can use json manipulator operator ->> to get the value you want as text out of json_array_elements output. To make it easier, you can call json_array_elements in FROM clause (which is a lateral call to a set-returning function):
SELECT
f.data AS original_json,
CAST((e.element->>'lat') AS numeric) AS lat,
CAST((e.element->>'lon') AS numeric) AS lon
FROM
field AS f,
json_array_elements(f.data->'vertices') AS e(element);
With that, you can simple create a table (or use INSERT into an existent one):
CREATE TABLE coordinates AS
SELECT
f.data AS original_json,
CAST((e.element->>'lat') AS numeric) AS lat,
CAST((e.element->>'lon') AS numeric) AS lon
FROM
field AS f,
json_array_elements(f.data->'vertices') AS e(element);
OBS: The LATERAL there is implicit, as the LATERAL keyword is optional for set-returning function calls, but you could make it really explicit, as:
FROM
field f
CROSS JOIN LATERAL json_array_elements(f.data->'vertices') AS e(element);
Also, LATERAL is 9.3+ only, although you are certainly above that as you are using json_array_elements (also 9.3+ only).
Related
I am trying to search for a string in a simple JSON Array e.g.
'["Accounting","Administration & Office Support","Advertising, Arts & Media"]'
SELECT classification
FROM Skills.skills
WHERE JSON_EXTRACT(classification, '$[0]') = "Accounting";
This work and returns all rows with "Accounting" in position 0.
My understanding is if I were to use as wildcard '$[*]' then this would search all positions. However, this does not return any rows.
What JSON function should I use in this case?
You can use JSON_SEARCH to search the array for the value:
SELECT *
FROM skills
WHERE JSON_SEARCH(classification, 'one', 'Accounting') IS NOT NULL
You can also use JSON_CONTAINS, but you need to be careful to add double quotes around the string to make it a valid JSON scalar:
SELECT *
FROM skills
WHERE JSON_CONTAINS(classification, '"Accounting"')
Demo on db-fiddle
I have got below data in some complex format of json, which is neither a pure json not array.
I have tried to pull the specific values by using below queries that did not work
SELECT json_array_extract_scalar(bankdetails, '$.bank') FROM ade.evangelist_oao.oaocustomfundingupdateaoapplicationsvc ;
Query failed (#20200717_121411_01261_cav8k): line 1:8: Unexpected
parameters (row(bank varchar), varchar(6)) for function
json_extract_scalar. Expected: json_extract_scalar(varchar(x),
JsonPath) , json_extract_scalar(json, JsonPath) [DB Errorcode=1]
SELECT json_extract_scalar(accounts, '$.fundingsrcs.method') FROM ade.evangelist_oao.oaocustomfundingupdateaoapplicationsvc ;
Query failed (#20200717_121600_01270_cav8k): line 1:8: Unexpected
parameters (array(row(fundingsrcs array(row(method varchar,amount
bigint)))), varchar(20)) for function json_extract_scalar. Expected:
json_extract_scalar(varchar(x), JsonPath) , json_extract_scalar(json,
JsonPath) [DB Errorcode=1]
How can i get the b840 value of bankdetails field and CUSTOMFUNDING value of accounts field in this case?
The actual type is ROW (not JSON), so you can retrieve the value with
SELECT bankdetails.bank FROM ...
I have the below string in a column in hive table which i am trying to query using apache drill:
{"cdrreasun":"52","cdxscarc":"20150407161405","cdrend":"20150407155201","cdrdnrar.1un":"24321.70","servlnqlp":"54.201.25.50","men":"42403","xa:lnqruup":"3","cemcau":"120","accuuncl":"21","cdrc:
5","volcuca":"1.7"}
Want to retrieve all values for key cdrreasun using apache drill SQL.
Can't use FLATTEN on the column as it says Flatten does not work with inputs of non-list types.
Can't use KVGEN as well as it works only with MAP datatype.
Drill has function convert_fromJSON which allows converting from String to JSON object. For more details about this function and examples of usage please see https://drill.apache.org/docs/data-type-conversion/#convert_to-and-convert_from
For the example you specified, you can run
convert_fromJSON(colWithJsonText)['cdrreasun']
I figured it out, hope it will be helpful for others.
We have to do it in 3 steps if the datatype is of type MAP:
KVGEN() -> FLATTEN() -> convert_from()
If it's of type STRING then KVGEN() function is not needed.
SELECT ratinggrouplist
,t3.cdrlist3.cdrreason AS cdrreason
,t3.cdrlist3.cdrstart AS cdrstart
,t3.cdrlist3.cdrend AS cdrend
,t3.cdrlist3.cdrduration AS cdrduration
FROM (
SELECT ratinggrouplist, convert_from(t2.cdrlist2.`element`, 'JSON') AS cdrlist3
FROM (
SELECT ratinggrouplist ,flatten(t1.cdrlist1.`value`) AS cdrlist2
FROM (
SELECT ratinggrouplist, kvgen(cdrlist) AS cdrlist1
FROM dfs.tmp.SOME_TABLE
) AS t1
) AS t2
) AS t3;
In a PostgreSQL 9.5 table I have an integer column social.
When I try to update it in a stored procedure given the following JSON data (an array with 2 objects, each having a "social" key) in the in_users variable of type jsonb:
'[{"sid":"12345284239407942","auth":"ddddc1808197a1161bc22dc307accccc",**"social":3**,"given":"Alexander1","family":"Farber","photo":"https:\/\/graph.facebook.com\/1015428423940942\/picture?type=large","place":"Bochum,
Germany","female":0,"stamp":1450102770},
{"sid":"54321284239407942","auth":"ddddc1808197a1161bc22dc307abbbbb",**"social":4**,"given":"Alxander2","family":"Farber","photo":null,"place":"Bochum,
Germany","female":0,"stamp":1450102800}]'::jsonb
Then the following code is failing:
FOR t IN SELECT * FROM JSONB_ARRAY_ELEMENTS(in_users)
LOOP
UPDATE words_social SET
social = t->'social',
WHERE sid = t->>'sid';
END LOOP;
with the error message:
ERROR: column "social" is of type integer but expression is of type jsonb
LINE 3: social = t->'social',
^
HINT: You will need to rewrite or cast the expression.
I have tried changing that line to:
social = t->'social'::int,
but then I get the error:
ERROR: invalid input syntax for integer: "social"
LINE 3: social = t->'social'::int,
^
Why doesn't PostgreSQL recognize that the data is integer?
From the JSON-TYPE-MAPPING-TABLE I was having the impression that JSON number would be auto-converted to PostgreSQL numeric type.
A single set-based SQL command is far more efficient than looping:
UPDATE words_social w
SET social = (iu->>'social')::int
FROM JSONB_ARRAY_ELEMENTS(in_users) iu -- in_user = function variable
WHERE w.sid = iu->>'sid'; -- type of sid?
To answer your original question:
Why doesn't PostgreSQL recognize that the data is integer?
Because you were trying to convert the jsonb value to integer. In your solution you already found that you need the ->> operator instead of -> to extract text, which can be cast to integer.
Your second attempt added a second error:
t->'social'::int
In addition to the above: operator precedence. The cast operator :: binds stronger than the json operator ->. Like you found yourself already, you really want:
(t->>'social')::int
Very similar case on dba.SE:
Querying JSONB in PostgreSQL
I've ended up using:
FOR t IN SELECT * FROM JSONB_ARRAY_ELEMENTS(in_users)
LOOP
UPDATE words_social SET
social = (t->>'social')::int
WHERE sid = t->>'sid';
IF NOT FOUND THEN
INSERT INTO words_social (social)
VALUES ((t->>'social')::int);
END IF;
END LOOP;
On PostgreSQL 9.3.4, I have a JSON type column called "person" and the data stored in it is in the format {dogs: [{breed: <>, name: <>}, {breed: <>, name: <>}]}. I want to retrieve the breed of dog at index 0. Here are the two queries I ran:
Doesn't work
db=> select person->'dogs'->>0->'breed' from people where id = 77;
ERROR: operator does not exist: text -> unknown
LINE 1: select person->'dogs'->>0->'bree...
^
HINT: No operator matches the given name and argument type(s). You might need to add explicit type casts.
Works
select (person->'dogs'->>0)::json->'breed' from es_config_app_solutiondraft where id = 77;
?column?
-----------
"westie"
(1 row)
Why is the type casting necessary? Isn't it inefficient? Am I doing something wrong or is this necessary for postgres JSON support?
This is because operator ->> gets JSON array element as text. You need a cast to convert its result back to JSON.
You can eliminate this redundant cast by using operator ->:
select person->'dogs'->0->'breed' from people where id = 77;