I have got below data in some complex format of json, which is neither a pure json not array.
I have tried to pull the specific values by using below queries that did not work
SELECT json_array_extract_scalar(bankdetails, '$.bank') FROM ade.evangelist_oao.oaocustomfundingupdateaoapplicationsvc ;
Query failed (#20200717_121411_01261_cav8k): line 1:8: Unexpected
parameters (row(bank varchar), varchar(6)) for function
json_extract_scalar. Expected: json_extract_scalar(varchar(x),
JsonPath) , json_extract_scalar(json, JsonPath) [DB Errorcode=1]
SELECT json_extract_scalar(accounts, '$.fundingsrcs.method') FROM ade.evangelist_oao.oaocustomfundingupdateaoapplicationsvc ;
Query failed (#20200717_121600_01270_cav8k): line 1:8: Unexpected
parameters (array(row(fundingsrcs array(row(method varchar,amount
bigint)))), varchar(20)) for function json_extract_scalar. Expected:
json_extract_scalar(varchar(x), JsonPath) , json_extract_scalar(json,
JsonPath) [DB Errorcode=1]
How can i get the b840 value of bankdetails field and CUSTOMFUNDING value of accounts field in this case?
The actual type is ROW (not JSON), so you can retrieve the value with
SELECT bankdetails.bank FROM ...
Related
I have a table in Bigquery which has 2 columns - job_id and json_column(string which is in JSON format). When I tried to read the data and identify some objects it gives me error as below:
SyntaxError:Unexpected end of JSON input at undefined line XXXX, columns xx-xx
It Always gives me line 5931 and second time I execute again it gives line 6215.
If it's related to JSON structure issue, how can I know which row/job_id that number 5931 corresponds to? If I subset for a specific job_id, it returns the values but when I tried to execute on the complete table, I got this error. I tried looking at the job_id at the row_numbers mentioned and code works fine for those job_ids.
Do you think its JSON structure issue and how to identify which row/job_id has this Issue?
Table Structure:
Code:
CREATE TEMPORARY FUNCTION CUSTOM_JSON_EXTRACT(json STRING, json_path STRING)
RETURNS ARRAY<STRING>
LANGUAGE js AS """
var result = jsonPath(JSON.parse(json), json_path);
if(result){return result;}
else {return [];}
"""
OPTIONS (
library="gs://json_temp/jsonpath-0.8.0.js"
);
SELECT job_id,dist,gm,sub_gm
FROM lz_fdp_op.fdp_json_file,
UNNEST(CUSTOM_JSON_EXTRACT(trim(conv_column), '$.Project.OpsLocationInfo.iDistrictId')) dist ,
UNNEST(CUSTOM_JSON_EXTRACT(trim(conv_column), '$.Project.GeoMarketInfo.Geo')) gm,
UNNEST(CUSTOM_JSON_EXTRACT(trim(conv_column), '$.Project.GeoMarketInfo.SubGeo')) sub_gm
Would this work for you?
WITH
T AS (
SELECT
'1000149.04.14' AS job_id,
'{"Project":{"OpsLocationInfo":{"iDistrictId":"A"},"GeoMarketInfo":{"Geo":"B","SubGeo":"C"}}}' AS conv_column
)
SELECT
JSON_EXTRACT_SCALAR(conv_column, '$.Project.OpsLocationInfo.iDistrictId') AS dist,
JSON_EXTRACT_SCALAR(conv_column, '$.Project.GeoMarketInfo.Geo') AS gm,
JSON_EXTRACT_SCALAR(conv_column, '$.Project.GeoMarketInfo.SubGeo') AS sub_gm
FROM
T
BigQuery JSON Functions docs:
https://cloud.google.com/bigquery/docs/reference/standard-sql/json_functions
how can I read multiple arrays in an object in JSON without using
unnest?
Can you explain better with an input sample your comment?
I am currently using h2 version 1.4.196. I have a CSV file as follows,
Status,message,code
1,hello,13
2,world,14
3,ciao,26
Following query works as expected & returns proper Status values.
select "Status" from CSVREAD('myfile.csv',
'Status,message,code', null) ;
Following query produces an exception. It seems like the literal string "Status" is being passed to convert instead of the values returned by the
execution of the query. I am not sure if my understanding of convert is
correct. Please help me with fixing this query to return the desired result. Thanks in advance.
select convert("Status", int) from CSVREAD('myfile.csv', 'Status,message,code', null) ;
Caused by: java.lang.NumberFormatException: For input string: "Status"
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Integer.parseInt(Integer.java:580)
at java.lang.Integer.parseInt(Integer.java:615)
at org.h2.value.Value.convertTo(Value.java:940)
----------------------Detailed Exception------------------------------
select convert("Status", int) from CSVREAD('myfile.csv',
'Status,message,code', null) ;
Data conversion error converting "Status"; SQL statement:
select convert("Status", int) from CSVREAD('myfile.csv',
'Status,message,code', null) [22018-196] 22018/22018 (Help)
org.h2.jdbc.JdbcSQLException: Data conversion error converting "Status"; SQL statement:
select convert("Status", int) from CSVREAD('myfile.csv',
'Status,message,code', null) [22018-196]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:345)
at org.h2.message.DbException.get(DbException.java:168)
at org.h2.value.Value.convertTo(Value.java:996)
at org.h2.expression.Function.getSimpleValue(Function.java:945)
at org.h2.expression.Function.getValueWithArgs(Function.java:1196)
at org.h2.expression.Function.getValue(Function.java:591)
at org.h2.command.dml.Select$LazyResultQueryFlat.fetchNextRow(Select.java:1459)
at org.h2.result.LazyResult.hasNext(LazyResult.java:79)
at org.h2.result.LazyResult.next(LazyResult.java:59)
at org.h2.command.dml.Select.queryFlat(Select.java:519)
at org.h2.command.dml.Select.queryWithoutCache(Select.java:625)
at org.h2.command.dml.Query.queryWithoutCacheLazyCheck(Query.java:114)
at org.h2.command.dml.Query.query(Query.java:371)
at org.h2.command.dml.Query.query(Query.java:333)
at org.h2.command.CommandContainer.query(CommandContainer.java:113)
at org.h2.command.Command.executeQuery(Command.java:201)
at org.h2.jdbc.JdbcStatement.executeInternal(JdbcStatement.java:186)
at org.h2.jdbc.JdbcStatement.execute(JdbcStatement.java:164)
at org.h2.server.web.WebApp.getResult(WebApp.java:1380)
at org.h2.server.web.WebApp.query(WebApp.java:1053)
at org.h2.server.web.WebApp$1.next(WebApp.java:1015)
at org.h2.server.web.WebApp$1.next(WebApp.java:1002)
at org.h2.server.web.WebThread.process(WebThread.java:164)
at org.h2.server.web.WebThread.run(WebThread.java:89)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NumberFormatException: For input string: "Status"
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Integer.parseInt(Integer.java:580)
at java.lang.Integer.parseInt(Integer.java:615)
at org.h2.value.Value.convertTo(Value.java:940)
... 22 more
The way you're using it you're defining the columns and reading all the rows from the csv, so you're getting the first row with Status as well. Then you try to convert the string Status to an integer which obviously results in exeption.
Instead you shouldn't define the columns but let h2 get them from the first row of the csv:
select convert(status,int) from CSVREAD('myfile.csv');
I have to extract data from a json file who contains spatial information. The content of this file is
{"vertices":[{"lat":46.744628268759314,"lon":6.569952920654968},
{"lat":46.74441692818192,"lon":6.570487107359068},
{"lat":46.74426116111054,"lon":6.570355867853787},
{"lat":46.74447250168793,"lon":6.569821681149689}],
"name":"demo-field",
"cropType":"sugarbeet",
"cropPlantDistance":0.18000000715255737,
"rowDistance":0.5,"numberOfRows":[28,12,12],"seedingDate":"2016-08-17T07:39+00:00"}
I've created a table then copied the content of this file into it
create table field(data json);
COPY field(data) FROM '/home/guest-pc5/field.json';
I now I can query my data
SELECT json_array_elements(data->'vertices') from field;
{"lat":46.744628268759314,"lon":6.569952920654968}
{"lat":46.74441692818192,"lon":6.570487107359068}
{"lat":46.74426116111054,"lon":6.570355867853787}
{"lat":46.74447250168793,"lon":6.569821681149689}
(4 rows)
The problem is that I can't use it like that. I would like to catch only values of "lat" and "lon" to put them in the field table
I've tried to use the function json_to_recordset without success
select * from json_to_recordset('[{"lat":46.744628268759314,"lon":6.569952920654968},{"lat":46.74441692818192,"lon":6.570487107359068},{"lat":46.74426116111054,"lon":6.570355867853787},{"lat":46.74447250168793,"lon":6.569821681149689}]') as (lat numeric, lon numeric);
ERROR: function json_to_recordset(unknown) does not exist
LINE 1: select * from json_to_recordset('[{"lat":46.744628268759314,...
^
HINT: No function matches the given name and argument types. You might need to add explicit type casts.
You can use json manipulator operator ->> to get the value you want as text out of json_array_elements output. To make it easier, you can call json_array_elements in FROM clause (which is a lateral call to a set-returning function):
SELECT
f.data AS original_json,
CAST((e.element->>'lat') AS numeric) AS lat,
CAST((e.element->>'lon') AS numeric) AS lon
FROM
field AS f,
json_array_elements(f.data->'vertices') AS e(element);
With that, you can simple create a table (or use INSERT into an existent one):
CREATE TABLE coordinates AS
SELECT
f.data AS original_json,
CAST((e.element->>'lat') AS numeric) AS lat,
CAST((e.element->>'lon') AS numeric) AS lon
FROM
field AS f,
json_array_elements(f.data->'vertices') AS e(element);
OBS: The LATERAL there is implicit, as the LATERAL keyword is optional for set-returning function calls, but you could make it really explicit, as:
FROM
field f
CROSS JOIN LATERAL json_array_elements(f.data->'vertices') AS e(element);
Also, LATERAL is 9.3+ only, although you are certainly above that as you are using json_array_elements (also 9.3+ only).
In a PostgreSQL 9.5 table I have an integer column social.
When I try to update it in a stored procedure given the following JSON data (an array with 2 objects, each having a "social" key) in the in_users variable of type jsonb:
'[{"sid":"12345284239407942","auth":"ddddc1808197a1161bc22dc307accccc",**"social":3**,"given":"Alexander1","family":"Farber","photo":"https:\/\/graph.facebook.com\/1015428423940942\/picture?type=large","place":"Bochum,
Germany","female":0,"stamp":1450102770},
{"sid":"54321284239407942","auth":"ddddc1808197a1161bc22dc307abbbbb",**"social":4**,"given":"Alxander2","family":"Farber","photo":null,"place":"Bochum,
Germany","female":0,"stamp":1450102800}]'::jsonb
Then the following code is failing:
FOR t IN SELECT * FROM JSONB_ARRAY_ELEMENTS(in_users)
LOOP
UPDATE words_social SET
social = t->'social',
WHERE sid = t->>'sid';
END LOOP;
with the error message:
ERROR: column "social" is of type integer but expression is of type jsonb
LINE 3: social = t->'social',
^
HINT: You will need to rewrite or cast the expression.
I have tried changing that line to:
social = t->'social'::int,
but then I get the error:
ERROR: invalid input syntax for integer: "social"
LINE 3: social = t->'social'::int,
^
Why doesn't PostgreSQL recognize that the data is integer?
From the JSON-TYPE-MAPPING-TABLE I was having the impression that JSON number would be auto-converted to PostgreSQL numeric type.
A single set-based SQL command is far more efficient than looping:
UPDATE words_social w
SET social = (iu->>'social')::int
FROM JSONB_ARRAY_ELEMENTS(in_users) iu -- in_user = function variable
WHERE w.sid = iu->>'sid'; -- type of sid?
To answer your original question:
Why doesn't PostgreSQL recognize that the data is integer?
Because you were trying to convert the jsonb value to integer. In your solution you already found that you need the ->> operator instead of -> to extract text, which can be cast to integer.
Your second attempt added a second error:
t->'social'::int
In addition to the above: operator precedence. The cast operator :: binds stronger than the json operator ->. Like you found yourself already, you really want:
(t->>'social')::int
Very similar case on dba.SE:
Querying JSONB in PostgreSQL
I've ended up using:
FOR t IN SELECT * FROM JSONB_ARRAY_ELEMENTS(in_users)
LOOP
UPDATE words_social SET
social = (t->>'social')::int
WHERE sid = t->>'sid';
IF NOT FOUND THEN
INSERT INTO words_social (social)
VALUES ((t->>'social')::int);
END IF;
END LOOP;
I'm getting this error when trying to run a query that inserts results into a table in sql.
im passing the table name as parameter,how to give the hierarchy value to the insert statement.
here is my code:
declare #pathhere hierarchyid
select #pathhere=Path from SectionDetails where Sectionid=#sectionid and SectionName=#sectionname and Batchid=#batchid and Deptid=#deptid and Schoolid=#schoolid
insert stmt:
set #sqlstmt = 'insert into '+#batch+'(StudentID, StudentName,SectionID,SectionName,BatchID,BatchName, DeptID,DeptName, SchoolID,Path)
values('''+#sectionid+''','''+#sectionname+''','''+#sectionid+''','''+#sectionname+''','''+#batchid+''','''+#batchname+''','''+ #deptid+''','''+#deptname+''', '''+#schoolid+''','+ CAST(#pathhere as hierarchyid)+')'
exec(#sqlstmt)
im getting error in this line:
'+ CAST(#pathhere as hierarchyid)+'
as Invalid operator for data type. Operator equals add, type equals hierarchyid.
can anyone pls help me out how to pass the hierarchy value
You're trying to create a string that can be executed as a statement. So you need to get your hierarchyid into nvarchar(max) instead.
try: #pathhere.ToString()