I am trying to insert some data into my Postgres table. There is a column which includes some json Data. But every time I try to insert it shows some error
Below is my query
INSERT INTO settings.tbl_settings
(sin_product_id,
vhr_sys_module_name,
vhr_grouping_name,
vhr_settings_sys_name,
vhr_label,
vhr_value,
arj_select_items,
txt_remarks,
vhr_widget_type,
sin_settings_category,
int_sys_action_id,
fk_created_user_id,
dtm_created)
VALUES
(1,
'XO_PURCHASE',
'XO Purchase',
'NEED_XO_PURCHASE_SUPPLIER_SIDE_POSTING',
'Need XO Purchase Supplier Side Posting',
'NO_POSTING',
'[{"strSysName":"NO_POSTING","strLabel":"No Posting"} ,{"strSysName":"POSTING","strLabel":"Posting"}]'::JSONB[],
'',
'SELECTBOX',
2,
0,
1,
TO_TIMESTAMP('27-08-2022 17.24.34', 'DD/MM/YYYY HH24:MI:SS')
);
It Returns Given Error
ERROR: malformed array literal: "[{"strSysName":"NO_POSTING","strLabel":"No Posting"} ,{"strSysName":"POSTING","strLabel":"Posting"}]"
LINE 27: '[{"strSysName":"NO_POSTING","st...
^
DETAIL: "[" must introduce explicitly-specified array dimensions.
SQL state: 22P02
Character: 1024
Your mistake is that when you perform the insert operation, you convert the string into a jsonb array (::JSONB[]), but you just need to jsonb, since the type of your arj_select_items column is most likely defined as jsonb.
Demo in dbfiddle
I have a table that has the following data
START_DATE
NAME
ID
01/18/2022
JOHN
10
01/19/2022
ADAM
20
I am trying to convert this to JSON in a custom format like below -
{
"labels":{
"name":"JOHN",
"id":[10]
}
"values":{
"startDate":"01/18/2022"
}
}
PARSE_JSON way of
SELECT parse_json('{"values": {startDate: A.STARTDATE}}')
FROM TABLE_A A;
resulted in error
Error parsing JSON: unknown keyword "A", pos 25
OBJECT_CONSTRUCT results in converting column name as key and column value as value.
Please advise how to have custom field names in JSON conversion in Snowflake.
Renamed objects names as per data given and changed Id to array:
create table test1 values(START_DATE date, NAME string,ID number);
insert into test1(START_DATE, NAME,ID ) values('01/18/2022','JOHN', 10);
insert into test1(START_DATE, NAME,ID ) values('01/19/2022','ADAM', 20);
Select
OBJECT_CONSTRUCT('ID',id::array,'NAME',name) as label_obj,
OBJECT_CONSTRUCT(
'start_date',
START_DATE::string) as start_dt_obj,
object_insert(object_construct('labels', label_obj), 'values', start_dt_obj) as final_json
from
Test1;
I have a hive table to load JSON data. There are two values in my JSON. Both have data type as string. If I keep them as bigint, then select on this table gives below error:
Failed with exception java.io.IOException:org.apache.hadoop.hive.serde2.SerDeException: org.codehaus.jackson.JsonParseException: Current token (VALUE_STRING) not numeric, can not use numeric value accessors
at [Source: java.io.ByteArrayInputStream#3b6c740b; line: 1, column: 21]
If I change it two string, then it works OK.
Now, because these columns are in string, I am not able to use from_unixtime method for these columns.
If I try to alter these columns data types from string to bigint, I get below error:
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Unable to alter table. The following columns have types incompatible with the existing columns in their respective positions : uploadtimestamp
Below is my create table statement:
create table ABC
(
uploadTimeStamp bigint
,PDID string
,data array
<
struct
<
Data:struct
<
unit:string
,value:string
,heading:string
,loc:string
,loc1:string
,loc2:string
,loc3:string
,speed:string
,xvalue:string
,yvalue:string
,zvalue:string
>
,Event:string
,PDID:string
,`Timestamp`:string
,Timezone:string
,Version:string
,pii:struct<dummy:string>
>
>
)
row format serde 'org.apache.hive.hcatalog.data.JsonSerDe'
stored as textfile;
My JSON:
{"uploadTimeStamp":"1488793268598","PDID":"123","data":[{"Data":{"unit":"rpm","value":"100"},"EventID":"E1","PDID":"123","Timestamp":1488793268598,"Timezone":330,"Version":"1.0","pii":{}},{"Data":{"heading":"N","loc":"false","loc1":"16.032425","loc2":"80.770587","loc3":"false","speed":"10"},"EventID":"Location","PDID":"skga06031430gedvcl1pdid2367","Timestamp":1488793268598,"Timezone":330,"Version":"1.1","pii":{}},{"Data":{"xvalue":"1.1","yvalue":"1.2","zvalue":"2.2"},"EventID":"AccelerometerInfo","PDID":"skga06031430gedvcl1pdid2367","Timestamp":1488793268598,"Timezone":330,"Version":"1.0","pii":{}},{"EventID":"FuelLevel","Data":{"value":"50","unit":"percentage"},"Version":"1.0","Timestamp":1488793268598,"PDID":"skga06031430gedvcl1pdid2367","Timezone":330},{"Data":{"unit":"kmph","value":"70"},"EventID":"VehicleSpeed","PDID":"skga06031430gedvcl1pdid2367","Timestamp":1488793268598,"Timezone":330,"Version":"1.0","pii":{}}]}
Any ways I can convert this string unixtimestamp to standard time or I can work with bigint for these columns?
If you are talking about Timestamp and Timezone then you can define them as int/big int types.
If you'll look on their definition you'll see that there are no qualifiers (") around the values, therefore they are of numeric types within in the JSON doc:
"Timestamp":1488793268598,"Timezone":330
create external table myjson
(
uploadTimeStamp string
,PDID string
,data array
<
struct
<
Data:struct
<
unit:string
,value:string
,heading:string
,loc3:string
,loc:string
,loc1:string
,loc4:string
,speed:string
,x:string
,y:string
,z:string
>
,EventID:string
,PDID:string
,`Timestamp`:bigint
,Timezone:smallint
,Version:string
,pii:struct<dummy:string>
>
>
)
row format serde 'org.apache.hive.hcatalog.data.JsonSerDe'
stored as textfile
location '/tmp/myjson'
;
+------------------------+-------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| myjson.uploadtimestamp | myjson.pdid | myjson.data |
+------------------------+-------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| 1486631318873 | 123 | [{"data":{"unit":"rpm","value":"0","heading":null,"loc3":null,"loc":null,"loc1":null,"loc4":null,"speed":null,"x":null,"y":null,"z":null},"eventid":"E1","pdid":"123","timestamp":1486631318873,"timezone":330,"version":"1.0","pii":{"dummy":null}},{"data":{"unit":null,"value":null,"heading":"N","loc3":"false","loc":"14.022425","loc1":"78.760587","loc4":"false","speed":"10","x":null,"y":null,"z":null},"eventid":"E2","pdid":"123","timestamp":1486631318873,"timezone":330,"version":"1.1","pii":{"dummy":null}},{"data":{"unit":null,"value":null,"heading":null,"loc3":null,"loc":null,"loc1":null,"loc4":null,"speed":null,"x":"1.1","y":"1.2","z":"2.2"},"eventid":"E3","pdid":"123","timestamp":1486631318873,"timezone":330,"version":"1.0","pii":{"dummy":null}},{"data":{"unit":"percentage","value":"50","heading":null,"loc3":null,"loc":null,"loc1":null,"loc4":null,"speed":null,"x":null,"y":null,"z":null},"eventid":"E4","pdid":"123","timestamp":1486631318873,"timezone":330,"version":"1.0","pii":null},{"data":{"unit":"kmph","value":"70","heading":null,"loc3":null,"loc":null,"loc1":null,"loc4":null,"speed":null,"x":null,"y":null,"z":null},"eventid":"E5","pdid":"123","timestamp":1486631318873,"timezone":330,"version":"1.0","pii":{"dummy":null}}] |
+------------------------+-------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
Even if you have defined Timestamp as a string you can still cast it to a bigint before using it in a function that requires a bigint.
cast (`Timestamp` as bigint)
hive> with t as (select '0' as `timestamp`) select from_unixtime(`timestamp`) from t;
FAILED: SemanticException [Error 10014]: Line 1:45 Wrong arguments
'timestamp': No matching method for class
org.apache.hadoop.hive.ql.udf.UDFFromUnixTime with (string). Possible
choices: FUNC(bigint) FUNC(bigint, string) FUNC(int)
FUNC(int, string)
hive> with t as (select '0' as `timestamp`) select from_unixtime(cast(`timestamp` as bigint)) from t;
OK
1970-01-01 00:00:00