I have a table and I have to generate a specific output in the form of list of array. I have tried with json_agg, array_agg, row_to_json, combination of almost all agg_functions and json_building functions, but not able to generate the output as needed.
If it is not possible, I can work with simple json structure, also, but before giving up, want to give it a try.
Table structure
create table sample_table
(
x_labl character varying,
x_val1 character varying,
x_val2 character varying,
y_labl character varying,
y_val1 character varying,
y_val2 character varying
);
Sample_Data
Want to generate an output like, as below:
"chartData" : [
["lablX", 1, 2], ["lablY", 10, 20]
]
Is this what you want?
select array[
array[x_labl, x_val1, x_val2],
array[y_labl, y_val1, y_val2 ]
] as chartData
from sample_table
This generates a resultset with just one column called chartData and as many rows as there are in the table. Each value is a multi-level array that follows the spec you provided.
If you want a json object instead:
select json_build_object(
'chartData',
jsonb_build_array(
jsonb_build_array(x_labl, x_val1, x_val2),
jsonb_build_array(y_labl, y_val1, y_val2)
)
) as js
from sample_table
Related
I live in Denmark. Here the Thousand separator is a dot (.), and we use comma (,) as comma-separator.
I know that you can use TRY_PARSE to convert a varchar into a money/float value.
An example:
declare
#JSON varchar(max)=
'
{
"Data Table":
[
{
"Value" : "27.123,49"
}
]
}
'
select
TRY_PARSE(Value as money using 'da-dk') "Correct Value"
FROM OpenJson(#json, '$."Data Table"')
WITH
(
"Value" nvarchar(255) N'$."Value"'
)
select
Value "Wrong Value"
FROM OpenJson(#json, '$."Data Table"')
WITH
(
"Value" money N'$."Value"'
)
This query gives me two results
My question is: Can I control the culture in the WiTH Clause of OpenJSON, so I get the correct result without having to use TRY_PARSE?
Target: SQL Server 2019
Not directly in OPENJSON(), no. ECMA-404 JSON Data Interchange Syntax specifically defines the decimal point as the U+002E . character - and doesn't provide for cultural allowances - which is why you're having to define culture-specific values as strings in the first place.
The correct way to do it is only using TRY_PARSE or TRY_CONVERT. eg
select try_parse('27.123,49' as money using 'da-DK')
This is driving me nuts, and I don't understand what's wrong with my approach.
I generate a JSON object in SQL like this:
select #output = (
select distinct lngEmpNo, txtFullName
from tblSecret
for json path, root('result'), include_null_values
)
I get a result like this:
{"result":[{"lngEmpNo":696969,"txtFullName":"Clinton, Bill"}]}
ISJSON() confirms that it's valid JSON, and JSON_QUERY(#OUTPUT, '$.result') will return the array [] portion of the JSON object... cool!
BUT, I'm trying to use JSON_QUERY to extract a specific value:
This gets me a NULL value. Why??????? I've tried it with the [0], without the [0], and of course, txtFullName[0]
SELECT JSON_QUERY(#jsonResponse, '$.result[0].txtFullName');
I prefixed with strict, SELECT JSON_QUERY(#jsonResponse, 'strict $.result[0].txtFullName');, and it tells me this:
Msg 13607, Level 16, State 4, Line 29
JSON path is not properly formatted. Unexpected character 't' is found at
position 18.
What am I doing wrong? What is wrong with my structure?
JSON_QUERY will only extract an object or an array. You are trying to extract a single value so, you need to use JSON_VALUE. For example:
SELECT JSON_VALUE(#jsonResponse, '$.result[0].txtFullName');
We store a fair amount of value/pairs in a json object within a snowflake column.
The quantity of value/pairs is not pre-defined (hence semi structured data)
What are my options to update one of the value pairs?
Am I supposed to extract the entire JSON, convert it to string, modify the string and update the object column entirely?
Or is there a nice little function where I could update just the pair(s) I want?
create or replace table TB as
select $1 TB_ID, parse_json($2) my_json
from values
(1, '{ "FruitShape":"Round", "FruitSize":55 } '),
(2, '{ "FruitShape":"Square" } '),
(3, '{ "FruitShape":"Oblong", "FruitSize":22, "FruitColor":"Chartreuse" })
;
This created 3 rows with up to 3 value/pairs per row.
Let's say I want to change the property on the first row from "round" to "square"
UPDATE TB
SET my_json = parse_json('{ "FruitShape":"square", "FruitSize":55 }')
WHERE TB_ID = 1;
Is this what I am supposed to do?
You should not try to update, just append a new row and then use a view to access the current values.
You can do it with a javascript UDF. For example:
create or replace function merge_objects("a" object, "b" object)
returns object
language javascript
as
$$
return {...a, ...b}
$$;
And call like this:
select merge_objects(
object_construct('a',1,'b',2),
object_construct('b',3,'c',4)
);
select merge_objects(
parse_json('{"a":1,"b":[1,2,3]}')::object,
parse_json('{"c":2,"b":[2,3,4]}')::object
);
Given the table:
C1 C2 C3
----------------
1 'v1' 1.1
2 'v2' 2.2
3 'v3' 3.3
Is there any "easy" way to return JSON in this format:
{
"columns": [ "C1", "C2", "C3" ],
"rows": [
[ 1, "v1", 1.1 ],
[ 2, "v2", 2.2 ],
[ 3, "v3", 3.3 ]
]
}
To generate an array with single values from a table there is a neat trick like this:
SELECT JSON_QUERY(REPLACE(REPLACE(
(
SELECT id
FROM table a
WHERE pk in (1,2)
FOR JSON PATH
), '{"id":',''),'}','')) 'ids'
Which generates
"ids": [1,2]
But to construct the nested array above the replacing gets really tedious, anyone know a good way to achieve this?
Well, you ask for an easy way but the following will not be easy :-)
The tricky part is to know which values need to be qouted and which can remain naked.
This needs generic type-analysis to find, which values are strings.
The only way I know to get on meta data (besides building dynamic sql using meta views like INFORMATIONSCHEMA.COLUMNS) is XML together with an AUTO-schema.
This XML is very near to your needs actually. There is a list of columns at the beginning, followed by a list of rows. But it is not JSON of course...
Try this out:
--This is a mockup table with the values you provided.
DECLARE #mockup TABLE(C1 INT,C2 VARCHAR(100),C3 DECIMAL(4,2));
INSERT INTO #mockup VALUES
(1,'v1',1.1)
,(2,'v2',2.2)
,(3,'v3',3.3);
--Now we create an XML out of this
DECLARE #xml XML =
(
SELECT *
FROM #mockup t
FOR XML RAW,XMLSCHEMA,TYPE
);
--Check the XML's content with SELECT #xml to see how it is looking internally
--Now the real query can start:
SELECT '{"columns":[' +
STUFF(#xml.query('declare namespace xsd="http://www.w3.org/2001/XMLSchema";
for $col in /xsd:schema/xsd:element//xsd:attribute
return
<x>,{concat("""",xs:string($col/#name),"""")}</x>
').value('.','nvarchar(max)'),1,1,'') +
'],"rows":[' +
STUFF(
(
SELECT
',[' + STUFF(b.query(' declare namespace xsd="http://www.w3.org/2001/XMLSchema";
for $attr in ./#*
return
<x>,{if(/xsd:schema/xsd:element//xsd:attribute[#name=local-name($attr)]//xsd:restriction/#base="sqltypes:varchar") then
concat("""",$attr,"""")
else
xs:string($attr)
}
</x>
').value('.','nvarchar(max)'),1,1,'') + ']'
FROM #xml.nodes('/*:row') B(b)
FOR XML PATH(''),TYPE
).value('.','nvarchar(max)'),1,1,'') +
']}';
The result
{"columns":["C1","C2","C3"],"rows":[[3,"v3",3.30],[1,"v1",1.10],[2,"v2",2.20]]}
Some explanation:
The first part will use XQuery to find all columns (xsd:attribute within XML-schema) and create the array of column names.
The second part will againt use XQuery in order to run through all rows and write their column values in a concatenated string. Each value can refer to its type within the schema. Whenever this type is sqltypes:varchar the value will be quoted. All other values remain naked.
This will not solve each and any case generically...
To be honest, this was more for my own curiosity :-) Wanted to find out, how one can solve this.
Quite probably the best answer is: Use another tool. SQL-Server is not the best choice here ;-)
I am trying to load a huge volume json data with nested structure to hive using a Json serde. some of the field names start with $ in nested structure. I am mapping hive filed names Using SerDeproperties, but how ever when i query the table, getting null in the field starting with $, tried with different syntax,but no luck.
Sample JSON:
{
"_id" : "319FFE15FF90",
"SomeThing" :
{
"$SomeField" : 22,
"AnotherField" : 2112,
"YetAnotherField": 1
}
. . . etc . . . .
Using a schema as follows:
create table testSample
(
`_id` string,
something struct
<
$somefield:int,
anotherfield:bigint,
yetanotherfield:int
>
)
row format serde 'org.openx.data.jsonserde.JsonSerDe'
with serdeproperties
(
"mapping.somefield" = "$somefield"
);
This schema builds OK, however, somefield(starting with $) in the above table is always returning null (all the other values exist and are correct).
We've been trying a lot of syntax combinations, but to no avail.
Does anyone know the trick to hap a nested field with a leading $ in its name?
You almost got it right. Try creating the table like this.
The mistake you're making is that when mapping in the serde properties (mapping.somefield ="$somefield") you're saying "when looking for the hive column named 'somefield', look for the json field '$somefield', but in hive you defined the column with the dollar sign, which if not outright illegal it's for sure not the best practice in hive.
create table testSample
(
`_id` string,
something struct
<
somefield:int,
anotherfield:bigint,
yetanotherfield:int
>
)
row format serde 'org.openx.data.jsonserde.JsonSerDe'
with serdeproperties
(
"mapping.somefield" = "$somefield"
);
I tested it with some test data:
{ "_id" : "123", "something": { "$somefield": 12, "anotherfield":13,"yetanotherfield":100}}
hive> select something.somefield from testSample;
OK
12
I am suddenly starting to see this problem as well but for normal column names as well (no special characters such as $)
I am populating an external table (Temp) from another internal table (Table2) and want the output of Temp table in JSON format. I want column names in camel case in the output JSON file and so am also using the Serdepoperties in the Temp table to specify correct names. However, I am seeing that when I do Select * from the Temp table, it gives NULL values for the columns whose names have been used in the mapping.
I am running Hive 0.13. Here are the commands:
Create table command:
CREATE EXTERNAL TABLE Temp (
data STRUCT<
customerId:BIGINT, region:STRING, marketplaceId:INT, asin:ARRAY<STRING>>
)
ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe'
WITH SERDEPROPERTIES (
'mapping.customerid' = 'customerId',
'mapping.marketplaceid' = 'marketplaceId'
)
LOCATION '/output';
INSERT INTO TABLE Temp
SELECT
named_struct ('customerId',customerId, 'region', region, 'marketplaceId', marketplaceId, 'asin', asin)
FROM Table2;
Select * from Temp:
{"customerid":null,"region":"EU","marketplaceid":null,"asin":["B000FC1PZC"]}
{"customerid":null,"region":"EU","marketplaceid":null,"asin":["B000FC1C9G"]}
See how "customerid" and "marketplaceid" are null. Generated JSON file is:
{"data":{"region":"EU","asin":["B000FC1PZC"]}}
{"data":{"region":"EU","asin":["B000FC1C9G"]}}
Now, if I remove the with serdeproperties, the table starts getting all values:
{"customerid":1,"region":"EU","marketplaceid":4,"asin":["B000FC1PZC"]}
{"customerid":2,"region":"EU","marketplaceid":4,"asin":["B000FC1C9G"]}
And then the JSON file so generated is:
{"data":{"region":"EU","marketplaceid":4,"asin":["B000FC1PZC"],"customerid":1}}
{"data":{"region":"EU","marketplaceid":4,"asin":["B000FC1C9G"],"customerid":2}}