I want to combine the following JSON from multiple rows into one single JSON object as a row.
{"Salary": ""}
{"what is your name?": ""}
{"what is your lastname": ""}
Expected output
{
"Salary": "",
"what is your name?": "",
"what is your lastname": ""
}
With only built-in functions, you need to expand the rows into key/value pairs and aggregate that back into a single JSON value:
select jsonb_object_agg(t.k, t.v)
from the_table, jsonb_each(ob) as t(k,v);
If your column is of type json rather than jsonb you need to cast it:
select jsonb_object_agg(t.k, t.v)
from the_table, jsonb_each(ob::jsonb) as t(k,v);
A slightly more elegant solution is to define a new aggregate that does that:
CREATE AGGREGATE jsonb_combine(jsonb)
(
SFUNC = jsonb_concat(jsonb, jsonb),
STYPE = jsonb
);
Then you can aggregate the values directly:
select jsonb_combine(ob)
from the_table;
(Again you need to cast your column if it's json rather than jsonb)
Online example
Related
I have data arriving as separate events in JSON form resembling:
{
"id":1234,
"data":{
"packet1":{"name":"packet1", "value":1},
"packet2":{"name":"packet2", "value":2}
}
}
I'd like to unnest the data to essentially have one row per 'packet' (there may be any number of packets).
id
name
value
1234
packet1
1
1234
packet2
2
I've looked at using the unnest function with the various JSON functions but it seems limited to working with arrays. I have not been able to find a way to treat the 'data' field as if it were an array.
At the moment, I cannot change these events to store packets in an array, and ideally the unnesting should be happening within BigQuery.
1. Regular expressions
There might be other ways but you can consider below approach using regular expressions.
WITH sample_table AS (
SELECT """{
"id":1234,
"data":{
"packet1":{"name":"packet1", "value":1},
"packet2":{"name":"packet2", "value":2}
}
}""" AS events
)
SELECT JSON_VALUE(events, '$.id') AS id, name, value
FROM sample_table,
UNNEST (REGEXP_EXTRACT_ALL(events, r'"name":"(\w+)"')) name WITH offset
JOIN UNNEST (REGEXP_EXTRACT_ALL(events, r'"value":([0-9.]+)')) value WITH offset
USING (offset);
Query results
2. Javascript UDF
or, you might consider below using Javascript UDF.
CREATE TEMP FUNCTION extract_pair(json STRING)
RETURNS ARRAY<STRUCT<name STRING, value STRING>>
LANGUAGE js AS """
result = [];
for (const [key, value] of Object.entries(JSON.parse(json))) {
result.push(value);
}
return result;
""";
WITH sample_table AS (
SELECT """{
"id":1234,
"data":{
"packet1":{"name":"packet1", "value":1},
"packet2":{"name":"packet2", "value":2}
}
}""" AS events
)
SELECT JSON_VALUE(events, '$.id') AS id, obj.*
FROM sample_table, UNNEST(extract_pair(JSON_QUERY(events, '$.data'))) obj;
#Jaytiger's suggestion of unnesting a regex extract led me to the following solution.
The example I showed was simplified - there are more fields within the packets. To avoid requiring separate regex for each field name, I used regex to split/extract each individual packet, and then read the JSON.
This iteration doesn't do everything in one step but works when just looking at packets.
with sample_data
AS (SELECT """{"packet1":{"name":"packet1", "value":1},
"packet2":{"name":"packet2", "value":2}}""" as packets)
select
json_value('{'||packet||'}', "$.name") name,
json_value('{'||packet||'}', "$.value") value
from sample_data,
unnest(regexp_extract_all(packets, r'\:{(.*?)\}')) packet
Many similar questions, but unfortunately non helped me solve my problem. I tried to && and #> and similar, but no success.
I have a postgres DB with a table, that has a "value" column typed "json". All rows have the same basic structure, a simple JSON object, with the att "value" holding an array of strings:
{
value: ['one', 'two', 'three']
}
I need to make a query accepting an array of strings and returns all the rows, in which the value array and the passed array of strings have at least one common element.
Following the upper example, if I send ['one', 'four'], it should return the row with value: ['one', 'two', 'three'], since there is an intersection - 'one'.
If I send the array ['four', 'five', 'six'], it will not return this row.
You can use the ?| operator for that. But as you are using json and not the recommended jsonb type, you need to cast your column:
select *
from the_table
where (value::jsonb -> 'value') ?| array['one', 'four']
I have a big JSON data in one column called response_return in a Postgres DB, with a response like:
{
"customer_payment":{
"OrderId":"123456789",
"Customer":{
"Full_name":"Francis"
},
"Payment":{
"AuthorizationCode":"9874565",
"Recurrent":false,
"Authenticate":false,
...
}
}
}
I tried to use Postgres functions like -> ,->> ,#> or #> to walk through headers to achieve AuthorizationCode for a query.
When I use -> in customer_payment in a SELECT, returns all after them. If I try with OrderId, it's returned NULL.
The alternatives and sources:
Using The JSON Datatype In PostgreSQL
Operator ->
Allows you to select an element based on its name.
Allows you to select an element within an array based on its index.
Can be used sequentially: ::json->'elementL'->'subelementM'->…->'subsubsubelementN'.
Return type is json and the result cannot be used with functions and operators that require a string-based datatype. But the result can be used with operators and functions that require a json datatype.
Query for element of array in JSON column
This is not helpful because I don't want filter and do not believe that need to transform to array.
If you just want to get a single attribute, you can use:
select response_return -> 'customer_payment' -> 'Payment' ->> 'AuthorizationCode'
from the_table;
You need to use -> for the intermediate access to the keys (to keep the JSON type) and ->> for the last key to return the value as a string.
Alternatively you can provide the path to the element as an array and use #>>
select response_return #>> array['customer_payment', 'Payment', 'AuthorizationCode']
from the_table;
Online example
I have a nullable column with the JSON type:
CREATE TABLE mytable (mycolumn JSON);
What I want to do is track events in an array as they come and keep each event in the form of an object inside this array. The desired contents of mycolumn after three events have been pushed into the array would be:
[
{"product": ["book"], "subgenre": ["scifi"], "genre": ["fiction"]},
{"product": ["book"], "subgenre": ["space"], "genre": ["fiction"]},
{"product": ["book"], "genre": ["romance"]},
]
The shape of the objects are irrelevant and unknown (the above are just examples). The only known is that each event will be an object with at least one property. Wether that property is an array, object, scalar, string, or null is unknown.
The column will be null initially and my tests revealed that I need to coalesce it into an array or pushing into it will fail.
The closest I got to making this work was:
UPDATE
mytable
SET
mycolumn = JSON_ARRAY_APPEND (
COALESCE (mycolumn, '[]'),
'$',
(
'{"product": ["book"], "subgenre": ["scifi"], "genre": ["fiction"], "type": ["newrelease"]}'
)
);
The problem is that this query interprets the whole object as a string and I end up with an array of strings instead of an array of objects:
SELECT mycolumn FROM mytable;
[
"{\"product\":[\"book\"],\"subgenre\":[\"scifi\"],\"genre\":[\"fiction\"]}",
"{\"product\":[\"book\"],\"subgenre\":[\"space\"],\"genre\":[\"fiction\"]}",
"{\"product\":[\"book\"],\"genre\":[\"romance\"]}"
]
Looks to me like you want to use something like the JSON_MERGE_PRESERVE function, not JSON_ARRAY_APPEND function.
The latter evaluates the third argument as a value, it doesn't evaluate the third argument as a JSON document.
In the UPDATE statement shown, the spurious parens around the third argument (to JSON_ARRAY_APPEND) have no meaning. That third argument is just a value. The value is a long-ish string that looks like JSON, but in this context, it's just a string.
Reference: https://dev.mysql.com/doc/refman/8.0/en/json-modification-functions.html#function_json-array-append
My suggestion to testing and development of expressions... it is easier and faster to use SELECT statements. Once we have expressions that are returning the expected/desired results, then we can move the expression into an UPDATE statement.
cast('{"product": ["book"], "subgenre": ["scifi"], "genre": ["fiction"], "type": ["newrelease"]}' as json)
This is a similar question:
MySQL append json object to array of json objects
I'm trying to write a SQL statement that will parse some JSON and return only rows where one of the arrays in the JSON Object contains a given value.
Example JSON:
Object 1:
{
"Key1": ["item1", "item2", "item3"]
}
Object 2:
{
"Key1": ["item1", "item3"]
}
I would like to only return rows where JSON_QUERY(object, '$.Key1').Contains("item2") is true (in this example, Object 1).
Of course, this magical function 'Contains()' does not exist in tsql, and I can't find any documentation of a function that performs as I'd like.
EDIT:
My current solution (which I'm not very fond of and would like to replace) checks if the string literal '"item1"' is contained within the value returned by JSON_QUERY. I don't like this, because it's possible an entry in the array could have a value like '123123"item1"123123', and then the conditional would return true.
Christian,
Sounds to me like your SQL Query could use a where clause using a LIKE comparison on a column
WHERE col1 LIKE 'some%funky%string'
The % is a wildcard declaritive
You can use OPENJSON to get a derived result set out of a json list or array:
The followin query uses JSON_QUERY to retrieve the list of strings within Key1. This is passed as argument into OPENJSON to retrive the list of strings as derived table:
DECLARE #jsonTable TABLE(ID INT IDENTITY, JsonString NVARCHAR(MAX));
INSERT INTO #jsonTable VALUES
(N'{
"Key1": ["item1", "item2", "item3"]
}')
,(N'{
"Key1": ["item1", "item3"]
}');
DECLARE #LookFor NVARCHAR(100)='Item2'
SELECT jt.ID
,jt.JsonString
,A.value
FROM #jsonTable jt
OUTER APPLY OPENJSON(JSON_QUERY(jt.JsonString,N'$.Key1')) AS A
--WHERE A.value=#LookFor
Uncomment the final WHERE to reduce the list to the rows with a value of Item2 (as defined in the variable #LookFor).