I am looking for find how to correctly query a Postgres JSONB field. Suppose I have a JSON object like so
{"key1": ["value1", "value2"], "key2": ["value1", "value3"]}
And I'm storing it in the field 'data', I can query for the existence of the key.
SELECT data from somethings WHERE data ? "key1"
Or the key and the value.
SELECT data from somethings WHERE data -> "key1" ? "value1"
But I am struggling to search by the existence of the key values. I'm looking for something like. Basically I want to find the existence of a value whether is a top-level key or one of the values in each array.
SELECT data from somethings WHERE data ? ".*" -> "value1"
I thought I was looking for jsonb_each for a bit but I am unsure how to leverage it. Any thoughts?
You can use a JSON path expression:
select *
from something
where data #? '$.* ? (#[*] == "value1")'
The $.* iterates over all keys, and the #[*] then iterates over all array elements for each key.
Related
I'm starting to explore the JSON1 library for sqlite and have been so far successful in the basic queries I've created. I'm now looking to create a more complicated query that pulls data from multiple levels.
Here's the example JSON object I'm starting with (and most of the data is very similar).
{
"height": 140.0,
"id": "cp",
"label": {
"bind": "cp_label"
},
"type": "color_picker",
"user_data": {
"my_property": 2
},
"uuid": "948cb959-74df-4af8-9e9c-c3cb53ac9915",
"value": {
"bind": "cp_color"
},
"width": 200.0
}
This json object is buried about seven levels deep in a json structure and I pulled it from the larger json construct using an sql statement like this:
SELECT value FROM forms, json_tree(forms.formJSON, '$.root')
WHERE type = 'object'
AND json_extract(value, '$.id') = #sControlID
// In this example, #sControlID is a variable that represents the `id` value we're looking for, which is 'cp'
But what I really need to pull from this object are the following:
the value from key type ("color_picker" in this example)
the values from keys bind ("cp_color" and "cp_label" in this example)
the keys value and label (which have values of {"bind":"<string>"} in this example)
For that last item, the key name (value and label in this case) can be any number of keywords, but no matter the keyword, the value will be an object of the form {"bind":"<some_string>"}. Also, there could be multiple keys that have a bind object associated with them, and I'd need to return all of them.
For the first two items, the keywords will always be type and bind.
With the json example above, I'd ideally like to retrieve two rows:
type key value
color_picker value cp_color
color_picker label cp_label
When I use json_extract methods, I end up retrieving the object {"bind":"cp_color"} from the json_tree table, but I also need to retrieve the data from the parent object. I feel like I need to do some kind of union, but my attempts have so far been unsuccessful. Any ideas here?
Note: if the {"bind":"<string>"} object doesn't exist as a child of the parent object, I don't want any rows returned.
Well, I was on the right track and eventually figured out it. I created a separate query for each of the items I was looking for, then INNER JOINed all the json_tree tables from each of the queries to have all the required fields available. Then I json_extracted the required data from each of the json fields I needed data from. In the end, it gave me exactly what I was looking for, though I'm sure it could be written more efficiently.
For anyone interested, this is what hte final query ended up looking like:
SELECT IFNULL(json_extract(parent.value, '$.type'), '_window_'), child.key, json_extract(child.value, '$.bind') FROM (SELECT json_tree.* FROM nui_forms, json_tree(nui_forms.formJSON, '$') WHERE type = 'object' AND json_extract(nui_forms.formJSON, '$.id') = #sWindowID) parent INNER JOIN (SELECT json_tree.* FROM nui_forms, json_tree(nui_forms.formJSON, '$') WHERE type = 'object' AND json_extract(value, '$.bind') != 'NULL' AND json_extract(nui_forms.formJSON, '$.id') = #sWindowID) child ON child.parent = parent.id;
If you have any tips on reducing its complexity, feel free to comment!
I have a huge json file of about 4500 lines.
I wish to extract the value of all the keys named "value".
The levels of json paths are not same.
JSON Sample :
{
k1:v1,
k2:v2,
k3:v3,
k4:{
k5:v5,
k6:{
k7:v7,
value:"value1"
}
}
k8:v8,
value:"value2"
}
There are multiple such "value" tags.
Is there a way using jq to get all the values ?
Use recursive descent.
.. | objects | if has("value") then .value else empty end
Here is a slightly shorter variation of oguz ismail's answer which uses Optional Object Identifier .foo? and Alternative operator // along with recursive descent.
.. | .value? // empty
Example output (using corrected JSON Sample)
"value2"
"value1"
Note that this shortcut won't produce the same output if the "value" key may be null as // can't distinguish between a null "value" and a null produced by the ? operator when the "value" key is missing. It that's a concern then testing for the presence of the "value" key with has is better.
Try it online!
I have a nullable column with the JSON type:
CREATE TABLE mytable (mycolumn JSON);
What I want to do is track events in an array as they come and keep each event in the form of an object inside this array. The desired contents of mycolumn after three events have been pushed into the array would be:
[
{"product": ["book"], "subgenre": ["scifi"], "genre": ["fiction"]},
{"product": ["book"], "subgenre": ["space"], "genre": ["fiction"]},
{"product": ["book"], "genre": ["romance"]},
]
The shape of the objects are irrelevant and unknown (the above are just examples). The only known is that each event will be an object with at least one property. Wether that property is an array, object, scalar, string, or null is unknown.
The column will be null initially and my tests revealed that I need to coalesce it into an array or pushing into it will fail.
The closest I got to making this work was:
UPDATE
mytable
SET
mycolumn = JSON_ARRAY_APPEND (
COALESCE (mycolumn, '[]'),
'$',
(
'{"product": ["book"], "subgenre": ["scifi"], "genre": ["fiction"], "type": ["newrelease"]}'
)
);
The problem is that this query interprets the whole object as a string and I end up with an array of strings instead of an array of objects:
SELECT mycolumn FROM mytable;
[
"{\"product\":[\"book\"],\"subgenre\":[\"scifi\"],\"genre\":[\"fiction\"]}",
"{\"product\":[\"book\"],\"subgenre\":[\"space\"],\"genre\":[\"fiction\"]}",
"{\"product\":[\"book\"],\"genre\":[\"romance\"]}"
]
Looks to me like you want to use something like the JSON_MERGE_PRESERVE function, not JSON_ARRAY_APPEND function.
The latter evaluates the third argument as a value, it doesn't evaluate the third argument as a JSON document.
In the UPDATE statement shown, the spurious parens around the third argument (to JSON_ARRAY_APPEND) have no meaning. That third argument is just a value. The value is a long-ish string that looks like JSON, but in this context, it's just a string.
Reference: https://dev.mysql.com/doc/refman/8.0/en/json-modification-functions.html#function_json-array-append
My suggestion to testing and development of expressions... it is easier and faster to use SELECT statements. Once we have expressions that are returning the expected/desired results, then we can move the expression into an UPDATE statement.
cast('{"product": ["book"], "subgenre": ["scifi"], "genre": ["fiction"], "type": ["newrelease"]}' as json)
This is a similar question:
MySQL append json object to array of json objects
I have a column named params in a table named reports which contains JSON.
I need to find which rows contain the text 'authVar' anywhere in the JSON array. I don't know the path or level in which the text could appear.
I want to just search through the JSON with a standard like operator.
Something like:
SELECT * FROM reports
WHERE params LIKE '%authVar%'
I have searched and googled and read the Postgres docs. I don't understand the JSON data type very well, and figure I am missing something easy.
The JSON looks something like this.
[
{
"tileId":18811,
"Params":{
"data":[
{
"name":"Week Ending",
"color":"#27B5E1",
"report":"report1",
"locations":{
"c1":0,
"c2":0,
"r1":"authVar",
"r2":66
}
}
]
}
}
]
In Postgres 11 or earlier it is possible to recursively walk through an unknown json structure, but it would be rather complex and costly. I would propose the brute force method which should work well:
select *
from reports
where params::text like '%authVar%';
-- or
-- where params::text like '%"authVar"%';
-- if you are looking for the exact value
The query is very fast but may return unexpected extra rows in cases when the searched string is a part of one of the keys.
In Postgres 12+ the recursive searching in JSONB is pretty comfortable with the new feature of jsonpath.
Find a string value containing authVar:
select *
from reports
where jsonb_path_exists(params, '$.** ? (#.type() == "string" && # like_regex "authVar")')
The jsonpath:
$.** find any value at any level (recursive processing)
? where
#.type() == "string" value is string
&& and
# like_regex "authVar" value contains 'authVar'
Or find the exact value:
select *
from reports
where jsonb_path_exists(params, '$.** ? (# == "authVar")')
Read in the documentation:
The SQL/JSON Path Language
jsonpath Type
I have searched quite much on this and still unanswerable. I'm using PostgreSQL. Column name is "sections" and column type is json[] in below example.
My column looks like this in database:
sections
[{"name" : "section1",
"attributes": [{"attrkey1": "value1",
"attrkey2": "value2"},
{"attrkey3": "value3",
"attrkey4": "value4"}]
},
{"name" : "section2",
"attributes": [{"attrkey3": "value5",
"attrkey6": "value6"},
{"attrkey1": "value7",
"attrkey8": "value8"}]
}]
It's json array and I want to get "attrkey3" in my result. For getting particular key from Json, I can use json_extract_path_text(json_column, 'json_property') which is working perfectly fine. But I have no idea how to get some property from json[].
If I talk about above example, I want to get value of property "attrkey2" to be shown in my result. I know it's an array so it might work differently than usual, e.g. all the values of my array would act as a different row so I might have to write subquery but no idea how to do it.
Also, I can't write index statically and get property of the json element from some particular index. My query will be generated dynamically so I would never know how many elements are inside json array.
I saw some static examples but don't know how to implement it in my case. Can someone tell me how to do this in query?
I'm not sure you have a json[] (PostgreSQL array of json values) typed column, or a json typed column, which appears to be a JSON array (like in your example).
Either case, you need to expand your array before querying. In case of json[], you need to use unnest(anyarray); in case of JSON arrays in a json typed column, you need to use json_array_elements(json) (and LATERAL joins -- they are implicit in my examples):
select t.id,
each_section ->> 'name' section_name,
each_attribute ->> 'attrkey3' attrkey3
from t
cross join unnest(array_of_json) each_section
cross join json_array_elements(each_section -> 'attributes') each_attribute
where (each_attribute -> 'attrkey3') is not null;
-- use "where each_attribute ? 'attrkey3'" in case of jsonb
select t.id,
each_section ->> 'name' section_name,
each_attribute ->> 'attrkey3' attrkey3
from t
cross join json_array_elements(json_array) each_section
cross join json_array_elements(each_section -> 'attributes') each_attribute
where (each_attribute -> 'attrkey3') is not null;
SQLFiddle
Unfortunately, you cannot use any index with your data. You need to fix your schema first, in order to do that.
If you wish to access a single element then use json_array -> index
For example, if you have json_arr=[1,2,3] then json_array -> 0 will return 1
And also, if there was a key value map data in array:
select each_data -> 'value' as value3
from t cross join jsonb_array_elements(t.sections -> 'attributes') each_attribute
where each_attribute -> 'key' = '"attrkey3"'
I am mentioning this because the great answer also provided a perfect solution for my case. By the way, also be aware of jsonb_array.. method for jsonb type attribute.