all.
I have a query in a ExecuteSQLRecord processor which produces a JSON column in the result:
SELECT t.id, json_agg(json_build_object("value1", t.column1, "value2", t.column2)) AS "test" FROM table t GROUP BY t.id
The processor uses a JSONRecordSetWriter to write the results. However, the processor returns a String from the database, which causes issues in further processing that expects a JSON value:
{
...
"test": "[{\"value1\": \"column1value\", \"value2\": \"column2value\"}, ...]"
}
Is there a way of turning it to a proper JSON value, i.e., in this format:
{
...
"test": [{"value1": "column1value", "value2": "column2value"}, ...]
}
without resorting to ReplaceText processors?
I have tried to replace the quotes in the SQL query, to use jsonb_agg() and jsonb_build_object(), and to use a QueryDatabaseTableRecord but none worked.
Thanks in advance.
Related
I want to combine the following JSON from multiple rows into one single JSON object as a row.
{"Salary": ""}
{"what is your name?": ""}
{"what is your lastname": ""}
Expected output
{
"Salary": "",
"what is your name?": "",
"what is your lastname": ""
}
With only built-in functions, you need to expand the rows into key/value pairs and aggregate that back into a single JSON value:
select jsonb_object_agg(t.k, t.v)
from the_table, jsonb_each(ob) as t(k,v);
If your column is of type json rather than jsonb you need to cast it:
select jsonb_object_agg(t.k, t.v)
from the_table, jsonb_each(ob::jsonb) as t(k,v);
A slightly more elegant solution is to define a new aggregate that does that:
CREATE AGGREGATE jsonb_combine(jsonb)
(
SFUNC = jsonb_concat(jsonb, jsonb),
STYPE = jsonb
);
Then you can aggregate the values directly:
select jsonb_combine(ob)
from the_table;
(Again you need to cast your column if it's json rather than jsonb)
Online example
How do I access the value of a object property, inside an array, using JSON_QUERY in MariaDB? I have a JSON field with following data, and want to access the value of the section property.
[
[
{"section": "search"}
]
]
The following SQL is returning NULL
SELECT JSON_QUERY('[[{"section": "search"}]]', '$[0][0].section') FROM mytable
For anyone else having the same issue, you have to use JSON_VALUE for accessing scalar values. The following query gives the expected result.
SELECT JSON_VALUE'[[{"section": "search"}]]', '$[0][0].section') FROM mytable;
JSON_QUERY vs JSON_VALUE
I have a structure like this:
{"payload": {
"Item1": {
"property1": "Sunday",
"property2": "suffering_8890"
},
"Item2": {
"property1": "Monday",
"property2": "misery_0776"
},
"Item3": {
"property1": "Tuesday",
"property2": "pain_6756"
}
}
}
I need the property2 value that contains a certain sub-string (i.e- "misery"). Ultimately I just need the 4-digit code, but even getting the full value will work. I am able to get a list of all the property2 values by using:
$..property2
This returns:
Result[0] = suffering_8890
Result[1] = misery_0776
Result[2] = pain_6756
How do I filter it so that it only gives me the result containing the substring "misery"?
With regards to full value you can use a Filter Operator like:
$..[?(#.property2 =~ /misery.*?/i)].property2
Demo:
You can extract the 4-digit value out of the variable using Regular Expression Extractor
If you want to do this in one shot:
Add JSR223 PostProcessor as a child of the request which returns above JSON
Put the following code into "Script" area
vars.put('misery', ((com.jayway.jsonpath.JsonPath.read(prev.getResponseDataAsString(), '$..[?(#.property2 =~ /misery.*?/i)].property2').get(0) =~ ('(\\d+)'))[0][1]))
Refer the extracted value as ${misery} where required
More information: Apache Groovy - Why and How You Should Use It
You can use a regular expression extractor as well for the mentioned scenario to fetch the four-digit as below
property2": "misery_(.*)"
This will help you to save the four-digit code in JMeter variables right away. If you insist you to find out the required value from JSON using jsonpath, you can use JSON filter as below :-
$..[?(#.property2.indexOf('misery')>=0)]..property2
I have a nullable column with the JSON type:
CREATE TABLE mytable (mycolumn JSON);
What I want to do is track events in an array as they come and keep each event in the form of an object inside this array. The desired contents of mycolumn after three events have been pushed into the array would be:
[
{"product": ["book"], "subgenre": ["scifi"], "genre": ["fiction"]},
{"product": ["book"], "subgenre": ["space"], "genre": ["fiction"]},
{"product": ["book"], "genre": ["romance"]},
]
The shape of the objects are irrelevant and unknown (the above are just examples). The only known is that each event will be an object with at least one property. Wether that property is an array, object, scalar, string, or null is unknown.
The column will be null initially and my tests revealed that I need to coalesce it into an array or pushing into it will fail.
The closest I got to making this work was:
UPDATE
mytable
SET
mycolumn = JSON_ARRAY_APPEND (
COALESCE (mycolumn, '[]'),
'$',
(
'{"product": ["book"], "subgenre": ["scifi"], "genre": ["fiction"], "type": ["newrelease"]}'
)
);
The problem is that this query interprets the whole object as a string and I end up with an array of strings instead of an array of objects:
SELECT mycolumn FROM mytable;
[
"{\"product\":[\"book\"],\"subgenre\":[\"scifi\"],\"genre\":[\"fiction\"]}",
"{\"product\":[\"book\"],\"subgenre\":[\"space\"],\"genre\":[\"fiction\"]}",
"{\"product\":[\"book\"],\"genre\":[\"romance\"]}"
]
Looks to me like you want to use something like the JSON_MERGE_PRESERVE function, not JSON_ARRAY_APPEND function.
The latter evaluates the third argument as a value, it doesn't evaluate the third argument as a JSON document.
In the UPDATE statement shown, the spurious parens around the third argument (to JSON_ARRAY_APPEND) have no meaning. That third argument is just a value. The value is a long-ish string that looks like JSON, but in this context, it's just a string.
Reference: https://dev.mysql.com/doc/refman/8.0/en/json-modification-functions.html#function_json-array-append
My suggestion to testing and development of expressions... it is easier and faster to use SELECT statements. Once we have expressions that are returning the expected/desired results, then we can move the expression into an UPDATE statement.
cast('{"product": ["book"], "subgenre": ["scifi"], "genre": ["fiction"], "type": ["newrelease"]}' as json)
This is a similar question:
MySQL append json object to array of json objects
I have a column of text type be contain JSON value.
{
"customer": [
{
"details": {
"customer1": {
"name": "john",
"addresses": {
"address1": {
"line1": "xyz",
"line2": "pqr"
},
"address2": {
"line1": "abc",
"line2": "efg"
}
}
}
"customer2": {
"name": "robin",
"addresses": {
"address1": null
}
}
}
}
]
}
How can I extract 'address1' JSON field of column with query?
First I am trying to fetch JSON value then I will go with parsing.
SELECT JSON customer from text_column;
With my query, I get following error.
com.datastax.driver.core.exceptions.SyntaxError: line 1:12 no viable
alternative at input 'customer' (SELECT [JSON] customer...)
com.datastax.driver.core.exceptions.SyntaxError: line 1:12 no viable
alternative at input 'customer' (SELECT [JSON] customer...)
Cassandra version 2.1.13
You can't use SELECT JSON in Cassandra v2.1.x CQL v3.2.x
For Cassandra v2.1.x CQL v3.2.x :
The only supported operation after SELECT are :
DISTINCT
COUNT (*)
COUNT (1)
column_name AS new_name
WRITETIME (column_name)
TTL (column_name)
dateOf(), now(), minTimeuuid(), maxTimeuuid(), unixTimestampOf(), typeAsBlob() and blobAsType()
In Cassandra v2.2.x CQL v3.3.x Introduce : SELECT JSON
With SELECT statements, the new JSON keyword can be used to return each row as a single JSON encoded map. The remainder of the SELECT statment behavior is the same.
The result map keys are the same as the column names in a normal result set. For example, a statement like “SELECT JSON a, ttl(b) FROM ...” would result in a map with keys "a" and "ttl(b)". However, this is one notable exception: for symmetry with INSERT JSON behavior, case-sensitive column names with upper-case letters will be surrounded with double quotes. For example, “SELECT JSON myColumn FROM ...” would result in a map key "\"myColumn\"" (note the escaped quotes).
The map values will JSON-encoded representations (as described below) of the result set values.
If your Cassandra version is 2.1x and below, you can use the Python-based approach.
Write a python script using Cassandra-Python API
Here you have to get your row first and then use python json's loads method, which will convert your json text column value into JSON object which will be dict in Python. Then you can play around with Python dictionaries and extract your required nested keys. See the below code snippet.
from cassandra.cluster import Cluster
from cassandra.auth import PlainTextAuthProvider
import json
if __name__ == '__main__':
auth_provider = PlainTextAuthProvider(username='xxxx', password='xxxx')
cluster = Cluster(['0.0.0.0'],
port=9042, auth_provider=auth_provider)
session = cluster.connect("keyspace_name")
print("session created successfully")
rows = session.execute('select * from user limit 10')
for user_row in rows:
customer_dict = json.loads(user_row.customer)
print(customer_dict().keys()