Manipulating the JSON column and split the values - json

I have the following text in one of my Postgres table as TEXT datatype:
[
{"type": "text", "values": ["General"], "valueType": "string", "fieldType": "text", "value": ["General"], "customFieldId": "ee", "name": "customer_group"},
{"type": "text", "values": ["Vienna"], "valueType": "string", "fieldType": "text", "value": ["Vienna"], "customFieldId": "eU", "name": "customer_city"},
{"type": "text", "values": ["Mario"], "valueType": "string", "fieldType": "text", "value": ["Mario"], "customFieldId": "eZ", "name": "first_name"},
{"type": "text", "values": ["2019-06-30"], "valueType": "date", "fieldType": "text", "value": ["2019-06-30"], "customFieldId": "ea", "name": "created_at_date"}
]
I need to split the values of this TEXT field to columns and rows. For that I have converted the TEXT column to JSON as below:
SELECT CAST( "customFieldValues" as JSON) "customFieldValues" FROM fr.contacts
But when I tried to manipulate this JSON value I'm getting NULL as result.
WITH CTE AS(SELECT CAST( "customFieldValues" as JSON) "customFieldValues" FROM fr.contacts
)
SELECT
"customFieldValues" ->>'customer_city' as dd
FROM CTE
Does anyone have any suggestions on this? How to get the column names and it's values in rows. I want to create a TABLE based on this data.
Any suggestions would be of great help.
below is the expected result,
customer_group customer_city first_name created_at_date
General Vienna Mario 2019-06-30

Disclaimer: It is still not clear:
Why is there one element values and one value? What is the difference?
Why are these elements arrays?
step-by-step demo:db<>fiddle
SELECT
MAX(value) FILTER (WHERE column_name = 'customer_group') AS customer_group,
MAX(value) FILTER (WHERE column_name = 'customer_city') AS customer_city,
MAX(value) FILTER (WHERE column_name = 'first_name') AS first_name,
MAX(value) FILTER (WHERE column_name = 'created_at_date') AS created_at_date
FROM (
SELECT
elems ->> 'name' AS column_name,
elems -> 'value' ->> 0 AS value,
data
FROM
mytable,
json_array_elements(data::json) elems
) s
GROUP BY data
Cast text to json with ::json
Expand the JSON array: One row for each element with json_array_elements()
Getting the value: -> 'value' gets the array, ->> 0 gets the text representation of the first array element (the only one here)
Getting the column: ->> 'name' gets the text representation of the column name
Classical pivot algorithm (turning rows to columns) with the FILTER clause.

Related

How to get a specific object in an JSON array in MySQL?

I have a JSON column "jobs" that looks like this:
[
{
"id": "1",
"done": "100",
"target": "100",
"startDate": "123123132",
"lastAction": "123123132",
"status": "0"
},
{
"id": "2",
"done": "10",
"target": "20",
"startDate": "2312321",
"lastAction": "2312321",
"status": "1"
}
]
I want to filter the array by object key values. For example: To find all items that have target > done, status != 0 and lastAction is yesterday to get response like this:
[
{
"id": "1",
"done": "19",
"target": "100",
"startDate": "123123132",
"lastAction": "123123132",
"status": "0"
}
]
I know I can extract the data to a JSON_TABLE() to do the filtering but I don't get the original object back(unless I recreate it back) and the solution is not dynamic.
Can this kind of array filtering can really be done in MySQL?
SELECT JSON_PRETTY(JSON_EXTRACT(jobs.jobs, CONCAT('$[', j.rownum-1, ']'))) AS object
FROM jobs
CROSS JOIN JSON_TABLE(
jobs.jobs, '$[*]' COLUMNS(
rownum for ordinality,
done int path '$.done',
target int path '$.target',
status int path '$.status'
)
) as j
WHERE j.target > j.done AND j.status != 0;
You also mentioned a condition on lastAction, but the example values you gave are not valid dates, so I'll leave that enhancement to you. The example above demonstrates the technique.
Yes it is possible to do it using the JSON_EXTRACT and JSON_SEARCH functions.
Let's say your table is named tbl_Jobs and the jobs column is of type JSON.
SELECT * FROM tbl_Jobs
WHERE JSON_EXTRACT(jobs, "$[*].target") = JSON_EXTRACT(jobs, "$[*].done")
AND JSON_EXTRACT(jobs, "$[*].status") != 0
AND JSON_SEARCH(jobs, 'one', DATE_SUB(CURDATE(), INTERVAL 1 DAY), NULL, "$[*].lastAction") IS NOT NULL

How to delete multiple values in JSONB array Postgresql array object

I have a JSONB array below
[
{
"name": "test",
"age": "21",
"phone": "6589",
"town": "54"
},
{
"name": "test12",
"age": "67",
"phone": "6546",
"town": "54"
},
{
"name": "test123",
"age": "21",
"phone": "6589",
"town": "54"
},
{
"name": "test125",
"age": "67",
"phone": "6546",
"town": "54"
}
]
Now I want to delete the object if the name is test or test125. How to delete multiple or single values in JSONB array?
An update statement including a subquery, which eleminates the unwanted elements with NOT IN operator and aggregates the rest by using jsonb_agg() function, would find out this operation :
Choose this :
1. UPDATE tab
SET jsdata = t.js_new
FROM
(
SELECT jsonb_agg( (jsdata ->> ( idx-1 )::int)::jsonb ) AS js_new
FROM tab
CROSS JOIN jsonb_array_elements(jsdata)
WITH ORDINALITY arr(j,idx)
WHERE j->>'name' NOT IN ('test','test125')
) t
or this one :
2. WITH t AS (
SELECT jsonb_agg( (jsdata ->> ( idx-1 )::int)::jsonb ) AS js_new
FROM tab
CROSS JOIN jsonb_array_elements(jsdata)
WITH ORDINALITY arr(j,idx)
WHERE j->>'name' NOT IN ('test','test125')
)
UPDATE tab
SET jsdata = js_new
FROM t
Demo
If you have the Postgres 12, you can use jsonb_path_query_array function to filter the jsonb here is the sample for your question:
with t (j) as ( values ('[
{"name":"test","age":"21","phone":"6589","town":"54"},
{"name":"test12","age":"67","phone":"6546","town":"54"},
{"name":"test123","age":"21","phone":"6589","town":"54"},
{"name":"test125","age":"67","phone":"6546","town":"54"}
]'::jsonb) )
select jsonb_path_query_array(j,
'$[*] ? (#.name != "test" && #.name != "test125")')
from t;
more info on https://www.postgresql.org/docs/12/functions-json.html
I would create a function that does that:
create function remove_array_elements(p_data jsonb, p_key text, p_value text[])
returns jsonb
as
$$
select jsonb_agg(e order by idx)
from jsonb_array_elements(p_data) with ordinality as t(e,idx)
where t.e ->> p_key <> ALL (p_value) ;
$$
language sql
immutable;
Then you can use it like this:
update the_table
set the_column = remove_array_elements(the_column, 'name', array['test', 'test125'])
where id = ...;
Online example

Query jsonb column to match an array of keys

I have a table items which has a jsonb column data.
The data column is something like this {"name": "aaa", "age": 23, "job": "dev"}.
How do I select items that the data has only the keys name, age?.
You can use the ? and the ?& operators.
For your usecase, it will be:
SELECT * FROM table WHERE (NOT data ? 'job') AND (data ?& array ['name', 'age'])
Use the delete operator -, example:
with items (data) as (
values
('{"name": "aaa", "age": 23}'::jsonb),
('{"name": "aaa", "age": 23, "job": "dev"}'),
('{"name": "aaa", "age": 23, "gender": "f"}')
)
select *
from items
where data - 'name'- 'age' = '{}'
data
----------------------------
{"age": 23, "name": "aaa"}
(1 row)
In Postgres 10+ you can use a text array:
select *
from items
where data - array['name', 'age'] = '{}'

How can I use MariaDB to pull values from multiple objects from a JSON array?

In MariaDB 10.2.19, I have a table named forms with a column template which always contains a JSON array of objects. Some of these objects will have properties I want to return: name (should always be present), rule, and parameters. How can I return just these three properties from the entire array, but only for objects on which rule is present?
A sample array (formatted for easier viewing):
[{
"label": "Employed?",
"class": "select",
"name": "employed",
"parameters": "Yes",
"rule": "in"
},
{
"label": "Breed of dog?",
"class": "select",
"name": "breed",
"parameters": "spaniel, collie, mix",
"rule": "in"
},
{
"label": "Number",
"class": "text",
"name": "breed"
}]
If you are using MySQL 8.0.4 or later one way is using JSON_TABLE:
mysql> SELECT * FROM foo;
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| data |
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| [{"name": "employed", "rule": "in", "class": "select", "label": "Employed?", "parameters": "Yes"}, {"name": "breed", "rule": "in", "class": "select", "label": "Breed of dog?", "parameters": "spaniel, collie, mix"}, {"name": "breed", "class": "text", "label": "Number"}] |
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0,00 sec)
mysql> SELECT name, parameters
FROM foo,
JSON_TABLE (
foo.data,
"$[*]" COLUMNS (
name VARCHAR(100) PATH "$.name",
rule VARCHAR(100) PATH "$.rule",
parameters VARCHAR(100) PATH "$.parameters")
) AS t
WHERE rule IS NOT NULL;
+----------+----------------------+
| name | parameters |
+----------+----------------------+
| employed | Yes |
| breed | spaniel, collie, mix |
+----------+----------------------+
2 rows in set (0,00 sec)
https://dev.mysql.com/doc/refman/8.0/en/json-table-functions.html
You can accomplish this by using a dedicated numbers_table table:
SELECT
JSON_VALUE(f.template,CONCAT('$[',n.number,'].name')) AS `name`,
JSON_VALUE(f.template,CONCAT('$[',n.number,'].rule')) AS `rule`,
JSON_VALUE(f.template,CONCAT('$[',n.number,'].parameters')) AS `parameters`
FROM forms AS f
JOIN numbers_table AS n
WHERE
n.number < JSON_LENGTH(f.template) AND
JSON_VALUE(f.template,CONCAT('$[',n.number,'].rule')) IS NOT NULL;
The numbers_table table contains a single column called number which starts at 0 and can be as long as your use cases require (I have 1000 values 0 to 999). It is very useful for extracting each element of a JSON_ARRAY field into its own row.
The first WHERE condition makes sure we only use as many numbers as there are elements in the JSON_ARRAY (template in your case).
The second WHERE condition eliminates the ones that don't have a rule as per your use case.

Redshift - SQL script to extract value from a key value pair

I have a columns containing JSON data as below. I am trying to extract values corresponding to each key pair in the column. Could anyone advice how could I do using SQL
[{"id": 101, "id1": {"key": "SaleId", "type": "identifier", "regex": null}, "id2": {"key": Name, "type": "identifier", "regex": null}, "id3": {"key": null, "type": "identifier", "regex": null}}]
Key values are id1, id2, id3
Expected output:
id1 : SaleId
id2 : Name
id3 : null
I am using Redshift. Thanks
I don't know anything about Redshift, so this might not Work.
It Works in JavaScript:
/"(id\d)":\s\{"key": "?(\w+)"?/g
You will then have to extract Group 1, containing the id and Group 2, containing the key.
The regex starts by matching a double quote, then creating a Group with the Word 'id' followed by a digit, a colon, a Space, a left curly brace, a double quote, the Word 'key', a colon, a Space, an optional double quote. Finally it creates a Group with one or more Word characters, followed by an optional double quote.
As I said, I don't know Redshift, for instance, you might have to escape double quotes.
You can do what you need like this
with t as
(
select '[{"id": 101, ' ||
'"id1": {"key": "SaleId", "type": "identifier", "regex": "null"}, ' ||
'"id2": {"key": "Name", "type": "identifier", "regex": "null"}, ' ||
'"id3": {"key": "null", "type": "identifier", "regex": "null"}}]' as str
)
select 'id1:' || json_extract_path_text(substring(str,2,length(str)-2),'id1','key'),
'id2:' || json_extract_path_text(substring(str,2,length(str)-2),'id2','key'),
'id3:' || json_extract_path_text(substring(str,2,length(str)-2),'id3','key')
from t;
The JSON string in your example is invalid because Name is not in double quotes.
Assuming this is a typo and this is meant to be a valid JSON string, then you can use JSON functions to extract the values you need from the column.
Example (I have added quotes around "Name"):
create temp table jsontest (myjsonstring varchar(1000))
;
insert into jsontest(myjsonstring)
values ('[{"id": 101, "id1": {"key": "SaleId", "type": "identifier", "regex": null}, "id2": {"key": "Name", "type": "identifier", "regex": null}, "id3": {"key": null, "type": "identifier", "regex": null}}]')
;
select 'id1', json_extract_path_text(json_extract_array_element_text(myjsonstring, 0) , 'id1', 'key') from jsontest
union all
select 'id2', json_extract_path_text(json_extract_array_element_text(myjsonstring, 0) , 'id2', 'key') from jsontest
union all
select 'id3', json_extract_path_text(json_extract_array_element_text(myjsonstring, 0) , 'id3', 'key') from jsontest
;