Sum fields inside json array in mysql - mysql

I have this table:
CREATE TABLE stackoverflow_question (
id int NOT NULL AUTO_INCREMENT,
name varchar(255) NOT NULL,
json_ob mediumtext default null,
PRIMARY KEY (id)
);
I do some inserts:
insert into stackoverflow_question values(null, 'albert', '[{name: "albert1", qt: 2},{name: "albert2", qt: 2}]');
insert into stackoverflow_question values(null, 'barbara', '[{name: "barbara1", qt: 4},{name: "barbara2", qt: 7}]');
insert into stackoverflow_question values(null, 'paul', '[{name: "paul1", qt: 9},{name: "paul2", qt: 11}]');
Eventually, I will need to sort this table by total quantity.
in the examples above, "paul" has quantity = 20, while "barbara" has quantity = 11. And "albert" has quantity = 4.
Is it possible to create a select statement where a new field is created on the fly? Something like this:
SELECT
SUM (loop json_ob and sum all the quantity fields) AS total_quantity,
id,
name
FROM
stackoverflow_question
ORDER BY total_quantity

If json_ob is actually a valid json object then you can use JSON_TABLE() to extract the quantities and aggregate:
SELECT s.*, SUM(t.qt) total_quantity
FROM stackoverflow_question s,
JSON_TABLE(json_ob, '$[*]' COLUMNS (qt INTEGER PATH '$.qt')) t
GROUP BY s.id
ORDER BY total_quantity DESC;
See the demo.

According to jsonlint your JSON is not valid.
That's why this SQL returns an error (ERROR 3141 (22032): Invalid JSON text in argument 1 to function json_table: "Missing a name for object member." at position 2.")
SELECT
j.name, j.qt
FROM JSON_TABLE('[{name: "paul1", qt: 9},{name: "paul2", qt: 11}]',
"$[*]" COLUMNS (name varchar(20) PATH "$.name", qt int PATH "$.qt")) j ;
and this will return the values:
SELECT
j.name, j.qt
FROM JSON_TABLE('[{"name": "paul1", "qt": 9},{"name": "paul2", "qt": 11}]',
"$[*]" COLUMNS (name varchar(20) PATH "$.name", qt int PATH "$.qt")) j ;
output:
name
qt
paul1
9
paul2
11
You can convert your relaxedJSON, to JSON, using tools like : www.relaxedjson.org

Related

How to get data from json in a mysql query correctly?

I'm trying to pull data from json and insert it into a table. New rows appear in the table by number as in json, but all fields have an empty value, NULL. Please help me figure it out.
[{"name":"ivan","city":"london","kurs":"1"},{"name":"lena","city":"tokio","kurs":"5"},{"name":"misha","city":"kazan","kurs":"3"}]
SET #json = CONVERT(LOAD_FILE('/var/lib/mysql-files/myfile.json') using utf8mb4);
REPLACE INTO test (name, city, kurs)
SELECT
JSON_VALUE(#json, '$.name') as name,
JSON_VALUE(#json, '$.city') as city,
JSON_VALUE(#json, '$.kurs') as kurs
FROM JSON_TABLE(#json,'$[*]' COLUMNS (data JSON PATH '$')) jsontable
Use this, (and change VARCAR(20) to the appropriate definition):
set #json = '[{"name":"ivan","city":"london","kurs":"1"},{"name":"lena","city":"tokio","kurs":"5"},{"name":"misha","city":"kazan","kurs":"3"}]';
select * from json_table(#json,'$[*]' columns(name varchar(20) path '$.name',
city varchar(20) path '$.city',
kurs varchar(20) path '$.kurs')) as jsontable;
see: DBFIDDLE
output:
name
city
kurs
ivan
london
1
lena
tokio
5
misha
kazan
3

Create json object and aggregate into json array in SqlServer

I have following query in ORACLE:
SELECT *
FROM "Supplier" s
OUTER APPLY(
SELECT JSON_ARRAYAGG(JSON_OBJECT(p."Id", p."Description", p."Price")) as "Products"
FROM "Products" p
WHERE p."SupplierId" = s."Id"
) sp
In OUTER APPLY subquery I am creating json from columns I need and then aggregating those objects into json array. I need those two functions because sometimes I use only one of them. The same operation I would like to do in SqlServer. This is solution I managed so far:
SELECT *
FROM "Supplier" as s
OUTER APPLY(
SELECT p."Id", p."Description", p."Price"
FROM "Products" as p
WHERE p."SupplierId" = s."Id"
FOR JSON PATH
) as sp("Products")
The problem is that SqlServer executing those two functions at once (this is purpose for FOR JSON PATH statement). So here are my questions:
1) Is there possible to create json object without wrapping it into array (oracle-like syntax)?
2) Is there possible to aggregate json objects into an array?
UPDATE
I am using SqlServer version 2019 15.0.2000.5
Expected result (single record of products in json format)
"Products":{
"Id":"FEB0646B709B45B5A306E10599716F28",
"Description":"Database Manager",
"Price":149.99
}
If I understand the question correctly, the following statements are possible soltion (of course, they are based on the example data and statements in the question):
How to create a single JSON object:
If you want to generate one single JSON object, you need to use FOR JSON PATh for each row in the OUTER APPLY statement with the appropriate path expression. JSON_QUERY() is needed, because it returns a valid JSON and FOR JSON doesn't escape special characters.
Tables:
CREATE TABLE Supplier (
Id int,
Description varchar(50),
DateStart date
)
CREATE TABLE Products (
Id varchar(5),
SupplierId int,
Description varchar(100),
Price numeric(10, 2)
)
INSERT INTO Supplier (Id, Description, DateStart)
VALUES (1, 'Oracle', '19900505')
INSERT INTO Products (Id, SupplierId, Description, Price)
VALUES ('11111', 1, 'Database Manager', 149.99)
INSERT INTO Products (Id, SupplierId, Description, Price)
VALUES ('22222', 1, 'Chassi', 249.99)
Statement:
SELECT *
FROM "Supplier" s
OUTER APPLY(
SELECT Products = JSON_QUERY((
SELECT
p."Id" AS 'Product.Id',
p."Description" AS 'Product.Description',
p."Price" AS 'Product.Price'
FOR JSON PATH, WITHOUT_ARRAY_WRAPPER
))
FROM "Products" as p
WHERE p."SupplierId" = s."Id"
) sp ("Products")
Result:
Id Description DateStart Products
1 Oracle 1990-05-05 {"Product":{"Id":"11111","Description":"Database Manager","Price":149.99}}
1 Oracle 1990-05-05 {"Product":{"Id":"22222","Description":"Chassi","Price":249.99}}
How to aggregate JSON objects into an array:
By default FOR JSON creates a JSON array with one JSON object for each row. You only need to set a root key:
Statement:
SELECT *
FROM "Supplier" s
OUTER APPLY(
SELECT p."Id", p."Description", p."Price"
FROM "Products" p
WHERE p."SupplierId" = s."Id"
FOR JSON PATH
) sp("Products")
Result:
Id Description DateStart Products
1 Oracle 1990-05-05 [{"Id":"11111","Description":"Database Manager","Price":149.99},{"Id":"22222","Description":"Chassi","Price":249.99}]
DECLARE #data varchar(max)
DECLARE #LIST NVARCHAR(MAX)
DECLARE #Temp TABLE (YourColumnName VARCHAR(MAX) NULL);
INSERT INTO #Temp SELECT DISTINCT columnName FROM YourTableName WHERE(Id > 1000);
SELECT #LIST = STRING_AGG(CONVERT(NVARCHAR(max), ISNULL(YourColumnName, 'N/A')), ',') FROM #Temp
SET #data =(select #LIST as Name1,#LIST as Name2 For Json PATH)
select #data

Data truncation: '' While Inserting or Updating Value

I am trying to Insert/update a string (JSON). While inserting/Updating the value I get data truncation error.
I have tried making a JSON type column and passing an JSON_OBJECT() type but that fails as well.
select '''[{"id":"1202","title":"Asian","notes":"","active":"1"}]''';
CREATE TABLE mktesttable (
id int NOT NULL,
s VARCHAR(34530) NOT NULL
);
INSERT INTO mktesttable
select 1, '''[{"id":"1202","title":"Asian","notes":"","active":"1"}]''';
select * from mktesttable;
// That Works
INSERT INTO mktesttable
SELECT
patient_data.id,
CONCAT(
'''[{"id":"', patient_data.race,
'","title":"', list_options.title,
'","notes":"', list_options.notes,
'","active":"', list_options.active,
'"}]'''
) as s
FROM
patient_data
INNER JOIN list_options
ON patient_data.race = list_options.id order by 1 desc
Yields same result (Id's and data varies) but doesn't work
Result Set
If you want to store JSON object, you should use the JSON datatype instead of strings. To create a JSON object, you can use JSON_OBJECT.
CREATE TABLE mktesttable (
id int NOT NULL,
s JSON NOT NULL
);
INSERT INTO mktesttable
SELECT
patient_data.id,
JSON_OBJECT(
'id', patient_data.race,
'title', list_options.title,
'notes', list_options.notes,
'active', list_options.active,
)
FROM
patient_data
INNER JOIN list_options
ON patient_data.race = list_options.id
ORDER BY patient_data.id desc
If you need a JSON array as shown in your sample data, then :
JSON_ARRAY(
JSON_OBJECT(
'id', patient_data.race,
'title', list_options.title,
'notes', list_options.notes,
'active', list_options.active,
)
)

How do I generate nested json objects using mysql native json functions?

Using only the native JSON fuctions (no PHP, etc) in MySQL version 5.7.12 (section 13.16 in the manual) I am trying to write a query to generate a JSON document from relational tables that contains a sub object. Given the following example:
CREATE TABLE `parent_table` (
`id` int(11) NOT NULL,
`desc` varchar(20) NOT NULL,
PRIMARY KEY (`id`)
);
CREATE TABLE `child_table` (
`id` int(11) NOT NULL,
`parent_id` int(11) NOT NULL,
`desc` varchar(20) NOT NULL,
PRIMARY KEY (`id`,`parent_id`)
);
insert `parent_table` values (1,'parent row 1');
insert `child_table` values (1,1,'child row 1');
insert `child_table` values (2,1,'child row 2');
I am trying to generate a JSON document that looks like this:
[{
"id" : 1,
"desc" : "parent row 1",
"child_objects" : [{
"id" : 1,
"parent_id" : 1,
"desc" : "child row 1"
}, {
"id" : 2,
"parent_id" : 1,
"desc" : "child row 2"
}
]
}]
I am new to MySQL and suspect there is a SQL pattern for generating nested JSON objects from one to many relationships but I'm having trouble finding it.
In Microsoft SQL (which I'm more familiar with) the following works:
select
[p].[id]
,[p].[desc]
,(select * from [dbo].[child_table] where [parent_id] = [p].[id] for json auto) AS [child_objects]
from [dbo].[parent_table] [p]
for json path
I attempted to write the equivalent in MySQL as follows:
select json_object(
'id',p.id
,'desc',p.`desc`
,'child_objects',(select json_object('id',id,'parent_id',parent_id,'desc',`desc`)
from child_table where parent_id = p.id)
)
from parent_table p;
select json_object(
'id',p.id
,'desc',p.`desc`
,'child_objects',json_array((select json_object('id',id,'parent_id',parent_id,'desc',`desc`)
from child_table where parent_id = p.id))
)
from parent_table p
Both attempts fail with the following error:
Error Code: 1242. Subquery returns more than 1 row
The reason you are getting these errors is that the parent json object is not expecting a result set as one of its inputs, you need to have simple object pairs like {name, string} etc bug report - may be available in future functionality... this just means that you need to convert your multi row results into a concatination of results separated by commas and then converted into a json array.
You almost had it with your second example.
You can achieve what you are after with the GROUP_CONCAT function
select json_object(
'id',p.id
,'desc',p.`desc`
,'child_objects',json_array(
(select GROUP_CONCAT(
json_object('id',id,'parent_id',parent_id,'desc',`desc`)
)
from child_table
where parent_id = p.id))
)
from parent_table p;
This almost works, it ends up treating the subquery as a string which leaves the escape characters in there.
'{\"id\": 1,
\"desc\": \"parent row 1\",
\"child_objects\":
[\"
{\\\"id\\\": 1,
\\\"desc\\\": \\\"child row 1\\\",
\\\"parent_id\\\": 1
},
{\\\"id\\\": 2,
\\\"desc\\\": \\\"child row 2\\\",
\\\"parent_id\\\": 1}\"
]
}'
In order to get this working in an appropriate format, you need to change the way you create the JSON output as follows:
select json_object(
'id',p.id
,'desc',p.`desc`
,'child_objects',(select CAST(CONCAT('[',
GROUP_CONCAT(
JSON_OBJECT(
'id',id,'parent_id',parent_id,'desc',`desc`)),
']')
AS JSON) from child_table where parent_id = p.id)
) from parent_table p;
This will give you the exact result you require:
'{\"id\": 1,
\"desc\": \"parent row 1\",
\"child_objects\":
[{\"id\": 1,
\"desc\": \"child row 1\",
\"parent_id\": 1
},
{\"id\": 2,
\"desc\": \"child row 2\",
\"parent_id\": 1
}]
}'
For MariaDb, CAST AS JSON does not work. But JSON_EXTRACT may be used to convert a string to a JSON object:
select json_object(
'id',p.id
,'desc',p.`desc`
,'child_objects',JSON_EXTRACT(IFNULL((select
CONCAT('[',GROUP_CONCAT(
json_object('id',id,'parent_id',parent_id,'desc',`desc`)
),']')
from child_table where parent_id = p.id),'[]'),'$')
) from parent_table p;
I tried group_concat solution but I found it's problems larger string because of group_concat limitations (group_concat_max_len).
I wrote the new function resolve the problem about converting a string to JSON object as bellow and how to use it.
Tested on MariaDB 10.5.12
Usage: https://i.stack.imgur.com/cWfd7.jpg
CREATE FUNCTION `ut_tf_array`(input_json longtext) RETURNS longtext CHARSET utf8mb4 COLLATE utf8mb4_unicode_ci
COMMENT 'Function for transform json array agg'
BEGIN
DECLARE transformed_data_list longtext ;
DECLARE record longtext ;
DECLARE i_count int ;
DECLARE i_count_items int ;
SET i_count = 0;
SET i_count_items = JSON_LENGTH(JSON_EXTRACT(input_json,'$'));
SET transformed_data_list = '[]';
-- return array with length = zero
IF input_json is NULL THEN
RETURN transformed_data_list;
END IF;
WHILE i_count < i_count_items DO
-- fetch into record
SELECT JSON_EXTRACT( JSON_EXTRACT( input_json ,'$') , CONCAT('$[',i_count,']')) INTO record;
-- append to transformed_data_list
SELECT JSON_ARRAY_APPEND(transformed_data_list, '$', JSON_EXTRACT(record, '$')) into transformed_data_list;
SET i_count := i_count + 1;
END WHILE;
-- done
RETURN transformed_data_list;
END
Below Query works for me.
SELECT JSON_ARRAYAGG(JSON_OBJECT('Id', p.id, 'desc', p.`desc`, 'child_objects', temp_json)) AS json_value
FROM (
SELECT p.id, p.`desc`,
JSON_ARRAYAGG(JSON_OBJECT('id', p.id, 'parent_id', p.parent_id, 'desc', p.`desc`)) AS temp_json
FROM parent_table p
GROUP BY p.id
) t;

MySQL error - "check syntax to use near '%s"

Anyone see why the query below would yield the error
"#1064 - You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '%s)"?
SELECT SQL_CALC_FOUND_ROWS id
FROM (
SELECT taba.id
FROM (
SELECT alum.id
FROM cvm_education AS edu
JOIN cvm_alumni AS alum ON alum.id = edu.alumni_id
WHERE cvm_alumni.profile_status =1
AND highest_edu
IN (
SELECT name
FROM cvm_filter_educationlevels
JOIN cvm_educationlevel AS edulevels ON educationlevel_id = edulevels.id
WHERE filter_id = % s
)
) AS taba
Cheers!
you need to quote the string value and use LIKE for pattern matching
WHERE filter_id LIKE '% s'
but if you really want to find % s literally, use =
WHERE filter_id = '% s'
Try this:
SELECT count(taba.id)
FROM (
SELECT alum.id
FROM cvm_education AS edu
JOIN cvm_alumni AS alum ON alum.id = edu.alumni_id
WHERE alum.profile_status =1
AND highest_edu
IN (
SELECT name
FROM cvm_filter_educationlevels
JOIN cvm_educationlevel AS edulevels ON educationlevel_id = edulevels.id
WHERE filter_id = 1
)
) AS taba ;
http://www.sqlfiddle.com/#!2/f8adc/15
Two important points:
I don't understand the use of SQL_CALC_FOUND_ROWS() if have chanced
it in count(). I think this provides the same desired result.
You haven't provided sample data so I wasn't able to try %s. I have
substituted it with a binary (1,0). Furthermore, I don't know your
exact code so I made some assumptions based on your query.
Sample data:
CREATE TABLE cvm_education(
ID int auto_increment primary key,
alumni_id int
);
CREATE TABLE cvm_alumni(
ID int auto_increment primary key,
profile_status int,
highest_edu varchar(30)
);
CREATE TABLE cvm_filter_educationlevels (
ID int auto_increment primary key,
educationlevel_id int,
name varchar(30)
);
CREATE TABLE cvm_educationlevel(
ID int auto_increment primary key,
filter_id int
);
INSERT INTO cvm_education (alumni_id)
VALUES (10), (1), (2), (3),(5), (6),(7),(8),(9);
INSERT INTO cvm_alumni (profile_status, highest_edu)
VALUES (1, "master"),
(0,"bachelor"),
(1,"bachelor"),
(0, "master"),
(1, "master"),
(0, "master"),
(1, "master"),
(1, "master"),
(1, "master"),
(1, "master");
INSERT INTO cvm_filter_educationlevels(educationlevel_id,name)
VALUES (1, "master"), (0,"bachelor");
INSERT INTO cvm_educationlevel(filter_ID)
VALUES (1), (0), (1), (0), (0), (1),(1),(1),(1);
The "% s" is invalid syntax. If that's a literal, then it needs to be enclosed in quotes:
WHERE filter_id = '% s'
(But that fix doesn't appear to be right. It almost looks as if the MySQL statement is being generated with a sprintf, and there was intended to be a '%s' placeholder that was supposed to be replaced with an value.)
Also, there's a closing parenthesis and alias missing from the end of the statement:
) foo
And this:
WHERE cvm_alumni.profile_status = 1
should be changed to this:
WHERE alum.profile_status = 1
(The table is assigned an alias, the column reference should be qualified with the alias, not the table_name)
It's also a good idea to qualify the references all column references, including educationlevel_id, highest_edu and name. (That's not necessarily a problem with the statement, unless MySQL is throwing an "ambiguous column" error, but I prefer to insulate my statements from any "ambiguous column" error that will crop up when new columns are added.)
SELECT SQL_CALC_FOUND_ROWS id
FROM (SELECT taba.id
FROM (
SELECT alum.id
FROM cvm_education edu
JOIN cvm_alumni alum
ON alum.id = edu.alumni_id
WHERE alum.profile_status = 1
AND `highest_edu` IN
(
SELECT `name`
FROM cvm_filter_educationlevels
JOIN cvm_educationlevel edulevels
ON `educationlevel_id` = edulevels.id
WHERE `filter_id` = '% s'
)
) taba
) foo