MYSQL group by and count on JSON column - mysql

I have an sql table with json data like this
id | int(11) Primary Key
reasons | json
On the json_data column i have arrays stored.
id| reasons
1 | ["price","inactive","small","other"]
2 | ["price","other"]
3 | ["price","inactive"]
I need to group by "reasons" and get the count
price | 3
inactive | 2
small | 1
other | 2
I have tried this;
$dealInvestors = Deal::select(
"JSON_CONCAT(decline_reasons,'$.$all_possible_reasons)",
'count(reasons) as count',
)
->groupBy('reasons')
->get();
But it doesn't work.
Can anyone help me out?

SELECT reason, COUNT(*)
FROM test
CROSS JOIN JSON_TABLE(test.reasons,
'$[*]' COLUMNS (reason VARCHAR(255) PATH '$')
) jsontable
GROUP BY reason
https://dbfiddle.uk/?rdbms=mysql_8.0&fiddle=43cdbb8bfca8aa3cd3e82ccf775a577e
PS. I doubt that this query can be converted to Laravel syntax. Use raw SQL.

Related

Query mysql json column array using AND NOT CONTAINS (mysql 5.7)

I have a table with a json column that contains an array of objects, like the following:
create table test_json (json_id int not null primary key, json_data json not null) select 1 as json_id, '[{"category":"circle"},{"category":"square", "qualifier":"def"}]' as json_data union select 2 as json_id, '[{"category":"triangle", "qualifier":"xyz"},{"category":"square"}]' as json_data;
+---------+----------------------------------------------------------------------------------------+
| json_id | json_data |
+--------------------------------------------------------------------------------------------------+
| 1 | [{"category":"circle"}, {"category":"square", "qualifier":"def"}] |
| 2 | [{"category":"triangle", "qualifier":"xyz"}, {"category":"square"}] |
+---------+----------------------------------------------------------------------------------------+
I'd like to be able to query this table to look for any rows (json_id's) that contain a json object in the array with both a "category" value of "square" and no "qualifier" property.
The sample table above is just a sample and I'm looking for a query that would work over hundreds of rows and hundreds of objects in the json array.
In MySQL 8.0, you would use JSON_TABLE() for this:
mysql> select json_id, j.* from test_json, json_table(json_data, '$[*]' columns (
category varchar(20) path '$.category',
qualifier varchar(10) path '$.qualifier')) as j
where j.category = 'square' and j.qualifier is null;
+---------+----------+-----------+
| json_id | category | qualifier |
+---------+----------+-----------+
| 2 | square | NULL |
+---------+----------+-----------+
It's not clear why you would use JSON for this at all. It would be better to store the data in the normal manner, one row per object, with category and qualifier as individual columns.
A query against normal columns is a lot simpler to write, and you can optimize the query easily with an index:
select * from mytable where category = 'square' and qualifier is null;
I found another solution using only MySQL 5.7 JSON functions:
select json_id, json_data from test_json
where json_extract(json_data,
concat(
trim(trailing '.category' from
json_unquote(json_search(json_data, 'one', 'square'))
),
'.qualifier')
) is null
This assumes the value 'square' only occurs as a value for a "category" field. This is true in your simple example, but I don't know if it will be true in your real data.
Result:
+---------+------------------------------------------------------------------------+
| json_id | json_data |
+---------+------------------------------------------------------------------------+
| 2 | [{"category": "triangle", "qualifier": "xyz"}, {"category": "square"}] |
+---------+------------------------------------------------------------------------+
I still think that it's a CodeSmell anytime you reference JSON columns in a condition in the WHERE clause. I understood your comment that this is a simplified example, but regardless of the JSON structure, if you need to do search conditions, your queries will be far easier to develop if your data is stored in conventional columns in normalized tables.
Your request is not clear. Both of your SQL records has not such properties but your JSON object has. Maybe you try to find any record that has such object. So the following is your answer:
create table test_json (json_id int not null primary key, json_data json not null) select 1 as json_id, '[{"category":"circle", "qualifier":"abc"},{"category":"square", "qualifier":"def"}]' as json_data union select 2 as json_id, '[{"category":"triangle", "qualifier":"xyz"},{"category":"square"}]' as json_data;
select * from test_json;
select * from test_json where 'square' in (JSON_EXTRACT(json_data, '$[0].category'),JSON_EXTRACT(json_data, '$[1].category'))
AND (JSON_EXTRACT(json_data, '$[0].qualifier') is NULL || JSON_EXTRACT(json_data, '$[1].qualifier') is NULL);
See Online Demo
Also see JSON Function Reference

MySQL - get a column showing COUNT() from an associated table on each row

I have a database in MySQL (5.5.60-MariaDB).
I'm doing a SELECT query to get rows from a table called revision_filters followed by various INNER JOIN's to get associated data. The query looks as follows and executes correctly:
SELECT RevisionFilters.id AS `RevisionFilters__id`,
RevisionFilters.date AS `RevisionFilters__date`,
RevisionFilters.comment AS `RevisionFilters__comment`,
filters.label AS `Filters__label`,
filters.anchor AS `Filters__anchor`,
groups.label AS `Groups__label`
FROM revision_filters RevisionFilters
INNER JOIN dev_hub_subdb.filters Filters
ON filters.id = ( RevisionFilters.filter_id )
INNER JOIN dev_hub_subdb.groups Groups
ON groups.id = ( filters.group_id )
INNER JOIN dev_hub_subdb.regulations Regulations
ON regulations.id = ( groups.regulation_id )
There is a table called revision_filters_substances. The structure of the table is as follows. In this instance revision_filter_id is a foreign key that relates to revision_filters.id.
mysql> describe revision_filters_substances;
+--------------------+-----------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+--------------------+-----------------------+------+-----+---------+----------------+
| id | mediumint(8) unsigned | NO | PRI | NULL | auto_increment |
| revision_filter_id | mediumint(8) unsigned | NO | MUL | NULL | |
| substance_id | mediumint(8) unsigned | NO | MUL | NULL | |
+--------------------+-----------------------+------+-----+---------+----------------+
What I want to do is adapt my SELECT query so that on each row returned I can get a COUNT of the number of rows in revision_filters_substances that correspond to the rows in the SELECT query for revision_filters.
In some instances, it's possible that there are no rows in revision_filters_substances corresponding to a particular revision_filters.id and in this case I need the count to return 0.
I've read https://dba.stackexchange.com/questions/133384/counting-rows-from-a-subquery
But I can't see how to adapt this to my query.
It says on the linked article
The subquery should immediately follow the FROM keyword.
So I've tried doing this immediately following FROM revision_filters RevisionFilters in the query I have already:
, (SELECT COUNT(id) FROM revision_filters_substances WHERE id = revision_filters.id) AS count_substances
But this errors:
Unknown column 'revision_filters.id' in 'where clause'
Please can someone advise if this is possible? I don't see how to specify 0 if there are no corresponding rows either, so also need advice on how to achieve that.
You have aliased the revision_filters table to RevisionFilters. Use RevisionFilters.id instead in the where clause of the Correlated Subquery.
Also, to handle "no rows", current subquery will return NULL; you would have to use Coalesce(..) around it to return 0.
SELECT .... ,
COALESCE(SELECT COUNT(id)
FROM revision_filters_substances
WHERE id = RevisionFilters.id, 0) AS count_substances
.... /* your rest of the query here (FROM, WHERE clauses etc) */

MySQL incremental columns

I have a table like below:
ID|Prototype_A|Prototype_B|Prototype_C|Prototype_D|
---------------------------------------------------
1 |Fast381A |Blue4812 | Green7181 | White4812 |
---------------------------------------------------
2 |Slow841C |Orange8312 | null | null |
---------------------------------------------------
3 |Plane281K | null | null | null |
---------------------------------------------------
I need my query to return all non null prototypes for that ID.
so for example:
1 : Fast381A,Blue4812,Green7181,White4812
2 : Slow841C,Orange8312
3 : Plane281K
Is there a way to wildcard select all columns like select(Prototype_*) or should I setup my table in a different format?
For example I've been taught this type of structure is bad practice:
ID|Prototypes|
---------------------------------------------------
1 |Fast381A,Blue4812,Green7181,White4812
---------------------------------------------------
2 |Slow841C,Orange8312
---------------------------------------------------
3 |Plane281K
---------------------------------------------------
A SQL query returns a fixed set of columns. If you want to combine the non-NULL values into a single column, I would recommend concat_ws():
select id,
concat_ws(',', Prototype_A, Prototype_B, Prototype_C, Prototype_D)
from t;
This ignores the NULL values. The query returns two columns, one is a list of prototypes.
And, the answer to your question is "Yes". You should consider changing your data structure. Having multiple columns storing the same thing, with just an index identifying them usually means that you want a separate table, with one row per id and per prototype.
EDIT:
You want a table like this:
create table ModelPrototypes (
ModelProtypeId int primary key auto_increment,
ModelId int not null,
ProtoTypeChar char(1),
Prototype varchar(255)
);
Then you would populate it with values like:
1 A Fast381A
1 B Blue4812
1 C Green7181
1 D White4812
I'm not sure if PrototypeChar is really needed, but the information is in your table.
There's no way to wildcard select columns.
What you could do:
Setup your table as
ID, Prototype_type, Prototype_name
Then use GROUP_CONCAT:
SELECT id, GROUP_CONCAT(Prototype_name SEPARATOR ',')
FROM table GROUP BY Prototype_name
"Should I setup my table in a different format?"
Yes. Your table might look as follows:
ID Prototype_Code Prototype
------------------------------
1 A Fast381A
1 B Blue4812
1 C Green7181
1 D White4812
2 A Slow841C
2 B Orange8312
3 A Plane281K

MySQL - how to optimize large set of conditions

My head is already spinning from this and I need your help.
MY DATABASE
imported CSV file: 22 columns and 11k rows
2 tables with the same data (both created from the CSV)
Added ID as PRIMARY KEY to both
All VARCHAR(60) Some columns are empty strings ' '
DB:
PID | CODE 1 | CODE 2 | CODE 3 | CODE 4 | CODE 5 | CODE X (up to 9) | ID
-------------------------------------------------------------------------
1 | a | b | c | | | | 1
2 | a | | b | d | | | 2
3 | x | | | | | y | 3
DB has 22 columns but I'm only including CODE columns (up to 9)
in which I might be interested in terms of SQL statement.
It'll be only read table - MyISAM engine then?
WHAT I'D LIKE TO DO
select PID = 1 from first table
and retrieve all PIDs from second table
IF
selected PID's column CODE 1
or
selected PID's column CODE 2 (which is b) etc (up to 9).
= any PID's CODE X
So I should get only PID 2.
edit: PID is not a ID, it's just an example code, it could be string: '002451' and I'm looking for other PIDs with the same CODES (e.g PID1 has code = a so it should find PID2 becasue one of its CODE columns contains a)
MY ATTEMPT
SELECT a.* FROM `TABLE1` a WHERE
(
SELECT * FROM `TABLE2` b WHERE b.`PID` = 1
AND
(
( b.`CODE 1` NOT IN ('') AND IN (a.`CODE 1`,a.`CODE 2`, A.`CODE 3`...) ) OR
( b.`CODE 2` NOT IN ('') AND (a.`CODE 1`,a.`CODE 2`, A.`CODE 3`...) ) OR...
I'd end up with large query - over 81 conditions. In terms of performance... well, it doesn't work.
I intuitively know that I should:
use INDEXES (on CODE 1 / CODE 2 / CODE 3 etc.?)
use JOIN ON (but I'm too stupid) - that's why I created 2 tables (let's assume I don't want TEMP. TABLES)
How to write the SQL / design the DB efficently?
The correct data structure is one row per pid and code. The simplest way is:
create table PCodes (
pid int not null,
code varchar(255),
constraint fk_PCodes_pid references p(pid)
);
Then you have the values in a single column and it is much simpler to check for matching codes.
In practice, you should have three tables:
create table Codes (
CodeId int not null auto_increment primary key,
Code varchar(255)
);
create table PCodes (
pid int not null,
codeid int not null,
constraint fk_PCodes_pid references p(pid),
constraint fk_PCodes_codeid references codes(codeid);
);
If the ordering of the codes is important for each "p", then include a priority or ordering column in the PCodes table.

SQLYOG - SQL - Merging two columns into 1 column

I have two columns displaying the same type of information but not necessarily the same data. Although some of the data overlaps each column may/may not contain information that will also include NULL values. Like so:
Company ID | Company Name | Company ID | Company Name
-----------+--------------+------------+-------------
1 | A | 1 | A
2 | B | NULL | NULL
NULL | NULL | 3 | C
I am trying to merge columns 1 and 2 to columns 3 and 4, respectively, so that I have two columns that look like this:
Company ID | Company Name
-----------+-------------
1 | A
2 | B
3 | C
Looking at similar stackoverflow questions, I have doubt this may be done easily. Is this possible? Please, let me know!
Anything helps.
As you don't seem to be around to answer questions for clarification right now, let's go ahead.
It seems, you do actually have the four columns in question in a single table - but than, there should be no duplicate column names. Once they are unique, the following should work:
UPDATE SomeTable
SET company_ID_1 = IFNULL(company_ID_1, company_ID_2)
, company_Name_1 = IFNULL(company_Name_1, company_Name_2)
WHERE
company_ID_1 IS NULL
OR
company_Name_1 IS NULL
;
If the presented is actually the output of a join, you could replace the same by:
SELECT
IFNULL(SomeTable1.company_ID, SomeTable2.company_ID) company_ID
, IFNULL(SomeTable1.company_Name, SomeTable2.company_Name) company_Name
FROM SomeTable1
LEFT JOIN SomeTable2
ON SomeTable1.company_ID = SomeTable2.company_ID
UNION ALL
SELECT
IFNULL(SomeTable1.company_ID, SomeTable2.company_ID) company_ID
, IFNULL(SomeTable1.company_Name, SomeTable2.company_Name) company_Name
FROM SomeTable1
RIGHT JOIN SomeTable2
ON SomeTable1.company_ID = SomeTable2.company_ID
WHERE SomeTable1.company_ID IS NULL
ORDER BY company_ID
;
See it in action: SQL Fiddle
Please comment, if and as this requires adjustment / further detail.