How to emulate JSON_OVERLAPS function on MySQL 5.7? - mysql

I have two columns from different tables that hold JSON-formatted data. The data stored in both columns are arrays. Example:
users
+----+------------------+
| id | options |
+----+------------------+
| 1 | ["AB","CD","XY"] |
| 2 | ["CD","GH"] |
+----+------------------+
items
+----+-------------+
| id | options |
+----+-------------+
| 10 | ["CD","EF"] |
| 11 | ["GH","XY"] |
| 12 | ["GH"] |
+----+-------------+
I wanted to write a query that returns all the rows from users which matches a given row from items, using options columns to perform the match. The rule is if any value in the array is present in both rows, they are a match. Example: user 1 would match items 10 (because of CD option) and 11 (because of XY option); user 2 would match items 10, 11 and 12 because all of them have CD or GH.
Looking at MySQL docs I found that JSON_OVERLAPS does exactly that. However, I'm running MySQL 5.7 and the function is only available starting at 8.0.17. There is also no much talking around this function on the web.
How could I emulate JSON_OVERLAPS behavior on MySQL 5.7 in a query?
Edit: Unfortunately, upgrading to MySQL 8 is not an option since we run MariaDB on production, which also doesn't have that function.

How to emulate JSON_OVERLAPS function on MySQL 5.7?
Edit: Unfortunately, upgrading to MySQL 8 is not an option since we run MariaDB on production, which also doesn't have that function.
Be warned like Strawberry, suggested already upgrading is more easy
Now that is out off the way. You still asked for it, lets have some fun.
I posted some answers in the past to simulate MySQL's 8 JSON_TABLE(), why did i mention this? Because i use this method to emulate MySQL's 8 JSON_OVERLAPS to simply JOIN both resultsets which emulate JSON_TABLE() to a final resultset
Which makes the query below (forgive the formatting)
Query
SELECT
*
FROM (
SELECT
items.id
, JSON_UNQUOTE(
JSON_EXTRACT(items.options, CONCAT('$[', number_generator.number , ']'))
) AS json_options
FROM (
SELECT
#items_row := #items_row + 1 AS number
FROM (
SELECT 0 UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9
) row1
CROSS JOIN (
SELECT 0 UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9
) row2
CROSS JOIN (
SELECT #items_row := -1
) init_user_params
) AS number_generator
CROSS JOIN (
SELECT
items.id
, items.options
, JSON_LENGTH(items.options) AS json_array_length
FROM
items
) AS items
WHERE
number BETWEEN 0 AND json_array_length - 1
) AS items
INNER JOIN (
SELECT
users.id
, JSON_UNQUOTE(
JSON_EXTRACT(users.options, CONCAT('$[', number_generator.number , ']'))
) AS json_options
FROM (
SELECT
#users_row := #users_row + 1 AS number
FROM (
SELECT 0 UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9
) row1
CROSS JOIN (
SELECT 0 UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9
) row2
CROSS JOIN (
SELECT #users_row := -1
) init_user_params
) AS number_generator
CROSS JOIN (
SELECT
users.id
, users.options
, JSON_LENGTH(users.options) AS json_array_length
FROM
users
) AS users
WHERE
number BETWEEN 0 AND json_array_length - 1
) AS users
USING(json_options)
Result
| json_options | id | id |
| ------------ | --- | --- |
| CD | 10 | 2 |
| CD | 10 | 1 |
| GH | 11 | 2 |
| GH | 12 | 2 |
| XY | 11 | 1 |
see demo

Related

How to query duplicates results MySQL

While every question I found is talking about removing duplicates I need these duplicates.
Let's say my database is
+-------+-----------+
| ID | letter |
+-------+-----------+
| 1 | A |
| 2 | B |
| 4 | Z |
+-------+-----------+
I need to query a person name so let say the name is "ABA" when I query like this
select * from letters where letter = 'A' or letter = 'B' or letter = 'A'
My result will be
+-------+-----------+
| ID | letter |
+-------+-----------+
| 1 | A |
| 2 | B |
+-------+-----------+
I want the output will include the 3rd letter as a separate row.
+-------+-----------+
| ID | letter |
+-------+-----------+
| 1 | A |
| 2 | B |
| 3 | A |
+-------+-----------+
Maybe I don't know the right term but I didn't find even one answer that give me half a solution.
there is one entry but can I can the entry again? if I query for "nina" get the full name and not just "nia"
Original answer (and recommended approach)
Use a recursive query to convert the name into rows with one letter each. Then join with your letters table.
with recursive word_letters (word, pos, letter) as
(
select #name, 1, substr(#name, 1, 1)
union all
select word, pos + 1, substr(word, pos + 1, 1)
from word_letters
where pos < length(word)
)
select letters.*
from word_letters
join letters on letters.letter = word_letters.letter
order by word_letters.pos;
Demo: https://dbfiddle.uk/?rdbms=mysql_8.0&fiddle=6547c21d2dd9223270615047d46d9783
UPDATE: Workaround for old MySQL versions
Build a table of positions (numbers) large enough to cover the longest word. Then join the word in order to get the position for each letter in it. Then join your other table.
select letters.*
from
(
select hundreds.digit * 100 + tens.digit * 10 + units.digit + 1 as pos
from (select 0 as digit union select 1 union select 2 union select 3 union select 4 union select 5 union select 6 union select 7 union select 8 union select 9) units
cross join (select 0 as digit union select 1 union select 2 union select 3 union select 4 union select 5 union select 6 union select 7 union select 8 union select 9) tens
cross join (select 0 as digit union select 1 union select 2 union select 3 union select 4 union select 5 union select 6 union select 7 union select 8 union select 9) hundreds
) positions
join (select #name as word) w on length(w.word) >= positions.pos
join letters on letters.letter = substr(w.word, positions.pos, 1)
order by positions.pos;
Demo: https://dbfiddle.uk/?rdbms=mysql_5.7&fiddle=db3762d5705ce5eb77e628c4d4058485
You can do it by using any of the below queries. For your reference visit the link below to test the output.
http://sqlfiddle.com/#!9/c5e9f3/1
The table created and the populated data is as below
create table tbl(ID int,letter varchar(2));
insert into tbl(ID,letter) values(1,'A');
insert into tbl(ID,letter) values(2,'B');
insert into tbl(ID,letter) values(3,'A');
insert into tbl(ID,letter) values(4,'Z');
insert into tbl(ID,letter) values(5,'C');
and now the query
select * from tbl where letter in ('A','B');
or
select * from tbl where letter ='A' or letter='B';

Splitting string with '+' seperator into seperate rows and apply aggregation

The data is not static and group of characters separted by + can vary. I want all the characters separated by + to be in row wise and then apply aggregation on the top of it. I am using mysql 5.7.14 in windows.
suppose data is:
group val
a+b 10
a 5
b 6
b+d+c 12
d 13
c+d 12
the output should be like:
grp_item val
a 15
b 28
c 24
d 24
Like i said the MySQL query is complex..
The general idea is a MySQL number generator which generates 1 to 10000 so it supports 10000 separated values with the + sign in the group column.
And it does not matter what data is between the + signs.
Query
SELECT
Table1_unique_groups.`group`
, SUM(Table1.val)
FROM (
SELECT
DISTINCT
SUBSTRING_INDEX(SUBSTRING_INDEX(Table1.`group`, '+', number_generator.number), '+', -1) AS `group`
FROM (
SELECT
#row := #row + 1 AS number
FROM (
SELECT 0 UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9
) record_1
CROSS JOIN (
SELECT 0 UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9
) record_2
CROSS JOIN (
SELECT 0 UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9
) record_4
CROSS JOIN (
SELECT 0 UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9
) record_5
CROSS JOIN (
SELECT #row := 0
) AS init_user_params
) AS number_generator
CROSS JOIN
Table1
) AS Table1_unique_groups
INNER JOIN
Table1
ON
FIND_IN_SET(Table1_unique_groups.`group`, REPLACE(Table1.group, '+', ','))
GROUP BY
Table1_unique_groups.`group`
Result
| group | SUM(Table1.val) |
| ----- | --------------- |
| a | 15 |
| b | 28 |
| c | 24 |
| d | 37 |
DB Fiddle demo

MySQL: How to get records even when no record on the date?

I am getting number of visits every day for generating a chart. Even when there are zero records, I want to get the record with count 0.
I am planning to create a table which will contain every day, and when fetching - data will join with this table and get count of the records from visit table. Is there any other way to do the same in mySQL?
Visit Table with Sample Data
Date | ........
----------------------
01/11/2014 | --------
03/11/2014 | --------
I want results even for 02/11/2014 with count 0. If I group by date - I will get count only when records exists on a particular date.
I'll try to read in between lines of your question... Sort of game where I write the question and the answer :-/
You have a table (my_stats) holding two fields, one is the date (my_date) the other is a integer (my_counter).
By some mean, you will need a table holding a list of all dates you want to use in your output.
This could be done with a temp table... (but not all hosting solution will allow you this) the other is to build it up on the fly, using a view or a stored procedure.
Then you will LEFT JOIN this table/view/stored procedure/etc... to your table my_visits based on the date field.
This will output you all dates, and when there won't be a match in mour my_visits you'll have a NULL value. ( IFNULL(my_visits.my_counter, 0) will give you a 0 (zero) when there is no matching value.
inspiration:
Get a list of dates between two dates +
How to get list of dates between two dates in mysql select query and a nice solution here that needs no loops, procedures, or temp tables generate days from date range
Based on that last link, here we go...
first a sample table
DROP TABLE IF EXISTS `my_stats`;
CREATE TABLE IF NOT EXISTS `my_stats` (
`my_date` date NOT NULL,
`my_counter` int(11) NOT NULL
) ENGINE=MyISAM DEFAULT CHARSET=utf8mb4;
INSERT INTO `my_stats` (`my_date`, `my_counter`) VALUES
('2017-11-01', 2),
('2017-11-02', 3),
('2017-11-03', 5),
('2017-11-05', 3),
('2017-11-07', 7);
And now a working exemple BETWEEN '2017-11-01' AND '2017-11-09'
SELECT date_range.date AS the_date,
IFNULL(my_stats.my_counter, 0) AS the_counter
FROM (
SELECT a.date
FROM (
SELECT Curdate() - INTERVAL (a.a + (10 * b.a) + (100 * c.a)) day
AS date
FROM (
SELECT 0 AS a
UNION ALL
SELECT 1
UNION ALL
SELECT 2
UNION ALL
SELECT 3
UNION ALL
SELECT 4
UNION ALL
SELECT 5
UNION ALL
SELECT 6
UNION ALL
SELECT 7
UNION ALL
SELECT 8
UNION ALL
SELECT 9
) AS a
CROSS JOIN (
SELECT 0 AS a
UNION ALL
SELECT 1
UNION ALL
SELECT 2
UNION ALL
SELECT 3
UNION ALL
SELECT 4
UNION ALL
SELECT 5
UNION ALL
SELECT 6
UNION ALL
SELECT 7
UNION ALL
SELECT 8
UNION ALL
SELECT 9
) AS b
CROSS JOIN (
SELECT 0 AS a
UNION ALL
SELECT 1
UNION ALL
SELECT 2
UNION ALL
SELECT 3
UNION ALL
SELECT 4
UNION ALL
SELECT 5
UNION ALL
SELECT 6
UNION ALL
SELECT 7
UNION ALL
SELECT 8
UNION ALL
SELECT 9
) AS c
) AS a
WHERE a.date BETWEEN '2017-11-01' AND '2017-11-09'
) AS date_range
LEFT JOIN my_stats
ON( date_range.date = my_stats.my_date )
ORDER BY the_date ASC
Output
+------------+-------------+
| the_date | the_counter |
+------------+-------------+
| 2017-11-01 | 2 |
| 2017-11-02 | 3 |
| 2017-11-03 | 5 |
| 2017-11-04 | 0 |
| 2017-11-05 | 3 |
| 2017-11-06 | 0 |
| 2017-11-07 | 7 |
| 2017-11-08 | 0 |
| 2017-11-09 | 0 |
+------------+-------------+

MySQL return result by quantity per row

I need a query that requires me to return something like below.
data table
--------------------------------
product name | quantity
apple | 3
egg | 2
query expected result
--------------------------------
product name
apple
apple
apple
egg
egg
Thanks for feedback
UPDATE #1
ok my bad the question is unclear. I want to display my result by looping my product by quantity. Probably my expected result on quantity makes some confusion. Therefore, is that possible to loop my record by quantity in mysql?
If you really need this in SQL you can leverage a tally(numbers) table, which you can create and populate with values 1-100 like so
CREATE TABLE tally(n INT NOT NULL AUTO_INCREMENT PRIMARY KEY);
INSERT INTO tally (n)
SELECT a.N + b.N * 10 + 1 n
FROM
(SELECT 0 AS N UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) a
,(SELECT 0 AS N UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) b
ORDER BY n;
Then your query would look
SELECT product_name
FROM data_table d JOIN tally t
ON t.n <= d.quantity
Note: Make sure that you have MAX(n) value in your tally table >= MAX(quantity) in your data_table for this to work properly
Output:
+--------------+
| product_name |
+--------------+
| apple |
| apple |
| apple |
| egg |
| egg |
+--------------+
Here is SQLFiddle demo

Get value even if it doesn't exist in table from SQL SELECT statement

I have a MySQL table that looks something like this:
|---ID---|---COUNTER---|
| 1 | 2 |
| 2 | 6 |
| 3 | 1 |
| 5 | 9 |
| 6 | 10 |
I'm looking for a SELECT statement that returns ID's and their COUNTER. The table only have ID's such as: 1,2,3,5,6. Is there a statement where you say: I want ID's 1 to 10 even if they doesn't exist in the table, and if the ID doesn't exist, return the ID anyway with the COUNTER value 0. For example:
|---ID---|---COUNTER---|
| 1 | 2 |
| 2 | 6 |
| 3 | 1 |
| 4 | 0 |
| 5 | 9 |
| 6 | 10 |
| 7 | 0 |
| 8 | 0 |
| 9 | 0 |
| 10 | 0 |
Do I have to create a SELECT statement that contains NOT EXIST parameters?
Thanks in advance, Steve-O
Without creating a temp table:
select t.num as id, coalesce(yt.counter, 0)
from your_table yt
right join (
select 1 as num union select 2 union select 3 union select 4 union select 5 union
select 6 union select 7 union select 8 union select 9 union select 10
) t on yt.id = t.num
order by t.num
and bit more general:
select t.num as id, coalesce(yt.counter, 0)
from your_table yt
right join (
select t1.num + t2.num * 10 + t3.num * 100 as num
from (
select 1 as num union select 2 union select 3 union select 4 union select 5 union
select 6 union select 7 union select 8 union select 9 union select 0
) t1
cross join (
select 1 as num union select 2 union select 3 union select 4 union select 5 union
select 6 union select 7 union select 8 union select 9 union select 0
) t2
cross join (
select 1 as num union select 2 union select 3 union select 4 union select 5 union
select 6 union select 7 union select 8 union select 9 union select 0
) t3
) t on yt.id = t.num
where t.num between (select min(id) from your_table) and (select max(id) from your_table)
You can define limit by yourself here I've used min and max of id value from your_table.
It's not very robust, but if you created a temporary table with the ID's you wanted in it, you could then left join to your table containing ID and Counter which would include all the values:
Declare #tempidtable as table ( imaginaryid int )
insert into #tempidtable ( imaginaryid ) values ( 1 )
insert into #tempidtable ( imaginaryid ) values ( 2 )
insert into #tempidtable ( imaginaryid ) values ( 3 )
select
#temptable.imaginaryid,
ISNULL(yourothertable.counter, 0)
from #tempidtable
left join yourothertable
on #tempidtable.imaginaryid = yourothertable.id
As Tomek says you could loop over the inserts to make it easier to maintain, or possible store the ids you want as a base in another table, using this as the basis for the join rather than a temp table.
Create a table with all possible ID's:
create table Numbers (nr int primary key);
declare i int default 1;
while i < 100000 do
insert into Numbers (nr) values (i);
set i = i + 1;
end while;
Then you can use left join to return all numbers:
select n.NR
, c.Counter
from Numbers n
left join
Counters c
on c.ID = n.NR
You can use left join to solve your issue. Read more about left join here
I think you will have to create (generate in loop) temporary table with the complete sequence of numbers from 1 to N (where N is the MAX(Id) of counted table). Then do left join to that table and apply GROUP BY clause.
You need the range of integers to do an outer join with your table based on ID. Generating a range of integers is dependent on the SQL vendor if you do not want to use a temporary table. See SQL SELECT to get the first N positive integers for hints on how to do this based on your SQL vendor.