How to get all missing numbers from id column - mysql

Users table has id field in mysql database, and table has below record in id field
|id|
2
3
5
6
9
10
15
16
18
21
25
I want to get missing records row like below:-
1
4
7
8
11
12
13
14
17
19
20
22
23
24


How cool is this ?
SELECT seq.`id` FROM (
SELECT #row := #row + 1 as `id`
FROM `users` t, (SELECT #row := 0) r
CROSS JOIN `users` t2
) as seq WHERE seq.`id` NOT IN (SELECT `id` FROM `users`)
AND seq.`id` <= (SELECT max(`id`) from `users`);
If you want the missing ids for analytical purpose, should love this one:
SELECT CONCAT(dt.missing, IF(dt.`found`-1 > dt.missing, CONCAT(' to ', dt.`found` - 1), '')) AS missing
FROM ( SELECT #rownum:=#rownum+1 AS `missing`,
IF (#rownum=id, 0, #rownum:=id) AS `found`
FROM ( SELECT #rownum:=0 ) AS r
JOIN users
ORDER BY id ) AS dt
WHERE dt.`found`!= 0;


Use a calendar table approach and anti-join approach:
CREATE TABLE nums (id INT);
INSERT INTO nums (id)
(
SELECT (t*10+u+1) x FROM
(SELECT 0 t UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION
SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9) A,
(SELECT 0 u UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION
SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9) B
);
SELECT
n.id
FROM nums n
LEFT JOIN users u
ON n.id = u.id
WHERE
u.id IS NULL
ORDER BY
n.id;
Demo
Note that in the demo and query I did not limit the height of the range of missing numbers which gets returned. I arbitrarily used a sequence of numbers 1..100, but you may restrict to any size you want.

Related

Determine missing number in the following sequence of odd number - 1,3,7,9,11,13

Table: Sequence
id
1
3
7
9
11
13
What I tried is :
select l.id + 1 as start
from sequence as l
left outer join sequence as r on l.id + 1 = r.id
where r.id is NULL;
This does not capture all required numbers but only the +1s from the number available in the initial table.
My required output is :
2
4
5
6
8
10
12

Using a calendar table approach:
WITH nums AS (
SELECT 1 AS id UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL
SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL
SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9 UNION ALL
SELECT 10 UNION ALL SELECT 11 UNION ALL SELECT 12 UNION ALL
SELECT 13 UNION ALL SELECT 14 UNION ALL SELECT 15
)
SELECT n.id
FROM nums n
LEFT JOIN Sequence s
ON s.id = n.id
WHERE
s.id IS NULL;
On earlier versions of MySQL, which don't support CTE, you may either create a bona fide nums sequence table, or just inline the CTE:
SELECT n.id
FROM (
SELECT 1 AS id UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL
SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL
SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9 UNION ALL
SELECT 10 UNION ALL SELECT 11 UNION ALL SELECT 12 UNION ALL
SELECT 13 UNION ALL SELECT 14 UNION ALL SELECT 15
) n
LEFT JOIN Sequence s
ON s.id = n.id
WHERE
s.id IS NULL;

mysql select odd number values in a column that includes delimiter separated values

My next database table will be set up more optimally. Unfortunately this one was already set up where one column [data] contains checkbox array values that were saved the following way:
value 1|~|value 1 value 2|~|value 2 value 3|~|value 3
Not optimal, I know.
What I need is a mysql query that select only the values in [data] column in front of the |~|. Basically think I need to select the only odd values.
Any help pointing me in the right direction is greatly appreciated. I tried an if statement in a query and it did not work. Of course I deleted that by mistake.

What I need is a mysql query that select only the
values in [data] column in front of the |~|.
One thing to note the numbers before |~| must be unique.
It will not show the same number twice.
Query
SELECT
DISTINCT
SUBSTRING (
record_data.column
, LOCATE('|~|', record_data.`column` , number_generator.number) - 1
, 1
) AS number
FROM (
SELECT
#row := #row + 1 AS number
FROM (
SELECT 0 UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9
) record_1
CROSS JOIN (
SELECT 0 UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9
) record_2
CROSS JOIN (
SELECT 0 UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9
) record_4
CROSS JOIN (
SELECT 0 UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9
) record_5
CROSS JOIN (
SELECT #row := 0
) AS init_user_params
) AS number_generator
CROSS JOIN (
SELECT
*
FROM (
SELECT 'value 1|~|value 1 value 2|~|value 2 value 3|~|value 3' AS `column`
) AS record
) AS record_data
WHERE
LOCATE('|~|', record_data.`column` , number_generator.number) <> 0
Result
| number |
| ------ |
| 1 |
| 2 |
| 3 |
demo

Get integer which is not in database

I have this MySQL table :
create table example (
My_Id INT UNSIGNED AUTO_INCREMENT PRIMARY KEY,
My_Integer INT UNSIGNED DEFAULT 0
) ENGINE=InnoDB
Suppose this table to be populated like that :
SELECT * FROM `example`
My_Id My_Integer
1 10
2 1
3 3
4 152
5 16
6 481
7 128
My goal is to retrieve, from an user input, which integer is not in my table 'example'.
For example, if the user enter 1, 3, 10, the result should be empty.
If the user enter 1, 2, 3, 10, the result should be "2".
I can't find a way to do that, other than to do one MySQL request per integer in order to know if the number is in the table 'example'.
Do someone know a way ?

Select numberToMatch
from example as a
right outer join (
select 1 as numberToMatch union all
select 2 union all
select 3 union all
select 10
) as b
on b.numberToMatch = a.My_Integer
where a.My_Integer is null
Using right outer join and select those null record in the left table.

You could genereate tally table and use OUTER JOIN:
SELECT t.n
FROM (
SELECT a.N + b.N * 10 + c.N * 100 + 1 n
FROM (SELECT 0 AS N UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3
UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6
UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) a
,(SELECT 0 AS N UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3
UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6
UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) b
,(SELECT 0 AS N UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3
UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6
UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) c
) t
LEFT JOIN example e
ON t.n = e.My_Integer
WHERE t.n IN (1,2,3,10) -- here goes numbers
AND e.My_Integer IS NULL;
LiveDemo
Output:
╔═══╗
║ n ║
╠═══╣
║ 2 ║
╚═══╝
It will work in range 1-999. You could expand it if needed.
Another way is to use UNION ALL:
SELECT s.col
FROM (SELECT 1 AS col
UNION ALL SELECT 2
UNION ALL SELECT 3
UNION ALL SELECT 10) AS s
LEFT JOIN example e
ON s.col = e.My_Integer
WHERE e.My_Integer IS NULL;
LiveDemo2

(i) Create a table from existing one and (ii) display detailed data for which total is stored in table

From a table t1
product_id | quantity | total_amount
I want to create a table t2 with the columns
id | product_id | total_amount/quantity AS product_price
I want each record (t2.product_id, t2.product_price) to be shown t1.quantity times for the respective product:
1 | 400
1 | 400
1 | 400
2 | 75
2 | 75
Could you please tell me, how can I do this using bare SQL?

This seems to create the table, you are looking for:
CREATE TABLE T2 AS
SELECT
#rowid := #rowid + 1 AS id
, product_id
, total_amount / quantity AS product_price
FROM t1, (SELECT #rowid := 0) AS firstID
ORDER BY product_id
;
For the second part of your request, you could try:
SELECT
T.product_id
, T.total_amount / T.quantity AS price
FROM T1 T CROSS JOIN
(SELECT A.n + B.n * 10 + 1 n
FROM
(SELECT 0 AS n UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3
UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6
UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) A
, (SELECT 0 AS n UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3
UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6
UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) B
ORDER BY n
) N
WHERE N.n <= T.quantity
ORDER BY product_id
;
See it in action: SQL Fiddle.
Please comment if this requires adjustment / further detail.

How to INSERT INTO a range of numbers into a MYSQL table [duplicate]

How do I generate a range of consecutive numbers (one per line) from a MySQL query so that I can insert them into a table?
For example:
nr
1
2
3
4
5
I would like to use only MySQL for this (not PHP or other languages).

Here is one way to do it set-based without loops. This can also be made into a view for re-use. The example shows the generation of a sequence from 0 through 999, but of course, it may be modified to suit.
INSERT INTO
myTable
(
nr
)
SELECT
SEQ.SeqValue
FROM
(
SELECT
(HUNDREDS.SeqValue + TENS.SeqValue + ONES.SeqValue) SeqValue
FROM
(
SELECT 0 SeqValue
UNION ALL
SELECT 1 SeqValue
UNION ALL
SELECT 2 SeqValue
UNION ALL
SELECT 3 SeqValue
UNION ALL
SELECT 4 SeqValue
UNION ALL
SELECT 5 SeqValue
UNION ALL
SELECT 6 SeqValue
UNION ALL
SELECT 7 SeqValue
UNION ALL
SELECT 8 SeqValue
UNION ALL
SELECT 9 SeqValue
) ONES
CROSS JOIN
(
SELECT 0 SeqValue
UNION ALL
SELECT 10 SeqValue
UNION ALL
SELECT 20 SeqValue
UNION ALL
SELECT 30 SeqValue
UNION ALL
SELECT 40 SeqValue
UNION ALL
SELECT 50 SeqValue
UNION ALL
SELECT 60 SeqValue
UNION ALL
SELECT 70 SeqValue
UNION ALL
SELECT 80 SeqValue
UNION ALL
SELECT 90 SeqValue
) TENS
CROSS JOIN
(
SELECT 0 SeqValue
UNION ALL
SELECT 100 SeqValue
UNION ALL
SELECT 200 SeqValue
UNION ALL
SELECT 300 SeqValue
UNION ALL
SELECT 400 SeqValue
UNION ALL
SELECT 500 SeqValue
UNION ALL
SELECT 600 SeqValue
UNION ALL
SELECT 700 SeqValue
UNION ALL
SELECT 800 SeqValue
UNION ALL
SELECT 900 SeqValue
) HUNDREDS
) SEQ

Here's a hardware engineer's version of Pittsburgh DBA's solution:
SELECT
(TWO_1.SeqValue + TWO_2.SeqValue + TWO_4.SeqValue + TWO_8.SeqValue + TWO_16.SeqValue) SeqValue
FROM
(SELECT 0 SeqValue UNION ALL SELECT 1 SeqValue) TWO_1
CROSS JOIN (SELECT 0 SeqValue UNION ALL SELECT 2 SeqValue) TWO_2
CROSS JOIN (SELECT 0 SeqValue UNION ALL SELECT 4 SeqValue) TWO_4
CROSS JOIN (SELECT 0 SeqValue UNION ALL SELECT 8 SeqValue) TWO_8
CROSS JOIN (SELECT 0 SeqValue UNION ALL SELECT 16 SeqValue) TWO_16;

If you need the records in a table and you want to avoid concurrency issues, here's how to do it.
First you create a table in which to store your records
CREATE TABLE `incr` (
`Id` int(11) NOT NULL auto_increment,
PRIMARY KEY (`Id`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8;
Secondly create a stored procedure like this:
DELIMITER ;;
CREATE PROCEDURE dowhile()
BEGIN
DECLARE v1 INT DEFAULT 5;
WHILE v1 > 0 DO
INSERT incr VALUES (NULL);
SET v1 = v1 - 1;
END WHILE;
END;;
DELIMITER ;
Lastly call the SP:
CALL dowhile();
SELECT * FROM incr;
Result
Id
1
2
3
4
5

Let's say you want to insert numbers 1 through 100 into your table. As long as you have some other table that has at least that many rows (doesn't matter the content of the table), then this is my preferred method:
INSERT INTO pivot100
SELECT #ROW := #ROW + 1 AS ROW
FROM someOtherTable t
join (SELECT #ROW := 0) t2
LIMIT 100
;
Want a range that starts with something other than 1? Just change what #ROW gets set to on the join.

As you all understand, this is rather hacky so use with care
SELECT
id % 12 + 1 as one_to_twelve
FROM
any_large_table
GROUP BY
one_to_twelve
;

DECLARE i INT DEFAULT 0;
WHILE i < 6 DO
/* insert into table... */
SET i = i + 1;
END WHILE;

Very similar to the accepted response, but using the new WITH syntax for mysql >= 8.0 which makes a lot more legible and the intent is also clearer
WITH DIGITS (N) AS (
SELECT 0 UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL
SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL
SELECT 8 UNION ALL SELECT 9)
SELECT
UNITS.N + TENS.N*10 + HUNDREDS.N*100 + THOUSANDS.N*1000
FROM
DIGITS AS UNITS, DIGITS AS TENS, DIGITS AS HUNDREDS, DIGITS AS THOUSANDS;

In MariaDB you can do:
SELECT * FROM seq_i_to_N
For example:
SELECT * FROM seq_0_to_1000
SELECT * FROM seq_1_to_1000000
Reference: https://www.percona.com/blog/2020/07/27/generating-numeric-sequences-in-mysql/

The "shortest" way i know (in MySQL) to create a table with a long sequence is to (cross) join an existing table with itself. Since any (common) MySQL server has the information_schema.COLUMNS table i would use it:
DROP TABLE IF EXISTS seq;
CREATE TABLE seq (i MEDIUMINT AUTO_INCREMENT PRIMARY KEY)
SELECT NULL AS i
FROM information_schema.COLUMNS t1
JOIN information_schema.COLUMNS t2
JOIN information_schema.COLUMNS t3
LIMIT 100000; -- <- set your limit here
Usually one join should be enough to create over 1M rows - But one more join will not hurt :-) - Just don't forget to set a limit.
If you want to include 0, you should "remove" the AUTO_INCEMENT property.
ALTER TABLE seq ALTER i DROP DEFAULT;
ALTER TABLE seq MODIFY i MEDIUMINT;
Now you can insert 0
INSERT INTO seq (i) VALUES (0);
and negative numbers as well
INSERT INTO seq (i) SELECT -i FROM seq WHERE i <> 0;
You can validate the numbers with
SELECT MIN(i), MAX(i), COUNT(*) FROM seq;

All other answers are good, however they all have speed issues for larger ranges because they force MySQL to generate every number then filter them.
The following only makes MySQL generate the numbers that are needed, and therefore is faster:
set #amount = 55; # How many numbers from zero you want to generate
select `t0`.`i`+`t1`.`i`+`t2`.`i`+`t3`.`i` as `offset`
from
(select 0 `i` union select 1 union select 2 union select 3 union select 4 union select 5 union select 6 union select 7 union select 8 union select 9) `t0`,
(select 0 `i` union select 10 union select 20 union select 30 union select 40 union select 50 union select 60 union select 70 union select 80 union select 90) `t1`,
(select 0 `i` union select 100 union select 200 union select 300 union select 400 union select 500 union select 600 union select 700 union select 800 union select 900) `t2`,
(select 0 `i` union select 1000 union select 2000 union select 3000 union select 4000 union select 5000 union select 6000 union select 7000 union select 8000 union select 9000) `t3`
where `t3`.`i`<#amount
and `t2`.`i`<#amount
and `t1`.`i`<#amount
and `t0`.`i`+`t1`.`i`+`t2`.`i`+`t3`.`i`<#amount;
With the above you can generate upto 10,000 numbers (0 to 9,999) with no slower speed overhead for lower numbers, regardless how low they are.

Here's a way to do it with json_table if you have MySql 8 and above:
set #noRows = 100;
SELECT tt.rowid - 1 AS value
FROM JSON_TABLE(CONCAT('[{}', REPEAT(',{}', #noRows - 1), ']'),
"$[*]" COLUMNS(rowid FOR ORDINALITY)
) AS tt;
(h/t - https://www.percona.com/blog/2020/07/27/generating-numeric-sequences-in-mysql/)

The idea I want to share is not a precise response for the question but can be useful for some so I would like to share it.
If you frequently need only a limited set of numbers then it can be beneficial to create a table with the numbers you may need and just use that table every time. For example:
CREATE TABLE _numbers (num int);
INSERT _numbers VALUES (0), (1), (2), (3), ...;
This can be applied only if you need numbers below a certain reasonable limit, so don't use it for generating sequence 1...1 million but can be used for numbers 1...10k, for example.
If you have this list of numbers in the _numbers table then you can write queries like this, for obtaining the individual characters of a string:
SELECT number, substr(name, num, 1)
FROM users
JOIN _numbers ON num < length(name)
WHERE user_id = 1234
ORDER BY num;
If you need larger numbers than 10k then you can join the table to itself:
SELECT n1.num * 10000 + n2.num
FROM _numbers n1
JOIN _numbers n2
WHERE n1 < 100
ORDER BY n1.num * 10000 + n2.num; -- or just ORDER BY 1 meaning the first column

This is based on a previous answer (https://stackoverflow.com/a/53125278/2009581), but is compatible with MySQL 5.7. It works for replicas and read-only users:
SELECT x1.N + x10.N*10 + x100.N*100 + x1000.N*1000
FROM (SELECT 0 AS N UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) x1,
(SELECT 0 AS N UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) x10,
(SELECT 0 AS N UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) x100,
(SELECT 0 AS N UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) x1000
WHERE x1.N + x10.N*10 + x100.N*100 + x1000.N*1000 <= #max;
It generates integers in the range of [0, #max].

with recursive cte..
with recursive rnums as (
select 1 as n
union all
select n+1 as n from rnums
where n <10
)
select * from rnums
;
Result would be..
+------+
| n |
+------+
| 1 |
| 2 |
| 3 |
| 4 |
| 5 |
| 6 |
| 7 |
| 8 |
| 9 |
| 10 |
+------+
10 rows in set (0.00 sec)