how find "holes" in auto_increment column? - mysql

when I DELETE, as example, the id 3, I have this:
id | name
1 |
2 |
4 |
5 |
...
now, I want to search for the missing id(s), because i want to fill the id again with:
INSERT INTO xx (id,...) VALUES (3,...)
is there a way to search for "holes" in the auto_increment index?
thanks!

You can find the top value of gaps like this:
select t1.id - 1 as missing_id
from mytable t1
left join mytable t2 on t2.id = t1.id - 1
where t2.id is null

The purpose of AUTO_INCREMENT is to generate simple unique and meaningless identifiers for your rows. As soon as you plan to re-use those IDs, they're no longer unique (not at least in time) so I have the impression that you are not using the right tool for the job. If you decide to get rid of AUTO_INCREMENT, you can do all your inserts with the same algorithm.
As about the SQL code, this query will match existing rows with the rows that has the next ID:
SELECT a.foo_id, b.foo_id
FROM foo a
LEFT JOIN foo b ON a.foo_id=b.foo_id-1
E.g.:
1 NULL
4 NULL
10 NULL
12 NULL
17 NULL
19 20
20 NULL
24 25
25 26
26 27
27 NULL
So it's easy to filter out rows and get the first gap:
SELECT MIN(a.foo_id)+1 AS next_id
FROM foo a
LEFT JOIN foo b ON a.foo_id=b.foo_id-1
WHERE b.foo_id IS NULL
Take this as a starting point because it still needs some tweaking:
You need to consider the case where the lowest available number is the lowest possible one.
You need to lock the table to handle concurrent inserts.
In my computer it's slow as hell with big tables.

I think the only way you can do this is with a loop:
Any other solutions wont show gaps bigger than 1:
insert into XX values (1)
insert into XX values (2)
insert into XX values (4)
insert into XX values (5)
insert into XX values (10)
declare #min int
declare #max int
select #min=MIN(ID) from xx
select #max=MAX(ID) from xx
while #min<#max begin
if not exists(select 1 from XX where id = #min+1) BEGIN
print 'GAP: '+ cast(#min +1 as varchar(10))
END
set #min=#min+1
end
result:
GAP: 3
GAP: 6
GAP: 7
GAP: 8
GAP: 9

First, I agree with the comments that you shouldn't try filling in holes. You won't be able to find all the holes with a single SQL statement. You'll have to loop through all possible numbers starting with 1 until you find a hole. You could write a sql function to do this for you that could then be used in a function. So if you wrote a function called find_first_hole you could then call it in an insert like:
INSERT INTO xx (id, ...) VALUES (find_first_hole(), ...)

This is a gaps&island problem, see my (and other) replies here and here. In most cases, gaps&islands problems are most elegantly solved using recursive CTE's, which are not available in mysql.

Related

Only update a single row in a query based on non-unique parameters?

I have a big table with duplicate keys that I am trying to connect to smaller table that has unique keys. I know for a fact there will not matches for everything. I only want a match from my smaller table to update a single row in the bigger table and then to move onto the next smaller table row for the next update. I need it like this because I am trying to create unique id's in the larger table as each row represents a real world product which has it's own heiarchy of real world objects.
So for example,
bigtable
barcodeSnippet t_stamp workId parentCase newId
aaaa time1 1 1 NULL
aaaa time1 1 1 NULL
aaaa time1 1 1 NULL
and my small table might have this
smalltable
id barcodeSnippet t_stamp workId parentCase
1 aaaa time1 1 1
2 aaaa time1 1 1
the end result I want in my bigtable is
bigtable
barcodeSnippet t_stamp workId parentCase newId
aaaa time1 1 1 1
aaaa time1 1 1 2
aaaa time1 1 1 NULL
where I only mached once per row, and was left over with a NULL since I had 3 rows in the big table and two matches in my smaller one.
My current query
UPDATE bigtable as bt
JOIN smallTable as st ON (bt.barcodeSnippet = b.barcodeSnippet AND
bt.parentCase= st.parentCase and bt.t_stamp = st.t_stamp and bt.workId =
st.workId)
SET bt.bottlesId = st.id;
does not work, and I don't see it's possible to use the LIMIT in a UPDATE for MySQL. I have seen other answers in MS SQL where you can use TOP 1, perhaps where newId IS NULL, but again I am using MySQL here.
I am thinking I might need to use a Stored Procedure/Cursor approach but even with that it seems like I will run into the issue of having to run an update statement and then I am back at square 1.
Any ideas? Using MySQL 5.6.
EDIT: Think I have a decent solution. I just updated with my query so I do have duplicates. However, now I added a row number column. I plan to join the table on itself and update it if the row number is < the row number, therefore I keep the top ID and can turn the others to null, which is suitable.
Something like this
UPDATE bigtable tb
JOIN bigtable tb2 ON tb.newId = tb2.newId
SET tb.newId = NULL
WHERE tb.rowNumber < tb2.rowNumber;
You use the auto_incremented id column with the superglobal $_GET:
1.php
<?php $var = $row["id"]; ?>
Something
Then in your 2.php
<?php $id = $_GET["id"];
$sql = "SELECT * FROM table WHERE id='$id';";
Of course you need to use prepared statements

Finding count of unique value before a character

I have a some entries in database table rows as follows.
101 - 1
101 - 2
101 - 3
102 - 1
102 - 2
102 - 3
103
I need to get the result of SELECT Query for count as '3' since there are 101 and 102 are the only number before the -.
So is there any way to find the unique value in db table columns before a character?
EDIT : I have entries even without the - .
In case your entries have always the format you have provided us, you just have to find the position of the '-' character, split the values, get the first n characters and count the distinct values
This works for SQL Server, otherwise informs us about what DBMS you are using or replace the functions with the ones of your DBMS on your own
SELECT COUNT(DISTINCT SUBSTRING(val,0,CHARINDEX('-', val))) from YourTable
create table T1
(
id int primary key identity,
col1 varchar(20)
)
insert into T1 values('101 - 1'),('101 - 2'),('101 - 3'),('102 - 1'),('102 - 2'),('102 - 3')
select SUBSTRING(col1,0,CHARINDEX(' ',col1)) as 'Value',count(*) as 'Count' from T1 group by SUBSTRING(col1,0,CHARINDEX(' ',col1))

MySql, Without loop filling a table with semi/random data

How to have this code/output in MySql:
Had a recursive cte in MSSQL to fill a table with random data without loop e.g begin/end. Searched for similar logic in MySql but most or all solutions were using begin/end or for loops. Wonder if you could suggest a solution without loop in MySql.
Thanks
--MSSQL cte:------------------------------------
with t1( idi,val ) as
(
select
idi=1
,val=cast( 1 as real)
union all
select
idi=idi+1
,val=cast(val+rand() as real)
from t1
where idi<5
)
select idi,val from t1
-----------------------------------------------
Output in MSSQL:( semi random values)
idi | val
-------------
1 | 1
2 | 1.11
3 | 1.23
4 | 1.35
5 | 1.46
Edit:
Regarding discussions which considers set based codes as loop based codes indeed, I could understand this but just out of interest gave it a try in MSSQL 2008r2, here is the result:
1- above code with 32000 recursion took 2.812 sec
2- above output created with WHILE BEGIN END loop for 32000 took 53.640 sec
Obviously this is a big difference in execution time.
Here is the loop based code:
insert into #t1(idi,val)
select
idi=1
,val=1
declare #ii int = 2
while #ii<32000
begin
insert into #t1(idi,val)
select
idi=idi+1
,val=val+rand()
from #t1
where idi=#ii-1
set #ii=#ii+1
end
select * from #t1
MySql doesn't support CTE.
You need a procedure or some tricky queries like this one:
set #id=0;
set #val=0;
SELECT #id:=#id+1 As id,
#val:=#val+rand() As val
FROM information_schema.tables x
CROSS JOIN information_schema.tables y
LIMIT 10

MySQL: Select from list of values

I was wondering if I could select given values from a list and populate rows? For example, SELECT 1 as one, 2 as two, 3 as three will populate columns:
one | two | three
------------------------
1 | 2 | 3
I'm looking for a script that populates rows, something like:
values
-------
1
2
3
4
Thanks!
you can union each one if you want like so
SELECT 1 AS numbers
UNION SELECT 2
UNION SELECT 3
a much simpler way to do something like this would be to make a table with an auto incremented id... insert into another column in the table an empty string... then just select the auto incremented id
CREATE TEMPORARY TABLE tmp (
id INTEGER NOT NULL AUTO_INCREMENT PRIMARY KEY,
val varchar(1)
);
INSERT INTO tmp (val)
values
(""),
(""),
(""),
(""),
(""),
(""),
(""),
(""),
(""),
("");
select id from tmp;
DEMO
To get a few numbers, the approach from John Ruddell is the probably the most convenient, I can easily incorporate an inline view in any query I'm needing to run.
When I need a lot of numbers, for example, 1 through 4000, I can do something like this:
CREATE TABLE digit (d INT(11) NOT NULL PRIMARY KEY);
INSERT INTO digit (d) VALUES (0),(1),(2),(3),(4),(5),(6),(7),(8),(9);
SELECT thousands.d*1000+hundreds.d*100+tens.d*10+ones.d+1 AS n
FROM digit ones
CROSS
JOIN digit tens
CROSS
JOIN digit hundreds
CROSS
JOIN digit thousands
WHERE thousands.d < 4
I can also add a HAVING clause if the boundaries of the numbers I need aren't quite as neat, e.g
HAVING n >= 121
AND n <= 2499
If I want to ensure the "numbers" are returned in order, I'll add an ORDER BY clause:
ORDER BY n

Efficient way to remove successive duplicate rows in MySQL

I have a table with columns like (PROPERTY_ID, GPSTIME, STATION_ID, PROPERTY_TYPE, VALUE) where PROPERTY_ID is primary key and STATION_ID is foreign key.
This table records state changes; each row represents property value of some station at given time. However, its data was converted from old table where each property was a column (like (STATION_ID, GPSTIME, PROPERTY1, PROPERTY2, PROPERTY3, ...)). Because usually only one property changed at time I have lots of duplicates.
I need to remove all successive rows with same values.
Example. Old table contained values like
time stn prop1 prop2
100 7 red large
101 7 red small
102 7 blue small
103 7 red small
The converted table is
(order by time,type) (order by type,time)
time stn type value time stn type value
100 7 1 red 100 7 1 red
100 7 2 large 101 7 1 red
101 7 1 red 102 7 1 blue
101 7 2 small 103 7 1 red
102 7 1 blue 100 7 2 large
102 7 2 small 101 7 2 small
103 7 1 red 102 7 2 small
103 7 2 small 103 7 2 small
should be changed to
time stn type value
100 7 1 red
100 7 2 large
101 7 2 small
102 7 1 blue
103 7 1 red
The table contains about 22 mln rows.
My current approach is to use procedure to iterate over the table and remove duplicates:
BEGIN
DECLARE done INT DEFAULT FALSE;
DECLARE id INT;
DECLARE psid,nsid INT DEFAULT null;
DECLARE ptype,ntype INT DEFAULT null;
DECLARE pvalue,nvalue VARCHAR(50) DEFAULT null;
DECLARE cur CURSOR FOR
SELECT station_property_id,station_id,property_type,value
FROM station_property
ORDER BY station_id,property_type,gpstime;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = TRUE;
OPEN cur;
read_loop: LOOP
FETCH cur INTO id,nsid,ntype,nvalue;
IF done THEN
LEAVE read_loop;
END IF;
IF (psid = nsid and ptype = ntype and pvalue = nvalue) THEN
delete from station_property where station_property_id=id;
END IF;
SET psid = nsid;
SET ptype = ntype;
SET pvalue = nvalue;
END LOOP;
CLOSE cur;
END
However, it is too slow. On test table with 20000 rows it removes 10000 duplicates for 6 minutes. Is there a way to optimize the procedure?
P.S. I still have my old table intact, so maybe it is better to try and convert it without the duplicates rather than dealing with duplicates after conversion.
UPDATE.
To clarify which duplicates I want to allow and which not.
If a property changes, then changes back, I want all 3 records to be saved, even though first and the last contains same station_id, type, and value.
If there are several successive (by GPSTIME) records with same station_id, type, and value, I want only the first one (which represents the change to that value) to be saved.
In short, a -> b -> b -> a -> a should be optimized to a -> b -> a.
SOLUTION
As #Kickstart suggested, I've created new table, populated with filtered data. To refer previous rows, I've used approach similar to one used in this question.
rename table station_property to station_property_old;
create table station_property like station_property_old;
set #lastsid=-1;
set #lasttype=-1;
set #lastvalue='';
INSERT INTO station_property(station_id,gpstime,property_type,value)
select newsid as station_id,gpstime,newtype as type,newvalue as value from
-- this subquery adds columns with previous values
(select station_property_id,gpstime,#lastsid as lastsid,#lastsid:=station_id as newsid,
#lasttype as lasttype,#lasttype:=property_type as newtype,
#lastvalue as lastvalue,#lastvalue:=value as newvalue
from station_property_old
order by newsid,newtype,gpstime) sub
-- we filter the data, removing unnecessary duplicates
where lastvalue != newvalue or lastsid != newsid or lasttype != newtype;
drop table station_property_old;
Possibly create a new table, populated with a select from the existing table using a GROUP BY. Something like this (not tested so excuse any typos):-
INSERT INTO station_property_new
SELECT station_property_id, station_id, property_type, value
FROM (SELECT station_property_id, station_id, property_type, value, COUNT(*) FROM station_property GROUP BY station_property_id, station_id, property_type, value) Sub1
Regarding chainging properties, cant you put a unique constraint to ensure the combination of station/type/value columns is unique. That way you will not be able to change it to a value which will result in a duplication.