Select previous data entry by datetime - mysql

I have a mysql database with a table with performance data. Each data table entry is for a specific counter.
DATA
----
dat_date BIGINT
value BIGINT
dat_date DATETIME(3)
FK_dt_id BIGINT
For each entries I need to compute the speed and acceleration. So I want to create a SELECT that will select a row with it PREVIOUS row.
FK_dt_id is a foreigh key on a counter type
I've try this :
SELECT d1.dat_date,
d1.value v1,
d1.PK_dat_id
FROM data d1
INNER JOIN (SELECT * FROM data d2 ORDER BY d2.dat_date) d2
ON (d2.dat_date < d1.dat_date
AND d2.FK_dt_id = d1.FK_dt_id)
It works but I can't get the previous primary key, only it's value.
Any ideas?

The fastest (not necesarily the easiest) way to do things like this (with "pure" MySQL syntax) is with a little trick using user variables:
select #pk_data_id as prev_pk_data_id
, #pk_data_id := (case
when #fk_counter_id = fk_counter_id then a.pk_data_id
else 0
end) as pk_data_id
, #fk_counter_id := a.fk_counter_id as fk_counter_id
from
(select #pk_data_id := 0, #fk_counter_id := 0) as init
, data as a
order by a.fk_counter_id, a.pk_data_id
Once you have this rowset, you can join it with your data table.
I'd put this result in a temporary table and use it later; something like this:
drop table if exists temp_tbl;
create temporary table temp_tbl
select #pk_data_id as prev_pk_data_id
, #pk_data_id := (case
when #fk_counter_id = fk_counter_id then a.pk_data_id
else 0
end) as pk_data_id
, #fk_counter_id := a.fk_counter_id as fk_counter_id
from
(select #pk_data_id := 0, #fk_counter_id := 0) as init
, data as a
order by a.fk_counter_id, a.pk_data_id;
alter table temp_tbl
add index dId (pk_data_id),
add index pdId (prev_pk_data_id),
add index cId (fk_counter_id);
-- Now use the temp table to get what you need
select d1.*
, d2.pk_data_id as prev_pk_data_id
, d2.data_value as prev_data_value
, d2.data_datetime as prev_data_datetime
from data as d1
-- If you don't use the temp table, substitute 'temp_tbl' with
-- the query from above
inner join temp_tbl as a
on d1.pk_data_id = a.pk_data_id
and d1.fk_counter_id = a.fk_counter_id
left join data as d2
on a.prev_pk_data_id = d2.pk_data_id
and a.fk_counter_id = d2.fk_counter_id

Related

Mysql - Accumulatively count the total on a row by row basis

I'm trying in MySql to count the number of users created each day and then get an accumulative figure on a row by row basis. I have followed other suggestions on here, but I cannot seem to get the accumulation to be correct.
The problem is that it keeps counting from the base number of 200 and not taking account of previous rows.
Where was I would expect it to return
My Sql is as follows;
SELECT day(created_at), count(*), (#something := #something+count(*)) as value
FROM myTable
CROSS JOIN (SELECT #something := 200) r
GROUP BY day(created_at);
To create the table and populate it you can use;
CREATE TABLE myTable (
id INT AUTO_INCREMENT,
created_at DATETIME,
PRIMARY KEY (id)
);
INSERT INTO myTable (created_at)
VALUES ('2018-04-01'),
('2018-04-01'),
('2018-04-01'),
('2018-04-01'),
('2018-04-02'),
('2018-04-02'),
('2018-04-02'),
('2018-04-03'),
('2018-04-03');
You can view this on SqlFiddle.
Use a subquery:
SELECT day, cnt, (#s := #s + cnt)
FROM (SELECT day(created_at) as day, count(*) as cnt
FROM myTable
GROUP BY day(created_at)
) d CROSS JOIN
(SELECT #s := 0) r;
GROUP BY and variables have not worked together for a long time. In more recent versions, ORDER BY also needs a subquery.

Find all NULL values and set them to lowest unused number using MySQL query

I want to find all NULL values in column parameter_id and set them to lowest unused parameter_id.
I have query which will find lowest unused parameter_id, I also know how to get list of NULL values.
SELECT MIN(t1.parameter_id)+1 FROM table AS t1 WHERE NOT EXISTS (SELECT * FROM table AS t2 WHERE t2.parameter_id = t1.parameter_id+1)
I can get list of all rows with parameter_id=NULL, then make query to find current lowest unused parameter_id and then update parameter_id to that lowest unused number. Since table has 50.000 rows, this approach would create thousands of queries (50.000 * 2 per row).
Is there way to run "single query" which will find all parameter_id=NULL and update them all to current lowest unused parameter_id?
Here is table decrtiption (MySQL 5.5):
id (INT) primary key, auto_increment
parameter_id (INT) default NULL
Sample data:
# id, parameter_id
1, NULL
2, 1
3, NULL
4, 5
5, 3
Desired result:
# id, parameter_id
1, 2
2, 1
3, 4
4, 5
5, 3
EDIT:
I distilled what I want to single query. I simply need to run this query until there is 0 rows affected by UPDATE.
UPDATE `table`
SET parameter_id=
(SELECT *
FROM
(SELECT MIN(t1.parameter_id)+1
FROM `table` AS t1
WHERE NOT EXISTS
(SELECT *
FROM `table` AS t2
WHERE t2.parameter_id = t1.parameter_id+1)) AS t4)
WHERE parameter_id IS NULL LIMIT 1
The following enumerates the unused parameter ids:
select t.*, (#rn := #rn + 1) as seqnum
from table t cross join
(select #rn := 0) params
where not exists (select 1 from table t2 where t2.parameter_id = t.id)
order by t.id;
(You might want to put this in a temporary table with an index on seqnum for the subsequent query.)
The problem is getting a join key for the update. Here is a bit of a kludge: I'm going to add a column, enumerate it, and then drop it:
alter table `table` add column null_seqnum;
update `table` t cross join (select #rn1 := 0) params
set null_seqnum = (#rn1 := #rn1 + 1)
where parameter_id is null;
update `table` t join
(select t.*, (#rn := #rn + 1) as seqnum
from `table` t cross join
(select #rn := 0) params
where not exists (select 1 from `table` t2 where t2.parameter_id = t.id)
order by t.id
) tnull
on t.null_seqnum = tnull.seqnum
set t.parameter_id = tnull.id;
alter table `table` drop column null_seqnum;

MySQL update column with unrelated table data

I have two tables like this in mysql
a.cardnumber (unique)
a.position (numerical 3 digits or null)
a.serial
b.serial (unique)
b.lastused
I want to update any rows in "a" where position is above 600 AND "a.serial" is blank with any serial from "b.serial" where "b.lastused" is either null or more than 30 days ago. When the serial is copied into "a.serial" I want to update "b.lastused" with today's date so I know that the relevant "b.serial" has been used today.
There is no relation to the two tables apart from the serial and any serial from b can be used with any cardnumber in a.
I've tried this using my limited knowledge of mysql but I keep getting an error from my mysql desktop program to say I have an error in my query :(
Any help much appreciated!
I'm assuming here that you want to use a separate b.serial for each row to be updated in a. (This isn't specifically stated, but it seems to me to be most likely; please feel free to correct my assumption if it is wrong.)
I setup a small example. It wasn't clear what the datatypes for each of the columns, so I used INT where I wasn't sure. I used DATE datatype (rather than DATETIME) for lastused.
CREATE TABLE a (`cardnumber` VARCHAR(10) NOT NULL PRIMARY KEY, `position` INT, `serial` INT);
CREATE TABLE b (`serial` INT NOT NULL PRIMARY KEY, lastused DATE);
INSERT INTO a VALUES ('x0000',555,NULL),('x0001',700,123),('a1111',601,NULL),('a2222',602,NULL);
INSERT INTO b VALUES (100,'2012-07-15'),(101,NULL),(102,'2010-01-01'),(103,NULL),(104,NULL);
SELECT * FROM a;
SELECT * FROM b;
Based on the conditions you give, the rows with cardnumbers 'a1111' and 'a2222' should get updated, the other two rows should not (position <= 600, serial already assigned).
Before we run an UPDATE, we want to first run a SELECT that returns the rows to be updated, along with the values that will be assigned. Once we get that, we can convert that to a multi-table UPDATE statement.
SELECT a.cardnumber AS `a.cardnumber`
, a.position AS `a.position`
, a.serial AS `a.serial`
, b.serial AS `b.serial`
, b.lastused AS `b.lastused`
FROM (
SELECT #i := #i + 1 AS i
, aa.*
FROM a aa
JOIN (SELECT #i := 0) ii
WHERE aa.position > 600 /* assuming `position` is numeric datatype */
AND aa.serial IS NULL /* assuming 'blank' represented by NULL */
ORDER BY aa.cardnumber
) ia
JOIN (
SELECT #j := #j + 1 AS j
, bb.serial
, bb.lastused
FROM b bb
JOIN (SELECT #j := 0) jj
WHERE bb.lastused IS NULL
OR bb.lastused < DATE_ADD(NOW(),INTERVAL -30 DAY)
ORDER BY bb.serial
) jb
ON ia.i = jb.j
JOIN a ON a.cardnumber = ia.cardnumber
JOIN b ON b.serial = jb.serial
To convert that to an UPDATE, replace the SELECT ... FROM with UPDATE, and add a SET clause to assign new values to the tables.
UPDATE (
SELECT #i := #i + 1 AS i
, aa.*
FROM a aa
JOIN (SELECT #i := 0) ii
WHERE aa.position > 600
AND aa.serial IS NULL
ORDER BY aa.cardnumber
) ia
JOIN (
SELECT #j := #j + 1 AS j
, bb.serial
, bb.lastused
FROM b bb
JOIN (SELECT #j := 0) jj
WHERE bb.lastused IS NULL
OR bb.lastused < DATE_ADD(NOW(),INTERVAL -30 DAY)
ORDER BY bb.serial
) jb
ON ia.i = jb.j
JOIN a ON a.cardnumber = ia.cardnumber
JOIN b ON b.serial = jb.serial
SET a.serial = b.serial
, b.lastused = DATE(NOW())
-- 4 row(s) affected
You can run the queries for the inline views seperately (ia, jb) to verify that these are getting the rows you want to update.
The join from ia to a, and from jb to b, should be on the primary keys unique key.
The purpose of the ia and jb inline views is to get sequential numbers assigned to those rows so we can match them to each other.
The joins to a and b are to get back to the row in the original table, which is what we want to update.
(Obviously, some adjustments need to be made if serial is not an INT, or lastused is a DATETIME rather than a DATE.)
But this is an example of how I would go about doing the UPDATE you want to do (as best I understood it.)
NOTE: This approach works with MySQL versions that support subqueries. For MySQL 4.0, you would need to run this in steps, storing the results from the "ia" and "jb" inline views (subqueries) into actual tables. Then reference those tables in the query in place of the inline views. The ii and jj subqueries can be removed, and replaced with separate SELECT #i := 0, #j := 0 statement prior to the execution of the queries that reference these variables.
let me know if this works
Update table_a
set serial =
(
select b.serial from table_b b
where b.lastused = NULL
OR b.lastused < (current date - 30) limit 1
)
where cardnumber in
(
select a.cardnumber
from table_a a
where a.position > 600
and a.serial = NULL
)
update table_b b
set b.lastused = current date
where b.lastused = NULL
OR b.lastused < (current date - 30)

How to get the smallest Integers not yet in a database column

I have a table in a MySQL DB with an UNIQUE INT(10) column. The table is pretty populated and the row contains non-consecutive entries of Integer numbers in that column. I would like to do a query, which gets me the smallest number (or the n smallest numbers) that is not in any row.
Example: The table contains rows with values (1, 2, 3, 5, 7, 8, 10, 12, 15) for the column. The sql statement should return i.e. the five lowest non-contained values, which are 4, 6, 9, 11, 13 in this case.
Is this possible with MySQL?
You can use a "numbers" table (it's handy for various operations):
CREATE TABLE num
( i UNSIGNED INT NOT NULL
, PRIMARY KEY (i)
) ;
INSERT INTO num (i)
VALUES
(1), (2), ..., (1000000) ;
Then:
SELECT
num.i
FROM
num
LEFT JOIN
tableX AS t
ON num.i = t.columnX
WHERE
t.columnX IS NULL
ORDER BY
num.i
LIMIT 5
or:
SELECT
num.i
FROM
num
WHERE
NOT EXISTS
( SELECT *
FROM tableX AS t
WHERE num.i = t.columnX
)
ORDER BY
num.i
LIMIT 5
Another approach, without using an auxilary table, would be to use MySQL variables. You can test it in SQL-Fiddle, test-2. The output is not the same as the previous (just to show that it can be done):
SELECT start_id, end_id
FROM
( SELECT
IF( t.columnX <> #id, #id, NULL) AS start_id
, IF( t.columnX <> #id, t.columnX-1, NULL) AS end_id
, #rows := #rows + (t.columnX - #id) AS r
, #id := t.columnX + 1 AS running_id
FROM
tableX AS t
CROSS JOIN
( SELECT #rows := 0
, #id := 1
) AS dummy
WHERE
#rows < 5
ORDER BY
t.columnX
) AS tmp
WHERE
start_id IS NOT NULL
This will work, but I think it is pretty inefficient. You won't need an extra table though (a table that would be (2^31-1)*4/1024^3 = 8GB for all positive numbers in INT). Also I advise you look at why you need this, because it might not be neccesary.
Also it will return the start and end of a range, but not all numbers in that range. (e.g. if you have numbers 1 and 5 it will return {0,2,4,6})
SELECT (t.num-1) AS bound FROM t
WHERE t.num-1 NOT IN (SELECT t.num FROM t)
UNION
SELECT (t.num+1) AS bound FROM t
WHERE t.num+1 NOT IN (SELECT t.num FROM t)
As I said this will be pretty inefficient, JOINs might be faster but you would need benchmark it.
SELECT (t.num-1) AS bound FROM t
LEFT JOIN t AS u ON t.num-1 = u.num
WHERE u.num IS NULL
UNION
SELECT (t.num+1) AS bound FROM t
LEFT JOIN t AS u ON t.num+1 = u.num
WHERE u.num IS NULL

getting the ranking of the rows in mysql ORDER BY statements

suppose I have
SELECT * FROM t ORDER BY j
is there a way to specify the query to also return an autoincremented column that go along with the results that specifies the rank of that row in terms of the ordering?
also this column should also work when using ranged LIMITs, eg
SELECT * FROM t ORDER BY j LIMIT 10,20
should have the autoincremented column return 11,12,13,14 etc....
Oracle, MSSQL etc support ranking functions that do exactly what you want, unfortunately, MySQL has some catching up to do in this regard.
The closest I've ever been able to get to approximating ROW_NUMBER() OVER() in MySQL is like this:
SELECT t.*,
#rank = #rank + 1 AS rank
FROM t, (SELECT #rank := 0) r
ORDER BY j
I don't know how that would rank using ranged LIMIT unless you used that in a subquery perhaps (although performance may suffer with large datasets)
SELECT T2.*, rank
FROM (
SELECT t.*,
#rank = #rank + 1 AS rank
FROM t, (SELECT #rank := 0) r
ORDER BY j
) t2
LIMIT 10,20
The other option would be to create a temporary table,
CREATE TEMPORARY TABLE myRank
(
`rank` INT(11) NOT NULL AUTO_INCREMENT,
`id` INT(11) NOT NULL,
PRIMARY KEY(id, rank)
)
INSERT INTO myRank (id)
SELECT T.id
FROM T
ORDER BY j
SELECT T.*, R.rank
FROM T
INNER JOIN myRank R
ON T.id = R.id
LIMIT 10,20
Of course, the temporary table would need to be persisted between calls.
I wish there was a better way, but without ROW_NUMBER() you must resort to some hackery to get the behavior you want.