MySQL riddle with subquery - mysql

EDIT: looks like an index issue, update at the bottom of the question
I have the following query + subquery whose results I cannot explain. I am starting with this minimal input data set (the application here is capturing data change, and the PK is the id + the tx_id).
mysql> select * from tag_version;
+----+-------------------+------------+-------+----------------+
| id | name | article_id | tx_id | operation_type |
+----+-------------------+------------+-------+----------------+
| 1 | some tag | 1 | 1 | 0 |
| 1 | updated tag | 1 | 2 | 1 |
| 1 | updated again tag | 1 | 3 | 1 |
| 2 | other tag | 1 | 2 | 0 |
+----+-------------------+------------+-------+----------------+
4 rows in set (0.00 sec)
The subquery, standalone
SELECT max(f.tx_id) as max_tx_id, f.id
from tag_version f
WHERE f.tx_id <= 2
GROUP BY f.id
Result is
+-----------+----+
| max_tx_id | id |
+-----------+----+
| 2 | 1 |
| 2 | 2 |
+-----------+----+
2 rows in set (0.00 sec)
The query, where I manually inject the subquery results, notice how they are equal to the above
select t.*
from tag_version t
where t.article_id = 1
AND (t.tx_id, t.id) IN (
(2,1),
(2,2)
)
With expected results
+----+-------------+------------+-------+----------------+
| id | name | article_id | tx_id | operation_type |
+----+-------------+------------+-------+----------------+
| 1 | updated tag | 1 | 2 | 1 |
| 2 | other tag | 1 | 2 | 0 |
+----+-------------+------------+-------+----------------+
2 rows in set (0.00 sec)
And lastly, using the subquery in place of the tuples...
select t.*
from tag_version t
where t.article_id = 1
AND (t.tx_id, t.id) IN (
SELECT max(f.tx_id) as tx_id, f.id
from tag_version f
WHERE f.tx_id <= 2
GROUP BY f.id
)
The result is Empty set (0.00 sec)! Can someone explain this? I get the same empty results when I re-write the query using EXISTS instead of IN
I noticed than when I remove the line WHERE f.tx_id <= 2 from the subquery, I actually get results (although the wrong ones):
+----+-------------------+------------+-------+----------------+
| id | name | article_id | tx_id | operation_type |
+----+-------------------+------------+-------+----------------+
| 1 | updated again tag | 1 | 3 | 1 |
| 2 | other tag | 1 | 2 | 0 |
+----+-------------------+------------+-------+----------------+
2 rows in set (0.00 sec)
Replacing the subquery with a JOIN actually returns the expected correct results
SELECT t.*
FROM tag_version t
JOIN (
SELECT max(f.tx_id) as max_tx_id, f.id
from tag_version f
WHERE f.tx_id <= 2
GROUP BY f.id
) as max_ids
ON max_ids.max_tx_id = t.tx_id
AND max_ids.id = t.id
where t.article_id = 1
Result:
+----+-------------+------------+-------+----------------+
| id | name | article_id | tx_id | operation_type |
+----+-------------+------------+-------+----------------+
| 1 | updated tag | 1 | 2 | 1 |
| 2 | other tag | 1 | 2 | 0 |
+----+-------------+------------+-------+----------------+
2 rows in set (0.00 sec)
In addition, running the same query+subquery on the same data set with both PostgreSQL and SQLite gives the expected correct results.
My MySQL version is Server version: 5.5.40-0ubuntu0.14.04.1 (Ubuntu).
I think the clue to figuring out what's happening is that I actually get results when I remove the WHERE from the subquery, but I can't make something useful out of it.
EDIT: updated with input data set
EDIT: add table information
The table create statement is as follows
CREATE TABLE `tag_version` (
`id` int(11) NOT NULL,
`name` varchar(255) DEFAULT NULL,
`article_id` int(11) DEFAULT NULL,
`tx_id` bigint(20) NOT NULL,
`operation_type` smallint(6) NOT NULL,
PRIMARY KEY (`id`,`tx_id`),
KEY `ix_tag_version_operation_type` (`operation_type`),
KEY `ix_tag_version_tx_id` (`tx_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Data population as well
insert into tag_version (id, name, article_id, tx_id, operation_type) VALUES
(1, 'some tag', 1, 1, 0),
(1, 'updated tag', 1, 2, 1),
(1, 'updated again tag', 1, 3, 1),
(2, 'other tag', 1, 2, 0)
;
When I remove the ix_tag_version_tx_id index, the query returns the correct results... An explanation of why would be useful.

I believe you made a mistake in showing the result of first code(subquery).
The output of this query:
SELECT max(f.tx_id) as qwer, f.id
from tag_version f
WHERE f.tx_id <= 2
GROUP BY f.id
-is not:
+--------------+----+
| max(f.tx_id) | id |
+--------------+----+
| 2 | 1 |
| 2 | 2 |
+--------------+----+
It is:
+------+----+
| qwer | id |
+------+----+
| 2 | 1 |
| 2 | 2 |
+------+----+
(*Notice: the code line max(f.tx_id) as qwer *)
Now try this code for the expexted output.
There is a change when selecting max(f.tx_id).
select t.*
from tag_version t
where t.`article_id` = 1
AND t.`operation_type` != 2
AND (t.`tx_id`, t.`id`) IN (
SELECT max(f.`tx_id`) as `tx_id`, f.`id`
from tag_version f
WHERE f.`tx_id` <= 2
GROUP BY f.`id`
)
Let me know if this gives you the result or any other error.

Related

Select count of rows matching a condition grouped by increments of Id in MySQL

I have a table that has an autoincremented numeric primary. I'm trying to get a count of rows that match a condition grouped by increments of their primary key. Given the data:
| id | value |
|----|-------|
| 1 | a |
| 2 | b |
| 3 | a |
| 4 | a |
| 5 | b |
| 6 | a |
| 7 | b |
| 8 | a |
| 9 | b |
| 10 | b |
| 11 | a |
| 12 | b |
If I wanted to know how many rows matched value = 'a' for every five rows, the result should be:
| count(0) |
|----------|
| 3 |
| 2 |
| 1 |
I can nest a series of subqueries in the SELECT statement, like such:
SELECT (SELECT count(0)
FROM table
WHERE value = 'a'
AND id > 0
AND id <= 5) AS `1-5`,
(SELECT count(0)
FROM table
WHERE value = 'a'
AND id > 5
AND id <=10) AS `6-10`,
...
But is there a way to do this with a GROUP BY statement or something similar where I don't have to manually write out the increments? If not, is there a more time efficient method than a series of subqueries in the SELECT statement as in the above example?
You could divide the ID by 5 and then ceil the result:
SELECT CONCAT((CEIL(id / 5.0) - 1) * 5, '-', CEIL(id / 5.0) * 5), COUNT(*)
FROM mytable
WHERE value = 'a'
GROUP BY CEIL(id / 5.0)
The following aggregated query should do the trick :
SELECT CEIL(id/5), COUNT(*)
FROM table
WHERE value = 'a'
GROUP BY CEIL(id/5)

MYSQL 5.7.20 - Left join with order by a coalesced column - very strange behavior

I'm facing a very strange issue that I hope you can explain to me.
What I'm trying to do is order a resultset based on a coalesced column in a subquery. Let me explain better.
I have two tables:
CREATE TABLE `user` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(255) COLLATE utf8_unicode_ci NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
CREATE TABLE `user_favorites_user` (
`source_user_id` int(11) NOT NULL,
`favorited_user_id` int(11) NOT NULL,
KEY `source_user_id` (`source_user_id`),
KEY `favorited_user_id` (`favorited_user_id`),
CONSTRAINT `user_favorites_user_ibfk_1` FOREIGN KEY (`source_user_id`) REFERENCES `user` (`id`),
CONSTRAINT `user_favorites_user_ibfk_2` FOREIGN KEY (`favorited_user_id`) REFERENCES `user` (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
As one user (let's say ID=1) is browsing the website, I would like to show him other users ordered with his favorites at the bottom.
So, I start with this query:
select user.*, coalesce(favorites.is_favorited,0) as is_favorited_coalesced from user
left join (
select 1 as is_favorited, favorited_user_id from user_favorites_user
where source_user_id = '1'
) favorites on favorites.favorited_user_id = user.id
So far so good, this is what I get and I expected:
+----+-------+------------------------+
| id | name | is_favorited_coalesced |
+----+-------+------------------------+
| 3 | user3 | 1 |
| 4 | user4 | 1 |
| 1 | user1 | 0 |
| 2 | user2 | 0 |
+----+-------+------------------------+
4 rows in set (0.00 sec)
Now, I want to order the resultset. I thought that an ORDER BY clause may suffice:
select user.*, coalesce(favorites.is_favorited,0) as is_favorited_coalesced from user
left join (
select 1 as is_favorited, favorited_user_id from user_favorites_user
where source_user_id = '1'
) favorites on favorites.favorited_user_id = user.id
order by is_favorited_coalesced asc
At this point, I get the same result as above:
+----+-------+------------------------+
| id | name | is_favorited_coalesced |
+----+-------+------------------------+
| 3 | user3 | 1 |
| 4 | user4 | 1 |
| 1 | user1 | 0 |
| 2 | user2 | 0 |
+----+-------+------------------------+
4 rows in set (0.00 sec)
I then thought the coalesce is not good for sorting on-the-fly, so I added a wrapper query but the result is still the same.
Why the ORDER BY is_favorited_coalesced is not working? What am I missing here?
EDIT:
I tried using:
order by coalesce(favorites.is_favorited,0) asc
instead of the alias but I got the same results:
select user.*, coalesce(favorites.is_favorited,0) as is_favorited_coalesced from user left join ( select 1 as is_favorited, favorited_user_id from user_favorites_user where source_user_id = '1' ) favorites on favorites.favorited_user_id = user.id order by coalesce(favorites.is_favorited,0)
--------------
+----+-------+------------------------+
| id | name | is_favorited_coalesced |
+----+-------+------------------------+
| 3 | user3 | 1 |
| 4 | user4 | 1 |
| 1 | user1 | 0 |
| 2 | user2 | 0 |
+----+-------+------------------------+
4 rows in set (0.00 sec)
EDIT 2
I've found another strange behavior. If I try to order by the ID column, this is what I get:
--------------
select user.*, coalesce(favorites.is_favorited,0) as is_favorited_coalesced from user left join ( select 1 as is_favorited, favorited_user_id from user_favorites_user where source_user_id = '1' ) favorites on favorites.favorited_user_id = user.id order by id asc
--------------
+----+-------+------------------------+
| id | name | is_favorited_coalesced |
+----+-------+------------------------+
| 1 | user1 | 1 |
| 2 | user2 | 1 |
| 3 | user3 | 1 |
| 4 | user4 | 1 |
+----+-------+------------------------+
4 rows in set (0.00 sec)
I have no clue why this is happening.
I'm using MySQL 5.7.20 on a virtualized Fedora 25 under windows using VirtualBox.
EDIT 3
As suggested in the comments I've run:
mysql> explain select user.*, coalesce(favorites.is_favorited,0) as is_favorited_coalesced from user left join ( select 1 as is_favorited, favorited_user_id from user_favorites_user where source_user_id = '1' ) favorites on favorites.favorited_user_id = user.id order by is_favorited_coalesced asc;show warnings;
+----+-------------+---------------------+------------+-------+----------------------------------+----------------+---------+------+------+----------+----------------------------------------------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+---------------------+------------+-------+----------------------------------+----------------+---------+------+------+----------+----------------------------------------------------+
| 1 | SIMPLE | user | NULL | ALL | NULL | NULL | NULL | NULL | 4 | 100.00 | NULL |
| 1 | SIMPLE | user_favorites_user | NULL | range | source_user_id,favorited_user_id | source_user_id | 4 | NULL | 2 | 100.00 | Using where; Using join buffer (Block Nested Loop) |
+----+-------------+---------------------+------------+-------+----------------------------------+----------------+---------+------+------+----------+----------------------------------------------------+
2 rows in set, 1 warning (0.00 sec)
+-------+------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Level | Code | Message |
+-------+------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Note | 1003 | /* select#1 */ select `so_test`.`user`.`id` AS `id`,`so_test`.`user`.`name` AS `name`,coalesce(1,0) AS `is_favorited_coalesced` from `so_test`.`user` left join (`so_test`.`user_favorites_user`) on(((`so_test`.`user_favorites_user`.`favorited_user_id` = `so_test`.`user`.`id`) and (`so_test`.`user_favorites_user`.`source_user_id` = '1'))) where 1 order by `is_favorited_coalesced` |
+-------+------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)
Also:
mysql> SELECT ##sql_mode;
+-------------------------------------------------------------------------------------------------------------------------------------------+
| ##sql_mode |
+-------------------------------------------------------------------------------------------------------------------------------------------+
| ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION |
+-------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)
EDIT 4:
I've run:
mysql> SELECT ##optimizer_switch;
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| ##optimizer_switch |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,engine_condition_pushdown=on,index_condition_pushdown=on,mrr=on,mrr_cost_based=on,block_nested_loop=on,batched_key_access=off,materialization=on,semijoin=on,loosescan=on,firstmatch=on,duplicateweedout=on,subquery_materialization_cost_based=on,use_index_extensions=on,condition_fanout_filter=on,derived_merge=on |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)
As stated in the comments.
Including the dataset for fast testing:
SET NAMES utf8;
SET time_zone = '+00:00';
SET foreign_key_checks = 0;
SET sql_mode = 'NO_AUTO_VALUE_ON_ZERO';
CREATE TABLE `user` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(255) COLLATE utf8_unicode_ci NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
INSERT INTO `user` (`id`, `name`) VALUES
(1, 'user1'),
(2, 'user2'),
(3, 'user3'),
(4, 'user4');
CREATE TABLE `user_favorites_user` (
`source_user_id` int(11) NOT NULL,
`favorited_user_id` int(11) NOT NULL,
KEY `source_user_id` (`source_user_id`),
KEY `favorited_user_id` (`favorited_user_id`),
CONSTRAINT `user_favorites_user_ibfk_1` FOREIGN KEY (`source_user_id`) REFERENCES `user` (`id`),
CONSTRAINT `user_favorites_user_ibfk_2` FOREIGN KEY (`favorited_user_id`) REFERENCES `user` (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
INSERT INTO `user_favorites_user` (`source_user_id`, `favorited_user_id`) VALUES
(1, 3),
(1, 4);
This is the bug Query returns wrong data if order by is present (or at least closely related).
It is (in a very similar form) still present in MySQL 8.0.12 (see e.g. your example in dbfiddle, although it will hopefully not show the incorrect behaviour once it is fixed): while it actually orders correctly now (probably because you do a calculation on it), it still returns the incorrect value for is_favorited:
select user.*, favorites.is_favorited,
coalesce(favorites.is_favorited,0) as is_favorited_coalesced from user
left join (
select 1 as is_favorited, favorited_user_id from user_favorites_user
where source_user_id = '1'
) favorites on favorites.favorited_user_id = user.id
order by is_favorited_coalesced desc
+----+-------+--------------+------------------------+
| id | name | is_favorited | is_favorited_coalesced |
+----+-------+--------------+------------------------+
| 1 | user1 | | 1 |
| 2 | user2 | | 1 |
| 3 | user3 | | 0 |
| 4 | user4 | | 0 |
+----+-------+--------------+------------------------+
It seems to be an optimizer problem related to (non-)materialization (MySQL 5.7 had a lot of those). You can workaround most of those bugs by forcing materialization of the derived table (e.g. by adding a limit):
select user.*, favorites.is_favorited,
coalesce(favorites.is_favorited,0) as is_favorited_coalesced from user
left join (
select 1 as is_favorited, favorited_user_id from user_favorites_user
where source_user_id = '1' limit 1000000
) favorites on favorites.favorited_user_id = user.id
order by is_favorited_coalesced desc
+----+-------+--------------+------------------------+
| id | name | is_favorited | is_favorited_coalesced |
+----+-------+--------------+------------------------+
| 1 | user1 | 1 | 1 |
| 2 | user2 | 1 | 1 |
| 3 | user3 | | 0 |
| 4 | user4 | | 0 |
+----+-------+--------------+------------------------+
As #RaymondNijland mentioned, there are other workarounds, e.g. disabling Derived Table Merging with set [GLOBAL|SESSION] optimizer_switch='derived_merge=off' before you run that query. You could also use this to disable the feature globally until the bug gets fixed, so you do not have to inspect every query if it's broken, and just enable it for queries you have verfied that they are not affected (so they can profit from that optimization again).

Mysql - Check if VARCHAR column has a missing value on its incrementation

I'm trying to find out if my values inserted are auto-incrementing correctly or if for any reason one has failed to be inserted, deleted or gone "missing". I've tried several answers from Stackoverflow but they were mainly pointing out autoincrementable int values so they did not help since mine is a VARCHAR value that follows the following sequence:
AA000001
AA000002
...
AA000100
...
AA213978
and so on...
Thanks for your time.
You can declare SQL Vars in Query and calculate the difference in each iteration, as shown in the example below:
Schema
create table MyTable
( ai int auto_increment primary key,
id varchar(100) not null
);
insert MyTable (id) values
('AA000001'),
('AA000002'),
('AA000005'),
('AA000008'),
('AA000009'),
('AA000010');
Query
select id
FROM
(
select
t.id,
SUBSTRING(t.id,3) as s,
CAST(SUBSTRING(t.id,3) AS UNSIGNED) - #lastId as diff,
if( #lastId = 0, 0, CAST(SUBSTRING(t.id,3) AS UNSIGNED) - #lastId) as Difference,
#lastId := CAST(SUBSTRING(t.id,3) AS UNSIGNED) as dummy
from
`MyTable` t,
( select #lastId := 0) SQLVars
order by
t.id
) d
WHERE diff>1;
This is the inside query (not the final result set of the above)
+----------+--------+------+------------+-------+
| id | s | diff | Difference | dummy |
+----------+--------+------+------------+-------+
| AA000001 | 000001 | 1 | 0 | 1 |
| AA000002 | 000002 | 1 | 1 | 2 |
| AA000005 | 000005 | 3 | 3 | 5 |
| AA000008 | 000008 | 3 | 3 | 8 |
| AA000009 | 000009 | 1 | 1 | 9 |
| AA000010 | 000010 | 1 | 1 | 10 |
+----------+--------+------+------------+-------+
Actual Results of Above Query:
+----------+
| id |
+----------+
| AA000005 |
| AA000008 |
+----------+
Here's the SQL Fiddle.
To simply test if there are missing values,
select count(*) <> max(right(col, 6))-min(right(col, 6))+1 || count(*) <> count(distinct col)

Make new column which is incremented by it's order

I need to make new column for my table Products -> called Order (new column). And using rails migration I need to add new column and instantly set it's order number, but it's need to be done by product_id.
What I mean I need something like:
product_id | order
1 ------------> 1
1 ------------> 2
1 ------------> 3
2 ------------> 1
2 ------------> 2
Is there a way of doing it?
EDIT :
You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ''order' = t1.'order'' at line 15:
update product_submissions t
join (
select
id,
product_id,
'order' from (
select id,
product_id,
#rn:= if(#prev = product_id,#rn:=#rn+1,1) as 'order',
#prev:=product_id
from product_submissions,
(select #rn:=0,#prev:=0)r
order by product_id,id
)x
)t1 on t1.id=t.id set t.'order' = t1.'order'
Consider the following
mysql> create table test (id int ,product_id int);
Query OK, 0 rows affected (0.14 sec)
mysql> insert into test values (1,1),(2,1),(3,1),(4,2),(5,2);
Now lets create the order
select
product_id,
`order` from (
select
product_id,
#rn:= if(#prev = product_id,#rn:=#rn+1,1) as `order`,
#prev:=product_id from test,(select #rn:=0,#prev:=0)r
order by product_id,id
)x ;
This will give you something as
+------------+-------+
| product_id | order |
+------------+-------+
| 1 | 1 |
| 1 | 2 |
| 1 | 3 |
| 2 | 1 |
| 2 | 2 |
+------------+-------+
Now lets use in update command, but before that lets add the column (in your case its already there)
mysql> alter table test add column `order` int ;
Query OK, 5 rows affected (0.29 sec)
mysql> select * from test ;
+------+------------+-------+
| id | product_id | order |
+------+------------+-------+
| 1 | 1 | NULL |
| 2 | 1 | NULL |
| 3 | 1 | NULL |
| 4 | 2 | NULL |
| 5 | 2 | NULL |
+------+------------+-------+
5 rows in set (0.00 sec)
Finally the update command
update test t
join (
select
id,
product_id,
`order` from (
select id,
product_id,
#rn:= if(#prev = product_id,#rn:=#rn+1,1) as `order`,
#prev:=product_id
from test,(select #rn:=0,#prev:=0)r
order by product_id,id
)x
)t1 on t1.id=t.id set t.`order` = t1.`order`
mysql> select * from test ;
+------+------------+-------+
| id | product_id | order |
+------+------------+-------+
| 1 | 1 | 1 |
| 2 | 1 | 2 |
| 3 | 1 | 3 |
| 4 | 2 | 1 |
| 5 | 2 | 2 |
+------+------------+-------+
5 rows in set (0.00 sec)

Update the next row of the target row in MySQL

Suppose I have a table that tracks if a payment is missed like this:
+----+---------+------------+------------+---------+--------+
| id | loan_id | amount_due | due_at | paid_at | missed |
+----+---------+------------+------------+---------+--------+
| 1 | 1 | 100 | 2013-08-17 | NULL | NULL |
| 5 | 1 | 100 | 2013-09-17 | NULL | NULL |
| 7 | 1 | 100 | 2013-10-17 | NULL | NULL |
+----+---------+------------+------------+---------+--------+
And, for example, I ran a query that checks if a payment is missed like this:
UPDATE loan_payments
SET missed = 1
WHERE DATEDIFF(NOW(), due_at) >= 10
AND paid_at IS NULL
Then suppose that the row with id = 1 gets affected. I want the amount_due of row with id = 1 be added to the amount_due of the next row so the table would look like this:
+----+---------+------------+------------+---------+--------+
| id | loan_id | amount_due | due_at | paid_at | missed |
+----+---------+------------+------------+---------+--------+
| 1 | 1 | 100 | 2013-08-17 | NULL | 1 |
| 5 | 1 | 200 | 2013-09-17 | NULL | NULL |
| 7 | 1 | 100 | 2013-10-17 | NULL | NULL |
+----+---------+------------+------------+---------+--------+
Any advice on how to do it?
Thanks
Take a look at this :
SQL Fiddle
MySQL 5.5.32 Schema Setup:
CREATE TABLE loan_payments
(`id` int, `loan_id` int, `amount_due` int,
`due_at` varchar(10), `paid_at` varchar(4), `missed` varchar(4))
;
INSERT INTO loan_payments
(`id`, `loan_id`, `amount_due`, `due_at`, `paid_at`, `missed`)
VALUES
(1, 1, 100, '2013-09-17', NULL, NULL),
(3, 2, 100, '2013-09-17', NULL, NULL),
(5, 1, 100, '2013-10-17', NULL, NULL),
(7, 1, 100, '2013-11-17', NULL, NULL)
;
UPDATE loan_payments AS l
LEFT OUTER JOIN (SELECT loan_id, MIN(ID) AS ID
FROM loan_payments
WHERE DATEDIFF(NOW(), due_at) < 0
GROUP BY loan_id) AS l2 ON l.loan_id = l2.loan_id
LEFT OUTER JOIN loan_payments AS l3 ON l2.id = l3.id
SET l.missed = 1, l3.amount_due = l3.amount_due + l.amount_due
WHERE DATEDIFF(NOW(), l.due_at) >= 10
AND l.paid_at IS NULL
;
Query 1:
SELECT *
FROM loan_payments
Results:
| ID | LOAN_ID | AMOUNT_DUE | DUE_AT | PAID_AT | MISSED |
|----|---------|------------|------------|---------|--------|
| 1 | 1 | 100 | 2013-09-17 | (null) | 1 |
| 3 | 2 | 100 | 2013-09-17 | (null) | 1 |
| 5 | 1 | 200 | 2013-10-17 | (null) | (null) |
| 7 | 1 | 100 | 2013-11-17 | (null) | (null) |
Unfortunately I don't have time at the moment to write out full-blown SQL, but here's the psuedocode I think you need to implement:
select all DISTINCT loan_id from table loan_payments
for each loan_id:
set missed = 1 for all outstanding payments for loan_id (as determined by date)
select the sum of all outstanding payments for loan_id
add this sum to the amount_due for the loan's next due date after today
Refer to this for how to loop using pure MySQL: http://dev.mysql.com/doc/refman/5.7/en/cursors.html
I fixed my own problem by adding a missed_at field. I put the current timestamp ($now) in a variable before I update the first row to missed = 1 and missed_at = $now then I ran this query to update the next row's amount_due:
UPDATE loan_payments lp1 JOIN loan_payments lp2 ON lp1.due_at > lp2.due_at
SET lp1.amount_due = lp2.amount_due + lp1.amount_due
WHERE lp2.missed_at = $now AND DATEDIFF(lp1.due_at, lp2.due_at) <= DAYOFMONTH(LAST_DAY(lp1.due_at))
I wish I could use just use LIMIT 1 to that query but it turns out that it's not possible for an UPDATE query with a JOIN.
So all in all, I used two queries to achieve what I want. It did the trick.
Please advise if you have better solutions.
Thanks!