I think this question is gonna be hard to solve.
I have a TABLE in my DDBB as this one:
+----+--------+-------+
| ID | MONTH | VALUE |
+----+--------+-------+
| 1 | 1-2000 | 20.00 |
| 1 | 2-2000 | 21.00 |
| 1 | 3-2000 | 7.00 |
| 1 | 4-2000 | 8.00 |
+----+--------+-------+
With the following definition:
ID INTEGER(7) ZEROFILL NOT NULL
MONTH VARCHAR(7) NOT NULL
VALUE DOUBLE(20,2)
What I'm trying to achieve is the way to retrieve the number of times, through a period, the field {VALUE} has increased from its previous values.
In the example above, if the period is from "1-2000" to "4-2000", {VALUE} has increased 2 times: [20.00->21.00, 7.00->8.00]
At the end, I will like to have the following output:
+----+------------+
| ID | NUM_OF_INC |
+----+------------+
| 1 | 2 |
+----+------------+
What I'm pointing as the main issue, is that {MONTH} is not a DATE type field (of course, it cannot be).
Is there any way to achieve this?
I'm afraid that the solution is to get all the values and then compare one by one from the engine that is executing the queries.
Due to your date format and MySQLs lack of CTEs to convert them a single time, the query gets pretty verbose; this searches the whole range but it's fairly easy to add a range check using the same pattern;
SELECT a.id, COUNT(*) NUM_OF_INC
FROM Table1 a
JOIN Table1 b
ON a.id = b.id
AND a.value < b.value
AND STR_TO_DATE(CONCAT(a.`MONTH`, '-1'), '%c-%Y-%d')
< STR_TO_DATE(CONCAT(b.`MONTH`, '-1'), '%c-%Y-%d')
LEFT JOIN Table1 c
ON a.id = c.id
AND STR_TO_DATE(CONCAT(a.`MONTH`, '-1'), '%c-%Y-%d')
< STR_TO_DATE(CONCAT(c.`MONTH`, '-1'), '%c-%Y-%d')
AND STR_TO_DATE(CONCAT(c.`MONTH`, '-1'), '%c-%Y-%d')
< STR_TO_DATE(CONCAT(b.`MONTH`, '-1'), '%c-%Y-%d')
WHERE c.id IS NULL
GROUP BY a.id;
An SQLfiddle to test with.
Sadly, this query will definitely not use any index you have on MONTH.
If it is an option consider changing the datatype of MONTH into something calculable. Then you can join the last month (Month - 1) and select on a difference > 0:
SELECT
t1.ID, count(*)
FROM
Entity t1
INNER JOIN Entity t2
ON t1.ID = t2.ID
AND t2.MONTH = t1.MONTH - 1
WHERE
t1.VALUE - t2.VALUE > 0
AND t1.MONTH BETWEEN :beginDate AND :endDate
GROUP BY t1.ID
If you can't change the data type. You have to change the t1.MONTH - 1 with some MySQL functions:
DATE_FORMAT(
SUBDATE(
STR_TO_DATE(CONCAT(t1.MONTH, "-1"), "%c-%Y-%d"),
INTERVAL 1 MONTH),
"%c-%Y")
as well as t1.MONTH BETWEEN :beginDate AND :endDate:
STR_TO_DATE(CONCAT(t1.MONTH, "-1"), "%c-%Y-%d")
BETWEEN :beginDate AND :endDate
Related
I want to return all rows that were public in May (2019-05), so if a row was turned to draft (and not back to public) at any point before the end of May, I don't want it. For example:
id | post_id | status | date
-------------------------
1 | 1 | draft | 2019-03-25
2 | 1 | public | 2019-04-02
3 | 1 | draft | 2019-05-25
4 | 2 | draft | 2019-03-10
5 | 2 | public | 2019-04-01
6 | 2 | draft | 2019-06-01
The desired result for the above would return post_id 2 because its last status change prior to the end of May was to public.
post_id 1 was put back in draft before the end of May, so it would not be included.
I'm not sure how to use the correct join or sub-queries to do this as efficiently as possible.
You seem to want the status as of 2019-05-31. A correlated subquery seems like the simplest solution:
select t.*
from t
where t.date = (select max(t2.date)
from t t2
where t2.post_id = t.post_id and
t2.date <= '2019-05-31'
);
To get the ones that are public, just add a WHERE condition:
select t.*
from t
where t.date = (select max(t2.date)
from t t2
where t2.post_id = t.post_id and
t2.date <= '2019-05-31'
) and
t.status = 'public';
For performance, you want an index on (post_id, date).
You can also phrase this using a JOIN:
select t.*
from t join
(select t2.post_id, max(t2.date) as max_date
from t t2
where t2.date <= '2019-05-31'
group by t2.post_id
) t2
on t2.max_date = t.date
where t.status = 'public';
I would expect the correlated subquery to have better performance with the right indexes. However, sometimes MySQL surprises me.
we need to determine whether
the status of each post_id is public prior to the month May (the subquery with max(date)),
any post_id exists with status not equals public within the month May,
and then exclude the post_id satisfying the matter 2.
So, you can use :
select distinct t1.post_id
from tab t1
where t1.post_id not in
(
select distinct t1.post_id
from tab t1
join
(
select post_id, max(date) as date
from tab
where '2019-05-01'> date
group by post_id ) t2
on t1.post_id = t2.post_id
where t1.status != 'public'
and t1.date < '2019-06-01'
and t1.date > '2019-04-30'
);
+---------+
| POST_ID |
+---------+
| 2 |
+---------+
Demo
I have a table which contains an amount per month and a previous month amount.
Each month I need to carry the last previous month amount if it does not exist.
To explain slightly better (and with examples) I might have the following data;
Month,Amount,Previous
2019-01-01,100,0
2019-02-01,100,100
2019-03-01,100,null
2019-04-01,100,null
2019-05-01,100,200
2019-06-01,100,null
So I want to carry the 100 to March and April and then the 200 to June so it looks like this;
Month,Amount,Previous
2019-01-01,100,0
2019-02-01,100,100
2019-03-01,100,100
2019-04-01,100,100
2019-05-01,100,200
2019-06-01,100,200
I'm just hitting blanks, I know there is a way but the mind simply isn't putting it together.
I think it's going to involve LEFT JOIN on the same table and getting a MIN month value with the amount where the date is greater than the last month value but is not greater than the next.
Or it's going to be doing a subquery in a WHERE clause and a LEFT JOIN.
So far I've managed the below but it duplicates the May and June rows for each previous value (100 and 200).
SELECT
*
FROM
table1 t1
LEFT JOIN
table1 t2 ON
t1.month > t2.month
Month,Amount,Previous
2019-01-01,100,0
2019-02-01,100,100
2019-03-01,100,100
2019-04-01,100,100
2019-05-01,100,200
2019-05-01,100,100
2019-06-01,100,200
2019-06-01,100,100
drop table if exists t;
create table t
(Month date,Amount int,Previous int);
insert into t values
('2019-01-01',100,0),
('2019-02-01',100,100),
('2019-03-01',100,null),
('2019-04-01',100,null),
('2019-05-01',100,200),
('2019-06-01',100,null);
select t.*,
case when previous is null then
(select previous from t t1 where t1.month < t.month and t1.previous is not null order by month desc limit 1)
else previous
end as previousdownfilled
from t;
+------------+--------+----------+--------------------+
| Month | Amount | Previous | previousdownfilled |
+------------+--------+----------+--------------------+
| 2019-01-01 | 100 | 0 | 0 |
| 2019-02-01 | 100 | 100 | 100 |
| 2019-03-01 | 100 | NULL | 100 |
| 2019-04-01 | 100 | NULL | 100 |
| 2019-05-01 | 100 | 200 | 200 |
| 2019-06-01 | 100 | NULL | 200 |
+------------+--------+----------+--------------------+
6 rows in set (0.00 sec)
where the case statement checks if things need to be done and the correlated cub query does it. But I suspect this work should be done in the code which creates this table.
You can use a Correlated Subquery here. In the subquery, we will utilize ORDER BY with LIMIT to get the immediate previous amount value.
SELECT
t1.month,
t1.amount,
(SELECT t2.amount FROM table1 t2
WHERE t2.month < t1.month
ORDER BY t2.month DESC LIMIT 1) AS previous
FROM
table1 t1
In MySL 8+, you can use window functions. Unfortunately, MySQL has not (yet) implemented the simplest approach -- lag() with the ignore nulls option.
But you can still do this pretty simply:
select month, amount,
max(previous) over (partition by grp) as previous
from (select t.*, count(previous) over (order by month) as grp
from t
) t;
I have a table like this:
+----+---------+------------+
| id | price | date |
+----+---------+------------+
| 1 | 340 | 2018-09-02 |
| 2 | 325 | 2018-09-05 |
| 3 | 358 | 2018-09-08 |
+----+---------+------------+
And I need to make a view which has a row for every day. Something like this:
+----+---------+------------+
| id | price | date |
+----+---------+------------+
| 1 | 340 | 2018-09-02 |
| 1 | 340 | 2018-09-03 |
| 1 | 340 | 2018-09-04 |
| 2 | 325 | 2018-09-05 |
| 2 | 325 | 2018-09-06 |
| 2 | 325 | 2018-09-07 |
| 3 | 358 | 2018-09-08 |
+----+---------+------------+
I can do that using PHP with a loop (foreach) and making a temp variable which holds the previous price til there is a new date.
But I need to make a view ... So I should do that using pure-SQL .. Any idea how can I do that?
You could use a recursive CTE to generate the records in the "gaps". To avoid that an infinite gap after the last date is "filled", first get the maximum date in the source data and make sure not to bypass that date in the recursion.
I have called your table tbl:
with recursive cte as (
select id,
price,
date,
(select max(date) date from tbl) mx
from tbl
union all
select cte.id,
cte.price,
date_add(cte.date, interval 1 day),
cte.mx
from cte
left join tbl
on tbl.date = date_add(cte.date, interval 1 day)
where tbl.id is null
and cte.date <> cte.mx
)
select id,
price,
date
from cte
order by 3;
demo with mysql 8
Here is an approach which should work without analytic functions. This answer uses a calendar table join approach. The first CTE below is the base table on which the rest of the query is based. We use a correlated subquery to find the most recent date earlier than the current date in the CTE which has a non NULL price. This is the basis for finding out what the id and price values should be for those dates coming in from the calendar table which do not appear in the original data set.
WITH cte AS (
SELECT cal.date, t.price, t.id
FROM
(
SELECT '2018-09-02' AS date UNION ALL
SELECT '2018-09-03' UNION ALL
SELECT '2018-09-04' UNION ALL
SELECT '2018-09-05' UNION ALL
SELECT '2018-09-06' UNION ALL
SELECT '2018-09-07' UNION ALL
SELECT '2018-09-08'
) cal
LEFT JOIN yourTable t
ON cal.date = t.date
),
cte2 AS (
SELECT
t1.date,
t1.price,
t1.id,
(SELECT MAX(t2.date) FROM cte t2
WHERE t2.date <= t1.date AND t2.price IS NOT NULL) AS nearest_date
FROM cte t1
)
SELECT
(SELECT t2.id FROM yourTable t2 WHERE t2.date = t1.nearest_date) id,
(SELECT t2.price FROM yourTable t2 WHERE t2.date = t1.nearest_date) price,
t1.date
FROM cte2 t1
ORDER BY
t1.date;
Demo
Note: To make this work on MySQL versions earlier than 8+, you would need to inline the CTEs above. It would result in verbose code, but, it should still work.
Since you are using MariaDB, it is rather trivial:
MariaDB [test]> SELECT '2019-01-01' + INTERVAL seq-1 DAY FROM seq_1_to_31;
+-----------------------------------+
| '2019-01-01' + INTERVAL seq-1 DAY |
+-----------------------------------+
| 2019-01-01 |
| 2019-01-02 |
| 2019-01-03 |
| 2019-01-04 |
| 2019-01-05 |
| 2019-01-06 |
(etc)
There are variations on this wherein you generate a large range of dates, but then use a WHERE to chop to what you need. And use LEFT JOIN with the sequence 'derived table' on the 'left'.
Use something like the above as a derived table in your query.
I have a table ACQUISITION, with 1 720 208 rows.
------------------------------------------------------
| id | date | value |
|--------------|-------------------------|-----------|
| 1820188 | 2011-01-22 17:48:56 | 1.287 |
| 1820187 | 2011-01-21 21:55:11 | 2.312 |
| 1820186 | 2011-01-21 21:54:00 | 2.313 |
| 1820185 | 2011-01-20 17:46:10 | 1.755 |
| 1820184 | 2011-01-20 17:45:05 | 1.785 |
| 1820183 | 2011-01-19 18:21:02 | 2.001 |
------------------------------------------------------
Following a problem I need to find every rows that have less than two minutes difference.
Ideally I should be able to find here:
| 1820187 | 2011-01-21 21:55:11 | 2.312 |
| 1820186 | 2011-01-21 21:54:00 | 2.313 |
| 1820185 | 2011-01-20 17:46:10 | 1.755 |
| 1820184 | 2011-01-20 17:45:05 | 1.785 |
I'm quite lost here, if you got any ideas.
Let us restate your question in a subtle fashion so we can make this query complete before the heat-death of the universe.
"I need to know the consecutive records in the table with timestamps closer together than two minutes."
We can tie the notion of "consecutive" to your id values.
Try this query and see if you get decent performance (http://sqlfiddle.com/#!9/28738/2/0)
SELECT a.date first_date, a.id first_id, a.value first_value,
b.id second_id, b.value second_value,
TIMESTAMPDIFF(SECOND, a.date, b.date) delta_t
FROM thetable AS a
JOIN thetable AS b ON b.id = a.id + 1
AND b.date <= a.date + INTERVAL 2 MINUTE
The self-join workload is brought to heel with ON b.id = a.id + 1. And, avoiding a function on one of the two date column values allows the query to exploit any index that's available on that column.
Creating a covering index on (id,date,value) will help performance of this query.
If the consecutive-row assumption doesn't work in this dataset, you can try this, to compare each row to the next ten rows. It will be slower. (http://sqlfiddle.com/#!9/28738/6/0)
SELECT a.date first_date, a.id first_id, a.value first_value,
b.id second_id, b.value second_value,
TIMESTAMPDIFF(SECOND, a.date, b.date) delta_t
FROM thetable AS a
JOIN thetable AS b ON b.id <= a.id + 10
AND b.id > a.id
AND b.date <= a.date + INTERVAL 2 MINUTE
If the id values are entirely worthless as a way of ordering your rows, you'll need this. And, it will be very slow. (http://sqlfiddle.com/#!9/28738/5/0)
SELECT a.date first_date, a.id first_id, a.value first_value,
b.id second_id, b.value second_value,
TIMESTAMPDIFF(SECOND, a.date, b.date) delta_t
FROM thetable AS a
JOIN thetable AS b ON b.date <= a.date + INTERVAL 2 MINUTE
AND b.date > a.date
AND b.id <> a.id
Do a SELF JOIN with the table and use TIMEDIFF() function like
SELECT t1.*
from ACQUISITION t1 JOIN ACQUISITION t2
ON TIMEDIFF(t1.`date`, t2.`date`) <= 2;
I need some help with a MySQL query I'm working on. I have data as follows.
Table 1
id date1 text number
---|------------|--------|-------
1 | 2012-12-12 | hi | 399
2 | 2011-11-11 | so | 399
5 | 2010-10-10 | what | 555
3 | 2009-09-09 | bye | 300
4 | 2008-08-08 | you | 300
Table 2
id number date2 ref
---|--------|------------|----
1 | 399 | 2012-06-06 | 40
2 | 399 | 2011-06-06 | 50
5 | 555 | 2011-03-03 | 60
For each row in Table 1, I want to get zero or one ref values from Table 2. There should be a row in the result for each row in Table 1. The number column isn't unique to either table, so the join must be made using the date1 & date2 columns, where date2 is the highest value for the number without exceeding date1 for that number.
The desired result from the above example would be like so.
date1 text number ref
------------|--------|--------|-----
2012-12-12 | hi | 399 | 40
2011-11-11 | so | 399 | 50
2010-10-10 | what | 555 | null
2009-09-09 | bye | 300 | null
2008-08-08 | you | 300 | null
You can see in the result's first row, ref is 40 was chosen because in table2 the record with ref=40 had a date2 that that was less than date1, and the highest date that met that condition.
In the result's second row, ref is 50 was chosen because in table2 the record with ref=50 had a date2 that that was less than date1, and the highest date that met that condition.
The rest of the results have null refs because date1 is always less or a corresponding number doesn't exist in table2.
I've got to a certain point but I'm stuck. The query I have so far is like this.
SELECT date1, text, number, ref
FROM table1
LEFT JOIN (
SELECT *
FROM (
SELECT *
FROM table2
WHERE date2 <= '2012-12-12'
ORDER BY date2 DESC
) tmp
GROUP BY msisdn
) tmp ON table1.number = table2.number;
The problem is that the hard coded date won't do, it should be based on date1, but I can't use date1 because it's in the outer query. Is there a way I can make this work?
I tried similar example with different tables just now and was able to get what you wanted. Below is a similar query modified to fit your needs. You might want to change < with <= if that is what you are looking for.
SELECT a.date1, a.text, b.ref
FROM table1 a LEFT JOIN table2 b ON
( a.number = b.number
AND a.date1 > b.date2
AND b.date2 = ( SELECT MAX(x.date2)
FROM table2 x
WHERE x.number = b.number
AND x.date2 < a.date1)
)
Untested:
SELECT t1.date1,
t1.text,
t1.number,
(SELECT a.ref
FROM TABLE_2 a
JOIN (SELECT t.number,
MAX(t.date2) AS max_date
FROM TABLE_2 t
WHERE t.number = t1.number
AND t.date2 <= t1.date1
GROUP BY t.number) b ON b.number = a.number
AND b.max_date = a.date2)
FROM TABLE_1 t1
The issue is the use of t1 in the derived table of the subselect...