Let's say I have a table like this that tracks the balance of an asset I have in an account:
Delta
NetBalance
Timestamp
2
2
2020-01-01 00:00:00.000
4
6
2020-01-02 00:00:00.000
-1
5
2020-01-03 00:00:00.000
Let's say I have another unrelated table that keeps of track of pricing for my asset:
Price
Timestamp
1.00
2020-01-01 00:00:00.000
1.02
2020-01-01 23:59:00.000
2.01
2020-01-02 10:00:00.000
2.02
2020-01-02 18:00:00.000
3.01
2020-01-03 12:00:00.000
3.02
2020-01-03 13:59:00.000
I'm looking for a query that will yield a result set with the columns from the first table, plus the closest price (from the exact moment, or the past) from the second table and its associated timestamp, so, something like this:
Delta
NetBalance
Timestamp
MostRecentPrice
MostRecentPriceTimestamp
2
2
2020-01-01 00:00:00.000
1.00
2020-01-01 00:00:00.000
4
6
2020-01-02 00:00:00.000
1.02
2020-01-01 23:59:00.000
-1
5
2020-01-03 00:00:00.000
2.02
2020-01-02 18:00:00.000
Working with MySQL here. Would prefer to avoid things like cross joins because the tables themselves are pretty huge, but open to suggestions.
You can try to use LAG window function get previous Timestamp from account then do join with unrelated table.
Then use ROW_NUMBER window function to get MostRecent data rows.
SELECT *
FROM (
SELECT *,
row_number() OVER(PARTITION BY MONTH(Timestamp),DAY(Timestamp) ORDER BY MostRecentPriceTimestamp DESC) rn
FROM (
SELECT a.Delta,
a.NetBalance,
a.Timestamp,
u.Timestamp MostRecentPriceTimestamp,
u.Price MostRecentPrice
FROM (
SELECT *,LAG(Timestamp,1,Timestamp) OVER(ORDER BY Timestamp) prev_Timestamp
FROM account a
) a
INNER JOIN unrelated u
ON u.Timestamp BETWEEN a.prev_Timestamp AND a.Timestamp
) t1
) t1
WHERE rn = 1
sqlfiddle
Suppose I have this exact dataset:
date
widget ID
widget price
widget expiry date
2020-01-01
A
1
2020-03-01
2020-01-01
B
2
2020-04-01
2020-01-01
C
3
2020-05-01
2020-01-01
D
4
2020-06-01
2020-01-02
A
1.1
2020-03-01
2020-01-02
B
2.05
2020-04-01
2020-01-02
C
3.7
2020-05-01
2020-01-02
D
3.8
2020-06-01
2020-01-03
A
1.15
2020-03-01
2020-01-03
B
2.09
2020-04-01
2020-01-03
C
3.54
2020-05-01
2020-01-03
D
4.2
2020-06-01
2020-01-04
A
1.19
2020-03-01
2020-01-04
B
2.14
2020-04-01
2020-01-04
C
3.73
2020-05-01
2020-01-04
D
4.30
2020-06-01
Say I wanted to simultaneously retrieve the full time series of the two following widgets using a single SQL query:
the widget which on date 2020-01-01 had price as close as possible to 1 and expiry date as close as possible to 2020-03-10.
the widget which on date 2020-01-03 had price as close as possible to 3.5 and expiry date as close as possible to 2020-05-15.
In other words, this exact table:
date
widget ID
widget price
widget expiry date
2020-01-01
A
1
2020-03-01
2020-01-01
C
3
2020-05-01
2020-01-02
A
1.1
2020-03-01
2020-01-02
C
3.7
2020-05-01
2020-01-03
A
1.15
2020-03-01
2020-01-03
C
3.54
2020-05-01
2020-01-04
A
1.19
2020-03-01
2020-01-04
C
3.73
2020-05-01
How would you recommend going about it?
Generalising this example, suppose you had a list of tuples like below, where price_i is a target price and expiry_date_i is a target expiry date.
(date_1, price_1, expiry_date_1), (date_2, price_2, expiry_date_2),
(date_3, price_3, expiry_date_3),...
How would you load all of the corresponding widgets' time series in one go?
For the time being I am retrieving these widgets' IDs separately using a SQL query like this one (in this example date='2020-01-01', price=1, expiry date='2020-03-10'). Then collecting all of these retrieved IDs I load the full widget time series.
WITH sample AS
(SELECT *, ABS(DATEDIFF(day,widget_expiry_date, '2020-03-10')) AS date_diff, ABS(widget_price - 1) As price_diff
FROM data WHERE date='2020-01-01'
ORDER BY date_diff ASC, price_diff ASC)
SELECT TOP 1 widget_ID FROM sample
As you can imagine this is extremely inefficient. I wonder if there is a smarter way about it?
Thank you for your time and apologies in advance for the noobish question.
Retrieving all the series in a single query
with params (date_, price_, expiry_date_) AS (
select date '2020-01-01', 1, date '2020-03-10' union all
select date '2020-01-03', 3.5, date '2020-05-15'
)
select data.*
from params p
join data on data.widgetID = (
SELECT widgetID
FROM data d
WHERE d.date = p.date_
ORDER BY ABS(DATEDIFF(d.widget_expiry_date, p.expiry_date_)) ASC, ABS(d.widget_price - p.price_) ASC
LIMIT 1);
db<>fiddle
you also can use window functions:
SELECT indate , widgetID , price , expirydate FROM (
SELECT *
, ROW_NUMBER() OVER (PARTITION BY indate ORDER BY ABS(price - 1), ABS(DATEDIFF(expirydate, '2020-03-10')) ) rn1
, ROW_NUMBER() OVER (PARTITION BY indate ORDER BY ABS(price - 3.5), ABS(DATEDIFF(expirydate, '2020-05-15')) ) rn2
FROM widgets
) t
WHERE rn1 =1 OR rn2 = 1
ORDER BY indate , widgetID
db<>fiddle here
I have a table named calendar with a single column (mydate DATE).
MYDATE
2020-01-01
2020-01-02
2020-01-03
...
I have also a table named delivery with three columns (id PK, giorno DATE, totale FLOAT)
ID GIORNO TOTALE
1 2020-01-01 0.10
2 2020-01-01 5
3 2020-01-02 12
4 2020-01-12 5
5 2020-02-02 13.50
This is what I'm trying to obtain:
Day Numbers of orders
2020-01-01 2
2020-01-02 1
2020-01-03 0
2020-01-04 0
2020-01-05 0
2020-01-06 0
2020-01-07 0
2020-01-08 0
2020-01-09 0
2020-01-10 0
2020-01-11 0
2020-01-12 1
...
I was trying this query:
SELECT c.mydate, IFNULL(d.totale, 0) value
FROM delivery d
RIGHT JOIN calendar c
ON ( c.mydate = d.giorno )
GROUP BY c.mydate
ORDER BY c.mydate
Consider:
select c.mydate, count(d.id) number_of_orders
from calendar c
left join delivery d on d.giorno = c.mydate
group by c.mydate
This works by left-joining the calendar table with the orders table, then aggregating by date, and finally counting the number of matching rows in the order table.
This is quite close to your original query (although this uses left join instead of right join), however this uses an aggregate function to count the orders.
I need to show income and expense per day
Income and expenses are different table
I need to show in following format, for ex:
17/08/2019 date have two incomes in the table
I need to calculate sum of the income in the date, showing on the result with the same day expense.
I have tried with some queries, but it's not working.
Date | Income | Expense | Profit
Select SUM(d.amount)
, SUM(e.amount)
, d.date
, e.date
FROM due d
JOIN expenses e
ON d.date = e.date
Expense table -table-name : expenses
id | date | details | amount
1 13-08-2019 daily 50
2 17-08-2019 cleaning 50
3 17-08-2019 cleaning 50
4 18-08-2019 Tea 150
5 18-08-2019 other 50
Income table -table-name : due
id | date | amount
4 12-08-2019 150
5 13-08-2019 100
6 18-08-2019 450
7 18-08-2019 50
result will be:
id | date | Income | Expense | Profit
1 12-08-2019 150 NULL 150
2 13-08-2019 100 50 50
3 17-08-2019 NULL 100 -100
4 18-08-2019 500 200 300
In the future, I'd suggest posting some table details by using SHOW CREATE TABLE table_name which will allow us to better assist you.
You should be able to use a union and some grouping to get what you are after:
SELECT
Date,
SUM(Income) as Income,
SUM(Expense) as Expense,
SUM(Income) - SUM(Expense) as Profit
FROM (
SELECT
due.date as Date,
due.amount as Income,
0 as Expense
FROM due
UNION ALL
SELECT
expenses.date as Date
0 as Income,
expenses.amount as Expense
FROM expenses
)
GROUP BY Date
For my website, I have a loyalty program where a customer gets some goodies if they've spent $100 within the last 30 days. A query like below:
SELECT u.username, SUM(total-shipcost) as tot
FROM orders o
LEFT JOIN users u
ON u.userident = o.user
WHERE shipped = 1
AND user = :user
AND date >= DATE(NOW() - INTERVAL 30 DAY)
:user being their user ID. Column 2 of this result gives how much a customer has spent in the last 30 days, if it's over 100, then they get the bonus.
I want to display to the user which day they'll leave the loyalty program. Something like "x days until bonus expires", but how do I do this?
Take today's date, March 16th, and a user's order history:
id | tot | date
-----------------------
84 38 2016-03-05
76 21 2016-02-29
74 49 2016-02-20
61 42 2015-12-28
This user is part of the loyalty program now but leaves it on March 20th. What SQL could I do which returns how many days (4) a user has left on the loyalty program?
If the user then placed another order:
id | tot | date
-----------------------
87 12 2016-03-09
They're still in the loyalty program until the 20th, so the days remaining doesn't change in this instance, but if the total were 50 instead, then they instead leave the program on the 29th (so instead of 4 days it's 13 days remaining). For what it's worth, I care only about 30 days prior to the current date. No consideration for months with 28, 29, 31 days is needed.
Some create table code:
create table users (
userident int,
username varchar(100)
);
insert into users values
(1, 'Bob');
create table orders (
id int,
user int,
shipped int,
date date,
total decimal(6,2),
shipcost decimal(3,2)
);
insert into orders values
(84, 1, 1, '2016-03-05', 40.50, 2.50),
(76, 1, 1, '2016-02-29', 22.00, 1.00),
(74, 1, 1, '2016-02-20', 56.31, 7.31),
(61, 1, 1, '2015-12-28', 43.10, 1.10);
An example output of what I'm looking for is:
userident | username | days_left
--------------------------------
1 Bob 4
This is using March 16th as today for use with DATE(NOW()) to remain consistent with the previous bits of the question.
The following is basically how to do what you want. Note that references to "30 days" are rough estimates and what you may be looking for is "29 days" or "31 days" as works to get the exact date that you want.
Retrieve the list of dates and amounts that are still active, i.e., within the last 30 days (as you did in your example), as a table (I'll call it Active) like the one you showed.
Join that new table (Active) with the original table where a row from Active is joined to all of the rows of the original table using the date fields. Compute a total of the amounts from the original table. The new table would have a Date field from Active and a Totol field that is the sum of all the amounts in the joined records from the original table.
Select from the resulting table all records where the Amount is greater than 100.00 and create a new table with Date and the minimum Amount of those records.
Compute 30 days ahead from those dates to find the ending date of their loyalty program.
You would need to take the following steps (per user):
join the orders table with itself to calculate sums for different (bonus) starting dates, for any of the starting dates that are in the last 30 days
select from those records only those starting dates which yield a sum of 100 or more
select from those records only the one with the most recent starting date: this is the start of the bonus period for the selected user.
Here is a query to do that:
SELECT u.userident,
u.username,
MAX(base.date) AS bonus_start,
DATE(MAX(base.date) + INTERVAL 30 DAY) AS bonus_expiry,
30-DATEDIFF(NOW(), MAX(base.date)) AS bonus_days_left
FROM users u
LEFT JOIN (
SELECT o.user,
first.date AS date,
SUM(o.total-o.shipcost) as tot
FROM orders first
INNER JOIN orders o
ON o.user = first.user
AND o.shipped = 1
AND o.date >= first.date
WHERE first.shipped = 1
AND first.date >= DATE(NOW() - INTERVAL 30 DAY)
GROUP BY o.user,
first.date
HAVING SUM(o.total-o.shipcost) >= 100
) AS base
ON base.user = u.userident
GROUP BY u.username,
u.userident
Here is a fiddle.
With this input as orders:
+----+------+---------+------------+-------+----------+
| id | user | shipped | date | total | shipcost |
+----+------+---------+------------+-------+----------+
| 61 | 1 | 1 | 2015-12-28 | 42 | 0 |
| 74 | 1 | 1 | 2016-02-20 | 49 | 0 |
| 76 | 1 | 1 | 2016-02-29 | 21 | 0 |
| 84 | 1 | 1 | 2016-03-05 | 38 | 0 |
| 87 | 1 | 1 | 2016-03-09 | 50 | 0 |
+----+------+---------+------------+-------+----------+
The above query will return this output (when executed on 2016-03-20):
+-----------+----------+-------------+--------------+-----------------+
| userident | username | bonus_start | bonus_expiry | bonus_days_left |
+-----------+----------+-------------+--------------+-----------------+
| 1 | John | 2016-02-29 | 2016-03-30 | 10 |
+-----------+----------+-------------+--------------+-----------------+
Simple solution
Seeing how you do your first query, I guessed that when you are at the point where you look for the "expiration date", you already know that the user meets the 100 points over last 30 days. Then you can do this :
SELECT DATE_ADD(MIN(date),INTERVAL 30 DAY)
FROM orders o
WHERE shipped = 1
AND user = :user
AND date >= (DATE(NOW() - INTERVAL 30 DAY))
It takes the minimum order date of a user over the last 30 days, and add 30 days to the result.
But that really is a poor design to achieve what you want.
You would better to think further and implement what's next.
Advanced solution
In order to reproduce all the following solution, I have used the Fiddle that Trincot kindly built, and expanded it to test on more data : 4 users having 4 orders.
SQL FIddle http://sqlfiddle.com/#!9/668939/1
Step 1 : Design
The following query will return all the users meeting the loyalty program criteria, along with their earlier order date within 30 days and the loyalty program expiration date calculated from the earlier date, and the number of days before it expires.
SELECT O.user, u.username, SUM(total-shipcost) as tot, MIN(date) AS mindate,
DATE_ADD(MIN(date),INTERVAL 30 DAY) AS expirationdate,
DATEDIFF(DATE_ADD(MIN(date),INTERVAL 30 DAY), DATE(NOW())) AS daysleft
FROM orders o
LEFT JOIN users u
ON u.userident = o.user
WHERE shipped = 1
AND date >= DATE(NOW() - INTERVAL 30 DAY)
GROUP BY user
HAVING tot >= 100;
Now, create a VIEW with the above query
CREATE VIEW loyalty_program AS
SELECT O.user, u.username, SUM(total-shipcost) as tot, MIN(date) AS mindate,
DATE_ADD(MIN(date),INTERVAL 30 DAY) AS expirationdate,
DATEDIFF(DATE_ADD(MIN(date),INTERVAL 30 DAY), DATE(NOW())) AS daysleft
FROM orders o
LEFT JOIN users u
ON u.userident = o.user
WHERE shipped = 1
AND date >= DATE(NOW() - INTERVAL 30 DAY)
GROUP BY user
HAVING tot >= 100;
It is important to understand that this is only a one-shot action on your database.
Step 2 : Use your new VIEW
Once you have the view, you can get easily, for all users, the "state" of the loyalty program:
SELECT * FROM loyalty_program
user username tot mindate expirationdate daysleft
1 John 153 February, 28 2016 March, 29 2016 9
2 Joe 112 February, 24 2016 March, 25 2016 5
3 Jack 474 February, 23 2016 March, 24 2016 4
4 Averel 115 February, 22 2016 March, 23 2016 3
For a specific user, you can get the date you are looking for like this:
SELECT expirationdate FROM loyalty_program WHERE username='Joe'
You can also request all the users for which the expiration date is today
SELECT user FROM loyalty_program WHERE expirationdate=DATE(NOW))
But there are other easy possibilities that you'll discover after having played with your VIEW.
Conclusion
Make your life easier: learn to use VIEWS !
I am assuming your table looks like this:
user | id | total | date
-------------------------------
12 84 38 2016-03-05
12 76 21 2016-02-29
23 74 49 2016-02-20
23 61 42 2015-12-28
then try this:
SELECT x.user, x.date, x.id, x.cum_sum, d,date, DATEDIFF(NOW(), x.date) from (SELECT a.user, a.id, a.date, a.total,
(SELECT SUM(b.total) FROM order_table b WHERE b.date <= a.date and a.user=b.user ORDER BY b.user, b.id DESC) AS cum_sum FROM order_table a where a.date>=DATE(NOW() - INTERVAL 30 DAY) ORDER BY a.user, a.id DESC) as x
left join
(SELECT c.user, c.date as start_date, c.id from (SELECT a.user, a.id, a.date, a.total,
(SELECT SUM(b.total) FROM order_table b WHERE b.date <= a.date and a.user=b.user ORDER BY b.user, b.id DESC) AS cum_sum FROM order_table a where a.date>=DATE(NOW() - INTERVAL 30 DAY) ORDER BY a.user, a.id DESC) as c WHERE FLOOR(c.cum_sum/100)=MIN(FLOOR(c.cum_sum/100)) and MOD(c.cum_sum,100)=MAX(MOD(c.cum_sum,100)) group by concat(c.user, "_", c.id)) as d on concat(x.user, "_", x.id)=concat(d.user, "_", d.id) where x.date=d.date;
You will get a table something like this:
user | Date | cum_sum | start_date | Time_left
----------------------------------------------------
12 2016-03-05 423 2016-03-05 24
13 2016-02-29 525 2016-02-29 12
23 2016-02-20 944 2016-02-20 3
29 2015-12-28 154 2015-12-28 4
i have not tested this. But what i am trying to do is to create a table in descending order of id and user, and get a cumulative total column along with it. I have created another table by using this table with cumulative total, with relevant date (i.e. date from which date difference is to be calculated) for each user. I have left joined these two tables, and put in the condition x.date=d.date. I have put start_date and date in the table to check if the query is working.
Also, this is not the most optimum way of writing this code, but i have tried to stay as safe as possible by using sub queries, since i did not have the data to test this. Let me know if you face any error.