I have the following data structure
timestamp(varchar) bid(decimal) ask(decimal)
20090501 03:01:01.582 0.000060 0.000000
20090501 15:01:01.582 0.000120 0.000060
20090501 16:01:01.582 -0.000080 0.000120
20090504 03:01:01.582 0.000040 0.000060
20090504 15:01:01.582 -0.000040 0.000040
20090504 16:01:01.582 0.000000 -0.000040
20090505 03:01:01.582 0.000050 0.000110
20090505 15:01:01.582 0.000000 0.000050
20090505 16:01:01.582 -0.000080 0.000000
Now I want the output like the following
timestamp sum (bid) sum(ask)
20090501 15:01:01.582 0.000180 0.000060
20090504 15:01:01.582 -0.000080 0.000220
20090505 15:01:01.582 0.000050 0.000120
Now the relational logic behind the result is every time 15:01 occurs it will sum all the bids and asks value within the interval of last 15:01 occurrence it means the sum of bids and asks between every 15:01 need to be calculated
I am trying this with MySQL, so any help on this is highly appreciable.
The code which I have done so far is on Sql server 2008 R2
select date=case when substring(timestamp,10,2) <= 15
then substring(timestamp,1,8) else DATEADD("dd",1,substring(timestamp,1,8)) end,
SUM(isnull([Bid Change],0)), SUM([Ask Change]), MAX(aveg),MIN(aveg) from tbltestnew1
group by (case when substring(timestamp,10,2) <= 15
then substring(timestamp,1,8) else DATEADD("dd",1,substring(timestamp,1,8)) end),
CURR;
This gives me the result considering 1 day interval of every 15:01 which is not the correct result!
With MSSQL you could use an outer apply like this:
select
cast(t.timestamp as date) date,
bid_sum,
ask_sum
from tbltestnew1 t
outer apply (
select top 1 timestamp tlag
from tbltestnew1
where t.timestamp > timestamp and cast(timestamp as time) = '15:01:01.582' order by timestamp desc
) tprev
outer apply (
select sum(bid) bid_sum, sum(ask) ask_sum
from tbltestnew1
where (tlag is not null and (cast(timestamp as datetime) between dateadd(second,1, tlag) and t.timestamp)
or (tlag is null and cast(timestamp as datetime) <= t.timestamp)
)
) x
where cast(t.timestamp as time) = '15:01:01.582';
Sample SQL Fiddle
This query would give this result:
| DATE | BID_SUM | ASK_SUM |
|------------|----------|---------|
| 2009-05-01 | 0.00018 | 0.00006 |
| 2009-05-04 | -0.00008 | 0.00022 |
| 2009-05-05 | 0.00005 | 0.00012 |
With MSSQL 2012+ you could have used the lag() window function to access previous rows (which is what the first outer apply does), it would look like this:
select cast(t.timestamp as date) date, sum_bid, sum_ask
from (select timestamp, ask, bid, lag(timestamp) over (order by timestamp) prev from tbltestnew1
where cast(timestamp as time) = '15:01:01.582') t
outer apply (
select sum(bid) sum_bid, sum(ask) sum_ask
from tbltestnew1
where (prev is not null and (cast(timestamp as datetime) between dateadd(second,1, prev) and t.timestamp)
or (prev is null and cast(timestamp as datetime) <= t.timestamp))
) oa
Of course you could reduce the number of casts by using a common table expression (or derived tables).
select
case when time(timestamp) > '15:01:00'
THEN DATE(DATE_ADD(timestamp INTERVAL 1 DAY))
ELSE DATE(timestamp) END AS count_date,
SUM(isnull([Bid Change],0)),
SUM([Ask Change]),
MAX(aveg),
MIN(aveg) from tbltestnew1
group by count_date;
Based on your sample data, it seems to be as simple as GROUP BY LEFT(timestamp, 8).
Related
I have two tables, both with a Time column as timestamp type which is filled by default when the row is created: Table1 is updated approximately every 10 seconds:
Time | Val_1a | Val_2a | Val_3a
2021-11-06 13:59:53 | 15 | 10 | 35
2021-11-06 14:00:02 | 12 | 15 | 34
.................
2021-11-06 14:05:25 | 11 | 13 | 35
2021-11-06 14:05:35 | 11 | 17 | 36
Table2 is updated every hour after mathematical operations on table1:
Time | Var_1b | Var_2b | Var_3b
2021-11-06 11:00:00 | 2 | 15 | 30
2021-11-06 12:00:00 | 8 | 12 | 32
2021-11-06 13:00:00 | 12 | 11 | 35
What I would like to get but I'm not able to do in any way, is:
Check that the last table1.Val_2a value is greater than the first table1.Val_2a value written at the beginning of the current hour (with the tables above, check if 17 > 15). If this condition is not met, the entire query must return 0 otherwise:
2a) If the last row in table2 refers to the previous day, then the query result is simply the difference of the two table1.Val_2a values (17 - 15 = 2)
2b) Otherwise their difference is calculated as at point 2a (17-15 = 2) and it is added to the table2.Var_1b value (2 + 12 = 14)
I hope I was able to explain it in a clearly way, and that it all is possible with a single query. Thanks everyone for the support
Sorry, if I add an Answer but I couldn't add the image into the comment.
This is the qwery I used to test the CASE clause
SELECT t1.dtm, t1.Val_2a2, t1.Val_2a1,
CASE WHEN Val_2a2 > Val_2a1
THEN Val_2a2-Val_2a1 ELSE 0 END AS ValF FROM (SELECT DATE_FORMAT(time, '%Y-%m-%d %H:00:00') dtm,
SUBSTRING_INDEX(GROUP_CONCAT(Val_2a ORDER BY time),',',1) Val_2a1,
SUBSTRING_INDEX(GROUP_CONCAT(Val_2a ORDER BY time DESC),',',1) Val_2a2 FROM table1 GROUP BY dtm) t1
and this is the unexpected result
Qwery result
It is possible in a single query but different people will have different method of doing it. Whatever the method is, I personally think that the most important part is to keep the logic intact. The details you've provided in your question got me assuming that this might be a kind of query you're looking for:
SELECT t1.dtm, t1.Val_2a2, t1.Val_2a1, t2.Val_1b2,
CASE WHEN Val_2a2 > Val_2a1
THEN Val_2a2-Val_2a1+Val_1b2 ELSE 0 END AS ValF
FROM
(SELECT DATE_FORMAT(time, '%Y-%m-%d %H:00:00') dtm,
SUBSTRING_INDEX(GROUP_CONCAT(Val_2a ORDER BY time),',',1) Val_2a1 ,
SUBSTRING_INDEX(GROUP_CONCAT(Val_2a ORDER BY time DESC),',',1) Val_2a2
FROM table1
GROUP BY dtm) t1
LEFT JOIN
(SELECT DATE(time) dtm,
SUBSTRING_INDEX(GROUP_CONCAT(Val_1b ORDER BY time DESC),',',1) Val_1b2
FROM table2
GROUP BY dtm) t2
ON DATE(t1.dtm)=t2.dtm;
Demo fiddle
hoping it can help someone else, after some more test this is the final qwery I got, considering I just need a value on the fly without needing of storing it.
Of course every consideration by the experts is more than appreciate.
Thanks to all
SELECT
CASE WHEN
(ABS(t1.Val_2a2) - ABS(t1.Val_2a1)) BETWEEN 0 AND 30
THEN t1.Val_2a2-t1.Val_2a1+t2.Val_1b2
ELSE t2.Val_1b2
END AS My_result
FROM
(SELECT DATE_FORMAT(Time, '%Y-%m-%d %H:00:00') dtm,
(SELECT Val_2a FROM table1 WHERE Time >= DATE_FORMAT(NOW(),"%Y-%m-%d %H:00:00") ORDER BY Time LIMIT 1) Val_2a1,
(SELECT Val_2a FROM table1 WHERE Time >= DATE_FORMAT(NOW(),"%Y-%m-%d %H:00:00") ORDER BY Time DESC LIMIT 1) Val_2a2
FROM table1
GROUP BY dtm
ORDER BY Time DESC LIMIT 1) t1
LEFT JOIN
(SELECT (Time) dtm,
(Val_1b) Val_1b2
FROM table2
GROUP BY dtm ORDER BY dtm DESC LIMIT 1) t2
ON DATE(t1.dtm)= DATE(t2.dtm)
I have the following query:
SELECT
DATE(`timeStamp`),COUNT(*)
FROM
`wf`.sh`
WHERE
(DATE(`timeStamp`) >= curdate()- INTERVAL 31 DAY)
GROUP BY
DATE(`timeStamp`)
HAVING
COUNT(DATE(`timeStamp`)) > 0
ORDER BY
DATE(`timeStamp`) ASC;
The purpose of this query is to retrieve the amount of users online in my system per day, in the space of a month.
Example dataset:
uID timeStamp
1 2016-11-28 00:27:01
1 2016-11-28 01:10:15
1234 2016-11-28 02:50:00
2 2016-11-28 06:11:09
47 2016-11-28 08:32:48
1246 2016-11-28 09:51:47
In its current format, this query returns the count of rows with duplicate dates, for example:
timeStamp COUNT(*)
2017-01-29 256
2017-01-30 224
2017-01-31 240
2017-02-01 95
2017-02-02 136
I have another field uID; I need to modify my query so that GROUP also ignores rows with a duplicate uID field for each day. I tried creating another GROUP BY but was given an error that 'incorrect GROUP BY clause' (or something of that nature).
Can this be done via pure MySQL?
You can use a subselect
SELECT
visitDate,COUNT(*)
FROM
(SELECT DISTINCT DATE(`timeStamp`) as visitDate, uID FROM `wf`.sh`) alias_t
WHERE
(visitDate >= curdate()- INTERVAL 31 DAY)
GROUP BY
visitDate
HAVING
COUNT(visitDate) > 0
ORDER BY
visitDate ASC;
I've searched for this topic but all I got was questions about grouping results by month. I need to retrieve rows grouped by month with summed up cost from start date to this whole month
Here is an example table
Date | Val
----------- | -----
2017-01-20 | 10
----------- | -----
2017-02-15 | 5
----------- | -----
2017-02-24 | 15
----------- | -----
2017-03-14 | 20
I need to get following output (date format is not the case):
2017-01-20 | 10
2017-02-24 | 30
2017-03-14 | 50
When I run
SELECT SUM(`val`) as `sum`, DATE(`date`) as `date` FROM table
AND `date` BETWEEN :startDate
AND :endDate GROUP BY year(`date`), month(`date`)
I got sum per month of course.
Nothing comes to my mind how to put in nicely in one query to achieve my desired effect, probably W will need to do some nested queries but maybe You know some better solution.
Something like this should work (untestet). You could also solve this by using subqueries, but i guess that would be more costly. In case you want to sort the result by the total value the subquery variant might be faster.
SET #total:=0;
SELECT
(#total := #total + q.sum) AS total, q.date
FROM
(SELECT SUM(`val`) as `sum`, DATE(`date`) as `date` FROM table
AND `date` BETWEEN :startDate
AND :endDate GROUP BY year(`date`), month(`date`)) AS q
You can use DATE_FORMAT function to both, format your query and group by.
DATE_FORMAT(date,format)
Formats the date value according to the format string.
SELECT Date, #total := #total + val as total
FROM
(select #total := 0) x,
(select Sum(Val) as Val, DATE_FORMAT(Date, '%m-%Y') as Date
FROM st where Date >= '2017-01-01' and Date <= '2017-12-31'
GROUP BY DATE_FORMAT(Date, '%m-%Y')) y
;
+---------+-------+
| Date | total |
+---------+-------+
| 01-2017 | 10 |
+---------+-------+
| 02-2017 | 30 |
+---------+-------+
| 03-2017 | 50 |
+---------+-------+
Can check it here: http://rextester.com/FOQO81166
Try this.
I use yearmonth as an integer (the year of the date multiplied by 100 plus the month of the date) . If you want to re-format, your call, but integers are always a bit faster.
It's the complete scenario, including input data.
CREATE TABLE tab (
dt DATE
, qty INT
);
INSERT INTO tab(dt,qty) VALUES( '2017-01-20',10);
INSERT INTO tab(dt,qty) VALUES( '2017-02-15', 5);
INSERT INTO tab(dt,qty) VALUES( '2017-02-24',15);
INSERT INTO tab(dt,qty) VALUES( '2017-03-14',20);
SELECT
yearmonths.yearmonth
, SUM(by_month.month_qty) AS running_qty
FROM (
SELECT DISTINCT
YEAR(dt) * 100 + MONTH(dt) AS yearmonth
FROM tab
) yearmonths
INNER JOIN (
SELECT
YEAR(dt) * 100 + MONTH(dt) AS yearmonth
, SUM(qty) AS month_qty
FROM tab
GROUP BY YEAR(dt) * 100 + MONTH(dt)
) by_month
ON yearmonths.yearmonth >= by_month.yearmonth
GROUP BY yearmonths.yearmonth
ORDER BY 1;
;
yearmonth|running_qty
201,701| 10.0
201,702| 30.0
201,703| 50.0
select succeeded; 3 rows fetched
Need explanations?
My solution has the advantage over the others that it will be re-usable without change when you move it to a more modern database - and you can convert it to using analytic functions when you have time.
Marco the Sane
I am trying to generate a table in the following format.
Proday | 2014-04-01 | 2014-03-01
--------------------------------
1 | 12 | 17
2 | 6 | 0
7 | 0 | 24
13 | 3 | 7
Prodays (duration between two timestamps) is a calculated value and the data for months is a COUNT. I can output the data for a single month, but am having troubles joining queries to additional months. The index (prodays) may not match for each month. e.g.. 2014-04-01 may not have any data for Prodays 7, whereas 2014-03-01 may not have Proday 2. Should indicate with 0 or null.
I suspect FULL OUTER JOIN is what should do the trick. But have read that's not possible in Mysql?
This is the query to get data for a single month:
SELECT round((protime - createtime) / 86400) AS prodays, COUNT(id) AS '2014-04-01'
FROM `tbl_users` as t1
WHERE status = 1 AND DATE_FORMAT(FROM_UNIXTIME(createtime),'%Y-%m-%d') >= '2014-04-01'
AND DATE_FORMAT(FROM_UNIXTIME(createtime),'%Y-%m-%d') <= LAST_DAY('2014-04-01')
GROUP BY prodays
ORDER BY `prodays` ASC
How can I join/union an additional query to create a column for 2014-03-01?
You want to use conditional aggregation -- that is, move the filtering logic from the where clause to the select clause:
SELECT round((protime - createtime) / 86400) AS prodays,
sum(DATE_FORMAT(FROM_UNIXTIME(createtime),'%Y-%m-%d') >= '2014-04-01' AND
DATE_FORMAT(FROM_UNIXTIME(createtime),'%Y-%m-%d') <= LAST_DAY('2014-04-01')
) as `2014-04-01`,
sum(DATE_FORMAT(FROM_UNIXTIME(createtime),'%Y-%m-%d') >= '2014-03-01' AND
DATE_FORMAT(FROM_UNIXTIME(createtime),'%Y-%m-%d') <= LAST_DAY('2014-03-01')
) as `2014-03-01`
FROM `tbl_users` as t1
WHERE status = 1
GROUP BY prodays
ORDER BY `prodays` ASC;
Let's say I have a table that says how many items of something are valid between two dates.
Additionally, there may be multiple such periods.
For example, given a table:
itemtype | count | start | end
A | 10 | 2014-01-01 | 2014-01-10
A | 10 | 2014-01-05 | 2014-01-08
This means that there are 10 items of type A valid 2014-01-01 - 2014-01-10 and additionally, there are 10 valid 2014-01-05 - 2014-01-08.
So for example, the sum of valid items at 2014-01-06 are 20.
How can I query the table to get the sum per day? I would like a result such as
2014-01-01 10
2014-01-02 10
2014-01-03 10
2014-01-04 10
2014-01-05 20
2014-01-06 20
2014-01-07 20
2014-01-08 20
2014-01-09 10
2014-01-10 10
Can this be done with SQL? Either Oracle or MySQL would be fine
The basic syntax you are looking for is as follows:
For my example below I've defined a new table called DateTimePeriods which has a column for StartDate and EndDate both of which are DATE columns.
SELECT
SUM(NumericColumnName)
, DateTimePeriods.StartDate
, DateTimePeriods.EndDate
FROM
TableName
INNER JOIN DateTimePeriods ON TableName.dateColumnName BETWEEN DateTimePeriods.StartDate and DateTimePeriods.EndDate
GROUP BY
DateTimePeriods.StartDate
, DateTimePeriods.EndDate
Obviously the above code won't work on your database but should give you a reasonable place to start. You should look into GROUP BY and Aggregate Functions. I'm also not certain of how universal BETWEEN is for each database type, but you could do it using other comparisons such as <= and >=.
There are several ways to go about this. First, you need a list of dense dates to query. Using a row generator statement can provide that:
select date '2014-01-01' + level -1 d
from dual
connect by level <= 15;
Then for each date, select the sum of inventory:
with
sample_data as
(select 'A' itemtype, 10 item_count, date '2014-01-01' start_date, date '2014-01-10' end_date from dual union all
select 'A', 10, date '2014-01-05', date '2014-01-08' from dual),
periods as (select date '2014-01-01' + level -1 d from dual connect by level <= 15)
select
periods.d,
(select sum(item_count) from sample_data where periods.d between start_date and end_date) available
from periods
where periods.d = date '2014-01-06';
You would need to dynamically set the number of date rows to generate.
If you only needed a single row, then a query like this would work:
with
sample_data as
(select 'A' itemtype, 10 item_count, date '2014-01-01' start_date, date '2014-01-10' end_date from dual union all
select 'A', 10, date '2014-01-05', date '2014-01-08' from dual)
select sum(item_count)
from sample_data
where date '2014-01-06' between start_date and end_date;