I have huge table with millions of records that store stock values by timestamp. Structure is as below:
Stock, timestamp, value
goog,1112345,200.4
goog,112346,220.4
Apple,112343,505
Apple,112346,550
I would like to query this table by timestamp. If the timestamp matches,all corresponding stock records should be returned, if there is no record for a stock for that timestamp, the immediate previous one should be returned. In the above ex, if I query by timestamp=1112345 then the query should return 2 records:
goog,1112345,200.4
Apple,112343,505 (immediate previous record)
I have tried several different ways to write this query but no success & Im sure I'm missing something. Can someone help please.
SELECT `Stock`, `timestamp`, `value`
FROM `myTable`
WHERE `timestamp` = 1112345
UNION ALL
SELECT `Stock`, `timestamp`, `value`
FROM `myTable`
WHERE `timestamp` < 1112345
ORDER BY `timestamp` DESC
LIMIT 1
select Stock, timestamp, value from thisTbl where timestamp = ? and fill in timestamp to whatever it should be? Your demo query is available on this fiddle
I don't think there is an easy way to do this query. Here is one approach:
select tprev.*
from (select t.stock,
(select timestamp from t.stock = s.stock and timestamp <= <whatever> order by timestamp limit 1
) as prevtimestamp
from (select distinct stock
from t
) s
) s join
t tprev
on s.prevtimestamp = tprev.prevtimestamp and s.stock = t.stock
This is getting the previous or equal timestamp for the record and then joining it back in. If you have indexes on (stock, timestamp) then this may be rather fast.
Another phrasing of it uses group by:
select tprev.*
from (select t.stock,
max(timestamp) as prevtimestamp
from t
where timestamp <= YOURTIMESTAMP
group by t.stock
) s join
t tprev
on s.prevtimestamp = tprev.prevtimestamp and s.stock = t.stock
Related
I am trying to speed up a MYSQL query.
In a column called "MISC", I first have to extract a "traceID" variable, that will be used to match row of another table.
Example of the MISC column:
PFFCC_Strip/fkk49322/PMethod=Diners/CardType=Diners/9999******9999/2010/TraceId=7122910
I am extracting the value "7122910" as traceID and find corresponding row with a left join. The traceId value being unique, only one row must be present on each table.
I cannot set Index on the tables to speed up process. Any approach that could make this query run faster? As it is, it takes a few seconds to run which is not possible.
select *
from
(select TraceID,PP,UDef2, Payment_Method, Approved, TransactionID, Amount
from pr) pr
left join
(select
PAYMENT_ID as Payment_ID_omega, TRANSACTION_TYPE,
REQUESTED_AMOUNT, AMOUNT, `STATUS` as StatusRef_omega,
REQUEST_DATE, Agent,
if (locate('TraceId=',MISC)>0, SUBSTRING_INDEX(MISC,'TraceId=',-1),'') as traceID
from BankingActivity ) omega
on pr.TraceID = omega.traceID
having
(REQUEST_DATE BETWEEN DATE_ADD(DATE(NOW()), INTERVAL -1 DAY) AND NOW())
ORDER BY pr.TraceID DESC
You can place your filters inside the query before join that must make a difference and you must have the index on table pr(TraceID) and BankingActivity(REQUEST_DATE, traceID). For more optimised query, Please post the execution plan.
select * from(select TraceID
,PP
,UDef2
,Payment_Method
,Approved
,TransactionID
,Amount
from pr) pr
left join (select PAYMENT_ID as Payment_ID_omega
,TRANSACTION_TYPE
,REQUESTED_AMOUNT
,AMOUNT
,`STATUS` as StatusRef_omega
,REQUEST_DATE
,Agent
,if (locate('TraceId=', MISC) > 0, SUBSTRING_INDEX(MISC,'TraceId=',-1),'') as traceID
from BankingActivity
WHERE REQUEST_DATE BETWEEN DATE_ADD(DATE(NOW()), INTERVAL -1 DAY) AND NOW()) omega
on pr.TraceID = omega.traceID
ORDER BY pr.TraceID DESC
Here is my table
Which have field type which means 1 is for income and 2 is for expense
Now requirement is for example in table there is two transaction made on 2-10-2018 so i want data as following
Expected Output
id created_date total_amount
1 1-10-18 10
2 2-10-18 20(It calculates all only income transaction made on 2nd date)
3 3-10-18 10
and so on...
it will return an new field which contains only incom transaction made on perticulur day
What i had try is
SELECT * FROM `transaction`WHERE type = 1 ORDER BY created_date ASC
UNION
SELECT()
//But it wont work
SELECT created_date,amount,status FROM
(
SELECT COUNT(amount) AS totalTrans FROM transaction WHERE created_date = created_date
) x
transaction
You can Also See Schema HERE http://sqlfiddle.com/#!9/6983b9
You can Count() the total number of expense transactions using conditional function If(), on a group of created_date.
Similarly, you can Sum() the amount of expense done using If(), on a created_date.
Try the following:
SELECT
`created_date`,
SUM(IF (`type` = 2, `amount`, 0)) AS total_expense_amount,
COUNT(IF (`type` = 2, `id`, NULL)) AS expense_count
FROM
`transaction`
GROUP BY `created_date`
ORDER BY `created_date` ASC
Do you just want a WHERE clause?
SELECT t.created_date, SUM(amount) as total_amount
FROM transaction t
WHERE type = 2
GROUP BY t.created_date
ORDER BY created_date ASC ;
This query returns all items where the difference between the timestamps is less than 180 seconds.
The problem is, after all this is done, I need to then limit the results to the one most recent entry per facebook_id.
I tried using GROUP BY facebook_id, but it doesn't work because if I GROUP BY facebook_id before ORDER BY 'time', it picks the older entry instead of the newer entry which is not what I want.
Is there any way to GROUP BY after ORDER BY?
SELECT facebook_id,
TIMESTAMPDIFF(SECOND, `time`, '$mytime') AS `timediff`
FROM `table`
WHERE `facebook_id` != $fbid
HAVING `timediff` <= '180'
ORDER BY `time` DESC
Thanks for your help!
Note: I did try the suggested solutions to this question but had no success. GROUP BY after ORDER BY
You can use a self join by calculating maximum value of time column and join with 2 conditions one with facebook_id and second to match the time from table to the max_time of second query which will return recent entry against each facebook_id
SELECT t.*,
TIMESTAMPDIFF(SECOND, `time`, '$mytime') AS `timediff`
FROM `table` t
JOIN (
SELECT facebook_id,MAX(`time`) max_time FROM `table` GROUP BY facebook_id
) t1
ON(t.facebook_id= t1.facebook_id AND t.`time` = t1.max_time)
WHERE t.`facebook_id` != $fbid
HAVING `timediff` <= '180'
ORDER BY t.`time` DESC
SELECT fid, timediff
FROM (
SELECT facebook_id as fid,
TIMESTAMPDIFF(SECOND, `time`, '$mytime') AS timediff
FROM `table`
WHERE `facebook_id` != $fbid
HAVING `timediff` <= '180'
ORDER BY `time` DESC
) entries
GROUP BY entries.fid
Please let me know if you have any questions!
I have a requirement where I need o group data into equal number ob rows. As mysql doesn't have rownum() I'm simulating this behaviour:
SET #row:=6;
SELECT MAX(agg.timestamp) AS timestamp, MAX(agg.value) AS value, COUNT(agg.value) AS count
FROM
(
SELECT timestamp, value, #row:=#row+1 AS row
FROM data
WHERE channel_id=52 AND timestamp >= 0 ORDER BY timestamp
) AS agg
GROUP BY row div 8
ORDER BY timestamp ASC;
Note: according to Can grouped expressions be used with variable assignments? this query may not be 100% correct, but it does work.
An additional requirement is to calculate the row difference between the grouped sets. I've looked for a solution joining the same table with a subquery:
SET #row:=6;
SELECT MAX(agg.timestamp) AS timestamp, MAX(agg.value) AS value, COUNT(agg.value) AS count
FROM
(
SELECT timestamp, value, #row:=#row+1 AS row
FROM data
WHERE channel_id=52 AND timestamp >= 0 ORDER BY timestamp
) AS agg
LEFT JOIN data AS prev
ON prev.channel_id = agg.channel_id
AND prev.timestamp = (
SELECT MAX(timestamp)
FROM data
WHERE data.channel_id = agg.channel_id
AND data.timestamp < MIN(agg.timestamp)
)
GROUP BY row div 8
ORDER BY timestamp ASC;
Unfortunately that errors:
Error Code: 1054. Unknown column 'agg.channel_id' in 'on clause'
Any idea how this query could be written?
You never selected channel_id from your sbuquery, so it's not returned to the parent query, and is therefore invisible. Try
SELECT MAX(agg.timestamp) AS timestamp, MAX(agg.value) AS value, COUNT(agg.value) AS count
FROM
(
SELECT timestamp, value, #row:=#row+1 AS row, channel_id
^^^^^^^^^^^^-- need this
FROM data
Since MySQL only sees and uses the fields you explicitly return from that subquery, and will NOT "dig deeper" into the table underlying the query, you need to select/return all of the fields you'll be using the parent queries.
How about this version:
SELECT MAX(agg.timestamp) AS timestamp, MAX(agg.value) AS value, COUNT(agg.value) AS count, COALESCE(prev.timestamp, 0) AS prev_timestamp
FROM (SELECT d.*, #row:=#row+1 AS row
FROM data d CROSS JOIN
(select #row := 6) vars
WHERE channel_id = 52 AND timestamp >= 0 ORDER BY timestamp
) agg LEFT JOIN
data prev
ON prev.channel_id = agg.channel_id AND
prev.timestamp = (SELECT MAX(timestamp)
FROM data
WHERE data.channel_id = agg.channel_id AND
data.timestamp < agg.timestamp
)
GROUP BY row div 8
ORDER BY timestamp ASC;
This includes all the columns in the subquery. And it puts the variable initialization in the same query.
I have a MySQL database-table with the following colums
ID
status (can contain values 0, 1, 2)
timepstamp
text
note
owner
I'd like to obtain the following information about the entries of aspecific owner from the table:
number of entries
number of entries where status=0
number of entries where status=1
number of entries where status=2
number of entries where LENGTH(note)>0
minimum timestamp
maximum timestamp
I used to read the complete datasets and then evaluate them with PHP using
SELECT status, timestamp, LENGTH(note)>0 WHERE owner="name";
I have the problem that some users have so many entries, that that I frequently get an out of memory error if I read the data to PHP. I thought that letting MySQL evaluating the data should be more performat. I could not manage to write a query that could fulfill this task.
SELECT
MIN(timestamp) AS mintime,
MAX(timestamp) AS maxtime,
COUNT(*) AS number,
...
WHERE owner="name"
Is it somehow possible to obtain the result in one go? For example with a nested WHERE or IFwithin a COUNT?
COUNT(WHERE status=0) AS inactive
COUNT(IF(status=1)) AS active
...
How would you solve the problem?
Give this a try -
SELECT
COUNT(*) AS total,
SUM(IF(status=0, 1, 0)) AS stat0,
SUM(IF(status=1, 1, 0)) AS stat1,
SUM(IF(status=2, 1, 0)) AS stat2,
SUM(IF(LENGTH(note)>0, 1, 0)) AS notes,
MIN(timestamp) AS mintime,
MAX(timestamp) AS maxtime
FROM tbl_name
WHERE owner="name"
GROUP BY owner
Try this:
SELECT
MIN(`timestamp`) AS `mintime`,
MAX(`timestamp`) AS `maxtime`,
COUNT(`ID`) AS `number`,
(SELECT COUNT(`ID`) FROM `Table` WHERE `owner` = 'owner' AND `status` = 0) AS `inactive`,
(SELECT COUNT(`ID`) FROM `Table` WHERE `owner` = 'owner' AND `status` = 1) AS `active`,
(SELECT COUNT(`ID`) FROM `Table` WHERE `owner` = 'owner' AND LENGTH(`note`)>0) AS `longentries`
FROM `Table`
WHERE `owner` = 'name'
You should probably consider normalising the database design, though, so that you'll have two separate tables, one for users and one for entries, like this:
USERS
id
name
status
ENTRIES
id
user_id
timestamp
text
note (if this is an entry-specific field; otherwise move it to the table USERS)
Hope this helps.