I am just learning MySql (SQL in general) and I have a question. I ran a process to populate a table with 72 records. This was done, however, I needed to run the process again and this time it populated the table again with a second record for each user for a total now of 144 records. How can I isolate the newest records created today?
A simple solution is to use current_date to figure out today's date and date() to remove the time portion of your column. Then:
where current_date = date(createdTS)
This is fine for a small dataset as yours. As general solution, you'd need a query that won't need to manipulate every row, e.g.
where createdTS >= current_date and createdTS < current_date + interval 1 day
You just have to use your createdTS column, (assuming you know what was the timestamp of both runs).
SELECT * FROM `my_table` WHERE `createdTS` > '2019-07-25 15:00:00'
You could also RANK() over and get only the newest run for each user (something like this)
Related
I don't have much experience with MySQL so not sure if it's issue with MySQL or my code.
I have a table lets say data and it has a created_at column of DATETIME type.
This table gets 20-30 new records per second, no updates at all.
I have a Cron job that runs every 15 minute and tries to get all records created in the last 15 minutes.
if it runs at 10:15:06am and the last run was at 10:00:03, it makes this query:
SELECT * FROM `data` WHERE (created_at >= '2021-07-30T10:00:03Z' AND created_at < '2021-07-30T10:15:06Z')
Current time is excluded, hence the created_at < current_time.
But the problem is, once in a while I get duplicate data error. That is it includes a few data rows from the current_time, which should have been excluded.
Like in this case, if 15 records were inserted at 10:15:06, the query result might have 4-5 records included in it. But it does not happen every time.
I am using Golang and for current time, I use time.Now(). Can this be because of millisecond or something else ? I am not making more than 1 database query, so I think it has to do something with DB, if I have extra records.
SELECT *
FROM LOGS
WHERE datetime > DATE_SUB(NOW(), INTERVAL 1 MONTH)
I have a big table LOGS (InnoDB). When I try to get last month's data, the query waits too long.
I created an index for column datetime but it seems not helping. How to speed up this query?
Since the database records are inserted in oldest to newest, you could create 2 calls. The first call requesting the ID of the oldest record:
int oldestRecordID = SELECT TOP 1 MIN(id)
FROM LOGS
WHERE datetime > DATE_SUB(NOW(), INTERVAL 1 MONTH)
Then with that ID just request all records where ID > oldestRecordID:
SELECT *
FROM LOGS
WHERE ID > oldestRecordID
It's multiple calls, but it could be faster however I am sure you could combine those 2 calls too.
Probably the only thing you can do is create a clustered index on datetime. This will ensure that the values are co-located.
However, I don't think this will solve your real problem. Why are you bringing back all records from a month. This is a lot of data.
In all likelihood, you could summarize the data in the database and only bring back the information you need rather than all the data.
Some background first. We have a MySQL database with a "live currency" table. We use an API to pull the latest currency values for different currencies, every 5 seconds. The table currently has over 8 million rows.
Structure of the table is as follows:
id (INT 11 PK)
currency (VARCHAR 8)
value (DECIMAL
timestamp (TIMESTAMP)
Now we are trying to use this table to plot the data on a graph. We are going to have various different graphs, e.g: Live, Hourly, Daily, Weekly, Monthly.
I'm having a bit of trouble with the query. Using the Weekly graph as an example, I want to output data from the last 7 days, in 15 minute intervals. So here is how I have attempted it:
SELECT *
FROM currency_data
WHERE ((currency = 'GBP')) AND (timestamp > '2017-09-20 12:29:09')
GROUP BY UNIX_TIMESTAMP(timestamp) DIV (15 * 60)
ORDER BY id DESC
This outputs the data I want, but the query is extremely slow. I have a feeling the GROUP BY clause is the cause.
Also BTW I have switched off the sql mode 'ONLY_FULL_GROUP_BY' as it was forcing me to group by id as well, which was returning incorrect results.
Does anyone know of a better way of doing this query which will reduce the time taken to run the query?
You may want to create summary tables for each of the graphs you want to do.
If your data really is coming every 5 seconds, you can attempt something like:
SELECT *
FROM currency_data cd
WHERE currency = 'GBP' AND
timestamp > '2017-09-20 12:29:09' AND
UNIX_TIMESTAMP(timestamp) MOD (15 * 60) BETWEEN 0 AND 4
ORDER BY id DESC;
For both this query and your original query, you want an index on currency_data(currency, timestamp, id).
I have a mysql table which stores users' availability, stored in 'start' and 'end' columns as date fields.
I have a form where other users can search through the 'availabilty' with various periods like, today, tomorrow and next week . I'm trying to figure out how to construct the query to get all the rows for users who are available 'next month'.
The 'start' values maybe from today and the 'end' value might might be three months away but if next month falls between 'start' and 'end' then I would want that row returned.
The nearest I can get is with the query below but that just returns rows where 'start' falls within next month. Many thanks,
sql= "SELECT * FROM mytable WHERE start BETWEEN DATE_SUB(LAST_DAY(DATE_ADD(NOW(), INTERVAL 1 MONTH)),INTERVAL DAY(LAST_DAY(DATE_ADD(NOW(), INTERVAL 1 MONTH)))-1 DAY) AND LAST_DAY(DATE_ADD(NOW(), INTERVAL 1 MONTH))";
As you are interested in anything that happens in the full month following the current date you could try something like this:
SELECT * FROM mytable WHERE
FLOOR(start/100000000)<=FLOOR(NOW()/100000000)+1 AND
FLOOR( end/100000000)>=FLOOR(NOW()/100000000)+1
This query make use of the fact that datetime values are stored in MySql internally as a number like
SELECT now()+0
--> 20150906130640
where the digits 09 refer to the current month. FLOOR(NOW()/100000000) filters out the first digits of the number (in this case:201509). The WHERE conditions now simply test whether the start date is anywhere before the end of the next month and the end date is at least in or after the period of the next month.
(In my version I purposely left out the condition that start needs to be "after today", since a period that has started earlier seems in my eyes still applicable for your described purpose. If, however, you wanted that condition included too you could simply add an AND start > now() at the end of your WHERE clause.)
Edit
As your SQLfiddle is set-up with a date instead of a (as I was assuming) datetime column your dates will be represented differently in mumeric format like 20150907 and a simple division by 100 will now get you the desired month-number for comparison (201509):
SELECT * FROM mytable WHERE
FLOOR(start/100)<=FLOOR(NOW()/100000000)+1 AND
FLOOR( end/100)>=FLOOR(NOW()/100000000)+1
The number returned by NOW() is still a 14-digit figure and needs to be divided by 100000000. See your updated fiddle here: SQLfiddle
I also added another record ('Charlie') which does not fulfill your requirements.
Update
To better accommodate change-of-year scenarios I updated my SqlFiddle. The where clause is now based on 12*YEAR(..)+MONTH(..) type functions.
I have Rails application which using MYSQL as database. For some condition, I have to delete all the records from table which was stored exactly 2 hours before the current time.
My query is :
DELETE FROM TABLE_NAME WHERE (NOW() - created_at) > 7200;
Here create_at is datetime column type. Storing the value in the format "2012-12-04 06:39:44"
My problem is, the above query fetch the records even though the record created time is just 40 to 50 minutes and got deleted. The only problem is the record got delete after it reach 40 to 50 minx from it create time.
Can any one please correct my query. I want the MySQL solution. Please help me
You probably need this if you want to delete records created exactly 2 hours ago:
DELETE FROM TABLE_NAME WHERE created_at = NOW() - INTERVAL 2 HOUR
or this, that will delete all records created more than 2 hours ago:
DELETE FROM TABLE_NAME WHERE created_at < NOW() - INTERVAL 2 HOUR
Try this ::
DELETE FROM TABLE_NAME WHERE TIMEDIFF(NOW(),created_at) < '02:00:00';
Try:
DELETE FROM TABLE_NAME WHERE created_at<DATE_SUB(NOW(),INTERVAL 2 HOUR)
This query will delete everything created MORE THAN 2 hours ago. Putting an equal sign would mean EXACTLY 2 hours ago (in second). Of course you can format date to consider only minutes, but that would slow down the query.
If created_at is indexed (and I think it should be) don't perform any functions on it so it can use index to perform delete faster.
I understand you want to delete all records created within a time lapse. So, you shouldn't apply a "greater than" operator to the subtract operation. Instead you should try to specify an appropriated time frame.
You could also take a look to the timediff function http://dev.mysql.com/doc/refman/5.5/en/date-and-time-functions.html#function_timediff
Sorry I'm not able to post the right statement for you, since I don't have a mysql server at hand.