My database has data imputed every 1 minute and is stored in the format 2020-04-05 16:20:04 under a column called timestamp.
I need a MySQL query to select data from every day at a specific hour (the second does not matter), for for example I want to get the data from 16:00 of every day from the past 30 days.
It currently, just grabs the data from the past 30 days and then the PHP application sorts it, however, this is causing very slow loading time, hence wanting to only select the wanted data from the database.
Example of data
Please try the following sql:
select
d.timestamp, hour(d.timestamp)
from
demo1 d
where
DATEDIFF(NOW(), d.timestamp) < 30 and hour(d.timestamp) = 16;
The create sql is as following:
CREATE TABLE `demo1` (
`id` int(11) not null auto_increment primary key,
`serverid` int(11) not null,
`timestamp` datetime not null,
KEY `idx_timestamp` (`timestamp`)
) engine = InnoDB;
insert into `demo1` (serverid, timestamp)
VALUES (1, "2020-07-05 16:20:04"),
(2, "2020-07-06 17:20:04"),
(3, "2020-07-07 16:40:04"),
(4, "2020-07-08 08:20:04"),
(5, "2020-07-05 15:20:04"),
(5, "2020-07-05 16:59:04"),
(5, "2020-06-04 16:59:04");
Zhiyong's response will work, but wont perform well. You need to figure out a way to get the query to use indexes.
You can add a simple index on timestamp and run the query this way:
SELECT
d.timestamp, d.*
FROM demo1 d
WHERE 1
AND d.timestamp > CURDATE() - INTERVAL 30 DAY
AND hour(d.timestamp) = 16;
In MySQL 5.7 and up, you can created a generated column (also called calculated column) top store the hour of the timestamp in a separate column. You can then index this column, perhaps as a composite index of hour + timestamp, so that the query above will perform really quickly.
ALTER TABLE demo1
ADD COLUMN hour1 tinyint GENERATED ALWAYS AS (HOUR(timestamp)) STORED,
ADD KEY (hour1, timestamp);
The result query would be:
SELECT
d.timestamp, d.*
FROM demo1 d
WHERE 1
AND d.timestamp > CURDATE() - INTERVAL 30 DAY
AND hour1 = 16;
More info on that here:
https://dev.mysql.com/doc/refman/5.7/en/create-table-generated-columns.html
https://dev.mysql.com/doc/refman/5.7/en/generated-column-index-optimizations.html
Related
Below are the table I've created and its records:
CREATE TABLE students (
id INTEGER PRIMARY KEY,
name TEXT NOT NULL,
gender TEXT NOT NULL,
created_at datetime
);
INSERT INTO students VALUES (1, 'Ryan', 'M', '2012-11-03 00:00:00');
INSERT INTO students VALUES (2, 'Joanna', 'F', '2013-11-03 00:00:00');
To fetch the records by time, I use the following SQL statement:
SELECT * FROM students WHERE created_at > 1620489600000;
and both records can be returned, which confuses me because 1620489600000 (Sat May 08 2021 16:00:00 GMT+0000) should be a timestamp way later than the create_at fields of both records.
Indeed I know this can also be achieved by specifying a time formatted as 2012-11-03 00:00:00, but I just wonder:
whether we can effectively query a datetime column in where clause with unix timestamp?
if not, why does the above select statement return both records?
To compare two different data types MySQL automatically converts one of them to the other. In this case it tries to make numbers of the dates. What is the result? See here
The date gets converted to
20121103000000
which is bigger than
1620489600000
So the result in your WHERE clause is true and returns all records.
If you need to go from a Unix Timestamp you could use this:
WHERE created_at > FROM_UNIXTIME(1620489600000 / 1000)
Notice that I divided by 1000 because Unix time has to be in seconds and not milli seconds.
Good Day
I have two tables named as Users and mileagesnew
The users table has an id which is unique (Primary Key) and the users.id column acts as a foreign key in the mileagesnew table.
The Users Table
The mileagesnew Table
I want to create a MySQL event to insert a new mileage record for each and every user in the Users table inside the mileagesnew table
So, the MySQL trigger i have right now is,
CREATE EVENT Daily_Mileage
ON SCHEDULE
EVERY 1 DAY
STARTS (TIMESTAMP(CURRENT_DATE) + INTERVAL 1 DAY + INTERVAL 1 HOUR)
DO
INSERT INTO `mileagesnew`(`user_id`, `date`, `mileage`, `fuel_mileage`, `fuel`, `fuel_cost`, `km`,
`kmpl`, `remarks`, `created_at`, `updated_at`) VALUES
('Need To Get The ID of every employee in users table here without
repeating',CURRENT_DATE,'0','0','0','0','0','0','0','',CURRENT_TIMESTAMP,CURRENT_TIMESTAMP);
I am not sure how i can get the id value of each records present in the users table without having any duplicates. Is this possible or if i am doing this via mysql should i write the query individually for all users
So far the event set will run only for one user id but not for other which is set as follows,
CREATE EVENT Daily_Mileage
ON SCHEDULE
EVERY 1 DAY
STARTS (TIMESTAMP(CURRENT_DATE) + INTERVAL 1 DAY + INTERVAL 1 HOUR)
DO
INSERT INTO Mileages (user_id, `date`, mileage, fuel_mileage, fuel, fuel_cost, km, kmpl, remarks)
SELECT id, CURRENT_DATE, 0, '0', '0', '0', 0, 0, ''
FROM Users;
PS. created_at and updated_at columns are already DEFAULT CURRENT_TIMESTAMP, so they're removed. Single quotes wrapped zeros for numeric columns are removed.
PPS. TIMESTAMP function usage in STARTS is excess, STARTS CURRENT_DATE + INTERVAL '1 1' DAY_HOUR is enough.
the table name is mileagesnew not Mileage – Developer
Adjust it in the query by yourself.
I have a table with column timestam which stores a timestamp with this format: "2012-12-10 21:24:30"
I am looking for a SQL Query that takes the current timestamp and subtracts it with the one in the column and gives the difference in this format:
"3 Hours and 2 mins Remaining"
Looking for a MySQL Query that does that.
use TIMESTAMPDIFF
TIMESTAMPDIFF(unit,datetime_expr1,datetime_expr2)
where unit argument, which should be one of the following values:
MICROSECOND (microseconds), SECOND, MINUTE, HOUR, DAY, WEEK, MONTH,
QUARTER, or YEAR.
my approach : set unit as SECOND and then use SEC_TO_TIME
SELECT SEC_TO_TIME(TIMESTAMPDIFF(SECOND,`table`.`time_column`,CURRENT_TIMESTAMP()))
// will return in hh:mm:ii format
update
Return yes if HourDifference are less than 48 hours? otherwise no
SELECT IF(TIMESTAMPDIFF(HOUR,`time_column`,CURRENT_TIMESTAMP())< 48 ,'yes','no')
Assume your table name is 'testtable'
Table create query is given below
CREATE TABLE `testtable` (
`id` INT(2) NOT NULL AUTO_INCREMENT,
`period` TIMESTAMP NULL DEFAULT NULL,
PRIMARY KEY (`id`)
)
Insert some data to test my query. Query to insert some data is given below
INSERT INTO `testtable` (`id`, `period`) VALUES
(1, '2012-12-10 17:21:09'),
(2, '2012-11-06 18:21:12'),
(3, '2012-12-06 18:21:18'),
(4, '2012-12-06 19:21:24'),
(5, '2012-12-06 18:21:27');
Now execute following query to get your answer
SELECT *,
CONCAT(HOUR(difftime), ' hours ', MINUTE(difftime), ' Minutes ',
SECOND(difftime), ' seconds remaining') AS timetaken
FROM (SELECT *,
SEC_TO_TIME(UNIX_TIMESTAMP(NOW()) - UNIX_TIMESTAMP(
ttable.period)) AS
diffTime
FROM testtable ttable) AS temptable1
Output is given below
The column 'timetaken' will display answer.
my question is how can I select all 24 hours in a day as data in a select?
There is a more polished way to do that:
select 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23
from dual
My target db is Mysql, but a sql-standard solution is appreciated!
MySQL doesn't have recursive functionality, so you're left with using the NUMBERS table trick -
Create a table that only holds incrementing numbers - easy to do using an auto_increment:
DROP TABLE IF EXISTS `example`.`numbers`;
CREATE TABLE `example`.`numbers` (
`id` int(10) unsigned NOT NULL auto_increment,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
Populate the table using:
INSERT INTO NUMBERS
(id)
VALUES
(NULL)
...for as many values as you need. In this case, the INSERT statement needs to be run at least 25 times.
Use DATE_ADD to construct a list of hours, increasing based on the NUMBERS.id value:
SELECT x.dt
FROM (SELECT TIME(DATE_ADD('2010-01-01', INTERVAL (n.id - 1) HOUR)) AS dt
FROM numbers n
WHERE DATE_ADD('2010-01-01', INTERVAL (n.id - 1) HOUR) <= '2010-01-02' ) x
Why Numbers, not Dates?
Simple - dates can be generated based on the number, like in the example I provided. It also means using a single table, vs say one per data type.
SELECT * from dual WHERE field BETWEEN 0 AND 24
So lets say I have some records that look like:
2011-01-01 Cat
2011-01-02 Dog
2011-01-04 Horse
2011-01-06 Lion
How can I construct a query that will return 2011-01-03 and 2011-01-05, ie the unused dates. I postdate blogs into the future and I want a query that will show me the days I don't have anything posted yet. It would look from the current date to 2 weeks into the future.
Update:
I am not too excited about building a permanent table of dates. After thinking about it though it seems like the solution might be to make a small stored procedure that creates a temp table. Something like:
CREATE PROCEDURE MISSING_DATES()
BEGIN
CREATE TABLE TEMPORARY DATES (FUTURE DATETIME NULL)
INSERT INTO DATES (FUTURE) VALUES (CURDATE())
INSERT INTO DATES (FUTURE) VALUES (ADDDATE(CURDATE(), INTERVAL 1 DAY))
...
INSERT INTO DATES (FUTURE) VALUES (ADDDATE(CURDATE(), INTERVAL 14 DAY))
SELECT FUTURE FROM DATES WHERE FUTURE NOT IN (SELECT POSTDATE FROM POSTS)
DROP TABLE TEMPORARY DATES
END
I guess it just isn't possible to select the absence of data.
You're right — SQL does not make it easy to identify missing data. The usual technique is to join your sequence (with gaps) against a complete sequence, and select those elements in the latter sequence without a corresponding partner in your data.
So, #BenHoffstein's suggestion to maintain a permanent date table is a good one.
Short of that, you can dynamically create that date range with an integers table. Assuming the integers table has a column i with numbers at least 0 – 13, and that your table has its date column named datestamp:
SELECT candidate_date AS missing
FROM (SELECT CURRENT_DATE + INTERVAL i DAY AS candidate_date
FROM integers
WHERE i < 14) AS next_two_weeks
LEFT JOIN my_table ON candidate_date = datestamp
WHERE datestamp is NULL;
One solution would be to create a separate table with one column to hold all dates from now until eternity (or whenever you expect to stop blogging). For example:
CREATE TABLE Dates (dt DATE);
INSERT INTO Dates VALUES ('2011-01-01');
INSERT INTO Dates VALUES ('2011-01-02');
...etc...
INSERT INTO Dates VALUES ('2099-12-31');
Once this reference table is set up, you can simply outer join to determine the unused dates like so:
SELECT d.dt
FROM Dates d LEFT JOIN Blogs b ON d.dt = b.dt
WHERE b.dt IS NULL
If you want to limit the search to two weeks in the future, you could add this to the WHERE clause:
AND d.dt BETWEEN NOW() AND ADDDATE(NOW(), INTERVAL 14 DAY)
The way to extract rows from the mysql database is via SELECT. Thus you cannot select rows that do not exist.
What I would do is fill my blog table with all possible dates (for a year, then repeat the process)
create table blog (
thedate date not null,
thetext text null,
primary key (thedate));
doing a loop to create all dates entries for 2011 (using a program, eg $mydate is the date you want to insert)
insert IGNORE into blog (thedate,thetext) values ($mydate, null);
(the IGNORE keyword to not create an error (thedate is a primary key) if thedate exists already).
Then you insert the values normally
insert into blog (thedate,thetext) values ($mydate, "newtext")
on duplicate key update thetext="newtext";
Finally to select empty entries, you just have to
select thedate from blog where thetext is null;
You probably not going to like this:
select '2011-01-03', count(*) from TABLE where postdate='2011-01-03'
having count(*)=0 union
select '2011-01-04', count(*) from TABLE where postdate='2011-01-04'
having count(*)=0 union
select '2011-01-05', count(*) from TABLE where postdate='2011-01-05'
having count(*)=0 union
... repeat for 2 weeks
OR
create a table with all days in 2011, then do a left join, like
select a.days_2011
from all_days_2011
left join TABLE on a.days_2011=TABLE.postdate
where a.days_2011 between date(now()) and date(date_add(now(), interval 2 week))
and TABLE.postdate is null;