I have the following:
data points collected per day with associated date - unix timestamp.
I am trying to achieve the following:
Filter data points per day. i.e suppose I have 4 data points per day, I am trying to reduce it to n data points per day using GROUP BY.
I am using the following query:
SELECT *,FLOOR(UNIX_TIMESTAMP(date)/((1440/n)*60))
AS timekey
FROM `tbl`
where date < 'date'
GROUP BY timekey)m
UNION
(SELECT * ,0 As timekey FROM `tbl`
where date > 'date' );
This according to me should generate timekey for the data points for any row with date < given_date_in_query.
n determines the number of points to be grouped by.
Then the GROUP BY should ultimately reduce the number of data points with respect to n.
The above query works for a table with more than 20000 rows.
for n=2 the datapoints before the specified date are reduced to 2 points(to 2 rows from 4).
Here is the create script:
CREATE TABLE `tbl_a` (
`id` int(25) NOT NULL AUTO_INCREMENT,
`pid` bigint(11) NOT NULL,
---
`date` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
---
PRIMARY KEY (`id`),
UNIQUE KEY `phase_id_2` (`phase_id`,`date`),
KEY `id` (`id`,`pid`),
KEY `date` (`date`)
) ENGINE=InnoDB AUTO_INCREMENT=44 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
The query works perfectly for the above table.
However when I try the same query with another db with comparatively less rows it does not work as expected.
It reduces only the date specified in the query and the rest remain untouched.
.i.e only the row corresponding to the given_date in the query is
reduced by the query and the rest of the rows remain untouched.
Test table create script:
CREATE TABLE `test` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`date` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
KEY `date` (`date`)
) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
Could this be an index error or a database engine exception?
Steps to recreate the problem:
Create test tbl :
CREATE TABLE `test` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`date` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
KEY `date` (`date`)
) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
Insert data:
INSERT INTO test (date) VALUES ('2015-11-01 00:00:00');
INSERT INTO test (date) VALUES ('2015-11-01 06:00:00');
INSERT INTO test (date) VALUES ('2015-11-01 12:00:00');
INSERT INTO test (date) VALUES ('2015-11-01 18:00:00');
INSERT INTO test (date) VALUES ('2016-11-01 00:00:00');
INSERT INTO test (date) VALUES ('2016-11-01 06:00:00');
INSERT INTO test (date) VALUES ('2016-11-01 12:00:00');
INSERT INTO test (date) VALUES ('2016-11-01 18:00:00');
INSERT INTO test (date) VALUES ('2016-11-02 00:00:00');
INSERT INTO test (date) VALUES ('2016-11-02 06:00:00');
INSERT INTO test (date) VALUES ('2016-11-02 12:00:00');
INSERT INTO test (date) VALUES ('2016-11-02 18:00:00');
INSERT INTO test (date) VALUES ('2017-11-02 00:00:00');
INSERT INTO test (date) VALUES ('2017-11-02 06:00:00');
INSERT INTO test (date) VALUES ('2017-11-02 12:00:00');
INSERT INTO test (date) VALUES ('2017-11-02 18:00:00');
Run Query:
SELECT * FROM
(SELECT *,FLOOR(UNIX_TIMESTAMP(date)/((1440/2)*60))
AS timekey
FROM `test`
where date < '2017-11-02'
GROUP BY timekey) m
UNION
(SELECT * ,0 As timekey FROM `test`
where date > '2017-11-02');
The above should:
reduce all data points prior to 2017-11-02 which should give a result of 10.
Related
I'm trying to count the records in my "records" table and insert in results table but I just want to count today's records.
Below you will see some alternatives that I tried (I'm using MySQL), but I keep getting this error:
You have a syntax error in your SQL next to '' on line 2
INSERT INTO results (Data,total)
VALUES (now(), (SELECT COUNT(*) FROM records WHERE Data = now());
This SQL also causes an error:
INSERT INTO results (Data, total)
VALUES (now(), (SELECT COUNT(record.ID) AS day FROM record
WHERE date(Data) = date(date_sub(now(), interval 0 day));
and then
INSERT INTO resultS (Data,total)
VALUES (now(), (SELECT COUNT(*) FROM records
WHERE Data >= DATE_SUB(CURRENT_DATE(), INTERVAL 1 DAY));
And yet another attempt:
INSERT INTO results (Data, Total)
VALUES (now(), (SELECT COUNT(*) FROM records
WHERE DATE(Data)= CURRENT_DATE() - INTERVAL 1 DAY));
This is my sql config man:
CREATE TABLE `records`
(
`ID` char(23) NOT NULL,
`Name` varchar(255) NOT NULL,
`Total` int(255) NOT NULL,
`Data` date NOT NULL,
`QrCode` varchar(255) NOT NULL,
`City` varchar(255) NOT NULL,
`Device` varchar(255) NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
CREATE TABLE `results`
(
`id` int(11) NOT NULL,
`total` int(11) NOT NULL,
`Data` date DEFAULT NULL,
`grown` int(11) NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
You have defined grown column as not null so you cannot put there NULL.
My query works :
INSERT INTO results
VALUES (1, (SELECT COUNT(1) FROM records WHERE Data= now()), now(), 1);
You should define default value for grown column. Same situation you have with column id. You should define sequence for column id:
id NOT NULL AUTO_INCREMENT;
INSERT INTO results (Data, total)
SELECT CURRENT_DATE(), COUNT(*)
FROM records
WHERE DATE(Data) = CURRENT_DATE();
I have 2 tables.
First holds job details, second one the history of those job runs. First one also contains job period, per customer which is minimum time to wait before running next job for same customer. The time comparison needs to happen on started_on field of second table.
I need to find out the job ids to run next.
Schemas
job_details table
CREATE TABLE `job_details` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`customer_id` varchar(128) NOT NULL,
`period_in_minutes` int(11) unsigned NOT NULL,
`status` enum('ACTIVE','INACTIVE','DELETED') DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
job_run_history table
CREATE TABLE `job_run_history` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`job_id` int(10) unsigned NOT NULL,
`started_on` timestamp NULL DEFAULT NULL,
`status` enum('STREAMING','STREAMED','UPLOADING','UPLOADED','NO_RECORDS','FAILED') DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `fk_job_id` (`job_id`),
CONSTRAINT `fk_job_id` FOREIGN KEY (`job_id`) REFERENCES `job_details` (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Sample data for job_details table:
INSERT INTO `job_details` (`id`, `customer_id`, `period_in_minutes`, `status`)
VALUES
(1, 'cust1', 1, 'ACTIVE'),
(2, 'cust2', 1, 'ACTIVE'),
(3, 'cust3', 2, 'ACTIVE');
Sample data for job_run_history table:
INSERT INTO `job_run_history`(`job_id`, `started_on`, `status`)
VALUES
(1, '2021-07-01 14:38:00', 'UPLOADED'),
(2, '2021-07-01 14:37:55', 'UPLOADED');
Expected output (When run at 2021-07-01 14:38:56):
id
2,3
id => 1 did NOT get selected because the last job started within last 1 minute
id => 2 DID get selected because the last job started more than last 1 minute ago
id => 3 DID get selected because it has no run history
I have tried this, but this doesn't compare with max of start_time, hence, doesn't work:
select jd.id, max(jrh.started_on) from job_details jd
left join job_run_history jrh on jrh.job_id=jd.id
where
jd.status='ACTIVE'
and (jrh.status is null or jrh.status not in ('STREAMING','STREAMED','UPLOADING'))
and (jrh.`started_on` is null or jrh.`started_on` < date_sub(now(), interval jd.`period_in_minutes`*60 second))
group by jd.id;
MySql Version: 5.7.34
Any help please? Thanks in advance..
I'd prefer to use UNION ALL (it must be more fast than one complex query):
-- the subquery for the rows which have matched ones in 2nd table
SELECT t1.id
FROM job_details t1
JOIN job_run_history t2 ON t1.id = t2.job_id
WHERE t1.status = 'ACTIVE'
AND t2.status not in ('STREAMING','STREAMED','UPLOADING')
AND CURRENT_TIMESTAMP - INTERVAL t1.period_in_minutes MINUTE > t2.started_on
UNION ALL
-- the subquery for the rows which have no matched ones in 2nd table
SELECT id
FROM job_details t1
WHERE NOT EXISTS ( SELECT NULL
FROM job_run_history t2
WHERE t1.id = t2.job_id )
AND status = 'ACTIVE';
https://dbfiddle.uk/?rdbms=mysql_5.7&fiddle=8dcad95bf43ce711fdf40deda627e879
select jd.id from job_details jd
left join job_run_history jrh on jd.id= jrh.job_id
where jd.status = 'ACTIVE'
group by jd.id
having
max(jrh.started_on) < current_timestamp - interval max(jd.period_in_minutes) minute
or
max(jrh.id) is null
I'm not sure what's this filter about since you didn't explain it in your question so I didn't put it in the query: jrh.status not in ('STREAMING','STREAMED','UPLOADING'). However, I'm sure you can implement it in the query I posted.
I need to increment value unique row, use only one request
Not use select and after condition INSERT OR UPDATE
Table columns (date, servie) are used as unique values
Only one INSERT
For example
CREATE TABLE `log` (
`date` date NOT NULL,
`service` varchar(255) NOT NULL,
`count` int(11) NOT NULL,
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
I found a single solution, but it not work for my
INSERT INTO log (`date`,`service`,`count`) VALUES ('2020-12-1','amazon',count+1)
ON DUPLICATE KEY UPDATE `date`='2020-12-01',`service`='amazon',`count`=`count`+1;
Are there any other solutions?
First, define id_key as a unique key in the table (or as the primary key):
create table log (
id_key int primary key
dt date not null,
service varchar(255) not null,
cnt int not null,
) engine=innodb default charset=utf8;
Then you can do:
insert into t1 (id_key, dt, service, cnt)
values (1, '2020-12-1', 'amazon', 1)
on duplicate key update
dt = values(dt),
service = values(service),
cnt = cnt + 1;
This attempts to insert a new row in the table, with a cnt of of 1. If id_key exists already, then it is updated instead: the date and service are updated, and the count is incremented.
If you want to use the date and service as unique keys, then do reflect that in the create table statement:
create table log (
id_key int primary key
dt date not null,
service varchar(255) not null,
cnt int not null,
unique (dt, service)
) engine = innodb default charset = utf8;
And then, it does not really make sense to update these columns on duplicate keys, so:
insert into t1 (id_key, dt, service, cnt)
values (1, '2020-12-1', 'amazon', 1)
on duplicate key update cnt = cnt + 1;
please see the the test data bellow. I want to get the avgtime (=timeonsite/visits) and display as "xx:xx:xx" result in mysql. how can I get it?
SET FOREIGN_KEY_CHECKS=0;
-- ----------------------------
-- Table structure for `t`
-- ----------------------------
DROP TABLE IF EXISTS `t`;
CREATE TABLE `t` (
`id` int(11) NOT NULL auto_increment,
`timeOnsite` time default NULL,
`visits` int(11) default NULL,
PRIMARY KEY (`id`)
) ENGINE=MyISAM AUTO_INCREMENT=4 DEFAULT CHARSET=utf8;
-- ----------------------------
-- Records of t
-- ----------------------------
INSERT INTO `t` VALUES ('1', '04:05:30', '20');
INSERT INTO `t` VALUES ('2', '03:00:00', '10');
INSERT INTO `t` VALUES ('3', '00:01:30', '17');
You can use TIME_TO_SEC function to change xx:xx:xx format to seconds.
SELECT TIME_TO_SEC('00:01:30') / 17; # return 5.2941
And then through SEC_TO_TIME you can convert seconds to time back as below :
SELECT SEC_TO_TIME(TIME_TO_SEC('00:01:30') / 17); # return 00:00:05
Are sure that you calculate avgtime in such way?
If yes, mysql select below:
select id, timeOnsite,visits, SEC_TO_TIME(TIME_TO_SEC(timeOnsite)/visits) as avgtime
from t
OK so I have a table (in my MySQL database) as follows:
CREATE TABLE IF NOT EXISTS `funddata` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`ticker` varchar(128) NOT NULL,
`price_date` date NOT NULL,
`price` double NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 AUTO_INCREMENT=0 ;
And some sample data:
INSERT INTO funddata (ticker, price_date, price) VALUES ("tick1", '2013-06-01', 36.2);
INSERT INTO funddata (ticker, price_date, price) VALUES ("tick2", '2013-06-01', 14.7);
INSERT INTO funddata (ticker, price_date, price) VALUES ("tick3", '2013-06-01', 102.5);
INSERT INTO funddata (ticker, price_date, price) VALUES ("tick1", '2013-07-01', 38.7);
INSERT INTO funddata (ticker, price_date, price) VALUES ("tick2", '2013-07-01', 16.2);
Now let's say I want to add some more prices for tick1. If the price I want to add already exists in my table for that date then I want to update what's there with the new price, else I just want to insert it as a new record.
Does it make sense to make a unique index out of (ticker, price_date) given that no 2 records should share the same ticker and date? If so how would I do this and how would I make use of such an index.
CREATE UNIQUE INDEX index_name ON funddata(priceDate, price);
Then use INSERT... ON DUPLICATE KEY UPDATE
But still,this break the first normal form.http://en.wikipedia.org/wiki/First_normal_form