Aggregate data use sql - mysql

This is the definition of the "exchange" table:
CREATE TABLE `exchange` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`rank` int(11) NOT NULL,
`name` varchar(255) NOT NULL,
`volume` varchar(255) NOT NULL,
`timestamp` datetime NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=75032 DEFAULT CHARSET=utf8;
About 209 records will store every 5 minutes.
How can I get this data structure?
rank, name, [volume,...](last 144 value), timestamp
I use this query:
SELECT `volume`
FROM `exchange`
WHERE `exchange`.`name` = 'binance'
ORDER BY `timestamp` DESC
LIMIT 144
Is there better way to get the data once? Thanks.

Frankly, your method might be the best method, particularly if you have an index on (name, timestamp).
You can try:
select e.volume
from exchange e
where e.timestamp >= (select e2.timestamp
from exchange e2
where e2.name = e.name
limit 1 offset 143
);
You can then aggregate the values as:
select e.name, sum(e.volume)
from exchange e
where e.timestamp >= (select e2.timestamp
from exchange e2
where e2.name = e.name
limit 1 offset 143
)
group by e.name;
Note: In MySQL 8+, this is much simpler using row_number():
select name, sum(volume)
from (select e.*,
row_number() over (partition by name order by timestamp desc) as seqnum
from exchange e
) e
where seqnum <= 144
group by name;

Related

How to get NULL at the top of the sort?

Two tables are defined:
CREATE TABLE `users` (
`user_id` mediumint(6) unsigned NOT NULL AUTO_INCREMENT,
`score` tinyint(1) unsigned DEFAULT NULL,
PRIMARY KEY (`user_id`)
);
CREATE TABLE `online` (
`user_id` mediumint(6) unsigned NOT NULL AUTO_INCREMENT,
`url` varchar(255) NOT NULL,
PRIMARY KEY (`user_id`)
);
How to combine the tables so that the result would be sorted by the score field from the largest to the smallest but at the top there were records with the value NULL?
This query does not sort the second sample:
(SELECT * FROM `online` JOIN `users` USING(`user_id`) WHERE `score` IS NULL)
UNION
(SELECT * FROM `online` JOIN `users` USING(`user_id`) WHERE `score` IS NOT NULL ORDER BY `score` DESC)
Use two keys in the sort:
SELECT *
FROM `online` o JOIN
`users`
USING (user_id)
ORDER BY (`score` IS NULL) DESC, Score DESC;
MySQL treats booleans as numbers in a numeric context, with "1" for true and "0" for false. So, DESC puts the true values first.
Incidentally, your version would look like it works if you used UNION ALL rather than UNION. However, it is not guaranteed that the results are in any particular order unless you explicitly have an ORDER BY.
The UNION incurs overhead for removing duplicates and in doing so rearranges the data.
Try:
select * from online join users using (user_id) order by ifnull(score, 10) desc;
You can use order by Nulls Last in the end of your sql to show nulls on the first.
You can try below -
select * from
(
SELECT *,1 as ord FROM `online` JOIN `users` USING(`user_id`) WHERE `score` IS NULL
UNION
SELECT *,2 FROM `online` JOIN `users` USING(`user_id`) WHERE `score` IS NOT NULL
)A ORDER BY ord asc,`score` DESC

How to count and order by max by other table in MySQL?

I'm having two table:
CREATE TABLE `apps` (
`id` INT(10) UNSIGNED NOT NULL AUTO_INCREMENT,
`name` VARCHAR(191) NOT NULL COLLATE 'utf8mb4_unicode_ci',
PRIMARY KEY (`id`)
)
COLLATE='utf8mb4_unicode_ci'
ENGINE=InnoDB
;
CREATE TABLE `downloads_stats` (
`app_id` INT(10) UNSIGNED NOT NULL,
`date` DATE NOT NULL,
`downloads` INT(10) UNSIGNED NOT NULL,
PRIMARY KEY (`app_id`, `date`),
)
COLLATE='utf8mb4_unicode_ci'
ENGINE=InnoDB
First is the list of my apps, the other stores the download number of each app. What is the best way to get 10 most downloaded apps ? I can do simple JOIN and then SUM and ORDER BY but is it the most efficient way?
Yes, I'd use LEFT JOIN with GROUP BY, e.g.:
SELECT a.id, a.name, SUM(ds.downloads) AS 'downloads'
FROM apps a LEFT JOIN downloads_stats ds ON a.id = ds.app_id
GROUP BY a.id, a.name
ORDER BY SUM(ds.downloads) DESC
LIMIT 10;
If you just want the app_id then JOIN is not needed, e.g.:
SELECT app_id, SUM(downloads) as 'downloaded'
FROM downloads_stats
GROUP BY app_id
ORDER BY SUM(downloads) DESC
Yes , you can use LEFT JOIN with GROUP BY
SELECT a.id , a.name , SUM(ds.downloads) AS 'downloads'
FROM apps a LEFT JOIN downloads_stats ds ON a.id = ds.app_id
GROUP BY a.id , a.name
ORDER BY downloads DESC
LIMIT 10;

sorting with Order By in mysql

I am using mysql as database and i have a table like the one below.
CREATE TABLE IF NOT EXISTS `logins` (
`id` int(255) NOT NULL AUTO_INCREMENT,
`userid` varchar(255) NOT NULL,
`date` varchar(255) NOT NULL,
`status` varchar(255) NOT NULL,
KEY `id` (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 AUTO_INCREMENT=346 ;
I want to sort mysql results with order by.The problem is when i use this sql it takes only the first recod of date. Which is an older date. I want the newest date. last login date of user.
SELECT * FROM `logins` WHERE `status`='valid' GROUP BY `userid` ORDER BY `date` DESC
Any suggestions?
To do this you use a sub query to get the latest record for each user id and then join that to the logins table to get the rest of the details
SELECT logins.*
FROM logins
INNER JOIN
(
SELECT userid, MAX(`date`) AS max_date
FROM `logins`
WHERE `status` = 'valid'
GROUP BY `userid`
) sub0
ON logins.userid = sub0.userid
AND logins.`date` = sub0.max_date
WHERE `status` = 'valid'
You almost had it. Assuming id and userId doesn't evolve from one login to another, asking the MAX date should give you the expected result.
SELECT id, userId, MAX(`date`) AS lastDate, 'valid'
FROM `logins`
WHERE `status`='valid'
GROUP BY `userid`
ORDER BY `lastDate` DESC
Please note that you would need a JOIN if there were data that change between logins in the table.

how to optimize mysql group by

table:
CREATE TABLE IF NOT EXISTS `l_not_200_page` (
`id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`server` tinyint(3) unsigned NOT NULL,
`domain` tinyint(3) unsigned NOT NULL,
`page` varchar(128) NOT NULL,
`query_string` varchar(384) NOT NULL,
`status` smallint(5) unsigned NOT NULL,
PRIMARY KEY (`id`),
KEY `idx_time_domain_status_page` (`time`,`domain`,`status`,`page`),
KEY `page` (`page`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 AUTO_INCREMENT=1 ;
explain:
EXPLAIN SELECT *
FROM `l_not_200_page`
WHERE TIME
BETWEEN TIMESTAMP( '2014-03-25' )
AND TIMESTAMP( '2014-03-25 23:59:59' )
AND domain =1
AND STATUS = 404
GROUP BY PAGE
1
SIMPLE
l_not_200_page
range
idx_time_domain_status_page
idx_time_domain_status_page
7
NULL
1
Using where; Using temporary; Using filesort
it's very slow, how to optimize ?
sql:
SELECT PAGE , COUNT( * ) AS cnt
FROM l_not_200_page
WHERE TIME
BETWEEN TIMESTAMP( '2014-03-26 12:00:00' )
AND TIMESTAMP( '2014-03-26 12:30:00' )
AND domain =1
AND STATUS = 499
GROUP BY PAGE ORDER BY cnt DESC
LIMIT 100
the daily amount of data about 900w
Change the index to:
create index `idx_domain_status_time_page` on l_not_200_page(`domain`, `status`, `time`, `page`)
When MySQL uses an index for a where clause, the best index has all the fields in equality comparisons followed by one with an inequality, such as between. With time as the first element, it doesn't use the index for domain and status (well, it uses an index scan instead of a direct lookup).
For further optimization, you can get rid of the group by by choosing one row per page:
SELECT lp.* FROM l_not_200_page lp WHERE TIME BETWEEN TIMESTAMP( '2014-03-25' ) AND TIMESTAMP( '2014-03-25 23:59:59' ) AND
domain = 1 AND STATUS = 404 AND
NOT EXISTS (select 1
from l_not_200_page lp2
where lp2.page = lp.page and
lp2.domain = 1 and lp2.status = 404 and
lp2.TIME BETWEEN TIMESTAMP('2014-03-25) AND TIMESTAMP('2014-03-25 23:59:59') AND
lp2.id > lp.id
)
For this, an additional index on (page, domain, status, time) would help.

Need help creating SQL query

I have thus two tables:
CREATE TABLE `workers` (
`id` int(7) NOT NULL AUTO_INCREMENT,
`number` int(7) NOT NULL,
`percent` int(3) NOT NULL,
`order` int(7) NOT NULL,
PRIMARY KEY (`id`)
);
CREATE `data` (
`id` bigint(15) NOT NULL AUTO_INCREMENT,
`workerId` int(7) NOT NULL,
PRIMARY KEY (`id`)
);
I want to return the first worker (order by order ASC) that his number of rows in the table data times percent(from table workers) /100 is smaller than number(from table workers.
I have tried this query:
SELECT workers.id, COUNT(data.id) AS `countOfData`
FROM `workers` as workers, `data` as data
WHERE data.workerId = workers.id
AND workers.percent * `countOfData` < workers.number
LIMIT 1
But I get the error:
#1054 - Unknown column 'countOfData' in 'where clause'
This should work:
SELECT A.id
FROM workers A
LEFT JOIN (SELECT workerId, COUNT(*) AS Quant
FROM data
GROUP BY workerId) B
ON A.id = B.workerId
WHERE (COALESCE(Quant,0) * `percent`)/100 < `number`
ORDER BY `order`
LIMIT 1
You could calculate the number of rows per worker in a subquery. The subquery can be joined to the worker table. If you use a left join, a worker with no data rows will be considered:
select *
from workers w
left join
(
select workerId
, count(*) as cnt
from data
group by
workerId
) d
on w.id = d.workerId
where coalesce(d.cnt, 0) * w.percent / 100 < w.number
order by
w.order
limit 1