select count(*) from bill limit 100000;
mysql> select count(*) from `bill` limit 100000;
+----------+
| count(*) |
+----------+
| 47497305 |
+----------+
1 row in set
limit limits the number of rows outputted in the result set, not the number of rows that are processed.
Therefore it doesn't have any impact on queries like count(*) .
To achieve this you would have to wrap query into another sub select. Although such query doesn't make too much sense:
SELECT COUNT(*) FROM (
SELECT * FROM bill LIMIT 100000
) t
Related
I have a table that has two fields - id and counter. I want to sort all records by the counter field and then pick one record by its id and find out where in the entire table it is positioned by its counter. For example:
id
counter
1
1
2
5
3
6
4
0
5
4
the ASC order by counter would be: 4, 1, 5, 2, 3.
And say I want to get the position for id=2, the value i am looking for is 4 because it is forth in the result set.
I have found some solutions that use ROW_NUMBER approach but those all seem to work with entire results. I just want this one specific value for specific row by its id.
I was trying to use a simpler solution that will simply store the rank in the table alongside the counter field by updating the table when counter changes:
UPDATE mytable
SET counter_rank = ( SELECT COUNT(*)
FROM mytable
WHERE counter >= ( SELECT counter
FROM mytable
WHERE id = ?
GROUP BY counter)
)
WHERE id = ?
which is nice solution but it does not work because if another counter changes, rank for all rows must be calculated, which is not a good idea for my case. So I have to look at the dynamic solution with row number approach.
Also grouping by the counter must be taken into account - multiple rows may share the position in the rankings by having the same counter value. so more than one row can return the same number/position.
If you are getting multiple rows with their ranks, I would use ROW_NUMBER as you discovered, but in a subquery. And then select the desired rows from that subquery. But if you are getting just one row, this is most easily done with a join counting previous rows:
select mytable.id, mytable.counter, count(t2.id) counter_rank
from mytable
left join mytable t2 on t2.counter < mytable.counter
where mytable.id=2
(add 1 if you want a 1-based counter, not a 0-based counter)
fiddle
SELECT id, counter,
ROW_NUMBER() OVER (ORDER BY counter) position
FROM mytable
/* ORDER BY {needed expression} */
;
For specific row:
WITH cte AS (
SELECT id, counter,
ROW_NUMBER() OVER (ORDER BY counter) position
FROM mytable
)
SELECT * FROM cte WHERE id = {needed value}
;
The query is not deterministic if counter is not unique, you must expand sorting expression in this case. For example, till ORDER BY couner, id.
And if your Version hasent window function you can use a query like this:
SELECT FIND_IN_SET('2', GROUP_CONCAT(id ORDER BY counter)) AS position
FROM pos
ORDER BY counter;
sample
MariaDB [test]> select * from pos;
+----+---------+
| id | counter |
+----+---------+
| 1 | 1 |
| 2 | 5 |
| 3 | 6 |
| 4 | 0 |
| 5 | 4 |
+----+---------+
5 rows in set (0.000 sec)
MariaDB [test]> SELECT FIND_IN_SET('2', GROUP_CONCAT(id ORDER BY counter)) AS position
-> FROM pos
-> ORDER BY counter;
+----------+
| position |
+----------+
| 4 |
+----------+
1 row in set (0.000 sec)
MariaDB [test]>
I have a table with 450000 records at MySQL.
The query below cost nearly 3s. Is it possible to lower the time cost?
SELECT * FROM table order by id desc limit 400000, 8000
Assuming id is indexed, then there's not a lot you can do - apart from what's been suggested above. That said, I'm surprised it's soo slow...
SELECT COUNT(*) FROM my_table;
+----------+
| COUNT(*) |
+----------+
| 450000 |
+----------+
1 row in set (0.12 sec)
SELECT * FROM my_table ORDER BY i DESC LIMIT 400000,8000;
...
8000 rows in set (0.20 sec)
I need this query to return results when the count is 0 instead of just producing an empty set. How can I adjust my query so that it produces the following table with a '0' for the count and the appropriate date?
mysql> select count(test.id), date(insert_datetime) date
from db.table test where date(insert_datetime)='2015-08-17'
group by date(insert_datetime);
+--------------+------------+
| count(test.id) | date |
+--------------+------------+
| 42 | 2015-08-17 |
+--------------+------------+
1 row in set (0.14 sec)
mysql> select count(test.id), date(insert_datetime) date
from db.table test where date(insert_datetime)='2015-08-16'
group by date(insert_datetime);
Empty set (0.00 sec)
This should do it:
SELECT theDate AS `date`, IFNULL(subC.theCount, 0) AS `theCount`
FROM (SELECT DATE(20150817) AS `theDate`) AS subD
LEFT JOIN (
SELECT COUNT(test.id) AS theCount, DATE(insert_datetime) AS `theDate`
FROM db.table AS test
WHERE insert_datetime BETWEEN 20150817000000 AND 20150817235959
GROUP BY theDate
) AS subC
USING (theDate)
;
As another user hinted in a now deleted comment: if you are going to need this for a date range, an "all dates" table may come in more handy than the subD subquery; making a SELECT DATE(X) UNION SELECT DATE(Y) UNION SELECT DATE(Z) UNION SELECT DATE(etc...) subquery gets ridiculous fairly quickly.
I am currently experiencing an extremely slow query when using group by and order by. I have an inclination that the indexes are not being used because group by is on a separate column then order by
sqlFiddle
Foo Table Structure
id -> pk (indexed)
bar_id -> foreign key (indexed)
data -> varchar
created_at -> timestamp (indexed)
Here is the query:
SELECT * FROM foo GROUP BY bar_id ORDER BY created_at DESC
I am basically trying to get the most recent records for each bar_id. However this is taking up to 11 seconds to finish. Is there a better way to do this type of query?
SELECT COUNT(*) FROM foo;
+----------+
| COUNT(*) |
+----------+
| 98304 |
+----------+
1 row in set (0.03 sec)
SELECT x.*
FROM foo x
JOIN
( SELECT bar_id
, MAX(created_at) max_created_at
FROM foo
GROUP
BY bar_id
) y
ON y.bar_id = x.bar_id
AND y.max_created_at = x.created_at;
531 rows in set (0.01 sec)
Note: I've modified your schema slightly.
http://sqlfiddle.com/#!2/a6296/2
I did a performance profiling on my database with the slow query log. It turned out this is the number one annoyance:
UPDATE
t1
SET
v1t1 =
(
SELECT
t2.v3t2
FROM
t2
WHERE
t2.v2t2 = t1.v2t1
AND t2.v1t2 <= '2012-04-24'
ORDER BY
t2.v1t2 DESC,
t2.v3t2 DESC
LIMIT 1
);
The subquery itself is already slow. I tried variations with DISTINCT, GROUP BY and more subqueries but nothing performed below 4 seconds. For example the following query
SELECT v2t2, v3t2
FROM t2
WHERE t2.v1t2 <= '2012-04-24'
GROUP BY v2t2
ORDER BY v1t2 DESC
takes:
mysql> SELECT ...
...
69054 rows in set (5.61 sec)
mysql> EXPLAIN SELECT ...
+----+-------------+-------------+------+---------------+------+---------+------+---------+----------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------------+------+---------------+------+---------+------+---------+----------------------------------------------+
| 1 | SIMPLE | t2 | ALL | v1t2 | NULL | NULL | NULL | 5203965 | Using where; Using temporary; Using filesort |
+----+-------------+-------------+------+---------------+------+---------+------+---------+----------------------------------------------+
mysql> SHOW CREATE TABLE t2;
...
PRIMARY KEY (`v3t2`),
KEY `v1t2_v3t2` (`v1t2`,`v3t2`),
KEY `v1t2` (`v1t2`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8
SELECT COUNT(*) FROM t1;
+----------+
| COUNT(*) |
+----------+
| 77070 |
+----------+
SELECT COUNT(*) FROM t2;
+----------+
| COUNT(*) |
+----------+
| 5203965 |
+----------+
I am trying to fetch the newest entry (v3t2) and its parent (v2t2). Should not be that big of a deal, should it? Does anyone have any advice which knobs I should turn? Any help or hint is greatly appreciated!
This should be a more appropriate SELECT statement:
SELECT
t1.v2t1,
(
SELECT
t2.v3t2
FROM
t2
WHERE
t2.v2t2 = t1.v2t1
AND t2.v1t2 <= '2012-04-24'
ORDER BY
t2.v1t2 DESC,
t2.v3t2 DESC
LIMIT 1
) AS latest
FROM
t1
Your ORDER BY ... LIMIT 1 is forcing database to perform a full scan of the table to return only 1 row. It looks like very much as a candidate for indexing.
Before you build the index, check the fileds selectivity by running:
SELECT count(*), count(v1t2), count(DISTINCT v1t2) FROM t2;
If you're having high number of non-NULL values in your column and number of distinct values is more then 40% of the non-NULLs, then building index is a good thing to go.
If index provides no help, you should analyze the data in your columns. You're using t2.v1t2 <= '2012-04-24' condition, which, in the case you have a historical set of records in your table, will give nothing to the planner, as all rows are expected to be in the past, thus full scan is the best choice anyway. Thus, indexe is useless.
What you should do instead, is think how to rewrite your query in a way, that only a limited subset of records is checked. Your construct ORDER BY ... DESC LIMIT 1 shows that you probably want the most recent entry up to '2012-04-24' (including). Why don't you try to rewrite your query to a something like:
SELECT v2t2, v3t2
FROM t2
WHERE t2.v1t2 => date_add('2012-04-24' interval '-10' DAY)
GROUP BY v2t2
ORDER BY v1t2 DESC;
This is just an example, knowing the design of your database and nature of your data more precise query can be built.
I would take a look at indexes that are built for the sub-select t2. You should have a index for v2t2 and possibly one for v1t2, and v3t2 because of the ordering. The index should reduce the time the sub select has to go looking for the results before using them in your update query.
Does this work any better? Gets rid of one of the sorts and groups by the key being used.
UPDATE
t1
SET
v1t1 =
(
SELECT
MAX(t2.v3t2)
FROM
t2
WHERE
t2.v2t2 = t1.v2t1
AND t2.v1t2 <= '2012-04-24'
GROUP BY t2.v1t2
ORDER BY t2.v1t2 DESC
LIMIT 1
);
Alternate Version
UPDATE `t1`
SET `v1t1` = (
SELECT MAX(`t2`.`v3t2`)
FROM `t2`
WHERE `t2`.`v2t2` = `t1`.`v2t1`
AND `t2`.`v1t2` = (
SELECT MAX(`t2`.`v1t2`)
FROM `t2`
WHERE `t2`.`v2t2` = `t1`.`v2t1
AND `t2`.`v1t2` <= '2012-04-24'
LIMIT 1
)
LIMIT 1
);
And add this index to t2:
KEY `v2t2_v1t2` (`v2t2`, `v1t2`)