I have 10,000 users on registrations table and want to limit this query to 3500 users from my application. But when I investigate the logs , sometimes it counts more than 3500. I can not understand why that query returns more than limit:
select count(*)
from registrations
where (selectedtime IS NULL
AND expirationtime < NOW()
)
LIMIT 3500;
I tried manually on DB and saw sometimes more than 3500
Your query only returns 1 row, which is less than 3500.
If you want to limit the number of rows that are being counted, you need to put that in a subquery.
SELECT COUNT(*)
FROM (
SELECT 1
FROM registrations
WHERE selectedtime IS NULL AND expirationtime < NOW()
LIMIT 3500) AS x
A SELECT statement with COUNT returns the number of rows retrieved by the SELECT statement.
For performance reasons, the desired result is to limit that count.
Including a LIMIT clause in the SELECT statement will not work since it only restricts the number of rows returned, which is always one.
The solution, what I call “Limited-Count”, is done by limiting a non-count SELECT statement and wrapping it in COUNT(*).
For example:
If your count statement looks like
select count(*) from registrations where (selectedtime IS NULL AND expirationtime < NOW()) LIMIT 3500;
You can limit the results by replacing the query into:
SELECT COUNT(*) AS total
FROM (
SELECT 1
FROM registrations
WHERE selectedtime IS NULL AND expirationtime < NOW()
LIMIT 3500) AS x
If you need to know how many rows your non-count SELECT statement would have returned without the LIMIT, you could use the information function, FOUND_ROWS(). It will fetch the total rows number without running the statement again.
This does what you thought you were doing:
select count(*)
from registrations
where (selectedtime IS NULL
AND expirationtime < NOW()
)
LIMIT ROWS EXAMINED 3500;
(Available since MariaDB 5.5.21 - see https://mariadb.com/kb/en/library/limit-rows-examined/ )
Related
I've been tasked with converting a Rails app from MySQL to Postgres asap and ran into a small issue.
The active record query:
current_user.profile_visits.limit(6).order("created_at DESC").where("created_at > ? AND visitor_id <> ?", 2.months.ago, current_user.id).distinct
Produces the SQL:
SELECT visitor_id, MAX(created_at) as created_at, distinct on (visitor_id) *
FROM "profile_visits"
WHERE "profile_visits"."social_user_id" = 21
AND (created_at > '2015-02-01 17:17:01.826897' AND visitor_id <> 21)
ORDER BY created_at DESC, id DESC
LIMIT 6
I'm pretty confident when working with MySQL but I'm honestly new to Postgres. I think this query is failing for multiple reasons.
I believe the distinct on needs to be first.
I don't know how to order by the results of max function
Can I even use the max function like this?
The high level goal of this query is to return the 6 most recent profile views of a user. Any pointers on how to fix this ActiveRecord query (or it's resulting SQL) would be greatly appreciated.
The high level goal of this query is to return the 6 most recent
profile views of a user.
That would be simple. You don't need max() nor DISTINCT for this:
SELECT *
FROM profile_visits
WHERE social_user_id = 21
AND created_at > (now() - interval '2 months')
AND visitor_id <> 21 -- ??
ORDER BY created_at DESC NULLS LAST, id DESC NULLS LAST
LIMIT 6;
I suspect your question is incomplete. If you want:
the 6 latest visitors with their latest visit to the page
then you need a subquery. You cannot get this sort order in one query level, neither with DISTINCT ON, nor with window functions:
SELECT *
FROM (
SELECT DISTINCT ON (visitor_id) *
FROM profile_visits
WHERE social_user_id = 21
AND created_at > (now() - interval '2 months')
AND visitor_id <> 21 -- ??
ORDER BY visitor_id, created_at DESC NULLS LAST, id DESC NULLS LAST
) sub
ORDER BY created_at DESC NULLS LAST, id DESC NULLS LAST
LIMIT 6;
The subquery sub gets the latest visit per user (but not older than two months and not for a certain visitor21. ORDER BY must have the same leading columns as DISTINCT ON.
You need the outer query to get the 6 latest visitors then.
Consider the sequence of events:
Best way to get result count before LIMIT was applied
Why NULLS LAST? To be sure, you did not provide the table definition.
PostgreSQL sort by datetime asc, null first?
Is it possible to get comma separated value of first n (say 10 rows of a column) rows using Mysql?
I have a query to get data greater than CURDATE(). And it will return more than 100 rows of result. What I want is, GROUP_CONCAT the first 10 rows of result.
This is my query:
SELECT GROUP_CONCAT(user_id) AS userids
FROM user_tasks
WHERE due_date > CURDATE() LIMIT 10;
am getting entire rows. I need first 10 rows only
Thanks
Use subquery:
SELECT
GROUP_CONCAT(user_id) AS userids
FROM
(SELECT
user_id
FROM
user_tasks
WHERE due_date > CURDATE()
LIMIT 10) AS users
You need to use a sub query to impose the limit, like this:
SELECT GROUP_CONCAT(sub_query.user_id) AS userids
FROM
(
SELECT user_id
FROM user_tasks
WHERE due_date > CURDATE()
LIMIT 10
) sub_query
I want to do a MySQL Query which selects a given number of Rows from a single table from a given offset like
SELECT * FROM table
WHERE timestamp < '2011-11-04 09:01:05'
ORDER BY timestamp DESC
LIMIT 100
My problem is that i always want all rows within a day if one row of a day will be included in the result.
It would be no problem to have a result with e.g. 102 rows instead of 100
Can i realize this with a single SQL statement?
Thanks for your help!
This seems to work on my system:
SELECT UserID, Created
FROM some_user
WHERE Created < '2011-11-04 09:10:11'
AND Created >= (
SELECT DATE(Created) -- note: DATE() strips out the time portion from datetime
FROM some_user
WHERE Created < '2011-11-04 09:10:11'
ORDER BY Created DESC
LIMIT 99, 1 -- note: counting starts from 0 so LIMIT 99, 1 returns 100th row
)
ORDER BY Created DESC
-- 0 rows affected, 102 rows found. Duration for 1 query: 0.047 sec.
There might be a faster alternative.
If I understand your question correctly, you're intrested in retrievieng 100 rows, + any rows that are on the same day as ones already retrieved. You can do this using a subquery:
SELECT table.*
FROM table, (
SELECT DISTINCT day
FROM (
SELECT TO_DAYS(timestamp) day
FROM table
WHERE timestamp < :?
LIMIT 100
)
) days
WHERE TO_DAYS(table.timestamp) = days.day
ORDER BY timestamp
Exclude the time part in the query and remove the LIMIT.
SELECT * FROM table
WHERE timestamp < '2011-11-04 00:00:00'
ORDER BY timestamp DESC
I'm trying to get a count of records matching certain criteria within a subset of the total records. I tried (and assumed this would work)
SELECT count(*)
FROM records
WHERE status = 'ADP'
LIMIT 0,10
and I assumed this would tell me how many records of status ADP were in that set of 10 records. It doesn't - it returns, in this case 30, which is the total number of ADP records in the table.
How do I just count up the records matching my criteria including the limit?
SELECT count(*)
FROM ( SELECT records
FROM table
WHERE status = 'ADP'
LIMIT 0,10
)
select count(*) from (select * from records where status='ADP' limit 0,10) as t;
If SELECT SUM(amount) FROM transactions ORDER BY order LIMIT 0, 50 sums the amount field for the first 50 records in a table, how do a sum all records after the first 50? In other words, I'd like to do something like SELECT SUM(amount) from transactions ORDER BY order LIMIT 50, *, but that doesn't work.
SELECT SUM(amount)
FROM (
SELECT amount
FROM transactions
ORDER BY
order
LIMIT 50, 1000000000000
) q
Note that your original query:
SELECT SUM(amount)
FROM transactions
ORDER BY
order
LIMIT 0, 50
does not do what you probably think it does. It is synonymous to this:
SELECT a_sum, order
FROM (
SELECT SUM(amount) AS a_sum, order
FROM transactions
) q
ORDER BY
order
LIMIT 0, 50
The inner query (which would normally fail in any other engine but works in MySQL due to its GROUP BY extension syntax) returns only 1 records.
ORDER BY and LIMIT are then applied to that one aggregated record, not to the records of transactions.
The documentation advices to use an incredible large number as second parameter to LIMIT:
To retrieve all rows from a certain offset up to the end of the result set, you can use some large number for the second parameter. This statement retrieves all rows from the 96th row to the last:
SELECT * FROM tbl LIMIT 95,18446744073709551615;
There is probably a more efficient way, but you could run a count query first, to retrieve total # of rows in your table:
SELECT count(*) FROM transactions
Stuff that into a variable and use that variable as your second argument for LIMIT. You could probably do this as a nested mysql query.