I have the following query, but after some time when users start putting in more and more items in the "ci_falsepositives" table, it gets really slow.
The ci_falsepositives table contains a reference field from ci_address_book and another reference field from ci_matched_sanctions.
How can I create a new query but still being able to sort on each field.
For example I can still sort on "hits" or "matches"
SELECT *, matches - falsepositives AS hits
FROM (SELECT c.*, IFNULL(p.total, 0) AS matches,
(SELECT COUNT(*)
FROM ci_falsepositives n
WHERE n.addressbook_id = c.reference
AND n.sanction_key IN
(SELECT sanction_key FROM ci_matched_sanctions)
) AS falsepositives
FROM ci_address_book c
LEFT JOIN
(SELECT addressbook_id, COUNT(match_id) AS total
FROM ci_matched_sanctions
GROUP BY addressbook_id) AS p
ON c.id = p.addressbook_id
) S
ORDER BY folder asc, wholename ASC
LIMIT 0,15
The problem has to be the SELECT COUNT(*) FROM ci_falsepositives sub-query. That sub-query can be written using an inner join between ci_falsepositives and ci_matched_sanctions, but the optimizer might do that for you anyway. What I think you need to do, though, is make that sub-query into a separate query in the FROM clause of the 'next query out' (that is, SELECT c.*, ...). Probably, that query is being evaluated multiple times - and that's what's hurting you when people add records to ci_falsepositives. You should study the query plan carefully.
Maybe this query will be better:
SELECT *, matches - falsepositives AS hits
FROM (SELECT c.*, IFNULL(p.total, 0) AS matches, f.falsepositives
FROM ci_address_book AS c
JOIN (SELECT n.addressbook_id, COUNT(*) AS falsepositives
FROM ci_falsepositives AS n
JOIN ci_matched_sanctions AS m
ON n.sanction_key = m.sanction_key
GROUP BY n.addressbook_id
) AS f
ON c.reference = f.addressbook_id
LEFT JOIN
(SELECT addressbook_id, COUNT(match_id) AS total
FROM ci_matched_sanctions
GROUP BY addressbook_id) AS p
ON c.id = p.addressbook_id
) AS s
ORDER BY folder asc, wholename ASC
LIMIT 0, 15
Related
This almost seems like a scope issue- the select statement in the subquery doesn't recognize table 'candidate':
SELECT
candidate.id AS id,
candidate.image AS image,
candidate.name AS name,
candidate.party AS party,
player.order AS player_order,
c_pcts.pct AS pct
FROM `candidate`
INNER JOIN players player ON player.candidate_id = candidate.id
INNER JOIN lineups lineup ON player.lineup_id = lineup.id
INNER JOIN (
SELECT
pct
FROM candidate_pcts p
INNER JOIN weekly_game game ON p.weekly_game_id = (
SELECT id FROM weekly_game ORDER BY date DESC LIMIT 1
) WHERE p.candidate_id = candidate.id
) c_pcts
WHERE lineup.id = '31'
ORDER BY player.order ASC
gives the error: "Unknown column 'candidate.id' in 'where clause'." If instead of "FROM candidate_pcts p" I put
FROM candidate_pcts p, candidate c
then it doesn't see 'p.weekly_game_id' ...huh?
Seems like I need to identify the 'candidate' table for the subquery somehow but everything I'm trying leads me only further astray. And I have tried a mess of things: order of the tables, explicitly identifying them everywhere i could think of, backticks. I should note that the nested subquery works like a charm. Here it is again:
SELECT
pct
FROM `candidate_pcts`
INNER JOIN weekly_game game ON candidate_pcts.weekly_game_id = (
SELECT id FROM weekly_game ORDER BY date DESC LIMIT 1
) WHERE candidate_pcts.candidate_id = '5'
with a hardcoded an id value there, of course. I can supply database structure if needed here, but this is long already. The 'weekly_game' table is simply a set of scores for each candidate each week and we only want the most recent week's score, thus the 'ORDER BY date DESC LIMIT 1' clause.
Thanks very much for your time.
Tables:
table candidate: {id, image, name, party}
table candidate_pcts: {id, candidate_id, pct, weekly_game_id}
table lineups: {id, date, user_id}
table players: {id,candidate_id,lineup_id,order}
table weekly_game: {id,date}
You are basically on the right track around the problem. In essence the nested sub-select does not know about candidate.id. It you break apart the query and just look at the sub-select in question:
SELECT
pct
FROM candidate_pcts p
INNER JOIN weekly_game game ON p.weekly_game_id = (
SELECT id FROM weekly_game ORDER BY date DESC LIMIT 1
) WHERE p.candidate_id = candidate.id
You can see there is NO reference whatsoever in that query to the candidate table other than in your where clause, thus this is an unknown column.
Since a subselect is, in essence, made before the outer select that references it, the subselect must be a standalone, executable query.
Thanks to all, especially Mike for that excellent explanation. What I did was restructured the query like so:
SELECT
candidate.id AS id,
candidate.image AS image,
candidate.name AS name,
candidate.party AS party,
player.order AS player_order,
pcts.pct AS pct
FROM `candidate`
INNER JOIN players player ON player.candidate_id = candidate.id
INNER JOIN lineups lineup ON player.lineup_id = lineup.id
LEFT JOIN (
SELECT
p.candidate_id AS pct_id, pct AS pct
FROM candidate_pcts p
INNER JOIN weekly_game game ON p.weekly_game_id = (
SELECT id FROM weekly_game ORDER BY date DESC LIMIT 1
)
) pcts
ON pct_id = candidate.id
WHERE lineup.id = '$lineup_id'
ORDER BY player.order ASC
I have written an sql statement that besides all the other columns should return the number of comments and the number of likes of a certain post. It works perfectly when I don't try to get the number of times it has been shared too. When I try to get the number of time it was shared instead it returns a wrong number of like that seems to be either the number of shares and likes or something like that. Here is the code:
SELECT
[...],
count(CS.commentId) as shares,
count(CL.commentId) as numberOfLikes
FROM
(SELECT *
FROM accountSpecifics
WHERE institutionId= '{$keyword['id']}') `AS`
INNER JOIN
account A ON A.id = `AS`.accountId
INNER JOIN
comment C ON C.accountId = A.id
LEFT JOIN
commentLikes CL ON C.commentId = CL.commentId
LEFT JOIN
commentShares CS ON C.commentId = CS.commentId
GROUP BY
C.time
ORDER BY
year, month, hour, month
Could you also tell me if you think this is an efficient SQL statement or if you would do it differently? thank you!
Do this instead:
SELECT
[...],
(select count(*) from commentLikes CL where C.commentId = CL.commentId) as shares,
(select count(*) from commentShares CS where C.commentId = CS.commentId) as numberOfLikes
FROM
(SELECT *
FROM accountSpecifics
WHERE institutionId= '{$keyword['id']}') `AS`
INNER JOIN account A ON A.id = `AS`.accountId
INNER JOIN comment C ON C.accountId = A.id
GROUP BY C.time
ORDER BY year, month, hour, month
If you use JOINs, you're getting back one result set, and COUNT(any field) simply counts the rows and will always compute the same thing, and in this case the wrong thing. Subqueries are what you need here. Good luck!
EDIT: as posted below, count(distinct something) can also work, but it's making the database do more work than necessary for the answer you want to end up with.
Quick fix:
SELECT
[...],
count(DISTINCT CS.commentId) as shares,
count(DISTINCT CL.commentId) as numberOfLikes
Better approach:
SELECT [...]
, Coalesce(shares.numberOfShares, 0) As numberOfShares
, Coalesce(likes.numberOfLikes , 0) As numberOfLikes
FROM [...]
LEFT
JOIN (
SELECT commentId
, Count(*) As numberOfShares
FROM commentShares
GROUP
BY commentId
) As shares
ON shares.commentId = c.commentId
LEFT
JOIN (
SELECT commentId
, Count(*) As numberOfLikes
FROM commentLikes
GROUP
BY commentId
) As likes
ON likes.commentId = c.commentId
i've a very confusionary quesry string that at this moment appears like this:
SELECT SQL_CALC_FOUND_ROWS m.*,
(
SELECT count(1) FROM votazioni_messaggi v
WHERE v.idMeggaggio=m.ID
) AS Votato,
AVG(voto) AS votazioneMedia,
n.nomeUtente,
FROM messages m
LEFT JOIN votazioni_messaggi v ON v.idMessaggio = m.ID
LEFT JOIN utenti n ON a.idUtente=n.ID
WHERE n.idUser='$idUser' AND m.Genre IN('animal', 'love')
GROUP BY m.ID,
LIMIT $partenza, 20
In other words, i've to select 20 messages at time that haves like genre "animals" and "love", for each of it retrieve info about user and calculate the vote average. Counting and grouping i can know if an user had already voted or not the message.
Now i've to set a last clause for ORDER messages BY insertion time (i've this data in messages.InsertionTime table).
The clause should be ORDER BY messages.InsertionTime but how i can know in what exact point i've to place it?
Try this:
SELECT SQL_CALC_FOUND_ROWS m.*,
(SELECT count(1) FROM votazioni_messaggi v
WHERE v.idMeggaggio=m.ID
) AS Votato,
AVG(voto) AS votazioneMedia,
n.nomeUtente,
FROM messages m
LEFT JOIN votazioni_messaggi v ON v.idMessaggio = m.ID
LEFT JOIN utenti n ON a.idUtente=n.ID
WHERE n.idUser='$idUser' AND m.Genre IN ('animal', 'love')
GROUP BY m.ID
ORDER BY m.InsertionTime
LIMIT $partenza, 20;
You had at least two problems. One was a comma at the end of the group by. The other was using the table name instead of the table alias.
im trying to get my query to group rows by month and year from the assignments table, and count the number of rows that has a certain value from the leads table. they are linked together as the assignments table has an id_lead field, which is the id of the row in the leads table.
d_new would be a count of the assignments for leads for the month whose website is newsite.com
d_subprime would be a count of the assignments for leads for the month whose website is not newsite.com
here are the tables being used:
`leads`
id (int)
website (varchar)
`assignments`
id_lead (int)
date_assigned (int)
heres my query which is not working:
SELECT
MONTHNAME(FROM_UNIXTIME(a.date_assigned)) as d_month,
YEAR(FROM_UNIXTIME(a.date_assigned)) as d_year,
(select COUNT(*) from leads where website='newsite.com' ) as d_new,
(select COUNT(*) from leads where website!='newsite.com') as d_subprime
FROM assignments as a
left join leads as l on (l.id = a.id_lead)
where id_dealership='$id_dealership2'
GROUP BY
d_month,
d_year
ORDER BY
d_year asc,
MONTH(FROM_UNIXTIME(a.date_assigned)) asc
$id_dealership is a variable containing a id of the dealership im trying to view the count for.
any help would be greatly appreciated.
You can sort of truncate your timestamps to months and use the obtained values for grouping, then derive the necessary date parts from them:
SELECT
YEAR(d_yearmonth) AS d_year,
MONTHNAME(d_yearmonth) AS d_month,
…
FROM (
SELECT
LAST_DAY(FROM_UNIXTIME(a.date_assigned)) as d_yearmonth,
…
FROM assignments AS a
LEFT JOIN leads AS l ON (l.id = a.id_lead)
WHERE id_dealership = '$id_dealership2'
GROUP BY
d_yearmonth
) AS s
ORDER BY
d_year ASC,
MONTH(d_yearmonth) ASC
Well, LAST_DAY() doesn't really truncate a timestamp, but it does turn all the values belonging to the same month into the same value, which is basically what we need.
And I guess the counts should be related to the rows you are actually selecting, which is not what your subqueries are. Something like this might do:
…
COUNT(d.website = 'newsite.com' OR NULL) AS d_new,
/* or: COUNT(d.website) - COUNT(NULLIF(d.website, 'newsite.com')) AS d_new */
COUNT(NULLIF(d.website, 'newsite.com')) AS d_subprime
…
Here's the entire query with all the modifications mentioned:
SELECT
YEAR(d_yearmonth) AS d_year,
MONTHNAME(d_yearmonth) AS d_month,
d_new,
d_subprime
FROM (
SELECT
LAST_DAY(FROM_UNIXTIME(a.date_assigned)) as d_yearmonth,
COUNT(d.website = 'newsite.com' OR NULL) AS d_new,
COUNT(NULLIF(d.website, 'newsite.com')) AS d_subprime
FROM assignments AS a
LEFT JOIN leads AS l ON (l.id = a.id_lead)
WHERE id_dealership = '$id_dealership2'
GROUP BY
d_yearmonth
) AS s
ORDER BY
d_year ASC,
MONTH(d_yearmonth) ASC
This should do the trick:
SELECT
YEAR(FROM_UNIXTIME(a.date_assigned)) as d_year,
MONTHNAME(FROM_UNIXTIME(a.date_assigned)) as d_month,
l.website,
COUNT(*)
FROM
assignments AS a
INNER JOIN leads AS l on (l.id = a.id_lead) /*are you sure, that you need a LEFT JOIN?*/
WHERE id_dealership='$id_dealership2'
GROUP BY
d_year, d_month, website
/*an ORDER BY is not necessary, MySQL does that automatically when grouping*/
If you really need a LEFT JOIN, be aware that COUNT() ignores NULL values. If you want to count those as well (which I can't imagine to make sense) write it like this:
SELECT
YEAR(FROM_UNIXTIME(a.date_assigned)) as d_year,
MONTHNAME(FROM_UNIXTIME(a.date_assigned)) as d_month,
l.website,
COUNT(COALESCE(l.id, 1))
FROM
assignments AS a
LEFT JOIN leads AS l on (l.id = a.id_lead)
WHERE id_dealership='$id_dealership2'
GROUP BY
d_year, d_month, website
Start with
SELECT
MONTHNAME(FROM_UNIXTIME(a.date_assigned)) as d_month,
YEAR(FROM_UNIXTIME(a.date_assigned)) as d_year,
SUM(IF(l.website='newsite.com',1,0) AS d_new,
SUM(IF(l.website IS NOT NULL AND l.website!='newsite.com',1,0) AS d_subprime
FROM assignments AS a
LEFT JOIN leads AS l ON l.id = a.id_lead
WHERE id_dealership='$id_dealership2'
GROUP BY
d_month,
d_year
ORDER BY
d_year asc,
MONTH(FROM_UNIXTIME(a.date_assigned)) asc
and work from here: The field id_dealership is neither in leads nor in assignments, so you need more work.
If you edit your question to account for id_dealership we might be able to help you further.
How could I count rows from a SELECT query as a value?
Such as
SELECT FUCNTIONIMLOOKINGFOR(SELECT * FROM anothertable) AS count FROM table;
So that count is an integer of how many rows the subquery SELECT * FROM anothertable returns.
EDIT
SELECT p.PostPID, p.PostUID, p.PostText, p.PostTime, u.UserUID, u.UserName, u.UserImage, u.UserRep,
(
SELECT COUNT(f.FlagTime)
FROM Flags as f
JOIN Posts as p
ON p.PostPID = f.FlagPID
) as PostFlags
FROM Posts AS p
JOIN Users AS u
ON p.PostUID = u.UserUID
ORDER BY PostTime DESC
LIMIT 0, 30
SELECT ( SELECT COUNT(id) FROM aTable ) as count FROM table
I assume your example is a truncated version of your actual query, so perhaps you should post what you are after to get a, possibly, more optimal query.
EDIT
Working directly from my brain, something like this should be more optimal.
SELECT p.PostPID, p.PostUID, p.PostText, p.PostTime, u.UserUID, u.UserName, u.UserImage, u.UserRep, COUNT(v.FlagTime) as postFlags
FROM Flags as f
JOIN Posts as p ON p.PostPID = f.FlagPID
JOIN Users AS u ON p.PostUID = u.UserUID
LIMIT 0, 30
GROUP BY p.PostPID
ORDER BY PostTime DESC
You can say
SELECT COUNT(*) FROM anothertable
which will return a numeric value, which you can use in another query, such as in the select list of another query, or as a condition in another query.
SELECT someVariable FROM table
WHERE (SELECT COUNT(*) FROM anotherTable) > 5
OR
SELECT someVariable, (SELECT COUNT(*) FROM anotherTable) as count FROM table