MySQL query with limited subqueries - mysql

Question

I'm trying to get some statistical data from a few tables. We have a users table, quiz table, a quiz question set table, and a quiz questions table. Each quiz has many sets, and each set has one or many questions. There's also a questions table which is where the question comes from (the quiz questions table links a question to question set, which then links to a quiz, which then links to the user). What I need is to see the number of questions answered correctly, and the number of questions answered, but only up to the past 50 questions. So if one user has answered 120 questions only the most recent 50 should be used in this query; if a user has answered 37 questions, then all of their questions should be used. I'd like to get this laid out so theres the user_id, questions_answered, questions_answered_correctly. I currently have this working, but I'm looking through each user and grabbing their 50 most recent questions, and with some additional tables limiting organization being joined on I have to do hundreds, if not thousands of these to get one statistical report.
I'm guessing I need to do a subquery somewhere to only pull the most recent questions from the user, but I'm not sure how a subquery like that would work. Here's what I have so far, but I'm sure I'm totally off on this. It executes, but incorrectly. Some of the results are over 50 when they shouldn't be:
SELECT users.id, (SELECT COUNT(grammar_quiz_questions.id) FROM `grammar_quiz_questions`
INNER JOIN `grammar_quiz_question_sets` ON `grammar_quiz_question_sets`.`id` = `grammar_quiz_questions`.`grammar_quiz_question_set_id`
INNER JOIN `grammar_quizzes` ON `grammar_quizzes`.`id` = `grammar_quiz_question_sets`.`grammar_quiz_id`
INNER JOIN `grammar_questions` ON `grammar_questions`.`id` = `grammar_quiz_questions`.`grammar_question_id`
WHERE (grammar_quiz_questions.finished is not null AND grammar_quizzes.user_id = users.id)
ORDER BY grammar_quiz_questions.finished DESC LIMIT 50) AS `questions_answered`, (SELECT COUNT(grammar_quiz_questions.id) FROM `grammar_quiz_questions`
INNER JOIN `grammar_quiz_question_sets` ON `grammar_quiz_question_sets`.`id` = `grammar_quiz_questions`.`grammar_quiz_question_set_id`
INNER JOIN `grammar_quizzes` ON `grammar_quizzes`.`id` = `grammar_quiz_question_sets`.`grammar_quiz_id`
INNER JOIN `grammar_questions` ON `grammar_questions`.`id` = `grammar_quiz_questions`.`grammar_question_id`
WHERE (grammar_quiz_questions.finished is not null AND grammar_quizzes.user_id = users.id AND grammar_quiz_question_sets.correct_on_first_attempt = 1)
ORDER BY grammar_quiz_questions.finished DESC LIMIT 50) AS `questions_answered_correctly`
FROM users
Thanks,
James

Answer 1

UPDATE:
Following update is not a complete answer to the question, but some nudges. I am not sure why you are querying on all these tables. are grammar_quiz_question_sets mutually exclusive subsets of grammar_quiz_questions? how about grammar_quizzes and grammar_questions, what is the set relation? Given that I don't know these answers, but you do, look at the code snippet following. I hope it guides you:
set #correct:=0;
select users.id, count(p.id), sum(if(r.correct_on_first_attempt = 1,1,0)) as correct
from grammar_quiz_questions p, grammar_quiz_question_sets r, users;
ORIGINAL:
I imagine you have a control and data access layer (java, php, python,etc) through which records are added and manipulated. Further, I imagine you need to grab statistics more than once in the lifetime of a user. Therefore, while you may need a query like yours to recalibrate once in a while -- if that will ever be necessary--, you need something less heady. Hence the following proposal.
1] create a statistics table:
create table statistics(
user_id int(11) not null, -- foreign key
questions_answered int(11) not null default 0,
questions_answered_correctly int(11) no null default 0
-- for primary key, you may use user_id or some auto record_id
)
2] the first time around, run your "heavy/administrative" query
3] subsequently, update the stats for a user after each quiz or each answered question. The idea here is that you will have that information in memory (i.e. in your programming layer) since you have to update the quiz table; during that time do some math to update the stats table. e.g. imagine java:
public void updateStats(int userId, int questions, int correct){
String query =
"insert into statistics(user_id,questions_answered,questions_answered_correctly) "+
"values("+userId+", "+questions+", "+correct+") "+
"on duplicate key update "+
"questions_answered=questions_answered+values(questions_answered), "+
"questions_answered_correctly = questions_answered_correctly + values(questions_answered_correctly)";
... //execute the statement
}
now for the "heavy" query, I am rewriting it below with a bit more clarity to encourage others to take a stab at it:
SELECT users.id,
(
SELECT COUNT(p.id)
FROM grammar_quiz_questions p, grammar_quiz_question_sets r, grammar_quizzes t, grammar_questions u
WHERE r.id = p.grammar_quiz_question_set_id
AND t.id = r.grammar_quiz_id
AND u.id = p.grammar_question_id
AND p.finished is not null
AND t.user_id = users.id
ORDER BY p.finished DESC LIMIT 50
) AS questions_answered,
(
SELECT COUNT(p.id)
FROM grammar_quiz_questions p, grammar_quiz_question_sets r, grammar_quizzes t, grammar_questions u
WHERE r.id = p.grammar_quiz_question_set_id
AND t.id = r.grammar_quiz_id
AND u.id = p.grammar_question_id
AND p.finished is not null
AND t.user_id = users.id
AND r.correct_on_first_attempt = 1
ORDER BY p.finished DESC LIMIT 50
) AS questions_answered_correctly
FROM users