Robust SQL query on demographics data set - mysql

Question

I have a rather complex SQL Server query (at least to me) to write on a demographics data set. I need to figure out how many respondents in the system mathc a specific demographic.
I have 2 main tables. I will list the relevant columns. Assume there are unique ID's on each row.
Table Respondents:
[RespondentID] [SystemEntryDate]
Table RespondentProfiles:
[QuestionID] [AnswerID]
The respondent ID on Respondents links to RespondentProfiles. For each question answered, a row is created. The question id corresponds to a specific question (say gender, ethnicity, state, and car ownership) and the answer id means something different depending on the question. Like 1 is male and 2 is female, or 1 might be white, 2 hispanic, 3 pacific islander, and so on.
I also have a table called Conditions. The conditions table looks like this:
[ConditionSetID] [QuestionID] [AnswerID]
The condition set id links to the conditions together into a collection of conditions. So i can pass a condition set id to the query, and it will return a count of how many respondents meet that criteria, as well as the min and max dates from that set.
My query will look something like this:
create procedure query
#ConditionSetID int
as
select count(distinct r.ID) as Respondents,
min(r.SystemEntryDate) as EarliestDate,
max(r.SystemEntryDate) as LatestDate
from Respondents r
join RespondentProfiles rp
on r.ID = rp.RespondentID
join Conditions c
on c.ConditionSetID = #ConditionSetID
and c.QuestionID = rp.QuestionID
where rp.QuestionID = c.QuestionID
and rp.Condition = c.AnswerID
As an example, I might have a respondent profiles table like this
[RespondentID] [QuestionID] [AnswerID]
10001 1 (gender) 1 (male)
10001 2 (ethnicity) 1 (white)
10001 3 (car) 23 (lexus)
10002 1 (gender) 2 (female)
10002 2 (ethnicity) 2 (black)
10002 3 (car) 24 (buick)
10003 1 (gender) 2 (female)
10003 2 (ethnicity) 1 (white)
10003 3 (car) 5 (honda)
10004 1 (gender) 1 (male)
10004 2 (ethnicity) 2 (black)
10004 3 (car) 24 (buick)
And if I pick a specific condition set, the rows id have might be like:
[QuestionID] [AnswerID]
1 (gender) 2 (female)
2 (ethnicity) 2 (black)
3 (car) 24 (buick)
This would be asking for all the black females who own a buick, which should give em a count of 1.
Or I could have:
[QuestionID] [AnswerID]
3 (car) 23 (lexus)
3 (car) 24 (buick)
This is asking for everyone who owns a buick or lexus, which would be 3 people.
And then as a final example:
[QuestionID] [AnswerID]
2 (ethnicity) 2 (black)
3 (car) 23 (lexus)
3 (car) 24 (buick)
This is asking for everyone who is black and owns a lexus or everyone who is black and owns a buick, which would be 2 people.
I know this isn't horribly complicated, but it is the most complex thing I've attempted yet, and any help would be greatly appreciated. I'm having a lot of trouble figuring out how to set up the where clause, and even general direction would be appreciated. There are also about 800,000 records in the respondentprofiles table, so it must be efficient.
The where clause I have set up isn't quite correct, because it will only get the records as if the different questions are being or'd together as opposed to and'ed. So it will return a row for that respondent even if only one answer matches, which is wrong. A particular respondent must meet all the conditions in the condition set to be selected.
Perhaps I need to select into a temp table question at a time or something? Or use some sort of grouping? I am just really confused on where to go with this. I hope I have provided enough information to adequately demonstrate my dilemma.

Answer 1

The examples below show how to get the respondent IDs of respondents who answered:
To question A, Yes
To question B, No
TO question C, Yes
Assuming you are actually using SQL server (you tagged both mysql and sql server in your question), you can use:
select id
from RespondentProfiles
where QuestionID = 'a'
and AnswerID = 'yes'
intersect
select id
from RespondentProfiles
where QuestionID = 'b'
and AnswerID = 'no'
intersect
select id
from RespondentProfiles
where QuestionID = 'c'
and AnswerID = 'yes'
Or if you are using MySQL you can use:
select id
from RespondentProfiles x
where QuestionID = 'a'
and AnswerID = 'yes'
join (select id
from RespondentProfiles
where QuestionID = 'b'
and AnswerID = 'no') y
on x.id = y.id
join (select id
from RespondentProfiles
where QuestionID = 'c'
and AnswerID = 'yes') z
on y.id = z.id
Just to add to my answer what I put in the comments - there is no need for your conditions table. You do not need to have such a table in order to query for respondents who answers 2+ questions a certain way. You can use inline views and/or subqueries to accomplish that. (or in the case of sql server, the intersect set operator)