query which creates missing rows based on anther table - mysql

I have many forms that users fill out. Each form contains a list of questions. In this first table is the form id and the id's of the questions.
form_id | question_id
1 | 1
1 | 2
1 | 3
2 | 4
2 | 5
This table has two forms one which has 3 questions and the other 2. I have a second table which has the answers that the users have given for the questions.
user_id | form_id | question_id | answer
476 | 1 | 1 | "answer1"
476 | 1 | 3 | "answer2"
693 | 1 | 1 | "answer3"
693 | 1 | 2 | "answer4"
235 | 2 | 5 | "answer5"
In this example, 2 users have filled out form 1 and 1 user has filled in form 2. But none have filled in all the questions. Is it possible to write a query which combines the two tables and will give me the answers that the user have given including the questions that they didn't answer? I'd like the results to look like this.
user_id | form_id | question_id | answer
476 | 1 | 1 | "answer1"
476 | 1 | 2 | NULL
476 | 1 | 3 | "answer2"
693 | 1 | 1 | "answer3"
693 | 1 | 2 | "answer4"
693 | 1 | 3 | NULL
235 | 2 | 4 | NULL
235 | 2 | 5 | "answer5"
The problem that I have when I use a left join like this
select * from template t
left join answers a on a.template_id = t.template_id
AND a.question_id = t.question_id
AND t.template_id = t.template_id;
is that the row that results is missing user_id.

Yes, the specified result can be returned by a query.
One way to achieve this is a join to an inline view, and an "outer join" operation to the second table.
The "trick" is getting a distinct list of user_id and form_id from the second table, using a query, for example:
SELECT user_id, form_id
FROM second_table
GROUP BY user_id, form_id
And then using that query as an inline view (wrapping it in parens, assigning a table alias, and referencing it like it was a table in an outer query.
All that's required after that is an "outer join" to the second table.
For example:
SELECT r.user_id
, q.form_id
, q.question_id
, a.answer
FROM first_table q
JOIN ( SELECT p.user_id, p.form_id
FROM second_table p
GROUP BY p.user_id, p.form_id
) r
ON r.form_id = q.form_id
LEFT
JOIN second_table a
ON a.user_id = r.user_id
AND a.form_id = r.form_id
AND a.question_id = q.question_id
ORDER
BY r.user_id
, q.form_id
, q.question_id
Note that the keyword "LEFT" specifies an outer join operation, returning all rows from the left side, along with matching rows from the right side. A typical "inner" join would exclude rows that didn't find a matching row from the table on the right side.

use
left join
something like:
select * from table1 left join table2 on table1.form_id= table2.form_id

Related

Select all items and count in related table by criteria

I have tables Match and Reaction as following:
REACTION
+----------+----------+----------+----------+
| user_id | game_id | item_id | reaction |
+----------+----------+----------+----------+
| 1 | 1 | 1 | 1 |
| 1 | 1 | 2 | 1 |
| 2 | 1 | 1 | 1 |
| 2 | 1 | 2 | 0 |
+----------+----------+----------+----------+
MATCH:
+----------+----------+
| game_id | item_id |
+----------+----------+
| 1 | 1 |
| 1 | 2 |
+----------+----------+
Now I want (if possible without subqueries) to select ALL item_ids from MATCH table AND count of rows where field reaction in table Reaction is equal to 1 for user with id = 2. For example, for defined tables I want to get following results:
+----------+----------+
| item_id | count |
+----------+----------+
| 1 |  1 |
| 2 | 0 |
+----------+----------+
I've tried something like
SELECT match.item_id, COUNT(reaction.user_id) as c
FROM match
LEFT JOIN reaction ON reaction.item_id = match.item_id
WHERE reaction.reaction = 1 AND match.game_id = 2
GROUP BY match.item_id
HAVING c > 0
but it didn't work as expected. I cannot get count for particular user.
I think you are close. I think you just need to move conditions on the second table to the ON clause:
SELECT m.item_id, COUNT(r.user_id) as c
FROM match m LEFT JOIN
reaction r
ON r.item_id = m.item_id AND
r.reaction = 1 AND
r.user_id = 2
WHERE m.game_id = 2
GROUP BY m.item_id;
I'm not sure what the HAVING clause is for, because you seem to want counts of 0.
Note that this also introduces table aliases so the query is easier to write and to read.
SELECT match.item_id, COUNT(reaction.user_id) as c
FROM match JOIN reaction ON (reaction.item_id = match.item_id and reaction.reaction = 1 AND match.game_id = 2)
GROUP BY match.item_id
HAVING COUNT(reaction.user_id)
I think you need to filter 'before' join -> so use the 'on' clause.
Filters in where are applied after the join is made while filter applied on on clause are applied before the join is made
You have not game_id = 2 so this should return no value
and you should not use left joined table columns in where condition otherwise these wprk as inner join ... in these cases you shou move the related condition in ON clause
SELECT match.item_id, COUNT(reaction.user_id) as c
FROM match
LEFT JOIN reaction ON reaction.item_id = match.item_id
AND reaction.reaction = 1
WHERE match.game_id = 2
GROUP BY match.item_id
HAVING c > 0
but try also
SELECT match.item_id, COUNT(reaction.user_id) as c
FROM match
LEFT JOIN reaction ON reaction.item_id = match.item_id
AND reaction.reaction = 1
GROUP BY match.item_id

MySQL left join that shows NULL for missing rows

I have 2 tables
Table:
recip
recipid | recipname
1 | Recip1
2 | Recip2
And table:
recipuser
recipid | userid
1 | 1
2 | 1
1 | 2
So userid 2 has 1 recip
The result I'm trying to achieve is to show all "recip" rows with matching or null for given user id, EG:
SELECT r.recipid, r.recipname, ru.userid
FROM recip r
left JOIN recipuser ru ON r.recipid = ru.recipid
WHERE ru.userid = 2 OR ru.userid IS NULL
Results in:
recipid | recipname | userid
1 | Recip1 | 2
I want to get:
recipid | recipname | userid
1 | Recip1 | 2
2 | Recip2 | NULL
How do I show all rows from recip with the userid or NULL for every row given a user id??
Thanks for your help.
Move the WHERE logic to the ON clause:
SELECT r.recipid, r.recipname, ru.userid
FROM recip r
LEFT JOIN recipuser ru
ON r.recipid = ru.recipid AND ru.userid = 2;
The problem with your current query is that the WHERE clause is filtering off the non matching record which you want to appear.

Optimization SQL for getting data from two joined tables (usernames for user-from-id and user-to-id msgs from two tables)

I have table "msgs" with messages between users (their ids):
+--------+-------------+------------+---------+---------+
| msg_id |user_from_id | user_to_id | message | room_id |
+--------+-------------+------------+---------+---------+
| 1 | 1 | 4 |Hello! | 2 |
| 2 | 1 | 5 |Hi there | 1 |
| 3 | 2 | 1 |CU soon | 2 |
| 4 | 3 | 7 |nice... | 1 |
+--------+-------------+------------+---------+---------+
I also have two tables with users names.
Table: user1
+--------+----------+
|user_id |user_name |
+--------+----------+
| 5 | Ann |
| 6 | Sam |
| 7 | Michael |
+--------+----------+
Table: user2
+--------+----------+
|user_id |user_name |
+--------+----------+
| 1 | John |
| 2 | Alice |
| 3 | Tom |
| 4 | Jane |
+--------+----------+
I need to get usernames for two users IDs in every row. Every user-id can be in first or second table with usernames.
I wrote this SQL query:
SELECT DISTINCT
m.msg_id,
m.user_from_id,
CASE WHEN c1.user_name IS NULL THEN c3.user_name ELSE c1.user_name END AS from_name,
m.user_to_id,
CASE WHEN c2.user_name IS NULL THEN c4.user_name ELSE c2.user_name END AS to_name,
m.message
FROM msgs m
LEFT JOIN users1 c1 ON c1.user_id=m.user_from_id
LEFT JOIN users1 c2 ON c2.user_id=m.user_to_id
LEFT JOIN users2 c3 ON c3.user_id=m.user_from_id
LEFT JOIN users2 c4 ON c4.user_id=m.user_to_id
WHERE m.room_id=1
LIMIT 0, 8
It works.
Execute query to get raw data without usernames (without any join) tooks about ~0.1 sec. But it's enough to join only one usernames table (user1 or user2 only) to get this data in about ~6.2 sec. (with join one table). I have quite a lot rows in this tables: 35K rows in msgs, 0.5K in user1, 25K in user2.
Executing query with join two tables (with all this data) is impossible.
How to optimize this query? I just need usernames for user_ids in first "msgs" table.
There are potentially many differences between the queries with and without the joins. I am going to assume that the ids have the appropriate indexes -- primary keys automatically do. If not, then check that.
The obvious solution is to use the original query as a subquery:
SELECT m.msg_id, m.user_from_id,
(CASE WHEN c1.user_name IS NULL THEN c3.user_name ELSE c1.user_name
END) AS from_name,
m.user_to_id,
(CASE WHEN c2.user_name IS NULL THEN c4.user_name ELSE c2.user_name
END) AS to_name,
m.message
FROM (SELECT m.*
FROM msgs m
WHERE m.room_id = 1
LIMIT 0, 8
) m LEFT JOIN
users1 c1
ON c1.user_id = m.user_from_id LEFT JOIN
users1 c2
ON c2.user_id = m.user_to_id LEFT JOIN
users2 c3
ON c3.user_id = m.user_from_id LEFT JOIN
users2 c4
ON c4.user_id = m.user_to_id;
For most data structures, the distinct is also unnecessary.
This also makes (the reasonable assumption) that user_id is unique in the users tables.
Also, use of LIMIT without ORDER BY is highly discouraged. The particular rows you get are indeterminate and might change from one execution to the next.

Select data from another table if exists, if not display null

I have two tables.
Invoices
ID | Amount
-----------
1 | 123.54
2 | 553.46
3 | 431.34
4 | 321.31
5 | 983.12
Credit Memos
ID | invoice_ID | Amount
------------------------
1 | 3 | 25.50
2 | 95 | 65.69
3 | 51 | 42.50
I want to get a result set like this out of those two tables
ID | Amount | Cr_memo
---------------------
1 | 123.54 |
2 | 553.46 |
3 | 431.34 | 25.50
4 | 321.31 |
5 | 983.12 |
I've been messing with joins and whatnot all morning with no real luck.
Here is the last query I tried, which pulled everything from the Credit Memo table...
SELECT A.ID, A.Amount FROM Invoices AS A
LEFT JOIN Credit_Memos AS B ON A.ID = B.invoice_ID
Any help or pointers are appreciated.
Your query would work fine. Just add Credit_memo.Amount with an alias:
SELECT Inv.ID,Inv.Amount,IFNULL(C.Amount,'') AS Cr_memo
FROM Invoices Inv LEFT JOIN
Credit_Memos C ON Inv.ID=C.invoice_ID
Result:
ID AMOUNT CR_MEMO
1 124
2 553
3 431 25.50
4 321
5 983
See result in SQL FIDDLE.
You almost got the answer Left Outer Join is what you need but you missed to select Cr_memo from Credit_Memos table. Since you don't want to show Null values when there is no Invoices_ID in Credit Memos table use IFNULL to make NULL's as Empty string
SELECT A.ID, A.Amount, IFNULL(B.Cr_memo,'') AS Cr_memo
FROM Invoices AS A
LEFT JOIN Credit_Memos AS B
ON A.ID = B.invoice_ID
The LEFT JOIN keyword returns all rows from the left table (table1), with the matching rows in the right table (table2). The result is NULL in the right side when there is no match.
SELECT A.ID, A.Amount, IFNULL(B.amount,0) AS Cr_memo FROM Invoices AS A
LEFT JOIN Credit_Memos AS B ON A.ID = B.invoice_ID
here is some useful link about left join link and another

Using SUM to count three tables with relevant records

I'm trying to use the SUM function to count rows from 3 tables, which is however, not working effectively since when the total_files and total_notes are returned, they both are the same when there is at least one file and then total_files will take the same value as total_notes which I don't understand why it's doing that.
It should count the number of rows which is relevant to each record that will get return as a record list with a count of total files, total notes and total contacts assigned to the record per record row (the data of files, notes and contacts do not get displayed only counted).
My query is shown below:
SELECT rec.street_number,
rec.street_name,
rec.city,
rec.state,
rec.country,
rec.latitude,
rec.longitude,
LEFT(rec.description, 250) AS description,
usr.username,
usr.full_name,
ppl.person_id,
ppl.first_name,
ppl.last_name,
SUM(IF(rlk.record_id = rec.record_id, 1, 0)) AS total_contacts,
SUM(IF(files.record_id = rec.record_id, 1, 0)) AS total_files,
SUM(IF(notes.record_id = rec.record_id, 1, 0)) AS total_notes,
(
SELECT COUNT(DISTINCT rec.record_id)
FROM records rec
WHERE rec.marked_delete = 0 AND rec.is_archive = 0
) AS total_records
FROM
(
records rec
INNER JOIN members usr ON rec.user_id = usr.user_id
LEFT OUTER JOIN record_links rlk ON rec.record_id = rlk.record_id
LEFT OUTER JOIN people ppl ON ppl.person_id = rlk.person_id AND rlk.record_id = rec.record_id
LEFT OUTER JOIN files files ON files.record_id = rec.record_id
LEFT OUTER JOIN notes notes ON notes.record_id = rec.record_id
)
WHERE rec.marked_delete = 0 AND rec.is_archive = 0
GROUP BY rec.record_id
ORDER BY rec.submit_date DESC
LIMIT 0, 25
Basically as you can see there is three SUM which will count relevant rows that comes from those tables, but I seriously don't understand how total_files would be taking the same value as total_notes is there something wrong I'm doing here?
It's because rec is joined to both notes and files.
Suppose record 1 has 2 notes and 1 file, record 2 has two note and two files, and record 3 has a note but no files.
Then the table rec LEFT OUTER JOIN files ... LEFT OUTER JOIN notes will look like this:
+-----------+---------+---------+
| record_id | file_id | note_id |
+-----------+---------+---------+
| 1 | 1 | 1 |
| 1 | 2 | 1 |
| 2 | 3 | 2 |
| 2 | 3 | 3 |
| 2 | 4 | 2 |
| 2 | 4 | 3 |
| 3 | NULL | 4 |
+-----------+---------+---------+
Note how every file_id gets joined to every note_id (within the same record_id). Also, since you have SUM(IF(files.record_id = rec.record_id,1,0)) and the join condition is files.record_id = rec.record_id, you are actually counting COUNT(files)*COUNT(notes) per record_id.
I'd recommend you instead COUNT(DISTINCT files.id) and COUNT(DISTINCT records.id). The column in the COUNT would be your primary key on files/notes, not files.record_id:
SELECT rec.record_id,
COUNT(DISTINCT files.id) AS total_files,
COUNT(DISTINCT notes.id) AS total_notes
FROM rec
-- note: LEFT OUTER JOIN is the same as LEFT JOIN in MySQL
LEFT JOIN files ON files.record_id=rec.record_id
LEFT JOIN notes ON notes.record_id=rec.record_id
GROUP BY record_id
+-----------+-------------+-------------+
| record_id | total_files | total_notes |
+-----------+-------------+-------------+
| 1 | 2 | 1 |
| 2 | 2 | 2 |
| 3 | 0 | 1 |
+-----------+-------------+-------------+
Of course, adjust to your query as necessary (add in those extra columns/joins).