We are doing some pro bone work for a good cause and I'm having a hell of a time with a query. The coding has been done by many volunteers over the years which has an inevitable outcome.
I have two tables, A and B. What I need is a sum of of score_hours on a join between the two where the data is unique for each instance of only A.
Please keep in mind that both tables are quite big (10 to 50k+ each depending on time in the month).
Table A:
id (pk, ai)
uid (int)
scores_date (timestamp (but for some reason only the actual date, not
the time))
score_hours (decimal 3,1)
Table B:
id (pk, ai)
uid (int)
shift_date (timestamp)
There are many records in table B that have the uid we are looking for on several dates (the dates are not unique). Table A has multiple records for uid but on different days. So it could have 1 uid a day, but not 2 instances of 1 uid a day.
There are obviously more selectors for both tables, but they don't match in any way between the tables (although I do need to query them with simple "AND") so this is what I have to work with. I do need to join them because of the rest of the query, but so far I'm not getting the records I need within a decent time.
My attempts were:
This almost made it. But the execution time was disgusting and failed with some simple selectors.
SELECT SUM(score_hours)
FROM A
WHERE
A.uid IN
(SELECT B.uid
FROM B
WHERE B.uid = "1")
This gives the right output but it joins one for every instance of a uid. Normally you can solve that by grouping, but the sum will still count all. So that is not an option:
SELECT SUM(score_hours)
FROM A
LEFT JOIN B ON A.uid = B.uid
WHERE A.uid = "1"
*edit: Not only do I need to JOIN on uid, but there has to be something like this in it:
DISTINCT(date(m.shift_datum)) = DATE(d.dagscores_date)
It is actually a very basic query, except for the fact that a SUM is needed on a record which is not unique in regards to the Left join and that I need to JOIN on two tables at the same time.
If you need more data please tell me so. I can provide all.
You need to remove the duplicates from the table you're joining with, otherwise the cross-product creates multiple rows that get added into the sum.
SELECT SUM(score_hours)
FROM A
JOIN (SELECT DISTINCT uid
FROM B) AS B
ON A.uid = B.uid
Related
I've never been able to get my head around INNER JOINs (or any other JOIN types for that matter) so I'm struggling to work out how to use it in my specific situation. In fact, I'm not even sure if it's what I need. I've looked at other examples and read tutorials but my brain just doesn't seem to work the way needed to truly get it (or it doesn't function at all).
Here's the scenario:
I have two tables -
phone_numbers - this table has a list of phone numbers that
belong to lots of different customers. A single customer can have
multiple numbers. For simplicity's sake, we'll say the fields are
'number_id', 'customer_id', 'phone_number'.
call_history - this table has a record of every single call that one of these
numbers in the first table could have had. There's a record for
every individual call going back years. Again, for simplicity,
we'll say the relevant fields are customer_id, phone_number,
call_start_time.
What I'm trying to accomplish is to find all of the numbers that belong to a particular customer_id in the phone numbers table and use that information to search through the call_history table and find the number of calls each phone number has received, and group that by the number of calls for each number, preferably also showing zeros where a number hasn't received any calls at all.
The reason the zero calls is important is because that's the data I'm interested in. Otherwise, I could just get all the information out of the call_history table. But what I'm trying to achieve is find the numbers with no activity.
All I've been able to accomplish is run one query to get all of the numbers belonging to one customer:
SELECT customer_id, phone_number FROM phone_numbers WHERE customer_id = Y;
Then run a second query to get all phone calls for that customer_id for a set duration:
SELECT customer_id, phone_number, COUNT(*) FROM call_history WHERE customer_id = Y and call_start_time >= DATE_SUB(SYSDATE(), INTERVAL 30 DAY) GROUP BY phone_number;
I've then had to use the data returned from both queries and use a VLOOKUP function in Excel to match number of calls for each individual number from the second query to the list of all numbers from the first query, thus leaving blanks in my "all numbers" table and identifying those numbers that had no calls for that time period.
I'm hoping there's some way to do all of this with a single query and return a table of results, listing the zero number of calls with it and eliminate the whole manual Excel bit as it's not overly efficient and prone to human error.
Without at least a workable example from you, it's not easy to re-create your situation. Anyway, INNER JOIN might not return the result as how you expected. In my short time with MySQL, I mainly use 2 types of JOIN; one is already mentioned and the other is LEFT JOIN. From what I can understand in your question, what you want to achieve can be done by using LEFT JOIN instead of INNER JOIN. I may not be the best person to explain this to you but this is how I understand it:
INNER JOIN - only return anything that match in ON clause between two (or more) tables.
LEFT JOIN - will return everything from the table on the left side of the join and return NULL if ON get no match in the table on the right side of the join .. unless you specify some WHERE condition from something on the right table.
Now, here is my query suggestion and hopefully it'll be useful for you:
SELECT A.customer_id, A.phone_number,
SUM(CASE WHEN call_start_time >= DATE_SUB(SYSDATE(), INTERVAL 30 DAY)
THEN 1 ELSE 0 END) AS Total
FROM phone_numbers A
LEFT JOIN call_history B
ON A.customer_id=B.customer_id
GROUP BY A.customer_id,A.phone_number;
What I did here is I LEFT JOIN phone_numbers table with call_history on customer_id and I re-position the WHERE call_start_time >= .. condition into a CASE expression in the SELECT since putting it at WHERE will turn this into a normal join or inner join instead.
Here is an example fiddle : https://www.db-fiddle.com/f/hriFWqVy5RGbnsdj8i3aVG/1
For Inner join You should have to do like this way..
SELECT customer_id,phone_number FROM phone_numbers as pn,call_history as ch where pn.customer_id = ch.customer_id and call_start_time >= DATE_SUB(SYSDATE(), INTERVAL 30 DAY) GROUP BY phone_number;
Just add table name whatever you want to join and add condition
I want to check 2 databases to see if the money-payments are the same as the total. That is possible, but I get a very long table:
select
transaction_id
,total_low+total_high a
, sum(money_received) b
from
archive_transaction inner join archive_transaction_payment
on archive_transaction.id=archive_transaction_payment.transaction_id
group by transaction_id;
Actually I only want the transactions where the total is wrong!!
So now I want to add a!=b and that gives an invalid query. How to proceed?
Table archive_transaction has 1 row per transaction, but archive_transaction_payment can have multiple payments for one transaction. This makes it complicated for me.
select
transaction_id
,total_low+total_high a
, sum(money_received) b
from archive_transaction inner join archive_transaction_payment
on archive_transaction.id=archive_transaction_payment.transaction_id
where
a!=b
group by transaction_id;
Joins are still problematic for me, but I found an answer without join to find faults in the database.
SELECT id
FROM archive_transaction a
WHERE total_low + total_high != (SELECT Sum(money_received)
FROM archive_transaction_payment b
WHERE a.id = b.transaction_id);
Now I get a short list of problems in my database. Thanks for helping me out.
I'm aware of the INSERT INTO table_name QUERY; however, I'm unsure how to go about achieving the desired result in this case.
Here's a slightly contrived example to explain what I'm looking for, but I'm afraid I cannot put it more succiently.
I have two tables in a database designed for a hotel.
BOOKING and CUSTOMER_BOOKING
Where BOOKING contains PK_room_number, room_type, etc. and CUSTOMER_BOOKING contains FK_room_number, FK_cusomer_id
CUSTOMER_BOOKING is a linking table (many customers can make many bookings, and many bookings can consist of many customers).
Ultimately, in the application back-end I want to be able to list all rooms that have less than 3 customers associated with them. I could execute this a separate query and save the result in the server-side scripting.
However, a more elegant solution (from my point of view) is to store this within the BOOKING table itself. That is to add a column no_of_bookings that counts the number of times the current PK_room_number appears as the foreign key FK_room_number within the CUSTOMER_BOOKING table. And why do this instead? Because it would be impossible for me to write a single complicated query which will both include the information from all ROOMS, among other tables, and also count the occurrences of bookings, without excluding ROOMS that don't have any bookings. A very bad thing for a hotel website attempting to show free rooms!
So it would look like this
BOOKING: PK_room_number (104B) room_type (double) room_price (high), no_of_bookings (3)
BOOKING: PK_room_number (108C) room_type (single) room_price (low), no_of_bookings (1)
CUSTOMER_BOOKING: FK_room_number (104B) FK_customer_id (4312)
CUSTOMER_BOOKING: FK_room_number (104B) FK_customer_id (6372)
CUSTOMER_BOOKING: FK_room_number (104B) FK_customer_id (1112)
CUSTOMER_BOOKING: FK_room_number (108C) FK_customer_id (9181)
How would I go about creating this?
Because it would be impossible for me to write a single complicated
query which will both include the information from all ROOMS, among
other tables, and also count the occurrences of bookings, without
excluding ROOMS that don't have any bookings.
I wouldn't say it's impossible and unless you're running into performance issues, it's easier to implement than adding a new summary column:
select b.*, count(cb.room_number)
from bookings b
left join customer_booking cb on b.room_number = cb.room_number
group by b.room_number
Depending on your query may need to use a derived table containing the booking counts for each room instead instead
select b.*, coalesce(t1.number_of_bookings,0) number_of_bookings
from bookings b
left join (
select room_number, count(*) number_of_bookings
from customer_booking
group by room_number
) t1 on t1.room_number = b.room_number
You have to left join the derived table and select coalesce(t1.number_of_bookings,0) in case a room does not have any entries in the derived table (i.e. 0 bookings).
A summary column is a good idea when you're running into performance issues with counting the # of bookings each time. In that case I recommend creating insert and delete triggers on the customer_booking table that either increment or decrement the number_of_bookings column.
You could do it in a single straight select like this:
select DISTINCT
b1.room_pk,
c1.no_of_bookings
from cust_bookings b1,
(select room_pk, count(1) as no_of_bookings
from cust_bookings
group by room_pk) c1
where b1.room_pk = c1.room_pk
having c1.no_of_bookings < 3
Sorry i used my own table names to test it but you should figure it out easily enough. Also, the "having" line is only there to limit the rows returned to rooms with less than 3 bookings. If you remove that line you will get everything and could use the same sql to update a column on the bookings table if you still want to go that route.
Consider below solutions.
A simple aggregate query to count the customers per each booking:
SELECT b.PK_room_number, Count(c.FK_customer_id)
FROM Booking b
INNER JOIN Customer_Booking c ON b.PK_room_number = c.FK_room_number
GROUP BY b.PK_room_number
HAVING Count(c.FK_customer_id) < 3; # ADD 3 ROOM MAX FILTER
And if you intend to use a new column no_of_booking, here is an update query (using aggregate subquery) to run right after inserting new value from web frontend:
UPDATE Booking b
INNER JOIN
(SELECT b.PK_room_number, Count(c.FK_customer_id) As customercount
FROM Booking b
INNER JOIN Customer_Booking c ON b.PK_room_number = c.FK_room_number
GROUP BY b.PK_room_number) As r
ON b.PK_room_number = r.PK_room_number
SET b.no_of_booking = r.customercount;
the following generates a list showing all of the bookings and a flag of 0 or 1 if the the room has a customer for each of the rooms. it will display some rooms multiple times if there are multiple customers.
select BOOKING.*,
case CUSTOMER_BOOKING.FK_ROOM_NUMBER is null THEN 0 ELSE 1 END AS BOOKING_FLAG
from BOOKING LEFT OUTER JOIN CUSTOMER_BOOKING
ON BOOKING.PK_room_numer = CUSTOMER_BOOKING.FK_room_number
summing and grouping we arrive at:
select BOOKING.*,
SUM(case when CUSTOMER_BOOKING.FK_ROOM_NUMBER is null THEN 0 ELSE 1 END) AS BOOKING_COUNT
from BOOKING LEFT OUTER JOIN CUSTOMER_BOOKING
ON BOOKING.PK_room_number = CUSTOMER_BOOKING.FK_room_number
GROUP BY BOOKING.PK_room_number
there are at least two other solutions I can think of off the top of my head...
Explain SQL (in phpmyadmin) of a query that is taking more than 5 seconds is giving me the above. I read that we can study the Explain SQL to optimize a query. Can anyone tell if this Explain SQL telling anything as such?
Thanks guys.
Edit:
The query itself:
SELECT
a.`depart` , a.user,
m.civ, m.prenom, m.nom,
CAST( GROUP_CONCAT( DISTINCT concat( c.id, '~', c.prenom, ' ', c.nom ) ) AS char ) AS coordinateur,
z.dr
FROM `0_activite` AS a
JOIN `0_member` AS m ON a.user = m.id
LEFT JOIN `0_depart` AS d ON ( m.depart = d.depart AND d.rank = 'mod' AND d.user_sec =2 )
LEFT JOIN `0_member` AS c ON d.user_id = c.id
LEFT JOIN `zone_base` AS z ON m.depart = z.deprt_num
GROUP BY a.user
Edit 2:
Structures of the two tables a and d. Top: a and bottom: d
Edit 3:
What I want in this query?
I first want to get the value of 'depart' and 'user' (which is an id) from the table 0_activite. Next, I want to get name of the person (civ, prenom and name) from 0_member whose id I am getting from 0_activite via 'user', by matching 0_activite.user with 0_member.id. Here depart is short of department which is also an id.
So at this point, I have depart, id, civ, nom and prenom of a person from two tables, 0_activite and 0_member.
Next, I want to know which dr is related with this depart, and this I get from zone_base. The value of depart is same in both 0_activite and 0_member.
Then comes the trickier part. A person from 0_member can be associated with multiple departs and this is stored in 0_depart. Also, every user has a level, one of what is 'mod', stands for moderator. Now I want to get all the people who are moderators in the depart from where the first user is, and then get those moderaor's name from 0_member again. I also have a variable user_sec, but this is probably less important in this context, though I cannot overlook it.
This is what makes the query a tricky one. 0_member is storing id, name of users, + one depart, 0_depart is storing all departs of users, one line for each depart, and 0_activite is storing some other stuffs and I want to relate those through userid of 0_activite and the rest.
Hope I have been clear. If I am not, please let me know and I will try again to edit this post.
Many many thanks again.
Aside from the few answers provided by the others here, it might help to better understand the "what do I want" from the query. As you've accepted a rather recent answer from me in another of your questions, you have filters applied by department information.
Your query is doing a LEFT join at the Department table by rank = 'mod' and user_sec = 2. Is your overall intent to show ALL records in the 0_activite table REGARDLESS of a valid join to the 0_Depart table... and if there IS a match to the 0_Depart table, you only care about the 'mod' and 2 values?
If you only care about those people specifically associated with the 0_depart with 'mod' and 2 conditions, I would reverse the query starting with THIS table first, then join to the rest.
Having keys on tables via relationship or criteria is always a performance benefit (vs not having the indexes).
Start your query with whatever would be your smallest set FIRST, then join to other tables.
From clarification in your question... I would start with the inner-most... Who it is and what departments are they associated with... THEN get the moderators (from department where condition)... Then get actual moderator's name info... and finally out to your zone_base for the dr based on the department of the MODERATOR...
select STRAIGHT_JOIN
DeptPerMember.*
Moderator.Civ as ModCiv,
Moderator.Prenom as ModPrenom,
Moderator.Nom as ModNom,
z.dr
from
( select
m.ID,
m.Depart,
m.Civ,
m.Prenom,
m.Nom
from
0_Activite as a
join 0_member m
on a.User = m.ID
join 0_Depart as d
on m.depart = d.depart ) DeptPerMember
join 0_Depart as DeptForMod
on DeptPerMember.Depart = DeptForMod.Depart
and DeptForMod.rank = 'mod'
and DeptForMod.user_sec = 2
join 0_Member as Moderator
on DeptForMod.user_id = Moderator.ID
join zone_base z
on Moderator.depart = z.deprt_num
Notice how I tier'd the query to get each part and joined to the next and next and next. I'm building the chain based on the results of the previous with clear "alias" references for clarification of content. Now, you can get whatever respective elements from any of the levels via their distinct "alias" references...
The output from EXPLAIN is showing us that the first and third tables listed (a & d) are not having any indexes utilised by the database engine in executing this query. The key column is NULL for both - which is a shame since both are 'large' tables (OK, they're not really large, but compared to the rest of the tables they're the big 'uns).
Judging from the query, an index on user on 0_activite and an index on (depart, rank, user_sec) on 0_depart would go some way to improving performance.
you can see that columns key and key_len are null this means its not using any key in the possible_keys column. So table a and d are both scanning all rows. (check larger numbers in rows column. you want this smaller).
To deal with 0_depart:
Make sure you have a key on (d.depart, d.rank,d.user_sec) which are part of the join of 0_depart.
To deal with 0_activite:
I'm not positive but a GROUP column should be indexed too so you need a key on a.user
What is a good way to calculate difference (in a sense what should be added and deleted from one table to get another) between tables in MySQL?
Neither of the answers posted so far (from BrynJ and Vadim) does a very thorough job. And doing the thorough job is also incredibly hard. Both answers assume that it is sufficient to know which ID numbers are present in each table. However, in general, tables have more than one column.
Let's call the tables A and B.
One important question is "do the two tables have the same schema"? If not, then one issue is which columns need to be added to A and which need to be added to B to make their schemas the same. This is a metadata query, answerable from the system catalog. Which values should be inserted in the columns added to the tables is an interesting question.
Let's assume that the tables actually have the same schema, including the same primary key and the same functional dependencies between columns. Let's also assume that there is an ID column (storing a unique integer), and Name column (a string), and a RefDate column of type DATE.
Table A Table B
ID Name RefDate ID Name RefDate
1 Frederick 2007-01-23 1 Josephine 2009-01-10
Now, what needs to be inserted, deleted, updated from each table to make them the same?
I think it is fair to say that there is no single answer to that without knowing a lot more context. It might be that Frederick has undergone gender-change surgery since 2007, and the entry in B represents her new identity. Or it might be a blunder; the database should not store both those records. Or there might be another resolution.
Unfortunately, the queries from BrynJ and Vadim would both indicate that there is no difference between A and B, which is a dubious proposition to me.
Incidentally, note that comparing rows when the rows might have nulls is more complex than when they don't. For example, consider comparing names:
No nulls:
(A.Name = B.Name)
With nulls:
(A.Name = B.Name OR (A.Name IS NULL AND B.Name IS NULL))
One more reason to shun nulls whenever you can.
You can also use a left outer join (the first tells you where a row exists in table a and not b, the second vice-versa):
SELECT a.id FROM a LEFT JOIN b ON a.id = b.id WHERE b.id IS NULL
SELECT b.id FROM b LEFT JOIN a ON b.id = a.id WHERE a.id IS NULL
SELECT DISTINCT id FROM a WHERE NOT EXISTS (SELECT * FROM b WHERE a.id = b.id);
SELECT DISTINCT id FROM b WHERE NOT EXISTS (SELECT * FROM a WHERE a.id = b.id);