inner join in mysql (to-many connections) - mysql

SELECT
a.cdrID as cdrID,
a.userName as userName,
a.callingStationID as callingStationID,
a.orgClientAccountID as orgClientAccountID,
a.terClientAccountID as terClientAccountID,
a.calledStationID as calledStationID,
a.setupTime as setupTime,
a.connectTime as connectTime,
a.disconnectTime as disconnectTime,
a.orgDestCode as orgDestCode,
a.orgBilledDuration as orgBilledDuration,
a.orgBilledAmount as orgBilledAmount,
a.terDestCode as terDestCode,
a.terBilledDuration as terBilledDuration,
a.terBilledAmount as terBilledAmount,
a.orgRateID as orgRateID,
a.terRateID as terRateID,
b.dtDestName as orgDestName,
c.dtDestName as terDestName,
d.clCustomerID as terClientName,
1 as cdrwsid,
cast((e.crFlatRate*a.orgBilledAmount)as decimal(10,4)) as cdrsale,
cast((f.crFlatRate*a.terBilledAmount)as decimal(10,4)) as cdrpurchase,
cast(((e.crFlatRate*a.orgBilledAmount)-(f.crFlatRate*a.terBilledAmount))as decimal(10,4)) as cdrprofit
FROM Successful.vbSuccessfulCDR_508 a
inner join iTelBilling.vbDestination b on a.orgDestCode=b.dtDestCode
inner join iTelBilling.vbDestination c on a.terDestCode=c.dtDestCode
inner join iTelBilling.vbClient d on a.terClientAccountID=d.clAccountID
inner join iTelBilling.vbCallRate e on a.orgRateID=e.crCallRateID
inner join iTelBilling.vbCallRate f on a.terRateID=f.crCallRateID
where setupTime between '1317761709564' and '1317804909564' and a.terBilledDuration!=0
I have problem with this query some time this query runs fine some time it got hanged on server and some time it through error to-many connections. Can any one tell me what to do.

The problem sounds like this query is running very long; This can be due to the fact, that you need to have a look at the indexes that the query uses. To get an overview (and perhaps optimize your indexes and pks) use the command:
> EXPLAIN SELECT
a.cdrID as cdrID,
a.userName as userName,
...
Another reason can be, that there are deadlock-situations or situations, where the query is running very long since a table is locked. If this happens, other users that execute that query (I assume you are using it in an webserver-context) are building up a "waiting row". Each user that executes this query (which is waiting) needs a connection of its own. If this happens, your server is running out of concurrent connections in a short time.
This can be solved in two ways:
1) Make sure your query has more performance (check the pks and indexes)
2) Increase your concurrent connections settings in your SQL-server:
This can be done by setting the following value to 200 connections (for example) in your my.cnf
max_connections = 200
3) Optimize your mySQL. Make sure your querycache, key-buffer, ... are set to a fitting value. Further informations on mySQL-Performance tuning you will find here.

It might be that the engine is trying to use your SMALLER lookup tables for performing the join instead of your PRIMARY table of CRD (Call Data Records), like phone system billing. You are trying to get proper origination / destination billing codes and rates. Sometimes MySQL will try to think for you by using the smaller tables first.
Ensure you have an index on your Successful table on the "setupTime". In addition, add "STRAIGHT_JOIN" clause to the top
SELECT STRAIGHT_JOIN ... rest of query.
This tells MySQL to process based on the tables you have ordered in that order. It appears the joins to your destination, client and call rate tables WOULD have the corresponding index on their join keys respectively... if not create them.

Related

Mysql query joining 5 tables

I am trying to join 5 tables in which i want to get different currency mentioned on different tables against same contract id.
It is giving me results when i join any three tables but when I add one more table in query the server gets unresponsive until I have to kill the process.
Please help me where I am doing a mistake.
SELECT c.department_id,
c.contract_id,
c.seller_id,
c.buyer_id,
c.contract_ratecurrency AS contractcurrency,
b.currency_id AS billcurrency,
s.saleinv_currency AS saleinvcurrency,
cm.currency_id AS commissioncurrency,
sl.currency_id AS cmlogcurrency,
c.contract_iscancel
FROM tbl_contracts C
JOIN tbl_contract_bill b ON c.contract_id=b.contract_id
JOIN tbl_contract_saleinvoice s ON c.contract_id =s.contract_id
JOIN tbl_commission_payment cm ON c.department_id = cm.department_id
JOIN tbl_saleinvoice_commission_log sl ON c.department_id = sl.department_id
WHERE (c.contract_ratecurrency <> s.saleinv_currency
OR c.contract_ratecurrency <> b.currency_id
OR s.saleinv_currency <> b.currency_id
OR cm.currency_id <> sl.currency_id
OR c.contract_ratecurrency <> cm.currency_id
OR s.saleinv_currency <> cm.currency_id
OR b.currency_id <> cm.currency_id)
AND (c.contract_iscancel =0)
requried result should be
ccontractid,csellerid,cbuyerid,ccurrency,bcurrency,scurrency,cmcurrency,slcurrency
101,25,50,1,1,2,3,1
102,28,16,2,3,1,3,2
It looks like you are having performance issues. To optimize your database structure you have multiple options:
Adding indexes on your keys.
Let's take a look to your join statement:
JOIN tbl_saleinvoice_commission_log sl ON c.department_id = sl.department_id
Adding a clustered index on department_id on
tbl_saleinvoice_commission_log table will help you a lot in
performance wise. For more information you can check this link.
Partitioning is another way to increase performance, but you need to check your database structure to see whether it works for you or not. For more information you can check this link.
Also I believe your tables are one to many, so you might need to check how many rows you are trying to retrieve. If your database server is not capable of processing big number of rows you might need to improve your hardware or CPU usage limits of your database daemon.

MS Access query on linked tables with multiple joins is very slow

I've a MySQL database and a MS Access front end. MySQL database tables are linked via ODBC connection to MS Access.
ANY query with multiple joined tables will run extremely slow in case of having anything in "WHERE" (or "HAVING") clause.
For example:
SELECT tblGuests.GuestName, Sum(tblPayments.Payment) AS SumOfPayment, tblRooms.RoomName
FROM (tblGuests LEFT JOIN tblPayments ON tblGuests.GuestID = tblPayments.GuestNo) LEFT JOIN tblRooms ON tblGuests.RoomNo = tblRooms.RoomID
WHERE tblGuests.NoShow=False
GROUP BY tblGuests.GuestName, tblRooms.RoomName;
will take for ages (approx. 3 minutes for 20K records.) Exactly the same script takes for 1-1.5 seconds in case of Pass Through Query, so the problem shouldn't be related to indexes or settings on server side. (By the way, indexes are set up on the necessary columns and relations are set up, too.)
The problem happens ONLY if there are more than 2 tables involved in the query AND there is something in the "WHERE" clause or in "HAVING".
For example if you modify the code above like
SELECT tblGuests.GuestName, Sum(tblPayments.Payment) AS SumOfPayment
FROM tblGuests LEFT JOIN tblPayments ON tblGuests.GuestID = tblPayments.GuestNo
WHERE tblGuests.NoShow=False
GROUP BY tblGuests.GuestName;
then it will be very quick again. (Only 2 tables are involved to the query.) Also
SELECT tblGuests.GuestName, Sum(tblPayments.HUFpayment) AS SumOfPayment, tblGuests.NoShow, tblRooms.RoomName
FROM (tblGuests LEFT JOIN tblPayments ON tblGuests.GuestID = tblPayments.GuestNo) LEFT JOIN tblRooms ON tblGuests.RoomNo = tblRooms.RoomID
GROUP BY tblGuests.GuestName, tblGuests.NoShow, tblRooms.RoomName;
will have no problem at all because there is no "WHERE" clause. However the very similar code I mentioned in the beginning of the post will be very slow, unless I run it directly on the server (or via Pass Through Query).
Do you have any idea what can cause this problem and how to avoid it (except to run Pass Through Queries all the time)?

Conditionals in WHEREs or JOINs?

Lets say I have the following query:
SELECT occurs.*, events.*
FROM occurs
INNER JOIN events ON (events.event_id = occurs.event_id)
WHERE event.event_state = 'visible'
Another way to do the same query and get the same results would be:
SELECT occurs.*, events.*
FROM occurs
INNER JOIN events ON (events.event_id = occurs.event_id
AND event.event_state = 'visible')
My question. Is there a real difference? Is one way faster than the other? Why would I choose one way over the other?
For an INNER JOIN, there's no conceptual difference between putting a condition in ON and in WHERE. It's a common practice to use ON for conditions that connect a key in one table to a foreign key in another table, such as your event_id, so that other people maintaining your code can see how the tables relate.
If you suspect that your database engine is mis-optimizing a query plan, you can try it both ways. Make sure to time the query several times to isolate the effect of caching, and make sure to run ANALYZE TABLE occurs and ANALYZE TABLE events to provide more info to the optimizer about the distribution of keys. If you do find a difference, have the database engine EXPLAIN the query plans it generates. If there's a gross mis-optimization, you can create an Oracle account and file a feature request against MySQL to optimize a particular query better.
But for a LEFT JOIN, there's a big difference. A LEFT JOIN is often used to add details from a separate table if the details exist or return the rows without details if they do not. This query will return result rows with NULL values for b.* if no row of b matches both conditions:
SELECT a.*, b.*
FROM a
LEFT JOIN b
ON (condition_one
AND condition_two)
WHERE condition_three
Whereas this one will completely omit results that do not match condition_two:
SELECT a.*, b.*
FROM a
LEFT JOIN b ON some_condition
WHERE condition_two
AND condition_three
Code in this answer is dual licensed: CC BY-SA 3.0 or the MIT License as published by OSI.

How to simplify/improve this mysql delete query

We regularly send a newsletter to our subscribers. We want to remove subscribers who never open our emails nor read them.
Here's the query I have put together for this - it removes subscribers/their events where they have not replied to 5 emails or more.
It seems a little awkward (and big!) and I was wondering if there was a simpler and more elegant/efficient way to do this query (maybe with joins??) as it does take a while.
DELETE FROM list_subscriber_events where
list_subscriber_events.subscriberid IN
(SELECT list_subscribers.emailaddress, list_subscriber_events.subscriberid, list_subscriber_events.eventtype, count(list_subscriber_events.eventtype) as total
FROM `list_subscriber_events`
LEFT JOIN list_subscribers on
list_subscriber_events.subscriberid=list_subscribers.subscriberid
AND list_subscribers.subscriberid<>''
AND list_subscriber_events.subscriberid<>''
AND list_subscribers.subscriberid NOT IN (select subscriberid from stats_emailopens)
AND list_subscribers.subscriberid NOT IN (select subscriberid from stats_linkclicks)
GROUP BY list_subscriber_events.subscriberid
HAVING count(list_subscriber_events.eventtype) > 5 );
To start with the IN statement in a DELETE query (or almost any query): IN tends to result in very high query execution times in mysql. The other NOT IN statements might be bad for performance also (you have to test the different cases), so this is a rewrite of the query to get rid of the NOT IN.
A rewrite of this query might be better in the following style:
CREATE VIEW myUsersToBeDeleted AS
SELECT lse.subscriberid
FROM `list_subscriber_events` lse
LEFT JOIN list_subscribers ls ON lse.subscriberid=ls.subscriberid
AND ls.subscriberid<>''
AND lse.subscriberid<>''
LEFT JOIN stats_emailopens se ON ls.subscriberid=se.subscriberid
LEFT JOIN stats_linkclicks sl ON ls.subscriberid=sl.subscriberid
WHERE sl.subscriberid IS NULL AND se.subscriberid IS NULL
GROUP BY lse.subscriberid
HAVING count(lse.eventtype) > 5 ;
The DELETE is then easier and quicker:
DELETE lse FROM list_subscriber_events lse, myUsersToBeDeleted b WHERE
lse.subscriberid=b.subscriberid;
Last hint: Migrate to MariaDB to get in general way better performance from using views. MySQL is pretty poor on that level.

Why does this make the server time out?

I'm trying to run
SELECT id, max(event_date) dt
FROM worker
LEFT JOIN events ON id = attendee
WHERE worker.type='CON'
GROUP BY id;
but the query takes much too long. Is there a way to make this faster?
worker and events are somewhat large (10,000 records) tables joined one-to-many. (Most workers have 0 associated events, but some have up to 10 or so.) But I wouldn't think this is so large that the query would fail. Ideas?
Posting the exact table schemas would allow for better analysis.
I'm guessing attendee and type are not indexed.
Try prefixing your query with "explain" (see http://dev.mysql.com/doc/refman/5.1/en/explain-output.html) for details on the execution plan.
Without a schema, this is forced to be a guess. But does the events table have both an id field and an attendee field?
Does being more explicit in your query help?
SELECT worker.id, max(events.event_date) dt
FROM worker
LEFT JOIN events ON worker.id = events.attendee
WHERE worker.type='CON'
GROUP BY worker.id;
(I've had to assume which tables each field associates to.)