I am trying to join 5 tables in which i want to get different currency mentioned on different tables against same contract id.
It is giving me results when i join any three tables but when I add one more table in query the server gets unresponsive until I have to kill the process.
Please help me where I am doing a mistake.
SELECT c.department_id,
c.contract_id,
c.seller_id,
c.buyer_id,
c.contract_ratecurrency AS contractcurrency,
b.currency_id AS billcurrency,
s.saleinv_currency AS saleinvcurrency,
cm.currency_id AS commissioncurrency,
sl.currency_id AS cmlogcurrency,
c.contract_iscancel
FROM tbl_contracts C
JOIN tbl_contract_bill b ON c.contract_id=b.contract_id
JOIN tbl_contract_saleinvoice s ON c.contract_id =s.contract_id
JOIN tbl_commission_payment cm ON c.department_id = cm.department_id
JOIN tbl_saleinvoice_commission_log sl ON c.department_id = sl.department_id
WHERE (c.contract_ratecurrency <> s.saleinv_currency
OR c.contract_ratecurrency <> b.currency_id
OR s.saleinv_currency <> b.currency_id
OR cm.currency_id <> sl.currency_id
OR c.contract_ratecurrency <> cm.currency_id
OR s.saleinv_currency <> cm.currency_id
OR b.currency_id <> cm.currency_id)
AND (c.contract_iscancel =0)
requried result should be
ccontractid,csellerid,cbuyerid,ccurrency,bcurrency,scurrency,cmcurrency,slcurrency
101,25,50,1,1,2,3,1
102,28,16,2,3,1,3,2
It looks like you are having performance issues. To optimize your database structure you have multiple options:
Adding indexes on your keys.
Let's take a look to your join statement:
JOIN tbl_saleinvoice_commission_log sl ON c.department_id = sl.department_id
Adding a clustered index on department_id on
tbl_saleinvoice_commission_log table will help you a lot in
performance wise. For more information you can check this link.
Partitioning is another way to increase performance, but you need to check your database structure to see whether it works for you or not. For more information you can check this link.
Also I believe your tables are one to many, so you might need to check how many rows you are trying to retrieve. If your database server is not capable of processing big number of rows you might need to improve your hardware or CPU usage limits of your database daemon.
Related
I'm writing a query in mysql to join two tables. And both tables have more than 50,000 records.
Table EMP Columns
empid,
project,
code,
Status
Table EMPINFO
empid,
project,
code,
projecttype,
timespent,
skills
In each table there is candidate key [empid, project, code]
So when I join the table using INNER join
like this INNER JOIN
ON a.empid = b.empid
and a.project = b.project
and a.code = b.code
I'm getting the result, but if I add count(*) in outer query to count number of records, it takes lot of time something connection gets failed.
Is there any way to speed up to get number of records ?
And I would like to hear more suggestions to speed up inner join query as well having same candidate key in both tables.
INDEX(empid, project, code) -- in any order.
Are these tables 1:1? If so, why do the JOIN in order to do the COUNT?
Please provide SHOW CREATE TABLE. (If there are datatype differences, this could be a big problem.)
Please provide the actual SELECT.
How much RAM do you have? Please provide SHOW VARIABLES LIKE '%buffer%';.
I have multiple suppliers of data, which I will call A, B, & C. A has a database that is updated monthly. B & C (my application actually gets more than 2 other data suppliers, and there are over 100) reference a table in A and tell which month from A they are using. A may update, add or delete records for each monthly release. Most of the records from A will stay the same. I currently use multiple databases, and specify the database to use in each join.
What is a good way to store the data from A so that B & C joins to the data will work efficiently? Does NoSQL or ORDBMS solve this issue?
If ...
you're using MySQL and
your user id has appropriate privileges and
your databases all live on a single server
you can use tables from multiple databases quite easily, just by qualifying the table names in your queries. For example, this sort of thing performs very well.
SELECT a.id, b.vendor
FROM A.stock a
JOIN B.shipments b ON a.sku = b.sku
I did not write this query. I am working on someone else's old code. I am looking into changing what is needed for this query but if I could simply speed up this query that would solve my problem temporarily. I am looking at adding indexes. when I did a show indexes there are so many indexes on the table orders can that also slow down a query?
I am no database expert. I guess I will learn more from this effort. :)
SELECT
orders.ORD_ID,
orders.ORD_TotalAmt,
orders.PAYMETH_ID,
orders.SCHOOL_ID,
orders.ORD_AddedOn,
orders.AMAZON_PurchaseDate,
orders.ORDSTATUS_ID,
orders.ORD_InvoiceNumber,
orders.ORD_CustFirstName,
orders.ORD_CustLastName,
orders.AMAZON_ORD_ID,
orders.ORD_TrackingNumber,
orders.ORD_SHIPPINGCNTRY_ID,
orders.AMAZON_IsExpedited,
orders.ORD_ShippingStreet1,
orders.ORD_ShippingStreet2,
orders.ORD_ShippingCity,
orders.ORD_ShippingStateProv,
orders.ORD_ShippingZipPostalCode,
orders.CUST_ID,
orders.ORD_ShippingName,
orders.AMAZON_ShipOption,
orders.ORD_ShipLabelGenOn,
orders.ORD_SHIPLABELGEN,
orders.ORD_AddressVerified,
orders.ORD_IsResidential,
orderstatuses.ORDSTATUS_Name,
paymentmethods.PAYMETH_Name,
shippingoptions.SHIPOPT_Name,
SUM(orderitems.ORDITEM_Qty) AS ORD_ItemCnt,
SUM(orderitems.ORDITEM_Weight * orderitems.ORDITEM_Qty) AS ORD_ItemTotalWeight
FROM
orders
LEFT JOIN orderstatuses ON
orders.ORDSTATUS_ID = orderstatuses.ORDSTATUS_ID
LEFT JOIN orderitems ON
orders.ORD_ID = orderitems.ORD_ID
LEFT JOIN paymentmethods ON
orders.PAYMETH_ID = paymentmethods.PAYMETH_ID
LEFT JOIN shippingoptions ON
orders.SHIPOPT_ID = shippingoptions.SHIPOPT_ID
WHERE
(orders.AMAZON_ORD_ID IS NOT NULL AND (orders.ORD_SHIPLABELGEN IS NULL OR orders.ORD_SHIPLABELGEN = '') AND orderstatuses.ORDSTATUS_ID <> 101 AND orderstatuses.ORDSTATUS_ID <> 40)
GROUP BY
orders.ORD_ID,
orders.ORD_TotalAmt,
orders.PAYMETH_ID,
orders.SCHOOL_ID,
orders.ORD_AddedOn,
orders.ORDSTATUS_ID,
orders.ORD_InvoiceNumber,
orders.ORD_CustFirstName,
orders.ORD_CustLastName,
orderstatuses.ORDSTATUS_Name,
paymentmethods.PAYMETH_Name,
shippingoptions.SHIPOPT_Name
ORDER BY
orders.ORD_ID
One simple thing you should consider is whether you really need to use left joins or you would be satisfied using inner joins for some of the joins. the new query would not be the same as the original query, so you would need to think carefully about what you really want back. If your foreign key relationships are indexed correctly, this could help substantially, especially between ORDERS and ORDERITEMS, because I would imagine these are your largest tables. The following post has a good explanation: INNER JOIN vs LEFT JOIN performance in SQL Server. There are lots of other things that can be done, but you will need to post the query plan so people can dive deeper.
It looks like just adding the index was all that was needed.
create index orderitems_ORD_ID_index on orderitems(ORD_ID);
SELECT
a.cdrID as cdrID,
a.userName as userName,
a.callingStationID as callingStationID,
a.orgClientAccountID as orgClientAccountID,
a.terClientAccountID as terClientAccountID,
a.calledStationID as calledStationID,
a.setupTime as setupTime,
a.connectTime as connectTime,
a.disconnectTime as disconnectTime,
a.orgDestCode as orgDestCode,
a.orgBilledDuration as orgBilledDuration,
a.orgBilledAmount as orgBilledAmount,
a.terDestCode as terDestCode,
a.terBilledDuration as terBilledDuration,
a.terBilledAmount as terBilledAmount,
a.orgRateID as orgRateID,
a.terRateID as terRateID,
b.dtDestName as orgDestName,
c.dtDestName as terDestName,
d.clCustomerID as terClientName,
1 as cdrwsid,
cast((e.crFlatRate*a.orgBilledAmount)as decimal(10,4)) as cdrsale,
cast((f.crFlatRate*a.terBilledAmount)as decimal(10,4)) as cdrpurchase,
cast(((e.crFlatRate*a.orgBilledAmount)-(f.crFlatRate*a.terBilledAmount))as decimal(10,4)) as cdrprofit
FROM Successful.vbSuccessfulCDR_508 a
inner join iTelBilling.vbDestination b on a.orgDestCode=b.dtDestCode
inner join iTelBilling.vbDestination c on a.terDestCode=c.dtDestCode
inner join iTelBilling.vbClient d on a.terClientAccountID=d.clAccountID
inner join iTelBilling.vbCallRate e on a.orgRateID=e.crCallRateID
inner join iTelBilling.vbCallRate f on a.terRateID=f.crCallRateID
where setupTime between '1317761709564' and '1317804909564' and a.terBilledDuration!=0
I have problem with this query some time this query runs fine some time it got hanged on server and some time it through error to-many connections. Can any one tell me what to do.
The problem sounds like this query is running very long; This can be due to the fact, that you need to have a look at the indexes that the query uses. To get an overview (and perhaps optimize your indexes and pks) use the command:
> EXPLAIN SELECT
a.cdrID as cdrID,
a.userName as userName,
...
Another reason can be, that there are deadlock-situations or situations, where the query is running very long since a table is locked. If this happens, other users that execute that query (I assume you are using it in an webserver-context) are building up a "waiting row". Each user that executes this query (which is waiting) needs a connection of its own. If this happens, your server is running out of concurrent connections in a short time.
This can be solved in two ways:
1) Make sure your query has more performance (check the pks and indexes)
2) Increase your concurrent connections settings in your SQL-server:
This can be done by setting the following value to 200 connections (for example) in your my.cnf
max_connections = 200
3) Optimize your mySQL. Make sure your querycache, key-buffer, ... are set to a fitting value. Further informations on mySQL-Performance tuning you will find here.
It might be that the engine is trying to use your SMALLER lookup tables for performing the join instead of your PRIMARY table of CRD (Call Data Records), like phone system billing. You are trying to get proper origination / destination billing codes and rates. Sometimes MySQL will try to think for you by using the smaller tables first.
Ensure you have an index on your Successful table on the "setupTime". In addition, add "STRAIGHT_JOIN" clause to the top
SELECT STRAIGHT_JOIN ... rest of query.
This tells MySQL to process based on the tables you have ordered in that order. It appears the joins to your destination, client and call rate tables WOULD have the corresponding index on their join keys respectively... if not create them.
I have a table structure like the following:
user
id
name
profile_stat
id
name
profile_stat_value
id
name
user_profile
user_id
profile_stat_id
profile_stat_value_id
My question is:
How do I evaluate a query where I want to find all users with profile_stat_id and profile_stat_value_id for many stats?
I've tried doing an inner self join, but that quickly gets crazy when searching for many stats. I've also tried doing a count on the actual user_profile table, and that's much better, but still slow.
Is there some magic I'm missing? I have about 10 million rows in the user_profile table and want the query to take no longer than a few seconds. Is that possible?
Typically databases are able to handle 10 million records in a decent manner. I have mostly used oracle in our professional environment with large amounts of data (about 30-40 million rows also) and even doing join queries on the tables has never taken more than a second or two to run.
On IMPORTANT lessson I realized whenever query performance was bad was to see if the indexes are defined properly on the join fields. E.g. Here having index on profile_stat_id and profile_stat_value_id (user_id I am assuming is the primary key) should have indexes defined. This will definitely give you a good performance increaser if you have not done that.
After defining the indexes do run the query once or twice to give DB a chance to calculate the index tree and query plan before verifying the gain
Superficially, you seem to be asking for this, which includes no self-joins:
SELECT u.name, u.id, s.name, s.id, v.name, v.id
FROM User_Profile AS p
JOIN User AS u ON u.id = p.user_id
JOIN Profile_Stat AS s ON s.id = p.profile_stat_id
JOIN Profile_Stat_Value AS v ON v.id = p.profile_stat_value_id
Any of the joins listed can be changed to a LEFT OUTER JOIN if the corresponding table need not have a matching entry. All this does is join the central User_Profile table with each of the other three tables on the appropriate joining column.
Where do you think you need a self-join?
[I have not included anything to filter on 'the many stats'; it is not at all clear to me what that part of the question means.]