I am working with the tpch database and have a query that I want to optimize for faster runtime.
I tried adding indexes to li.l_orderkey, o.o_custkey, and c.c_mktsegment, but they are not improving performance. Could someone please provide suggestions? Thanks.
Connection:
conn = mysql.connect(host = 'relational.fit.cvut.cz', port = int(3306), user = 'guest', passwd = 'relational', db = 'tpch')
Query:
SELECT
c.c_mktsegment,
COUNT(o.o_orderkey) AS num_orders,
SUM(li.l_quantity) AS total_quantity,
SUM(li.l_extendedprice) AS total_price
FROM lineitem li
JOIN orders o
ON li.l_orderkey = o.o_orderkey
JOIN customer c
ON o.o_custkey = c.c_custkey
WHERE li.l_commitdate BETWEEN '1997-01-01T00:00:00Z' AND '1997-12-31T00:00:00Z'
GROUP BY c.c_mktsegment;
In order for a query to use a specific index, it should be covering - including all columns in the query. Your lineitem items seems to have a lot of records, so the most of the data is read there.
You can try creating index on lineitem table with the following columns:
l_commitdate
l_orderkey
l_quantity
l_extendedprice
The idea is to reduce the read columns count (data row size) and to be able to filter by l_commitdate.
Another approach will be pre-calculating this information using trigger or the routine which is adding/updating/deleting the line items records, but it will be more complex.
Related
I am trying to join 5 tables in which i want to get different currency mentioned on different tables against same contract id.
It is giving me results when i join any three tables but when I add one more table in query the server gets unresponsive until I have to kill the process.
Please help me where I am doing a mistake.
SELECT c.department_id,
c.contract_id,
c.seller_id,
c.buyer_id,
c.contract_ratecurrency AS contractcurrency,
b.currency_id AS billcurrency,
s.saleinv_currency AS saleinvcurrency,
cm.currency_id AS commissioncurrency,
sl.currency_id AS cmlogcurrency,
c.contract_iscancel
FROM tbl_contracts C
JOIN tbl_contract_bill b ON c.contract_id=b.contract_id
JOIN tbl_contract_saleinvoice s ON c.contract_id =s.contract_id
JOIN tbl_commission_payment cm ON c.department_id = cm.department_id
JOIN tbl_saleinvoice_commission_log sl ON c.department_id = sl.department_id
WHERE (c.contract_ratecurrency <> s.saleinv_currency
OR c.contract_ratecurrency <> b.currency_id
OR s.saleinv_currency <> b.currency_id
OR cm.currency_id <> sl.currency_id
OR c.contract_ratecurrency <> cm.currency_id
OR s.saleinv_currency <> cm.currency_id
OR b.currency_id <> cm.currency_id)
AND (c.contract_iscancel =0)
requried result should be
ccontractid,csellerid,cbuyerid,ccurrency,bcurrency,scurrency,cmcurrency,slcurrency
101,25,50,1,1,2,3,1
102,28,16,2,3,1,3,2
It looks like you are having performance issues. To optimize your database structure you have multiple options:
Adding indexes on your keys.
Let's take a look to your join statement:
JOIN tbl_saleinvoice_commission_log sl ON c.department_id = sl.department_id
Adding a clustered index on department_id on
tbl_saleinvoice_commission_log table will help you a lot in
performance wise. For more information you can check this link.
Partitioning is another way to increase performance, but you need to check your database structure to see whether it works for you or not. For more information you can check this link.
Also I believe your tables are one to many, so you might need to check how many rows you are trying to retrieve. If your database server is not capable of processing big number of rows you might need to improve your hardware or CPU usage limits of your database daemon.
I have a database that holds the details of various different events and all of the odds that bookmakers are offering on those events. I have the following query which I am using to get the best odds for each different type of bet for each event:
SELECT
eo1.id,
eo1.event_id,
eo1.market_id,
IF(markets.display_name IS NULL, markets.name, markets.display_name) AS market_name,
IF(market_values.display_name IS NULL, market_values.name, market_values.display_name) AS market_value_name,
eo2.bookmaker_id,
eo2.value
FROM event_odds AS eo1
JOIN markets ON eo1.market_id = markets.id AND markets.enabled = 1
JOIN market_values on eo1.market_value_id = market_values.id
JOIN bookmakers on eo1.bookmaker_id = bookmakers.id AND bookmakers.enabled = 1
JOIN event_odds AS eo2
ON
eo1.event_id = eo2.event_id
AND eo1.market_id = eo2.market_id
AND eo1.market_value_id = eo2.market_value_id
AND eo2.value = (
SELECT MAX(value)
FROM event_odds
WHERE event_odds.event_id = eo1.event_id
AND event_odds.market_id = eo1.market_id
AND event_odds.market_value_id = eo1.market_value_id
)
WHERE eo1.`event_id` = 6708
AND markets.name != '-'
GROUP BY eo1.market_id, eo1.market_value_id
ORDER BY markets.sort_order, market_name, market_values.id
This returns exactly what I want however since the database has grown in size it's started to be very slow. I currently have just over 500,000 records in the event odds table and the query takes almost 2 minutes to run. The hardware is decent spec, all of the columns are indexed correctly and the table engine being used is MyISAM for all tables. How can I optimise this query so it runs quicker?
For this query, you want to be sure you have an index on event_odds(event_id, market_id, market_value_id, value).
In addition, you want indexes on:
markets(id, enabled, name)
bookmakers(id, enabled)
Note that composite indexes are quite different from multiple indexes with one column each.
Create a MySQL view for this SQL. Try to fetch data from that MySQL view then. This would help in increasing the speed and can reduce complexity. Try pagination for listing using limit. This will also reduce the load on server. Try to indexes for typical columns
I'm writing a query in mysql to join two tables. And both tables have more than 50,000 records.
Table EMP Columns
empid,
project,
code,
Status
Table EMPINFO
empid,
project,
code,
projecttype,
timespent,
skills
In each table there is candidate key [empid, project, code]
So when I join the table using INNER join
like this INNER JOIN
ON a.empid = b.empid
and a.project = b.project
and a.code = b.code
I'm getting the result, but if I add count(*) in outer query to count number of records, it takes lot of time something connection gets failed.
Is there any way to speed up to get number of records ?
And I would like to hear more suggestions to speed up inner join query as well having same candidate key in both tables.
INDEX(empid, project, code) -- in any order.
Are these tables 1:1? If so, why do the JOIN in order to do the COUNT?
Please provide SHOW CREATE TABLE. (If there are datatype differences, this could be a big problem.)
Please provide the actual SELECT.
How much RAM do you have? Please provide SHOW VARIABLES LIKE '%buffer%';.
i have a problem with a query for a web site. This is the situation:
I have 3 table:
articoli = where there are all article
clasart = where there are all the matches between the code article and class code - 32314 rows
classificazioni = where there are all matches between class code and name of class - 2401 rows
and this is the query
SELECT a.clar_classi , b.CLA_DESCRI
FROM clasart a JOIN (
SELECT art.AI_CAPOCODI, art.ai_codirest
FROM (select * from clasart where clar_azienda = 'SRL') a
JOIN (
SELECT AI_CAPOCODI, AI_CODIREST,AI_DT_CREAZ,
AI_DESCRIZI, AI_CATEMERC, concat(AI_CAPOCODI, AI_CODIREST) as codice, aI_grupscon
FROM articoli
WHERE AI_AZIENDA = 'SRL' AND AI_CATEMERC LIKE '0101______' AND AI_FLAG_NOW = 0 AND AI_CAPOCODI <> 'zzz'
) art ON trim(a.CLAR_ARTICO) = art.AI_CODIREST
JOIN classificazioni b ON a.CLAR_CLASSI = b.CLA_CODICE
WHERE b.CLA_CODICE LIKE 'AA51__'
group by CLAR_ARTICO) art ON trim(CLAR_ARTICO) = concat(art.AI_CAPOCODI, art.ai_codirest)
JOIN classificazioni b ON a.CLAR_CLASSI = b.CLA_CODICE
WHERE CLAR_AZIENDA = 'SRL' AND CLAR_CLASSI like 'CO____'
The time of run is 16 second. The time increase to 16 second when join with classificazioni.
You can help me? Thanks
Introduce following indexing using the queries below and after that the query will start running within a second or two:
ALTER TABLE articoli ADD INDEX idx_artc_az_cat_flg_cap (AI_AZIENDA, AI_FLAG_NOW, AI_CAPOCODI, AI_CATEMERC);
The above query will introduce the multi-column indexes on articoli table. The indexing work similar way how hash tables or keys of the array work to directly identifying the row on which the target value(s) match. Using multi-column will result in comparison of less number of rows.
Do not use trim(a.CLAR_ARTICO): make sure that before insertion the values are trimmed but not at the time of joining. This can result in skipping the index files and the join comparison can be expensive this way.
Let's move to next steps:
Introduce index on clar_azienda using following query:
ALTER TABLE clasart ADD INDEX idx_cls_az (clar_azienda);
If art.AI_CODIREST is not a primary/foreign key you'll need to introduce index there using the query below:
ALTER TABLE classificazioni ADD INDEX idx_clsi_cd (CLA_CODICE);
We are almost done, you'll just need to index CLAR_AZIENDA as well the same way how I indexed the above columns. Let me also tell you what is what in index column last query so you can write your own.
ALTER TABLE <tableName> ADD INDEX <indexKey (<column to be indexed>);
Let me know if you still have issues, remember you can run these queries after selecting your database from PhpMyAdmin (SQL tabl) or on mysql console.
SELECT COUNT(*)
FROM song AS s
JOIN user AS u
ON(u.user_id = s.user_id)
WHERE s.is_active = 1 AND s.public = 1
The s.active and s.public are index as well as u.user_id and s.user_id.
song table row count 310k
user table row count 22k
Is there a way to optimize this? We're getting 1 second query times on this.
Ensure that you have a compound "covering" index on song: (user_id, is_active, public). Here, we've named the index covering_index:
SELECT COUNT(s.user_id)
FROM song s FORCE INDEX (covering_index)
JOIN user u
ON u.user_id = s.user_id
WHERE s.is_active = 1 AND s.public = 1
Here, we're ensuring that the JOIN is done with the covering index instead of the primary key, so that the covering index can be used for the WHERE clause as well.
I also changed COUNT(*) to COUNT(s.user_id). Though MySQL should be smart enough to pick the column from the index, I explicitly named the column just in case.
Ensure that you have enough memory configured on the server so that all of your indexes can stay in memory.
If you're still having issues, please post the results of EXPLAIN.
Perhaps write it as a stored procedure or view... You could also try selecting all the IDs first then running the count on the result... if you do it all as one query it may be faster. Generally optimisation is done by using nested selects or making the server do the work so in this context that is all I can think of.
SELECT Count(*) FROM
(SELECT song.user_id FROM
(SELECT * FROM song WHERE song.is_active = 1 AND song.public = 1) as t
JOIN user AS u
ON(t.user_id = u.user_id))
Also be sure you are using the correct kind of join.