I am trying to update multiple tables that use the same column called "Team". I created a update statement but very slow and takes way to long. Can I get some tips to optimize and run faster?
update QB, RB, WR, passing, rushing, receiving
set qb.team='GB',
rb.team='GB',
wr.team='GB',
passing.team='GB',
rushing.team='GB',
receiving.team='GB'
where qb.team=('GNB') or
(rb.team='GNB') or
(wr.team='GNB') or
(passing.team='GNB') or
(rushing.team='GNB') or
(receiving.team='GNB');
You're doing a huge cross join on all six of your tables. This means that the criteria in your WHERE clause are scanning through a very large number of joined rows. Specifically you're scanning the product of the number of rows in all six tables.
Instead, you should write your query like this.
update QB
join RB ON QB.something = RB.something
join WR ON QB.something = WR.something ... etc
SET QB.team = 'GB', RB.team='GB' ... etc
WHERE something
Related
I have a database that holds the details of various different events and all of the odds that bookmakers are offering on those events. I have the following query which I am using to get the best odds for each different type of bet for each event:
SELECT
eo1.id,
eo1.event_id,
eo1.market_id,
IF(markets.display_name IS NULL, markets.name, markets.display_name) AS market_name,
IF(market_values.display_name IS NULL, market_values.name, market_values.display_name) AS market_value_name,
eo2.bookmaker_id,
eo2.value
FROM event_odds AS eo1
JOIN markets ON eo1.market_id = markets.id AND markets.enabled = 1
JOIN market_values on eo1.market_value_id = market_values.id
JOIN bookmakers on eo1.bookmaker_id = bookmakers.id AND bookmakers.enabled = 1
JOIN event_odds AS eo2
ON
eo1.event_id = eo2.event_id
AND eo1.market_id = eo2.market_id
AND eo1.market_value_id = eo2.market_value_id
AND eo2.value = (
SELECT MAX(value)
FROM event_odds
WHERE event_odds.event_id = eo1.event_id
AND event_odds.market_id = eo1.market_id
AND event_odds.market_value_id = eo1.market_value_id
)
WHERE eo1.`event_id` = 6708
AND markets.name != '-'
GROUP BY eo1.market_id, eo1.market_value_id
ORDER BY markets.sort_order, market_name, market_values.id
This returns exactly what I want however since the database has grown in size it's started to be very slow. I currently have just over 500,000 records in the event odds table and the query takes almost 2 minutes to run. The hardware is decent spec, all of the columns are indexed correctly and the table engine being used is MyISAM for all tables. How can I optimise this query so it runs quicker?
For this query, you want to be sure you have an index on event_odds(event_id, market_id, market_value_id, value).
In addition, you want indexes on:
markets(id, enabled, name)
bookmakers(id, enabled)
Note that composite indexes are quite different from multiple indexes with one column each.
Create a MySQL view for this SQL. Try to fetch data from that MySQL view then. This would help in increasing the speed and can reduce complexity. Try pagination for listing using limit. This will also reduce the load on server. Try to indexes for typical columns
I've a MySQL database and a MS Access front end. MySQL database tables are linked via ODBC connection to MS Access.
ANY query with multiple joined tables will run extremely slow in case of having anything in "WHERE" (or "HAVING") clause.
For example:
SELECT tblGuests.GuestName, Sum(tblPayments.Payment) AS SumOfPayment, tblRooms.RoomName
FROM (tblGuests LEFT JOIN tblPayments ON tblGuests.GuestID = tblPayments.GuestNo) LEFT JOIN tblRooms ON tblGuests.RoomNo = tblRooms.RoomID
WHERE tblGuests.NoShow=False
GROUP BY tblGuests.GuestName, tblRooms.RoomName;
will take for ages (approx. 3 minutes for 20K records.) Exactly the same script takes for 1-1.5 seconds in case of Pass Through Query, so the problem shouldn't be related to indexes or settings on server side. (By the way, indexes are set up on the necessary columns and relations are set up, too.)
The problem happens ONLY if there are more than 2 tables involved in the query AND there is something in the "WHERE" clause or in "HAVING".
For example if you modify the code above like
SELECT tblGuests.GuestName, Sum(tblPayments.Payment) AS SumOfPayment
FROM tblGuests LEFT JOIN tblPayments ON tblGuests.GuestID = tblPayments.GuestNo
WHERE tblGuests.NoShow=False
GROUP BY tblGuests.GuestName;
then it will be very quick again. (Only 2 tables are involved to the query.) Also
SELECT tblGuests.GuestName, Sum(tblPayments.HUFpayment) AS SumOfPayment, tblGuests.NoShow, tblRooms.RoomName
FROM (tblGuests LEFT JOIN tblPayments ON tblGuests.GuestID = tblPayments.GuestNo) LEFT JOIN tblRooms ON tblGuests.RoomNo = tblRooms.RoomID
GROUP BY tblGuests.GuestName, tblGuests.NoShow, tblRooms.RoomName;
will have no problem at all because there is no "WHERE" clause. However the very similar code I mentioned in the beginning of the post will be very slow, unless I run it directly on the server (or via Pass Through Query).
Do you have any idea what can cause this problem and how to avoid it (except to run Pass Through Queries all the time)?
i have a problem with a query for a web site. This is the situation:
I have 3 table:
articoli = where there are all article
clasart = where there are all the matches between the code article and class code - 32314 rows
classificazioni = where there are all matches between class code and name of class - 2401 rows
and this is the query
SELECT a.clar_classi , b.CLA_DESCRI
FROM clasart a JOIN (
SELECT art.AI_CAPOCODI, art.ai_codirest
FROM (select * from clasart where clar_azienda = 'SRL') a
JOIN (
SELECT AI_CAPOCODI, AI_CODIREST,AI_DT_CREAZ,
AI_DESCRIZI, AI_CATEMERC, concat(AI_CAPOCODI, AI_CODIREST) as codice, aI_grupscon
FROM articoli
WHERE AI_AZIENDA = 'SRL' AND AI_CATEMERC LIKE '0101______' AND AI_FLAG_NOW = 0 AND AI_CAPOCODI <> 'zzz'
) art ON trim(a.CLAR_ARTICO) = art.AI_CODIREST
JOIN classificazioni b ON a.CLAR_CLASSI = b.CLA_CODICE
WHERE b.CLA_CODICE LIKE 'AA51__'
group by CLAR_ARTICO) art ON trim(CLAR_ARTICO) = concat(art.AI_CAPOCODI, art.ai_codirest)
JOIN classificazioni b ON a.CLAR_CLASSI = b.CLA_CODICE
WHERE CLAR_AZIENDA = 'SRL' AND CLAR_CLASSI like 'CO____'
The time of run is 16 second. The time increase to 16 second when join with classificazioni.
You can help me? Thanks
Introduce following indexing using the queries below and after that the query will start running within a second or two:
ALTER TABLE articoli ADD INDEX idx_artc_az_cat_flg_cap (AI_AZIENDA, AI_FLAG_NOW, AI_CAPOCODI, AI_CATEMERC);
The above query will introduce the multi-column indexes on articoli table. The indexing work similar way how hash tables or keys of the array work to directly identifying the row on which the target value(s) match. Using multi-column will result in comparison of less number of rows.
Do not use trim(a.CLAR_ARTICO): make sure that before insertion the values are trimmed but not at the time of joining. This can result in skipping the index files and the join comparison can be expensive this way.
Let's move to next steps:
Introduce index on clar_azienda using following query:
ALTER TABLE clasart ADD INDEX idx_cls_az (clar_azienda);
If art.AI_CODIREST is not a primary/foreign key you'll need to introduce index there using the query below:
ALTER TABLE classificazioni ADD INDEX idx_clsi_cd (CLA_CODICE);
We are almost done, you'll just need to index CLAR_AZIENDA as well the same way how I indexed the above columns. Let me also tell you what is what in index column last query so you can write your own.
ALTER TABLE <tableName> ADD INDEX <indexKey (<column to be indexed>);
Let me know if you still have issues, remember you can run these queries after selecting your database from PhpMyAdmin (SQL tabl) or on mysql console.
I am trying to move some data from a database to another. I am currently having over a million entries in my database and I was expecting this to take long but already passed 50min and no result :) .
Here is my query:
UPDATE xxx.product AS p
LEFT JOIN xx.tof_art_lookup AS l ON p.model_view = l.ARL_SEARCH_NUMBER
SET p.model = l.ARL_DISPLAY_NR
WHERE p.model_view = l.ARL_SEARCH_NUMBER;
Any help on how to improve this query will be welcome. Thanks in advance!
Indexes on p.model_view, l.ARL_SEARCH_NUMBER if you not gonna get rid of JOINs.
Actually, it might be optimized depending on actual data amounts and their values (NULLs presence) by use of:
1. Monitoring query execution plan and , if it's not good, putting query hints for compiler or exchange JOINs for subqueries so compiler uses another type of join inside it (merge/nested loops/hashs/whatever)
2. Making a stored procedure with more comlicated but faster logic
3. Doing updates by small portions
Identify what makes slow.
check JOIN is optimized
run SELECT only:
SELECT COUNT(*)
FROM xxx.product p LEFT JOIN xx.tof_art_lookup l
ON p.model_view = l.ARL_SEARCH_NUMBER;
how long takes? and EXPLAIN SELECT ... check proper INDEX is used for JOIN.
If everything is fine for JOIN, then UPDATEing row is slow. this situation is hard to make things faster.
UPDATE = DELETE and INSERT
I didn't tried this. but sometimes, this strategy is faster.. UPDATE is DELETE old row and INSERT new row using new value.
// CREATE new table and INSERT
CREATE TABLE xxx.new_product
SELECT p.model_model, l. ARL_DISPLAY_NR, ...
FROM xxx.product p LEFT JOIN xx.tof_art_lookup l
ON p.model_view = l.ARL_SEARCH_NUMBER;
// drop xxx.procuct
// rename xxx.new_product to xxx.product
divide table into small chunk, and run concurrently
I think your job is CPU bounded and your UPDATE query uses just one CPU can't have benefit many cores. xxx.product TABLE has no constraint for join, there for 1M rows are updated sequencially
My suggestion following.
give some conditions to xxx.product so that xxx.product divided 20 group. (I don't no which column would be better for you, as I have no information about xxx.product)
then run 20 queries at once concurrently.
for example:
// for 1st chunk
UPDATE xxx.product AS p
...
WHERE p.model_view = l.ARL_SEARCH_NUMBER
AND p.column BETWEEN val1 AND val2; <= this condition spliting xxx.product
// for 2nd chunk
UPDATE xxx.product AS p
...
WHERE p.model_view = l.ARL_SEARCH_NUMBER
AND p.column BETWEEN val2 AND val3;
...
...
// for 20th chunk
UPDATE xxx.product AS p
...
WHERE p.model_view = l.ARL_SEARCH_NUMBER
AND p.column BETWEEN val19 AND val20;
It is important to find BETWEEN value distribute table evenly. Histogram may help you. Getting data for histogram plot
I am working in mysql with queries, but I am new to this. I am joining 5 tables where each table has an identifier and one table is the master. Each related table may have more than one associated record to the master table. I am attempting to join these tables but I can't seem to get rid of the duplicated data.
I want all of the related records to be displayed, but I don't want the data in the master table to display for all results in the related tables. I have tried so many different methods but nothing has worked. Currently I have 4 queries that work for the separate tables, but I have not successfully joined them to have the results display the multiple records in the related table but just one record from the master table.
Here are my individual queries that work:
SELECT
GovernmaxAdditionsExtract.AdditionDescr,
GovernmaxAdditionsExtract.BaseArea,
GovernmaxAdditionsExtract.Value
FROM
GovernmaxExtract
INNER JOIN GovernmaxAdditionsExtract
ON GovernmaxExtract.mpropertyNumber = GovernmaxAdditionsExtract.PropertyNumber
WHERE (((GovernmaxExtract.mpropertyNumber)="xxx-xxx-xx-xxx"));
SELECT
GovernmaxExtract.mpropertyNumber,
GovernmaxDwellingExtract.CardNumber,
GovernmaxDwellingExtract.MainBuildingType,
GovernmaxDwellingExtract.BaseArea
FROM
GovernmaxExtract INNER JOIN
GovernmaxDwellingExtract ON GovernmaxExtract.mpropertyNumber = GovernmaxDwellingExtract.PropertyNumber
WHERE (((GovernmaxExtract.mpropertyNumber)="xxx-xxx-xx-xxx"));
Using these sub queries, I tried to put together 2 of the tables, but now I am getting all records back and it is not reading my input parameter:
SELECT GE.mpropertynumber
FROM
GovernmaxExtract AS GE,
(SELECT
GovernmaxAdditionsExtract.AdditionDescr,
GovernmaxAdditionsExtract.BaseArea,
GovernmaxAdditionsExtract.Value
FROM GovernmaxExtract INNER JOIN
GovernmaxAdditionsExtract ON
governmaxextract.mpropertyNumber = GovernmaxAdditionsExtract.PropertyNumber) AS AE
WHERE GE.mpropertynumber = 'xxx-xxx-xx-xxx'
I tried nested queries, lots of different joins, and I am just not able to wrap my head around this. I am pretty sure I want to do a nested query since I want the main data from the Governmax table to display once with the main data and all records with all info for the associated tables. Maybe I am going about it all wrong.
Our original code was:
SELECT
ge.*,
gde.*,
gfe.*,
gae.*,
goie.*
FROM governmaxextract AS ge
LEFT JOIN governmaxdwellingextract AS gde
ON ge.mpropertyNumber = gde.PropertyNumber
LEFT JOIN governmaxfeaturesextract AS gfe
ON gde.PropertyNumber = gfe.PropertyNumber
LEFT JOIN governmaxadditionsextract AS gae
ON gde.PropertyNumber = gae.PropertyNumber
RIGHT JOIN governmaxotherimprovementsextract AS goie
ON gde.PropertyNumber = goie.PropertyNumber
WHERE ge.mpropertyNumber = '$codeword'
ORDER BY goie.CardNumber
But this gives multiple rows from the master table for each record in the associated tables. I thought about concatenate, but I need the data from the associated tables to be displayed individually. Not sure what to try next. Any help is much appreciated.
Sorry, and there is no way to do that like you want. JOIN's can't do that.
I suggest to keep solution with separate queries.
Btw - You could play with UNION operator,
http://en.wikipedia.org/wiki/Union_(SQL)#UNION_operator
P.s.
You could extract main data separately, then extract data from related tables at once using UNION. With UNIOM it will give one result row per each row in related table.
In order to join an two of the Detail tables together without generating duplicate rows, you will have to perform the following operation on each one:
Group on the foreign key to the Master table, and aggregate all other columns being projected onto the join.
Numeric columns are commonly aggregated with SUM(), COUNT(), MAX(), and MIN(). MAX() and MIN() are also applicable to character data. A PIVOT operation is also sometimes useful as an aggregation operator for this type of circumstance.
Once you have two of the Detail tables grouped and aggregated in this way, they will join without duplicates. Additional Detail tables can be added to the join by first grouping and aggregating them also, in the same fashion.