I am running into a small problem,
This is a demo query
select
A.order_id,
if(
A.production_date != '0000-00-00',
A.production_date,
if(
SOME INNER QUERY != '0000-00-00',
SOME INNER QUERY ,
SOME OTHER INNER QUERY
)
) as production_start_date
from
orders A
So basically, suppose the SOME INNER QUERY is taking 10 seconds to do its calculations, fetching data from 8 different tables, checking past history for same order type etc. and if its result is a date, I fetch that date in first condition. But now it will take 20 seconds as 10 seconds for calculation for if condition, and 10 seconds to re-execute to return the result.
Is there any way I can reduce this?
if any one is interested in looking at actual query http://pastebin.com/zqzbpEei
Assuming your query looks like this (sorry, I gave up trying to locate the actual query):
IF(
(SELECT aField FROM aTable WHERE bigCondition) != '0000-00-00',
SELECT aField FROM aTable WHERE bigCondition,
SELECT anotherField FROM anotherTable
)
You can rewrite it as follows:
SELECT IF (
someField != '0000-00-00',
someField,
SELECT anotherField FROM anotherTable
)
FROM aTable WHERE bigCondition
This way you compute bigCondition only once.
This query is quite ugly indeed.
Your major problem seems to be the misuse (and abuse, big time) of the IF() construct. It should be reserved to simple conditions and operations. The same applies to logical operators. Do not operate on entire queries. For instance, I see this one bit appears a few times in your query:
IF(
(SELECT v1.weekends FROM vendor v1 WHERE v1.vendor_id = A.vendor_id) IS NULL
OR (SELECT v1.weekends FROM vendor v1 WHERE v1.vendor_id = A.vendor_id) = '',
'6', -- by the way, why is this a string?! This is an integer, isn't it?
(SELECT v1.weekends FROM vendor v1 WHERE v1.vendor_id = A.vendor_id)
)
This is Bad. The condition should be moved into the SELECT directly. Rewrite it as below:
SELECT
IF (v1.weekends IS NULL OR v1.weekends = '', 6, v1.weekends)
FROM vendor v1 WHERE v1.vendor_id = A.vendor_id
That's two SELECT saved. Do this for every IF() that contains a query, and I am ready to bet you are going to speed up your query by several orders of magnitude.
There is a lot more to say about your current code. Unfortunately, you will probably need to refactor some parts of your ORM. Add new, more specialised methods to some classes, and make them use new queries that you crafted manually. Then refactor your current operation so that it uses these new methods.
Related
Is there any way to do that in a single query? Or do I have to manage it externally? It is not a JOIN of any kind.
SELECT
IF (
(SELECT indicator FROM configuration_table) = 1,
(SELECT series_id FROM series_table LIMIT 1),
''
) as if_exp
FROM
series_table
This executes but returns the first ID over and over, and if I take out the LIMIT 1, it doesn't work as it expects only one result. But what I need is that, if this condition is met:
(SELECT indicator FROM configuration_table) = 1,
Then I need all this data returned:
SELECT series_id, series_code, series_name FROM series_table
Is it possible somehow? Should I be doing two queries and managing the data from php? Thank you very much.
The easiest way would be:
IF ((SELECT indicator FROM configuration_table) = 1) THEN
SELECT series_id, series_code, series_name FROM series_table
END IF
You did not show us what to do, when the condition is false. We do not know the relationship between configuration_table and series_table, so we can't find a way to make it in a single query.
I have copied this answer from IF Condition Perform Query, Else Perform Other Query this answer.
SELECT CASE WHEN ( (SELECT indicator FROM configuration_table) = 1 )
THEN
SELECT series_id, series_code, series_name FROM series_table
ELSE
<QUERY B>
END
Here Query B should replaced by your desired query.
I have 1 variable which is used as WHERE condition in 3 initial queries. Based on the response from one of those I need to create queries to 2 more queries.
$var1 = $_GET['id']; $var2 = $_GET['truck'];
//first two are easy
SELECT `name`, `address` FROM `company` WHERE `id`='".$var1."' LIMIT 1; //q1
SELECT `value`, `date` FROM `checks` WHERE `truck`='".$var2."'; //q2
//the 3rd query may have multiple results and for every result i need **q4** and **q5** to be executed
SELECT `loadNumber`, `cfnNumber` FROM `loads` WHERE `truck`='".$var2."' ; //q3
//notice that WHERE conditions use values from **q3**
SELECT `value`, `date` FROM `finances` WHERE `load`='".loadNumber."'; //q4
SELECT `address` FROM `stops` WHERE `load`='".loadNumber."'; //q5
My question is about optimization as I am trying to combine all these queries into one if possible, hoping it will mean lesser server load time. I am not very familiar with JOINS, so ultimately this is how my code looks like with real data. And here is the result, also with real data. Is there a need to optimize/join these queries in order to decrease server load?
I would suggest to combine just last 3 queries, keep first 2 as is:
SELECT
l.`loadNumber`, l.`cfnNumber` ,
f.`value`, f.`date`,
s.`address`
FROM `loads` l
LEFT JOIN `finances` f
ON f.`load`= l.loadNumber
LEFT JOIN `stops` s
ON s.`load`= l.loadNumber
WHERE `truck`='".$var2."'
Do a subquery. Be something like this:
SELECT `item1_t1`, `item2_t1` FROM `table1` WHERE `item2_t1` in (SELECT `item1_t2` FROM `table2` WHERE `item2_t2`='".$var."');
i have an SQL Requests:
SELECT DISTINCT id_tr
FROM planning_requests a
WHERE EXISTS(
SELECT 1 FROM planning_requests b
WHERE a.id_tr = b.id_tr
AND trainer IS NOT NULL
AND trainer != 'FREE'
)
AND EXISTS(
SELECT 1 FROM planning_requests c
WHERE a.id_tr = c.id_tr
AND trainer IS NULL
)
but this requests take 168.9490 sec to execute for returning 23162 rows of 2545088 rows
should i use LEFT JOIN or NOT IN ? and how can i rewrite it thx
You can speed this up by adding indexes. I would suggest: planning_requests(id_tr, trainer).
You can do this as:
create index planning_requests_id_trainer on planning_requests(id_tr, trainer);
Also, I think you are missing an = in the first subquery.
EDIT:
If you have a lot of duplicate values of id_tr, then in addition to the above indexes, it might make sense to phrase the query as:
select id_tr
from (select distinct id_tr
from planning_requests
) a
where . . .
The where conditions are being run on every row of the original table. The distinct is processed after the where.
I think your query can be simplified to this:
SELECT DISTINCT a.id_tr
FROM planning_requests a
JOIN planning_requests b
ON b.id_tr = a.id_tr
AND b.trainer IS NULL
WHERE a.trainer < 'FREE'
If you index planning_requests(trainer), then MySQL can utilize an index range to get all the rows that aren't FREE or NULL. All numeric strings will meet the < 'FREE' criteria, and it also won't return NULL values.
Then, use JOIN to make sure each record from that much smaller result set has a matching NULL record.
For the JOIN, index planning_requests(id_tr, trainer).
It might be simpler if you don't mix types in a column like FREE and 1.
I have the following query:
SELECT *
FROM products
INNER JOIN product_meta
ON products.id = product_meta.product_id
JOIN sales_rights
ON product_meta.product_id = sales_rights.product_id
WHERE ( products.categories REGEXP '[[:<:]]5[[:>:]]' )
AND ( active = '1' )
AND ( products.show_browse = 1 )
AND ( product_meta.software_platform_mac IS NOT NULL )
AND ( sales_rights.country_id = '240'
OR sales_rights.country_id = '223' )
GROUP BY products.id
ORDER BY products.avg_rating DESC
LIMIT 0, 18;
Running the query with the omission of the ORDER BY section and the query runs in ~90ms, with the ORDER BY section and the query takes ~8s.
I've browsed around SO and have found the reason for this could be that the sort is being executed before all the data is returned, and instead we should be running ORDER BY on the result set instead? (See this post: Slow query when using ORDER BY)
But I can't quite figure out the definitive way on how I do this?
I've browsed around SO and have found the reason for this could be
that the sort is being executed before all the data is returned, and
instead we should be running ORDER BY on the result set instead?
I find that hard to believe, but if that's indeed the issue, I think you'll need to do something like this. (Note where I put the parens.)
select * from
(
SELECT products.id, products.avg_rating
FROM products
INNER JOIN product_meta
ON products.id = product_meta.product_id
JOIN sales_rights
ON product_meta.product_id = sales_rights.product_id
WHERE ( products.categories REGEXP '[[:<:]]5[[:>:]]' )
AND ( active = '1' )
AND ( products.show_browse = 1 )
AND ( product_meta.software_platform_mac IS NOT NULL )
AND ( sales_rights.country_id = '240'
OR sales_rights.country_id = '223' )
GROUP BY products.id
) as X
ORDER BY avg_rating DESC
LIMIT 0, 18;
Also, edit your question and include a link to that advice. I think many of us would benefit from reading it.
Additional, possibly unrelated issues
Every column used in a WHERE clause should probably be indexed somehow. Multi-column indexes might perform better for this particular query.
The column products.categories seems to be storing multiple values that you filter with regular expressions. Storing multiple values in a single column is usually a bad idea.
MySQL's GROUP BY is indeterminate. A standard SQL statement using a GROUP BY might return fewer rows, and it might return them faster.
If you can, you may want to index your ID columns so that the query will run quicker. This is a DBA-level solution, rather than a SQL solution - tuning the database will help overall performance.
The issue in the instance of this query, was that by using GROUP BY and ORDER BY in a query, MySQL is unable to use the index if the GROUP BY and ORDER BY expressions are different.
Related Reading:
http://dev.mysql.com/doc/refman/5.0/en/order-by-optimization.html
http://mysqldba.blogspot.co.uk/2008/06/how-to-pick-indexes-for-order-by-and.html
This query will be done in a cached autocomplete text box, possibly by thousands of users at the same time. What I have below works, bit I feel there may be a better way to do what I am doing.
Any advice?
UPDATED -- it can be 'something%':
SELECT a.`object_id`, a.`type`,
IF( b.`name` IS NOT NULL, b.`name`,
IF( c.`name` IS NOT NULL, c.`name`,
IF( d.`name` IS NOT NULL, d.`name`,
IF ( e.`name` IS NOT NULL, e.`name`, f.`name` )
)
)
) AS name
FROM `user_permissions` AS a
LEFT JOIN `divisions` AS b
ON ( a.`object_id` = b.`division_id`
AND a.`type` = 'division'
AND b.`status` = 1 )
LEFT JOIN `departments` AS c
ON ( a.`object_id` = c.`department_id`
AND a.`type` = 'department'
AND c.`status` = 1 )
LEFT JOIN `sections` AS d
ON ( a.`object_id` = d.`section_id`
AND a.`type` = 'section'
AND d.`status` = 1 )
LEFT JOIN `units` AS e
ON ( a.`object_id` = e.`unit_id`
AND a.`type` = 'unit'
AND e.`status` = 1 )
LEFT JOIN `positions` AS f
ON ( a.`object_id` = f.`position_id`
AND a.`type` = 'position'
AND f.`status` = 1 )
WHERE a.`user_id` = 1 AND (
b.`name` LIKE '?%' OR
c.`name` LIKE '?%' OR
d.`name` LIKE '?%' OR
e.`name` LIKE '?%' OR
f.`name` LIKE '?%'
)
Two simple, fast queries is often better than one huge, inefficient query.
Here's how I'd design it:
First, create a table for all your names, in MyISAM format with a FULLTEXT index. That's where your names are stored. Each of the respective object type (e.g. departments, divisions, etc.) are dependent tables whose primary key reference the primary key of the main named objects table.
Now you can search for names with this much simpler query, which runs blazingly fast:
SELECT a.`object_id`, a.`type`, n.name, n.object_type
FROM `user_permissions` AS a
JOIN `named_objects` AS n ON a.`object_id = n.`object_id`
WHERE MATCH(n.name) AGAINST ('name-to-be-searched')
Using the fulltext index will run hundreds of times faster than using LIKE in the way you're doing.
Once you have the object id and type, if you want any other attributes of the respective object type you can do a second SQL query joining to the table for the appropriate object type:
SELECT ... FROM {$object_type} WHERE object_id = ?
This will also go very fast.
Re your comment: Yes, I'd create the table with names even if it's redundant.
Other than changing the nested Ifs to use a Coalesce() function (MySql has Coalesce() doesn't it)? There is not much you can do as long as you need to filter on that input parameter with a like expresion. Putting a filter on a column using a Like expression, where the Like parameter has a wildcard at the begining, as you do, makes the query argument non-SARG-able, which means that the query processor must do a complete table scan of all the rows in the table to evaluate the filter predicate.
It cannot use an index, because an index is based on the column values, and with your Like parameter, it doesn't know which index entries to read from (since the parameter starts with a wild card)
if MySql has Coalesce, you can replace your Select with:
SELECT a.`object_id`, a.`type`,
Coalesce(n.name, c.name, d.Name, e.Name) name
If you can replace the search argument parameter so that it does not start with a wildcard, then just ensure that there is an index on the name column in each of the tables, and (if there are not indices on that column now), the query performance will increase enormously.
There are 500 things you can do. Optimize once you know where your bottlenecks are. Until then, work on getting those users onto your app. Its a much higher priority.