MySQL Select Query running very slow - mysql

I have a MySQL query that works but is very slow. I am guessing due to the amount of joins.
SELECT
order_header.order_head_id,
order_header.order_date,
order_header.status,
suppliers.supplier,
categories.category,
order_header.user,
order_header.sage_ref,
SUM(order_lines.total_price) AS price
FROM
order_header
LEFT JOIN
order_lines ON order_header.order_head_id = order_lines.order_head_id
LEFT JOIN
suppliers ON order_header.supplier_id = suppliers.supp_id
LEFT JOIN
categories ON order_header.category = categories.cat_id
WHERE
order_header.status LIKE '%'
AND order_header.order_head_id LIKE '%'
AND order_header.user LIKE '%'
GROUP BY order_header.order_head_id
ORDER BY order_head_id DESC
LIMIT 50;
Results of the EXPLAIN query
SHOW CREATE TABLE results
CREATE TABLE `categories` (
`cat_id` int(11) NOT NULL AUTO_INCREMENT,
`category` varchar(45) DEFAULT NULL,
`status` varchar(45) DEFAULT NULL,
PRIMARY KEY (`cat_id`)
) ENGINE=InnoDB AUTO_INCREMENT=63 DEFAULT CHARSET=latin1
CREATE TABLE `order_header` (
`order_head_id` int(11) NOT NULL AUTO_INCREMENT,
`status` varchar(45) DEFAULT NULL,
`category` varchar(45) NOT NULL,
`order_date` date DEFAULT NULL,
`supplier_id` varchar(45) NOT NULL,
`user` varchar(45) DEFAULT NULL,
`sage_ref` varchar(45) DEFAULT NULL,
`query_notes` varchar(500) DEFAULT NULL,
PRIMARY KEY (`order_head_id`)
) ENGINE=InnoDB AUTO_INCREMENT=2249 DEFAULT CHARSET=latin1
CREATE TABLE `order_lines` (
`order_lines_id` int(11) NOT NULL AUTO_INCREMENT,
`order_head_id` int(11) DEFAULT NULL,
`qty` int(11) DEFAULT NULL,
`description` varchar(255) DEFAULT NULL,
`unit_price` decimal(65,2) DEFAULT NULL,
`total_price` decimal(65,2) DEFAULT NULL,
PRIMARY KEY (`order_lines_id`)
) ENGINE=InnoDB AUTO_INCREMENT=3981 DEFAULT CHARSET=latin1
CREATE TABLE `suppliers` (
`supp_id` int(11) NOT NULL AUTO_INCREMENT,
`supplier` varchar(255) DEFAULT NULL,
`status` varchar(225) DEFAULT NULL,
PRIMARY KEY (`supp_id`)
) ENGINE=InnoDB AUTO_INCREMENT=161 DEFAULT CHARSET=latin1
SQL Version 5.6.30
I am not great on MySQL and was wondering if anyone can see a way to improve the query so that it runs quicker.
Your help would be gratefully appreciated.
Many thanks,
John

It can make sense to wrap the first (left) join into a GROUP BY subquery. GROUP BY and LIMIT will limit the number of row which will be used in the following two joins:
SELECT
x.order_head_id,
x.order_date,
x.status,
suppliers.supplier,
categories.category,
x.user,
x.sage_ref,
x.price
FROM (
SELECT
order_header.supplier_id,
order_header.category,
order_header.order_head_id,
order_header.order_date,
order_header.status,
order_header.user,
order_header.sage_ref,
SUM(order_lines.total_price) AS price
FROM order_header
LEFT JOIN order_lines ON order_header.order_head_id = order_lines.order_head_id
WHERE order_header.status LIKE '%'
AND order_header.order_head_id LIKE '%'
AND order_header.user LIKE '%'
GROUP BY order_header.order_head_id
ORDER BY order_head_id DESC
LIMIT 50
) x
LEFT JOIN suppliers ON x.supplier_id = suppliers.supp_id
LEFT JOIN categories ON x.category = categories.cat_id
ORDER BY order_head_id DESC

Related

MySql query is slow with join - how to speed it up

I have to export 554k records from our mysql db. At the current rate it will take 5 days to export the data and the slowness is mainly caused by the query below. The data structure consists of
Companies
--Contacts
----(Contact)Activities
For the contacts, we have an index on company_id. On the activities table, we have an index for contact_id and company_id which map back to the respective contacts and companies tables.
I need to grab each contact and the latest activity date that they have. This is the query that I'm running and it takes about .5 second to execute.
Select *
from contacts
left outer join (select occurred_at
,contact_id
from activities
where occurred_at is not null
group by contact_id
order by occurred_at desc) activities
on contacts.id = activities.contact_id
where company_id = 20
If I remove the join and just select * from contacts where company_id=20 the query executes in .016 sec.
If I use Explain for info on the join query I get this
Any ideas on how I can speed this up?
Edit:
Here are the table definitions.
CREATE TABLE `companies` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(255) COLLATE utf8_unicode_ci DEFAULT NULL,
`street_address` varchar(255) COLLATE utf8_unicode_ci DEFAULT NULL,
`city` varchar(255) COLLATE utf8_unicode_ci DEFAULT NULL,
`state` varchar(255) COLLATE utf8_unicode_ci DEFAULT NULL,
`county` varchar(255) COLLATE utf8_unicode_ci DEFAULT NULL,
`website` varchar(255) COLLATE utf8_unicode_ci DEFAULT NULL,
`external_id` int(11) DEFAULT NULL,
`created_at` datetime DEFAULT NULL,
`updated_at` datetime DEFAULT NULL,
`user_id` int(11) DEFAULT NULL,
`falloff_date` date DEFAULT NULL,
`zipcode` varchar(255) COLLATE utf8_unicode_ci DEFAULT NULL,
`phone` varchar(255) COLLATE utf8_unicode_ci DEFAULT NULL,
`company_id` int(11) DEFAULT NULL,
`order_count` int(11) NOT NULL DEFAULT '0',
`active_job_count` int(11) NOT NULL DEFAULT '0',
`duplicate_of` int(11) DEFAULT NULL,
`warm_date` datetime DEFAULT NULL,
`employee_size` int(11) DEFAULT NULL,
`dup_checked` tinyint(1) DEFAULT '0',
`rating` int(11) DEFAULT NULL,
`delinquent` tinyint(1) DEFAULT '0',
`cconly` tinyint(1) DEFAULT '0',
PRIMARY KEY (`id`),
KEY `index_companies_on_name` (`name`),
KEY `index_companies_on_user_id` (`user_id`),
KEY `index_companies_on_company_id` (`company_id`),
KEY `index_companies_on_external_id` (`external_id`),
KEY `index_companies_on_state_and_dup_checked` (`id`,`state`,`dup_checked`,`duplicate_of`),
KEY `index_companies_on_dup_checked` (`id`,`dup_checked`),
KEY `index_companies_on_dup_checked_name` (`dup_checked`,`name`),
KEY `index_companies_on_county` (`county`,`state`)
) ENGINE=InnoDB AUTO_INCREMENT=15190300 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
CREATE TABLE `contacts` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`first_name` varchar(255) COLLATE utf8_unicode_ci DEFAULT NULL,
`last_name` varchar(255) COLLATE utf8_unicode_ci DEFAULT NULL,
`title` varchar(255) COLLATE utf8_unicode_ci DEFAULT NULL,
`phone` varchar(255) COLLATE utf8_unicode_ci DEFAULT NULL,
`extension` varchar(255) COLLATE utf8_unicode_ci DEFAULT NULL,
`fax` varchar(255) COLLATE utf8_unicode_ci DEFAULT NULL,
`email` varchar(255) COLLATE utf8_unicode_ci DEFAULT NULL,
`active` tinyint(1) DEFAULT NULL,
`main` tinyint(1) DEFAULT NULL,
`company_id` int(11) DEFAULT NULL,
`created_at` datetime DEFAULT NULL,
`updated_at` datetime DEFAULT NULL,
`external_id` int(11) DEFAULT NULL,
`second_phone` varchar(255) COLLATE utf8_unicode_ci DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `index_contacts_on_company_id` (`company_id`),
KEY `index_contacts_on_first_name` (`first_name`),
KEY `index_contacts_on_last_name` (`last_name`),
KEY `index_contacts_on_phone` (`phone`),
KEY `index_contacts_on_email` (`email`)
) ENGINE=InnoDB AUTO_INCREMENT=11241088 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
CREATE TABLE `activities` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`kind` int(11) DEFAULT NULL,
`contact_id` int(11) DEFAULT NULL,
`call_status` int(11) DEFAULT NULL,
`occurred_at` datetime DEFAULT NULL,
`notes` text COLLATE utf8_unicode_ci,
`user_id` int(11) DEFAULT NULL,
`scheduled_for` datetime DEFAULT NULL,
`priority` tinyint(1) DEFAULT NULL,
`company_id` int(11) DEFAULT NULL,
`created_at` datetime DEFAULT NULL,
`updated_at` datetime DEFAULT NULL,
`from_user_id` int(11) DEFAULT NULL,
`to_user_id` int(11) DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `index_activities_on_contact_id` (`contact_id`),
KEY `index_activities_on_user_id` (`user_id`),
KEY `index_activities_on_company_id` (`company_id`)
) ENGINE=InnoDB AUTO_INCREMENT=515340 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
This is a greatest-n-per-group query, which comes up frequently on Stack Overflow.
Here's a solution that uses a MySQL 8.0 window function:
WITH latest_activities AS (
SELECT contact_id, occurred_at,
ROW_NUMBER() OVER (PARTITION BY contact_id ORDER BY occurred_at DESC) AS rn
FROM activities
)
SELECT *
FROM contacts AS c
LEFT OUTER JOIN latest_activities
ON c.id = latest_activities.contact_id AND latest_activities.rn = 1
WHERE c.company_id = 20
Here's a solution that should work on pre-8.0 versions:
SELECT c.*, a.*
FROM contacts AS c
LEFT OUTER JOIN activities AS a ON a.contact_id = c.id
LEFT OUTER JOIN activities AS a2 ON a2.contact_id = c.id
AND a2.occurred_at > a.occurred_at
WHERE c.company_id = 20
AND a2.contact_id IS NULL;
Another solution:
SELECT c.*, a.*
FROM contacts AS c
LEFT OUTER JOIN activities AS a ON a.contact_id = c.id
LEFT OUTER JOIN (
SELECT c2.contact_id, MAX(a2.occurred_at) AS occurred_at
FROM activities AS a2
INNER JOIN contacts AS c2 ON a2.contact_id = c2.id
WHERE c2.company_id = 20
GROUP BY c2.contact_id ORDER BY NULL
) AS latest_activities
ON latest_activities.contact_id = c.id
AND latest_activities.occurred_at = a.occurred_at
WHERE c.company_id = 20
It would be helpful to create a new index on activities (contact_id, occurred_at).
Don't use subqueries in the FROM clause if you can help it. They impede the MySQL optimizer. So, if you want one row:
Select c.*, a.occurred_at
from contacts c left outer join
from activities a
on c.id = a.contact_id and
a.occurred_at is not null
where c.company_id = 20
order by a.occurred_at desc
limit 1;
If you want one row per contact_id:
Select c.*, a.occurred_at
from contacts c left outer join
from activities a
on c.id = a.contact_id and
a.occurred_at is not null and
a.occurred_at = (select max(a2.occurred_at)
from activities a2
where a2.contact_id = a.contact_id
)
where c.company_id = 20
order by a.occurred_at desc
limit 1;
This can make use of an index on activities(contact_id, occured_at). and contact(company_id, contact_id).
Your query is doing one thing that is a clear no-no -- and no longer supported by the default settings in the most recent versions of MySQL. You have unaggregated columns in a select that are not in the group by. The contact_id should be generating an error.
I feel like I am overlooking something with how complicated the other answers are, but I would think this would be all you need.
SELECT c.*
, MAX(a.occurred_at) AS occurred_at
FROM contacts AS c
LEFT JOIN activities AS a
ON c.id = a.contact_id AND a.occurred_at IS NOT NULL
WHERE c.company_id = 20
GROUP BY c.id;
Notes: (1) this assumes you didn't actually want the duplicate contact_id from your original subquery to be in the final results. (2) This also assumes your server is not configured to require a full group by; if it is, you will need to manually expand c.* into the full column list, and copy that list to the GROUP BY clause as well.
Expanding on dnoeth's comments to your question; if you are not querying each company separately for a particular reason (chunking for load, code structure handling this also handles other stuff company by company, whatever), you could tweak the above query like so to get all your results in one query.
SELECT con.*
, MAX(a.occurred_at) AS occurred_at
FROM companies AS com
INNER JOIN contacts AS con ON com.id = con.company_id
LEFT JOIN activities AS a
ON con.id = a.contact_id AND a.occurred_at IS NOT NULL
WHERE [criteria for companies chosen to be queried]
GROUP BY con.id
ORDER BY con.company_id, con.id
;

MySQL: Find "last modified" date from two tables

I have two MySQL tables: tech_requests and comments. I want to display each tech_request one time in a list ordered by the "last modified" date, whether that be the date of the tech_request creation or the latest comment tied to that tech_request. I was trying to use UNION but I got stuck. Any ideas would be much appreciated. Here are the tables:
CREATE TABLE `tech_requests` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`staff_member_id` int(3) NOT NULL,
`date_time` datetime NOT NULL,
`request` text NOT NULL,
`building_id` int(2) NOT NULL,
`technician_id` int(2) DEFAULT NULL,
`completed` tinyint(1) NOT NULL,
`subject` varchar(30) NOT NULL DEFAULT '',
`category_id` int(2) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=203 DEFAULT CHARSET=utf8;
CREATE TABLE `comments` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`tech_request_id` int(11) NOT NULL,
`technician_id` int(2) NOT NULL,
`date_time` datetime NOT NULL,
`comment` text NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=234 DEFAULT CHARSET=utf8;
Are you looking for something like this?
SELECT r.id, r.staff_member_id, ...,
GREATEST(r.date_time, COALESCE(c.date_time, 0)) last_modified
FROM tech_requests r LEFT JOIN
(
SELECT tech_request_id, MAX(date_time) date_time
FROM comments c
GROUP BY tech_request_id
) c
ON r.id = c.tech_request_id
ORDER BY last_modified
Here is SQLFiddle demo

Join two tables only if a value is null or a specific number

I have three tables in a database:
Product table - +100000 entries
Attribute table (list of possible attributes of a product)
Product attribtue table (which contains the value of the attribute of a product)
I am looking for 8 random products and one of their attributes (attribute_id = 2), but if a product hasn't this attribute it should appear at the return of the query. I have been trying some sql queries without any succesful result because my return only shows the products that have the attribute and hide the others.
My three tables are like this:
CREATE TABLE `product` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`sku` varchar(20) COLLATE utf8_unicode_ci NOT NULL DEFAULT '',
`name` varchar(90) COLLATE utf8_unicode_ci NOT NULL DEFAULT '',
`provider_id` int(11) unsigned DEFAULT NULL,
`url` varchar(255) COLLATE utf8_unicode_ci DEFAULT NULL,
`active` int(1) unsigned NOT NULL DEFAULT '0',
PRIMARY KEY (`id`),
UNIQUE KEY `sku_UNIQUE` (`sku`)
) ENGINE=InnoDB AUTO_INCREMENT=123965 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
CREATE TABLE `attribute` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`name` varchar(50) NOT NULL DEFAULT '',
`data_type` varchar(50) DEFAULT '',
PRIMARY KEY (`id`),
) ENGINE=InnoDB AUTO_INCREMENT=30 DEFAULT CHARSET=latin1;
CREATE TABLE `product_attribute` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`product_id` int(11) unsigned NOT NULL,
`attribute_id` int(11) unsigned NOT NULL DEFAULT '6',
`value` longtext NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `product_id` (`product_id`,`attribute_id`)
) ENGINE=InnoDB AUTO_INCREMENT=869437 DEFAULT CHARSET=latin1;
And this is one of the queries I tried, I thought it was correct but it have the same problem as the others...
SELECT product.id, product.sku, product.name,provider.name as provider_name,
product_attribute.value as author
FROM (`product`)
LEFT JOIN `provider` ON `product`.`provider_id` = `provider`.`id`
LEFT JOIN `product_attribute` ON `product`.`id` = `product_attribute`.`product_id`
WHERE `product`.`active` = '1' AND `product`.`url` IS NOT NULL
AND (`product_attribute`.`attribute_id` = 8 OR `product_attribute`.`attribute_id` IS NULL)
AND `product`.`provider_id` = '7' ORDER BY RAND() LIMIT 8
I was trying with left, inner and right join and nothing works.
You should put the condition for the left-joined table in the join, not the where clause
...
from product
left join provider ON product.provider_id = provider.id
left join product_attribute on product.id = product_attribute.product_id
and product_attribute.attribute_id = 8
where `product`.`active` = '1'
and `product`.`url` IS NOT NULL
and `product`.`provider_id` = '7'
...

Get same IDs from Detail Table

I have 2 tables in my mysql database:
CREATE TABLE IF NOT EXISTS `RECIPES` (
`recipes_id` int(11) NOT NULL AUTO_INCREMENT,
`title` varchar(50) COLLATE utf8_bin NOT NULL,
`text` varchar(2000) COLLATE utf8_bin NOT NULL,
`count_persons` int(11) NOT NULL,
`duration` int(11) NOT NULL,
`user_id` int(11) NOT NULL,
`date` datetime NOT NULL,
`accepted` int(11) NOT NULL DEFAULT '0',
PRIMARY KEY (`recipes_id`),
KEY `recipes_user_fk` (`user_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin AUTO_INCREMENT=88 ;
CREATE TABLE IF NOT EXISTS `RECIPES_POS` (
`recipes_pos_id` int(11) NOT NULL AUTO_INCREMENT,
`recipes_id` int(11) NOT NULL,
`ingredients_id` int(11) NOT NULL,
`ingredients_value` int(11) NOT NULL,
PRIMARY KEY (`recipes_pos_id`),
KEY `recipe_pos_rec_id` (`recipes_id`),
KEY `recipes_pos_ingredient_fk` (`ingredients_id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 AUTO_INCREMENT=58 ;
In the Recipe_Pos Table are many entries. This table shows what ingredients are used in the Recipe.
Now i want to find the recipe which contains incredients like powder and sugar:
SELECT r.recipes_id FROM RECIPES r, RECIPES_POS rp WHERE r.recipes_id = rp.recipes_id AND rp.ingredients_id =6 AND rp.ingredients_id =4
this statment is wrong because a entry in Recipe_Pos can'T contains both incredients.
Whats the right query? It should works with only 1 incredient and more
select r.recipes_id
from RECIPES r
inner join RECIPES_POS rp on r.recipes_id = rp.recipes_id
where rp.ingredients_id in (4, 6)
group by r.recipes_id
having count(distinct rp.ingredients_id) = 2

MySQL join problem

I want to join a pages table and menu table.
CREATE TABLE IF NOT EXISTS `pages` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(255) NOT NULL DEFAULT '',
`keywords` varchar(255) NOT NULL DEFAULT '',
`description` varchar(255) NOT NULL DEFAULT '',
`path` varchar(255) NOT NULL DEFAULT '',
`content` text NOT NULL,
`status` enum('active','inactive') NOT NULL DEFAULT 'active',
`category_id` int(11) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1 AUTO_INCREMENT=21 ;
CREATE TABLE IF NOT EXISTS `menus` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(255) NOT NULL,
`shortdesc` varchar(255) NOT NULL,
`page_id` varchar(60) NOT NULL,
`status` enum('active','inactive') NOT NULL,
`parentid` int(11) NOT NULL,
`order` int(11) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1 AUTO_INCREMENT=79 ;
I have errors with the following SQL.
function generateTree(&$tree, $parentid = 0) {
$res = $this->db->query('SELECT M.*, P.name AS PageName
WHERE M.parentid = $parentid
ORDER BY M.order asc, M.parentid asc
FROM menus AS M
LEFT JOIN pages AS P
ON P.id = M.page_id');
...
...
Can you tell what I am doing wrong?
Thanks in advance.
You've got your SQL syntax mixed up
$res = $this->db->query('
SELECT
M.*, P.name AS PageName
FROM
menus AS M
LEFT JOIN pages AS P ON P.id = M.page_id
WHERE
M.parentid = $parentid
ORDER BY
M.order asc, M.parentid asc
');
BTW, you should bot be using variables in the SQL string. Use parameterized queries instead (mysyqli*).