I am new to performance tuning in MySQL & need your help concerning a view that will replace a table later-on in our design.
The table to be replaced is called users and has the following attributes:
The users2 view has the following attributes:
When I execute a normal SELECT on both objects, they respond at the same time:
SELECT *
FROM `users`
SELECT *
FROM `users2`
But an ordered version of these queries result in a different performance: The table is a little slower (takes less than two seconds), the view need about ten times this time:
SELECT *
FROM `users`
ORDER BY `lastName`, `firstName`
SELECT *
FROM `users2`
ORDER BY `lastName`, `firstName`
To find out the reason, I let EXPLAIN the two comments:
Obviously, an ALL on table 'a' (addresses) on the attribute Countries_ID is making trouble, so I made the following:
ALTER TABLE addresses ADD INDEX (Countries_ID);
This index didn't change anything at all. So, I ask you for your opinion what can be done better.
Notice 1: Is there a way to create an index on temporary column Countries_ID_2?
Notice 2: The users2 view was created with the following SQL query:
CREATE OR REPLACE VIEW users2 AS
(SELECT p.username
, p.password
, p.firstName
, p.lastName
, p.eMail AS email
, a.settlement AS city
, s.name AS country
, pl.languages
, p.description
, p.ID AS ID
, p.phone1
, p.phone2
, CONCAT_WS(' ', a.street, a.addition) AS address
, p.status
, p.publicMail
, ad.name AS Betreuer
FROM addresses a
INNER JOIN addresses_have_persons ap ON a.ID = ap.Addresses_ID
INNER JOIN countries c ON a.Countries_ID = c.ID
INNER JOIN persons p ON a.ID = p.addressID
AND ap.Persons_ID = p.ID
INNER JOIN states s ON a.States_ID = s.ID
INNER JOIN persons_language pl ON p.ID = pl.ID
LEFT JOIN advisors ad ON p.advisorID = ad.ID
-- LEFT JOIN titles t ON t.ID = ad.titleID
);
Notice 3: Although a lot of fields in the persons table are NULL, there is not a single row where these fields are altogether NULL.
EDIT:
CREATE OR REPLACE VIEW persons_language AS
(SELECT lp.Persons_ID AS ID
, GROUP_CONCAT(DISTINCT l.name ORDER BY l.name SEPARATOR ', ') AS languages
FROM languages l
, languages_have_persons lp
WHERE l.ID = lp.Languages_ID
GROUP BY lp.Persons_ID);
Without the ORDER BY, the language names are not alphabetically ordered, which I currently want. Perhaps, we could decide to get them in any order, but we'll see.
Currently, I made the following modifications without any performance improvement:
ALTER TABLE addresses ADD INDEX (Countries_ID);
ALTER TABLE addresses ADD INDEX (States_ID);
ALTER TABLE addresses_have_persons ADD INDEX (Addresses_ID);
ALTER TABLE languages ADD INDEX (name);
ALTER TABLE persons ADD INDEX (addressID);
ALTER TABLE persons ADD INDEX (address2ID);
ALTER TABLE persons ADD INDEX (address3ID);
ALTER TABLE persons ADD INDEX (advisorID);
EDIT 2:
I discuss this issue also on another site. The discussions there let me do the following changes to be nearer to the third normal form:
CREATE OR REPLACE TABLE accounts
(ID INT NOT NULL AUTO_INCREMENT PRIMARY KEY
, username VARCHAR(50) NOT NULL UNIQUE
, password VARCHAR(255) NOT NULL
, eMail VARCHAR(100) NOT NULL
, Persons_ID INT NOT NULL
);
INSERT INTO accounts (username, password, eMail, Persons_ID)
SELECT username, password, eMail, ID
FROM persons;
The table persons does contain only the most necessary things and has the following structure now:
The new table persons_information carries all additional information:
I recreated the users2 with the following command:
CREATE OR REPLACE VIEW users2 AS
(SELECT ac.username
, ac.password
, p.firstName
, p.lastName
, ac.eMail AS email
, adr.settlement AS city
, s.name AS country
, pl.languages
, pi.description
, ac.Persons_ID AS ID
, pi.phone1
, pi.phone2
, CONCAT_WS(' ', adr.street, adr.addition) AS address
, p.status
, pi.publicMail
, adv.name AS Betreuer
FROM accounts ac
INNER JOIN persons p ON ac.Persons_ID = p.ID
INNER JOIN persons_information pi ON p.ID = pi.ID
INNER JOIN addresses adr ON adr.ID = pi.addressID
INNER JOIN addresses_have_persons ap ON adr.ID = ap.Addresses_ID
AND ap.Persons_ID = p.ID
INNER JOIN countries c ON adr.Countries_ID = c.ID
INNER JOIN states s ON adr.States_ID = s.ID
INNER JOIN persons_language pl ON p.ID = pl.ID
LEFT JOIN advisors adv ON pi.advisorID = adv.ID
-- LEFT JOIN titles t ON t.ID = adv.titleID
);
The SELECT _ FROM users2 is fast, but if I add an ORDER BY lastName, firstName, it takes about 25 seconds to get the response.
Here are the results of the *EXPLAIN SELECT * FROM users2* command:
And here for the other command:
I also (re)created following indexes:
ALTER TABLE addresses ADD INDEX (Countries_ID);
ALTER TABLE addresses ADD INDEX (States_ID);
ALTER TABLE addresses_have_persons ADD INDEX (Persons_ID);
ALTER TABLE languages ADD INDEX (name);
ALTER TABLE persons_information ADD INDEX (addressID);
ALTER TABLE persons_information ADD INDEX (address2ID);
ALTER TABLE persons_information ADD INDEX (address3ID);
ALTER TABLE persons_information ADD INDEX (advisorID);
I think one reason for the problem is the persons_language view that is created as follows:
CREATE OR REPLACE VIEW persons_language AS
(SELECT lp.Persons_ID AS ID
, GROUP_CONCAT(DISTINCT l.name ORDER BY l.name SEPARATOR ', ') AS languages
FROM languages l
INNER JOIN languages_have_persons lp ON l.ID = lp.Languages_ID
GROUP BY lp.Persons_ID);
EDIT 3:
For those interested, I add the EXPLAIN for the persons_language view:
EDIT 4:
After the database meeting today, we decided to delete all objects related to the address information & recreated the view with
CREATE OR REPLACE VIEW `users2` AS
(SELECT ac.username
, ac.password
, p.firstName
, p.lastName
, ac.eMail AS email
, pl.languages
, pi.description
, ac.Persons_ID AS ID
, pi.phone1
, pi.phone2
, p.status
, pi.publicMail
, adv.name AS Betreuer
FROM accounts ac
INNER JOIN persons p ON ac.Persons_ID = p.ID
INNER JOIN persons_information pi ON p.ID = pi.ID
INNER JOIN persons_language pl ON p.ID = pl.ID
INNER JOIN advisors adv ON pi.advisorID = adv.ID
WHERE ac.password IS NOT NULL
);
I also created an index with
CREATE INDEX LanguagesPersonsIndex ON `languages_have_persons` (`Languages_ID`, `Persons_ID`);
The EXPLAIN command shows that the new indices are in use and that the delay after a SELECT with an ORDER BY clause with the new, smaller view is about 18 s. Here is the new result:
My question is: What could I do more to improve the performance?
The key fault must be the problem.
But depending on data volume on the joined tables, it'll anyway be slower.
Try To:
Implement KeyIndexes on ALL attributes used to stablish relationships. (ap.Addresses_ID, a.Countries_ID, p.addressID, ap.Persons_ID, a.States_ID, p.advisorID).
Declare PK on All 'ID' columns.
Don't use ORDER or GROUP in the views construction.
Declare Key Index for attributes that are most used on searches, ordering or grouping.
Tip: The 'INNER' (INNER JOIN) isn't necessary. Is the same of 'JOIN'
Your VIEW "persons_language" would be better like this:
SELECT lp.Persons_ID AS ID, GROUP_CONCAT(DISTINCT l.name ORDER BY l.name SEPARATOR ', ') AS languages
FROM languages_have_persons lp
JOIN languages l ON l.ID = lp.Languages_ID
GROUP BY lp.Persons_ID;
It's more appropriate because the clauses 'FROM' and 'JOIN' are processed before 'WHERE' clause.
You may boost your mysql memory and cache configurations.
Look the my mysql server's configurations (Runs an ERP with weight tables and views):
join_buffer_size= 256M
key_buffer = 312M
key_buffer_size = 768M
max_allowed_packet = 160M
thread_stack = 192K
thread_cache_size = 8
query_cache_limit = 64M
innodb_buffer_pool_size = 1512M
table_cache = 1024M
read_buffer_size = 4M
query_cache_size = 768M
query_cache_limit = 128M
SELECT *
FROM `users`
ORDER BY `lastName`, `firstName`
needs
INDEX(last_name, first_name) -- in that order
Beware of VIEWs; some VIEWs optimize well, some do not.
Please provide SHOW CREATE TABLE for both addresses and addresses_have_persons.
In persons_language, why do you need DISTINCT? Doesn't it have PRIMARY KEY(person, language) (or in the opposite order)? Let's see SHOW CREATE TABLE.
Please provide the EXPLAIN for any query you want to discuss.
Related
My input is only userid. How do i combine three tables to get area name. I am pretty sure my syntax is correct
TABLE NAME: userinfo userid PRIMARY KEY
TABLE NAME: userinfo_attarea employee_id FOREIGN KEY userid REFERENCE
userinfo area_id FOREIGN KEY area_id REFERENCE personnel area
TABLE NAME: personnel area area_id PRIMARY KEY areaname
I tried with this but failed,
SELECT areaname FROM userinfo a
INNER JOIN (SELECT *FROM userinfo_attarea b
INNER JOIN SELECT *FROM personnel_area c
ON b.areaid = c.areaid
) b ON a.userid = b.employee_id;
i think you this query will help you
select areaname from personel_area pa inner join
userinfo_attarea ut on pa.area_id=ut.area_id
inner join userinfo ui on ut.employee_id=ui.userid
select b.areaname
from userinfo a INNER JOIN ( SELECT c.areaname FROM ( select *
from userinfo_attarea ) b
INNER JOIN ( select *
from personnel_area ) c on b.areaid = c.areaid
) b on a.userid = b.employee_id;
Your correct syntax would be:
SELECT areaname FROM userinfo a
INNER JOIN (
SELECT * FROM userinfo_attarea b
INNER JOIN personnel_area c
ON b.areaid = c.areaid
) b ON a.userid = b.employee_id;
The SLELECT * FROM on third line is unnecessary.
Also, I strongly recommend proper indentation and correct use of spaces :) (* FROM instead of *FROM). It will increase clarity of your code.
Also, you need to take care of column names, as sometimes they can be ambigious, so you should list them explicitly in inner query and give them unique aliases.
Your syntax is NOT correct. You are using parentheses unnecessarily in the FROM clause. You are NOT using parentheses for subqueries.
The subqueries are entirely unnecessary. In addition, you have a poor choice of table aliases (random letters rather than table abbreviations) and you haven't qualified your table names.
SELECT pa.areaname
FROM userinfo ui INNER JOIN
userinfo_attarea uia
ON uia.employee_id = ui.userid INNER JOIN
personnel_area pa
ON uia.areaid = pa.areaid;
I have the following query and it works. But it takes insanely long to process due to it's buildup. I'm therefor in the need of assistance to get this query faster.
SQL Query
In the query PRODUCT_ID should be replaced by ' ' and the productID number.
SELECT
b.order_id,
customer_meta.meta_value AS customer_id,
users.user_email,
qty_table.meta_value AS qty,
user_meta1.meta_value AS firstname,
user_meta2.meta_value AS lastname,
user_meta3.meta_value AS company,
user_meta4.meta_value AS address,
user_meta5.meta_value AS city,
user_meta6.meta_value AS postcode,
user_meta7.meta_value AS state,
user_meta8.meta_value AS user_phone
FROM
wp_woocommerce_order_itemmeta a,
wp_woocommerce_order_items b,
wp_postmeta customer_meta,
wp_users users,
wp_woocommerce_order_itemmeta qty_table,
wp_usermeta user_meta1,
wp_usermeta user_meta2,
wp_usermeta user_meta3,
wp_usermeta user_meta4,
wp_usermeta user_meta5,
wp_usermeta user_meta6,
wp_usermeta user_meta7,
wp_usermeta user_meta8
WHERE
a.meta_key = '_product_id'
AND a.meta_value = PRODUCT_ID
AND a.order_item_id = b.order_item_id
AND customer_meta.meta_key = '_customer_user'
AND customer_meta.post_id = b.order_id
AND user_meta1.meta_key = 'first_name'
AND user_meta1.user_id = users.id
AND user_meta2.meta_key = 'last_name'
AND user_meta2.user_id = users.id
AND user_meta3.meta_key = 'billing_company'
AND user_meta3.user_id = users.id
AND user_meta4.meta_key = 'billing_address_1'
AND user_meta4.user_id = users.id
AND user_meta5.meta_key = 'billing_city'
AND user_meta5.user_id = users.id
AND user_meta6.meta_key = 'billing_postcode'
AND user_meta6.user_id = users.id
AND user_meta7.meta_key = 'billing_state'
AND user_meta7.user_id = users.id
AND user_meta8.meta_key = 'billing_phone'
AND user_meta8.user_id = users.id
AND users.ID = customer_meta.meta_value
AND qty_table.meta_key = '_qty'
AND qty_table.order_item_id = b.order_item_id
ORDER BY user_meta3.meta_value ASC
I need all the information, since I want to list all the users with their Firstname, lastname, company, address, postcode, etc for a given product which has been bought. So the query in itself works, but is a killer in process time.
I could use max( CASE WHEN ... ... THEN ...END ) as a_name but I only know how to do it successfully if I use a left join.
Any tips on how to get this query to run better?
WP, are you listening? WooCommerce, are you listening? I am getting tired of optimizing your database app.
First and foremost, EAV is a terrible schema design. But I won't rant about that. I'll just point out the index(es) that are probably missing or mal-formed:
wp_usermeta: PRIMARY KEY(user_id, meta_key)
Without any (191) tacked on.
Similarly for wp_woocommerce_order_itemmeta.
I may have more abuse to dish out; please provide SHOW CREATE TABLE for the tables being used.
Years later, after talking to #RickJames, here's another answer to your question.
You have refactored the query from comma-join to LEFT JOINs. That is good. Your comma-join syntax omitted users with missing metadata from the result set.
If you haven't already done so, make sure you upgrade your MySQL server so it's version 5.7 or later. If you use MariaDB, upgrade it so it's version 10.3 or later The version is important for performance.
Install and use the WordPress plugin Index WP MySQL for Speed. Rick and I put it together to address precisely your problem. It reindexes wp_usermeta (and some other WordPress tables) to make these sorts of queries more efficient.
Or, if you prefer not to use a plugin to reindex your tables, you can do it yourself via phpmyadmin or some other SQL client. (These statements won't work if you're stuck on an old version of MySQL. The plugin contains a workaround for old versions that's a bit too complex to explain in a StackOverflow answer.)
ALTER TABLE wp_usermeta ENGINE=InnoDB, ROW_FORMAT=Dynamic;
ALTER TABLE wp_usermeta ADD UNIQUE KEY umeta_id (umeta_id);
ALTER TABLE wp_usermeta DROP PRIMARY KEY;
ALTER TABLE wp_usermeta ADD PRIMARY KEY (user_id, meta_key, umeta_id);
ALTER TABLE wp_usermeta DROP KEY user_id;
ALTER TABLE wp_usermeta DROP KEY meta_key;
ALTER TABLE wp_usermeta ADD KEY meta_key (meta_key, user_id);
This set of indexes allows your queries to retrieve those left-joined meta_value attributes much more efficiently. Why? In brief, the table needs an index on (user_id, meta_key) for the purpose. And in InnoDB, the data in the table is actually stored in the primary key's index (it's called a clustered index). So MySQL can very quickly retrieve the meta_values you need.
Adding the indexes described by Rick James will make the original query much faster.
But I have managed to change it to use LEFT JOIN and this is by far the fastest and is shown here if anybody else would need it:
SELECT
b.order_id,
customer_meta.meta_value as customer_id,
u.display_name as customer_name,
u.user_email,
qty_table.meta_value as qty,
um_last_name.meta_value as last_name,
um_first_name.meta_value as first_name,
um_address1.meta_value as address1,
um_address2.meta_value as address2,
um_address_city.meta_value as city,
um_address_state.meta_value as state,
um_company.meta_value as company,
um_email.meta_value as billing_email
FROM wp_woocommerce_order_itemmeta a
LEFT JOIN wp_woocommerce_order_items b on a.order_item_id = b.order_item_id
LEFT JOIN wp_postmeta customer_meta on customer_meta.post_id = b.order_id AND customer_meta.meta_key = '_customer_user'
LEFT JOIN wp_users u on u.ID = customer_meta.meta_value
LEFT JOIN wp_woocommerce_order_itemmeta qty_table on qty_table.order_item_id = b.order_item_id and qty_table.meta_key = '_qty'
LEFT JOIN wp_usermeta um_last_name on u.ID = um_last_name.user_id and um_last_name.meta_key = 'last_name'
LEFT JOIN wp_usermeta um_first_name on u.ID = um_first_name.user_id and um_first_name.meta_key = 'first_name'
LEFT JOIN wp_usermeta um_address1 on u.ID = um_address1.user_id and um_address1.meta_key = 'billing_address_1'
LEFT JOIN wp_usermeta um_address2 on u.ID = um_address2.user_id and um_address2.meta_key = 'billing_address_2'
LEFT JOIN wp_usermeta um_address_city on u.ID = um_address_city.user_id and um_address_city.meta_key = 'billing_city'
LEFT JOIN wp_usermeta um_address_state on u.ID = um_address_state.user_id and um_address_state.meta_key = 'billing_state'
LEFT JOIN wp_usermeta um_email on u.ID = um_email.user_id and um_email.meta_key = 'billing_email'
LEFT JOIN wp_usermeta um_company on u.ID = um_company.user_id and um_company.meta_key = 'billing_company'
WHERE
a.meta_key = '_product_id' AND a.meta_value = PRODUCT_ID
So, I've got two tables: users and tasks;
users:
user_id username password first_name last_name isAdmin
tasks:
task_id name description assessment assigned_user_id fk creator_id fk created_on last_modified status
What I want to do is replace assigned_user_id and creator_id with first_name + last_name from users table. So I execute the following query:
SELECT `task_id`, `description`,`assessment`,
(SELECT CONCAT(`first_name`, ' ',`last_name`) WHERE `Tasks`.`assigned_user_id` = `Users`.`user_id`) AS assigned_user,
(SELECT CONCAT(`first_name`, ' ',`last_name`) WHERE `Tasks`.`creator_id`=`Users`.`user_id`) AS creator_user,
`created_on`,`last_modified`,`status`
FROM `Tasks`
LEFT JOIN `Users`
ON Tasks.assigned_user_id = Users.user_id
OR Tasks.creator_id = Users.user_id
WHERE task_id=2
The problem is that it returns 2 rows. One is with assigned_user_id null and filled creator_id and the other is the other way around:
task_id description assessment assigned_user creator_user created_on last_modified status
2 SHA SA PII LI 24 NULL Petyo Chuliuv 2016-07-22 2016-07-22 1
2 SHA SA PII LI 24 Gosho Toshov NULL 2016-07-22 2016-07-22 1
Question is: How to return a single row with assigned_user and creator_user filled and where I did wrong? Thanks in advance!
I tested this on SQL Server and reproduced the same issue as you so hopefully I can be of help.
When I did the test the two SELECT CONCAT statements were using the same user_id both times. So the issue seems that it is not checking for both ids at once but both ids at separate times. So if I were to use your example it first uses Petyo's id in both of the SELECT CONCAT statements (only filling the creator_user role so the other one becomes false) and then it uses Gosho's id in both of the SELECT CONCAT statements which also only fills one field (the assigned_user field) and making the other one null.
So what you need to do is JOIN the Users table again. One for the assigned, one for the create. Something like this...
SELECT `task_id`, `description`,`assessment`,
(SELECT CONCAT(`U1.first_name`, ' ',`U1.last_name`)) AS assigned_user,
(SELECT CONCAT(`U2.first_name`, ' ',`U2.last_name`)) AS creator_user,
`created_on`,`last_modified`,`status`
FROM `Tasks`
LEFT JOIN `Users` U1
ON Tasks.assigned_user_id = U1.user_id
LEFT JOIN `Users` U2
ON Tasks.creator_id = U2.user_id
WHERE task_id=2
Before you had an OR. It does not look at one side, look for the id, then look at the other one, look for the id, then use it at once. It is exactly that. If the current user_id it is looking for happens to be one of those two then it uses that single user_id.
You need to join to your users table twice and alias them...
Somethin like...
SELECT `task_id`, `description`,`assessment`,
(SELECT CONCAT(`assignedUsers.first_name`, ' ',`assignedUsers.last_name`) AS assigned_user,
(SELECT CONCAT(`createdUsers.first_name`, ' ',`createdUsers.last_name`) AS creator_user,
`created_on`,`last_modified`,`status`
FROM `Tasks`
LEFT JOIN `Users` assignedUsers ON Tasks.assigned_user_id = assignedUsers .user_id
LEFT JOIN `Users` createdUsers ON Tasks.creator_id = createdUsers .user_id
WHERE task_id=2
Since you have two foreign keys and you want to fetch the corresponding data you just have to INNER JOIN the users table twice.
You used LEFT JOIN which will fetch all the data from the first table,in your case tasks, even if no match is found in the second table (in your case this did not made any difference but in cases where an id is not set or the user does not exist anymore maybe this is a problem that is up to you to decide...) and you also used OR in the JOIN conditions which resulted in duplicate tasks in the results.
So you must INNER JOIN twice.One time to get the assigned user and one to get the creator.
Havent tested but this should work :
SELECT t.`task_id`, t.`description`,t.`assessment`,
CONCAT(u1.`firstname`,' ',u1.`lastname`) as creator,
CONCAT(u2.`firstname`,' ',u2.`lastname`) as assigned_user,
t.`created_on`,t.`last_modified`,t.`status`
FROM `tasks` t
INNER JOIN `users` u1 ON t.creator_id=u1.id
INNER JOIN `users` u2 ON t.assigned_user_id=u2.id
WHERE t.`task_id`=2
Thank you all guys but I fixed it by doing:
SELECT `task_id`, `description`,`assessment`,
(SELECT CONCAT(`first_name`, ' ', `last_name`)
FROM `Users`
WHERE `Tasks`.`assigned_user_id` = `Users`.`user_id`) AS assigned_user,
(SELECT CONCAT(`first_name`, ' ', `last_name`)
FROM `Users`
WHERE `Tasks`.`creator_id`=`Users`.`user_id`) AS creator_user,
`created_on`,`last_modified`,`status`
FROM `Tasks`
WHERE task_id=3
I just added FROM Users and WHERE clauses in each inner SELECT,
so I didn't have to do any joins... As always it was way more simple than I thought. Thanks again, much appreciated!
How can I optimize this mysql statement?
SELECT DISTINCT p.name
FROM Something_Meta s1
JOIN Something_Meta s2 ON s1.fk_somethingId = s2.fk_somethingId
JOIN Products p ON s2.fk_productId = p.id
JOIN
(
select fk_id from Restricted where fk_foo != 233 and fk_id NOT IN
(
Select fk_id from Restricted where fk_foo = 233
)
)
r ON r.fk_id = p.id
WHERE s1.fk_somethingId = 63 AND s2.fk_somethingId <> s1.fk_somethingId
order by p.name ASC
My tables are like that
Product (id,name )
Restricted (id,fk_id,fk_foo )
Something_Meta (id,fk_id,fk_somethingId )
fk_id is foreign key to product (id)
Probably that sql statement needs optimization..
select fk_id from Restricted where fk_foo != 233 and fk_id NOT IN
(
Select fk_id from Restricted where fk_foo = 233
)
The whole query statement needs more than 1.5 sec to run which are many secs for a website for a single query.
You could try an index on (fk_foo, fk_id) which would cover your entire query:
create index ix_restricted_fk_foo_fk_id(fk_foo, fk_id) on restricted
First off : the DISTINCT is a bit of a red flag. If you need to filter out the doubles then there's probably an error somewhere (either in the query or the design) causing the doubles.
Second; you could write your query like this:
SELECT DISTINCT p.name
FROM Something_Meta s1
JOIN Something_Meta s2 ON s1.fk_somethingId = s2.fk_somethingId
JOIN Products p ON s2.fk_productId = p.id
WHERE s1.fk_somethingId = 63 AND s2.fk_somethingId <> s1.fk_somethingId
AND NOT EXISTS ( SELECT *
FROM Restricted r
WHERE r.fk_foo != 233
AND r.fk_id = p.id )
AND EXISTS ( SELECT *
FROM Restricted r2
WHERE r2.fk_foo = 233
AND r2.fk_id = p.id )
order by p.name ASC
As I don't have a clue about the Something_Meta tables I'll simply focus on the Restricted table and suggest you put an index on fk_foo and fk_id. Said index is NOT part of the query, but rather of the table, so you have to define it upfront, once.
CREATE INDEX idx_Restricted ON Restricted (fk_id, fk_foo)
Once the index is there; any query that might benefit from it will automagically use it in the background; no need for you to adapt the query for it.
Side-note; since you obviously are looking for the product I find it curious you don't 'focus' your query on Products.
SELECT p.name
FROM Products p
JOIN etc...
At the moment I am experiencing some slower MySQL queries in my application which I want to speed up. Unfortunately I’m not quite sure which is the correct way to do it.
I have the following (fictitious) tables: Book, Page and Word.
Word is child of Page by word_page_id
Page is child of Book by page_book id
I already have individual indexes on page_book_id, word_page_id, book_user_id and book_flag_delete.
SELECT `book`.*, COUNT(word_id) AS `word_amount` FROM `book`
LEFT JOIN `page` ON page_book_id = book_id
LEFT JOIN `word` ON word_page_id = paragraph_id
WHERE (book_user_id = 1) AND (book_flag_delete IS NULL)
GROUP BY `book_id`
ORDER BY `book_id` ASC LIMIT 100
SELECT COUNT(DISTINCT `book_id`) AS `book_row_count` FROM `book`
LEFT JOIN `page` ON page_book_id = book_id
LEFT JOIN `word` ON word_page_id = page_id
WHERE (book_user_id = 59) AND (book_flag_delete IS NULL)
Any ideas how to speed up such queries?
Is there extra indexing involved?
Set indexes on the fields you use for joining.
Further make sure that these have both the same datatype, encoding, and collation, else the index will also not be used.
mysql> EXPLAIN <query> will show you the actually used fields (key column in output) and the available indexes (possible_keys output field).
For this query:
SELECT b.*, COUNT(w.word_id) AS `word_amount`
FROM `book` b LEFT JOIN
`page` p
ON p.page_book_id = b.book_id LEFT JOIN
`word` w
ON w.word_page_id = p.paragraph_id
WHERE (b.book_user_id = 1) AND (b.book_flag_delete IS NULL)
GROUP BY b.`book_id`
ORDER BY b.`book_id` ASC
LIMIT 100;
The best indexes are: book(user_id, book_flag_delete, book_id), page(page_book_id, paragraph_id), and word(word_page_id, word_id).
However, the overall group by might be expensive. You might try writing the query as:
SELECT b.*,
(SELECT COUNT(w.word_id)
FROM `page` p JOIN
`word` w
ON w.word_page_id = p.paragraph_id
WHERE p.page_book_id = b.book_id
) AS `word_amount`
FROM `book` b LEFT JOIN
WHERE (b.book_user_id = 1) AND (b.book_flag_delete IS NULL)
ORDER BY b.`book_id` ASC
LIMIT 100;
The same indexes indexes work here. But, this query should avoid a group by on all the data at once (instead, it uses the indexes for the aggregation).
The optimal schema for a many-to-many mapping table is
CREATE TABLE XtoY (
# No surrogate id for this table
x_id MEDIUMINT UNSIGNED NOT NULL, -- For JOINing to one table
y_id MEDIUMINT UNSIGNED NOT NULL, -- For JOINing to the other table
# Include other fields specific to the 'relation'
PRIMARY KEY(x_id, y_id), -- When starting with X
INDEX (y_id, x_id) -- When starting with Y
) ENGINE=InnoDB;
The details on 'why' are in my index cookbook
In your select you're gonna want to refrain from using the wildcard "*" to grab columns. Plus utilize aliases ALWAYS!! This will keep your db from having to create a "virtual" alias.
select book1.column1, book1.column2, page1.column1
from book book1
left join page page1
on page1.page_book_id = book1.book_id
..... blah