I'm a beginner in php and I want to ask you if the query and table schema I have set up is the right way for performance. Note: If you want me to follow a different way, please provide sample for me, thanks
$digerilanlar = DB::get('
SELECT Count(siparisler.hid) AS siparissayisi,
siparisler.hid, ilanlar.id, ilanlar.seflink, ilanlar.kategori, ilanlar.baslik,
ilanlar.yayin, ilanlar.tutar, ilanlar.sure, ilanlar.onecikan, ilanlar.guncellemetarihi,
uyeler.nick, uyeler.foto, uyeler.online, uyeler.ban FROM ilanlar
inner join uyeler ON uyeler.id=ilanlar.ilansahibi
LEFT JOIN siparisler ON ilanlar.id = siparisler.hid
WHERE ilanlar.kategori= '.$kat->id.' and ilanlar.yayin=1 and uyeler.ban=0
GROUP BY ilanlar.id
ORDER BY guncellemetarihi DESC
LIMIT 0,12');
DATABASE DESİGN
Table engine MyISAM MYSQL versiyon 5.7.14
TABLE:İLANLAR
ilansahibi (int)= index
kategori (int)= index
yayin (int)= index
TABLE:UYELER
ban (int)= index
TABLE:SİPARİSLER
hid (int)= index
This will probably require two temp tables and two sorts:
GROUP BY ilanlar.id
ORDER BY guncellemetarihi DESC
Assuming that guncellemetarihi is update_date, this is not identical, but probably gives you what you want, but with only one temp table and sort:
GROUP BY guncellemetarihi, id
ORDER BY guncellemetarihi DESC, id DESC
COUNT(x) checks x for being NOT NULL. If that is not necessary, simply do COUNT(*).
SELECT COUNT(hid), hid
does not make sense. The COUNT implies that there may be multiple "hids", but hid implies that there is only one. (Since I don't understand to objective, I cannot advise which direction to change things.)
This composite INDEX may help:
ilanlar: INDEX(kategori, yayin, ilansahibi, id)
You should switch from ENGINE=MyISAM to ENGINE=InnoDB.
More on making indexes: Index Cookbook
To discuss further, please provide SHOW CREATE TABLE and EXPLAIN SELECT ...
Related
I am using mysql.
And create index on 'playCount' 'desc' in table D.
However, it did not apply.
So, I create index on 'aId ASC, playCount DESC' in table D.
But, it did not apply too.
Order by is so slow, please tell me how to create an index on my code.
explain SELECT `A`.`id` AS `id`, `A`.`title` AS `title`, `A`.`img` AS `img`
FROM `A` `A`
INNER JOIN `B` `B` ON `B`.`aId`=`A`.`id`
INNER JOIN `C` `C` ON `C`.`id`=`B`.`cId`
LEFT JOIN `D` `D` ON `D`.`aId`=`A`.`id`
GROUP BY `A`.`id`
ORDER BY `D`.`playCount` DESC
LIMIT 10;
There may be at least 2 reasons why the ORDER BY may be ignored.
That query will be performed this way:
Join together all valid combinations (based on the ONs) of rows among those tables. This generates a potentially large temp table. This temp table will include a bunch of columns -- title, etc.
Perform the GROUP BY. This probably requires sorting the temp table above. This may shrink back down to a smaller temp table. Since this GROUP BY must be performed before the ORDER BY, no index relating to the ORDER BY can help.
Sort (again) to get the desired ORDER BY.
Deliver the first 10 rows. This effectively tosses any bulky thing (title?) (except for the first 10) that had been carried around since step 1.
If there were a WHERE clause, the addition of an INDEX might help.
INDEX(aId ASC, playCount DESC) -- Well, I need to ask what version of MySQL you are using. Mixing ASC and DESC has always been allowed, and the sorting has always worked correctly. But DESC has been ignored in the index until version 8.0. (Still, as I have already pointed out, the index cannot be used.)
If you want to discuss this further, please provide SHOW CREATE TABLE for each table, EXPLAIN SELECT ..., the approximate size of each table, and whether the tables are related 1:1 or many:many or many:1.
tag_relation table has tag_id and comment_id fields only and both of them are indexed. (there is no primary) it has InnoDB type.
Following query takes long time to execute. How can I make it faster?
All comment_id, tag_id, status, datetime fields are indexed. I really have no idea how to optimize it further.
SELECT
text
FROM comment
INNER JOIN tag_relation
ON tag_relation.comment_id=comment.comment_id
WHERE tag_id='1022278'
AND status=1
ORDER BY comment.datetime DESC LIMIT 0,20
Main cause of slowness is tag_relation table which has 1.5 million records. When it has less records execution time was faster.
Query plan:
This is your query:
SELECT c.text
FROM comment c INNER JOIN
tag_relation tr
ON tr.comment_id = c.comment_id
WHERE t.tag_id = 1022278 AND c.status = 1
ORDER BY c.datetime DESC
LIMIT 0, 20;
First, notice that I removed the single quotes from the value 1022278. If this is really a number, the single quotes can sometimes confuse SQL optimizers. There are two ways to go about optimizing this query, depending on the selectivity of the various conditions. The first is to have the indexes:
tag_relation(tag_id, comment_id)
comment(comment_id, status, datetime, text)
The second is a covering index for comments, and the most important part is the comment_id column.
The second is:
comment(status, comment_id, datetime)
tag_relation(comment_id, tag_id)
The basic issue is which table is scanned first for the join. Using this index, the query would be processed as:
SELECT c.text
FROM comment c INNER JOIN
tag_relation
WHERE c.status = 1 AND
EXISTS (SELECT 1
FROM tag_relation tr
WHERE tr.comment_id = c.comment_id AND tr.tag_id = 1022278
)
ORDER BY c.datetime DESC
LIMIT 0, 20;
I'm not 100% sure that this avoids the file sort on the result set, but it might work.
If I get it right you have one index for tag_id and another index for comment_id. Try creating an composite index like:
create index ... on tag_relation(tag_id, comment_id)
This will make the index with tag_id redundant so it can be dropped.
AFAIK MySQL cannot do index anding, but even if it could a composite index would be more efficient.
I think the problem is in the "status" field. Although it is indexed, the index is not being used. It says "using where" for that table. You can force the use of the index for status but I'm not sure it will be useful, depending on selectivity, i.e., how many different values can "status" take. Alternatively, the documentation says that if "status" allows for NULL then you'll see the "using where". Does it allow for NULLs? If so, consider restricting it.
I just noticed that I overlooked the "ORDER BY", comment.datetime will need an index.
If you already have an index, then try a subquery:
SELECT text
FROM tag_relation
INNER JOIN (SELECT c.comment_id, c.text, c.datetime
FROM comment c
WHERE c.status = 1) comment
ON tag_relation.comment_id = comment.comment_id
WHERE tag_id='1022278'
ORDER BY comment.datetime DESC LIMIT 0,20
I have a left join:
$query = "SELECT a.`id`, a.`documenttitle`, a.`committee`, a.`issuedate`, b.`tagname`
FROM `#__document_management_documents` AS a
LEFT JOIN `#__document_managment_tags` AS b
ON a.id = b.documentid
".$tagexplode."
".$issueDateText."
AND a.committee in (".$committeeQueryTextExplode.")
AND a.documenttitle LIKE '".$documentNameFilter."%'
GROUP BY a.id ORDER BY a.documenttitle ASC
";
It's really slow abaout 7 seconds on 4000 records
Any ideas what I might be doing wrong
SELECT a.`id`, a.`documenttitle`, a.`committee`, a.`issuedate`, b.`tagname`
FROM `w4c_document_management_documents` AS a
LEFT JOIN `document_managment_tags` AS b
ON a.id = b.documentid WHERE a.issuedate >= ''
AND a.committee in ('1','8','9','10','11','12','13','16','17','18','19','20','21','22','23','24','25','26','27','28','29','30','31','32','33','34','35','36','37','38','39','40','41','42','43','44','45','46','47')
AND a.documenttitle LIKE '%' GROUP BY a.id ORDER BY a.documenttitle ASC
I would put an index on a.committee, and full text index the doctitle col. The IN and LIKE are immediate flags to me. Issue date should also have an index because you are >= it
Try running the following commands in a MySQL client:
show index from #__document_management_documents;
show index from #_document_management_tags;
Check to see if there are keys/indexes on the id and documentid fields from the respective tables. If there aren't, MySQL will be doing a full table scan to lookup the values. Creating indexes on these fields makes the search time logarithmic, because it sorts them in a binary tree which is stored in the index file. Even better is to use primary keys (if possible), because that way the row data is stored in the leaf, which saves MySQL another I/O operation to lookup the data.
It could also simply be that the IN and >= operators have bad performance, in which case you might have to rewrite your queries or redesign your tables.
As mentioned above, try to find if your columns have index. You can even do "EXPLAIN" command in your MySQL client at the start of your query to see if the query is actually using indexes. You will see in the 'key' columns and 'Extra' column. Get more information here
This will help you optimize your query. Also group by causes using temporary and filesort which causes MySQL to create a temporary table and going through each rows. If you could use PHP to group by it would be faster.
Say I have an Order table that has 100+ columns and 1 million rows. It has a PK on OrderID and FK constraint StoreID --> Store.StoreID.
1) select * from 'Order' order by OrderID desc limit 10;
the above takes a few milliseconds.
2) select * from 'Order' o join 'Store' s on s.StoreID = o.StoreID order by OrderID desc limit 10;
this somehow can take up to many seconds. The more inner joins I add, slows it down further more.
3) select OrderID, column1 from 'Order' o join 'Store' s on s.StoreID = o.StoreID order by OrderID desc limit 10;
this seems to speed the execution up, by limiting the columns we select.
There are a few points that I dont understand here and would really appreciate it if anyone more knowledgeable with mysql (or rmdb query execution in general) can enlighten me.
Query 1 is fast since it's just a reverse lookup by PK and DB only needs to return the first 10 rows it encountered.
I don't see why Query 2 should take for ever. Shouldn't the operation be the same? i.e. get the first 10 rows by PK and then join with other tables. Since there's a FK constraint, it is guaranteed that the relationship will be satisfied. So DB doesn't need to join more rows than necessary and then trim the result, right? Unless, FK constraint allows null FK? In which case I guess a left join would make this much faster than an inner join?
Lastly, I'm guess query 3 is simply faster because less columns are used in those unnecessary joins? But why would the query execution need the other columns while joining? Shouldn't it just join using PKs first, and then get the columns for just the 10 rows?
Thanks!
My understanding is that the mysql engine applies limit after any join's happen.
From http://dev.mysql.com/doc/refman/5.0/en/select.html, The HAVING clause is applied nearly last, just before items are sent to the client, with no optimization. (LIMIT is applied after HAVING.)
EDIT: You could try using this query to take advantage of the PK speed.
select * from (select * from 'Order' order by OrderID desc limit 10) o
join 'Store' s on s.StoreID = o.StoreID;
All of your examples are asking for tablescans of the existing tables, so none of them will be more or less performant than the degree to which mysql can cache the data or results. Some of your queries have order by or join criteria, which can take advantage of indexes purely to make the joining process more efficient, however, that still is not the same as having a set of criteria that will trigger the use of indexes.
Limit is not a criteria -- it can be thought of as filtration once a result set is determined. You save time on the client, once the result set is prepared, but not on the server.
Really, the only way to get the answers you are seeking is to become familiar with:
EXPLAIN EXTENDED your_sql_statement
The output of EXPLAIN will show you how many rows are being looked at by mysql, as well as whether or not any indexes are being used.
Suppose you have a query like this...
SELECT T.TaskID, T.TaskName, TAU.AssignedUsers
FROM `tasks` T
LEFT OUTER JOIN (
SELECT TaskID, GROUP_CONCAT(U.FirstName, ' ',
U.LastName SEPARATOR ', ') AS AssignedUsers
FROM `tasks_assigned_users` TAU
INNER JOIN `users` U ON (TAU.UserID=U.UserID)
GROUP BY TaskID
) TAU ON (T.TaskID=TAU.TaskID)
Multiple people can be assigned to a given task. The purpose of this query is to show one row per task, but with the people assigned to the task in a single column
Now... suppose you have the proper indexes setup on tasks, users, and tasks_assigned_users. The MySQL Optimizer will still not use the TaskID index when joining tasks to the derived table. WTF?!?!?
So, my question is... how can you make this query use the index on tasks_assigned_users.TaskID? Temporary tables are lame, so if that's the only solution... the MySQL Optimizer is stupid.
Indexes used:
tasks
PRIMARY - TaskID
users
PRIMARY - UserID
tasks_assigned_users
PRIMARY - (TaskID,UserID)
Additional index UNIQUE - (UserID,TaskID)
EDIT: Also, this page says that derived tables are executed/materialized before joins occur. Why not re-use the keys to perform the join?
EDIT 2: MySQL Optimizer won't let you put index hints on derived tables (presumably because there are no indexes on derived tables)
EDIT 3: Here is a really nice blog post about this: http://venublog.com/2010/03/06/how-to-improve-subqueries-derived-tables-performance/ Notice that Case #2 is the solution I'm looking for, but it appears that MySQL does not support this at this time. :(
EDIT 4: Just found this: "As of MySQL 5.6.3, the optimizer more efficiently handles subqueries in the FROM clause (that is, derived tables):... During query execution, the optimizer may add an index to a derived table to speed up row retrieval from it." Seems promising...
There is a solution to this in MySQL Server 5.6 - the preview release (at the time of this writing).
http://dev.mysql.com/doc/refman/5.6/en/from-clause-subquery-optimization.html
Although, I'm not sure if the MySQL Optimizer will re-use indexes that already exist when it "adds indexes to the derived table"
Consider the following query:
SELECT * FROM t1
JOIN (SELECT * FROM t2) AS derived_t2 ON t1.f1=derived_t2.f1;
The documentation says: "The optimizer constructs an index over column f1 from derived_t2 if doing so would permit the use of ref access for the lowest cost execution plan."
OK, that's great, but does the optimizer re-use indexes from t2? In other words, what if an index existed for t2.f1? Does this index get re-used, or does the optimizer recreate this index for the derived table? Who knows?
EDIT: The best solution until MySQL 5.6 is to create a temporary table, create an index on that table, and then run the SELECT query on the temp table.
The problem I see is that by doing a subquery there is no underlying indexed table.
If you are having a performance I'd do the grouping at the end, something like this:
SELECT T.TaskID, T.TaskName, GROUP_CONCAT(U.FirstName, ' ', U.LastName SEPARATOR ', ') AS AssignedUsers
FROM `tasks` T
LEFT OUTER JOIN `tasks_assigned_users` TAU ON (T.TaskID=TAU.TaskID)
INNER JOIN `users` U ON (TAU.UserID=U.UserID)
GROUP BY T.TaskID, T.TaskName
I'm afraid, it's not possible. You have to create a temporary table or a view to use an index.