I have two MySQL tables: "list" and "more"
The "list" table has upwards of 100,000 results.
The "more" table has upwards of 50,000 results.
The data from these tables cannot be combined.
The issue is, I have a column called "short_title" in both tables. They are both VARCHAR(255) and will contain a string like "short-title-here".
I'm using a simple query like:
SELECT L.title, M.more_info
FROM `list` L, `more` M
WHERE M.short_title = L.short_title
Since there are so many results in each table, and I need to be matching the results based on the "short_title" column which is a text field, it makes the queries EXTREMELY slow.
There is an INDEX on the "short_title" column in the "list" table, and the "short_title" column in my "more" table is UNIQUE
Is there anything I can do to the column (example: making them fulltext) that will make these queries faster?
Thank you in advance
**** UPDATE ****
I've changed my query to INNER JOIN the two tables.
The results of the explain query can be found here:
Try indexing the short_title fields.
This is your query (written using explicit joins):
SELECT L.title, M.more_info
FROM list L join
more M
on M.short_title = L.short_title;
You state that short_title is being stored as text. You can create a regular index on it by doing:
create index more_short_title on more(short_title(255));
The 255 is the number of bytes to be indexed. This should speed up the queries.
Also, with a name like short_title, why not just use a varchar() column rather than text?
Related
now i'm trying to figure out, what should i do, to improve my query result.
Now, it's 47.55.
So, should i create any indexes for columns? Tell me please
SELECT bw.workloadId, lrer.lecturerId, lrer.lastname, lrer.name, lrer.fathername, bt.title, ac.activityname, cast(bw.exactday as char(45)) as "date", bw.exacttime as "time" FROM base_workload as bw
right join unioncourse as uc on uc.idunioncourse = bw.idunioncourse
right join basecoursea as bc on bc.idbasecoursea = uc.idbasecourse
right join lecturer as lrer on lrer.lecturerId = uc.lecturerId
right join basetitle as bt on bt.idbasetitle = bc.idbasetitle
right join activity as ac on ac.activityId = bc.activityId
where lrer.lecturerId is not null AND bc.idbasecoursea is not null and bw.idunioncourse != ""
ORDER BY bw.exactday, bw.exacttime ASC;
From MySQL 8.0 documentation:
Indexes are used to find rows with specific column values quickly. Without an index, MySQL must begin with the first row and then read through the entire table to find the relevant rows. The larger the table, the more this costs. If the table has an index for the columns in question, MySQL can quickly determine the position to seek to in the middle of the data file without having to look at all the data. This is much faster than reading every row sequentially.
MySQL use indexes for these operations:
To find the rows matching a WHERE clause quickly.
To eliminate rows from consideration.
If the table has a multiple-column index, any leftmost prefix of the index can be used by the optimizer to look up rows.
To retrieve rows from other tables when performing joins.
To find the MIN() or MAX() value for a specific indexed column key_col.
To sort or group a table if the sorting or grouping is done on a leftmost prefix of a usable index (for example, ORDER BY key_part1, key_part2).
In some cases, a query can be optimized to retrieve values without consulting the data rows.
As of your requirements, you could use index on the WHERE clause for faster data retrieval.
I think you can get rid of
lrer.lecturerId is not null
AND bc.idbasecoursea is not null
By changing the first 3 RIGHT JOINs to JOINs.
What is the datatype of exactday? What is the purpose of
cast(bw.exactday as char(45)) as "date"
The CAST may be unnecessary.
Re bw.exactday, bw.exacttime: It is usually better to use a single column for DATETIME instead of two columns (DATE and TIME).
What are the PRIMARY KEYs of the tables?
Please convert to LEFT JOIN if possible; I can't wrap my head around RIGHT JOINs.
This index on bw may help: INDEX(exactday, exacttime).
I must run this query with MySQL:
select requests.id, requests.id_temp, categories.id
from opadithree.requests inner join
opadi.request_detail_2
on substring(requests.id_sub_temp, 3) = request_detail_2.id inner join
opadithree.categories
on request_detail_2.theme = categories.cu_code
where categories.atc = false and id_sub_temp like "2_%";
However for some reason the query is too slow. The table requests has 15583 rows. The table request_detail_2 66469 rows and the table categories has 13452 rows.
The most problematic column id_sub_temp has data strings in the following formats: "2_number" or "3_number".
Do you know some trick to make the query faster?
Here are the indexes I'd try:
First, I need an index so your WHERE condition on id_sub_temp can find the rows needed efficiently. Then add the column id_temp so the result can select that column from the index instead of forcing it to read the row.
CREATE INDEX bk1 ON requests (id_sub_temp, id_temp);
Next I'd like the join to categories to filter by atc=false and then match the cu_code. I tried reversing the order of these columns so cu_code was first, but that resulted in an expensive index-scan instead of a lookup. Maybe that was only because I was testing with empty tables. Anyway, I don't think the column order is important in this case.
CREATE INDEX bk2 ON categories (atc, cu_code);
The join to request_detail_2 is currently by primary key, which is already pretty efficient.
I want to get the rows from table es_goodsSearch where the column goodsNm
contains any word in column named word in es_manageCleanWords table which contains about 10,000 rows in entire table.
So I made query in following ways.
SELECT * FROM es_goodsSearch g
LEFT JOIN es_manageCleanWords as emc
ON g.goodsNm LIKE CONCAT('%', emc.word,'%');
The problem is query is too slow to use it on production. How can I solve this problem?
Here is my explain results and index results of following table.
es_goodsSearch explain image
es_manageCleanWords explain image
es_goodsSearch index info
es_manageCleanWords index info
I have the following query:
SELECT region.id, region.world_id, min_x, min_y, min_z, max_x, max_y, max_z, version, mint_version
FROM minecraft_worldguard.region
LEFT JOIN minecraft_worldguard.region_cuboid
ON region.id = region_cuboid.region_id
AND region.world_id = region_cuboid.world_id
LEFT JOIN minecraft_srvr.lot_version
ON id=lot
WHERE region.world_id = 10
AND region_cuboid.world_id=10;
The Mysql slow query log tells me that it takes more than 5 seconds to execute, returns 2300 rows but examines 15'404'545 rows to return it.
The three tables each have bout 6500 rows only with unique keys on the id and lot fields as well as keys on the world_id fields. I tried to minimize the amount of rows examined by filtering both cuboid and world by their ID and the double WHERE on world_id, but it did not seem to help.
Any idea how I can optimize this query?
Here is the sqlfiddle with the indexes as of current status.
MySQL can't use index in this case because joined fields has different data types:
`lot` varchar(20) COLLATE utf8_unicode_ci NOT NULL
`id` varchar(128) COLLATE utf8_bin NOT NULL
If you change types of this fields to general type (for example, region.id to utf8_unicode_ci), MySQL uses primary key (fiddle).
According to docs:
Comparison of dissimilar columns (comparing a string column to a
temporal or numeric column, for example) may prevent use of indexes if
values cannot be compared directly without conversion.
You have joined the two tables "minecraft_worldguard.region" and "minecraft_worldguard.region_cuboid", on region.world_id and region_cuboid.world_id. So WHERE clause wouldn't require two conditions.
The two columns in the WHERE clause have been equated in the JOIN condition, hence you wouldn't require checking both the conditions in the WHERE clause. Remove one of them in the WHERE clause and add an index on the column that is remaining on the WHERE condition.
In your example, leave the WHERE clause as below:
WHERE region.world_id = 10
and add an index on the region.world_id column, that would improve the performance a bit.
NOTE: observe that I am suggesting you to discard "AND region_cuboid.world_id=10;" part of the WHERE clause.
Hope that helps.
First, when writing queries that have multiple tables, it is a very good thing to get used to "alias" references to the tables so you don't have to retype the entire long name throughout. Also, it is a really good idea to identify which tables the columns are coming from to allow users to better understand what is where which can also help improve performance (such as suggesting a covering index).
That said, I have applied aliases to your original query, but AM GUESSING the table per the respective columns, but you can obviously identify quickly and adjust.
SELECT
R.id,
R.world_id,
RC.min_x,
RC.min_y,
RC.min_z,
RC.max_x,
RC.max_y,
RC.max_z,
LV.version,
LV.mint_version
FROM
minecraft_worldguard.region R
LEFT JOIN minecraft_worldguard.region_cuboid RC
ON R.id = RC.region_id
AND R.world_id = RC.world_id
LEFT JOIN minecraft_srvr.lot_version LV
ON R.id = LV.lot
WHERE
R.world_id = 10
I also removed from the where clause your "region_cuboid.world_id = 10" as that is redundant as a result of the JOIN clause based on region AND world.
For suggestion of indexes, and if I have the proper alias references to the columns, I would suggest a covering index on the region table of
( world_id, id ). The "World_id" in the first position quickly qualifies the WHERE clause, and the "id" is there for the RC and LV tables.
For the region_cuboid table, I would also have an index on ( world_id, region_id) to match the region table being joined to it.
For the lot_version table, and index on (lot) or a covering index on (lot, version, mint_version)
Is it possible to create a full text index on a VIEW?
If so, given two columns column1 and column2 on a VIEW, what is the SQL to get this done?
The reason I'd like to do this is I have two very large tables, where I need to do a FULLTEXT search of a single column on each table and combine the results. The results need to be ordered as a single unit.
Suggestions?
EDIT: This was my attempt at creating a UNION and ordering by each statements scoring.
(SELECT a_name AS name, MATCH(a_name) AGAINST('$keyword') as ascore
FROM a WHERE MATCH a_name AGAINST('$keyword'))
UNION
(SELECT s_name AS name,MATCH(s_name) AGAINST('$keyword') as sscore
FROM s WHERE MATCH s_name AGAINST('$keyword'))
ORDER BY (ascore + sscore) ASC
sscore was not recognized.
MySQL doesn't allow any form of indexes on a view, just on it's underlying tables. The reason for this is because MySQL only materializes a view when you select from it, due to the possibility of the underlying tables changing data all the time. If you had a view that returned 10 million rows, you'd have to apply a full text index to it every time you selected from it, and that takes a lot of time.
If you want full index functionality, then you might as well stick with the SQL script you've posted, and manually (or cronjob a script to) update the fulltext index of both tables on a nightly basis (or hourly if you're in that high traffic market).
A kindof ugly workaround that might be inefficient but gets the job done would be to join the view with the underlying table to access the fulltext columns:
/* join the view with the underlying table, that is used in the view: */
select item.* from items_view as item
left join items as raw_item on raw_item.id = item.id
/* use the `item` from the view as usual: */
where item.foo = 'bar'
/* but use `raw_item` when you need the fulltext index: */
where match (raw_item.content) against ('foo bar')
Of course, this isn't a relevant solution if you need to create a new column in the view and give it the fulltext index, but if you only need to access the original data in this way and you just need the view for other unrelated operations, then this might be the way to go.