I have a MySQL 5.7 table with a binary(16) id column used as PRIMARY key.
The engine is InnoDb.
Rows are created with Doctrine using https://github.com/ramsey/uuid-doctrine#innodb-optimised-binary-uuids as per https://www.percona.com/blog/2014/12/19/store-uuid-optimized-way/
For instance the uuid 55a54172-f5e4-11e8-aa0d-fe02fd3f406 is stored as HEX(id) = 11E8F5E455A54172AA0DFE02FD3F406 in the database
I'm trying to search for rows with the same string beginning.
Something like this with pseudo code
SELECT *
FROM log
WHERE id LIKE 11E8F5E4
I've tried to query
SELECT HEX(`id`) AS `id`
FROM `log`
WHERE (`id` & UNHEX('11E8F5E4')) = UNHEX('11E8F5E4')
LIMIT 10
which is fast but rows with a different prefix like 11E8E8AD47E47970B1C9525400B61105 are returned
How may I do it while using the Btree index (for perf reason)?
I tried the straightforward approach with LIKE, and it seems to be working fine.
You need to make a binary string which has a % at the end:
SELECT ...
FROM table
WHERE field LIKE CONCAT(UNHEX('11E8F5E4'),'%')
Related
There is my SQL query. Table system_mailer is for logging sent e-mails. When i want to search some data, query is 10 seconds long. It is possible on any way to optimize this query?
SELECT `ID`
FROM `system_mailer`
WHERE `group` = 'selling_center'
AND `group_parameter_1` = '1'
AND `group_parameter_2` = '2138'
Timins is around couple of seconds, how could it be optimised?
You might find the following index on 4 columns would help performance:
CREATE INDEX idx ON system_mailer (`group`, group_parameter_1, group_parameter_2, ID);
MySQL should be able to use this index on your current query. By the way, if you are using InnoDB, and ID is the primary key, then you might be able to drop it from the explicit index definition, and just use this:
CREATE INDEX idx ON system_mailer (`group`, group_parameter_1, group_parameter_2);
Please avoid naming your columns and tables with reserved MySQL keywords like group. Because you made this design decision, you will now be forced to forever escape that column name with backticks (ugly).
just be sure you have a composite index on table system_mailer for the columns
(`group`, `group_parameter_1`, `group_parameter_2`)
and you can use redudancy adding the id to index for avoid data table access in query
(`group`, `group_parameter_1`, `group_parameter_2`, ID)
I have the following table in PostgreSQL:
CREATE TABLE index_test
(
id int PRIMARY KEY NOT NULL,
text varchar(2048) NOT NULL,
last_modified timestamp NOT NULL,
value int,
item_type varchar(2046)
);
CREATE INDEX idx_index_type ON index_test ( item_type );
CREATE INDEX idx_index_value ON index_test ( value )
I make the following selects:
explain select * from index_test r where r.item_type='B';
explain select r.value from index_test r where r.value=56;
The explanation of execution plan looks like this:
Seq Scan on index_test r (cost=0.00..1.04 rows=1 width=1576)
Filter: ((item_type)::text = 'B'::text)'
As far as I understand, this is a full table scan. The question is: why my indexes are not used?
May be, the reason is that I have too few rows in my table? I have only 20 of them. Could you please provide me with a SQL statement to easily populate my table with random data to check the indexes issue?
I have found this article: http://it.toolbox.com/blogs/db2luw/how-to-easily-populate-a-table-with-random-data-7888, but it doesn't work for me. The efficiency of the statement does not matter, only the simplicity.
Maybe, the reason is that I have too few rows in my table?
Yes. For a total of 20 rows in a table a seq scan is always going to be faster than an index scan. Chances are that those rows are located in a single database block anyway, so the seq scan would only need a single I/O operation.
If you use
explain (analyze true, verbose true, buffers true) select ....
you can see a bit more details about what is really going on.
Btw: you shouldn't use text as a column name, as that is also a datatype in Postgres (and thus a reserved word).
The example you have found is for DB2, in pg you can use generate_series to do it.
For example like this:
INSERT INTO index_test(data,last_modified,value,item_type)
SELECT
md5(random()::text),now(),floor(random()*100),md5(random()::text)
FROM generate_series(1,1000);
SELECT max(value) from index_test;
http://sqlfiddle.com/#!12/52641/3
The second query in above fiddle should use index only scan.
Is there any way to force MySQL to push a predicate into a view?
Example:
CREATE TABLE t1(
id INT(11) NOT NULL AUTO_INCREMENT
PRIMARY KEY (id)
);
CREATE VIEW v1
AS SELECT * FROM t1;
The query below will not use PRIMARY KEY index in MySQL:
SELECT *
FROM v1
WHERE id = 1
Instead it will select everything from t1, create a derived table and then filter it out for id = 1.
Is there any way to overcome this?
PS: my real life example is a little bit more complex than the one above, but for simplicity I used the example above
PPS: here's a related Stack Overflow question: How do I get MySQL to use an INDEX for view query?
Yes but you'd have to switch to the fully-compatible MariaDB 10.2.2
This is on by default and can be switched off using optimizer_switch:
SET GLOBAL optimizer_switch='condition_pushdown_for_derived=off'
Having this table:
CREATE TABLE `example` (
`id` int(11) unsigned NOT NULL auto_increment,
`keywords` varchar(200) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB;
We would like to optimize the following query:
SELECT id FROM example WHERE keywords LIKE '%whatever%'
The table is InnoDB, (so no FULLTEXT for now) which would be the best index to use in order to optimize such query?
We've tried a simple :
ALTER TABLE `example` ADD INDEX `idxSearch` (`keywords`);
But an explain query shows that need to scan the whole table
if our queries where LIKE 'whatever%' instead, this index performs well, but otherwise has no value.
Is there anyway to optimize this for innoDB ?
Thanks!
Indexes are built from the start of the string towards the end. When you use LIKE 'whatever%' type clause, MySQL can use those start-based indexes to look for whatever very quickly.
But switching to LIKE '%whatever%' removes that anchor at the start of the string. Now the start-based indexes can't be used, because your search term is no longer anchored at the start of the string - it's "floating" somewhere in the middle and the entire field has to be search. Any LIKE '%... query can never use indexes.
That's why you use fulltext indexes if all you're doing are 'floating' searches, because they're designed for that type of usage.
Of major note: InnoDB now supports fulltext indexes as of version 5.6.4. So unless you can't upgrade to at least 5.6.4, there's nothing holding you back from using InnoDB *AND fulltext searches.
I would like to comment that surprisingly, creating an index also helped speed up queries for like '%abc%' queries in my case.
Running MySQL 5.5.50 on Ubuntu (leaving everything on default), I have created a table with a lot of columns and inserted 100,000 dummy entries. In one column, I inserted completely random strings with 32 characters (i.e. they are all unique).
I ran some queries and then added an index on this column.
A simple
select id, searchcolumn from table_x where searchcolumn like '%ABC%'
returns a result in ~2 seconds without the index and in 0.05 seconds with the index.
This does not fit the explanations above (and in many other posts). What could be the reason for that?
EDIT
I have checked the EXPLAIN output. The output says rows is 100,000, but Extra info is "Using where; Using index". So somehow, the DBMS has to search all rows, but still is able to utilise the index?
I have the following data in a MySQL table table:
ID: int(11) [this is the primary key]
Date: date
and I run the MySQL query:
SELECT * from table WHERE Date=CURDATE() and ID=1;
This takes between 0.6 and 1.2 seconds.
Is there any way to optimize this query to get results quicker?
My objective is to find out if I already have a record for today for this ID.
Add indexes on ID and Date.
See CREATE INDEX manual.
You could add a limit 1 at the end, since you are searching for a primary key the max results is 1.
And if you only want to know wether it exists or not you could replace * with ID to select only the ID.
Furthermore, if you haven't already, you really need to add indexes.
SET #cur_date = CURDATE()
...WHERE Date = #cur_date ...
and then create an index of Date, ID (order is important, it should match the order you query on).
In general, calling functions before you do the query and storing them to variables lets SQL treat them like numbers instead of functions, which tends to allow it to use a faster query algorithm.