Optimize Mysql query with boolean value - mysql

I have this table:
CREATE TABLE IF NOT EXISTS `products` (
`product_id` int(11) NOT NULL AUTO_INCREMENT,
`supplier_id` int(11) NOT NULL,
`allowed` varchar(256) NOT NULL,
`blocked` varchar(256) NOT NULL,
`approved` tinyint(1) NOT NULL DEFAULT '0',
PRIMARY KEY (`product_id`),
KEY `supplier_id` (`supplier_id`),
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
'approved' is a boolean 0/1 field
'blocked' and 'allowed' hold country codes such as "US CA FR"
I run this query:
SELECT DISTINCT supplier_id
FROM products
WHERE (
supplier_id=0 OR
supplier_id = 1207077 OR
supplier_id = 1207087 OR
supplier_id = 1207079 OR
supplier_id = 1207082 OR
supplier_id = 1207083 OR
supplier_id = 1207086 OR
supplier_id = 1207084 OR
supplier_id = 1207078 OR
supplier_id = 1207085 OR
supplier_id = 1207094 OR
supplier_id = 1207097 OR
supplier_id = 1207095 OR
supplier_id = 1207089 OR
supplier_id = 1207091
) AND (
(`blocked` NOT LIKE '%US%' AND `allowed` ='') OR
`allowed` LIKE '%US%'
) AND approved=1;
It runs in about 0.02s. Any suggestions on how to optimize it? Thank you.

The execution speed is the same because OR and non left anchored LIKE clauses cannot use indexes appropriately. You've got bad table design in that US FR etc field, that should be in another table that you join against. If you are stuck with your design and the table is vey large, then create a derived table for the supplier_id OR clauses and then JOIN against the same table in order to find the rest of the matches. This may also require a UNION since you have other OR's. For more information:
http://dev.mysql.com/doc/refman/5.6/en/index-btree-hash.html

I managed to optimize it as follows:
SELECT DISTINCT supplier_id FROM products WHERE ((`blocked` NOT LIKE '%US%' AND
`allowed` ='') OR `allowed` LIKE '%US%') AND approved=1 AND supplier_id IN (1207077,
1207087, 1207079, 1207082, 1207083, 1207086, 1207084, 1207078, 1207085, 1207094,
1207097, 1207095, 1207089, 1207091);
It runs at 0.0004s now.
Thank you winmutt for pointing me to the right direction :-)

Related

Find records, which have several specific records in a joined table

I have two tables as follows:
create table gift_certificate
(
id int auto_increment
primary key,
name varchar(64) not null,
description mediumtext not null,
price decimal default 0 not null,
duration int default 1 not null,
create_date datetime not null,
last_update_date datetime not null
)
and
create table tag
(
id int auto_increment
primary key,
name varchar(64) not null,
constraint tag_name_uindex
unique (name)
)
with a linking table:
create table gift_certificate__tag
(
certificate_id int not null,
tag_id int not null,
primary key (certificate_id, tag_id),
constraint gift_certificate__tag_gift_certificate_id_fk
foreign key (certificate_id) references gift_certificate (id),
constraint gift_certificate__tag_tag_id_fk
foreign key (tag_id) references tag (id)
)
I need to search for gift certificates by several tags (“and” condition). I only came up with a solution for one tag
select distinct gc.*, tag.* from gift_certificate gc
left outer join gift_certificate__tag joint on gc.id=joint.certificate_id
left outer join tag on joint.tag_id=tag.id
where tag.name='puppy'
order by gc.id desc;
Would be grateful for some support
You can aggregate the joint table by certificate and use HAVING to only keep certificates that have all the tags. Then select all matching certificates using an IN clause. For instance:
select *
from gift_certificate
where id in
(
select joint.certificate_id
from gift_certificate__tag joint
join tag on joint.tag_id=tag.id
group by joint.certificate_id
having max(case when tag.name = 'puppy' then 1 else 0 end) = 1
and max(case when tag.name = 'something' then 1 else 0 end) = 1
);
As true = 1 and false = 0 in MySQL, you can shorten the expression to
having max(tag.name = 'puppy')
if you find this readable. Or
having sum(tag.name = 'puppy') > 0

Can I improve my movie selecting SQL query

I've created a database to store movies data. My tables are the following:
movies:
CREATE TABLE IF NOT EXISTS `movies` (
`movieId` int(11) NOT NULL AUTO_INCREMENT,
`imdbId` varchar(255) DEFAULT NULL,
`imdbRating` float DEFAULT NULL,
`movieTitle` varchar(255) NOT NULL,
`movieLength` varchar(255) NOT NULL,
`imdbRatingCount` varchar(255) NOT NULL,
`poster` varchar(255) NOT NULL,
`year` varchar(255) NOT NULL,
PRIMARY KEY (`movieId`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=1 ;
I have a table in which i store movie actors:
CREATE TABLE IF NOT EXISTS `actors` (
`actorId` int(10) NOT NULL AUTO_INCREMENT,
`actorName` varchar(255) NOT NULL,
PRIMARY KEY (`actorId`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=1 ;
And one other in which i store the relation between the movies and actors: (movieActor)
CREATE TABLE IF NOT EXISTS `movieActor` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`movieId` int(10) NOT NULL,
`actorId` int(10) NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Now when i want to select a list of movies in which are the selected actors my query is:
SELECT *
FROM movies m inner join
(SELECT movieId FROM movieActor WHERE actorId IN(1,2,3) GROUP BY movieId having count(*) = 3) ma ON m.movieId = ma.movieId
WHERE imdbRating IS NOT NULL ORDER BY imdbRating DESC
This is working perfectly, but i don't know that this is the optimal table structure and query to accomplish this. Are there any better table structure to store data or query the list?
First of all, use indexes on your tables. In my opinion it should be useful to have 3 indexes on movieActor. MovieId - ActorID - MovieIdActorId.
Second try tu use foreign keys. These help to identify the best execution plan for your dbs.
Third try to avoid generating temp tables in your execution plan of your query. Subselects often creates temp tables which are used when the database has to temporarily save something in the RAM. To check this, write EXPLAIN in front of goer query.
I would write it like this:
SELECT m.*, movieActor
FROM movies m inner join
movieActor ma ON m.movieId = ma.movieId
WHERE imdbRating IS NOT NULL
and actorId IN(1,2,3)
GROUP BY movieId
having count(*) = 3)
ORDER BY imdbRating DESC
(Not tested)
Just try to optimize it with the EXPLAIN keyword. It also can help you to create the right indexes.

Assistance with MySQL JOIN

I have two tables, one with categories and subcategories. Each category and subcategory has an id and if it's a subcategory, it's got a topid != 0 referring what it's a subcategory of. The other table "markers" has a field 'cat' which correlates with the category field 'name' Now I want to select everything from markers with category.id = 4 OR category.topid = 4 so I tried this query:
SELECT * FROM `xymply_markers`
JOIN `xymply_categories`
ON xymply_markers.cat = xymply_categories.name
WHERE xymply_categories.topid=4
OR xymply_categories.id=4
Which doesn't return me anything even tho I do have such elements in my table "markers". Any assistance would be appreciated!
Table schemas:
`xymply_categories` (
`id` int(11) NOT NULL auto_increment,
`topid` int(11) NOT NULL,
`name` text NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=MyISAM AUTO_INCREMENT=15 DEFAULT CHARSET=utf8 AUTO_INCREMENT=15 ;
`xymply_markers` (
`created` date NOT NULL,
`id` int(11) NOT NULL auto_increment,
`sdate` date NOT NULL,
`hdate` date NOT NULL,
`name` varchar(60) NOT NULL,
`address` varchar(80) NOT NULL,
`unit` varchar(6) NOT NULL,
`lat` decimal(10,7) NOT NULL,
`lng` decimal(10,7) NOT NULL,
`type` varchar(30) NOT NULL,
`userid` int(11) NOT NULL,
`adtext` text NOT NULL,
`phone` varchar(20) NOT NULL,
`email` varchar(30) NOT NULL,
`url` varchar(30) NOT NULL,
`cat` varchar(4) NOT NULL,
UNIQUE KEY `id` (`id`)
) ENGINE=MyISAM AUTO_INCREMENT=151 DEFAULT CHARSET=utf8 AUTO_INCREMENT=151 ;
Sample Data:
xymply_categories:
id 1
topid 0
name 'vehicle'
--------------
id 2
topid 1
name 'bike'
--------------
id 3
topid 1
name 'truck'
xymply_markers:
id 1
sdate 2012-03-01
hdate 2012-04-01
name 'TEST'
address '1234 TEST'
unit''
lat 49.0
lng -123.0
adtext 'TEST'
phone '1234567890'
email 'email#email.com'
url 'www.url.com'
cat 'bike'
--------------
id 1
sdate 2012-03-01
hdate 2012-04-01
name 'TEST'
address '1234 TEST'
unit''
lat 49.5
lng -123.5
adtext 'TEST'
phone '1234567890'
email 'email#email.com'
url 'www.url.com'
cat 'vehicle'
One problem is that the xymply_markers.cat field is VARCHAR(4) but the xymply_categories.name field is TEXT, and contains values longer than 4 characters. Either you're not giving us the accurate schema, or you're confused about which columns join, or you're never going to see any trucks or vehicles. Columns which join should have the same type almost without exception (I've never seen a good reason for an exception).
You are then asking about id = 4 or topid = 4, but the sample data you show only has id = 1 or topid = 1. Do you actually have data where id = 4 or topid = 4 in the system?
Between these two lots of confusion, it is hard to know what we're up against. If you have data that joins and has the relevant topid or id values, then your query should work.
I have a field called 'id' in both tables. How can I control which one I'm accessing with PHP after I read data into the array with $row = #mysql_fetch_assoc($result)?
The simplest way is to ensure that each result column has a unique name, creating one with an 'alias', as in:
SELECT c.id AS category_id,
c.topid,
c.name AS category_name,
m.id AS marker_id,
m.name AS marker_name,
...
PHP will associate the alias names with the the data in the row.

Some help needed with a SQL query

I need some help with a MySQL query. I have two tables, one with offers and one with statuses. An offer can has one or more statuses. What I would like to do is get all the offers and their latest status. For each status there's a table field named 'added' which can be used for sorting.
I know this can be easily done with two queries, but I need to make it with only one because I also have to apply some filters later in the project.
Here's my setup:
CREATE TABLE `test`.`offers` (
`id` INT UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY ,
`client` TEXT NOT NULL ,
`products` TEXT NOT NULL ,
`contact` TEXT NOT NULL
) ENGINE = MYISAM ;
CREATE TABLE `statuses` (
`id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`offer_id` int(11) NOT NULL,
`options` text NOT NULL,
`deadline` date NOT NULL,
`added` datetime NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1
Should work but not very optimal imho :
SELECT *
FROM offers
INNER JOIN statuses ON (statuses.offer_id = offers.id
AND statuses.id =
(SELECT allStatuses.id
FROM statuses allStatuses
WHERE allStatuses.offer_id = offers.id
ORDER BY allStatuses.added DESC LIMIT 1))
Try this:
SELECT
o.*
FROM offers o
INNER JOIN statuses s ON o.id = s.offer_id
ORDER BY s.added
LIMIT 1

MySQL query killing my server

Looking at this query there's got to be something bogging it down that I'm not noticing. I ran it for 7 minutes and it only updated 2 rows.
//set product count for makes
$tru->query->run(array(
'name' => 'get-make-list',
'sql' => 'SELECT id, name FROM vehicle_make',
'connection' => 'core'
));
while($tempMake = $tru->query->getArray('get-make-list')) {
$tru->query->run(array(
'name' => 'update-product-count',
'sql' => 'UPDATE vehicle_make SET product_count = (
SELECT COUNT(product_id) FROM taxonomy_master WHERE v_id IN (
SELECT id FROM vehicle_catalog WHERE make_id = '.$tempMake['id'].'
)
) WHERE id = '.$tempMake['id'],
'connection' => 'core'
));
}
I'm sure this query can be optimized to perform better, but I can't think of how to do it.
vehicle_make = 45 rows
taxonomy_master = 11,223 rows
vehicle_catalog = 5,108 rows
All tables have appropriate indexes
UPDATE: I should note that this is a 1-time script so overhead isn't a big deal as long as it runs.
CREATE TABLE IF NOT EXISTS `vehicle_make` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(32) NOT NULL,
`product_count` int(11) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1 AUTO_INCREMENT=46 ;
CREATE TABLE IF NOT EXISTS `taxonomy_master` (
`product_id` int(10) NOT NULL,
`v_id` int(10) NOT NULL,
`vehicle_requirement` varchar(255) DEFAULT NULL,
`is_sellable` enum('True','False') DEFAULT 'True',
`programming_override` varchar(25) DEFAULT NULL,
PRIMARY KEY (`product_id`,`v_id`),
KEY `idx2` (`product_id`),
KEY `idx3` (`v_id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;
CREATE TABLE IF NOT EXISTS `vehicle_catalog` (
`v_id` int(10) NOT NULL,
`id` int(11) NOT NULL,
`v_make` varchar(255) NOT NULL,
`make_id` int(11) NOT NULL,
`v_model` varchar(255) NOT NULL,
`model_id` int(11) NOT NULL,
`v_year` varchar(255) NOT NULL,
PRIMARY KEY (`v_id`,`v_make`,`v_model`,`v_year`),
UNIQUE KEY `idx` (`v_make`,`v_model`,`v_year`),
UNIQUE KEY `idx2` (`v_id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;
Update: The successful query to get what I needed is here....
SELECT
m.id,COUNT(t.product_id) AS CountOf
FROM taxonomy_master t
INNER JOIN vehicle_catalog v ON t.v_id=v.id
INNER JOIN vehicle_make m ON v.make_id=m.id
GROUP BY m.id;
without the tables/columns this is my best guess from reverse engineering the given queries:
UPDATE m
SET product_count =COUNT(t.product_id)
FROM taxonomy_master t
INNER JOIN vehicle_catalog v ON t.v_id=v.id
INNER JOIN vehicle_make m ON v.make_id=m.id
GROUP BY m.name
The given code loops over each make, and then runs a query the counts for each. My answer just does them all in one query and should be a lot faster.
have an index for each of these:
vehicle_make.id cover on name
vehicle_catalog.id cover make_id
taxonomy_master.v_id
EDIT
give this a try:
CREATE TEMPORARY TABLE CountsOf (
id int(11) NOT NULL
, CountOf int(11) NOT NULL DEFAULT 0.00
);
INSERT INTO CountsOf
(id, CountOf )
SELECT
m.id,COUNT(t.product_id) AS CountOf
FROM taxonomy_master t
INNER JOIN vehicle_catalog v ON t.v_id=v.id
INNER JOIN vehicle_make m ON v.make_id=m.id
GROUP BY m.id;
UPDATE taxonomy_master,CountsOf
SET taxonomy_master.product_count=CountsOf.CountOf
WHERE taxonomy_master.id=CountsOf.id;
instead of using nested query ,
you can separated this query to 2 or 3 queries,
and in php insert the result of the inner query to the out query ,
its faster !
#haim-evgi Separating the queries will not increase the speed significantly, it will just shift the load from the DB server to the Web server and create overhead of moving data between the two servers.
I am not sure with the appropriate indexes you run such query 7 minutes. Could you please show the table structure of the tables involved in these queries.
Seems like you need the following indices:
INDEX BTREE('make_id') on vehicle_catalog
INDEX BTREE('v_id') on taxonomy_master