MySQL: UPDATE with SUM and JOIN - mysql

I'm trying to do a SUM and store it in another table. The SUM is simple :
SELECT award.alias_id,
SUM(award.points) AS points
FROM award
INNER JOIN achiever ON award.id = achiever.award_id
I now want to store that. I figured out how to do it on a row-by-row basis :
UPDATE aliaspoint
SET points = (SELECT SUM(award.points) AS points
FROM award
INNER JOIN achiever ON award.id = achiever.award_id
WHERE achiever.alias_id = 2000)
WHERE alias_id = 2000;
I thought something like this might work but I get:
ERROR 1111 (HY000): Invalid use of group function
UPDATE aliaspoint
INNER JOIN achiever ON aliaspoint.alias_id = achiever.alias_id
INNER JOIN award ON achiever.award_id = award.id
SET aliaspoint.points = SUM(award.points)
And some table definitions to help :
mysql> show create table aliaspoint;
| metaward_aliaspoint | CREATE TABLE `aliaspoint` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`alias_id` int(11) NOT NULL,
`points` double DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `alias_id` (`alias_id`),
KEY `aliaspoint_points` (`points`)
) ENGINE=MyISAM AUTO_INCREMENT=932081 DEFAULT CHARSET=latin1 |
mysql> show create table achiever;
| metaward_achiever | CREATE TABLE `achiever` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`modified` datetime NOT NULL,
`created` datetime NOT NULL,
`award_id` int(11) NOT NULL,
`alias_id` int(11) NOT NULL,
`count` int(11) NOT NULL,
PRIMARY KEY (`id`),
KEY `achiever_award_id` (`award_id`),
KEY `achiever_alias_id` (`alias_id`)
) ENGINE=MyISAM AUTO_INCREMENT=87784996 DEFAULT CHARSET=utf8 |
mysql> show create table award;
| metaward_award | CREATE TABLE `award` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`points` double DEFAULT NULL,
PRIMARY KEY (`id`),
) ENGINE=MyISAM AUTO_INCREMENT=131398 DEFAULT CHARSET=utf8 |

You're missing the GROUP BY clause in:
SET points = (SELECT SUM(award.points) AS points
FROM award
INNER JOIN achiever ON award.id = achiever.award_id
WHERE achiever.alias_id = 2000)
There isn't enough information on the AWARD and ACHIEVER tables, so I recommend testing this before updating the UPDATE statement:
SELECT t.id, -- omit once confirmed data is correct
a.alias_id, -- omit once confirmed data is correct
SUM(t.points) AS points
FROM AWARD t
JOIN ACHIEVER a ON a.award_id = t.id
GROUP BY t.id, a.alias_id
Once you know the summing is correct, update the INSERT statement:
SET points = (SELECT SUM(t.points)
FROM AWARD t
JOIN ACHIEVER a ON a.award_id = t.id
WHERE a.alias_id = 2000 --don't include if you don't need it
GROUP BY t.id, a.alias_id)

Related

Can I improve my movie selecting SQL query

I've created a database to store movies data. My tables are the following:
movies:
CREATE TABLE IF NOT EXISTS `movies` (
`movieId` int(11) NOT NULL AUTO_INCREMENT,
`imdbId` varchar(255) DEFAULT NULL,
`imdbRating` float DEFAULT NULL,
`movieTitle` varchar(255) NOT NULL,
`movieLength` varchar(255) NOT NULL,
`imdbRatingCount` varchar(255) NOT NULL,
`poster` varchar(255) NOT NULL,
`year` varchar(255) NOT NULL,
PRIMARY KEY (`movieId`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=1 ;
I have a table in which i store movie actors:
CREATE TABLE IF NOT EXISTS `actors` (
`actorId` int(10) NOT NULL AUTO_INCREMENT,
`actorName` varchar(255) NOT NULL,
PRIMARY KEY (`actorId`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=1 ;
And one other in which i store the relation between the movies and actors: (movieActor)
CREATE TABLE IF NOT EXISTS `movieActor` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`movieId` int(10) NOT NULL,
`actorId` int(10) NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Now when i want to select a list of movies in which are the selected actors my query is:
SELECT *
FROM movies m inner join
(SELECT movieId FROM movieActor WHERE actorId IN(1,2,3) GROUP BY movieId having count(*) = 3) ma ON m.movieId = ma.movieId
WHERE imdbRating IS NOT NULL ORDER BY imdbRating DESC
This is working perfectly, but i don't know that this is the optimal table structure and query to accomplish this. Are there any better table structure to store data or query the list?
First of all, use indexes on your tables. In my opinion it should be useful to have 3 indexes on movieActor. MovieId - ActorID - MovieIdActorId.
Second try tu use foreign keys. These help to identify the best execution plan for your dbs.
Third try to avoid generating temp tables in your execution plan of your query. Subselects often creates temp tables which are used when the database has to temporarily save something in the RAM. To check this, write EXPLAIN in front of goer query.
I would write it like this:
SELECT m.*, movieActor
FROM movies m inner join
movieActor ma ON m.movieId = ma.movieId
WHERE imdbRating IS NOT NULL
and actorId IN(1,2,3)
GROUP BY movieId
having count(*) = 3)
ORDER BY imdbRating DESC
(Not tested)
Just try to optimize it with the EXPLAIN keyword. It also can help you to create the right indexes.

Optimize tables MySQL

I have a query that is executed in 35s, which is waaaaay too long.
Here are the 3 tables concerned by the query (each table is approx. 13000 lines long, and should be much longer in the future) :
Table 1 : Domains
CREATE TABLE IF NOT EXISTS `domain` (
`id_domain` int(11) NOT NULL AUTO_INCREMENT,
`domain_domain` varchar(255) NOT NULL,
`projet_domain` int(11) NOT NULL,
`date_crea_domain` int(11) NOT NULL,
`date_expi_domain` int(11) NOT NULL,
`active_domain` tinyint(1) NOT NULL,
`remarques_domain` text NOT NULL,
PRIMARY KEY (`id_domain`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
Table 2 : Keywords
CREATE TABLE IF NOT EXISTS `kw` (
`id_kw` int(11) NOT NULL AUTO_INCREMENT,
`kw_kw` varchar(255) NOT NULL,
`clics_kw` int(11) NOT NULL,
`cpc_kw` float(11,3) NOT NULL,
`date_kw` int(11) NOT NULL,
PRIMARY KEY (`id_kw`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
Table 3 : Linking between domain and keyword
CREATE TABLE IF NOT EXISTS `kw_domain` (
`id_kd` int(11) NOT NULL AUTO_INCREMENT,
`kw_kd` int(11) NOT NULL,
`domain_kd` int(11) NOT NULL,
`selected_kd` tinyint(1) NOT NULL,
PRIMARY KEY (`id_kd`),
KEY `kw_to_domain` (`kw_kd`,`domain_kd`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
The query is as follows :
SELECT ng.*, kd.*, kg.*
FROM domain ng
LEFT JOIN kw_domain kd ON kd.domain_kd = ng.id_domain
LEFT JOIN kw kg ON kg.id_kw = kd.kw_kd
GROUP BY ng.id_domain
ORDER BY kd.selected_kd DESC, kd.id_kd DESC
Basically, it selects all domains, with, for each one of these domains, the last associated keyword.
Does anyone have an idea on how to optimize the tables or the query ?
The following will get the last keyword, according to your logic:
select ng.*,
(select kw_kd
from kw_domain kd
where kd.domain_kd = ng.id_domain and kd.selected_kd = 1
order by kd.id_kd desc
limit 1
) as kw_kd
from domain ng;
For performance, you want an index on kw_domain(domain_kd, selected_kd, kw_kd). In this case, the order of the fields matters.
You can use this as a subquery to get more information about the keyword:
select ng.*, kg.*
from (select ng.*,
(select kw_kd
from kw_domain kd
where kd.domain_kd = ng.id_domain and kd.selected_kd = 1
order by kd.id_kd desc
limit 1
) as kw_kd
from domain ng
) ng left join
kw kg
on kg.id_kw = ng.kw_kd;
In MySQL, group by can have poor performance, so this might work better, particularly with the right indexes.

How to INNER JOIN around a loop of tables

I have four tables as follows:
CREATE TABLE IF NOT EXISTS `categories` (
`id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`name` varchar(255) NOT NULL,
PRIMARY KEY (`id`),
) ENGINE=MyISAM DEFAULT CHARSET=utf8 ;
CREATE TABLE IF NOT EXISTS `categories_friends` (
`category_id` int(10) unsigned NOT NULL,
`friend_id` int(10) unsigned NOT NULL,
UNIQUE KEY `category_id` (`friend_id`,`category_id`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8;
CREATE TABLE IF NOT EXISTS `friends` (
`id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`user_id` int(10) unsigned NOT NULL,
`friend_id` int(10) unsigned NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `user_id` (`user_id`,`friend_id`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 ;
CREATE TABLE IF NOT EXISTS `ratings` (
`id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`user_id` int(10) unsigned NOT NULL,
`category_id` int(10) unsigned NOT NULL,
`title` varchar(255) NOT NULL,
`description` text NOT NULL,
`rating` tinyint(2) unsigned NOT NULL,
`public` tinyint(1) NOT NULL DEFAULT '0',
`created` datetime NOT NULL,
PRIMARY KEY (`id`),
) ENGINE=MyISAM DEFAULT CHARSET=utf8 ;
I am trying to perform the following query on those tables:
SELECT *
FROM `favred`.`ratings` AS `Rating`
INNER JOIN `favred`.`friends` AS `JFriend`
ON (`JFriend`.`friend_id` = `Rating`.`user_id`)
INNER JOIN `favred`.`categories_friends` AS `JCategoriesFriend`
ON (`JCategoriesFriend`.`category_id` = `Rating`.`category_id`
AND `JCategoriesFriend`.`friend_id` = `JFriend`.`id`)
INNER JOIN `favred`.`categories` AS `JCategory`
ON (`JCategory`.`id` = `Rating`.`category_id`
AND `JCategory`.`id` = `JCategoriesFriend`.`category_id`)
WHERE `JFriend`.`user_id` = 1
AND `Rating`.`user_id` <> 1
AND `JCategory`.`id` IN (4, 14)
GROUP BY `Rating`.`id`
The query above is not working, as it returns no results (although there is data in the tables that should return), what I'm trying to do is to find all the Ratings that were not authored by me (ID:1), but were authored by my Friends, but only if I've selected to view a specific Category for that Friend, with the resulting set being filtered by a given set of specific Categories.
The INNER JOINs loop around through Rating --> Friend --> CategoriesFreind --> Category --> back to Rating.
If I remove the additional portion of the INNER JOIN's ON clauses as follows:
SELECT *
FROM `favred`.`ratings` AS `Rating`
INNER JOIN `favred`.`friends` AS `JFriend`
ON (`JFriend`.`friend_id` = `Rating`.`user_id`)
INNER JOIN `favred`.`categories_friends` AS `JCategoriesFriend`
ON (`JCategoriesFriend`.`friend_id` = `JFriend`.`id`)
INNER JOIN `favred`.`categories` AS `JCategory`
ON (`JCategory`.`id` = `JCategoriesFriend`.`category_id`)
WHERE `JFriend`.`user_id` = 1
AND `Rating`.`user_id` <> 1
AND `JCategory`.`id` IN (4, 14)
GROUP BY `Rating`.`id`
then the query will return results, but because the INNER JOIN joining the CategoriesFriend to the Rating is not being filtered by the 'JCategory'.'id' IN (4, 14) clause, it returns all Ratings by that friend instead of filtered as it should be.
Any suggestions on how to modify my query to get it to pull the filtered results?
And I'm using CakePHP, so a query that would fit into it's unique query format would be preferred although not required.
first ,why are you use the JFriend.id, does it mean something,or is it as the same as user_id?
try this one,the same logic but it's from top to bottom ,I feel:
SELECT * FROM categories as JCategory
INNER JOIN categories_friends as JCategoriesFriend ON JCategoriesFriend.category_id = JCategory.id
INNER JOIN friends AS JFriend ON JFriend.friend_id = JCategoriesFriend.friend_id
INNER JOIN ratings AS Rating ON Rating.user_id = JFriend.friend_id
WHERE JCategory.id IN (4,14) AND JFriend.user_id = 1 AND Rating.user_id <> 1 GROUP BY Rating.id
I got one result from all the data that I made for the testing.
if it does not work also,try make some correct data,maybe the data is not right...
the testing data below:
categories: id | name (14| 141414)
categories_friends: category_id| friend_id (14| 2)
friends: id | user_id | friend_id (4| 1| 2)
ratings: id | user_id | category_id | title (2| 2| 14 | 'haha')
So I wondered if the INNER JOINs were being a little too limiting and specific in their ON clauses. So I thought that maybe a LEFT JOIN would work better...
SELECT *
FROM `favred`.`ratings` AS `Rating`
INNER JOIN `favred`.`friends` AS `JFriend`
ON (`JFriend`.`friend_id` = `Rating`.`user_id`)
LEFT JOIN `favred`.`categories_friends` AS `JCategoriesFriend`
ON (`JCategoriesFriend`.`friend_id` = `JFriend`.`id`
AND `JCategoriesFriend`.`category_id` = `Rating`.`category_id`)
WHERE `JFriend`.`user_id` = 1
AND `JRatingsUser`.`id` IS NULL
AND `Rating`.`user_id` <> 1
GROUP BY `Rating`.`id`
That query worked for me.
I did away with linking to the categories table directly, and linked indirectly through the categories_friends table which sped up the query a little bit, and everything is working great.

mySql subtract row of different table

I want to subtract between two rows of different table:
I have created a view called leave_taken and table called leave_balance.
I want this result from both table:
leave_taken.COUNT(*) - leave_balance.balance
and group by leave_type_id_leave_type
Code of both table
-----------------View Leave_Taken-----------
CREATE ALGORITHM = UNDEFINED DEFINER=`1`#`localhost` SQL SECURITY DEFINER
VIEW `leave_taken`
AS
select
`leave`.`staff_leave_application_staff_id_staff` AS `staff_leave_application_staff_id_staff`,
`leave`.`leave_type_id_leave_type` AS `leave_type_id_leave_type`,
count(0) AS `COUNT(*)`
from
(
`leave`
join `staff` on((`staff`.`id_staff` = `leave`.`staff_leave_application_staff_id_staff`))
)
where (`leave`.`active` = 1)
group by `leave`.`leave_type_id_leave_type`;
----------------Table leave_balance----------
CREATE TABLE IF NOT EXISTS `leave_balance` (
`id_leave_balance` int(11) NOT NULL AUTO_INCREMENT,
`staff_id_staff` int(11) NOT NULL,
`leave_type_id_leave_type` int(11) NOT NULL,
`balance` int(3) NOT NULL,
`date_added` date NOT NULL,
PRIMARY KEY (`id_leave_balance`),
UNIQUE KEY `id_leave_balance_UNIQUE` (`id_leave_balance`),
KEY `fk_leave_balance_staff1` (`staff_id_staff`),
KEY `fk_leave_balance_leave_type1` (`leave_type_id_leave_type`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 AUTO_INCREMENT=3 ;
------- Table leave ----------
CREATE TABLE IF NOT EXISTS `leave` (
`id_leave` int(11) NOT NULL AUTO_INCREMENT,
`staff_leave_application_id_staff_leave_application` int(11) NOT NULL,
`staff_leave_application_staff_id_staff` int(11) NOT NULL,
`leave_type_id_leave_type` int(11) NOT NULL,
`date` date NOT NULL,
`active` int(11) NOT NULL DEFAULT '1',
`date_updated` date NOT NULL,
PRIMARY KEY (`id_leave`,`staff_leave_application_id_staff_leave_application`,`staff_leave_application_staff_id_staff`),
KEY `fk_table1_leave_type1` (`leave_type_id_leave_type`),
KEY `fk_table1_staff_leave_application1` (`staff_leave_application_id_staff_leave_application`,`staff_leave_application_staff_id_staff`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 AUTO_INCREMENT=32 ;
Well, I still don't think you've provided enough information. It would be very helpful to have some sample data and your expected output (in tabular format). That said, I may have something you can start working with. This query finds all staff members, calculates their current leave (grouped by type), and determines the difference between that and their balance by leave type. Take a look at it, and more importantly (perhaps) the sqlfiddle here that I used which has the sample data in it (very important to determining if this is the correct path for your data).
SELECT
staff.id_staff,
staff.name,
COUNT(`leave`.id_leave) AS leave_count,
leave_balance.balance,
(COUNT(`leave`.id_leave) - leave_balance.balance) AS leave_difference,
`leave`.leave_type_id_leave_type AS leave_type
FROM
staff
JOIN `leave` ON staff.id_staff = `leave`.staff_leave_application_staff_id_staff
JOIN leave_balance ON
(
staff.id_staff = leave_balance.staff_id_staff
AND `leave`.leave_type_id_leave_type = leave_balance.leave_type_id_leave_type
)
WHERE
`leave`.active = 1
GROUP BY
staff.id_staff, leave_type;
Good luck!

MySQL query killing my server

Looking at this query there's got to be something bogging it down that I'm not noticing. I ran it for 7 minutes and it only updated 2 rows.
//set product count for makes
$tru->query->run(array(
'name' => 'get-make-list',
'sql' => 'SELECT id, name FROM vehicle_make',
'connection' => 'core'
));
while($tempMake = $tru->query->getArray('get-make-list')) {
$tru->query->run(array(
'name' => 'update-product-count',
'sql' => 'UPDATE vehicle_make SET product_count = (
SELECT COUNT(product_id) FROM taxonomy_master WHERE v_id IN (
SELECT id FROM vehicle_catalog WHERE make_id = '.$tempMake['id'].'
)
) WHERE id = '.$tempMake['id'],
'connection' => 'core'
));
}
I'm sure this query can be optimized to perform better, but I can't think of how to do it.
vehicle_make = 45 rows
taxonomy_master = 11,223 rows
vehicle_catalog = 5,108 rows
All tables have appropriate indexes
UPDATE: I should note that this is a 1-time script so overhead isn't a big deal as long as it runs.
CREATE TABLE IF NOT EXISTS `vehicle_make` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(32) NOT NULL,
`product_count` int(11) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1 AUTO_INCREMENT=46 ;
CREATE TABLE IF NOT EXISTS `taxonomy_master` (
`product_id` int(10) NOT NULL,
`v_id` int(10) NOT NULL,
`vehicle_requirement` varchar(255) DEFAULT NULL,
`is_sellable` enum('True','False') DEFAULT 'True',
`programming_override` varchar(25) DEFAULT NULL,
PRIMARY KEY (`product_id`,`v_id`),
KEY `idx2` (`product_id`),
KEY `idx3` (`v_id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;
CREATE TABLE IF NOT EXISTS `vehicle_catalog` (
`v_id` int(10) NOT NULL,
`id` int(11) NOT NULL,
`v_make` varchar(255) NOT NULL,
`make_id` int(11) NOT NULL,
`v_model` varchar(255) NOT NULL,
`model_id` int(11) NOT NULL,
`v_year` varchar(255) NOT NULL,
PRIMARY KEY (`v_id`,`v_make`,`v_model`,`v_year`),
UNIQUE KEY `idx` (`v_make`,`v_model`,`v_year`),
UNIQUE KEY `idx2` (`v_id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;
Update: The successful query to get what I needed is here....
SELECT
m.id,COUNT(t.product_id) AS CountOf
FROM taxonomy_master t
INNER JOIN vehicle_catalog v ON t.v_id=v.id
INNER JOIN vehicle_make m ON v.make_id=m.id
GROUP BY m.id;
without the tables/columns this is my best guess from reverse engineering the given queries:
UPDATE m
SET product_count =COUNT(t.product_id)
FROM taxonomy_master t
INNER JOIN vehicle_catalog v ON t.v_id=v.id
INNER JOIN vehicle_make m ON v.make_id=m.id
GROUP BY m.name
The given code loops over each make, and then runs a query the counts for each. My answer just does them all in one query and should be a lot faster.
have an index for each of these:
vehicle_make.id cover on name
vehicle_catalog.id cover make_id
taxonomy_master.v_id
EDIT
give this a try:
CREATE TEMPORARY TABLE CountsOf (
id int(11) NOT NULL
, CountOf int(11) NOT NULL DEFAULT 0.00
);
INSERT INTO CountsOf
(id, CountOf )
SELECT
m.id,COUNT(t.product_id) AS CountOf
FROM taxonomy_master t
INNER JOIN vehicle_catalog v ON t.v_id=v.id
INNER JOIN vehicle_make m ON v.make_id=m.id
GROUP BY m.id;
UPDATE taxonomy_master,CountsOf
SET taxonomy_master.product_count=CountsOf.CountOf
WHERE taxonomy_master.id=CountsOf.id;
instead of using nested query ,
you can separated this query to 2 or 3 queries,
and in php insert the result of the inner query to the out query ,
its faster !
#haim-evgi Separating the queries will not increase the speed significantly, it will just shift the load from the DB server to the Web server and create overhead of moving data between the two servers.
I am not sure with the appropriate indexes you run such query 7 minutes. Could you please show the table structure of the tables involved in these queries.
Seems like you need the following indices:
INDEX BTREE('make_id') on vehicle_catalog
INDEX BTREE('v_id') on taxonomy_master