JOIN LEFT with multiple conditions - mysql

I'm having following tables structure
CREATE TABLE IF NOT EXISTS `review_author` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(100) NOT NULL,
`email` varchar(255) NOT NULL,
`client_id` int(11) DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `IDX_37D99F0819EB6921` (`client_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=2110 ;
AND
CREATE TABLE IF NOT EXISTS `brokers_comments` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`hb_broker_id` int(11) NOT NULL,
`client_id` int(11) DEFAULT NULL,
`user_name` varchar(100) NOT NULL,
`user_email` varchar(100) NOT NULL,
`state` int(11) NOT NULL DEFAULT '0',
`text` varchar(3000) NOT NULL,
PRIMARY KEY (`id`),
KEY `IDX_5365DFFB9FE55EF7` (`hb_broker_id`),
KEY `IDX_5365DFFB19EB6921` (`client_id`),
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=1583 ;
Before extracting value i did following query:
INSERT INTO review_author (
name,
email,
client_id
)
SELECT
brokers_comments.user_name,
brokers_comments.user_email,
brokers_comments.client_id
FROM brokers_comments
LEFT JOIN review_author
ON brokers_comments.user_name=review_author.name AND
brokers_comments.user_email=review_author.email AND
brokers_comments.client_id=review_author.client_id
WHERE review_author.id IS NULL
Not in review_author should be all author from table brokers_comments and now i'm trying to get authors id using following query:
SELECT
review_author.id
FROM brokers_comments
LEFT JOIN review_author
ON brokers_comments.user_name=review_author.name AND
brokers_comments.user_email=review_author.email AND
brokers_comments.client_id=review_author.client_id
WHERE review_author.id IS NOT NULL
but i'm getting about 110 results from total 1531 records from table brokers_comments.
UPDATE
I couldn't manage to insert data in http://sqlfiddle.com/ so following link are dump for two tables review_author and brokers_comments.
Again my issue is to transfer distinct columns(user_name, user_email, client_id) from table brokers_comments to table review_author and then select review_author.id based on relation name/email/client_id from both tables.
http://wrttn.in/7ca325
http://wrttn.in/3a7885
Insert new author was wrong and made duplication. Below is new correct form.
INSERT INTO review_author (
name,
email,
client_id
)
SELECT user_name, user_email, client_id
FROM brokers_comments AS broker
WHERE NOT EXISTS
(
SELECT 1
FROM review_author AS author
WHERE author.email = broker.user_email
)
GROUP BY broker.user_email
P.S. I somebody will make a working online mysql database please put in comments so i could put it there.
Resolved
Only now i realised that user_email must be unique. Based on this i made following select statement:
SELECT
author.id
FROM brokers_comments AS broker
LEFT JOIN review_author AS author
ON broker.user_email = author.email

It seems you use excess fields in JOIN clause since client_id is a key, you need to join tables only on this field. Possible cause of that you getting not same number of records is different name/email for same client_id in those two tables. So, your two queries should be like this:
INSERT INTO review_author (
name,
email,
client_id
)
SELECT
brokers_comments.user_name,
brokers_comments.user_email,
brokers_comments.client_id
FROM brokers_comments
LEFT JOIN review_author
ON brokers_comments.client_id=review_author.client_id
WHERE review_author.id IS NULL
and
SELECT
review_author.id
FROM brokers_comments
LEFT JOIN review_author
ON brokers_comments.client_id=review_author.client_id
WHERE review_author.id IS NOT NULL

Related

how to count items inside group_concat method with mysql query

I have one to many table relationship :
one user for multiple event
one event for multiple event_attribute
Now, I group by userId and want to know how many for each event attribute ?
I am using group_concat like this:
group_concat(
concat(event_event_attribute.event_attr_id,
count( distinct event_event_attribute.value)
) group by event_attr_id)
)
group by userId
So here, I first group by userId, then group concat event-attribute, at least I hope to have :
(attr1, 10),(attr2, 30)....
all in one row.
But this does not work at all
Any suggestions?
To be more specific, this is the DB schema I am using:
CREATE TABLE `user` (
`id` int(11) NOT NULL,
`name` varchar(45) DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `id_UNIQUE` (`id`)
);
CREATE TABLE `event` (
`id` int(11) NOT NULL,
`name` varchar(45) DEFAULT NULL,
`user_id` int(11) DEFAULT NULL,
PRIMARY KEY (`id`)
);
CREATE TABLE `event_attr` (
`id` int(11) NOT NULL,
`att_name` varchar(45) DEFAULT NULL,
`event_id` varchar(45) DEFAULT NULL,
PRIMARY KEY (`id`)
);
INSERT INTO `user` VALUES (1,'user1'),(2,'user2'),(3,'user3');
INSERT INTO `event` VALUES (1,'event1',1),(2,'event2',1),(3,'event3',1),(4,'event4',2),(5,'event5',2),(6,'event6',3);
INSERT INTO `event_attr` VALUES (1,'att1','1'),(2,'att2','1'),(3,'att3','1'),(4,'att1','2'),(5,'att2',NULL);
Now if I am running:
select u.id, group_concat(e.name)
from user u
join event e on u.id=e.user_id
group by u.id
I will get:
1 event1,event2,event3
2 event4,event6
3 event 6
That is fine. But one step forward, I need to know count for each event_attt for each user, such as:
1 evet_att1:3;event_att2:2
2 event_att3:1
Then it is not possible. Can I use just one query to get above expected response?
should be the inverse alias concat the aggreagted values and not aggregated the concat
select concat (group_concat(event_event_attribute.event_attr_id )
,' - ',
count( distinct event_event_attribute.value) )
from event_event_attribute
group by userid
Otherwise could be you need an subquery for obtain the count group by event_attr_id
select group_concat(
concat(event_attr_id), ',', count_value)
)
from t (
select user_id, event_event_attribute.event_attr_id, count( distinct event_event_attribute.value) count_value
from event_event_attribute
group by event_attr_id
) t
group by user_id

How to INNER JOIN around a loop of tables

I have four tables as follows:
CREATE TABLE IF NOT EXISTS `categories` (
`id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`name` varchar(255) NOT NULL,
PRIMARY KEY (`id`),
) ENGINE=MyISAM DEFAULT CHARSET=utf8 ;
CREATE TABLE IF NOT EXISTS `categories_friends` (
`category_id` int(10) unsigned NOT NULL,
`friend_id` int(10) unsigned NOT NULL,
UNIQUE KEY `category_id` (`friend_id`,`category_id`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8;
CREATE TABLE IF NOT EXISTS `friends` (
`id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`user_id` int(10) unsigned NOT NULL,
`friend_id` int(10) unsigned NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `user_id` (`user_id`,`friend_id`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 ;
CREATE TABLE IF NOT EXISTS `ratings` (
`id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`user_id` int(10) unsigned NOT NULL,
`category_id` int(10) unsigned NOT NULL,
`title` varchar(255) NOT NULL,
`description` text NOT NULL,
`rating` tinyint(2) unsigned NOT NULL,
`public` tinyint(1) NOT NULL DEFAULT '0',
`created` datetime NOT NULL,
PRIMARY KEY (`id`),
) ENGINE=MyISAM DEFAULT CHARSET=utf8 ;
I am trying to perform the following query on those tables:
SELECT *
FROM `favred`.`ratings` AS `Rating`
INNER JOIN `favred`.`friends` AS `JFriend`
ON (`JFriend`.`friend_id` = `Rating`.`user_id`)
INNER JOIN `favred`.`categories_friends` AS `JCategoriesFriend`
ON (`JCategoriesFriend`.`category_id` = `Rating`.`category_id`
AND `JCategoriesFriend`.`friend_id` = `JFriend`.`id`)
INNER JOIN `favred`.`categories` AS `JCategory`
ON (`JCategory`.`id` = `Rating`.`category_id`
AND `JCategory`.`id` = `JCategoriesFriend`.`category_id`)
WHERE `JFriend`.`user_id` = 1
AND `Rating`.`user_id` <> 1
AND `JCategory`.`id` IN (4, 14)
GROUP BY `Rating`.`id`
The query above is not working, as it returns no results (although there is data in the tables that should return), what I'm trying to do is to find all the Ratings that were not authored by me (ID:1), but were authored by my Friends, but only if I've selected to view a specific Category for that Friend, with the resulting set being filtered by a given set of specific Categories.
The INNER JOINs loop around through Rating --> Friend --> CategoriesFreind --> Category --> back to Rating.
If I remove the additional portion of the INNER JOIN's ON clauses as follows:
SELECT *
FROM `favred`.`ratings` AS `Rating`
INNER JOIN `favred`.`friends` AS `JFriend`
ON (`JFriend`.`friend_id` = `Rating`.`user_id`)
INNER JOIN `favred`.`categories_friends` AS `JCategoriesFriend`
ON (`JCategoriesFriend`.`friend_id` = `JFriend`.`id`)
INNER JOIN `favred`.`categories` AS `JCategory`
ON (`JCategory`.`id` = `JCategoriesFriend`.`category_id`)
WHERE `JFriend`.`user_id` = 1
AND `Rating`.`user_id` <> 1
AND `JCategory`.`id` IN (4, 14)
GROUP BY `Rating`.`id`
then the query will return results, but because the INNER JOIN joining the CategoriesFriend to the Rating is not being filtered by the 'JCategory'.'id' IN (4, 14) clause, it returns all Ratings by that friend instead of filtered as it should be.
Any suggestions on how to modify my query to get it to pull the filtered results?
And I'm using CakePHP, so a query that would fit into it's unique query format would be preferred although not required.
first ,why are you use the JFriend.id, does it mean something,or is it as the same as user_id?
try this one,the same logic but it's from top to bottom ,I feel:
SELECT * FROM categories as JCategory
INNER JOIN categories_friends as JCategoriesFriend ON JCategoriesFriend.category_id = JCategory.id
INNER JOIN friends AS JFriend ON JFriend.friend_id = JCategoriesFriend.friend_id
INNER JOIN ratings AS Rating ON Rating.user_id = JFriend.friend_id
WHERE JCategory.id IN (4,14) AND JFriend.user_id = 1 AND Rating.user_id <> 1 GROUP BY Rating.id
I got one result from all the data that I made for the testing.
if it does not work also,try make some correct data,maybe the data is not right...
the testing data below:
categories: id | name (14| 141414)
categories_friends: category_id| friend_id (14| 2)
friends: id | user_id | friend_id (4| 1| 2)
ratings: id | user_id | category_id | title (2| 2| 14 | 'haha')
So I wondered if the INNER JOINs were being a little too limiting and specific in their ON clauses. So I thought that maybe a LEFT JOIN would work better...
SELECT *
FROM `favred`.`ratings` AS `Rating`
INNER JOIN `favred`.`friends` AS `JFriend`
ON (`JFriend`.`friend_id` = `Rating`.`user_id`)
LEFT JOIN `favred`.`categories_friends` AS `JCategoriesFriend`
ON (`JCategoriesFriend`.`friend_id` = `JFriend`.`id`
AND `JCategoriesFriend`.`category_id` = `Rating`.`category_id`)
WHERE `JFriend`.`user_id` = 1
AND `JRatingsUser`.`id` IS NULL
AND `Rating`.`user_id` <> 1
GROUP BY `Rating`.`id`
That query worked for me.
I did away with linking to the categories table directly, and linked indirectly through the categories_friends table which sped up the query a little bit, and everything is working great.

MySQL View: Join tables without causing the data to duplicate on every row?

I am basically having the exact same problem as here:
SQL View: Join tables without causing the data to duplicate on every row?
Except on that question he was using SQL, and I am using mysql. I am wondering if the same query is possible in mysql. If so, I may have the wrong syntax?
I am trying to do something like
select a.name as account_Name,
p.description as property_DESCRIPTION,
p.address as property_ADDRESS,
null as vehicles_DESCRIPTION,
null as vehicles_MAKE,
null as vehicles_MODEL
from Accounts a
inner join Properties p
on a.accountid = p.accountid
UNION ALL
select a.name as account_Name,
null as property_DESCRIPTION,
null as property_ADDRESS,
v.description as vehicles_DESCRIPTION,
v.make as vehicles_MAKE,
v.model as vehicles_MODEL
from Accounts a
inner join vehicles v
on a.accountid = v.accountid
Here is my actual code:
SELECT user.first_name, user.last_name, upi.image_id, NULL AS friends.friend_user_id FROM user
INNER JOIN user_profile_images as upi ON (user.user_id = upi.user_id)
UNION
SELECT user.first_name, user.last_name, NULL AS upi.image_id, friends.friend_user_id FROM user
INNER JOIN friends ON (user.user_id = friends.user_id)
WHERE user.user_id = '$profile_id'
where I have 3 tables: user, user_profile_images, and friends. Both user_profile_images and friends are related to the user through the user_id. So a user can have multiple profile images as well as multiple friend entries. I can post the table diagrams if it doesnt make sense. But what I want is basically a view of all the info, with fields NULL if they don't apply to the overall view.
If I do the query with 2 tables, either with user and user_profile_images, or user and friends, I get the desired results, but adding the third table gives me duplicate rows.
The solution, as #MarcB suggests, is to use UNION rather than UNION ALL.
However, I have a question for you - why use the UNION at all? The following is equivalent, except that if (say) account 1 has one property and one vehicle, instead of getting:
account_Name property_DESCRIPTION vehicles_MAKE
account1 property1 NULL
account1 NULL vehicle1
You'll get
account_Name property_DESCRIPTION vehicles_MAKE
account1 property1 vehicle1
Query:
SELECT a.name as account_Name,
p.description as property_DESCRIPTION,
p.address as property_ADDRESS,
v.description as vehicles_DESCRIPTION,
v.make as vehicles_MAKE,
v.model as vehicles_MODEL
FROM Accounts a
LEFT JOIN Properties p
on a.accountid = p.accountid
LEFT JOIN vehicles v
on a.accountid = v.accountid
WHERE p.description IS NOT NULL AND v.make IS NOT NULL
Note - the last line (IS NOT NULL for both p and v) simulates the 'accounts table' part of the INNER JOIN and makes sure that only accounts with at least a property OR a vehicle are shown. Substitute the id columns of p and v there.
If I were going after data like that where two tables are related by an id on the third I would consider using outer joins. In the comments of the question you referenced, outer joins were mentioned as a possible solution. The code for that would look like this.
select a.name as account_Name,
p.description as property_DESCRIPTION,
p.address as property_ADDRESS,
v.description as vehicles_DESCRIPTION,
v.make as vehicles_MAKE,
v.model as vehicles_MODEL
from accounts a
left outer join properties p on p.accountid = a.accountid
left outer join vehicles v on v.accountid = a.accountid;
Here is how the solution was tested. First I created the three tables.
CREATE TABLE `accounts` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(50) COLLATE utf8_bin NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_bin;
CREATE TABLE `properties` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`accountid` int(11) NOT NULL,
`description` varchar(255) COLLATE utf8_bin NOT NULL,
`address` varchar(255) COLLATE utf8_bin NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_bin;
CREATE TABLE `vehicles` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`accountid` int(11) NOT NULL,
`description` varchar(255) COLLATE utf8_bin NOT NULL,
`make` varchar(100) COLLATE utf8_bin NOT NULL,
`model` varchar(100) COLLATE utf8_bin NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_bin;
Next I inserted data into each table.
INSERT INTO `demo`.`accounts` (
`accountid` ,
`name`
)
VALUES (
NULL , 'techport80.com'
);
INSERT INTO `demo`.`properties` (
`id` ,
`accountid` ,
`description` ,
`address`
)
VALUES (
NULL , '1', 'office', '123 may street');
INSERT INTO `demo`.`vehicles` (
`id` ,
`accountid` ,
`description` ,
`make` ,
`model`
)
VALUES (
NULL , '1', 'motorcycle', 'honda', 'shadow'
);
At this point if I test the solution, I will receive one row with account, property and vehicle information.
INSERT INTO `demo`.`vehicles` (
`id` ,
`accountid` ,
`description` ,
`make` ,
`model`
)
VALUES (
NULL , '1', 'passenger car', 'Ford', 'Mustang'
);
Now if I test my solution, I see 2 row, one for each vehicle. But the two rows are not exactly duplicate. Though there are some column with the same data. Most notably, the account information.
INSERT INTO `demo`.`properties` (
`id` ,
`accountid` ,
`description` ,
`address`
)
VALUES (
NULL , '1', 'home', '321 yam street'
);
Next I added a second address. Now four rows are returned. One for each vehicle and one for each address. But still, none of the rows are duplicate.
Finally I added another vehicle. This was to cause an off balance of vehicle vs properties.
INSERT INTO `demo`.`vehicles` (
`id` ,
`accountid` ,
`description` ,
`make` ,
`model`
)
VALUES (
NULL , '1', 'Van', 'Chev', 'Cargo'
);
Now we have 6 rows. One per each vehicle and each vehicle is also related to two properties. (So 2 x 3)
Having gone thru this exercise, I wonder if a hierarchical view of the data would be a better model for this data. Perhaps XML or JSON could be used represent the data. For this task, you could use a stored function, but I personally would first consider a programming language like PHP, C#, C++, or a slew of others.
HTH

Some help needed with a SQL query

I need some help with a MySQL query. I have two tables, one with offers and one with statuses. An offer can has one or more statuses. What I would like to do is get all the offers and their latest status. For each status there's a table field named 'added' which can be used for sorting.
I know this can be easily done with two queries, but I need to make it with only one because I also have to apply some filters later in the project.
Here's my setup:
CREATE TABLE `test`.`offers` (
`id` INT UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY ,
`client` TEXT NOT NULL ,
`products` TEXT NOT NULL ,
`contact` TEXT NOT NULL
) ENGINE = MYISAM ;
CREATE TABLE `statuses` (
`id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`offer_id` int(11) NOT NULL,
`options` text NOT NULL,
`deadline` date NOT NULL,
`added` datetime NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1
Should work but not very optimal imho :
SELECT *
FROM offers
INNER JOIN statuses ON (statuses.offer_id = offers.id
AND statuses.id =
(SELECT allStatuses.id
FROM statuses allStatuses
WHERE allStatuses.offer_id = offers.id
ORDER BY allStatuses.added DESC LIMIT 1))
Try this:
SELECT
o.*
FROM offers o
INNER JOIN statuses s ON o.id = s.offer_id
ORDER BY s.added
LIMIT 1

MySQL query killing my server

Looking at this query there's got to be something bogging it down that I'm not noticing. I ran it for 7 minutes and it only updated 2 rows.
//set product count for makes
$tru->query->run(array(
'name' => 'get-make-list',
'sql' => 'SELECT id, name FROM vehicle_make',
'connection' => 'core'
));
while($tempMake = $tru->query->getArray('get-make-list')) {
$tru->query->run(array(
'name' => 'update-product-count',
'sql' => 'UPDATE vehicle_make SET product_count = (
SELECT COUNT(product_id) FROM taxonomy_master WHERE v_id IN (
SELECT id FROM vehicle_catalog WHERE make_id = '.$tempMake['id'].'
)
) WHERE id = '.$tempMake['id'],
'connection' => 'core'
));
}
I'm sure this query can be optimized to perform better, but I can't think of how to do it.
vehicle_make = 45 rows
taxonomy_master = 11,223 rows
vehicle_catalog = 5,108 rows
All tables have appropriate indexes
UPDATE: I should note that this is a 1-time script so overhead isn't a big deal as long as it runs.
CREATE TABLE IF NOT EXISTS `vehicle_make` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(32) NOT NULL,
`product_count` int(11) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1 AUTO_INCREMENT=46 ;
CREATE TABLE IF NOT EXISTS `taxonomy_master` (
`product_id` int(10) NOT NULL,
`v_id` int(10) NOT NULL,
`vehicle_requirement` varchar(255) DEFAULT NULL,
`is_sellable` enum('True','False') DEFAULT 'True',
`programming_override` varchar(25) DEFAULT NULL,
PRIMARY KEY (`product_id`,`v_id`),
KEY `idx2` (`product_id`),
KEY `idx3` (`v_id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;
CREATE TABLE IF NOT EXISTS `vehicle_catalog` (
`v_id` int(10) NOT NULL,
`id` int(11) NOT NULL,
`v_make` varchar(255) NOT NULL,
`make_id` int(11) NOT NULL,
`v_model` varchar(255) NOT NULL,
`model_id` int(11) NOT NULL,
`v_year` varchar(255) NOT NULL,
PRIMARY KEY (`v_id`,`v_make`,`v_model`,`v_year`),
UNIQUE KEY `idx` (`v_make`,`v_model`,`v_year`),
UNIQUE KEY `idx2` (`v_id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;
Update: The successful query to get what I needed is here....
SELECT
m.id,COUNT(t.product_id) AS CountOf
FROM taxonomy_master t
INNER JOIN vehicle_catalog v ON t.v_id=v.id
INNER JOIN vehicle_make m ON v.make_id=m.id
GROUP BY m.id;
without the tables/columns this is my best guess from reverse engineering the given queries:
UPDATE m
SET product_count =COUNT(t.product_id)
FROM taxonomy_master t
INNER JOIN vehicle_catalog v ON t.v_id=v.id
INNER JOIN vehicle_make m ON v.make_id=m.id
GROUP BY m.name
The given code loops over each make, and then runs a query the counts for each. My answer just does them all in one query and should be a lot faster.
have an index for each of these:
vehicle_make.id cover on name
vehicle_catalog.id cover make_id
taxonomy_master.v_id
EDIT
give this a try:
CREATE TEMPORARY TABLE CountsOf (
id int(11) NOT NULL
, CountOf int(11) NOT NULL DEFAULT 0.00
);
INSERT INTO CountsOf
(id, CountOf )
SELECT
m.id,COUNT(t.product_id) AS CountOf
FROM taxonomy_master t
INNER JOIN vehicle_catalog v ON t.v_id=v.id
INNER JOIN vehicle_make m ON v.make_id=m.id
GROUP BY m.id;
UPDATE taxonomy_master,CountsOf
SET taxonomy_master.product_count=CountsOf.CountOf
WHERE taxonomy_master.id=CountsOf.id;
instead of using nested query ,
you can separated this query to 2 or 3 queries,
and in php insert the result of the inner query to the out query ,
its faster !
#haim-evgi Separating the queries will not increase the speed significantly, it will just shift the load from the DB server to the Web server and create overhead of moving data between the two servers.
I am not sure with the appropriate indexes you run such query 7 minutes. Could you please show the table structure of the tables involved in these queries.
Seems like you need the following indices:
INDEX BTREE('make_id') on vehicle_catalog
INDEX BTREE('v_id') on taxonomy_master