I want to get all the neighborhoods (based on different zips) which the user is not a member of already.
I have a users table and several other tables like this:
table name: neighborhood
CREATE TABLE neighborhood(
`neighborhood_id` INT(11) NOT NULL AUTO_INCREMENT,
`name` VARCHAR(255) NOT NULL,
`description` TEXT DEFAULT NULL,
`neighborhood_postal_code` VARCHAR(255) NOT NULL,
`region_neighborhood` VARCHAR(255) NOT NULL,
`created_at` DATETIME DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`neighborhood_id`),
INDEX `neighborhood_region_neighborhood_FI_1` (`region_neighborhood`)
) ENGINE = InnoDB;
table name: user_neighborhood
CREATE TABLE user_neighborhood(
`user_id` INT(11) NOT NULL,
`neighborhood_id` INT(11) NOT NULL,
`activity_circle` INT(1) DEFAULT 0,
`duo_circle` INT(1) DEFAULT 0,
FOREIGN KEY (`user_id`) REFERENCES `users` (`user_id`),
FOREIGN KEY (`neighborhood_id`) REFERENCES `neighborhood` (`neighborhood_id`)
) ENGINE = InnoDB;
I have tried the following query, but the result is not correct:
SELECT n.*
FROM `neighborhood` as n
left join user_neighborhood as un on n.neighborhood_id = un.neighborhood_id
where un.user_id != 1 and n.neighborhood_postal_code IN ('2000', '2100')
UPDATE: I managed to make the query seem correct at first instance using a subquery like this:
select *
from neighborhood
where neighborhood_id NOT IN (select neighborhood_id from user_neighborhood where user_id != 1)
AND neighborhood_postal_code IN ('2000', '2100')
However, it also returns (some) of the neighborhoods i am in already. It doesnot make much sense to me why only some..
Why exactly are you adding user_id != 1 in your subquery? I think if you know the id of the user you want to fetch for lets say user_id is 10 then use where user_id = 10 in subquery like:
select *
from neighborhood
where neighborhood_id NOT IN (select distinct neighborhood_id from user_neighborhood where user_id = 10)
AND neighborhood_postal_code IN ('2000', '2100')
But if you want to fetch all the neighbors which have no user then you can use this Query:
select *
from neighborhood
where neighborhood_id NOT IN (select distinct neighborhood_id from user_neighborhood)
AND neighborhood_postal_code IN ('2000', '2100')
Hope this helps!
I am trying to use the following MySQL query:
SELECT *
FROM top_lines t
LEFT OUTER JOIN last_24_topline AS l ON l.`member_no` = t.`member_no`
AND l.`mfg` = t.`line_no`
WHERE l.account_no = 32049 OR l.account_no IS NULL
However this is returning no rows, as there are no account_no rows in last_24_topline that match. From all that I understand and have read this query should still return all the rows from top_lines, even though no rows match in last_24_topline since I am checking for a value or null, but it is not. Are there any options or settings in MySQL (5.7.2) that would cause this behavior?
Just for information, this query works as expected:
SELECT *
FROM top_lines t
LEFT OUTER JOIN last_24_topline l ON l.`member_no` = t.`member_no`
AND l.`mfg` = t.`line_no`
AND l.`account_no` = 32049
I'm unable to use this construct however since I am using entity framework and you can only pass columns in and not values to the joins
CREATE TABLE `last_24_topline` (
`id` bigint(20) NOT NULL AUTO_INCREMENT,
`member_no` varchar(30) NOT NULL,
`branch_no` int(11) DEFAULT NULL,
`employee_no` varchar(25) DEFAULT NULL,
`account_no` varchar(25) DEFAULT NULL,
`salesperson_name` varchar(255) DEFAULT NULL,
`customer_name` varchar(255) DEFAULT NULL,
`mfg` varchar(5) DEFAULT NULL,
`mfg_description` varchar(255) DEFAULT NULL,
`last_three` decimal(10,2) DEFAULT '0.00',
`last_twelve` decimal(10,2) DEFAULT '0.00',
`ly_last_three` decimal(10,2) DEFAULT '0.00',
`ly_last_twelve` decimal(10,2) DEFAULT '0.00',
PRIMARY KEY (`id`),
KEY `ix_branch_no` (`branch_no`),
KEY `ix_employee_no` (`employee_no`),
KEY `ix_member_line_account` (`member_no`,`mfg`,`account_no`),
KEY `ix_member_line` (`member_no`,`mfg`),
KEY `ix_account_no` (`account_no`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8
CREATE TABLE `top_lines` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`member_no` varchar(30) NOT NULL,
`line_no` varchar(5) NOT NULL,
`line_description` varchar(255) NOT NULL,
PRIMARY KEY (`id`),
KEY `ix_line_no` (`member_no`,`line_no`)
) ENGINE=InnoDB AUTO_INCREMENT=41 DEFAULT CHARSET=latin1
insert into `top_lines`(`id`,`member_no`,`line_no`,`line_description`) values (1,'520','772','FED ROTOR/DRUM');
insert into `top_lines`(`id`,`member_no`,`line_no`,`line_description`) values (2,'520','952','FED SST CERAMIC');
insert into `top_lines`(`id`,`member_no`,`line_no`,`line_description`) values (3,'520','954','FED SST FRICTION');
insert into `top_lines`(`id`,`member_no`,`line_no`,`line_description`) values (4,'520','162','EVS FRICTION');
INSERT INTO `last_24_topline` (`id`, `member_no`, `branch_no`, `employee_no`, `account_no`, `salesperson_name`, `customer_name`, `mfg`, `mfg_description`, `last_three`, `last_twelve`, `ly_last_three`, `ly_last_twelve`) VALUES('1','520','0','10856','463854','FORD, JAMES,','JIFFY LUBE','459','FEDERATED AIR FILTER','0.00','15.21','0.00','0.00');
INSERT INTO `last_24_topline` (`id`, `member_no`, `branch_no`, `employee_no`, `account_no`, `salesperson_name`, `customer_name`, `mfg`, `mfg_description`, `last_three`, `last_twelve`, `ly_last_three`, `ly_last_twelve`) VALUES('2','520','0','10856','463854','FORD, JAMES,','JIFFY LUBE','460','FILTERS','0.00','0.00','0.00','16.48');
INSERT INTO `last_24_topline` (`id`, `member_no`, `branch_no`, `employee_no`, `account_no`, `salesperson_name`, `customer_name`, `mfg`, `mfg_description`, `last_three`, `last_twelve`, `ly_last_three`, `ly_last_twelve`) VALUES('3','520','0','10856','463854','FORD, JAMES,','JIFFY LUBE','863','SMP T SERIES','0.00','0.00','0.00','50.67');
I would expect, even with no data in last_24_topline that matches, for the first query to produce a result set containing all the rows in top_lines with null values for the columns from last_24_topline.
Expected results:
So, creating the same schema into another database and inserting only the example data I provided above, I get the results I expect. I am testing further with copying the full rows to the second database to see if it still gives the expected results.
update
Copying all data into the new tables causes the problem to resurface. I'm trying to pare down to the minimum necessary to replicate the issue.
Try detecting for empty string too, maybe the fields are not NULL, but empty strings.
SELECT *
FROM
top_lines t
LEFT JOIN
last_24_topline AS l ON l.member_no = t.member_no AND l.mfg = t.line_no
WHERE
(l.account_no = '' OR l.account_no = '32049' OR l.account_no IS NULL)
If you want more help, i will need a sample data for "table last_24_topline" and the expected output after the join.
As a second try, you can use this one:
SELECT *
FROM
top_lines t
LEFT JOIN
last_24_topline AS l ON l.member_no = t.member_no AND l.mfg = t.line_no
WHERE
l.id IS NULL
OR
(l.id IS NOT NULL AND l.account_no = '32049')
Use a column involved in the join instead of l.account_no this way rows from top_lines will be returned if there is no matching row in the left table.
SELECT *
FROM top_lines t
LEFT OUTER JOIN last_24_topline AS l ON l.`member_no` = t.`member_no`
AND l.`mfg` = t.`line_no`
WHERE l.account_no = 32049 OR l.`member_no` IS NULL
Alternatively place the account number filter directly in to the join
SELECT *
FROM top_lines t
LEFT OUTER JOIN last_24_topline AS l ON l.`member_no` = t.`member_no`
AND l.`mfg` = t.`line_no`
AND l.account_no = 32049
On the query without the account number in the join, the join does match row in the last_24_toplines table, but it does not match the account number in the where clause, so it is filtered out and not seen as a row from top_lines without a matching row from last_24_topline.
So, for example this row from the top_line table
Will match this row from the last_24_toplines
but both will then be filtered out, because the account_no didn't match what was in the where clause: WHERE l.account_no = 32049 OR l.account_no IS NULL
The query with the check for account_no within the join will still match the row from top_line, but will not have a matching row from last_24_topline, so you will get a row with top_lines data with null last_24_topline.
To allow our customers to store some of their own data along with our data structure I have created two extra tables:
CREATE TABLE external_columns
(
`id` INT(11) PRIMARY KEY NOT NULL,
`column` VARCHAR(30) NOT NULL,
`sid` INT(11) NOT NULL,
`bid` INT(11) NOT NULL,
`label` VARCHAR(30) NOT NULL,
`table` VARCHAR(30) NOT NULL,
`default` TINYTEXT NOT NULL
);
CREATE TABLE external_data
(
`id` INT(11) PRIMARY KEY NOT NULL,
`extcol_id` INT(11) NOT NULL,
`sid` INT(11) NOT NULL,
`bid` INT(11) NOT NULL,
`data` MEDIUMTEXT NOT NULL,
`row_id` INT(11) NOT NULL,
CONSTRAINT `external_data_external_columns_id_fk`
FOREIGN KEY (extcol_id) REFERENCES external_columns (id)
);
CREATE UNIQUE INDEX combinedUniqueIndex
ON external_data (extcol_id, sid, bid, row_id);
sid and bid are system values that identify the customer the data belongs to. row_id refers to the primary key of table referenced in table.
To get data for a certain row I have created this prepared statement:
SELECT `data`.*, `columns`.`column`, `columns`.`default`
FROM `external_columns` as `columns`
LEFT JOIN `external_data` as `data`
ON `columns`.`id` = `data`.`extcol_id`
WHERE (
`columns`.`sid` = :sid
AND `columns`.`bid` = :bid
AND `data`.`row_id` = :row_id
AND `columns`.`table` = :tableName
)
This works fine as long as for each external_column there is an entry in external_data for the given :row_id. But I want to make sure that there is always a row for each column, even if there is no data for the given :row_id. Is there a way to do this with one query?
Very close, by placing AND data.row_id = :row_id in your WHERE, you have effectively written an INNER JOIN as nulled data.row_ids won't match.
You should move this condition to the LEFT JOIN conditions:
SELECT `data`.*, `columns`.`column`, `columns`.`default`
FROM `external_columns` as `columns`
LEFT JOIN `external_data` as `data`
ON `data`.`extcol_id`= `columns`.id
AND `data`.`row_id` = :row_id
WHERE `columns`.`sid` = :sid
AND `columns`.`bid` = :bid
AND `columns`.`table` = :tableName
Personal Preferences:
Don't need the WHERE parentheses and I always tend to put the table conditions for a JOIN in the JOIN conditions where applicable and JOIN table on the LHS to make indexing options more obvious..
No difference for INNER JOINs but essential for certain LEFT JOINs.
I've created a database to store movies data. My tables are the following:
movies:
CREATE TABLE IF NOT EXISTS `movies` (
`movieId` int(11) NOT NULL AUTO_INCREMENT,
`imdbId` varchar(255) DEFAULT NULL,
`imdbRating` float DEFAULT NULL,
`movieTitle` varchar(255) NOT NULL,
`movieLength` varchar(255) NOT NULL,
`imdbRatingCount` varchar(255) NOT NULL,
`poster` varchar(255) NOT NULL,
`year` varchar(255) NOT NULL,
PRIMARY KEY (`movieId`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=1 ;
I have a table in which i store movie actors:
CREATE TABLE IF NOT EXISTS `actors` (
`actorId` int(10) NOT NULL AUTO_INCREMENT,
`actorName` varchar(255) NOT NULL,
PRIMARY KEY (`actorId`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=1 ;
And one other in which i store the relation between the movies and actors: (movieActor)
CREATE TABLE IF NOT EXISTS `movieActor` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`movieId` int(10) NOT NULL,
`actorId` int(10) NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Now when i want to select a list of movies in which are the selected actors my query is:
SELECT *
FROM movies m inner join
(SELECT movieId FROM movieActor WHERE actorId IN(1,2,3) GROUP BY movieId having count(*) = 3) ma ON m.movieId = ma.movieId
WHERE imdbRating IS NOT NULL ORDER BY imdbRating DESC
This is working perfectly, but i don't know that this is the optimal table structure and query to accomplish this. Are there any better table structure to store data or query the list?
First of all, use indexes on your tables. In my opinion it should be useful to have 3 indexes on movieActor. MovieId - ActorID - MovieIdActorId.
Second try tu use foreign keys. These help to identify the best execution plan for your dbs.
Third try to avoid generating temp tables in your execution plan of your query. Subselects often creates temp tables which are used when the database has to temporarily save something in the RAM. To check this, write EXPLAIN in front of goer query.
I would write it like this:
SELECT m.*, movieActor
FROM movies m inner join
movieActor ma ON m.movieId = ma.movieId
WHERE imdbRating IS NOT NULL
and actorId IN(1,2,3)
GROUP BY movieId
having count(*) = 3)
ORDER BY imdbRating DESC
(Not tested)
Just try to optimize it with the EXPLAIN keyword. It also can help you to create the right indexes.
Looking at this query there's got to be something bogging it down that I'm not noticing. I ran it for 7 minutes and it only updated 2 rows.
//set product count for makes
$tru->query->run(array(
'name' => 'get-make-list',
'sql' => 'SELECT id, name FROM vehicle_make',
'connection' => 'core'
));
while($tempMake = $tru->query->getArray('get-make-list')) {
$tru->query->run(array(
'name' => 'update-product-count',
'sql' => 'UPDATE vehicle_make SET product_count = (
SELECT COUNT(product_id) FROM taxonomy_master WHERE v_id IN (
SELECT id FROM vehicle_catalog WHERE make_id = '.$tempMake['id'].'
)
) WHERE id = '.$tempMake['id'],
'connection' => 'core'
));
}
I'm sure this query can be optimized to perform better, but I can't think of how to do it.
vehicle_make = 45 rows
taxonomy_master = 11,223 rows
vehicle_catalog = 5,108 rows
All tables have appropriate indexes
UPDATE: I should note that this is a 1-time script so overhead isn't a big deal as long as it runs.
CREATE TABLE IF NOT EXISTS `vehicle_make` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(32) NOT NULL,
`product_count` int(11) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1 AUTO_INCREMENT=46 ;
CREATE TABLE IF NOT EXISTS `taxonomy_master` (
`product_id` int(10) NOT NULL,
`v_id` int(10) NOT NULL,
`vehicle_requirement` varchar(255) DEFAULT NULL,
`is_sellable` enum('True','False') DEFAULT 'True',
`programming_override` varchar(25) DEFAULT NULL,
PRIMARY KEY (`product_id`,`v_id`),
KEY `idx2` (`product_id`),
KEY `idx3` (`v_id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;
CREATE TABLE IF NOT EXISTS `vehicle_catalog` (
`v_id` int(10) NOT NULL,
`id` int(11) NOT NULL,
`v_make` varchar(255) NOT NULL,
`make_id` int(11) NOT NULL,
`v_model` varchar(255) NOT NULL,
`model_id` int(11) NOT NULL,
`v_year` varchar(255) NOT NULL,
PRIMARY KEY (`v_id`,`v_make`,`v_model`,`v_year`),
UNIQUE KEY `idx` (`v_make`,`v_model`,`v_year`),
UNIQUE KEY `idx2` (`v_id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;
Update: The successful query to get what I needed is here....
SELECT
m.id,COUNT(t.product_id) AS CountOf
FROM taxonomy_master t
INNER JOIN vehicle_catalog v ON t.v_id=v.id
INNER JOIN vehicle_make m ON v.make_id=m.id
GROUP BY m.id;
without the tables/columns this is my best guess from reverse engineering the given queries:
UPDATE m
SET product_count =COUNT(t.product_id)
FROM taxonomy_master t
INNER JOIN vehicle_catalog v ON t.v_id=v.id
INNER JOIN vehicle_make m ON v.make_id=m.id
GROUP BY m.name
The given code loops over each make, and then runs a query the counts for each. My answer just does them all in one query and should be a lot faster.
have an index for each of these:
vehicle_make.id cover on name
vehicle_catalog.id cover make_id
taxonomy_master.v_id
EDIT
give this a try:
CREATE TEMPORARY TABLE CountsOf (
id int(11) NOT NULL
, CountOf int(11) NOT NULL DEFAULT 0.00
);
INSERT INTO CountsOf
(id, CountOf )
SELECT
m.id,COUNT(t.product_id) AS CountOf
FROM taxonomy_master t
INNER JOIN vehicle_catalog v ON t.v_id=v.id
INNER JOIN vehicle_make m ON v.make_id=m.id
GROUP BY m.id;
UPDATE taxonomy_master,CountsOf
SET taxonomy_master.product_count=CountsOf.CountOf
WHERE taxonomy_master.id=CountsOf.id;
instead of using nested query ,
you can separated this query to 2 or 3 queries,
and in php insert the result of the inner query to the out query ,
its faster !
#haim-evgi Separating the queries will not increase the speed significantly, it will just shift the load from the DB server to the Web server and create overhead of moving data between the two servers.
I am not sure with the appropriate indexes you run such query 7 minutes. Could you please show the table structure of the tables involved in these queries.
Seems like you need the following indices:
INDEX BTREE('make_id') on vehicle_catalog
INDEX BTREE('v_id') on taxonomy_master