I have two tables, one is for kits that can contain various sub-products, and another is a list of sub-products that can't be added together in the same kit.
In simplified form:
CREATE TABLE `kits` (
`subProdID` INT(11) NOT NULL DEFAULT '0',
`kitID` VARCHAR(50) NULL DEFAULT NULL,
`kitName` VARCHAR(512) NULL DEFAULT NULL,
PRIMARY KEY (`subProdID`)
);
CREATE TABLE `subProd_incompatible` (
`IncompID` INT(11) NOT NULL DEFAULT '0',
`subProdID` INT(11) NULL DEFAULT NULL,
`subProdIncompID` INT(11) NULL DEFAULT NULL,
PRIMARY KEY (`VersaIncompID`)
)
In subProd_incompatible, subProdID and subProdIncompID all exist as subProdIDs in kits.
Right now I use procedural code outside the database to produce a list of kits that all contain products that do not conflict according to the subProd_incompatible table, but I'd like to do it in SQL if at all possible.
What I'd like is to
SELECT `kitID`, `kitName`, GROUP_CONCAT(`subProdID`)
FROM ("a subquery") AS Q
GROUP BY `GroupingCriterion`
"a subquery" should return the columns of kits, along with a generated GroupingCriterion.
Here is an idea. For each kit, get the list of the matches to the incompatible table for each subproduct. Then, aggregate by the kit and incompatible id and see if any have two products. If so, you have incompatible products in the kit:
select k.kitid, i.InCompId
from kits k join
subProd_incompatible i
on k.subProdId in (i.subProdId, i.subProdIncompID)
group by k.kitid, i.InCompId
having count(distinct k.subProdId) = 2;
Related
I have 3 tables. The first one is called map_life, the second one is called scripts and the third one is called npc_data.
I'm running the following query to get all the properties from map_life while also getting the script column from scripts and the storage_cost column from npc_data if the ids match.
SELECT life.*
, script.script
, npc.storage_cost
FROM map_life life
LEFT
JOIN scripts script
ON script.objectid = life.lifeid
AND script.script_type = 'npc'
LEFT
JOIN npc_data npc
ON npc.npcid = life.lifeid
As you can see, map_life id is lifeid, while scripts id is objectid and npc_data id is npcid.
This query is taking about 5 seconds to execute, and I have no idea why. Here's the CREATE statements for all those 3 tables, maybe I'm missing something?
CREATE TABLE `mcdb83`.`map_life` (
`id` bigint(21) unsigned NOT NULL AUTO_INCREMENT,
`mapid` int(11) NOT NULL,
`life_type` enum('npc','mob','reactor') NOT NULL,
`lifeid` int(11) NOT NULL,
`life_name` varchar(50) DEFAULT NULL COMMENT 'For reactors, specifies a handle so scripts may interact with them; for NPC/mob, this field is useless',
`x_pos` smallint(6) NOT NULL DEFAULT '0',
`y_pos` smallint(6) NOT NULL DEFAULT '0',
`foothold` smallint(6) NOT NULL DEFAULT '0',
`min_click_pos` smallint(6) NOT NULL DEFAULT '0',
`max_click_pos` smallint(6) NOT NULL DEFAULT '0',
`respawn_time` int(11) NOT NULL DEFAULT '0',
`flags` set('faces_left') NOT NULL DEFAULT '',
PRIMARY KEY (`id`),
KEY `lifetype` (`mapid`,`life_type`)
) ENGINE=InnoDB AUTO_INCREMENT=32122 DEFAULT CHARSET=latin1;
CREATE TABLE `mcdb83`.`scripts` (
`script_type` enum('npc','reactor','quest','item','map_enter','map_first_enter') NOT NULL,
`helper` tinyint(3) NOT NULL DEFAULT '-1' COMMENT 'Represents the quest state for quests, and the index of the script for NPCs (NPCs may have multiple scripts).',
`objectid` int(11) NOT NULL DEFAULT '0',
`script` varchar(30) NOT NULL DEFAULT '',
PRIMARY KEY (`script_type`,`helper`,`objectid`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 COMMENT='Lists all the scripts that belong to NPCs/reactors/etc. ';
CREATE TABLE `mcdb83`.`npc_data` (
`npcid` int(11) NOT NULL,
`storage_cost` int(11) NOT NULL DEFAULT '0',
`flags` set('maple_tv','is_guild_rank') NOT NULL DEFAULT '',
PRIMARY KEY (`npcid`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
For this query:
SELECT l.*, s.script, npc.storage_cost
FROM map_life l LEFT JOIN
scripts s
ON s.objectid = l.lifeid AND
s.script_type = 'npc' LEFT JOIN
npc_data npc
ON npc.npcid = l.lifeid;
You want indexes on: scripts(object_id, script_type, script) and npc_data(npcid, storage_cost). The order of the columns in these indexes is important.
map_life.lifeid does not have any indexes defined, therefore the joins will result in full table scans. Define an index on map_life.lifeid field.
In scripts table the primary key is defined on the following fields in that order: script_type, helper, objectid. The join is done on objectid and there is a constant filter criterion on script_type. Because the order of the fields in the index is wrong, MySQL cannot use the primary key for this query. For this query the order of the fields in the index should b: objectid, script_type, helper.
The above will significantly speed up the joins. You may further increase the speed of the query if your indexes actually cover all fields that are in the query because in this case MySQL does not even have to touch the tables.
Consider adding an index with the following fields and order to the scripts table: object_id, script_type, script and npcid, storage_cost index to npc_data table. However, these indexes may slow down insert / update / delete statements, so do some performance testing before adding these indexes to production environment.
I read many posts on forum, but still I have confusion on creating index to speed up join queries in mysql, here is my doubt
I have two tables, one is category table which just contains few thousand lines, and contains all information about data, and another one is geo_data table which contains huge amount of data, I join geo_data table based on 2 keys s_key1 and s_key2. Following is structure of table
category table
CREATE TABLE `category` (
`Id` int(11) NOT NULL AUTO_INCREMENT,
`s_key1` int(11) DEFAULT NULL,
`s_key2` int(11) DEFAULT NULL,
`STD_DATE` datetime DEFAULT NULL,
`LATITUDE` float DEFAULT NULL,
`LONGITUDE` float DEFAULT NULL,
`COUNTRY_CD` varchar(15) DEFAULT NULL,
`INSTR_CODE` varchar(15) DEFAULT NULL,
`CANADACR_CD` varchar(15) DEFAULT NULL,
`PROBST_T` varchar(15) DEFAULT NULL,
`TYPE` varchar(15) DEFAULT NULL,
PRIMARY KEY (`Id`)
) ENGINE=MyISAM AUTO_INCREMENT=32350 DEFAULT CHARSET=latin1;
geo_data table
CREATE TABLE `geo_data` (
`s_key1` int(11) DEFAULT NULL,
`s_key2` int(11) DEFAULT NULL,
`MAGNETIC` float DEFAULT NULL,
`GRAVITY` float DEFAULT NULL,
`BATHY` float DEFAULT NULL,
`CORE` float DEFAULT NULL
) ENGINE=MyISAM DEFAULT CHARSET=latin1;
I have many tables like geo_data table which contains s_key1, s_key2 and other columns, in my application I often use fields std_date,latitude,longitude,country_cd,type from category table
I do inner join, sometimes left join depending on the requirement, for example my query looks like below
SELECT
c.s_key1,
c.s_key2,
c.std_date,
c.latitude,
c.longitude,
g.magnetic,
g.bathy
FROM
category c, geo_data g
WHERE
c.s_key1 = g.s_key1 && c.s_key2 = g.s_key2;
and sometimes my where clause will have something like this too
WHERE
c.latitude between -30 to 30 AND
c.longitude between 10 to 140 AND
c.country_cd = 'INDIA' AND
c.type = 'NON_PROFIT';
So what's the right way of creating index to speed up my query, whether below one right ? please someone help
create index `myindex` on
`category` (s_key1,s_key2,std_date,latitude,longitude,country_cd)
create index `myindex` on
`geo_data` (s_key1,s_key2)
and One more doubt whether both tables (category,geo_data) should have index key to speed up performance or only geo_data table ?
From the where condition it makes sense to simplify the first index as:
create index `myindex` on
`category` (s_key1,s_key2)
The original however can improve the performance in terms that it doesn't have to access the full table row to get the other values. However it makes the index bigger and therefore slower. So it depends on whether this is optimization for only this query or there are more of them which use only the s_key1 and s_key2 (or with combination with other columns).
Regarding the clarification - for lat/lng check it will make sense to move std_date after lat/lng (or remove completely):
create index `myindex` on
`category` (s_key1,s_key2,latitude,longitude,std_date,country_cd)
I have researched tirelessly to try and understand mysql joins with multiple tables and how reference tables come into play but to no avail. Also this is my first post so be gentle if I missed any rules.
Here are my tables:
customers | CREATE TABLE `customers` (
`customer_num` int(11) NOT NULL AUTO_INCREMENT,
`customer_name` varchar(50) DEFAULT NULL,
`city` varchar(50) DEFAULT NULL,
`state` varchar(50) DEFAULT NULL,
`country` varchar(50) DEFAULT NULL,
PRIMARY KEY (`customer_num`)
orders | CREATE TABLE `orders` (
`order_num` int(11) NOT NULL AUTO_INCREMENT,
`order_date` date DEFAULT NULL,
`shipped_date` date DEFAULT NULL,
`status` varchar(15) DEFAULT NULL,
`comments` text,
`customer_num` int(11) DEFAULT NULL,
PRIMARY KEY (`order_num`)
p_o | CREATE TABLE `p_o` (
`product_num` int(10) unsigned NOT NULL,
`order_num` int(10) unsigned NOT NULL,
KEY `order_num` (`order_num`)
products | CREATE TABLE `products` (
`product_num` int(11) NOT NULL AUTO_INCREMENT,
`year` int(4) unsigned NOT NULL DEFAULT '2014',
`make` varchar(20) NOT NULL,
`model` varchar(20) NOT NULL,
`price` int(6) NOT NULL DEFAULT '0',
PRIMARY KEY (`product_num`)
Customers has a one-many relationship with orders, while orders references to p_o which has both product_num and order_num, and p_o also connects to products.
My goal is to make a query (that is somewhat optimal, not all selects and wheres) that will show all of this information in one table. I created this but cannot get them to show more than just two tables information.
select * from customers t1 join orders t2 on t1.customer_num=t2.customer_num;
That join shows all of the information from orders and customers, and I intend on it being inner join. I've tried all sorts of ways to get them all to join into one but none of them work, and I do believe it's on my end. Also I'm using a module that makes dynamic mysql tables and doesn't support union, but even if you have a solution that uses union I will take it.
Any help you can offer is greatly appreciated, I've been working on this for far too many hours.
I think you should get the information needed using the following:
SELECT
`orders`.`order_num`,
`orders`.`order_date`,
`orders`.`shipped_date`,
`orders`.`status`,
`orders`.`comments`,
`orders`.`customer_num`,
`customers`.`customer_name`,
`customers`.`city`,
`customers`.`state`,
`customers`.`country`,
`products`.`product_num`,
`products`.`year`,
`products`.`make`,
`products`.`model`,
`products`.`price`
FROM
`orders`
inner join `customers`
ON `customers`.`customer_num` = `orders`.`customer_num`
inner join `p_o`
ON `p_o`.`order_num` = `orders`.`order_num`
inner join `products`
ON `products`.`product_num` = `p_o`.`product_num`
I'd like (and I hope) to help you understand the process of multiple join basing on this problem.
The in-words description of this multiple join operation could be like this:
I need to extract detailed information about orders i.e.: order header info, order customer detailed info and products ordered within each of the order numbers.
According to my database structure I can see that every order header from orders table has field with customer number whom the order belongs to, so I want to connect the detailed data from customers table to every order from orders table.
Then I know that I will also need the detailed specification of each product for every specific order.
My DB structure says that I can access product detailed information from products table through "connection" table p_othen I'm connecting p_o to my already-joined set {customers, orders} basing on the order_num column.
Having this information in the result set by now ({customers, orders, p_o}) I only need to connect products table to the p_o table on the product_code key.
In the end I list the columns needed from final result set consisting from four tables {customers, orders, p_o, products}.
I hope it could help you some way.
I have two tables. One is called map_life, and the second one is called scripts. The map_life table has a lot of rows, that are identified by a column called lifeid. The rows at the table scripts are identified by the column objectid. I want to create a query that gets all the rows from the table map_life and also adds the column scriptfrom scripts table if lifeidmatches objectid, and that the objecttype is npc.
I created the following query.
SELECT id
,lifeid
,x_pos
,y_pos
,foothold
,min_click_pos
,max_click_pos
,respawn_time
,flags
,script.script
FROM map_life life
LEFT JOIN scripts script
ON script.objectid = life.lifeid
AND script.script_type = 'npc'
However, that query takes a lot of time. Any way I can tune it? Thanks.
EDIT: I have ran EXPLAIN command, there are the results.
"id","select_type","table","type","possible_keys","key","key_len","ref","rows","Extra"
1,"SIMPLE","life","ALL","","","","",47600,""
1,"SIMPLE","script","ref","PRIMARY","PRIMARY","1","const",1834,"Using where"
EDIT 2: Here are the create statmenets of each table.
map_life
DROP TABLE IF EXISTS `mcdb`.`map_life`;
CREATE TABLE `mcdb`.`map_life` (
`id` bigint(21) unsigned NOT NULL AUTO_INCREMENT,
`mapid` int(11) NOT NULL,
`life_type` enum('npc','mob','reactor') NOT NULL,
`lifeid` int(11) NOT NULL,
`life_name` varchar(50) DEFAULT NULL COMMENT 'For reactors, specifies a handle so scripts may interact with them; for NPC/mob, this field is useless',
`x_pos` smallint(6) NOT NULL DEFAULT '0',
`y_pos` smallint(6) NOT NULL DEFAULT '0',
`foothold` smallint(6) NOT NULL DEFAULT '0',
`min_click_pos` smallint(6) NOT NULL DEFAULT '0',
`max_click_pos` smallint(6) NOT NULL DEFAULT '0',
`respawn_time` int(11) NOT NULL DEFAULT '0',
`flags` set('faces_left') NOT NULL DEFAULT '',
PRIMARY KEY (`id`,`lifeid`) USING BTREE,
KEY `lifetype` (`mapid`,`life_type`)
) ENGINE=InnoDB AUTO_INCREMENT=47557 DEFAULT CHARSET=latin1;
scripts
DROP TABLE IF EXISTS `mcdb`.`scripts`;
CREATE TABLE `mcdb`.`scripts` (
`script_type` enum('npc','reactor','quest','item','map','map_enter','map_first_enter') NOT NULL,
`helper` tinyint(3) NOT NULL DEFAULT '-1' COMMENT 'Represents the quest state for quests, and the index of the script for NPCs (NPCs may have multiple scripts).',
`objectid` int(11) NOT NULL DEFAULT '0',
`script` varchar(40) NOT NULL DEFAULT '',
PRIMARY KEY (`script_type`,`helper`,`objectid`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 COMMENT='Lists all the scripts that belong to NPCs/reactors/etc. ';
You should probably add an index to the 'script_type' field depending on the type. If it's not using a type that can be indexed, you should change the type if possible and index
Here is a link that discusses more about indexes with MySQL, http://dev.mysql.com/doc/refman/5.0/en/mysql-indexes.html
Your primary key on scripts is:
PRIMARY KEY (`script_type`,`helper`,`objectid`)
The order of multi-column keys is important.
From the docs:
Any index that does not span all AND levels in the WHERE clause is not
used to optimize the query. In other words, to be able to use an
index, a prefix of the index must be used in every AND group.
Your primary key on scripts does include both the script_type and objectid columns, which are both used in the join's ON clause:
ON script.objectid = life.lifeid
AND script.script_type = 'npc'
but the primary key also includes the helper column between those two, so MySQL can only use the primary key index for searching using the first column (script_type).
So, for every join, MySQL must search through all scripts records where script_type is 'npc' to find the particular objectid record to join on.
MySQL could full utilize the primary key index if your ON clause included all three columns like this:
ON script.objectid = life.lifeid
AND script.helper = 1
AND script.script_type = 'npc'
If you often query the scripts table without specifying the helper column, consider changing the order of the columns in the primary key to put the helper column last:
PRIMARY KEY (`script_type`,`objectid`,`helper`)
Then, your original ON clause is appropriate for the index because the index prefix includes all of the columns in your predicate (script_type,objectid):
ON script.objectid = life.lifeid
AND script.script_type = 'npc'
Alternatively, add an additional index with just the two columns mentioned in the ON clause:
KEY `scrypt_type_objectid` (`script_type`,`objectid`)
Can anyone suggest a database design for the following:
A user can make a catalogue
Within a catalogue a user can make a range - i.e. a range of products
Within a range a user can add multiple products
Within a range a user can add multiple ranges -> range->range->range all with products in them.
I currently have in my database -
catalogue_range with - id, name, description
and
catalogue_product with - id, range_id, name, description
can anyone see what I'm trying to produce?
My aim is to be able to make multiple catalogue ranges within a catalogue range and add multiple products to each of these catalogue ranges.
Here is my current SQL:
`catalogue_range` (
`id` char(40) NOT NULL,
`profile_id` char(40) NOT NULL,
`type` enum('pdf','db') DEFAULT NULL,
`status` enum('new','draft','live') NOT NULL,
`name` varchar(64) NOT NULL,
`description` varchar(1000) NOT NULL,
`updated` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
KEY `profile_id` (`profile_id`)
)
`catalogue_product` (
`id` char(40) NOT NULL,
`catalogue_id` char(40) NOT NULL,
`order` smallint(5) unsigned NOT NULL,
`name` varchar(50) NOT NULL,
`description` varchar(250) NOT NULL,
PRIMARY KEY (`id`),
KEY `catalogue_id` (`catalogue_id`)
)
Thanks in advance.
catalogue(catalogue id, your private attributes)
product(product id, #catalogue id, your private attributes)
range(range id, #range id parent, your private attributes)
product range(#product id, #range id)
You will need stored procedures/applicative algorithms to compile:
the list of product of a range (to calculate recursive sqls mysql doesn't offer analytic functions as oracle does)
the list of ranges of a catalogue/range
Hope it helps.
S.
Assuming that a product can only exist in one catalogue at a time, your design is almost alright as it is. What you are missing is a recursive foreign key on catalogue_range. Add something like the following to your catalogue_range table definition:
`parent_range_id` char(40) NULL,
FOREIGN KEY (`parent_range_id`) REFERENCES catalogue_range(`id`)
The top level range(s) for any given user will have a NULL parent_range_id, others will refer to the containing range. Note that hierarchies aren't necessarily easy to work with in SQL. You may also want to look into techniques for making hierarchical data more SQL-friendly, such as nested sets.