MySQL: Split comma separated list into multiple rows - mysql

I have an unnormalized table with a column containing a comma separated list that is a foreign key to another table:
+----------+-------------+ +--------------+-------+
| part_id | material | | material_id | name |
+----------+-------------+ +--------------+-------+
| 339 | 1.2mm;1.6mm | | 1 | 1.2mm |
| 970 | 1.6mm | | 2 | 1.6mm |
+----------+-------------+ +--------------+-------+
I want to read this data into a search engine that offers no procedural language.
So is there a way to either make a join on this column or run a query on this data that inserts appropriate entries into a new table?
The resulting data should look like this:
+---------+-------------+
| part_id | material_id |
+---------+-------------+
| 339 | 1 |
| 339 | 2 |
| 970 | 2 |
+---------+-------------+
I could think of a solution if the DBMS supported functions returning a table but MySQL apparently doesn't.

In MySQL this can be achieved as below
SELECT id, length FROM vehicles WHERE id IN ( 117, 148, 126)
+---------------+
| id | length |
+---------------+
| 117 | 25 |
| 126 | 8 |
| 148 | 10 |
+---------------+
SELECT id,vehicle_ids FROM load_plan_configs WHERE load_plan_configs.id =42
+---------------------+
| id | vehicle_ids |
+---------------------+
| 42 | 117, 148, 126 |
+---------------------+
Now to get the length of comma separated vehicle_ids use below query
Output
SELECT length
FROM vehicles, load_plan_configs
WHERE load_plan_configs.id = 42 AND FIND_IN_SET(
vehicles.id, load_plan_configs.vehicle_ids
)
+---------+
| length |
+---------+
| 25 |
| 8 |
| 10 |
+---------+
For more info visit http://amitbrothers.blogspot.in/2014/03/mysql-split-comma-separated-list-into.html

I've answered two similar questions in as many days but not had any responses so I guess people are put off by the use of the cursor but as it should be a one off process I personally dont think that matters.
As you stated MySQL doesnt support table return types yet so you have little option other than to loop the table and parse the material csv string and generate the appropriate rows for part and material.
The following posts may prove of interest:
split keywords for post php mysql
MySQL procedure to load data from staging table to other tables. Need to split up multivalue field in the process
Rgds

MySQL does not have temporary table reuse and functions do not return rows.
I can't find anything in Stack Overflow to convert string of csv integers into rows so I wrote my own in MySQL.
DELIMITER $$
DROP PROCEDURE IF EXISTS str_split $$
CREATE PROCEDURE str_split(IN str VARCHAR(4000),IN delim varchar(1))
begin
DECLARE delimIdx int default 0;
DECLARE charIdx int default 1;
DECLARE rest_str varchar(4000) default '';
DECLARE store_str varchar(4000) default '';
create TEMPORARY table IF NOT EXISTS ids as (select parent_item_id from list_field where 1=0);
truncate table ids;
set #rest_str = str;
set #delimIdx = LOCATE(delim,#rest_str);
set #charIdx = 1;
set #store_str = SUBSTRING(#rest_str,#charIdx,#delimIdx-1);
set #rest_str = SUBSTRING(#rest_str from #delimIdx+1);
if length(trim(#store_str)) = 0 then
set #store_str = #rest_str;
end if;
INSERT INTO ids
SELECT (#store_str + 0);
WHILE #delimIdx <> 0 DO
set #delimIdx = LOCATE(delim,#rest_str);
set #charIdx = 1;
set #store_str = SUBSTRING(#rest_str,#charIdx,#delimIdx-1);
set #rest_str = SUBSTRING(#rest_str from #delimIdx+1);
select #store_str;
if length(trim(#store_str)) = 0 then
set #store_str = #rest_str;
end if;
INSERT INTO ids(parent_item_id)
SELECT (#store_str + 0);
END WHILE;
select parent_item_id from ids;
end$$
DELIMITER ;
call str_split('1,2,10,13,14',',')
You will also need to cast to different types if you are not using ints.

You can also use REGEXP
SET #materialids=SELECT material FROM parttable where part_id=1;
SELECT * FROM material_id WHERE REGEXP CONCAT('^',#materialids,'$');
This will help if you want to get just one part. Not the whole table of course

Related

MySQL trigger to update a column by reordering its values

Sorry if the title is miss-leading, I couldn't come up with a better one that is related to my issue.
I've been trying to solve this for a while now, and I couldn't find the solution.
I have a table categories:
+----+--------+----------+
| ID | Name | Position |
+----+--------+----------+
| 1 | Dogs | 4 |
| 2 | Cats | 3 |
| 3 | Birds | 10 |
| 4 | Others | 2 |
+----+--------+----------+
I need to keep the Position column in order, in a way not to miss an values as well, so the final table should look like:
+----+--------+----------+
| ID | Name | Position |
+----+--------+----------+
| 1 | Dogs | 3 |
| 2 | Cats | 2 |
| 3 | Birds | 4 |
| 4 | Others | 1 |
+----+--------+----------+
What I tried doing, is creating a trigger on UPDATE and on INSERT that would try to prevent this. The trigger I created ( same one before INSERT) :
DELIMITER //
CREATE TRIGGER sortPostions BEFORE UPDATE ON categories
FOR EACH ROW BEGIN
SET #max_pos = 0;
SET #min_pos = 0;
SET #max_ID = 0;
SET #min_ID = 0;
SELECT position, id INTO #max_pos,#max_ID FROM categories WHERE position = ( SELECT MAX(position) FROM categories);
SELECT position, id INTO #min_pos,#min_ID FROM categories WHERE position = ( SELECT MIN(position) FROM categories);
IF NEW.position >= #max_pos AND NEW.id != #max_ID THEN
SET NEW.position = #max_pos + 1;
END IF;
IF NEW.position <= #min_pos AND NEW.id != #min_ID THEN
SET NEW.position = #min_pos - 1;
END IF;
IF NEW.position < 0 THEN
SET NEW.position = 0;
END IF;
END//
DELIMITER ;
But unfortunately it's not working as intended. It's not fixing missing values and I think this is not a perfect solution.
I went ahead and created a procedure:
BEGIN
SET #n = 0;
UPDATE categories
SET position = #n:=#n+1
ORDER BY position ASC;
END
But I wasn't able to call this procedure from a trigger, as it seems that MySQL doesn't allow that. I get the following error:
#1442 - Can't update table 'categories' in stored function/trigger because it is already used by statement which invoked this stored function/trigger.
mysql -V output:
mysql Ver 14.14 Distrib 5.5.57, for debian-linux-gnu (x86_64) using readline 6.3
What's the perfect solution to solve this problem ?
Thanks a lot!
You can't do this in a trigger. MySQL does not allow you to do an INSERT/UPDATE/DELETE in a trigger (or in a procedure called by the trigger) against the same table for which the trigger was spawned.
The reason is that it can result in infinite loops (your update spawns the trigger, which updates the table, which spawns the trigger again, which updates the table again...). Also because it can create lock conflicts if more than one of these requests happens concurrently.
You should do this in application code if you must renumber the position of the rows. Do it with a separate statement after your initial query has completed.
Another option is don't worry about making the positions consecutive. Just make sure they are in the right order. Then when you query the table, generate row numbers on demand.
SELECT (#n:=#n+1) AS row_num, c.*
FROM (SELECT #n:=0 AS n) AS _init
CROSS JOIN categories AS c
ORDER BY c.position ASC;
+---------+----+--------+----------+
| row_num | id | name | position |
+---------+----+--------+----------+
| 1 | 4 | Others | 2 |
| 2 | 2 | Cats | 3 |
| 3 | 1 | Dogs | 4 |
| 4 | 3 | Birds | 10 |
+---------+----+--------+----------+
In MySQL 8.0, you'll be able to do this with more standard syntax using ROW_NUMBER().
SELECT ROW_NUMBER() OVER w AS row_num, c.*
FROM categories AS c
WINDOW w AS (ORDER BY c.position ASC);
Gives the same output as the query using the user-variable.

MySQL second auto increment field based on foreign key

I've come across this problem numerous times but haven't found a "MySQL way" to solve the issue as such - I have a database that contains users and reports. Each report has an id which I display as a report number to my users.
The main complaint is that users are confused as to why reports have gone missing from their system. This is not actually the case. It is actually that they are recognizing a gap between their IDs and assume that these are missing reports, when in actual fact, it is simply becasue another user has filled in this auto-incrementing gap.
I need to know if there is a way to do this in MySQL:
Is it possible that I can have a second auto-increment field called report_number which is based on a user_id field which has a different set of auto-increments per user?
e.g.
|------|---------|---------------|
| id | user_id | report_number |
|------|---------|---------------|
| 1 | 1 | 1 |
| 2 | 1 | 2 |
| 3 | 1 | 3 |
| 4 | 2 | 1 |
| 5 | 1 | 4 |
| 6 | 1 | 5 |
| 7 | 2 | 2 |
| 8 | 3 | 1 |
| 9 | 3 | 2 |
|------|---------|---------------|
I am using InnoDB for this as it is quite heavily weighted with foreign-keys. It appears to complain when I add a second auto increment field, but I wasn't sure if there was a different way to do this?
MyISAM supports the second column with auto increment, but InnoDB doesn't.
For InnoDB you might create a trigger BEFORE INSERT to get the max value of the reportid and add one to the value.
DELIMITER $$
CREATE TRIGGER report_trigger
BEFORE INSERT ON reports
FOR EACH ROW BEGIN
SET NEW.`report_id` = (SELECT MAX(report_id) + 1 FROM reports WHERE user_id = NEW.user_id);
END $$
DELIMITER ;
If you can use MyISAM instead, in the documentation of MySQL page there is an example:
http://dev.mysql.com/doc/refman/5.0/en/example-auto-increment.html
CREATE TABLE animals (
grp ENUM('fish','mammal','bird') NOT NULL,
id MEDIUMINT NOT NULL AUTO_INCREMENT,
name CHAR(30) NOT NULL,
PRIMARY KEY (grp,id)
) ENGINE=MyISAM;
INSERT INTO animals (grp,name) VALUES
('mammal','dog'),('mammal','cat'),
('bird','penguin'),('fish','lax'),('mammal','whale'),
('bird','ostrich');
SELECT * FROM animals ORDER BY grp,id;
Which returns:
+--------+----+---------+
| grp | id | name |
+--------+----+---------+
| fish | 1 | lax |
| mammal | 1 | dog |
| mammal | 2 | cat |
| mammal | 3 | whale |
| bird | 1 | penguin |
| bird | 2 | ostrich |
+--------+----+---------+
Right one with IFNULL:
DELIMITER $$
CREATE TRIGGER salons_trigger
BEFORE INSERT ON salon
FOR EACH ROW BEGIN
SET NEW.salon_id = IFNULL((SELECT MAX(salon_id) + 1 FROM salon WHERE owner = NEW.owner), 1);
END $$
DELIMITER ;
I think mysql doesnt support two auto_increment columns. you can create report number using information schema.
select NULL from information_schema.columns
MySQl does not support two auto incremented fields, if you need then create another table, set the other field which you want to be as auto incremented and you must set up a relationship with these two tables.

Search contacts upto multiple levels [duplicate]

I have a database with a tree of names that can go down a total of 9 levels deep and I need to be able to search down a signal branch of the tree from any point on the branch.
Database:
+----------------------+
| id | name | parent |
+----------------------+
| 1 | tom | 0 |
| 2 | bob | 0 |
| 3 | fred | 1 |
| 4 | tim | 2 |
| 5 | leo | 4 |
| 6 | sam | 4 |
| 7 | joe | 6 |
| 8 | jay | 3 |
| 9 | jim | 5 |
+----------------------+
Tree:
tom
fred
jay
bob
tim
sam
joe
leo
jim
For example:
If I search "j" from the user "bob" I should get only "joe" and "jim". If I search "j" form "leo" I should only get "jim".
I can't think of any easy way do to this so any help is appreciated.
You should really consider using the Modified Preorder Tree Traversal which makes such queries much easier. Here's your table expressed with MPTT. I have left the parent field, as it makes some queries easier.
+----------------------+-----+------+
| id | name | parent | lft | rght |
+----------------------+-----+------+
| 1 | tom | 0 | 1 | 6 |
| 2 | bob | 0 | 7 | 18 |
| 3 | fred | 1 | 2 | 5 |
| 4 | tim | 2 | 8 | 17 |
| 5 | leo | 4 | 12 | 15 |
| 6 | sam | 4 | 9 | 16 |
| 7 | joe | 6 | 10 | 11 |
| 8 | jay | 3 | 3 | 4 |
| 9 | jim | 5 | 13 | 14 |
+----------------------+-----+------+
To search j from user bob you'd use the lft and rght values for bob:
SELECT * FROM table WHERE name LIKE 'j%' AND lft > 7 AND rght < 18
Implementing the logic to update lft and rght for adding, removing and reordering nodes can be a challenge (hint: use an existing library if you can) but querying will be a breeze.
There isn't a nice/easy way of doing this; databases don't support tree-style data structures well.
You will need to work on a level-by-level basis to prune results from child-to-parent, or create a view that gives all 9 generations from a given node, and match using an OR on the descendants.
Have you thought about using a recursive loop? i use a loop for a cms i built on top of codeigniter that allows me to start anywhere in the site tree and will then subsequently filter trhough all the children> grand children > great grand children etc. Plus it keeps the sql down to short rapid queries opposed to lots of complicated joins. It may need some modifying in your case but i think it could work.
/**
* build_site_tree
*
* #return void
* #author Mike Waites
**/
public function build_site_tree($parent_id)
{
return $this->find_children($parent_id);
}
/** end build_site_tree **/
// -----------------------------------------------------------------------
/**
* find_children
* Recursive loop to find parent=>child relationships
*
* #return array $children
* #author Mike Waites
**/
public function find_children($parent_id)
{
$this->benchmark->mark('find_children_start');
if(!class_exists('Account_model'))
$this->load->model('Account_model');
$children = $this->Account_model->get_children($parent_id);
/** Recursively Loop over the results to build the site tree **/
foreach($children as $key => $child)
{
$childs = $this->find_children($child['id']);
if (count($childs) > 0)
$children[$key]['children'] = $childs;
}
return $children;
$this->benchmark->mark('find_children_end');
}
/** end find_children **/
As you can see this is a pretty simplfied version and bear in mind this has been built into codeigniter so you will need to modyfy it to suite but basically we have a loop that calls itself adding to an array each time as it goes. This will allow you to get the whole tree, or even start from a point in the tree as long as you have the parent_id avaialble first!
Hope this helps
The new "recursive with" construct will do the job, but I don't know id MySQL supports it (yet).
with recursive bobs(id) as (
select id from t where name = 'bob'
union all
select t.id from t, bobs where t.parent_id = bobs.id
)
select t.name from t, bobs where t.id = bobs.id
and name like 'j%'
There is no single SQL query that will return the data in tree format - you need processing to traverse it in the right order.
One way is to query MySQL to return MPTT:
SELECT * FROM table ORDER BY parent asc;
root of the tree will be the first item of the table, its children will be next, etc., the tree being listed "breadth first" (in layers of increasing depth)
Then use PHP to process the data, turning it into an object that holds the data structure.
Alternatively, you could implement MySQL search functions that given a node, recursively search and return a table of all its descendants, or a table of all its ancestors. As these procedures tend to be slow (being recursive, returning too much data that is then filtered by other criteria), you want to only do this if you know you're not querying for that kind of data again and again, or if you know that the data set remains small (9 levels deep and how wide?)
You can do this with a stored procedure as follows:
Example calls
mysql> call names_hier(1, 'a');
+----+----------+--------+-------------+-------+
| id | emp_name | parent | parent_name | depth |
+----+----------+--------+-------------+-------+
| 2 | ali | 1 | f00 | 1 |
| 8 | anna | 6 | keira | 4 |
+----+----------+--------+-------------+-------+
2 rows in set (0.00 sec)
mysql> call names_hier(3, 'k');
+----+----------+--------+-------------+-------+
| id | emp_name | parent | parent_name | depth |
+----+----------+--------+-------------+-------+
| 6 | keira | 5 | eva | 2 |
+----+----------+--------+-------------+-------+
1 row in set (0.00 sec)
$sqlCmd = sprintf("call names_hier(%d,'%s')", $id, $name); // dont forget to escape $name
$result = $db->query($sqlCmd);
Full script
drop table if exists names;
create table names
(
id smallint unsigned not null auto_increment primary key,
name varchar(255) not null,
parent smallint unsigned null,
key (parent)
)
engine = innodb;
insert into names (name, parent) values
('f00',null),
('ali',1),
('megan',1),
('jessica',3),
('eva',3),
('keira',5),
('mandy',6),
('anna',6);
drop procedure if exists names_hier;
delimiter #
create procedure names_hier
(
in p_id smallint unsigned,
in p_name varchar(255)
)
begin
declare v_done tinyint unsigned default(0);
declare v_dpth smallint unsigned default(0);
set p_name = trim(replace(p_name,'%',''));
create temporary table hier(
parent smallint unsigned,
id smallint unsigned,
depth smallint unsigned
)engine = memory;
insert into hier select parent, id, v_dpth from names where id = p_id;
/* http://dev.mysql.com/doc/refman/5.0/en/temporary-table-problems.html */
create temporary table tmp engine=memory select * from hier;
while not v_done do
if exists( select 1 from names n inner join tmp on n.parent = tmp.id and tmp.depth = v_dpth) then
insert into hier select n.parent, n.id, v_dpth + 1
from names n inner join tmp on n.parent = tmp.id and tmp.depth = v_dpth;
set v_dpth = v_dpth + 1;
truncate table tmp;
insert into tmp select * from hier where depth = v_dpth;
else
set v_done = 1;
end if;
end while;
select
n.id,
n.name as emp_name,
p.id as parent,
p.name as parent_name,
hier.depth
from
hier
inner join names n on hier.id = n.id
left outer join names p on hier.parent = p.id
where
n.name like concat(p_name, '%');
drop temporary table if exists hier;
drop temporary table if exists tmp;
end #
delimiter ;
-- call this sproc from your php
call names_hier(1, 'a');
call names_hier(3, 'k');

INSERT / UPDATE SQL random & unique VARCHAR

I need to be able to INSERT/UPDATE UNIQUE RANDOM UTF8 ALPHANUMERICAL VARCHAR 55 into a table field called 'key'.
Can't find out any good query example, does anyone can show me or link me something?
This answer is based on mysql.
This select will create 55 char long random strings:
select substr(concat(md5(rand()),md5(rand())),1,55);
to fill your table column you might want to try out:
create table example (keycol varchar(55));
insert into example (keycol) values (substr(concat(md5(rand()),md5(rand())),1,55));
The result will be:
select keycol from example;
+---------------------------------------------------------+
| keycol |
+---------------------------------------------------------+
| 4517f4be669301a4a529b53fc18d646dec42d4d07d911d33a67c863 |
| 3caa1c98f0f9ee39515aa6f4ddb3f84fa41abd5392f610c5d24bcd9 |
| 8e52cb4ce29e58514671c9b68f19832f26ddf53f277621ac420bd2e |
| 3adcccfb6cb729ce1c0a14fb75f6fd54f58992dc0751527c969e007 |
| c28c5879589dc90f4fb0963673e5668fa5789d325423ba043e0243b |
| 8f7a2af97d73261008f0d0d7249480fde56a3a91f2ce6e8bf0b0070 |
| ff4f74f25b92da3eaab282218c23a75d4cfa77c8f8bfdf74d7ebdf9 |
+---------------------------------------------------------+

Getting limited amount of records from hierarchical data

Let's say I have 3 tables (significant columns only)
Category (catId key, parentCatId)
Category_Hierarchy (catId key, parentTrail, catLevel)
Product (prodId key, catId, createdOn)
There's a reason for having a separate Category_Hierarchy table, because I'm using triggers on Category table that populate it, because MySql triggers work as they do and I can't populate columns on the same table inside triggers if I would like to use auto_increment values. For the sake of this problem this is irrelevant. These two tables are 1:1 anyway.
Category table could be:
+-------+-------------+
| catId | parentCatId |
+-------+-------------+
| 1 | NULL |
| 2 | 1 |
| 3 | 2 |
| 4 | 3 |
| 5 | 3 |
| 6 | 4 |
| ... | ... |
+-------+-------------+
Category_Hierarchy
+-------+-------------+----------+
| catId | parentTrail | catLevel |
+-------+-------------+----------+
| 1 | 1/ | 0 |
| 2 | 1/2/ | 1 |
| 3 | 1/2/3/ | 2 |
| 4 | 1/2/3/4/ | 3 |
| 5 | 1/2/3/5/ | 3 |
| 6 | 1/2/3/4/6/ | 4 |
| ... | ... | ... |
+-------+-------------+----------+
Product
+--------+-------+---------------------+
| prodId | catId | createdOn |
+--------+-------+---------------------+
| 1 | 4 | 2010-02-03 12:09:24 |
| 2 | 4 | 2010-02-03 12:09:29 |
| 3 | 3 | 2010-02-03 12:09:36 |
| 4 | 1 | 2010-02-03 12:09:39 |
| 5 | 3 | 2010-02-03 12:09:50 |
| ... | ... | ... |
+--------+-------+---------------------+
Category_Hierarchy makes it simple to get category subordinate trees like this:
select c.*
from Category c
join Category_Hierarchy h
on (h.catId = c.catId)
where h.parentTrail like '1/2/3/%'
Which would return complete subordinate tree of category 3 (that is below 2, that is below 1 which is root category) including subordinate tree root node. Excluding root node is just one more where condition.
The problem
I would like to write a stored procedure:
create procedure GetLatestProductsFromSubCategories(in catId int)
begin
/* return 10 latest products from each */
/* catId subcategory subordinate tree */
end;
This means if a certain category had 3 direct sub categories (with whatever number of nodes underneath) I would get 30 results (10 from each subordinate tree). If it had 5 sub categories I'd get 50 results.
What would be the best/fastest/most efficient way to do this? If possible I'd like to avoid cursors unless they'd work faster compared to any other solution as well as prepared statements, because this would be one of the most frequent calls to DB.
Edit
Since a picture tells 1000 words I'll try to better explain what I want using an image. Below image shows category tree. Each of these nodes can have an arbitrary number of products related to them. Products are not included in the picture.
So if I'd execute this call:
call GetLatestProductsFromSubCategories(1);
I'd like to effectively get 30 products:
10 latest products from the whole orange subtree
10 latest products from the whole blue subtree and
10 latest products from the whole green subtree
I don't want to get 10 latest products from each node under catId=1 node which would mean 320 products.
Final Solution
This solution has O(n) performance:
CREATE PROCEDURE foo(IN in_catId INT)
BEGIN
DECLARE done BOOLEAN DEFAULT FALSE;
DECLARE first_iteration BOOLEAN DEFAULT TRUE;
DECLARE current VARCHAR(255);
DECLARE categories CURSOR FOR
SELECT parentTrail
FROM category
JOIN category_hierarchy USING (catId)
WHERE parentCatId = in_catId;
DECLARE CONTINUE HANDLER FOR SQLSTATE '02000' SET done = TRUE;
SET #query := '';
OPEN categories;
category_loop: LOOP
FETCH categories INTO current;
IF `done` THEN LEAVE category_loop; END IF;
IF first_iteration = TRUE THEN
SET first_iteration = FALSE;
ELSE
SET #query = CONCAT(#query, " UNION ALL ");
END IF;
SET #query = CONCAT(#query, "(SELECT product.* FROM product JOIN category_hierarchy USING (catId) WHERE parentTrail LIKE CONCAT('",current,"','%') ORDER BY createdOn DESC LIMIT 10)");
END LOOP category_loop;
CLOSE categories;
IF #query <> '' THEN
PREPARE stmt FROM #query;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
END IF;
END
Edit
Due to the latest clarification, this solution was simply edited to simplify the categories cursor query.
Note: Make the VARCHAR on line 5 the appropriate size based on your parentTrail column.