Related
I want to extract the substrings from a string in MySQL. The string contains multiple substrings separated by commas(','). I need to extract these substrings using any MySQL functions.
For example:
Table Name: Product
-----------------------------------
item_code name colors
-----------------------------------
102 ball red,yellow,green
104 balloon yellow,orange,red
I want to select the colors field and extract the substrings as red, yellow and green as separated by comma.
A possible duplicate of this: Split value from one field to two
Unfortunately, MySQL does not feature a split string function.
As in the link above indicates there are User-defined Split function.
A more verbose version to fetch the data can be the following:
SELECT SUBSTRING_INDEX(SUBSTRING_INDEX(colors, ',', 1), ',', -1) as colorfirst,
SUBSTRING_INDEX(SUBSTRING_INDEX(colors, ',', 2), ',', -1) as colorsecond
....
SUBSTRING_INDEX(SUBSTRING_INDEX(colors, ',', n), ',', -1) as colornth
FROM product;
Based on https://blog.fedecarg.com/2009/02/22/mysql-split-string-function/, here is a way to access a value from a delimiter separated array:
/*
usage:
SELECT get_from_delimiter_split_string('1,5,3,7,4', ',', 1); -- returns '5'
SELECT get_from_delimiter_split_string('1,5,3,7,4', ',', 10); -- returns ''
*/
CREATE FUNCTION get_from_delimiter_split_string(
in_array varchar(255),
in_delimiter char(1),
in_index int
)
RETURNS varchar(255) CHARSET utf8mb4 COLLATE utf8mb4_unicode_ci
RETURN REPLACE( -- remove the delimiters after doing the following:
SUBSTRING( -- pick the string
SUBSTRING_INDEX(in_array, in_delimiter, in_index + 1), -- from the string up to index+1 counts of the delimiter
LENGTH(
SUBSTRING_INDEX(in_array, in_delimiter, in_index) -- keeping only everything after index counts of the delimiter
) + 1
),
in_delimiter,
''
);
here are the docs for the string operators for reference: https://dev.mysql.com/doc/refman/8.0/en/string-functions.html
Check the use of SPLIT_STR function here at line 64 to 69
SELECT INSTR('359616044513513-2574', '-')
I want to assign values to variables like dev=359616044513513 and id=2574
How can I do it in stored procedure of MYSQL?
Try this:
SELECT SUBSTRING_INDEX('359616044513513-2574', '-', 1),
SUBSTRING_INDEX('359616044513513-2574', '-', -1) INTO #dev, #id;
I have a denormalised records in my table:
ID, CODES
1 |1|2|3|4
2 |5|6|7|8
In second column there are int values, saved in varchar field separated by | symbol.
I want to convert them to normal Many2Many relational form, using link table.
So I want to create a table like this
ID CODE
1 1
1 2
1 3
1 4
....
2 8
I understand that I can iterate through the records in mysql stored function, split string and insert value. But I am interested: is it possible to convert data this way without stored procedure/function, but using only query (create table ... select ...)?
Thanks.
UPD: There is variable number of codes in different rows. Each line has from 1 to 15 codes.
Here's how it works, inclusive test data and so on.
But consider that this is just a fun answer. The way to go is clearly a stored procedure or a function or whatever.
drop table testvar;
create table testvar (id int, codes varchar(20));
insert into testvar values (1, '|1|2|3|4'), (2, '|5|6|7|8');
drop table if exists inserttest;
create table inserttest (id int, code int);
select #sql:=left(concat('insert into inserttest values ', group_concat( '(', id, ',', replace(right(codes, length(codes) - 1), '|', concat( '),(', id, ',' )), '),' separator '')), length(concat('insert into inserttest values ', group_concat( '(', id, ',', replace(right(codes, length(codes) - 1), '|', concat( '),(', id, ',' )), '),' separator ''))) -1)
from testvar;
prepare stmt1 from #sql;
execute stmt1;
select * from inserttest;
The Oracle way is:
insert into newtestvar
select t.id, to_number(substr(t.codes, p1 + 1, p2))
from (
select testvar.id, testvar.codes, s.num,
instr(testvar.codes, '|',1,s.num) p1,
instr(testvar.codes||'|', '|',1,s.num + 1)- instr(testvar.codes, '|',1,s.num) - 1 p2
from testvar, (select level num from dual connect by level <= 15) s
where s.num <= (length(testvar.codes)-length(replace(testvar.codes, '|')))
) t;
I hope you can adapt it for mysql.
How do you insert selected rows from table_source to table_target using SQL in MySQL where:
Both tables have the same schema
All columns should transfer except for the auto-increment id
Without explicitly writing all the column names, as that would be tedious
The trivial INSERT INTO table_target SELECT * FROM table_source fails on duplicate entries for primary key.
Either you list all of the fields you want in the insert...select, or you use something else externally to build the list for you.
SQL does not have something like SELECT * except somefield FROM, so you'll have to bite the bullet and write out the field names.
Column names have to be specified -
INSERT INTO table_target SELECT NULL, column_name1, column_name2, column_name3, ...
FROM table_source;
Just pass NULL as a value for the auto-increment id field.
Of course, primary key must be unique. It depends on what you want to achieve, but you could exclude rows with a primary key that already exists.
INSERT INTO table_target SELECT * FROM table_source
WHERE table_source.id NOT IN (SELECT id FROM table_target)
UPDATE: since you also need the extra rows, you should resolve the conflict first, does table_source have relationships? If not you could change those keys:
UPDATE table_source SET id = id + 1000
WHERE id IN (SELECT id FROM table_target)
Where 1000, is a constant, big enough so they go after the end of your table.
Tedious but safe and correct.
Writing INSERT statements without providing a list of columns leads to code that's hard to debug and, more importantly, very fragile code that will break if the definition of the table is changed.
If you absolutely can't write the column names out yourself then it's relatively easy to build a tool into your code that will create the comma-separated list for you.
This is my final solution to mass update with 'replace insert' command.
SET ##session.group_concat_max_len = ##global.max_allowed_packet;
SET #schema_db = 'db';
SET #tabl = 'table';
SET #cols = (SELECT CONCAT('`',GROUP_CONCAT(COLUMN_NAME SEPARATOR '`, `'), '`') FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_SCHEMA = #schema_db AND TABLE_NAME = #tabl GROUP BY TABLE_NAME);
SET #Querystr = CONCAT('REPLACE INTO ',#schema_db,'.',#tabl,' SELECT ', #cols,' FROM import.tbl_', #tabl);
PREPARE stmt FROM #Querystr;
EXECUTE stmt;
I think you could use syntax like:
INSERT INTO table (a,b,c) VALUES (1,2,3)
ON DUPLICATE KEY UPDATE c=c+1;
REF: http://dev.mysql.com/doc/refman/5.0/en/insert-on-duplicate.html
Hope it helps
INSERT IGNORE just "bypass" the duplicate rows.
http://dev.mysql.com/doc/refman/5.5/en/insert.html
You can probably do it with prepared statements.
PREPARE table_target_insert FROM 'INSERT INTO table_target SELECT ? FROM table_source';
SET #cols:='';
SELECT #cols:=GROUP_CONCAT(IF(column_name = 'id','NULL',column_name) separator ",") FROM information_schema.columns WHERE table_name='table_source';
EXECUTE table_target_insert USING #cols;
It seems as if columns can not be given as a place holder in a MySQL Prepared Statement. I have compiled the following solution for testing:
SET #schema_db = 'DB';
SET #table = 'table';
SET #cols = (SELECT CONCAT(GROUP_CONCAT(COLUMN_NAME SEPARATOR ', '), "\n") FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_SCHEMA = #schema_db AND TABLE_NAME = #table GROUP BY TABLE_NAME);
SET #Querystr = CONCAT('SELECT',' ', #cols,' ','FROM',' ',#schema_db,'.',#table,' ', 'Limit 5');
PREPARE stmt FROM #Querystr;
EXECUTE stmt;
You can use dynamic query:
DECLARE #Columns VARCHAR(MAX)=''
DECLARE #Query VARCHAR(MAX)=''
SELECT
#Columns = ISNULL(#Columns +',', '') + T.COLUMN_NAME
FROM
(
select name as COLUMN_NAME from sys.all_columns
where object_id = (select object_id from sys.tables where name = 'Source_Table')
and is_identity = 0
)T
set #Query = 'insert into Target_Table (' + SUBSTRING(#Columns,2 , 9999) + ') select ' + SUBSTRING(#Columns,2 , 9999) + ' from Source_Table';
PRINT #Query
EXEC(#Query)
The easiest way to do it is to use phpmyadmin to write the list of columns, then to change it as needed, in the example below I want to duplicate row with id=1078 and in this table I have id unique auto increment and alias unique.therefore I created my query as follow, with id & alias replaced by a desired value. and it worked like a charm.
INSERT INTO sy3_menuselect 1079, menutype, title, "alias", note, path, link, type, published, parent_id, level, component_id, checked_out, checked_out_time, browserNav, access, img, template_style_id, params, lft, rgt, home, language, client_id from sy3_menuwhere id=1078
Alternatively, to auto increment id, use the following Join statement:
INSERT INTO sy3_menuselect *
from (SELECT MAX(id+1 )from sy3_menu)a
join (select menutype, title, "alias", note, path, link, type, published, parent_id, level, component_id, checked_out, checked_out_time, browserNav, access, img, template_style_id, params, lft, rgt, home, language, client_idfrom sy3_menuwhere id=1079)b
I've got a table field membername which contains both the last name and the first name of users. Is it possible to split those into 2 fields memberfirst, memberlast?
All the records have this format "Firstname Lastname" (without quotes and a space in between).
Unfortunately MySQL does not feature a split string function. However you can create a user defined function for this, such as the one described in the following article:
MySQL Split String Function by Federico Cargnelutti
With that function:
DELIMITER $$
CREATE FUNCTION SPLIT_STR(
x VARCHAR(255),
delim VARCHAR(12),
pos INT
)
RETURNS VARCHAR(255) DETERMINISTIC
BEGIN
RETURN REPLACE(SUBSTRING(SUBSTRING_INDEX(x, delim, pos),
LENGTH(SUBSTRING_INDEX(x, delim, pos -1)) + 1),
delim, '');
END$$
DELIMITER ;
you would be able to build your query as follows:
SELECT SPLIT_STR(membername, ' ', 1) as memberfirst,
SPLIT_STR(membername, ' ', 2) as memberlast
FROM users;
If you prefer not to use a user defined function and you do not mind the query to be a bit more verbose, you can also do the following:
SELECT SUBSTRING_INDEX(SUBSTRING_INDEX(membername, ' ', 1), ' ', -1) as memberfirst,
SUBSTRING_INDEX(SUBSTRING_INDEX(membername, ' ', 2), ' ', -1) as memberlast
FROM users;
SELECT variant (not creating a user defined function):
SELECT IF(
LOCATE(' ', `membername`) > 0,
SUBSTRING(`membername`, 1, LOCATE(' ', `membername`) - 1),
`membername`
) AS memberfirst,
IF(
LOCATE(' ', `membername`) > 0,
SUBSTRING(`membername`, LOCATE(' ', `membername`) + 1),
NULL
) AS memberlast
FROM `user`;
This approach also takes care of:
membername values without a space: it will add the whole string to memberfirst and sets memberlast to NULL.
membername values that have multiple spaces: it will add everything before the first space to memberfirst and the remainder (including additional spaces) to memberlast.
The UPDATE version would be:
UPDATE `user` SET
`memberfirst` = IF(
LOCATE(' ', `membername`) > 0,
SUBSTRING(`membername`, 1, LOCATE(' ', `membername`) - 1),
`membername`
),
`memberlast` = IF(
LOCATE(' ', `membername`) > 0,
SUBSTRING(`membername`, LOCATE(' ', `membername`) + 1),
NULL
);
It seems that existing responses are over complicated or not a strict answer to the particular question.
I think, the simple answer is the following query:
SELECT
SUBSTRING_INDEX(`membername`, ' ', 1) AS `memberfirst`,
SUBSTRING_INDEX(`membername`, ' ', -1) AS `memberlast`
;
I think it is not necessary to deal with more-than-two-word names in this particular situation. If you want to do it properly, splitting can be very hard or even impossible in some cases:
Johann Sebastian Bach
Johann Wolfgang von Goethe
Edgar Allan Poe
Jakob Ludwig Felix Mendelssohn-Bartholdy
Petőfi Sándor
Virág Vendelné Farkas Margit
黒澤 明
In a properly designed database, human names should be stored both in parts and in whole. This is not always possible, of course.
If your plan is to do this as part of a query, please don't do that (a). Seriously, it's a performance killer. There may be situations where you don't care about performance (such as one-off migration jobs to split the fields allowing better performance in future) but, if you're doing this regularly for anything other than a mickey-mouse database, you're wasting resources.
If you ever find yourself having to process only part of a column in some way, your DB design is flawed. It may well work okay on a home address book or recipe application or any of myriad other small databases but it will not be scalable to "real" systems.
Store the components of the name in separate columns. It's almost invariably a lot faster to join columns together with a simple concatenation (when you need the full name) than it is to split them apart with a character search.
If, for some reason you cannot split the field, at least put in the extra columns and use an insert/update trigger to populate them. While not 3NF, this will guarantee that the data is still consistent and will massively speed up your queries. You could also ensure that the extra columns are lower-cased (and indexed if you're searching on them) at the same time so as to not have to fiddle around with case issues.
And, if you cannot even add the columns and triggers, be aware (and make your client aware, if it's for a client) that it is not scalable.
(a) Of course, if your intent is to use this query to fix the schema so that the names are placed into separate columns in the table rather than the query, I'd consider that to be a valid use. But I reiterate, doing it in the query is not really a good idea.
use this
SELECT SUBSTRING_INDEX(SUBSTRING_INDEX( `membername` , ' ', 2 ),' ',1) AS b,
SUBSTRING_INDEX(SUBSTRING_INDEX( `membername` , ' ', -1 ),' ',2) AS c FROM `users` WHERE `userid`='1'
In MySQL this is working this option:
SELECT Substring(nameandsurname, 1, Locate(' ', nameandsurname) - 1) AS
firstname,
Substring(nameandsurname, Locate(' ', nameandsurname) + 1) AS lastname
FROM emp
Not exactly answering the question, but faced with the same problem I ended up doing this:
UPDATE people_exit SET last_name = SUBSTRING_INDEX(fullname,' ',-1)
UPDATE people_exit SET middle_name = TRIM(SUBSTRING_INDEX(SUBSTRING_INDEX(fullname,last_name,1),' ',-2))
UPDATE people_exit SET middle_name = '' WHERE CHAR_LENGTH(middle_name)>3
UPDATE people_exit SET first_name = SUBSTRING_INDEX(fullname,concat(middle_name,' ',last_name),1)
UPDATE people_exit SET first_name = middle_name WHERE first_name = ''
UPDATE people_exit SET middle_name = '' WHERE first_name = middle_name
The only case where you may want such a function is an UPDATE query which will alter your table to store Firstname and Lastname into separate fields.
Database design must follow certain rules, and Database Normalization is among most important ones
I had a column where the first and last name were both were in one column. The first and last name were separated by a comma. The code below worked. There is NO error checking/correction. Just a dumb split. Used phpMyAdmin to execute the SQL statement.
UPDATE tblAuthorList SET AuthorFirst = SUBSTRING_INDEX(AuthorLast,',',-1) , AuthorLast = SUBSTRING_INDEX(AuthorLast,',',1);
13.2.10 UPDATE Syntax
This takes smhg from here and curt's from Last index of a given substring in MySQL and combines them. This is for mysql, all I needed was to get a decent split of name to first_name last_name with the last name a single word, the first name everything before that single word, where the name could be null, 1 word, 2 words, or more than 2 words. Ie: Null; Mary; Mary Smith; Mary A. Smith; Mary Sue Ellen Smith;
So if name is one word or null, last_name is null. If name is > 1 word, last_name is last word, and first_name all words before last word.
Note that I've already trimmed off stuff like Joe Smith Jr. ; Joe Smith Esq. and so on, manually, which was painful, of course, but it was small enough to do that, so you want to make sure to really look at the data in the name field before deciding which method to use.
Note that this also trims the outcome, so you don't end up with spaces in front of or after the names.
I'm just posting this for others who might google their way here looking for what I needed. This works, of course, test it with the select first.
It's a one time thing, so I don't care about efficiency.
SELECT TRIM(
IF(
LOCATE(' ', `name`) > 0,
LEFT(`name`, LENGTH(`name`) - LOCATE(' ', REVERSE(`name`))),
`name`
)
) AS first_name,
TRIM(
IF(
LOCATE(' ', `name`) > 0,
SUBSTRING_INDEX(`name`, ' ', -1) ,
NULL
)
) AS last_name
FROM `users`;
UPDATE `users` SET
`first_name` = TRIM(
IF(
LOCATE(' ', `name`) > 0,
LEFT(`name`, LENGTH(`name`) - LOCATE(' ', REVERSE(`name`))),
`name`
)
),
`last_name` = TRIM(
IF(
LOCATE(' ', `name`) > 0,
SUBSTRING_INDEX(`name`, ' ', -1) ,
NULL
)
);
Method I used to split first_name into first_name and last_name when the data arrived all in the first_name field. This will put only the last word in the last name field, so "john phillips sousa" will be "john phillips" first name and "sousa" last name. It also avoids overwriting any records that have been fixed already.
set last_name=trim(SUBSTRING_INDEX(first_name, ' ', -1)), first_name=trim(SUBSTRING(first_name,1,length(first_name) - length(SUBSTRING_INDEX(first_name, ' ', -1)))) where list_id='$List_ID' and length(first_name)>0 and length(trim(last_name))=0
UPDATE `salary_generation_tbl` SET
`modified_by` = IF(
LOCATE('$', `other_salary_string`) > 0,
SUBSTRING(`other_salary_string`, 1, LOCATE('$', `other_salary_string`) - 1),
`other_salary_string`
),
`other_salary` = IF(
LOCATE('$', `other_salary_string`) > 0,
SUBSTRING(`other_salary_string`, LOCATE('$', `other_salary_string`) + 1),
NULL
);
In case someone needs to run over a table and split a field:
First we use the function mention above:
CREATE DEFINER=`root`#`localhost` FUNCTION `fn_split_str`($str VARCHAR(800), $delimiter VARCHAR(12), $position INT) RETURNS varchar(800) CHARSET utf8
DETERMINISTIC
BEGIN
RETURN REPLACE(
SUBSTRING(
SUBSTRING_INDEX($str, $delimiter, $position),
LENGTH(
SUBSTRING_INDEX($str, $delimiter, $position -1)
) + 1
),
$delimiter, '');
END
Second, we run in a while loop on the string until there isn't any results (I've added $id for JOIN clause):
CREATE DEFINER=`root`#`localhost` FUNCTION `fn_split_str_to_rows`($id INT, $str VARCHAR(800), $delimiter VARCHAR(12), $empty_table BIT) RETURNS int(11)
BEGIN
DECLARE position INT;
DECLARE val VARCHAR(800);
SET position = 1;
IF $empty_table THEN
DROP TEMPORARY TABLE IF EXISTS tmp_rows;
END IF;
SET val = fn_split_str($str, ',', position);
CREATE TEMPORARY TABLE IF NOT EXISTS tmp_rows AS (SELECT $id as id, val as val where 1 = 2);
WHILE (val IS NOT NULL and val != '') DO
INSERT INTO tmp_rows
SELECT $id, val;
SET position = position + 1;
SET val = fn_split_str($str, ',', position);
END WHILE;
RETURN position - 1;
END
Finally we can use it like that:
DROP TEMPORARY TABLE IF EXISTS tmp_rows;
SELECT SUM(fn_split_str_to_rows(ID, FieldToSplit, ',', 0))
FROM MyTable;
SELECT * FROM tmp_rows;
You can use the id to join to other table.
In case you are only splitting one value you can use it like that
SELECT fn_split_str_to_rows(null, 'AAA,BBB,CCC,DDD,EEE,FFF,GGG', ',', 1);
SELECT * FROM tmp_rows;
We don't need to empty the temporary table, the function will take care of that.
mysql 5.4 provides a native split function:
SPLIT_STR(<column>, '<delimiter>', <index>)