I have a MySQL stored procedure (shown below) that's supposed to build a list of IDs from a table of hierarchically related records. I'm having to re-engineer an older stored procedure to switch from using a simple CONCAT function to GROUP_CONCAT because the former couldn't handle the sizes of the lists being generated (i.e., the lists are going well over the 1024 character limit of the CONCAT function).
DELIMITER $$
CREATE PROCEDURE SPTest (top_id INT)
BEGIN
DECLARE ids_all TEXT;
SET SESSION group_concat_max_len = 1000000;
SET ids_all = top_id;
SET #str = GROUP_CONCAT('SELECT GROUP_CONCAT(id SEPARATOR \', \') ',
'FROM tbl WHERE nha_id = ', top_id, ' INTO #ids_tmp' SEPARATOR '');
PREPARE stmt FROM #str;
EXECUTE stmt;
WHILE #ids_tmp != "" DO
SET ids_all = GROUP_CONCAT(ids_all, #ids_tmp SEPARATOR ', ');
SET #str = GROUP_CONCAT('SELECT GROUP_CONCAT(id SEPARATOR \', \') ',
'FROM tbl WHERE nha_id IN (', #ids_tmp, ') INTO #ids_tmp' SEPARATOR '');
PREPARE stmt FROM #str;
EXECUTE stmt;
END WHILE;
SELECT ids_all AS ids;
END $$
DELIMITER ;
The problem is this routine is generating the following error when I try to call it (and sometimes it returns this error when I try to create the Stored Procedure):
ERROR 1111 (HY000): Invalid use of group function
When I manually create the same kinds of queries this code would build and run them at the command-line, they work perfectly fine -- no errors of any kind, and I get the results I expect. I've seen posts (both here on Stack Exchange and elsewhere) which say that MySQL doesn't support nested aggregate functions, but the concatenation here is just being done to strings. So I thought that maybe somehow it was seeing the GROUP_CONCAT in the string and burping because of that, so I tried putting "XXXX" in place of "GROUP_CONCAT" in the string and then using the REPLACE function to switch it back, but that made no difference. I also tried it with WHERE and HAVING for the criteria clause, but neither one worked. I've also done an extensive web search and was unable to find anything that was in any way helpful.
I don't know what else to try, mainly because I can't see what's wrong with the syntax here. Any help would be appreciated.
Update 1:
I have since tried a modified version of the script where the GROUP_CONCAT merges data from a subquery like this:
SET #qrystr = GROUP_CONCAT('SELECT GROUP_CONCAT(c SEPARATOR ", ") ',
'FROM (SELECT id AS c FROM tbl WHERE nha_id IN (', #ids_tmp,
') AS d INTO #ids_tmp' SEPARATOR '');
but that made no difference either.
You can't use GROUP_CONCAT() as a scalar function; it must be used in the context of a set of rows. Similarly, you can't use any other aggregate function without a table reference:
SET #x = MAX(<expr>); -- makes no sense
Ideally you should upgrade to MySQL 8.0 if you haven't already, and use a recursive CTE query instead:
WITH RECURSIVE hierarchy AS (
SELECT top_id AS id
UNION
SELECT tbl.id FROM tbl JOIN hierarchy ON tbl.nha_id = hierarchy.id
)
SELECT GROUP_CONCAT(id SEPARATOR ', ') AS ids_all FROM hierarchy;
That would eliminate the need for using loops or prepare/execute or temp variables.
Thanks to Bill Karwin's answer regarding the need to use SELECT with GROUP_CONCAT, I was able to see how my code needed to be changed. Rather than using a SET statement to assign the values to the variables, I need to use a SELECT ... INTO ... construct, as shown here:
DELIMITER $$
CREATE PROCEDURE ASSEMBLY_LIST_TEST (top_id INT)
BEGIN
DECLARE ids_all TEXT DEFAULT '';
SET SESSION group_concat_max_len = 1000000;
SET ids_all = top_id;
# Find the 1st-level children of 'top_id' to start building the list
SELECT GROUP_CONCAT('SELECT GROUP_CONCAT(id SEPARATOR ", ") FROM tbl ',
'WHERE nha_id = ', top_id, ' INTO #ids_tmp' SEPARATOR '') INTO #qrystr;
PREPARE stmt FROM #qrystr;
EXECUTE stmt;
# Recursively find the children of each level of children of the previous loop & add to the list
WHILE #ids_tmp != '' DO
SELECT GROUP_CONCAT(ids_all, ', ', #ids_tmp SEPARATOR '') INTO ids_all;
SELECT GROUP_CONCAT('SELECT GROUP_CONCAT(id SEPARATOR ", ") FROM tbl ',
'WHERE nha_id IN (', #ids_tmp, ') INTO #ids_tmp' SEPARATOR '') INTO #qrystr;
PREPARE stmt FROM #qrystr;
EXECUTE stmt;
END WHILE;
SELECT ids_all AS ids;
END $$
DELIMITER ;
This now produces exactly the result I was looking for. Of course, the recursive query approach is a much better solution for what I want this to do, but maybe my solution will help someone else.
I have to run a query in SQL Server using data from a MySQL database. When I needed to do the opposite, I found an easy way to accomplish what I needed writing an update query using the select statement in SQL Server.
In SQL Server I wrote:
SELECT 'update sgidb.Example set MySQLCol1 = ' + cast(MSSQLCol1 as varchar(max)) + ' where MySQLCol2 = ' + cast(MSSQLCol2 as varchar(max)) + ';' FROM MSSQLTable
That resulted in a bunch of update statements with the keys I needed like:
'update sgidb.Example set MySQLCol1 = 12 where MySQLCol2 = 45;
But when I tried to do the same in MySQL I got a bunch of syntax errors. The web told me MySQL don't need the + operator to concatenate strings in a sentence, but it didn't work, neither writing the concatenate function explicitly. Any ideas?
You can use the CONCAT function which is available in MySQL as well as in SQL, like this:
SELECT CONCAT('update sgidb.Example set MySQLCol1 = ' , MSSQLCol1 , ' where MySQLCol2 = ' , MSSQLCol2 , ';' )FROM MSSQLTable
Now in the above solution you need to take care of the blank space after or before or even after and before the statement.
For tackling the above situation what you can do is to use the function CONCAT_WS, which is available in MySQL as well as in SQL:
SELECT CONCAT_WS(' ', 'update sgidb.Example set MySQLCol1 =' , MSSQLCol1 , 'where MySQLCol2 =' , MSSQLCol2 , ';' )FROM MSSQLTable
CONCAT_WS function adds two or more strings together with a separator.
Now no need to take care of the spaces that you need to put to avoid the syntax error anymore.
Please note that, CONCAT_WS is going to handle null as well. But in case of CONCAT, if any of the variable/field is null then the entire CONCATENATED result becomes null.
Here's how you can do it
SELECT concat("update sgidb.Example set MySQLCol1 = ",MSSQLCol1,"where MySQLCol2 = ",MSSQLCol2,";") FROM MSSQLTable;
Both SQL Server and mysql have got CONCAT function.
You can use the below query in both RDBMS.
SELECT CONCAT('update sgidb.Example set MySQLCol1 = ' , MSSQLCol1 , ' where MySQLCol2 = ' , MSSQLCol2 , ';' )FROM MSSQLTable
You may try concat to concatenate your string.
Example as:
SELECT CONCAT('MySQL CAST example #',CAST(2 AS CHAR));
#ouput
MySQL CAST example #2
In your above update query your set and where column may have varchar, but if you closely look into your query ' is missing in both of the column names. Please try the below query.
SELECT CONCAT('update sgidb.Example set MySQLCol1 = ''',
CAST(MSSQLCol1 AS VARCHAR(MAX)),
''' where MySQLCol2 = ''',
CAST(MSSQLCol2 AS VARCHAR(MAX)) ,
''';' )
FROM MSSQLTable;
I've got a table field membername which contains both the last name and the first name of users. Is it possible to split those into 2 fields memberfirst, memberlast?
All the records have this format "Firstname Lastname" (without quotes and a space in between).
Unfortunately MySQL does not feature a split string function. However you can create a user defined function for this, such as the one described in the following article:
MySQL Split String Function by Federico Cargnelutti
With that function:
DELIMITER $$
CREATE FUNCTION SPLIT_STR(
x VARCHAR(255),
delim VARCHAR(12),
pos INT
)
RETURNS VARCHAR(255) DETERMINISTIC
BEGIN
RETURN REPLACE(SUBSTRING(SUBSTRING_INDEX(x, delim, pos),
LENGTH(SUBSTRING_INDEX(x, delim, pos -1)) + 1),
delim, '');
END$$
DELIMITER ;
you would be able to build your query as follows:
SELECT SPLIT_STR(membername, ' ', 1) as memberfirst,
SPLIT_STR(membername, ' ', 2) as memberlast
FROM users;
If you prefer not to use a user defined function and you do not mind the query to be a bit more verbose, you can also do the following:
SELECT SUBSTRING_INDEX(SUBSTRING_INDEX(membername, ' ', 1), ' ', -1) as memberfirst,
SUBSTRING_INDEX(SUBSTRING_INDEX(membername, ' ', 2), ' ', -1) as memberlast
FROM users;
SELECT variant (not creating a user defined function):
SELECT IF(
LOCATE(' ', `membername`) > 0,
SUBSTRING(`membername`, 1, LOCATE(' ', `membername`) - 1),
`membername`
) AS memberfirst,
IF(
LOCATE(' ', `membername`) > 0,
SUBSTRING(`membername`, LOCATE(' ', `membername`) + 1),
NULL
) AS memberlast
FROM `user`;
This approach also takes care of:
membername values without a space: it will add the whole string to memberfirst and sets memberlast to NULL.
membername values that have multiple spaces: it will add everything before the first space to memberfirst and the remainder (including additional spaces) to memberlast.
The UPDATE version would be:
UPDATE `user` SET
`memberfirst` = IF(
LOCATE(' ', `membername`) > 0,
SUBSTRING(`membername`, 1, LOCATE(' ', `membername`) - 1),
`membername`
),
`memberlast` = IF(
LOCATE(' ', `membername`) > 0,
SUBSTRING(`membername`, LOCATE(' ', `membername`) + 1),
NULL
);
It seems that existing responses are over complicated or not a strict answer to the particular question.
I think, the simple answer is the following query:
SELECT
SUBSTRING_INDEX(`membername`, ' ', 1) AS `memberfirst`,
SUBSTRING_INDEX(`membername`, ' ', -1) AS `memberlast`
;
I think it is not necessary to deal with more-than-two-word names in this particular situation. If you want to do it properly, splitting can be very hard or even impossible in some cases:
Johann Sebastian Bach
Johann Wolfgang von Goethe
Edgar Allan Poe
Jakob Ludwig Felix Mendelssohn-Bartholdy
Petőfi Sándor
Virág Vendelné Farkas Margit
黒澤 明
In a properly designed database, human names should be stored both in parts and in whole. This is not always possible, of course.
If your plan is to do this as part of a query, please don't do that (a). Seriously, it's a performance killer. There may be situations where you don't care about performance (such as one-off migration jobs to split the fields allowing better performance in future) but, if you're doing this regularly for anything other than a mickey-mouse database, you're wasting resources.
If you ever find yourself having to process only part of a column in some way, your DB design is flawed. It may well work okay on a home address book or recipe application or any of myriad other small databases but it will not be scalable to "real" systems.
Store the components of the name in separate columns. It's almost invariably a lot faster to join columns together with a simple concatenation (when you need the full name) than it is to split them apart with a character search.
If, for some reason you cannot split the field, at least put in the extra columns and use an insert/update trigger to populate them. While not 3NF, this will guarantee that the data is still consistent and will massively speed up your queries. You could also ensure that the extra columns are lower-cased (and indexed if you're searching on them) at the same time so as to not have to fiddle around with case issues.
And, if you cannot even add the columns and triggers, be aware (and make your client aware, if it's for a client) that it is not scalable.
(a) Of course, if your intent is to use this query to fix the schema so that the names are placed into separate columns in the table rather than the query, I'd consider that to be a valid use. But I reiterate, doing it in the query is not really a good idea.
use this
SELECT SUBSTRING_INDEX(SUBSTRING_INDEX( `membername` , ' ', 2 ),' ',1) AS b,
SUBSTRING_INDEX(SUBSTRING_INDEX( `membername` , ' ', -1 ),' ',2) AS c FROM `users` WHERE `userid`='1'
In MySQL this is working this option:
SELECT Substring(nameandsurname, 1, Locate(' ', nameandsurname) - 1) AS
firstname,
Substring(nameandsurname, Locate(' ', nameandsurname) + 1) AS lastname
FROM emp
Not exactly answering the question, but faced with the same problem I ended up doing this:
UPDATE people_exit SET last_name = SUBSTRING_INDEX(fullname,' ',-1)
UPDATE people_exit SET middle_name = TRIM(SUBSTRING_INDEX(SUBSTRING_INDEX(fullname,last_name,1),' ',-2))
UPDATE people_exit SET middle_name = '' WHERE CHAR_LENGTH(middle_name)>3
UPDATE people_exit SET first_name = SUBSTRING_INDEX(fullname,concat(middle_name,' ',last_name),1)
UPDATE people_exit SET first_name = middle_name WHERE first_name = ''
UPDATE people_exit SET middle_name = '' WHERE first_name = middle_name
The only case where you may want such a function is an UPDATE query which will alter your table to store Firstname and Lastname into separate fields.
Database design must follow certain rules, and Database Normalization is among most important ones
I had a column where the first and last name were both were in one column. The first and last name were separated by a comma. The code below worked. There is NO error checking/correction. Just a dumb split. Used phpMyAdmin to execute the SQL statement.
UPDATE tblAuthorList SET AuthorFirst = SUBSTRING_INDEX(AuthorLast,',',-1) , AuthorLast = SUBSTRING_INDEX(AuthorLast,',',1);
13.2.10 UPDATE Syntax
This takes smhg from here and curt's from Last index of a given substring in MySQL and combines them. This is for mysql, all I needed was to get a decent split of name to first_name last_name with the last name a single word, the first name everything before that single word, where the name could be null, 1 word, 2 words, or more than 2 words. Ie: Null; Mary; Mary Smith; Mary A. Smith; Mary Sue Ellen Smith;
So if name is one word or null, last_name is null. If name is > 1 word, last_name is last word, and first_name all words before last word.
Note that I've already trimmed off stuff like Joe Smith Jr. ; Joe Smith Esq. and so on, manually, which was painful, of course, but it was small enough to do that, so you want to make sure to really look at the data in the name field before deciding which method to use.
Note that this also trims the outcome, so you don't end up with spaces in front of or after the names.
I'm just posting this for others who might google their way here looking for what I needed. This works, of course, test it with the select first.
It's a one time thing, so I don't care about efficiency.
SELECT TRIM(
IF(
LOCATE(' ', `name`) > 0,
LEFT(`name`, LENGTH(`name`) - LOCATE(' ', REVERSE(`name`))),
`name`
)
) AS first_name,
TRIM(
IF(
LOCATE(' ', `name`) > 0,
SUBSTRING_INDEX(`name`, ' ', -1) ,
NULL
)
) AS last_name
FROM `users`;
UPDATE `users` SET
`first_name` = TRIM(
IF(
LOCATE(' ', `name`) > 0,
LEFT(`name`, LENGTH(`name`) - LOCATE(' ', REVERSE(`name`))),
`name`
)
),
`last_name` = TRIM(
IF(
LOCATE(' ', `name`) > 0,
SUBSTRING_INDEX(`name`, ' ', -1) ,
NULL
)
);
Method I used to split first_name into first_name and last_name when the data arrived all in the first_name field. This will put only the last word in the last name field, so "john phillips sousa" will be "john phillips" first name and "sousa" last name. It also avoids overwriting any records that have been fixed already.
set last_name=trim(SUBSTRING_INDEX(first_name, ' ', -1)), first_name=trim(SUBSTRING(first_name,1,length(first_name) - length(SUBSTRING_INDEX(first_name, ' ', -1)))) where list_id='$List_ID' and length(first_name)>0 and length(trim(last_name))=0
UPDATE `salary_generation_tbl` SET
`modified_by` = IF(
LOCATE('$', `other_salary_string`) > 0,
SUBSTRING(`other_salary_string`, 1, LOCATE('$', `other_salary_string`) - 1),
`other_salary_string`
),
`other_salary` = IF(
LOCATE('$', `other_salary_string`) > 0,
SUBSTRING(`other_salary_string`, LOCATE('$', `other_salary_string`) + 1),
NULL
);
In case someone needs to run over a table and split a field:
First we use the function mention above:
CREATE DEFINER=`root`#`localhost` FUNCTION `fn_split_str`($str VARCHAR(800), $delimiter VARCHAR(12), $position INT) RETURNS varchar(800) CHARSET utf8
DETERMINISTIC
BEGIN
RETURN REPLACE(
SUBSTRING(
SUBSTRING_INDEX($str, $delimiter, $position),
LENGTH(
SUBSTRING_INDEX($str, $delimiter, $position -1)
) + 1
),
$delimiter, '');
END
Second, we run in a while loop on the string until there isn't any results (I've added $id for JOIN clause):
CREATE DEFINER=`root`#`localhost` FUNCTION `fn_split_str_to_rows`($id INT, $str VARCHAR(800), $delimiter VARCHAR(12), $empty_table BIT) RETURNS int(11)
BEGIN
DECLARE position INT;
DECLARE val VARCHAR(800);
SET position = 1;
IF $empty_table THEN
DROP TEMPORARY TABLE IF EXISTS tmp_rows;
END IF;
SET val = fn_split_str($str, ',', position);
CREATE TEMPORARY TABLE IF NOT EXISTS tmp_rows AS (SELECT $id as id, val as val where 1 = 2);
WHILE (val IS NOT NULL and val != '') DO
INSERT INTO tmp_rows
SELECT $id, val;
SET position = position + 1;
SET val = fn_split_str($str, ',', position);
END WHILE;
RETURN position - 1;
END
Finally we can use it like that:
DROP TEMPORARY TABLE IF EXISTS tmp_rows;
SELECT SUM(fn_split_str_to_rows(ID, FieldToSplit, ',', 0))
FROM MyTable;
SELECT * FROM tmp_rows;
You can use the id to join to other table.
In case you are only splitting one value you can use it like that
SELECT fn_split_str_to_rows(null, 'AAA,BBB,CCC,DDD,EEE,FFF,GGG', ',', 1);
SELECT * FROM tmp_rows;
We don't need to empty the temporary table, the function will take care of that.
mysql 5.4 provides a native split function:
SPLIT_STR(<column>, '<delimiter>', <index>)