mySQL: Query performance with User Defined Functions - mysql

I'm trying to make a multi-language mySQL database.
I was considering using user defined functions to determine which column in a table to read - where different columns will store the different translations - however I was concerned about query performance.
For example, if I wanted to return a list of Cities, and the name of country - with the latter returned in multiple languages. At what point would the below approach impact performance? - as I might do an equivalent on a table with 2-5,000 rows.
Country table structure:
SELECT
`CountryID`,
`Name_English`,
`Name_French`,
`Name_Spanish`
FROM `Country`
WHERE `CountryID` = fCountryID;
City Table Structure:
SELECT
`City`.`CityName` 'City'
,getCountry(1, `City`.`CountryID`) 'Country'
FROM `City`;
Example Function Call:
SELECT
`City`.`CityName` 'City'
,getCountry(1, `City`.`CountryID`) 'Country'
FROM `City`;
Full Function:
delimiter $$
CREATE DEFINER=root#localhost FUNCTION
getCountry(fLanguageID INT, fCountryID SMALLINT)
RETURNS varchar(100) CHARSET utf8 COLLATE utf8_unicode_ci
BEGIN
DECLARE returnCountry VARCHAR(100);
IF (fLanguageID = 1) -- English
THEN
SET returnCountry = (
SELECT `Name_English` FROM `Country`
WHERE `CountryID` = fCountryID
);
ELSEIF (fLanguageID = 2) -- French
THEN
SET returnCountry = (
SELECT `Name_French` FROM `Country`
WHERE `CountryID` = fCountryID
);
ELSEIF (fLanguageID = 3) -- Spanish
THEN
SET returnCountry = (
SELECT `Name_Spanish` FROM `Country`
WHERE `CountryID` = fCountryID
);
END IF;
RETURN returnCountry;
SELECT
`CountryID`,
`Name_English`,
`Name_French`,
`Name_Spanish`
FROM `Country`
WHERE `CountryID` = fCountryID;
SELECT
`City`.`CityName` 'City'
,getCountry(1, `City`.`CountryID`) 'Country'
FROM `City`;
END$$

Related

How to `SELECT FROM` a table that is a part of a query itself using MySQL?

Say, if I have multiple tables that have the same schema:
CREATE TABLE `tbl01`
(
`id` INT UNSIGNED AUTO_INCREMENT PRIMARY KEY,
`name` TINYTEXT,
`data` INT
);
CREATE TABLE `tbl02`
(
`id` INT UNSIGNED AUTO_INCREMENT PRIMARY KEY,
`name` TINYTEXT,
`data` INT
);
CREATE TABLE `tbl03`
(
`id` INT UNSIGNED AUTO_INCREMENT PRIMARY KEY,
`name` TINYTEXT,
`data` INT
);
-- etc. ------------------
INSERT INTO `tbl01` (`name`, `data`) VALUES
('row 1', 1),
('row 2', 1),
('row 3', 3);
INSERT INTO `tbl02` (`name`, `data`) VALUES
('cube', 1),
('circle', 0);
INSERT INTO `tbl03` (`name`, `data`) VALUES
('one', 1);
and then one table that contains names of all other tables in one of its columns:
CREATE TABLE `AllTbls`
(
`id` INT UNSIGNED AUTO_INCREMENT PRIMARY KEY,
`tblnm` VARCHAR(64) NOT NULL UNIQUE,
`desc` TINYTEXT,
`flgs` BIGINT UNSIGNED
);
INSERT INTO `AllTbls` (`tblnm`, `desc`, `flgs`) VALUES
('tbl01', 'Table 1', 0),
('tbl02', 'Table two', 1),
('tbl03', '3rd table', 0);
So if I want to write a query to retrieve contents of AllTbls and also in one column to include count of rows in each of corresponding tables, I thought the following would be the way to do it:
SELECT *, `tblnm` as TblName, (SELECT COUNT(*) FROM TblName) as cntRws
FROM `AllTbls` ORDER BY `id` ASC LIMIT 0,30;
But this returns an error:
#1146 - Table 'database.TblName' doesn't exist
I know that I can do this in multiple queries (using a loop in a programming language), but is it possible to do it in one query?
PS. I'm using MySQL v.5.7.28
The simple answer is: "you can't"
Table names are not supposed to be used like variables, to hold data, in this way. What you're supposed to have is one table:
tblContractCounts
Client, ContractCount
-------------------
IBM, 1
Microsoft, 3
Google, 2
Not three tables:
tblIBMContractCounts
ContractCount
1
tblMicrosoftContractCounts
ContractCount
3
tblGoogleContractCounts
ContractCount
2
If your number of tables is known and fixed you can perhaps remedy things by creating a view that unions them all back together, or embarking on an operation to put them all into one table, with separate views named the old names so things carry in working til you can change them. If new tables are added all the time it's a flaw in the data modelling and need to be corrected. In that case you'd have to use a programming language (front end or stored procedure) to build a single query:
//pseudo code
strSql = ""
for each row in dbquery("Select name from alltbls")
strSql += "select '" + row.name + "' as tbl, count(*) as ct from " + row.name + " union all "
next row
strSql += "select 'dummy', 0"
result = dbquery(strSql)
It doesn't have to be your front end that does this - you could also do this in mysql and leverage the dynamic sql / EXECUTE. See THIS ANSWER how we can concatenate a string using logic like above so that the string contains an sql query and then execute the query. The information schema will give you the info you need to get a list of all current table names
But all you're doing is working around the fact that your data modelling is broken; I recommend to fix that instead
ps: the INFORMATION_SCHEMA has rough counts for tables with their names, which may suffice for your needs in this particular case
select table_name, table_rows from infornation_schema.tables where table_name like ...
I managed to solve the problem using the following stored procedure.
-- DROP PROCEDURE sp_Count_Rows;
Delimiter $$
CREATE PROCEDURE sp_Count_Rows()
BEGIN
DECLARE table_name TEXT DEFAULT "";
DECLARE finished INTEGER DEFAULT 0;
DECLARE table_cursor
CURSOR FOR
SELECT tblnm FROM alltbls;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET finished = 1;
OPEN table_cursor;
DROP TABLE IF EXISTS RowsCount;
CREATE TABLE IF NOT EXISTS RowsCount(Tlbnm text, ctnRws int);
table_loop: LOOP
FETCH table_cursor INTO table_name;
IF finished = 1 THEN
LEAVE table_loop;
END IF;
SET #s = CONCAT("insert into RowsCount select '", table_name ,"', count(*) as cntRws from ", table_name);
PREPARE stmt1 FROM #s;
EXECUTE stmt1;
DEALLOCATE PREPARE stmt1;
END LOOP table_loop;
CLOSE table_cursor;
SELECT * FROM RowsCount;
DROP TABLE RowsCount;
END
$$
And then when you call the procedure
CALL sp_Count_Rows();
You get this result

MySQL stored function to read column from row in table

We need a function / stored procedure to return a column in a row in a table, where we pass the table, id field name, id and column to return.
We have a table which is an activity log with columns such as
log_datetime - the date/time the user did something
log_table - the table affected
log_idcol - the column in log_table that holds the id
log_id - the id of the row in log_table
log_titlecol - the column in log_table which holds the 'title' of the
item
So in a report/ MySql statement I need to be able to list the title which is held in the column name held in log_titlecol, read from the table with the name in the log_table column, and id of whatever is in log_idcol.
so like
SELECT log_titlecol FROM log_table where log_idcol = log_id
where all the 'log_; parts are replaceable. I can't see a way to do this in MySQL, so is there a way to do this in a stored procedure/function?
For example:
CREATE TABLE jokes (
JokeID smallint(4) NOT NULL, **(autoincrement, unique)**
Title varchar(50) CHARACTER SET utf8 DEFAULT NULL,
Note text CHARACTER SET utf8
)
CREATE TABLE idioms (
IdiomID smallint(4) NOT NULL, **(autoincrement, unique)**
IdiomTitle varchar(50) CHARACTER SET utf8 DEFAULT NULL,
Note text CHARACTER SET utf8
)
CREATE TABLE `log` (
id smallint(4) NOT NULL, **(autoincrement, unique)**
log_datetime datetime,
log_table varchar(50) CHARACTER SET utf8 DEFAULT NULL,
log_idcol varchar(50) CHARACTER SET utf8 DEFAULT NULL,
log_id smallint(4) NOT NULL,
log_titlecol varchar(50) CHARACTER SET utf8 DEFAULT NULL,
)
INSERT INTO jokes (Title, Notes)
VALUES ('Funny joke','This is note1')
INSERT INTO jokes (Title, Notes)
VALUES ('Another Funny joke','This is another note')
INSERT INTO idioms (IdiomTitle, Notes)
VALUES ('Bird in the hand','What this means..')
INSERT INTO jokes (Title, Notes)
VALUES ('Another Funny joke','This is another note')
INSERT INTO log (log_datetime, log_table, log_idcol, log_id , log_titlecol)
VALUES (now(), 'jokes','JokeID',1,'Title')
INSERT INTO log (log_datetime, log_table, log_idcol, log_id , log_titlecol)
VALUES (now(), 'jokes','JokeID',2,'Title')
INSERT INTO log (log_datetime, log_table, log_idcol, log_id , log_titlecol)
VALUES (now(), 'idioms','IdiomID',2,'IdiomTitle')
So now how can I have a report from log showing the date time and the Title column from the Jokes table, row id as in the log_id column? I need a function.
eg SELECT log_datetime, log_table, log_id, GetTitle(log_table, log_idcol, log_id, log_titlecol) from log
Where GetTitle is a function which will return the column held in 'log_titlecol' form the table passed as log_table, from the row with an id (held in the log_idcol column) of log_id
So for example the output would show:
2018-01-01 12:00 jokes 1 Funny Joke
2018-01-01 12:10 jokes 2 Another Funny joke
2018-01-01 12:11 idioms 1 Bird in the hand
I have tried
CREATE PROCEDURE Getcol(IN tab TEXT CHARSET utf8mb4, IN col TEXT CHARSET utf8mb4, IN idcol TEXT CHARSET utf8mb4, IN id INT(15), OUT outcol TEXT CHARSET utf8mb4)
DETERMINISTIC
COMMENT 'Return a column from any table'
BEGIN
SET #Expression = CONCAT('SELECT ', col,' INTO #outcol FROM ', tab, ' where ', idcol, ' = ', id);
PREPARE myquery FROM #Expression;
EXECUTE myquery;
SELECT #outcol;
END
I can call this like
CALL GetCol('Jokes','Title','JokeID',1)
And this works to return the Title column for ID 1, but I am cant seem to then put this call into a function
CREATE FUNCTION getrowcol(tab TEXT, col TEXT, idcol TEXT, id INT) RETURNS text CHARSET utf8mb4
NO SQL
COMMENT 'Return a column from any table'
BEGIN
DECLARE outvar TEXT;
CALL GetCol(tab, col, idcol, id, #out1);
SELECT #out1 INTO outvar;
RETURN outvar;
END
This returns blank. How can I return #out1?
This is essentially the same question (with no answer)
https://dba.stackexchange.com/questions/151328/dynamic-sql-stored-procedure-called-by-a-function
Try something like,
DELIMITER //
CREATE PROCEDURE demo(tab VARCHAR(50), tit VARCHAR(50))
BEGIN
SET #Expression = CONCAT('SELECT l.log_datetime, j.',tit,' FROM `log` l INNER JOIN ',tab,' j;');
PREPARE myquery FROM #Expression;
EXECUTE myquery;
END
//
DELIMITER ;
Call it like
CALL demo('jokes', 'title')
from Reference

Function that generate Code returns the same things

There is a MySQL function in our web system to generate Code. The structure of the code is
district_cd(length:2) + date(length:8) + sequence no(length:5,start at 1).<like : ab2016090800001>
The sequence no was saved in table and will be updated (+1) when generate a new code.
But sometimes it returned two same codes and makes us fall in trouble. Here are the captures to replicate this problem, I will attach the DDL after this.
Step 1.Client1->change to manual commit then generate a code, but do not commit.
SET autocommit = 0;
select * from applies;
select * from sequence where apply_date = "2016-09-08";
select nextval("ab");
insert into applies (apply_id,apply_no,created,district_cd) values (2,"ab2016090800002","ab",now());
select * from sequence where apply_date = "2016-09-08";
Step2.Client2->change to manual commit then generate a code, stuck as Client1 locked
SET autocommit = 0;
select * from applies;
select * from sequence where apply_date = "2016-09-08";
insert into applies (apply_id,apply_no,created,district_cd) values (3,"ab20160908123456780","ab",now());
Step3.Client1->commit;
commit;
select * from sequence where apply_date = "2016-09-08";
Step4.Client2->code was generated and two records appeared in sequence table
select * from sequence where apply_date = "2016-09-08";
capture of Step4
Step5.Client2->commit;one of the two records that appeared in sequence table was deleted.The codes generated are duplicated.
commit;
select * from sequence where apply_date = "2016-09-08";
select * from applies;
capture of Step5
※DDL
Table:applies (apply_no:save the code)
CREATE TABLE `applies` (
`apply_id` varchar(100) NOT NULL DEFAULT '',
`apply_no` varchar(100) NOT NULL DEFAULT '',
`district_cd` varchar(100) DEFAULT NULL,
`created` datetime DEFAULT NULL,
PRIMARY KEY (`apply_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Table:sequence (current_value:save current sequnce value)
CREATE TABLE `sequence` (
`district_cd` varchar(3) NOT NULL DEFAULT '',
`current_value` int(11) NOT NULL DEFAULT '0',
`apply_date` date NOT NULL DEFAULT '0000-00-00',
PRIMARY KEY (`district_cd`,`current_value`,`apply_date`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Function:currval->get current sequence value by district_cd
DELIMITER ;;
CREATE DEFINER=`usr`#`%` FUNCTION `currval`(d VARCHAR(3)) RETURNS int(11)
DETERMINISTIC
BEGIN
DECLARE value INTEGER;
DECLARE needInitSequence INTEGER;
DECLARE today DATE;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET needInitSequence = 1;
SET value = 0;
SET today = current_date();
SELECT `current_value` INTO value
FROM `sequence`
WHERE `district_cd` = d AND `apply_date` = today limit 1;
IF needInitSequence = 1 THEN
INSERT INTO `sequence` (`district_cd`, `current_value`, `apply_date`) VALUES (d, value, today);
END IF;
RETURN value;
END
;;
DELIMITER ;
Function:nextval->generate code by district_cd
DELIMITER ;;
CREATE DEFINER=`usr`#`%` FUNCTION `nextval`(d VARCHAR(3)) RETURNS varchar(16) CHARSET utf8
DETERMINISTIC
BEGIN
DECLARE value INTEGER;
SET value = currval(d);
UPDATE `sequence`
SET `current_value` = `current_value` + 1
WHERE `district_cd` = d AND `apply_date` = current_date();
RETURN concat(d, date_format(now(), '%Y%m%d'), LPAD(currval(d), 5, '0'));
END
;;
DELIMITER ;
Triggers of applies->a business logic,if the length of apply_no is greater than 18,it will call the function:nextval to generate a new code
DELIMITER ;;
CREATE TRIGGER `convert_long_no` BEFORE INSERT ON `applies` FOR EACH ROW BEGIN
IF ((SELECT LENGTH(NEW.apply_no)) >= 18) THEN
SET NEW.apply_no = (SELECT nextval(NEW.district_cd));
END IF;
END
;;
DELIMITER ;
My Questions:
Why did the function:nextval returns two same codes?
Why did two records appear in sequnce when update the record.

MySQL use ExtractValue(XML, 'Value/Values') to get all multiple values (split one column into rows)

I have a non-normal field containing multiple values because it is Xml data that wasn't intended to be queried, until now. Can MySQL split this xml column into multiple rows?
Table
NameA | <Xml><Values<Value>1</Value><Value>2</Value><Value>3</Value></Values></Xml>
NameB | <Xml><Values<Value>1</Value><Value>2</Value></Values></Xml>
NameC | <Xml><Values<Value>1</Value><Value>2</Value><Value>3</Value><Value>4</Value></Values></Xml>
I want
NameA | 1
NameA | 2
NameA | 3
NameB | 1
Like this MSSQL/TSQL solution
SELECT
I.Name,
Value.value('.','VARCHAR(30)') AS Value
FROM
Item AS I
CROSS APPLY
Xml.nodes('/Xml/Values/Value') AS T(Value)
WHERE
I.TypeID = 'A'
But in MySQL I can only get
NameA | 123
NameB | 12
NameC | 1234
with
SELECT
I.`Name`,
ExtractValue(Xml,'/Xml/Values/Value') AS ListOfValues
FROM
Item AS I
WHERE
I.TypeID = 'A'
;
Are there any elegant ways to split xml in MySQL?
No. You must solve this just like other mysql split column problems.
Can MySQL split a column?
Mysql string split
I.e. Specifically based on this answer
DROP FUNCTION IF EXISTS STRSPLIT;
DELIMITER $$
CREATE FUNCTION STRSPLIT($Str VARCHAR(20000), $delim VARCHAR(12), $pos INTEGER)
RETURNS VARCHAR(20000)
BEGIN
DECLARE output VARCHAR(20000);
SET output = REPLACE(SUBSTRING(SUBSTRING_INDEX($Str, $delim, $pos)
, LENGTH(SUBSTRING_INDEX($Str, $delim, $pos - 1)) + 1)
, $delim
, '');
IF output = ''
THEN SET output = null;
END IF;
RETURN output;
END $$
You can iterate through the values like so
DROP PROCEDURE IF EXISTS GetNameValues $$
CREATE PROCEDURE GetNameValues()
BEGIN
DECLARE i INTEGER;
DROP TEMPORARY TABLE IF EXISTS TempList;
CREATE TEMPORARY TABLE TempList(
`Name` VARCHAR(256) COLLATE utf8_unicode_ci NOT NULL,
`ValueList` VARCHAR(20000) COLLATE utf8_unicode_ci NOT NULL
);
DROP TEMPORARY TABLE IF EXISTS Result;
CREATE TEMPORARY TABLE Result(
`Name` VARCHAR(256) COLLATE utf8_unicode_ci NOT NULL,
`Value` VARCHAR(128) COLLATE utf8_unicode_ci NOT NULL
);
INSERT INTO
TempList
SELECT
I.`Name`,
ExtractValue(Xml,'/Xml/Values/Value') AS ValueList
FROM
Item AS I
WHERE
I.TypeID = 'A'
;
SET i = 1;
REPEAT
INSERT INTO
Result
SELECT
`Name`,
CAST(STRSPLIT(ValueList, ' ', i) AS CHAR(128)) AS Value
FROM
TempList
WHERE
CAST(STRSPLIT(ValueList, ' ', i) AS CHAR(128)) IS NOT NULL
;
SET i = i + 1;
UNTIL ROW_COUNT() = 0
END REPEAT;
SELECT * FROM Result ORDER BY `Name`;
END $$
DELIMITER ;
CALL GetNameValues();
Hope this helps someone one day.

Need help in converting INSERT\UPDATE to MERGE

Is this a good candidate for a MERGE command?
Must the source data also be another table or can it be variables that are passed?
If it must be a table, is inserting the passed variables into a temp table sane?
Could you help me with the syntax?
CREATE PROCEDURE [dbo].[usp_ConvertToMerge]
#GL_DT date
,#SRC_SYS_ID varchar(60)
,#MLR_SRC_SYS_CD char(3)
,#TRSRY_FEED_DT date
,#Data varchar(20)
AS
BEGIN
IF NOT EXISTS (SELECT
#GL_DT
FROM
MLR_REBATE_IBOR_INFO_2
WHERE
[GL_DT] = #GL_DT
AND [SRC_SYS_ID] = #SRC_SYS_ID
AND [MLR_SRC_SYS_CD] = #MLR_SRC_SYS_CD
AND [TRSRY_FEED_DT] = #TRSRY_FEED_DT)
BEGIN
INSERT INTO [dbo].[MLR_REBATE_IBOR_INFO_2]
([GL_DT],
[SRC_SYS_ID],
[MLR_SRC_SYS_CD],
[TRSRY_FEED_DT],
[Data])
SELECT
#GL_DT
,#SRC_SYS_ID
,#MLR_SRC_SYS_CD
,#TRSRY_FEED_DT
,#Data
END
ELSE
BEGIN
UPDATE [dbo].[MLR_REBATE_IBOR_INFO_2]
SET [Data] = #Data
WHERE [GL_DT] = #GL_DT
AND [SRC_SYS_ID] = #SRC_SYS_ID
AND [MLR_SRC_SYS_CD] = #MLR_SRC_SYS_CD
AND [TRSRY_FEED_DT] = #TRSRY_FEED_DT
END
END
GO
I think I did it:
CREATE PROCEDURE MyMergeTest
#GL_DT date
,#SRC_SYS_ID char(20)
,#MLR_SRC_SYS_CD char(3)
,#TRSRY_FEED_DT date
,#Data varchar(20)
AS
BEGIN
MERGE MLR_REBATE_IBOR_INFO_2 AS target
USING
(
SELECT
#GL_DT
,#SRC_SYS_ID
,#MLR_SRC_SYS_CD
,#TRSRY_FEED_DT
,#Data
) AS source
(
GL_DT
,SRC_SYS_ID
,MLR_SRC_SYS_CD
,TRSRY_FEED_DT
,Data
)
ON (
target.GL_DT = source.GL_DT AND
target.SRC_SYS_ID = source.SRC_SYS_ID AND
target.MLR_SRC_SYS_CD = source.MLR_SRC_SYS_CD AND
target.TRSRY_FEED_DT = source.TRSRY_FEED_DT
)
WHEN MATCHED THEN
UPDATE SET Data = source.Data
WHEN NOT MATCHED THEN
INSERT
(
[GL_DT],
[SRC_SYS_ID],
[MLR_SRC_SYS_CD],
[TRSRY_FEED_DT],
[Data]
)
VALUES
(
[GL_DT], --<<it looks like these can eiether be the variable eg, #GL_DT, or prefixed by 'source.'
[SRC_SYS_ID],
[MLR_SRC_SYS_CD],
[TRSRY_FEED_DT],
[Data]
);
END