Dynamic SQL Procedure within function - mysql

I am trying to create a stored procedure that among other things, updates a table with information from other table:
UPDATE table1 T1, table2 T2
set T1.rank = T1.rank + T2.rank
T1.tags = merge(T1.tags, T2.tags)
WHERE T1.id = T2.id
tags is a string of comma separated words and merge is a function (not a procedure) that breaks the strings into temporary tables and returns the a new string with unique tags using prepared statements. I know can't use prepared statements inside functions but the part that uses those is inside a procedure that is called inside the function.
So I get an error starting that I can't use dynamic sql procedure inside a function
and I need the function so I can use the return value as the new value in the update.
Any Possible way to achive this?

You can do that (an update with merging tags) in one statement with pure SQL. The trick is in using a number aka tally table, which can be created and populated in seconds (also with one SQL statement).
Here is your UPDATE statement
UPDATE table1 t1 JOIN table2 t2
ON t1.id = t2.id JOIN
(
SELECT id, GROUP_CONCAT(DISTINCT tag ORDER BY tag) tags
FROM
(
SELECT a.id, SUBSTRING_INDEX(SUBSTRING_INDEX(tags, ',', n.id), ',', -1) tag
FROM
(
SELECT t1.id, CONCAT(t1.tags, ',', t2.tags) tags
FROM table1 t1 JOIN table2 t2
ON t1.id = t2.id
) a CROSS JOIN tally n
WHERE n.id <= 1 + (LENGTH(tags) - LENGTH(REPLACE(tags, ',', '')))
AND n.id < 100 -- change that number to accommodate max possible number of tags
) b
GROUP BY id
) c ON t1.id = c.id
SET t1.tags = c.tags,
t1.rank = t1.rank + t2.rank
Here is schema and statement that populates tally table with up to 100k rows.
CREATE TABLE tally (id INT NOT NULL AUTO_INCREMENT PRIMARY KEY);
INSERT INTO tally
SELECT NULL
FROM (SELECT 0 n UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) a
, (SELECT 0 n UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) b
, (SELECT 0 n UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) c
, (SELECT 0 n UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) d
, (SELECT 0 n UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) e
;
Here is SQLFiddle demo

Related

Mysql Flatten an array

The table contains json data and i wanted to extract labelKey from this json
table: d_json
data
{"tag":null,"options":[{"labelKey":"key10","value":"yes","selected":true},{"labelKey":"key11","value":"no","selected":false}]}
{"tag":null,"options":[{"labelKey":"key20","value":"yes","selected":true},{"labelKey":"key21","value":"no","selected":false},{"labelKey":"key22","value":"no","selected":false}]}
I used following query to extract "labelKey"
SELECT JSON_EXTRACT(JSON_EXTRACT(j.data,'$.options'),'$[*].labelKey') FROM d_json j AS result;
It returns following result
result
["key10", "key12"]
["key20", "key21", "key22"]
However i want result in flat, each row contains one element instead of array, such as
result
"key10"
"key11"
"key21"
"key22"
"key23"
Not getting any clue how to flatten resulted arrays
On mysql v8+, you can do that using JSON_TABLE function like this:
SELECT p.*
FROM d_json,
JSON_TABLE(data, '$.options[*]' COLUMNS (
labelKey VARCHAR(40) PATH '$.labelKey')
) p;
Result:
labelKey
key10
key11
key20
key21
key22
Here's a demo fiddle
Edit:
On older MySQL version, try this:
SELECT TRIM(SUBSTRING_INDEX(SUBSTRING_INDEX(val2,',',rn),',',-1))
FROM (SELECT 1 rn UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5) AS r
CROSS JOIN
(SELECT REPLACE(REPLACE(GROUP_CONCAT(val),'[',''),']','') AS val2
FROM
(SELECT JSON_EXTRACT(JSON_EXTRACT(j.data,'$.options'),'$[*].labelKey') AS val
FROM d_json j) v1
) v2;
Demo fiddle
The idea is to CROSS JOIN with a sequence of row numbers then use that same sequence to extract from a GROUP_CONCAT values using SUBSTRING_INDEX. In the query example above, I'm using a hardcoded row sequences in the form of:
(SELECT 1 rn UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5) AS r
Ideally, the best approach is to figure out the sequence required and generate it dynamically.
Update:
Generating numbering sequences on older MySQL version is a challenge especially if we're aiming for a dynamic generating. There's a method that is not dynamic but can generate a large numbering sequence from a very long query but if you're planning to be using this sequences for a long time, I suggest you just create a table for it:
CREATE TABLE number_seq (
sequences INT);
INSERT INTO number_seq
SELECT #row := #row + 1 AS rn FROM
(SELECT 0 UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION
SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9) t1 CROSS JOIN
(SELECT 0 UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION
SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9) t2 CROSS JOIN
(SELECT 0 UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION
SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9) t3 CROSS JOIN
(SELECT #row:=0) numbers;
The query above will generate a range of numbers from 1-1000 and insert into a table. Once you have that table, you just need to write your query like this:
SELECT TRIM(SUBSTRING_INDEX(SUBSTRING_INDEX(val2,',',sequences),',',-1))
FROM (SELECT sequences FROM
(SELECT (LENGTH(val2)-LENGTH(REPLACE(val2,',','')))+1 AS valLen FROM
(SELECT REPLACE(REPLACE(GROUP_CONCAT(val),'[',''),']','') AS val2 FROM
(SELECT JSON_EXTRACT(JSON_EXTRACT(j.data,'$.options'),'$[*].labelKey') AS val
FROM d_json j) v1
) v2 ) v3 JOIN number_seq t ON sequences <= valLen) r
CROSS JOIN
(SELECT REPLACE(REPLACE(GROUP_CONCAT(val),'[',''),']','') AS val2
FROM
(SELECT JSON_EXTRACT(JSON_EXTRACT(j.data,'$.options'),'$[*].labelKey') AS val
FROM d_json j) v1
) v2;
The highlight of change from the previous query is the switch between the hardcoded numbering sequences with a query that basically get the total values separated by comma in the final JSON_EXTRACT and join it against the created number_seq table to get the rows needed. This part here:
SELECT sequences FROM
(SELECT (LENGTH(val2)-LENGTH(REPLACE(val2,',','')))+1 AS valLen FROM
(SELECT REPLACE(REPLACE(GROUP_CONCAT(val),'[',''),']','') AS val2 FROM
(SELECT JSON_EXTRACT(JSON_EXTRACT(j.data,'$.options'),'$[*].labelKey') AS val
FROM d_json j) v1
) v2 ) v3 JOIN number_seq t ON sequences <= valLen
Here's an updated fiddle for reference https://dbfiddle.uk/?rdbms=mysql_5.7&fiddle=ace8babce8d7bbb97f7e016a754e93a9

SQL Subtraction one by one

I was thinking of subtracting digits one by one but didn't find a way to implement it after a big effort.
Row 1: 100211210
Row 2: 100010220
Result: 000201010
And the result has to be non-negative.
Select SUBSTR(t1.row,1,1)-(t2.row,1, 1)|| SUBSTR(t1.row,2,2)-SUBSTR(t2.row,2, 2)||... so on from table t1 where t1.row NOT IN (Select row in table t2);
This will check in the same table if the row exists then it will skip if not subtract digit by digit or you can use loop in pl/sql by declaring values for substr as i,j for both and then subtracting.
select GROUP_CONCAT(CAST(ABS(substring('123456782',c.count,1)-substring('323456789',c.count,1)) AS CHAR) separator '')
from (select c1.1*10+c2.1 count from (select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9 union all select 0) c1,
(select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9 union all select 0) c2 order by count) c
where c.count>0 and c.count<=length('123456782')
2 string is the same length, and the last parameter is the lenght of the strings

mysql find numbers in query that are NOT in table

Is there a simple way to compare a list of numbers in my query to a column in a table to return the ones that are NOT in the db?
I have a comma separated list of numbers (1,57, 888, 99, 76, 490, etc etc) that I need to compare to the number column in a table in my DB. SOME of those numbers are in the table, some are not. I need the query to return those that are in my comma separated list, but are NOT in the DB...
I would put the list of numbers to be checked in a table of their own, then use WHERE NOT EXISTS to check whether they exist in the table to be queried. See this SQLFiddle demo for an example of how this might be accomplished:
If you're comfortable with this syntax, you can even avoid putting into a temp table:
SELECT * FROM (
SELECT 1 AS mycolumn
UNION
SELECT 2
UNION
SELECT 3
UNION
SELECT 4
UNION
SELECT 5
UNION
SELECT 6
UNION
SELECT 7
) a
WHERE NOT EXISTS ( SELECT 1 FROM mytable b
WHERE b.mycolumn = a.mycolumn )
UPDATE per comments from OP
If you can insert your very long list of numbers into a table, then query as follows to get the numbers that are not found in the other table:
SELECT mynumber
FROM mytableof37000numbers a
WHERE NOT EXISTS ( SELECT 1 FROM myothertable b
WHERE b.othernumber = a.mynumber)
Alternately
SELECT mynumber
FROM mytableof37000numbers a
WHERE a.mynumber NOT IN ( SELECT b.othernumber FROM myothertable b )
Hope this helps.
May be this is what you are looking for.
Convert your CSV to rows using SUBSTRING_INDEX. Use NOT IN operator to find the values which is not present in DB
Then Convert the result back to CSV using Group_Concat.
select group_concat(value) from(
SELECT SUBSTRING_INDEX(SUBSTRING_INDEX(t.a, ',', n.n), ',', -1) value
FROM csv t CROSS JOIN
(
SELECT a.N + b.N * 10 + 1 n
FROM
(SELECT 0 AS N UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) a
,(SELECT 0 AS N UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) b
ORDER BY n
) n
WHERE n.n <= 1 + (LENGTH(t.a) - LENGTH(REPLACE(t.a, ',', '')))) ou
where value not in (select a from db)
SQLFIDDLE DEMO
CSV TO ROWS referred from this ANSWER
You could use the 'IN' clause of MySQL. Maybe check this out IN clause tutorial

Give out IDs which are present in the list and not present in table

i have a list of ID's like (1,2,3,4,5,6). I want to find out how many of these id's in list does not exists in the table. how do i do this ?
-- table --
id name
1 a
2 b
3 c
4 d
5 e
6 f
7 g
List of Id's which i get is 1,2,3,4,5,6,7,8,9,10,11,12,13
so when query i should get out result like 8,9,10,11,12,13 which are not in the table
This will return the list of IDs in the list that aren't in the table.
SELECT t1.id
FROM (SELECT 1 id
UNION
SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5 UNION SELECT 6) t1
LEFT JOIN MyTable t2
ON t1.id = t2.id
WHERE t2.id IS NULL
If you do things like this often, you might want to create a permanent table containing constants, e.g.
CREATE TABLE numbers (id int primary key);
INSERT INTO numbers (id) VALUES (0), (1), (2), ..., (1000);
Then you can do:
SELECT t1.id
FROM numbers
JOIN MyTable t2
ON t1.id = t2.id
WHERE t1.id IN (1, 2, 3, 4, 5, 6)
AND t2.id IS NULL
Try this:
SELECT
6-c
FROM (
SELECT
COUNT(DISTINCT(ID)) as c
FROM
my_table
WHERE
ID IN (1,2,3,4,5,6)
) as tmp
And to find out which ones are missing :
SELECT
zID
FROM (
SELECT 1 as zID UNION
SELECT 2 as zID UNION
SELECT 3 as zID UNION
SELECT 4 as zID UNION
SELECT 5 as zID UNION
SELECT 6 as zID UNION
SELECT 7 as zID UNION
SELECT 8 as zID UNION
SELECT 9 as zID UNION
SELECT 10 as zID UNION
SELECT 11 as zID UNION
SELECT 12 as zID UNION
SELECT 13 as zID UNION
SELECT 14 as zID UNION
SELECT 15 as zID UNION
) as t1
LEFT JOIN (
SELECT
DISTINCT(ID) as mID
FROM
my_table
) as t2
ON t1.zID = t2.mID
WHERE
t2.mID IS NULL

Mysql explode function

I have a table witch has a field with the following record:
1,2,3,4,5,6
I would like to ask the following two things:
1) How can i make a foreign key in another table? The rule would be:
For any value seperated by comma in field `field_name` must be record of other_table.field_id
2) How can i do something like: SELECT explode(field) AS ex FROM table_name ?
the name's of row maybe can retrieve as ex[0]-->1, ex[1]-->2
While it is possible to do a join on a comma separated field (using FIND_IN_SET for example), I don't think there is a way to do this for a foreign key.
MySQL doesn't have an explode function, and your idea would seem to suggest a varying number of columns on each row.
You can split them onto different rows if necessary but it is ugly. And more a good reason to NOT use comma separated fields
SELECT DISTINCT SUBSTRING_INDEX(SUBSTRING_INDEX(field, ',', 1 + units.i + tens.i * 10 + hundreds.i * 100), ',', -1)
FROM table_name
CROSS JOIN (SELECT 0 AS i UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9) units
CROSS JOIN (SELECT 0 AS i UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9) tens
CROSS JOIN (SELECT 0 AS i UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9) hundreds