How to catch empty input value within MySQL statement? - mysql

In MySQL I need to find multiple IDs within a table, but in some cases the search ID is missing. There is no way around this and I can not put this into programming application logic as it is a grafana dashboard filter. In case no filter it will provide no value at all for the variable.
Minimal example:
SELECT *
FROM tbl
where
-- try to catch empty value
case
when "$filter_ids" then
id in ($filer_ids)
-- id in ('1','2')
-- find_in_set(id, 1,2)
end
AND other_id = 4
-- Possible values for $filter_ids:
-- ''
-- '1'
-- '1','18'
-- Alternativ cases, also possible:
-- empty for no value
-- 1
-- 1,18
I tried both, find_in_set and IN. But both result in an MySQL error in case there is no value (no filter set).
How could I catch this in MySQL?


You can add an empty string to IN or FIND_IN_SET, if you have no ids. that would nozt produce an error
CREATE tABLE TEST(id int)
SELECT * FROM TEST WHERE id in ("") OR FIND_IN_SET(id,'')
| id |
| -: |
db<>fiddle here

Related

mysql query message save into table or external file

I have multiple store procedures to do the ETL work in mysql. Normally, it is running on the server for over night.
inside the store procedures there are multiple update statement like
update table1 set column1=3 when column2 = 4
if there any way, I can keep the mysql workbench result like
Rows matched: 100 Changed: 50 Warnings: 0
for each statement I run either into mysql table or external file?
prefer mysql native method. if not, any python I could possible use?

"Rows changed" can be retrieved with ROW_COUNT() function.
"Rows matched" needs in a trick with user-defined variable usage.
CREATE TABLE test (id INT, val INT);
INSERT INTO test VALUES
(1,1), (1,2), (1,3), (2,4);
Now we want to perform UPDATE test SET val = 3 WHERE id = 1; and count the amounts.
UPDATE test
-- add user-defined variable for matched rows counting
CROSS JOIN ( SELECT #matched := 0 ) init_variable
-- increment matched rows counter (this expression is always TRUE)
SET val = CASE WHEN #matched := #matched + 1
-- update the column
THEN 3
END
WHERE id = 1;
SELECT #matched matched, ROW_COUNT() changed;
matched | changed
------: | ------:
3 | 2
db<>fiddle here
If more than one column should be updated in a query then only one expression must be accompanied with the counter increment.

How to JSON extract from dynamic key value pair in MySQL?

I have the user table with user_id and user_details. it contains the JSON data in string format as shown below:
[{"name":"question-1","value":"sachin","label":"Enter your name?"},
{"name":"question-2","value":"abc#example.com","label":"Enter your email?"},
{"name":"question-3","value":"xyz","label":"Enter your city?"}]
I have tried with the json_extract but it return the result if json has data as shown below:
{"name":"question-1","value":"sachin","label":"Enter your name?"}
then it return the result as,
Name | Label
question-1 | Enter your name?
Expected Result :
I want to extract all name and label from json in sql query.
Example-1:
Consider that we have the following data in user_details column,
[{"name":"question-1","value":"sachin","label":"Enter your name?"},
{"name":"question-2","value":"abc#example.com","label":"Enter your email?"},
{"name":"question-3","value":"xyz","label":"Enter your city?"}]
then the sql query should return the result in following format ,
Name | Label
question-1 | Enter your name?
question-2 | Enter your email?
question-3 | Enter your city?
How to get this using JSON_EXTRACT in MySQL?

I assume that you are not using a table.
SET #data = '[{"name":"question-1","value":"sachin","label":"Enter your name?"},
{"name":"question-2","value":"abc#example.com","label":"Enter your email?"},
{"name":"question-3","value":"xyz","label":"Enter your city?"}]';
SELECT JSON_EXTRACT(#data,'$[*].name') AS "name", JSON_EXTRACT(#data,'$[*].label') AS "label";
it will return
name | label
["question-1", "question-2", "question-3"] | ["Enter your name?", "Enter your email?", "Enter your city?"]
SQL should be like below according to your table and column name:
SELECT JSON_EXTRACT(user_details,'$[*].name') AS "name", JSON_EXTRACT(user_details,'$[*].label') AS "label" FROM user;
you can match them by using some loops for arrays. I do not know if this is the best way but it satisfy my needs.

Another answer given by How to extract rows from a json array using the mysql udf json_extract 0.4.0? is to parse yourself the JSON with common_schema. Pretty tricky if you are not used to complex SQL.
You could create an own aggregated table as proposed in topic List all array elements of a MySQL JSON field if you know how many elements will be given by the field but I guess this is not your case.
However, it seems better, as mentioned in both answers, not to store such json lists in your SQL database. Maybe could you make a related table containing one line per each dictionary and then link it to your main table with a foreign key.

I was working in a report where there was a big json array list in one column. I modified the datamodel to store the relationship 1 to * instead of storing everything in one single column. For doing this process, I had to use a while in a stored procedure since I do not know the maximum size:
DROP PROCEDURE IF EXISTS `test`;
DELIMITER #
CREATE PROCEDURE `test`()
PROC_MAIN:BEGIN
DECLARE numNotes int;
DECLARE c int;
DECLARE pos varchar(10);
SET c = 0;
SET numNotes = (SELECT
ROUND (
(
LENGTH(debtor_master_notes)
- LENGTH( REPLACE ( debtor_master_notes, "Id", "") )
) / LENGTH("Id")
) AS countt FROM debtor_master
order by countt desc Limit 1);
DROP TEMPORARY TABLE IF EXISTS debtorTable;
CREATE TEMPORARY TABLE debtorTable(debtor_master_id int(11), json longtext, note int);
WHILE(c <numNotes) DO
SET pos = CONCAT('$[', c, ']');
INSERT INTO debtorTable(debtor_master_id, json, note)
SELECT debtor_master_id, JSON_EXTRACT(debtor_master_notes, pos), c+1
FROM debtor_master
WHERE debtor_master_notes IS NOT NULL AND debtor_master_notes like '%[%' AND JSON_EXTRACT(debtor_master_notes, pos) IS NOT NULL AND JSON_EXTRACT(debtor_master_notes, pos) IS NOT NULL;
SET c = c + 1;
END WHILE;
SELECT * FROM debtorTable;
END proc_main #
DELIMITER ;

You don't use JSON_EXTRACT(). You use JSON_TABLE():
mysql> create table mytable ( id serial primary key, data json);
Query OK, 0 rows affected (0.01 sec)
mysql> insert into mytable set data = '[{"name":"question-1","value":"sachin","label":"Enter your name?"},
'> {"name":"question-2","value":"abc#example.com","label":"Enter your email?"},
'> {"name":"question-3","value":"xyz","label":"Enter your city?"}]';
Query OK, 1 row affected (0.00 sec)
mysql> SELECT j.* FROM mytable,
JSON_TABLE(data, '$[*]' COLUMNS (
name VARCHAR(20) PATH '$.name',
label VARCHAR(50) PATH '$.label'
)) AS j;
+------------+-------------------+
| name | label |
+------------+-------------------+
| question-1 | Enter your name? |
| question-2 | Enter your email? |
| question-3 | Enter your city? |
+------------+-------------------+
JSON_TABLE() requires MySQL 8.0.4 or later. If you aren't running at least that version, you will have to upgrade.
Honestly, if you need to access the individual fields, it's less work to store your data in normal columns, and avoid using JSON.

Iterate through a table and replace values in another table using a stored procedure or function

Hopelessly stuck at the following and up until now none of my programming speed dial buddies has been able to help out (most of them not MySQL experts):
I have different tables where the column names and datatypes are auto generated from the 'import table data wizard' using a CSV file, and the table does not contain an AUTO INCREMENT column (yet). This particular table consists of approx: 30.000 rows It starts at row=id(1) from a table that looks like this:
I am trying to correct values in one column that are comma delimited using one 'corrections' table. And to do this I am writing a stored procedure containing a WHILE loop to interate through the corrections table row for row, and check wheter or not an Alias is found in the table that was imported.
| id | material | alias01 | alias02 | alias03 | *up to 12
1 Katoen Cotton Supima Pima
2 Polyester Polyster
3 Lyocell Lycocell Lyocel
4 Linnen Linen
5 Viscose Visose Viskose Viscoe Voscose
6 Scheerwol
7 Polyamide
8 Nylon
9 Leer Leder Lamsleder Varkensleder
10 Polyurethaan Polyurethan PU Polyuretaan
For testing purposes to test any kind of results i am only using alias01 for now ( it needs to check alias01, then 02 etc... but i'll try to solve that at a later time).
It needs to compare `Length' ( alias_string_length = found_string_length) to make sure that a string that consist of 'wo' is not found in 'wool' or 'wol'.
The values from the column that need corrections look like this (the comma's dont need to be there it's just what i was given to work with):
| material |
,Katoen,Elastaan,Voering,Acetaat,Polyester
,Nylon,Polyester,Elastaan
,Katoen
,Leder,in,Leder,Loopzool,Leder
,Polyester
,Polyester,Elastaan,Voering,Polyester
Update
Thanks to Drew's tip i changed the procedure. I added a tmp table that holds materials AND a unique id for each row, and iterate through each one with the alias01. It takes around 11 seconds to do 9000 rows but 0 row(s) affected,. Any tips on increasing speed are most welcome, but insight in what might be the issue would help alot more.
CREATE DEFINER=`root`#`localhost` PROCEDURE `replace_materials`()
BEGIN
set #rownumber = 1;
set #totalrows = 28;
set #um ='';
set #cm ='';
set #corrected ='';
set #correctme ='';
TRUNCATE TABLE tmp;
INSERT INTO tmp (material) SELECT material FROM vantilburgonline.productinfo;
WHILE (#rownumber < #totalrows) DO
SET #um = (SELECT alias01 FROM vantilburgonline.materials WHERE id=#rownumber);
-- gives 'um' value from column alias01, from table materials, row(X)
SET #cm = (SELECT material FROM vantilburgonline.materials WHERE id=#rownumber);
-- gives 'cm' value from column material, from table materials, row(X)
set #tmprow = 1;
set #totaltmprow =9000;
WHILE (#tmprow < #totaltmprow) DO
SET #correctme = (SELECT material FROM vantilburgonline.tmp WHERE id = #tmprow);
-- gives the value from column material from table tmp to correctme(X).
SET #correctme = REPLACE(#correctme,#um,#cm);
-- should run through column material from table productinfo and replace 'alias01' with correct 'material'.
SET #tmprow = #tmprow +1;
END WHILE;
SET #rownumber = #rownumber +1;
END WHILE;
END
though i'm certain alias01 contains strings it should've found in the materials. Also Workbench was using 9GB at this point and i was only able to counter that by restarting..

I would recommend an alteration from your materials table which is unwieldy with multiple columns (alias01 .. alias12). A transition to a normalized, extensible system. It would have a materials table and a materials_alias table. As it sits alongside your current table that you created, I named them with a 2.
Schema
drop table if exists materials2;
create table materials2
( material varchar(100) primary key, -- let's go with a natural key
active bool not null -- turn it LIVE and ON for string replacement of alias back to material name
-- so active is TRUE for ones to do replacement, or FALSE for skip
-- facilitates your testing of your synonyms, translations, slangs, etc
)engine=INNODB;
insert materials2 (material,active) values
('KARTON',true),
('Polyester',false),
('Lyocell',false),
('Linnen',true),
('Viscose',true),
('Scheerwol',false),
('Nylon',false),
('Leer',true),
('Polyurethaan',true),
('Polyacryl',true),
('Acryl',false),
('Modal',true),
('Acetaat',true),
('Papier',false),
('Wol',true),
('Zijde',true),
('Temcal',false),
('Polyamide',true),
('Wol-Merino',true),
('Elastan',true),
('Elastomultiester',true);
-- 21 rows
-- a few rows were skipped. The intent of them read as gibberish to me. Please review.
-- we need to restructure the materials2_alias table (after the first attempt)
-- 1. it might need special handling when `alias` is a legitimate substring of `material` (those 2 columns)
-- 2. it needs a unique composite index
drop table if exists materials2_alias;
create table materials2_alias
( id int auto_increment primary key,
material varchar(100) not null,
alias varchar(100) not null,
ais bool not null, -- Alias is Substring (alias is a legitimate substring of material, like Wo and Wol, respectively)
unique key(material,alias), -- Composite Index, do not allow dupe combos (only 1 row per combo)
foreign key `m2alias_m2` (material) references materials2(material)
)engine=INNODB;
insert materials2_alias (material,alias,ais) values
('KARTON','Cotton',false),('KARTON','Katoen',false),('KARTON','Pima',false),
('Polyester','Polyster',false),
('Lyocell','Lycocell',false),('Lyocell','Lyocel',false),
('Linnen','Linen',false),
('Viscose','Visose',false),('Viscose','Viskose',false),('Viscose','Viscoe',false),('Viscose','Voscose',false),
('Leer','Leder',false),('Leer','Lamsleder',false),('Leer','Varkensleder',false),('Leer','Schapenleder',false),('Leer','Geitenleder',false),
('Polyurethaan','Polyurethan',false),('Polyurethaan','PU',false),('Polyurethaan','Polyuretaan',false),('Polyurethaan','Polyurathane',false),('Polyurethaan','Polyurtaan',false),('Polyurethaan','Polyueretaan',false),
('Polyacryl','Polyacrylic',false),
('Acetaat','Leder',false),('Acetaat','Lamsleder',false),
('Wol','Schuurwol',false),('Wol','Wool',false),('Wol','WO',false),('Wol','Scheerwol',false),
('Zijde','Silk',false),('Zijde','Sede',false),
('Polyamide','Polyamie',false),('Polyamide','Polyamid',false),('Polyamide','Poliamide',false),
('Wol-Merino','Merino',false),
('Elastan','Elastaan',false),('Elastan','Spandex',false),('Elastan','Elataan',false),('Elastan','Elastane',false),
('Elastomultiester','elastomutltiester',false),('Elastomultiester','Elasomultiester',false);
-- this cleans up the above, where false should have been true
update materials2_alias
set ais=true
where instr(material,alias)>0;
-- 4 rows
There are several alter table statements and other things. I will try to document them or link to them. I am merely trying to capture something to share considering it is several hundred lines of code from you. But mine comes down to a simple chunk of code you would put in a loop.
The Update put in a loop:
UPDATE productinfo pi
join materials2_alias ma
on instr( pi.material, concat(',',ma.alias,',') )>0
join materials2 m
on m.material=ma.material and m.active=true
set pi.material=replace(lower(pi.material),lower(ma.alias),lower(ma.material)),
pi.touchCount=pi.touchCount+1;
A few notes on the update:
-- Note, pi.material starts and ends with a comma.
-- I forced that during the ETL. But `ma.alias` does not contain commas.
-- So add the commas with a concat() within the "Update with a Join" pattern shown
--
-- Note that the commas solved the problem with the Wol - Wo
Well, the following 4 in particular.
select * from materials2_alias
where ais=true
order by material,alias;
+----+------------+----------+-----+
| id | material | alias | ais |
+----+------------+----------+-----+
| 6 | Lyocell | Lyocel | 1 |
| 33 | Polyamide | Polyamid | 1 |
| 28 | Wol | WO | 1 |
| 35 | Wol-Merino | Merino | 1 |
+----+------------+----------+-----+
-- instr() is not case sensitive except for binary strings
-- REPLACE(str,from_str,to_str); -- case sensitive
-- http://dev.mysql.com/doc/refman/5.7/en/string-functions.html#function_replace
--
-- so the update uses lower() or this won't work due to replace() case sensitivity
--
Stored Procedure:
DROP PROCEDURE if exists touchCounts;
DELIMITER $$
CREATE PROCEDURE touchCounts()
BEGIN
select touchCount,count(*) as rowCount
from productinfo
group by touchCount
order by touchCount;
END $$
DELIMITER ;
When that stored procedure returns the same count of rows on a successive call (the next call), you are done modifying the material column via the update.
That stored procedure could naturally return an out parameter for the rowcount. But it is late and time to sleep.
For your last data set from your side, the update statement would need to be called 4 times. That is like 13 seconds on my mediocre laptop. The idea is naturally flexible, for hundreds of aliases per material if you want.
I parked it up on github as it is too much otherwise.

PostgreSQL: SET equivalent to MySQL

I want to insert the id of one table into another in PostgreSQL. First I want to select the id of table1 where it equals a specific value and store that into a variable. In MySQL I can do:
SET #last_id_in = (SELECT id FROM table1 WHERE col1 = 1);
Then I can insert where the value would equal #last_id_in. How can I do something similar in PostgreSQL?

Depending on where you want your variable -- i.e. in a PL/pgSQL function, or in a script from which you're calling into Postgres...
PL/pgSQL, something like this:
INSERT INTO table2
SELECT id FROM table
WHERE col1 = 1
RETURNING id INTO last_id_in;
where last_id_in is previously declared as integer (or whatever type id is).
Script:
INSERT INTO table2
SELECT id FROM table1
WHERE col1 = 1
RETURNING id;
And then the cursor in whatever library you're interacting with Postgres with should contain that value for you to use elsewhere.
If the goal is simply to insert an element from one table into another, then the script version above can leave off the RETURNING clause, as there's no intervening steps.

Copy values from one column to another in the same table

How can I make a copy values from one column to another?
I have:
Database name: list
-------------------
number | test
-------------------
123456 | somedata
123486 | somedata1
232344 | 34
I want to have:
Database name: list
----------------
number | test
----------------
123456 | 123456
123486 | 123486
232344 | 232344
What MySQL query should I have?

Short answer for the code in question is:
UPDATE `table` SET test=number
Here table is the table name and it's surrounded by grave accent (aka back-ticks `) as this is MySQL convention to escape keywords (and TABLE is a keyword in that case).
BEWARE!
This is pretty dangerous query which will wipe everything in column test in every row of your table replacing it by the number (regardless of it's value)
It is more common to use WHERE clause to limit your query to only specific set of rows:
UPDATE `products` SET `in_stock` = true WHERE `supplier_id` = 10

UPDATE `table_name` SET `test` = `number`
You can also do any mathematical changes in the process or use MySQL functions to modify the values.

try this:
update `list`
set `test` = `number`

BEWARE : Order of update columns is critical
GOOD: What I want saves existing Value of Status to PrevStatus
UPDATE Collections SET PrevStatus=Status, Status=44 WHERE ID=1487496;
BAD: Status & PrevStatus both end up as 44
UPDATE Collections SET Status=44, PrevStatus=Status WHERE ID=1487496;

try following:
UPDATE `list` SET `test` = `number`
If list is table name and test and number are columns
it creates copy of all values from "number" and paste it to "test"

Following worked for me..
Ensure you are not using Safe-mode in your query editor application. If you are, disable it!
Then run following sql command
for a table say, 'test_update_cmd', source value column col2, target
value column col1 and condition column col3: -
UPDATE test_update_cmd SET col1=col2 WHERE col3='value';
Good Luck!

IF Anyone wants to put Condition
UPDATE bohf SET Sq=IDNo WHERE Table = 'AOF' AND FormSq BETWEEN 13 AND 17

update `table`
set `firstcolumn` = `secondcolumn`