MySQL database cleanse using stored function, procedure - mysql

I have a table like this
CREATE TABLE `test` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`tel_1` text,
`tel_2` text,
`tel_3` text,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
+----+------------------------------------+-----------------------------------------------+-------+
| id | tel_1 | tel_2 | tel_3 |
+----+------------------------------------+-----------------------------------------------+-------+
| 1 | 123-412-3455 | 1276ー364739−181 | NULL |
| 2 | 714-212-3839 Not Using | No Info | NULL |
| 3 | 12+13E | NULL | NULL |
| 4 | 0123ー3432-1233 Ext : 602 | NULL | NULL |
+----+------------------------------------+-----------------------------------------------+-------+
And I'd like to cleanse the data something like this below:
+----+--------------+------------+---------------+------------+-------+------------+
| id | tel_1 | tel_1_desc | tel_2 | tel_2_desc | tel_3 | tel_3_desc |
+----+--------------+------------+---------------+------------+-------+------------+
| 1 | 1234123455 | NULL | 1276364739181 | NULL | NULL | NULL |
| 2 | 7142123839 | Not Using | NULL | No Info | NULL | NULL |
| 3 | 12+13E | NULL | NULL | NULL | NULL | NULL |
| 4 | 012334321233 | Ext : 602 | NULL | NULL | NULL | NULL |
+----+--------------+------------+---------------+------------+-------+------------+
Here are the list what I need :
remove character from tel_X, mostly '-'
leave extra comment on tel_X_desc
make not UTF-8 number to correct format, from '0123' to
'0123'
I managed to MySQL create stored function for #3, however I cannot LOOP this for each records and update......
CREATE FUNCTION `multibyte2cv`(`str` TEXT) RETURNS text CHARSET utf8
BEGIN
DECLARE int_len INT(2);
DECLARE int_z VARCHAR(10) DEFAULT '1234567890';
DECLARE int_h VARCHAR(10) DEFAULT '1234567890';
SET int_len = CHAR_LENGTH(int_z);
WHILE int_len > 0 DO
SET str = REPLACE(str, SUBSTRING(int_z,int_len,1), SUBSTRING(int_h,int_len,1));
SET int_len = int_len - 1;
END WHILE;
RETURN str;
END;
I am pretty new to database field and trying to find something that I can solve this issue....

Call the function in an UPDATE query:
UPDATE test
SET tel_1 = multibyte2cv(tel_1), tel_2 = multibyte2cv(tel_2), tel_3 = multibyte2cv(tel_3)

Related

Creating a scheduled event in MySQL

I have a table names offer inside a MySQL database named fouras :
mysql> desc offer;
+------------------------+---------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+------------------------+---------------+------+-----+---------+----------------+
| id | bigint(20) | NO | PRI | NULL | auto_increment |
| description | varchar(5000) | NO | | NULL | |
| end_date | date | NO | | NULL | |
| is_accepted | bit(1) | NO | | NULL | |
| is_active | bit(1) | NO | | NULL | |
| is_draft | bit(1) | NO | | NULL | |
| is_processed | bit(1) | NO | | NULL | |
| is_removed | bit(1) | NO | | NULL | |
| max_reservation_number | int(11) | NO | | NULL | |
| promotion_first_param | varchar(255) | YES | | NULL | |
| promotion_product | varchar(255) | YES | | NULL | |
| promotion_second_param | varchar(255) | YES | | NULL | |
| promotion_type | varchar(255) | NO | | NULL | |
| publish_date | date | YES | | NULL | |
| remove_time_stamp | bigint(20) | YES | | NULL | |
| start_date | date | NO | | NULL | |
| title | varchar(255) | NO | | NULL | |
| views_number | int(11) | YES | | NULL | |
| local_business | bigint(20) | YES | MUL | NULL | |
+------------------------+---------------+------+-----+---------+----------------+
19 rows in set (0.00 sec)
Now, i want to periodically check if an offer has expired (end_date > today), for that i'm trying to use a MySQL scheduled event :
CREATE EVENT IF NOT EXISTS check_expired_offers
ON SCHEDULE EVERY 10 MINUTE
DO
BEGIN
DECLARE id INT;
DECLARE end_date DATE;
DECLARE offer_cursor CURSOR FOR SELECT id, end_date FROM fouras.offer;
OPEN offer_sursor;
offer_loop: LOOP
FETCH offer_cursor into id, end_date;
IF end_date < NOW() THEN
UPDATE fouras.offer set is_active = false;
END IF;
END LOOP
END;
But MySQL throws an error when i try to add this event:
Error Code: 1064. You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '' at line 5 0,127 sec
The error was about the delimiter, this tutorial helped me a lot
==========================
Found delimiter issue.
==========================
Here's the modified event:
DELIMITER $$
CREATE EVENT IF NOT EXISTS check_expired_offers
ON SCHEDULE EVERY 10 MINUTE
DO
BEGIN
DECLARE finished INTEGER DEFAULT 0;
DECLARE id INT;
DECLARE end_date DATE;
DECLARE offer_cursor CURSOR FOR SELECT id, end_date FROM fouras.offer;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET finished = 1;
OPEN offer_cursor;
offer_loop:
LOOP
FETCH offer_cursor into id, end_date;
IF finished = 1 THEN
LEAVE offer_loop;
END IF;
IF end_date < NOW() THEN
UPDATE fouras.offer set is_active = false;
END IF;
END LOOP ;
END $$
DELIMITER ;
Learn delimiters in MySQL
Note: I've also used a variable there named finished.
Where finished is a variable to indicate that the cursor has reached the end of the result set. Notice that the handler declaration must appear after variable and cursor declaration inside the stored programs.
The following diagram illustrates how MySQL cursor works.

How to update the huge table column?

I have a table with 9918751 records in it, please find below is data structure,
+--------------------+--------------+------+-----+-------------------+----------------+
| Field | Type | Null | Key | Default | Extra |
+--------------------+--------------+------+-----+-------------------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| start_ip | varchar(32) | YES | UNI | NULL | |
| end_ip | varchar(32) | YES | UNI | NULL | |
| country | varchar(255) | YES | | NULL | |
| region | varchar(255) | YES | | NULL | |
| city | varchar(255) | YES | | NULL | |
| country_conf | varchar(255) | YES | | NULL | |
| region_conf | varchar(255) | YES | | NULL | |
| city_conf | varchar(255) | YES | | NULL | |
| country_code | int(11) | YES | | NULL | |
| region_code | int(11) | YES | | NULL | |
| city_code | int(11) | YES | | NULL | |
| two_letter_country | varchar(20) | YES | | NULL | |
| creation_datetime | timestamp | NO | | CURRENT_TIMESTAMP | |
+--------------------+--------------+------+-----+-------------------+----------------+
In my table the start_ip and end_ip column contains valid IPv4 IP Address(like 192.168.1.1, 12.23.34.22 etc), Now I want to update all rows(9918751) to convert the IP address to integer using INET_ATON() function. When I am running below query
update geo_location_info set start_ip = INET_ATON(start_ip);
This query was taking too much time as to update the 9918751 rows and it was not completing the update. Please let me know any alternative (store procedure ?)
The basic idea is to use a loop for the update. The problem is that the function call takes a while and queries can time out. In addition, updating almost 10 million rows puts a certain burden on the logging mechanisms for maintaining data integrity. Assuming the id is the populated incrementally, you might do this in groups of 10,000 or maybe 100,000. Unfortunately, you can't really use #Stewarts idea, but almost half of all the rows are likely to start with 1.
The basic loop is:
set #len := 10000, #i := 1;
while #i * #len < 10000000 do
update geo_location_info
set start_ip = INET_ATON(start_ip)
where id between #len*#i and #len*(#i + 1) - 1;
set #i := #i + 1;
end while;
Unfortunately, in MySQL, you need to put this into a stored procedure -- because control flow mechanisms (such as while) are only allowed in stored programs. So, something more like:
delimiter $$
create procedure do_update ()
begin
set #len := 10000, #i := 1;
select #max := max(id) from geo_location_info;
while #i * #len <= #max do
update geo_location_info
set start_ip = INET_ATON(start_ip)
where id between #len*#i and #len*(#i + 1) - 1;
set #i := #i + 1;
end while;
end $$
delimiter ;

mysql after insert trigger which updates another table's column

i'm trying to write a trigger, I have following tables:
BookingRequest:
+-----------+---------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+---------+------+-----+---------+----------------+
| idRequest | int(11) | NO | PRI | NULL | auto_increment |
| roomClass | int(11) | NO | | NULL | |
| inDate | date | NO | | NULL | |
| outDate | date | NO | | NULL | |
| numOfBeds | int(11) | NO | | NULL | |
| status | int(11) | NO | MUL | NULL | |
| idUser | int(11) | NO | MUL | NULL | |
+-----------+---------+------+-----+---------+----------------+
status table:
+------------+--------------------------------------------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+------------+--------------------------------------------------+------+-----+---------+-------+
| idStatus | int(11) | NO | PRI | NULL | |
| nameStatus | enum('underConsideration','approved','rejected') | YES | | NULL | |
+------------+--------------------------------------------------+------+-----+---------+-------+
OccupiedRoom:
+--------------+---------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+--------------+---------+------+-----+---------+----------------+
| idOccupation | int(11) | NO | PRI | NULL | auto_increment |
| idRoom | int(11) | NO | | NULL | |
| idRequest | int(11) | NO | | NULL | |
+--------------+---------+------+-----+---------+----------------+
i need a trigger which will change status in BookingReques to 1 if request with the same id is inserted into OccupiedRoom table, so i tried something like this
create trigger occupy_trig after insert on OccupiedRoom
for each row
begin
if BookingRequest.idRequest= NEW.idRequest
then
update BookingRequest
set status = '1';
where idRequest = NEW.idRequest;
end if;
END;
and it doesn't work, so any suggestions would be very appriciated
Try this:
DELIMITER $$
CREATE TRIGGER occupy_trig
AFTER INSERT ON `OccupiedRoom` FOR EACH ROW
begin
DECLARE id_exists Boolean;
-- Check BookingRequest table
SELECT 1
INTO #id_exists
FROM BookingRequest
WHERE BookingRequest.idRequest= NEW.idRequest;
IF #id_exists = 1
THEN
UPDATE BookingRequest
SET status = '1'
WHERE idRequest = NEW.idRequest;
END IF;
END;
$$
DELIMITER ;
With your requirements you don't need BEGIN END and IF with unnecessary SELECT in your trigger. So you can simplify it to this
CREATE TRIGGER occupy_trig AFTER INSERT ON occupiedroom
FOR EACH ROW
UPDATE BookingRequest
SET status = 1
WHERE idRequest = NEW.idRequest;
Maybe remove the semi-colon after set because now the where statement doesn't belong to the update statement. Also the idRequest could be a problem, better write BookingRequest.idRequest
DELIMITER //
CREATE TRIGGER contacts_after_insert
AFTER INSERT
ON contacts FOR EACH ROW
BEGIN
DECLARE vUser varchar(50);
-- Find username of person performing the INSERT into table
SELECT USER() INTO vUser;
-- Insert record into audit table
INSERT INTO contacts_audit
( contact_id,
deleted_date,
deleted_by)
VALUES
( NEW.contact_id,
SYSDATE(),
vUser );
END; //
DELIMITER ;

combining two columns in to one using multiple line triggers in mysql

The following is my table schema:
+--------------+--------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+--------------+--------------+------+-----+---------+-------+
| LicenceID | int(11) | NO | PRI | NULL | |
| PassingRTO | varchar(4) | NO | | NULL | |
| DLNO | int(15) | YES | | NULL | |
| DateOfIssue | date | NO | | NULL | |
| DateOfExpiry | date | NO | | NULL | |
| COV | varchar(6) | NO | | NULL | |
| DateOfBirth | date | NO | | NULL | |
| BloodGroup | varchar(3) | YES | | NULL | |
| FullName | varchar(50) | NO | | NULL | |
| FathersName | varchar(50) | YES | | NULL | |
| Address | varchar(150) | NO | | NULL | |
| PinCode | int(6) | NO | | NULL | |
| IssuingAuth | int(7) | NO | | NULL | |
| IDIA | int(11) | YES | | NULL | |
| Valid | tinyint(4) | NO | | NULL | |
+--------------+--------------+------+-----+---------+-------+
What i want to do is When i insert a new row, i want my DLNO as PassingRTO+LicenceID and IDIA as PassingRTO+IssuingAuth.
I tried the same using -
create trigger insert_combined
after insert on LicencesDB
for each row
BEGIN
set new.IDIA = concat(new.PassingRTO, new.IssuingAuth);
set new.DLNO = concat(new.PassingRTO,new.LicenceID);
END;
but gives me an error as -
ERROR 1362 (HY000): Updating of NEW row is not allowed in after
trigger ERROR 1193 (HY000): Unknown system variable 'DLNO' ERROR 1064
(42000): You have an error in your SQL syntax; check the manual that
corresponds to your MySQL server version for the right syntax to use
near 'END' at line 1
Now I have two questions can we do this using multiple lines in triggers? and can't we combine two columns at the time of creating the table itself? like col1 = col2 + col3?
Thanks in advance!
don;t forget to change the DELIMITER
DELIMITER $$
create trigger insert_combined
after insert on LicencesDB
for each row
BEGIN
set new.IDIA = concat(new.PassingRTO, new.IssuingAuth);
set new.DLNO = concat(new.PassingRTO,new.LicenceID);
END $$
DELIMITER ;
Please check this link on mysql triggers. When you want to make an update on the current row being entered you can't use the after clause.
check http://dev.mysql.com/doc/refman/5.1/en/create-trigger.html
When you want to use
SET NEW.col_name = value
in your trigger, please note that you CANNOT use this with the AFTER the action, and must use it BEFORE the action.
Therefore, this will work:
CREATE TRIGGER sdata_insert BEFORE INSERT ON `sometable`
FOR EACH ROW
BEGIN
SET NEW.guid = UUID();
END
;
And this will NOT work:
CREATE TRIGGER sdata_insert AFTER INSERT ON `sometable`
FOR EACH ROW
BEGIN
SET NEW.guid = UUID();
END
;
Also you can probably do it in the code itself before even entering into the database.

MySQL cursor not fetching inside procedure

I have the following tables
albums:
+----------------+-------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+----------------+-------------+------+-----+---------+----------------+
| album_id | int(11) | NO | PRI | NULL | auto_increment |
| band_id | int(11) | YES | MUL | NULL | |
| release_date | varchar(45) | YES | | NULL | |
| name | varchar(45) | YES | | NULL | |
| format | varchar(45) | YES | | NULL | |
| music_genre_id | int(11) | YES | MUL | NULL | |
| label_id | int(11) | YES | MUL | NULL | |
| avg_rating | float | YES | | NULL | |
+----------------+-------------+------+-----+---------+----------------+
and music_ratings
+-----------------+---------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------------+---------+------+-----+---------+----------------+
| music_rating_id | int(11) | NO | PRI | NULL | auto_increment |
| user_id | int(11) | YES | MUL | NULL | |
| album_id | int(11) | YES | MUL | NULL | |
| rating | int(11) | YES | | NULL | |
+-----------------+---------+------+-----+---------+----------------+
After every insert into the *music_rating* I want to update the average rating in the albums table. I have a trigger for this, which calls a procedure. The thing is, the procedure does not work, for some reason the cursor is not fetching data from the table. (I called the procedure separately to make sure it isn't the trigger acting up. The tables have a couple of rows already, so it's not that.)
My procedure is pretty straight forward and looks like this
DELIMITER $$
CREATE PROCEDURE avg_album_calc(IN id_album INT)
BEGIN
DECLARE done INT DEFAULT 0;
DECLARE rating INT;
DECLARE cur CURSOR FOR SELECT `rating` FROM `music_ratings` WHERE `album_id`=id_album;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = 1;
OPEN cur;
SET #ct=0;
SET #sm=0;
REPEAT
FETCH cur INTO rating;
IF NOT done
THEN
SET #ct = #ct +1;
SET #sm = #sm + rating;
END IF;
UNTIL done END REPEAT;
UPDATE albums SET avg_rating = #sm/#ct WHERE album_id = id_album;
CLOSE cur;
END$$
DELIMITER ;
I echoed the result of the cursor with a SELECT rating after the FETCH cur INTO rating; command, and it shows up as null.
You do not need to calculate and store avg_rating in the albums table. You can calculate in on the fly -
SELECT a.album_id, a.name, AVG(mr.rating) FROM albums a
LEFT JOIN music_ratings mr
ON a.album_id = mr.album_id
GROUP BY a.album_id