My task is to write a stored procedure that will first validate the data from a temporary table and then insert the data into the main table.
For this I am planning to iterate over each row of the temp table, validate it using some other stored procedure or user defined function and then insert the data into the main table.
My problem is how to iterate over the rows of temp table without using CURSORS because they are very slow and memory consuming. I want to use some looping structure instead of CURSOR.
Of course if any one has any other algorithm for the above problem there suggestions are welcome.
PS: I am using MYSQL DB
Without the use of Cursor, you could iterate using a temporary table and a While..Do statement.
Let's say you have two tables
CREATE TABLE `user` (
`id` int(11) NOT NULL auto_increment,
`name` varchar(45) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=MyISAM;
And
CREATE TABLE `tmp_user` (
`id` int(11) NOT NULL auto_increment,
`name` varchar(45) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=MyISAM;
Create the following routine, and adjust the validation process:
DELIMITER $$
USE `routines_sample`$$
CREATE PROCEDURE `nocursor`()
BEGIN
Declare validId int;
Declare validName varchar(45);
-- drop the temporary table if exists
Drop table if exists `routines_sample`.tmp_validation;
-- create the temporary table, here you could use Engine=Memory
Create table `routines_sample`.tmp_validation (`id` int not null, `name` varchar(45) not null, `valid` bit(1) not null) Engine=MyISAM;
-- insert into the temporary table with a valid flag = 0 (false)
Insert into `routines_sample`.tmp_validation (`id`, `name`, `valid`)
Select tu.id, tu.name, 0
From `routines_sample`.tmp_user tu;
-- while exists data to validate on temporary table do something
While exists(Select `id` From `tmp_validation` Where `valid` = 0) Do
Select `id`, `name` Into #validId, #validName From tmp_validation Where `valid` = 0 Limit 1;
-- do your validation
Select #validId, #validName;
-- don't forget to update your validation table, otherwise you get an endless loop
Update `tmp_validation`
Set `valid` = 1
Where `id` = #validId;
END WHILE;
-- insert only valid rows to your destination table
Insert into `routines_sample`.`user` (`name`)
Select `name` From `tmp_validation`
Where `valid` = 1;
-- drop the temporary table
DROP TABLE tmp_validation;
END$$
DELIMITER ;
Related
I have two similar tables:
CREATE TABLE `t1` (
`id` int(11) NOT NULL AUTO_INCREMENT ,
`c1` int(11) NOT NULL DEFAULT '0',
`c2` int(11) NOT NULL DEFAULT '0',
PRIMARY KEY (`id`),
KEY `idx_c1` (`c1`)
) ENGINE=InnoDB;
CREATE TABLE `t2` (
`id` int(11) NOT NULL AUTO_INCREMENT ,
`c1` int(11) NOT NULL DEFAULT '0',
`c2` int(11) NOT NULL DEFAULT '0',
PRIMARY KEY (`id`),
KEY `idx_c1` (`c1`)
) ENGINE=InnoDB;
I want to fill both tables with random values:
drop procedure if exists random_records;
truncate table t1;
truncate table t2;
delimiter $$
create procedure random_records(n int)
begin
set #i=1;
set #m=100000;
while #i <= n do
insert into t1(c1,c2) values(rand()*#m,rand()*#m);
insert into t2(c1,c2) values(rand()*#m,rand()*#m);
set #i=#i+1;
end while;
end $$
delimiter ;
call random_records(100);
select * from t1 limit 10;
select * from t2 limit 10;
select count(*) from t1;
select count(*) from t2;
Here is what i see in table t1:
I don't understand why there is a lot of '0' and'1'
Function count() returns 210 for t1 and 208 for t2 - one mystery more
The most likely reason for the presence of many zeros and ones in the c1 and c2 columns of both tables is that the rand() function is returning very small numbers. This is because the #m variable, which is used to scale the random numbers generated by rand(), is set to a relatively low value of 100,000.
As a result, the random numbers generated are mostly between 0 and 0.00001, which is why you are seeing many zeros and ones in the tables. To fix this, you can increase the value of #m to a higher number, such as 1,000,000 or even 10,000,000, to generate larger random numbers.
As for the discrepancy in the number of rows in the two tables, it is likely due to the fact that the insert statements in the random_records procedure are not being executed atomically.
This means that there is a chance that one of the insert statements could fail, resulting in fewer rows being inserted into one of the tables. To fix this, you can wrap the insert statements in a transaction to ensure that they are executed as a single unit of work.
For example, you can modify the random_records procedure as follows:
drop procedure if exists random_records;
truncate table t1;
truncate table t2;
delimiter $$
create procedure random_records(n int)
begin
set #i=1;
set #m=1000000;
start transaction;
while #i <= n do
insert into t1(c1,c2) values(rand()*#m,rand()*#m);
insert into t2(c1,c2) values(rand()*#m,rand()*#m);
set #i=#i+1;
end while;
commit;
end $$
delimiter ;
This should ensure that the insert statements are executed atomically and that the number of rows in both tables is consistent.
I have some SQL Server schema changes that I'm trying to convert to MySQL. I know about CREATE TABLE IF NOT EXISTS in MySQL. I don't think I can use that here.
What I want to do is create a table in MySQL, with an index, and then insert some values all as part of the "if not exists" predicate. This was what I came up with, though it doesn't seem to be working:
SET #actionRowCount = 0;
SELECT COUNT(*) INTO #actionRowCount
FROM information_schema.tables
WHERE table_name = 'Action'
LIMIT 1;
IF #actionRowCount = 0 THEN
CREATE TABLE Action
(
ActionNbr INT AUTO_INCREMENT,
Description NVARCHAR(256) NOT NULL,
CONSTRAINT PK_Action PRIMARY KEY(ActionNbr)
);
CREATE INDEX IX_Action_Description
ON Action(Description);
INSERT INTO Action
(Description)
VALUES
('Activate'),
('Deactivate'),
('Specified');
END IF
I can run it once, and it'll create the table, index, and values. If I run it a second time, I get an error: Table Action already exists. I would have thought that it wouldn't run at all if the table already exists.
I use this pattern a lot when bootstrapping a schema. How can I do this in MySQL?
In mysql compound statements can only be used within stored programs, which includes the if statement as well.
Therefore, one solution is to include your code within a stored procedure.
The other solution is to use the create table if not exists ... with the separate index creation included within the table definition and using insert ignore or insert ... select ... to avoidd inserting duplicate values.
Examples of options:
Option 1:
CREATE TABLE IF NOT EXISTS `Action` (
`ActionNbr` INT NOT NULL AUTO_INCREMENT PRIMARY KEY,
`Description` VARCHAR(255) NOT NULL,
INDEX `IX_Action_Description` (`Description`)
) SELECT 'Activate' `Description`
UNION
SELECT 'Deactivate'
UNION
SELECT 'Specified';
Option 2:
DROP PROCEDURE IF EXISTS `sp_create_table_Action`;
DELIMITER //
CREATE PROCEDURE `sp_create_table_Action`()
BEGIN
IF NOT EXISTS(SELECT NULL
FROM `information_schema`.`TABLES` `ist`
WHERE `ist`.`table_schema` = DATABASE() AND
`ist`.`table_name` = 'Action') THEN
CREATE TABLE `Action` (
`ActionNbr` INT AUTO_INCREMENT,
`Description` NVARCHAR(255) NOT NULL,
CONSTRAINT `PK_Action` PRIMARY KEY (`ActionNbr`)
);
CREATE INDEX `IX_Action_Description`
ON `Action` (`Description`);
INSERT INTO `Action`
(`Description`)
VALUES
('Activate'),
('Deactivate'),
('Specified');
END IF;
END//
DELIMITER ;
CALL `sp_create_table_Action`;
I am trying to insert data to Second server's DB table after inserted to First server' table like below structure,
Second server :
use SecondServerDB;
CREATE TABLE user (
`ID` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(50) NOT NULL,
`age` smallint(6) NOT NULL,
PRIMARY KEY (`ID`)
) ENGINE=InnoDB;
After created table in Second sever, I have created table in First server like below,
First server :
use FirstServerDB;
CREATE TABLE remote_user2 (
`ID` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(50) NOT NULL,
`age` smallint(6) NOT NULL,
PRIMARY KEY (`ID`)
) ENGINE=FEDERATED
DEFAULT CHARSET=latin1
CONNECTION='mysql://MysqlUserName:MysqlUserPassword#SecondServer.MyDomain.com:3306/SecondServerDB/user';
After created FEDERATED table in second server, Created triggers in same server like below,
DELIMITER $$
CREATE TRIGGER remote_insert_Testing AFTER INSERT ON remote_user2 FOR EACH ROW
BEGIN
INSERT INTO user (ID,name, age) VALUES (NEW.ID,NEW.name, NEW.Age);
END $$
DELIMITER ;
If i run insert query like below,
INSERT INTO remote_user2 (ID,name, age) VALUES (1,'Name',27);
Then it shows, Error Code: 1146. Table 'FirstServerDB.user' doesn't exist.
where am i doing mistake?
Note : I googled and found out What i am trying is possible. How to create trigger to insert data to a database on another server for possible
Try using the fully qualified table names in your trigger:
DELIMITER $$
CREATE TRIGGER remote_insert_Testing
AFTER INSERT ON FirstServerDB.remote_user2 FOR EACH ROW
BEGIN
INSERT INTO SecondServerDB.user (ID,name, age)
VALUES (NEW.ID, NEW.name, NEW.Age);
END $$
DELIMITER ;
If you don't mind moving the data through a normal query, you can try the following:
INSERT INTO SecondServerDB.user (ID, name, age)
SELECT ID, name, age
FROM FirstServerDB.remote_user2
Hi I am trying to create a trigger in Joomla component installation, However it seems doesn't work
error msg
You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '' at line 4 SQL=CREATE TRIGGER tg_jws_worksheet_worksheet_insert BEFORE INSERT ON jws_worksheet_worksheet FOR EACH ROW BEGIN INSERT INTO jws_worksheet_worksheet_seq (id) VALUES (NULL);
here is the code
--
-- Table structure for table `client`
--
DROP TABLE IF EXISTS `#__worksheet_client`;
CREATE TABLE IF NOT EXISTS `#__worksheet_client` (
`ClientID` int(11) NOT NULL,
`CompanyId` int(11) NOT NULL,
`FullName` text NOT NULL,
`Email` text,
`ContactNo` text NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
-- --------------------------------------------------------
--
-- Table structure for table `company`
--
DROP TABLE IF EXISTS `#__worksheet_company`;
CREATE TABLE IF NOT EXISTS `#__worksheet_company` (
`CompanyId` int(5) NOT NULL,
`ABN` int(12) NOT NULL,
`CompanyName` text NOT NULL,
`Street` text,
`City` text,
`State` text,
`PostCode` int(5) DEFAULT NULL,
`Country` text,
`Phone` text,
`HourRate` int(11) NOT NULL,
`Notes` text NOT NULL
) ENGINE=InnoDB AUTO_INCREMENT=2 DEFAULT CHARSET=latin1;
-- --------------------------------------------------------
--
-- Table structure for table `worksheet_seq`
--
DROP TABLE IF EXISTS `#__worksheet_worksheet_seq`;
CREATE TABLE IF NOT EXISTS `#__worksheet_worksheet_seq` (
`id` int(11) NOT NULL
) ENGINE=InnoDB AUTO_INCREMENT=4 DEFAULT CHARSET=latin1;
-- --------------------------------------------------------
--
-- Table structure for table `worksheet`
--
DROP TABLE IF EXISTS `#__worksheet_worksheet`;
CREATE TABLE IF NOT EXISTS `#__worksheet_worksheet` (
`worksheetNo` varchar(9) NOT NULL,
`CompanyId` int(5) NOT NULL,
`UserId` int(5) NOT NULL,
`CompanyName` text NOT NULL,
`Category` varchar(20) NOT NULL,
`JobTitle` text NOT NULL,
`JobDesc` text NOT NULL,
`StartDate` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`EndDate` timestamp NOT NULL DEFAULT '0000-00-00 00:00:00',
`TimeSpd` int(11) NOT NULL,
`Tech` text NOT NULL,
`Status` varchar(8) NOT NULL,
`Hardware` text,
`Price` int(11) NOT NULL,
`file` text
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
SET FOREIGN_KEY_CHECKS=1;
--
-- Indexes for dumped tables
--
--
-- Indexes for table `client`
--
ALTER TABLE `#__worksheet_client`
ADD PRIMARY KEY (`ClientID`), ADD KEY `CompanyId` (`CompanyId`);
--
-- Indexes for table `company`
--
ALTER TABLE `#__worksheet_company`
ADD PRIMARY KEY (`CompanyId`);
--
-- Indexes for table `worksheet`
--
ALTER TABLE `#__worksheet_worksheet`
ADD PRIMARY KEY (`worksheetNo`), ADD KEY `CompanyId` (`CompanyId`);
--
-- Indexes for table `worksheet_seq`
--
ALTER TABLE `#__worksheet_worksheet_seq`
ADD PRIMARY KEY (`id`);
--
-- AUTO_INCREMENT for dumped tables
--
--
-- AUTO_INCREMENT for table `client`
--
ALTER TABLE `#__worksheet_client`
MODIFY `ClientID` int(11) NOT NULL AUTO_INCREMENT;
--
-- AUTO_INCREMENT for table `company`
--
ALTER TABLE `#__worksheet_company`
MODIFY `CompanyId` int(5) NOT NULL AUTO_INCREMENT,AUTO_INCREMENT=2;
--
-- AUTO_INCREMENT for table `worksheet_seq`
--
ALTER TABLE `#__worksheet_worksheet_seq`
MODIFY `id` int(11) NOT NULL AUTO_INCREMENT,AUTO_INCREMENT=4;
--
-- Constraints for dumped tables
--
--
-- Constraints for table `client`
--
ALTER TABLE `#__worksheet_client`
ADD CONSTRAINT `client_ibfk_1` FOREIGN KEY (`CompanyId`) REFERENCES `#__worksheet_company` (`CompanyId`);
--
-- Constraints for table `worksheet`
--
ALTER TABLE `#__worksheet_worksheet`
ADD CONSTRAINT `worksheet_ibfk_1` FOREIGN KEY (`CompanyId`) REFERENCES `#__worksheet_company` (`CompanyId`);
--
-- Triggers `worksheet`
--
DROP TRIGGER IF EXISTS `tg_#__worksheet_worksheet_insert`;
CREATE TRIGGER `tg_#__worksheet_worksheet_insert` BEFORE INSERT ON `#__worksheet_worksheet`
FOR EACH ROW
BEGIN
INSERT INTO `#__worksheet_worksheet_seq` (`id`) VALUES (NULL);
SET NEW.worksheetNo = CONCAT('WS-', LPAD(LAST_INSERT_ID(), 5, '0'));
END
create trigger usually requires redefining the delimiter:
DELIMITER $$
DROP TRIGGER IF EXISTS `tg_#__worksheet_worksheet_insert`$$
CREATE TRIGGER `tg_#__worksheet_worksheet_insert` BEFORE INSERT ON `#__worksheet_worksheet`
FOR EACH ROW
BEGIN
INSERT INTO `#__worksheet_worksheet_seq` (`id`) VALUES (NULL);
SET NEW.worksheetNo = CONCAT('WS-', LPAD(LAST_INSERT_ID(), 5, '0'));
END$$
DELIMITER ;
I had the same issue with Joomla 2.5, unable to create a trigger during the install or update procedure while the sql statements work fine.
You will not be able to create a trigger, function, procedure or any sql statement containing more than 1 semi-colon (;). That's why all other statements seem to work in your script.
I tried many appoaches which did not work including these:
In the manifest file use sql-file:
<install>
<sql>
<file driver="mysql" charset="utf8">sql/install.mysql.utf8.sql</file>
</sql>
</install>
In the manifest file use install-queries:
<install>
<queries>
<query id="1">
CREATE TRIGGER `...` AFTER UPDATE ON `#__...` FOR EACH ROW
BEGIN
...codeline...;
...codeline...;
END
</query>
<query id="2">
...
</query>
</queries>
</install>
(sorry for using blockquotes in stead of code sample, otherwise post will not be shown correctly)
The only way it did work was via the installscript option available since Joomla 2.5, see https://docs.joomla.org/J2.5:Developing_a_MVC_Component/Adding_an_install-uninstall-update_script_file.
This did not work out-of-the-box. I had to remove the access check (the first line with define) and put the following code in the install($parent) and update($parent) functions to add the trigger:
$db = JFactory::getDbo();
$db->setQuery("DROP TRIGGER IF EXISTS `...`");
$db->query();
$query="CREATE TRIGGER `...` AFTER UPDATE ON `#__...` FOR EACH ROW
BEGIN
...codeline...;
...codeline...;
END";
$db->setQuery($query);
$db->query();
if ($error = $db->getErrorMsg()) {
$this->setError($error);
return false;
}
...next sqlstatement or code...
return true;
Make sure the classname in this installer php file matches your component name including init caps, something like:
class com_YourComponentNameInstallerScript
And also, don't forget to open the file with <?php tag but do not put the ?> end tag at the end.
Save the file as php file in the root of your install package at the same level as the manifest file, and add this filename to the manifest file at somewhere between the component description and administration or site tags:
<scriptfile>yourcomponentname_install.php</scriptfile>
This method will also work with creating Stored Procedures and Functions.
I'm trying to create a temporary table and select results from it, inside a mysql procedure.
Here's a piece of the SQL
DELIMITER $$
CREATE PROCEDURE `mydb`.`simulate_results` (currdate DATETIME, enddate DATETIME, idcam INT)
BEGIN
DECLARE MYVAR1,
MYVAR2,
MYVAR3,
IDX INT;
CREATE TEMPORARY TABLE IF NOT EXISTS tmp (
`Id` INT NOT NULL AUTO_INCREMENT,
`Field1` INT NOT NULL,
`Field2` INT NOT NULL,
`Field3` INT NOT NULL, PRIMARY KEY (`Id`)) ENGINE=MEMORY;
INSERT INTO tmp (`Field1`,`Field2`,`Field3`) VALUES (0,0,0);
INSERT INTO tmp (`Field1`,`Field2`,`Field3`) VALUES (1,0,0);
SELECT Field1,Field2,Field3,Id INTO MYVAR1,MYVAR2,MYVAR3,IDXFROM tmp ORDER BY RAND() LIMIT 1;
SELECT MYVAR1; (...)
The only variable that comes filled from the SELECT INTO statemente is IDX (primary key). The others are always NULL.
Is there something wrong with this code?
Have you looked at your temporary table to see what kind of data is lingering in it? The temporary table will still be there after the procedure completes and if you run the procedure again from the same session, you'll be writing new rows into the existing temporary table.
I would add DROP TEMPORARY TABLE IF EXISTS tmp before CREATE TEMPORARY TABLE ... to be sure you don't have old data hanging around.