I am trying to create a procedure which will fill up the table until certain amount of elements.
At the current moment I have
CREATE PROCEDURE PopulateTable(
IN dbName tinytext,
IN tableName tinytext,
IN amount INT)
BEGIN
DECLARE current_amount INT DEFAULT 0;
SET current_amount = SELECT COUNT(*) FROM dbName,'.',tableName;
WHILE current_amount <= amount DO
set #dll=CONCAT('INSERT INTO `',dbName,'`.`',tableName,'`(',
'`',tableName,'_name`) ',
'VALUES(\'',tableName,'_',current_amount,'\')');
prepare stmt from #ddl;
execute stmt;
SET current_amount = current_amount + 1;
END WHILE;
END;
So what I am trying to do is, once user calls the procedure, the function will check and see how many current elements exist, and fill up the remaining elements.
First problem I have is that I do not know how to count the elements, so my SELECT COUNT(*) FROM dbName,'.',tableName; does not work.
I also want to a suggestion since I am kind of new to databases if what I am doing is correct or if there is a better way to do this?
Also just if this is of any help the table I am trying to do this to only has 2 fields, one of them being id, which is auto incremented and is primary and the other being profile_name which I am populating.
Thanks to anyone for their help!
Firstly, I think you'll have a delimiter problem if you try to execute the code you pasted. The procedure declaration delimiter has to be different from the one you use into the procedure code (here ';'). You will have to use DELIMITER statement;
Your procedure belongs to a schema; I'm not sure you can query tables from other shemas (especially without USE statement);
This is not a good idea if your database contains tables whitch are not supposed to be populated through your procedure;
If you have a limited number of concerned tables, I think it will be a better idea to define one procedure for each table. In this way, table name will be explicit in each procedure code and it will avoid the use of prepared statements;
Be careful about you 'amount' parameter : are you sure that your server can handle requests if I pass the maximum value for INT as amount ?
I think you should use '<' instead of '<=' in your WHILE condition
If you want to insert a large number of lines, you'll obtain better performances performing "grouped" inserts, or generating a temporary table (for example with MEMORY engine) containing all your lines and performing a unique INSERT selecting your temporary table's content.
Related
Is there such an opportunity? Let's say, we have a range of integers. We loop over them and find out that they don't exceed 65535. So, now we know that SMALLINT would be a suitable type and are going to create a table with a column of the appropriate type! How can we do this?
I've tried various combinations like:
SET #ty := "INT";
CREATE TABLE tab (
id #ty
);
but they all have failed and ended up complaining on #ty.
So, the other question is whether we can keep in a server script #variable (as INT or as "INT")?
Thank you very much for your time!
By loop over them and find out that they don't exceed 65535 ,if you mean the highest values does not exceed 65535, then we can use max() function to get the highest value. If you mean the number of values within the range, the count() function should be used instead. Next we can perform a condition check to determine which numeric type should be used. The rest is to create the wanted table based on the result from the condition check using PREPARED STATEMENT. To illustrate , we create a table and insert values to simulate the number range. Then use a procedure to create the intended table dynamically.
create table numbers (n int);
insert numbers values(1),(2),(70000);
delimiter //
drop procedure if exists create_table //
create procedure create_table()
begin
declare num_range int;
select max(n) from numbers into num_range; -- supposing you mean the highest values does not exceed 65535
If num_range<65536 then
set #create_tb_stmt=concat('create table `tab` (id smallint);');
else
set #create_tb_stmt=concat('create table `tab` (id int);');
end if;
drop table if exists `tab` ;
PREPARE stmt FROM #create_tb_stmt;
EXECUTE stmt;
end//
call create_table // -- call the procedure to make it happen
In order to document SQL code in a more linear fashion, I wanted to make appear the description of a table that is going to be created before the creation statement. So my first idea was to put that in a user-defined variable with which I could fill the comment instruction, but it seems to be unsupported (at least in the 5.6 version I have to deal with):
set #description = 'The following table is a useless dummy test.';
drop table if exists `test`;
create table `test` (dummy int) comment #test; -- syntax error
show table status where name='test';
Is there an other way to achieve the initial goal? Of course it's always possible to use -- SQL comments before the creation statement, but then it wouldn't appear in the recorded database structure, or at the price of a awful duplication.
We might be able to do this using dynamic SQL:
SET #description = 'The following table is a useless dummy test.';
SET #sql = CONCAT('CREATE TABLE test (dummy int) COMMENT=''', #description, '''');
PREPARE stmt FROM #sql;
EXECUTE stmt;
Note that each prepared statement is precisely just that; a single statement. So, to do this from the command line, you might need 3 statements to cover the full logic you want to run.
Although it doesn't match the question, here is a solution to its underlying problem: having the description of a table directly next to it's creation statement.
The MySQL syntax require to put the comment attached to the table at the end, but if the table have many fields, each declared on one or several lines, then the table comment can quickly be sent of sight of its declaration start.
Now, one can easily create a table with it's comment as sole attached statement, then alter it to include fields it should contains.
CREATE TALBE `test` COMMENT 'Informative description';
ALTER TABLE test
ADD id
INT UNSIGNED NOT NULL AUTO_INCREMENT
COMMENT 'Description of the id field',
ADD data
BLOB
COMMENT 'Description of the data field',
;
I am trying to update a MySQL InnoDB table with c. 100 million rows. The query takes close to an hour, which is not a problem.
However, I'd like to split this update into smaller chunks in order not to block table access. This update does not have to be an isolated transaction.
At the same time, the splitting of the update should not be too expensive in terms of additional overhead.
I considered looping through the table in a procedure using :
UPDATE TABLENAME SET NEWVAR=<expression> LIMIT batchsize, offset,
But UPDATE does not have an offset option in MySQL.
I understand I could try to UPDATE ranges of data that are SELECTed on a key, together with the LIMIT option, but that seems rather complicated for that simple task.
I ended up with the procedure listed below. It works but I am not sure whether it is efficient with all the queries to identify consecutive ranges. It can be called with the following arguments (example):
call chunkUpdate('SET var=0','someTable','theKey',500000);
Basically, the first argument is the update command (e.g. something like "set x = ..."), followed by the mysql table name, followed by a numeric (integer) key that has to be unique, followed by the size of the chunks to be processed. The key should have an index for reasonable performance. The "n" variable and the "select" statements in the code below can be removed and are only for debugging.
delimiter //
CREATE PROCEDURE chunkUpdate (IN cmd VARCHAR(255), IN tab VARCHAR(255), IN ky VARCHAR(255),IN sz INT)
BEGIN
SET #sqlgetmin = CONCAT("SELECT MIN(",ky,")-1 INTO #minkey FROM ",tab);
SET #sqlgetmax = CONCAT("SELECT MAX(",ky,") INTO #maxkey FROM ( SELECT ",ky," FROM ",tab," WHERE ",ky,">#minkey ORDER BY ",ky," LIMIT ",sz,") AS TMP");
SET #sqlstatement = CONCAT("UPDATE ",tab," ",cmd," WHERE ",ky,">#minkey AND ",ky,"<=#maxkey");
SET #n=1;
PREPARE getmin from #sqlgetmin;
PREPARE getmax from #sqlgetmax;
PREPARE statement from #sqlstatement;
EXECUTE getmin;
REPEAT
EXECUTE getmax;
SELECT cmd,#n AS step, #minkey AS min, #maxkey AS max;
EXECUTE statement;
set #minkey=#maxkey;
set #n=#n+1;
UNTIL #maxkey IS NULL
END REPEAT;
select CONCAT(cmd, " EXECUTED IN ",#n," STEPS") AS MESSAGE;
END//
Background - I have a DB created from a single large flat file. Instead of creating a single large table with 106 columns. I created a "columns" table which stores the column names and the id of the table that holds that data, plus 106 other tables to store the data for each column. Since not all the records have data in all columns, I thought this might be a more efficient way to load the data (maybe a bad idea).
The difficulty with this was rebuilding a single record from this structure. To facilitate this I created the following procedure:
DROP PROCEDURE IF EXISTS `col_val`;
delimiter $$
CREATE PROCEDURE `col_val`(IN id INT)
BEGIN
DROP TEMPORARY TABLE IF EXISTS tmp_record;
CREATE TEMPORARY TABLE tmp_record (id INT(11), val varchar(100)) ENGINE=MEMORY;
SET #ctr = 1;
SET #valsql = '';
WHILE (#ctr < 107) DO
SET #valsql = CONCAT('INSERT INTO tmp_record SELECT ',#ctr,', value FROM col',#ctr,' WHERE recordID = ',#id,';');
PREPARE s1 FROM #valsql;
EXECUTE s1;
DEALLOCATE PREPARE s1;
SET #ctr = #ctr+1;
END WHILE;
END$$
DELIMITER ;
Then I use the following SQL where the stored procedure parameter is the id of the record I want.
CALL col_val(10);
SELECT c.`name`, t.`val`
FROM `columns` c INNER JOIN tmp_record t ON c.ID = t.id
Problem - The first time I run this it works great. However, each subsequent run returns the exact same record even though the parameter is changed. How does this persist even when the stored procedure should be dropping and re-creating the temp table?
I might be re-thinking the whole design and going back to a single table, but the problem illustrates something I would like to understand.
Unsure if it matters but I'm running MySQL 5.6 (64 bit) on Windows 7 and executing the SQL via MySQL Workbench v5.2.47 CE.
Thanks,
In MySQL stored procedures, don't put an # symbol in front of local variables (input parameters or locally declared variables). The #id you used refers to a user variable, which is kind of like a global variable for the session you're invoking the procedure from.
In other words, #id is a different variable from id.
That's the explanation of the immediate problem you're having. However, I would not design the tables as you have done.
Since not all the records have data in all columns, I thought this might be a more efficient way to load the data
I recommend using a conventional single table, and use NULL to signify missing data.
Ok, First off, I am not a mysql guru. Second, I did search, but saw nothing relevant related to mysql, and since my DB knowledge is limited, guessing syntactical differences between two different Database types just isn't in the cards.
I am trying to determine if a particular value already exists in a table before inserting a row. I've decided to go about this using two Stored procedures. The first:
CREATE PROCEDURE `nExists` ( n VARCHAR(255) ) BEGIN
SELECT COUNT(*) FROM (SELECT * FROM Users WHERE username=n) as T;
END
And The Second:
CREATE PROCEDURE `createUser` ( n VARCHAR(255) ) BEGIN
IF (nExists(n) = 0) THEN
INSERT INTO Users...
END IF;
END
So, as you can see, I'm attempting to call nExists from createUser. I get the error that no Function exists with the name nExists...because it's a stored procedure. I'm not clear on what the difference is, or why such a difference would be necessary, but I'm a Java dev, so maybe I'm missing some grand DB-related concept here.
Could you guys help me out by any chance?
Thanks
I'm not sure how it helped you, but...
why SELECT COUNT(*) FROM (SELECT * FROM Users WHERE username=n) and not just SELECT COUNT(*) FROM Users WHERE username=n?
Just make the user name (or whatever the primary application index is) a UNIQUE index and then there is no need to test: Just try to insert a new record. If it already exists, handle the error. If it succeeds, all is well.
It can (and should) all be one stored procedure.