I want to implement a batch MySQL script to do something in a database. The thing is that, for each master id that I have I want to insert 4 tuples. But this tuples should be added in a transaction which means if one of these 4 tuples is failed the transaction should be rollback. Then I need to have some catching mechanism to capture that the query is failed. I CAN ONLY USE PURE MYSQL neither PHP, nor PERL etc. Even I cannot create any store procedure to do that. In Microsoft SQL Server there is ##error variable that solved my problem but in MYSQL we do not have any system variables showing the error code.
how can I do that?
Cheers,
This is an ugly workaround, but it worked for me when I was trying to import a batch of SQL queries and wrap the entire thing within a transaction, so that I could roll back if any of the SQL queries errored.
Since the size of the batch was massive, a SQL procedure with condition handler was not an option either.
You have to do this manually, so it really isn't a solution unless you are batching:
First, make sure your entire batch is stored in an SQL file. The SQL file should only contain the batch queries, and no transaction control queries.
Then start up a MySQL command line client and type in transaction commands manually:
mysql> SET AUTOCOMMIT = 0;
mysql> START TRANSACTION;
Then tell the command line client to run the batch file:
mysql> SOURCE path/to/file.sql
After that you can simply manually COMMIT; or ROLLBACK; depending on how happy you are with the result of your queries.
This is such a kludge, though. Anyone have a better approach?
Related
I have sql in a file, which contains a number of statements. How can I get mysql to check the validity of the sql without executing it?
Something like
mysql --check-only input.sql
At a minimum, I need syntax to be checked.
Ideally, it would also attempt to e.g. insert data and check constraints. Basically wrapping the whole file in an implicit transaction which is automatically rolled-back.
Can mysql CLI do this?
I suggest to use the following, as you seem to be okay with using transactions:
mysql --init-command="SET autocommit=0;BEGIN;" < input.sql
What this will do is it will just start a transaction, process the SQL inside input.sql (including checks on syntax and inserts). Then, the process will finish without a COMMIT; being issued which will have the server rollback the whole thing.
I am in process of data migration from old architecture to a new one. I have written SQL queries (insert query) for this purpose and I have used insert ignore to escape the duplication errors, and with this, they are converted into warnings. Now I want to log all these errors and warnings into a file so that stakeholders can refer to these files and check errors and warnings during the process of insertion with columns and value that was tried.
Following are the things that I checked and did:-
Used tee command
Used OUTFILE
Checked the error.log, general log, and MySQL log but all these logs either contained server errors or just the query that was executed.
I tried to execute the queries in MySQL Workbench and was able to see
duplication warnings with column and value in the action output. But
I think this will get lost with new query execution.
Try to do the below steps
Create log table with required column as per your wish
Create Stored procedure and move all your script to that stored procedure
In that stored procedure, whenever the exception raised then log those exception to your log table (i mean insert one record into that log table whenever the exception raised)
Take report from log table and give to stakeholders
The purpose of the trigger is exporting an specific data from a table with bcp after inserting data on that table, so I thought doing it in this way, I know that the trigger waits for bcp, that is waiting for a lock on the table to be released, but that lock is held until after the trigger, and for this reason it doesn't work. How can I do it? or do I need to add some function or something for that works?
I'm using SQL Server 2008.
ALTER TRIGGER [TRIGGER] on [TABLE] after INSERT AS BEGIN
DECLARE #CMD NVARCHAR(1000)
SET #CMD = 'cd.. && "C:\Program Files\Microsoft SQL Server\100\Tools\Binn\bcp.exe" "SELECT TOP 1 CODE FROM[TABLE] WITH (NOLOCK) ORDER BY ID DESC" queryout "\\FOLDER\FOLDER\FILE.txt" -T -c -S "[SERVERNAME]"'
EXEC master..XP_CMDSHELL #CMD
END
Don't use bcp in a trigger. Even if you could get it to work it will slow down your database, probably to the point where it will not be usable. bcp is a command line utility and should be treated as one
I recommend that you use an SQL Server Agent to execute extra actions on a scheduled or triggered basis
You can also read this tutorial that will help you get started with Agents
If you do not have SQL Server Agent (Express does not include it), then you have a few other options:
Write your own Agent. Here is an example
Call a stored procedure after insert. This answer used that method to solve similar problem to the one you posted
Use a stored procedure to write data and process the export code
Use a scheduled task processes data on a schedule (this is where bcp can be used without killing server performance)
Not sure if this would work but you could try changing to an INSTEAD OF trigger and do the INSERT and bcp within the trigger
I suspect this may circumvent the lock on the table.
further reading - http://www.c-sharpcorner.com/UploadFile/37db1d/creating-and-managing-triggers-in-sql-server-20052008/
I am getting the following message while creating a stored procedure in MySQL Workbench:
"Review the SQL script to be applied on the database"
I have several tables inside the database but the stored procedure I am writing will be
used only for one table. Since, the SQL script of stored procedure is gonna apply on the whole database, I am wondering if it's gonna affect other tables as well? I don't want other tables to get disturbed because of this script.
Please provide your inputs as I am doing this for the first time.
Question #2:
Why do I see "DELIMITER $$" as the first statement while creating a routine before the following statement?
CREATE PROCEDURE `mydatabase`.`myfirstroutine` ()
BEGIN
Thanks
1) MySQL Workbench offers the option to review the generated SQL script before it is sent to the server. This way you can check it for possible problems.
2) The DELIMITER command is usually necessary to switch the current delimiter that ends a single statement (which is by default a semicolon) to something else because the stored procedure code itself needs the semicolon to separate individual commands. However the sp code must be sent as a whole to the server.
A few more details: the DELIMITER keywword is a client keyword only, that means the server doesn't know it and doesn't need it. It's an invention for clients to properly separate sql commands before sending them to the server (you cannot send a list of commands to a server, only individual statements).
In MySQL Workbench however, especially in the object editors where you edit e.g. the sp text, adding the DELIMITER command is essentially nonsense, because there's only this sp code, hence nothing to separate. This might disappear in future version but for now just ignore it.
Is it possible for MySQL database to invoke an external exe file when a new row is added to one of the tables in the database?
I need to monitor the changes in the database, so when a relevant change is made, I need to do some batch jobs outside the database.
Chad Birch has a good idea with using MySQL triggers and a user-defined function. You can find out more in the MySQL CREATE TRIGGER Syntax reference.
But are you sure that you need to call an executable right away when the row is inserted? It seems like that method will be prone to failure, because MySQL might spawn multiple instances of the executable at the same time. If your executable fails, then there will be no record of which rows have been processed yet and which have not. If MySQL is waiting for your executable to finish, then inserting rows might be very slow. Also, if Chad Birch is right, then will have to recompile MySQL, so it sounds difficult.
Instead of calling the executable directly from MySQL, I would use triggers to simply record the fact that a row got INSERTED or UPDATED: record that information in the database, either with new columns in your existing tables or with a brand new table called say database_changes. Then make an external program that regularly reads the information from the database, processes it, and marks it as done.
Your specific solution will depend on what parameters the external program actually needs.
If your external program needs to know which row was inserted, then your solution could be like this: Make a new table called database_changes with fields date, table_name, and row_id, and for all the other tables, make a trigger like this:
CREATE TRIGGER `my_trigger`
AFTER INSERT ON `table_name`
FOR EACH ROW BEGIN
INSERT INTO `database_changes` (`date`, `table_name`, `row_id`)
VALUES (NOW(), "table_name", NEW.id)
END;
Then your batch script can do something like this:
Select the first row in the database_changes table.
Process it.
Remove it.
Repeat 1-3 until database_changes is empty.
With this approach, you can have more control over when and how the data gets processed, and you can easily check to see whether the data actually got processed (just check to see if the database_changes table is empty).
you could do what replication does: hang on the 'binary log'. setup your server as a 'master server', and instead of adding a 'slave server', run mysqlbinlog. you'll get a stream of every command that modifies your database.
step in 'between' the client and server: check MySQLProxy. you point it to your server, and point your client(s) to the proxy. it lets you interpose Lua scripts to monitor, analyze or transform any SQL command.
I think it's going to require adding a User-Defined Function, which I believe requires recompilation:
MySQL FAQ - Triggers: Can triggers call an external application through a UDF?
I think it's really a MUCH better idea to have some external process poll changes to the table and execute the external program - you could also have a column which contains the status of this external program run (e.g. "pending", "failed", "success") - and just select rows where that column is "pending".
It depends how soon the batch job needs to be run. If it's something which needs to be run "sooner or later" and can fail and need to be retried, definitely have an app polling the table and running them as necessary.