Sometimes an application requires quite a few SQL queries before it can do anything useful. I was wondering if there is a way to send those as a batch to the database, to avoid the overhead of going back and forth between the client and the server?
If there is no standard way to do it, I'm using the python bindings of MySQL.
PS: I know MySQL has an executemany() function, but that's only for the same query executed many times with different parameters, right?
This process works best on inserts
Make all you SQL queries into Stored Procedures. These eventually will become child stored procedures
Create Master Store procedure to run all other Stored Procedures.
Modify master Stored procedure to accept values required by child Stored Procedures
Modify master Stored procedure to accept commands using "if" statements to know which
child stored procedures to run
If you need return data from Database use 1 stored procedure at the time.
Related
I'm exclusively using Stored Procedures when doing database operations with MySQL. There is a need to insert thousands of records periodically. Is there a way to pass a collection as a parameter to stored procedure? If there is, how does the stored procedure go about inserting the received parameter as one bulk insert?
My understanding of stored procedures is that the parameters cannot contain any code that will be executable commands like INSERT DELETE UPDATE only variables. So you cannot pass a whole INSERT comand to a stored procedure. Hence store procedures give protection from SQL injection. Here is the excellent and very Turing like explanation of the rationale for the separation of commands and parameters in stored procedures by #Polynomials https://security.stackexchange.com/questions/25684/how-can-i-explain-sql-injection-without-technical-jargon/25710#25710
So batch inserts will either be LOAD DATA INFILE as #O. Jones said in comments above or an INSERT stored procedure where you use the language of your choice to iterate the population of the parameters insert by insert.
We have a large database and we do manipulations on it ever day by using the basic mysql queries.
Can anyone please tell me, what is the use of Mysql Stored Procedures?
The real use of the Stored Procedures comes into picture when have any application accessing database.
For example: Imagine that you have written all your database operations in the form of queries in your data access code.
Suppose, that you need to make any change to query , then you need to rebuild and redeploy the entire application in order see your changes.
But, if you are using stored procs and refering them in application, you can just make changes in your database with out need for redeploying the application.
So, obviously better security , maintainability and much more
Note: This is one scenario where stored procs are better than normal queries.
Usage of Stored Procs also avoids SQL Injection Attacks
In very simple words, stored procedures allow you to store your quires along with database, you can combine multiple quires in single procedure. now whenever you want to execute those quires just "CALL yourProcedure;"
Need to perform specific query daily ?
Read about MySQL events = stored procedures with scheduling capability !
https://dev.mysql.com/doc/refman/5.1/en/events.html
I am trying to work with EF, for the first time ever. I'm not sure I fully understand EF yet.
I already have a database with data in it, so I've generated my models from DB.
Our current setup runs EVERYTHING through stored procedures, even selects.
However, unless I'm mistaking, the models select directly into the tables, when I have generated them.
Can I change this behaviour, so it calls the select procedure instead?
No - at least with EF4. I can't speak for EF5
You can use stored procedures to insert and update, but those stored procedures must have all the parameters EF expects, so you're probably going to have to wrap your existing procedures in new procedures.
For select, you can use a FunctionImport and ExecuteFunction to populate an Entity.
I have searched through the internet, and understand that the only way to change the body of a store procedure is by dropping and creating it again. There seems nothing wrong with the mechanism but if I have a client application (or thousands of distributed clients) that keeps invoking the store procedure to update some data on the server database, dropping the procedure would result in data lost and/or corruption.
I'm thinking if there is a syntax like "CREATE PROCEDURE IF EXIST..." or something functions similarly so the update operation would be carried out smoothly. Yet I didn't find such thing being available in MySQL.
So how do you guys think this issue can be addressed? Awesome thoughts?
You cannot modify a stored procedure (though you can change its characteristics) in MySQL. From the ALTER PROCEDURE page.
This statement can be used to change the characteristics of a stored
procedure. More than one change may be specified in an ALTER PROCEDURE
statement. However, you cannot change the parameters or body of a
stored procedure using this statement; to make such changes, you must
drop and re-create the procedure using DROP PROCEDURE and CREATE
PROCEDURE.
While it is possible to lose data while performing this update (though it should be a relatively small window), it's unlikely that your data will be corrupted. I'd take a look at message queuing technologies if your system needs to be guarded against data loss from database downtime.
Very brief background:
We are making use of CLR stored procedures to apply access control, using Active Directory, on query results to restrict what the end user can see accordingly. In a nutshell, this is done by removing rows from a datatable where the user does not satisfy the criteria for access to the result (document in this case).
This filtering was previously done on the client before displaying the results. SQL 2008 and a much more powerful server is the motivation for moving this access filtering off the client.
What I am wondering is, is there any performance benefit to be had from calling the original regular T-SQL stored procedure from the CLR stored procedure equivalent, instead of having 'inline' T-SQL passed into the comand object (which in this case is just the original T-SQL that was made a stored procedure) ? I cannot find anywhere where someone has mentioned this (in part probably because it would be very confusing as an example of CLR SPs, I guess :-) ).
It seems to me that you might, as the T-SQL stored proc has already been optimised and compiled ?
Is anyone able to confirm this for me ?
Hope I've been clear enough. Thanks very much,
Colm.
If your SQL CLR stored procedure does a specific query properly (nicely parametrized) and executes it fairly frequently, then that T-SQL query will be just run once through the whole "determine the optimal execution plan" sequence and then stored in the plan cache of your SQL Server (and not evicted from it any faster than a similar T-SQL stored procedure).
As such, it will be just as "pre-compiled" as your original T-SQL stored procedure. From that point of view, I don't see any benefit.
If you could tweak your SQL statement from within your SQL CLR procedure in such a way that it would actually not even include those rows into the result set that you'll toss out in the end anyway, then your SQL-CLR stored procedure executing a properly parametrized T-SQL query might even be a bit faster than having a standard T-SQL stored procedure return too much data from which you need to exclude some rows again.