I'm writing a stored procedure that will perform a SELECT, loop through it with a cursor, do some mildly complicated processing, and then return some of the rows in a modified form.
One way to do this is to create a temporary table, insert the results of the processing there, and then finish the procedure with a select * from temp_table;
I'm wondering if I can skip that part and just stream the computed rows directly to the client, without involving the overhead of temporary storage.
Every unbounded SELECT within a stored procedure writes a separate result-set to the wire. The only way to combine them into a single result-set is to use a temporary table, and then SELECT from it when you're done.
Otherwise, the multiple result sets must be iterated by the client by calling more_results, which tells you whether more results are available. It returns false when your procedure finishes.
The mechanism MySQL uses to actually write these multiple result sets is very lightweight, so it depends on whether this works for what you're trying to do, whether this is useful in your application.
When should I use a function rather than a stored procedure in SQL, and vice versa? What is the purpose of each?
Functions are computed values and cannot perform permanent environmental changes to SQL Server (i.e., no INSERT or UPDATE statements allowed).
A function can be used inline in SQL statements if it returns a scalar value or can be joined upon if it returns a result set.
A point worth noting from comments, which summarize the answer. Thanks to #Sean K Anderson:
Functions follow the computer-science definition in that they MUST return a value and cannot alter the data they receive as parameters
(the arguments). Functions are not allowed to change anything, must
have at least one parameter, and they must return a value. Stored
procs do not have to have a parameter, can change database objects,
and do not have to return a value.
Here's a table summarizing the differences:
Stored Procedure
Function
Returns
Zero or more values
A single value (which may be a scalar or a table)
Can use transaction?
Yes
No
Can output to parameters?
Yes
No
Can call each other?
Can call a function
Cannot call a stored procedure
Usable in SELECT, WHERE and HAVING statements?
No
Yes
Supports exception handling (via try/catch)?
Yes
No
Functions and stored procedures serve separate purposes. Although it's not the best analogy, functions can be viewed literally as any other function you'd use in any programming language, but stored procs are more like individual programs or a batch script.
Functions normally have an output and optionally inputs. The output can then be used as the input to another function (a SQL Server built-in such as DATEDIFF, LEN, etc) or as a predicate to a SQL Query - e.g., SELECT a, b, dbo.MyFunction(c) FROM table or SELECT a, b, c FROM table WHERE a = dbo.MyFunc(c).
Stored procs are used to bind SQL queries together in a transaction, and interface with the outside world. Frameworks such as ADO.NET, etc. can't call a function directly, but they can call a stored proc directly.
Functions do have a hidden danger though: they can be misused and cause rather nasty performance issues: consider this query:
SELECT * FROM dbo.MyTable WHERE col1 = dbo.MyFunction(col2)
Where MyFunction is declared as:
CREATE FUNCTION MyFunction (#someValue INTEGER) RETURNS INTEGER
AS
BEGIN
DECLARE #retval INTEGER
SELECT localValue
FROM dbo.localToNationalMapTable
WHERE nationalValue = #someValue
RETURN #retval
END
What happens here is that the function MyFunction is called for every row in the table MyTable. If MyTable has 1000 rows, then that's another 1000 ad-hoc queries against the database. Similarly, if the function is called when specified in the column spec, then the function will be called for each row returned by the SELECT.
So you do need to be careful writing functions. If you do SELECT from a table in a function, you need to ask yourself whether it can be better performed with a JOIN in the parent stored proc or some other SQL construct (such as CASE ... WHEN ... ELSE ... END).
Differences between stored procedures and user-defined functions:
Stored procedures cannot be used in Select statements.
Stored procedures support Deferred Name Resolution.
Stored procedures are generally used for performing business logic.
Stored procedures can return any datatype.
Stored procedures can accept greater numbers of input parameter than user defined functions. Stored procedures can have up to 21,000 input parameters.
Stored procedures can execute Dynamic SQL.
Stored procedures support error handling.
Non-deterministic functions can be used in stored procedures.
User-defined functions can be used in Select statements.
User-defined functions do not support Deferred Name Resolution.
User-defined functions are generally used for computations.
User-defined functions should return a value.
User-defined functions cannot return Images.
User-defined functions accept smaller numbers of input parameters than stored procedures. UDFs can have up to 1,023 input parameters.
Temporary tables cannot be used in user-defined functions.
User-defined functions cannot execute Dynamic SQL.
User-defined functions do not support error handling. RAISEERROR OR ##ERROR are not allowed in UDFs.
Non-deterministic functions cannot be used in UDFs. For example, GETDATE() cannot be used in UDFs.
STORE PROCEDURE
FUNCTION (USER DEFINED FUNCTION)
Procedure can return 0, single or multiple values
Function can return only single value
Procedure can have input, output parameters
Function can have only input parameters
Procedure cannot be called from a function
Functions can be called from procedure
Procedure allows select as well as DML statement in it
Function allows only select statement in it
Exception can be handled by try-catch block in a procedure
Try-catch block cannot be used in a function
We can go for transaction management in procedure
We can not go for transaction management in function
Procedure cannot be utilized in a select statement
Function can be embedded in a select statement
Procedure can affect the state of database means it can perform CRUD operation on database
Function can not affect the state of database means it can not perform CRUD operation on database
Procedure can use temporary tables
Function can not use temporary tables
Procedure can alter the server environment parameters
Function can not alter the environment parameters
Procedure can use when we want instead is to group a possibly- complex set of SQL statements
Function can use when we want to compute and return a value for use in other SQL statements
Write a user-defined function when you want to compute and return a value for use in other SQL statements; write a stored procedure when you want instead is to group a possibly-complex set of SQL statements. These are two pretty different use cases, after all!
Basic Difference
Function must return a value but in Stored Procedure it is optional( Procedure can return zero or n values).
Functions can have only input parameters for it whereas Procedures can have input/output parameters .
Function takes one input parameter it is mandatory but Stored Procedure may take o to n input parameters..
Functions can be called from Procedure whereas Procedures cannot be called from Function.
Advance Difference
Procedure allows SELECT as well as DML(INSERT/UPDATE/DELETE) statement in it whereas Function allows only SELECT statement in it.
Procedures can not be utilized in a SELECT statement whereas Function can be embedded in a SELECT statement.
Stored Procedures cannot be used in the SQL statements anywhere in the WHERE/HAVING/SELECT section whereas Function can be.
Functions that return tables can be treated as another rowset. This can be used in JOINs with other tables.
Inline Function can be though of as views that take parameters and can be used in JOINs and other Rowset operations.
Exception can be handled by try-catch block in a Procedure whereas try-catch block cannot be used in a Function.
We can go for Transaction Management in Procedure whereas we can't go in Function.
source
a User Defined Function is an important tool available to a sql server programmer. You can use it inline in a SQL statement like so
SELECT a, lookupValue(b), c FROM customers
where lookupValue will be an UDF. This kind of functionality is not possible when using a stored procedure. At the same time you cannot do certain things inside a UDF. The basic thing to remember here is that UDF's:
cannot create permanent changes
cannot change data
a stored procedure can do those things.
For me the inline usage of a UDF is the most important usage of a UDF.
Stored Procedures are used as scripts. They run a series of commands for you and you can schedule them to run at certain times. Usually runs multiples DML statements like INSERT, UPDATE, DELETE, etc. or even SELECT.
Functions are used as methods. You pass it something and it returns a result. Should be small and fast - does it on the fly. Usually used in a SELECT statement.
SQL Server functions, like cursors, are meant to be used as your last weapon! They do have performance issues and therefore using a table-valued function should be avoided as much as possible. Talking about performance is talking about a table with more than 1,000,000 records hosted on a server on a middle-class hardware; otherwise you don't need to worry about the performance hit caused by the functions.
Never use a function to return a result-set to an external code (like ADO.Net)
Use views/stored procs combination as much as possible. you can recover from future grow-performance issues using the suggestions DTA (Database Tuning Adviser) would give you (like indexed views and statistics) --sometimes!
for further reference see: http://databases.aspfaq.com/database/should-i-use-a-view-a-stored-procedure-or-a-user-defined-function.html
Stored procedure:
Is like a miniature program in SQL Server.
Can be as simple as a select statement, or as complex as a long
script that adds, deletes, updates, and/or reads data from multiple
tables in a database.
(Can implement loops and cursors, which both allow you to work with
smaller results or row by row operations on data.)
Should be called using EXEC or EXECUTE statement.
Returns table variables, but we can't use OUT parameter.
Supports transactions.
Function:
Can not be used to update, delete, or add records to the database.
Simply returns a single value or a table value.
Can only be used to select records. However, it can be called
very easily from within standard SQL, such as:
SELECT dbo.functionname('Parameter1')
or
SELECT Name, dbo.Functionname('Parameter1') FROM sysObjects
For simple reusable select operations, functions can simplify code.
Just be wary of using JOIN clauses in your functions. If your
function has a JOIN clause and you call it from another select
statement that returns multiple results, that function call will JOIN
those tables together for each line returned in the result set. So
though they can be helpful in simplifying some logic, they can also be a
performance bottleneck if they're not used properly.
Returns the values using OUT parameter.
Does not support transactions.
To decide on when to use what the following points might help-
Stored procedures can't return a table variable where as function can do that.
You can use stored procedures to alter the server environment parameters where as using functions you can't.
cheers
Start with functions that return a single value. The nice thing is you can put frequently used code into a function and return them as a column in a result set.
Then, you might use a function for a parameterized list of cities. dbo.GetCitiesIn("NY") That returns a table that can be used as a join.
It's a way of organizing code. Knowing when something is reusable and when it is a waste of time is something only gained through trial and error and experience.
Also, functions are a good idea in SQL Server. They are faster and can be quite powerful. Inline and direct selects. Careful not to overuse.
Here's a practical reason to prefer functions over stored procedures. If you have a stored procedure that needs the results of another stored procedure, you have to use an insert-exec statement. This means that you have to create a temp table and use an exec statement to insert the results of the stored procedure into the temp table. It's messy. One problem with this is that insert-execs cannot be nested.
If you're stuck with stored procedures that call other stored procedures, you may run into this. If the nested stored procedure simply returns a dataset, it can be replaced with a table-valued function and you'll no longer get this error.
(this is yet another reason we should keep business logic out of the database)
I realize this is a very old question, but I don't see one crucial aspect mentioned in any of the answers: inlining into query plan.
Functions can be...
Scalar:
CREATE FUNCTION ... RETURNS scalar_type AS BEGIN ... END
Multi-statement table-valued:
CREATE FUNCTION ... RETURNS #r TABLE(...) AS BEGIN ... END
Inline table-valued:
CREATE FUNCTION ... RETURNS TABLE AS RETURN SELECT ...
The third kind (inline table-valued) are treated by the query optimizer essentially as (parametrized) views, which means that referencing the function from your query is similar to copy-pasting the function's SQL body (without actually copy-pasting), leading to the following benefits:
The query planner can optimize the inline function's execution just as it would any other sub-query (e.g. eliminate unused columns, push predicates down, pick different JOIN strategies etc.).
Combining several inline function doesn't require materializing the result from the first one before feeding it to the next.
The above can lead to potentially significant performance savings, especially when combining multiple levels of functions.
NOTE: Looks like SQL Server 2019 will introduce some form of scalar function inlining as well.
It is mandatory for Function to return a value while it is not for stored procedure.
Select statements only accepted in UDF while DML statements not required.
Stored procedure accepts any statements as well as DML statements.
UDF only allows inputs and not outputs.
Stored procedure allows for both inputs and outputs.
Catch blocks cannot be used in UDF but can be used in stored procedure.
No transactions allowed in functions in UDF but in stored procedure they are allowed.
Only table variables can be used in UDF and not temporary tables.
Stored procedure allows for both table variables and temporary tables.
UDF does not allow stored procedures to be called from functions while stored procedures allow calling of functions.
UDF is used in join clause while stored procedures cannot be used in join clause.
Stored procedure will always allow for return to zero. UDF, on the contrary, has values that must come - back to a predetermined point.
Functions can be used in a select statement where as procedures cannot.
Stored procedure takes both input and output parameters but Functions takes only input parameters.
Functions cannot return values of type text, ntext, image & timestamps where as procedures can.
Functions can be used as user defined datatypes in create table but procedures cannot.
***Eg:-create table <tablename>(name varchar(10),salary getsal(name))
Here getsal is a user defined function which returns a salary type, when table is created no storage is allotted for salary type, and getsal function is also not executed, But when we are fetching some values from this table, getsal function get’s executed and the return
Type is returned as the result set.
Generally using stored procedures is better for perfomances.
For example in previous versions of SQL Server if you put the function in JOIN condition the cardinality estimate is 1 (before SQL 2012) and 100 (after SQL 2012 and before of SQL 2017) and the engine can generate a bad execution plan.
Also if you put it in WHERE clause the SQL Engine can generate a bad execution plan.
With SQL 2017 Microsoft introduced the feature called interleaved execution in order to produce a more accurate estimate but the stored procedure remains the best solution.
For more details look the following article of Joe Sack
https://techcommunity.microsoft.com/t5/sql-server/introducing-interleaved-execution-for-multi-statement-table/ba-p/385417
In SQL Server, functions and stored procedure are two different types of entities.
Function: In SQL Server database, the functions are used to perform some actions and the action returns a result immediately.
Functions are two types:
System defined
User defined
Stored Procedures: In SQL Server, the stored procedures are stored in server and it can be return zero, single and multiple values.
Stored Procedures are two types:
System Stored Procedures
User Defined Procedures
My table contains the following fields, Name,Age,Salary,Phone,DOB. Based on a settings table, I have to select only some fields. For example, I say in settings, only Name and Phone is required. How can I do it using stored procedure ?
EDIT :
Which one is good.
Select the required fields from the table.
Select all columns and in ASP.NET page, use .Visibility property to hide or show columns
SQL is a fixed column language: columns can not be added or removed "on the fly"
You would need to use dynamic SQL to build a SELECT statement, or use IF statements to execute different ones. However, you open up caching, security and injection issues.
Personally, I'd ignore columns in the client code and have a simple, single efficient SQL query. The contract or API between SQL Server and the client should be static and predictable. If the settings table is applied in SQL Server, your client doesn't know what columns to expect. If your client does know, then it can ignore them.
After your edit, option 2, kind of.
But the data should be removed before being rendered in the page.
Keep it simple: don't try to optimise anything yet
You would need to have multiple different selects - based on your settings table - in your stored proc to return the different sets of data.
CREATE PROCEDURE dbo.YourProcedure(...)
AS BEGIN
DECLARE #Setting INT -- ?? whatever it is
SELECT #Setting = Choice FROM dbo.YourSettingsTable WHERE ....... ???
IF #Setting = 1
SELECT Name, Phone
FROM dbo.YourDataTable
ELSE
SELECT Name, Age, DOB, Phone, Salary
FROM dbo.YourDataTable
END
Using this approach, however, has its dangers - since the stored proc might return one set of data or quite another, your SQL Server query optimizer might make a very good decision on how to access the data for one setting - but when your setting changes, that execution plan will be totally outdated, thus potentially leading to horrible performance......
On the other hand - it might be easier to determine that setting before calling your stored proc - and then just pass in that setting as a stored proc parameter.
Or even better yet: have separate stored procs for each "scenario" - and then from the caller, call the appropriate stored proc depending on the value of your setting....
Create the sql you want dynamically then execute it with exec.
declare #sql varchar(500)
set #sql = 'select 123'
exec (#sql)
The above code should help you understand what you need to know.
Have a stored procedure for each set of fields you want to select
Allow the list of field names to be passed in as a parameter
I've been trying to figure out what's wrong with a set of queries I've got and I'm just confused at this point.
It's supposed to be in a stored procedure which gets called by a GUI application.
There's only one "tiny" problem, it's first a simple UPDATE, then an INSERT using a SELECT with a subselect and finally another UPDATE. Running these queries together by hand I get a total execution time of 0.057s, not too shabby.
Now, I try creating a stored procedure with these queries in it and five input variables, I run this procedure and on the first attempt it took 47.096s with subsequent calls to it showing similar execution times (35 to 50s). Running the individual queries from the MySQL Workbench still show execution times of less than 0.1s
There really isn't anything fancy about these queries, so why is the stored procedure taking an eternity to execute while the queries by themselves only take a fraction of a second? Is there some kind of MySQL peculiarity that I'm missing here?
Additional testing results:
It seems that if I run the queries in MySQL Workbench but use variables instead of just putting the values of the variables in the queries it runs just as slow as the stored procedure. So I tried changing the stored procedure to just use static values instead of variables and suddenly it ran blazingly fast. Apparently for some reason using a variable makes it run extremely slow (for example, the first UPDATE query goes from taking approximately 0.98s with three variables to 0.04-0.05s when I use the values of variables directly in the query, regardless of if it's in the stored procedure or running the query directly).
So, the problem isn't the stored procedure, it's something related to my use of variables (which is unavoidable).
I had the same problem. After researching for a while, I found out the problem was the collation issue while MySQL was comparing text.
TL;DR: the table was created in one collation while MySQL "thought" the variable was in another collation. Therefore, MySQL cannot use the index intended for the query.
In my case, the table was created with (latin1, latin1_swedish_ci) collation. To make MySQL to use the index, I had to change the where clause in the stored procedure from
UPDATE ... WHERE mycolumn = myvariable
to
UPDATE ... WHERE mycolumn =
convert(myvariable using latin1) collate latin1_swedish_ci
After the change, the stored procedure looked something like this:
CREATE PROCEDURE foo.'bar'()
BEGIN
UPDATE mytable SET mycolumn1 = variable1
WHERE mycolumn2 =
convert(variable2 using latin1) collate latin1_swedish_ci
END;
where (latin1, latin1_swedish_ci) is the same collation that my tableA was created with.
To check if MySQL uses the index or not, you can change the stored procedure to run an explain statement as followed:
CREATE PROCEDURE foo.'bar'()
BEGIN
EXPLAIN SELECT * FROM table WHERE mycolumn2 = variable2
END;
In my case, the explain result showed that no index was used during the execution of the query.
Note that MySQL may use the index when you run the query alone, but still won't use the index for the same query inside a stored procedure, which maybe because somehow MySQL sees the variable in another collation.
More information on the collation issue can be found here:
http://lowleveldesign.wordpress.com/2013/07/19/diagnosing-collation-issue-mysql-stored-procedure/
Back up link:
http://www.codeproject.com/Articles/623272/Diagnosing-a-collation-issue-in-a-MySQL-stored-pro
I had a similar problem.
Running a mysql routine was horrible slow.
But a colleague helped me.
The problem was that AUTOCOMMIT was true;
So every insert into and select was creating a complete transaction.
Then I run my routine with
SET autocommit=0;
at the beginning and
SET autocommit=1;
at the end. The performance went from nearly 500s to 4s
Since I didn't want to waste too much time trying to figure out why using variables in my stored procedures made them extremely slow I decided to employ a fix some people would consider quite ugly. I simply executed each query directly from the data access layer of my application. Not the prettiest way to do it (since a lot of other stuff for this app uses stored procedures) but it works and now the user won't have to wait 40+ seconds for certain actions as they happen almost instantly.
So, not really a solution or explanation of what was happening, but at least it works.
Upvote for a very interesting and important question. I found this discussion of some of the reasons that a stored procedure might be slow. I'd be interested to see readers' reactions to it.
The main recommendation that I took from the interchange: it helps to add more indexes.
Something that we ran across today that makes procedures slow, even when they run very fast as direct queries, is having parameter (or, presumably, variable) names that are the same as column names. The short version is, don't use a parameter name that is the same as one of the columns in the query in which it will be used. For example, if you had a field called account_id and a parameter named the same, change it to something like in_account_id and your run time can go from multiple seconds to hundredths of a second.
I'm a bit confused about what is better to use in the following case:
I have a quite complex query that is used for reporting purposes. A simplified version looks like
SELECT [type], COUNT(*) as total_num, COUNT(DISTINCT user_id) as uq_user_num
FROM table1
INNER JOIN table2 ON (...)
...
WHERE table3.last_action_date BETWEEN :start_date AND :end_date
GROUP BY [type]
I can create an inline function or a stored procedure that takes start_date and end_parameters and executes this query.
I incline to function because this task does not involve any data modification or complex logic. Also, I may want to use the result in APPLY later (it's not really important at the moment).
Does it make sense to use function, not procedure? Is it any difference from performance point of view (execution plan caching, etc) ?
Thank you.
Using a multi-statement table valued function is similar to using a proc from plan caching and cached plan reuse perspective.. Using an inline table valued function is similar to using a view from the plan cache and plan reuse perspective(reuse only happen is exact same statement is used. ie same parameters).
Considering the same you should use a multi-statement table valued function.
You may want to consider using a View too. A view is efficient if the results do no change given the parameters provided, which is what you have here. In this case, the results will not change if you make two calls with the same start and end date.
However, two of the main differences between a stored proc and a function are that you cannot call updates/ inserts from a function and you cannot call a stored proc as part of a select statement, but you can with a function.
See this thread for more info:
Function vs. Stored Procedure in SQL Server