I'm working on a stored procedure that will accept a string and return a new string of text. Input parameters are #OrderId and #OrderText which is a string with dollar sign enclosed variables like so... $Order Name$ sent to $Customer$
The valid variables are in a Variables table (values such as Order Name, Customer, a total of 25 of them which should remain fairly static). Variables can only be used once in the string.
The stored procedure needs to return the string but with the variables replaced with their respective values.
Example1
Input: 123, $Order Name$ sent to $Customer$
Returns: Toolkit sent to StackCustomer Inc.
Example2
Input: 456, $Customer$ requests $delivery method$ for $order type$
Returns: ABC Inc requests fast shipping for pallet orders.
Each of the variables can be retrieved using a function.
DECLARE #OrderId int = 123
DECLARE #InputText VARCHAR(500) = '$Order Name$ sent to $Customer$'
select
#InputText = case when #InputText like '%$order name$%'
then replace(#InputText, '$Order Name$', getOrderName(id) else '' end,
#InputText = case when #InputText like '%$customer$'
then replace(#InputText, '$Customer$', getCustomer(id) else '' end
-- repeat 25 times
Is there a better way? My main concern is maintainability - if a variable is added, renamed, or removed, this stored proc will need to be changed (although I'm told it would only happen a couple times a year, if that). Would dynamic sql be able to help in this case?
Personally, I would create a keywords table to maintain it. something like this
CREATE TABLE [keywords] (
key_value VARCHAR(100) NOT NULL,
function_value VARCHAR(100) NOT NULL
)
INSERT INTO [keywords]
VALUES
('$Customer$','getCustomer(id)'),
('$Order Name$' ,'getOrderName(id)'),
('$order type$','getOrderType(id)')
Then use dynamic sql create REPLACE SQL
DECLARE #OrderId int = 123
DECLARE #InputText VARCHAR(500) = '$Order Name$ sent to $Customer$'
DECLARE #sql VARCHAR(8000) = 'SELECT '
SELECT
#sql = #sql +
' #InputText = replace(#InputText, ''' + key_value + ''', ' + function_value + ')'
+ ' ,'
FROM keywords
WHERE #InputText LIKE '%' + key_value + '%'
SELECT #sql = LEFT(#sql, LEN(#sql) -1)
PRINT #sql
EXEC(#sql)
SQLFiddle
I am not really getting why you need to do tokenization if you are going to use input variables for the procedure.
There are token templates already available for SQL Management Studio in the form of:
<(tokenName),(datatype),(defaultvalue)>
You can get their data filled directly with SQL Managment Studio with CTRL + SHIFT + M, or with ALT > Q > S in 2012, or by text finding their values in an environment.
If you are trying to put in an input to be accessed by an outside developing platform that changes the strings like ADO.NET or the Entity Framework in C#/VB.NET I still am not getting why you would not just make more input variables.
You could do this quite easily:
Declare #OrderPlace varchar(128), #Customer varchar(64), #StringCombine varchar(512);
Select
#OrderPlace = 'Place I am at'
, #Customer = 'Mr Customer';
Select #StringCombine = #Customer + ' order at ' + #OrderPlace
Then if a different language accessed your SQL server procedure it would just have to put in two parameters #Customer and #OrderPlace. You could even set #StringCombine to be an output variable. This is much more preferable then text replacing characters and then running a string. This could be able for SQL injection attacks potentially so santizing your inputs is a big part of returning data, especially if you are altering something for SQL to run before it is ran.
You mentioned maintainability and this is much more robust because if I change the logic to the proc but DO NOT change the variables names, I did not have to change anything else. If I have to change an ADO.NET or other library for references and then SQL code, that is lot more work. Generally when working with tokens I strive for reuse of code where one part of it can go down and only hurt that part of it, not take the whole thing down.
Related
I have made a stored procedure. I want it to filter the data by different parameters. If I pass one parameter, it should be filtered by one; if I pass two, it should be filtered by two, and so on, but it is not working.
Can anyone help me please?
DROP PROCEDURE IF EXISTS medatabase.SP_rptProvince2;
CREATE PROCEDURE medatabase.`SP_rptProvince2`(
IN e_Region VARCHAR(45)
)
BEGIN
DECLARE strQuery VARCHAR(1024);
DECLARE stmtp VARCHAR(1024);
SET #strQuery = CONCAT('SELECT * FROM alldata where 1=1');
IF e_region IS NOT NULL THEN
SET #strQuery = CONCAT(#strQuery, ' AND (regionName)'=e_Region);
END IF;
PREPARE stmtp FROM #strQuery;
EXECUTE stmtp;
END;
AFAIK, you can't have a variable argument list like that. You can do one of a couple of things:
Take a fixed maximum number of parameters, and check them for null-ness before concatenating:
CREATE PROCEDURE SP_rptProvince2(a1 VARCHAR(45), a2 VARCHAR(45), ...)
...
IF a1 IS NOT NULL THEN
SET #strQuery = CONCAT(#strQuery, ' AND ', a2);
END IF;
If you need predetermined fields to which the criteria in the argument apply (like the e_Region parameter in your existing code), then you modify the CONCAT operation appropriately.
Possible invocation:
CALL SP_rptProvince2('''North''', 'column3 = ''South''')
Take a single parameter that is much bigger than just 45 characters, and simply append it to the query (assuming it is not null).
Clearly, this places the onus on the user to provide the correct SQL code.
Possible invocation:
CALL SP_rptProvince2('RegionName = ''North'' AND column3 = ''South''')
There's not a lot to choose between the two. Either can be made to work; neither is entirely satisfactory.
You might note that there was a need to protect the strings in the arguments with extra quotes; that is the sort of thing that makes this problematic.
I found a JSON-based approach which works with the latest MySQL/MariaDB systems. Check the link below (Original Author is Federico Razzoli): https://federico-razzoli.com/variable-number-of-parameters-and-optional-parameters-in-mysql-mariadb-procedures
Basically, you take a BLOB parameter which is actually a JSON object and then do JSON_UNQUOTE(JSON_EXTRACT(json object, key)) as appropriate.
Lifted an extract here:
CREATE FUNCTION table_exists(params BLOB)
RETURNS BOOL
NOT DETERMINISTIC
READS SQL DATA
COMMENT '
Return whether a table exists.
Parameters must be passed in a JSON document:
* schema (optional). : Schema that could contain the table.
By default, the schema containing this procedure.
* table : Name of the table to check.
'
BEGIN
DECLARE v_table VARCHAR(64)
DEFAULT JSON_UNQUOTE(JSON_EXTRACT(params, '$.table'));
DECLARE v_schema VARCHAR(64)
DEFAULT JSON_UNQUOTE(JSON_EXTRACT(params, '$.schema'));
IF v_schema IS NULL THEN
RETURN EXISTS (
SELECT TABLE_NAME
FROM information_schema.TABLES
WHERE
TABLE_SCHEMA = SCHEMA()
AND TABLE_NAME = v_table
);
ELSE
RETURN EXISTS (
SELECT TABLE_NAME
FROM information_schema.TABLES
WHERE
TABLE_SCHEMA = v_schema
AND TABLE_NAME = v_table
);
END IF;
END;
Below is a function that pads an input string to desired length on the specified side of the input string. When executed by SSIS the meta data is not returning the desired length. I need it to return the appropriate data length to SSIS.
CREATE FUNCTION [dbo].[udf_FillField]
(
#StringOrg nvarchar(max), --Original string
#LengthOfStringOutPut int, --Length of output sting
#FillChr nvarchar(1), --Char to pad the original string
#LRJustified nvarchar(1) --Left or right justified
)
RETURNS nvarchar(max)
AS
BEGIN
-- Declare Variable(s)
DECLARE #Result nvarchar(max)
-- Left Justified
IF #LRJustified = 'L'
BEGIN
SET #Result =
LEFT(ISNULL(#StringOrg, ''), #LengthOfStringOutPut) +
REPLICATE(#FillChr, (#LengthOfStringOutPut - LEN(LEFT(ISNULL(#StringOrg, ''), #LengthOfStringOutPut))))
END
-- Right Justified
IF #LRJustified = 'R'
BEGIN
SET #Result =
REPLICATE(#FillChr, (#LengthOfStringOutPut - LEN(LEFT(ISNULL(#StringOrg, ''), #LengthOfStringOutPut)))) +
LEFT(ISNULL(#StringOrg, ''), #LengthOfStringOutPut)
END
-- Incorrect Parameter Input
IF #LRJustified <> 'R' AND #LRJustified <> 'L'
BEGIN
SET #Result =
NULL
END
-- Output Data
RETURN #Result
END
Your UDF is defined as returning nvarchar(max). SSIS is going to look at the metadata for the function, see that it specifies that it returns nvarchar(max), or DT_NTEXT as SSIS will classify it, and it's done with validation.
The only way you're going to be able to use your function is explicitly cast the results to the appropriate size.
SELECT
CAST(dbo.udf_FillField(T.col1, 100, N'X', N'R') AS nvarchar(100)) AS Col1
FROM
dbo.MyTable AS T;
As a side note, UDFs feel like code reuse, that's what we're taught as developers. But in SQL Server, they are an abomination because the Optimizer has no way of producing an accurate cost for how much work the function will require so it guesses, and 100 times out of 100, it's probably wrong. You will get a consistent implementation of whatever business logic you're encapsulating but the penalty to performance can be severe based on data volumes and logic required.
We've been experiencing some severe performance issues on our SQL 2008 r2 DB. When we run the Activity Monitor in SQL Server Management Studio, and a SP that returns who/what is active, it shows the following three transactions as being very expensive:
Query 1:
SET #DateMonth = '0' + #DateMonth
Query 2:
SET #DateMonth = CAST(datepart(mm, #TheDate) AS VARCHAR(2))
Function:
CREATE FUNCTION [dbo].[DateToNGDate] (#TheDate datetime)
RETURNS VARCHAR(10)
AS
BEGIN
DECLARE #DateYear VARCHAR(4)
DECLARE #DateMonth VARCHAR(2)
DECLARE #DateDay VARCHAR(2)
SET #DateYear = CAST(datepart(yyyy, #TheDate) AS VARCHAR(4))
SET #DateMonth = CAST(datepart(mm, #TheDate) AS VARCHAR(2))
IF (LEN (#DateMonth) = 1) SET #DateMonth = '0' + #DateMonth
SET #DateDay = CAST(datepart(dd, #TheDate) AS VARCHAR(2))
IF (LEN (#DateDay) = 1) SET #DateDay = '0' + #DateDay
RETURN #DateYear+#DateMonth+#DateDay
END
That last one comes back like that, but i'm pretty sure it isn't creating the function (it already exists), rather it is just running it. It is also the one that comes up the most as appearing to be a performance killer (it's used throughout our code).
I'm sure these aren't what is actually causing the problem, but why would they appear as they are?
Scalar-valued functions are a known performance issue in SQL Server. One option is to define your function like this:
CREATE FUNCTION [dbo].[DateToNGDateX] (#TheDate datetime)
RETURNS table as return
(
select
cast(
CAST(datepart(yyyy, #TheDate) AS VARCHAR(4)) -- DateYear
+ right('0' + CAST(datepart(mm, #TheDate) AS VARCHAR(2)),2) -- DateMonth
+ right('0' + CAST(datepart(dd, #TheDate) AS VARCHAR(2)),2) -- DateDay
as varchar(30)) as Value
)
and reference it like this:
select Value from [dbo].[DateToNGDateX] ('20140110');
However, for the specific functionality you desire, test this as well:
select convert(char(8), cast('20140110' as date), 112);
which will return a date formatted as yyyymmdd.
Scalar UDFs are very slow if your select statement is returning a lot of rows, because they are going to be executed once for every row that is returned. Maybe you can rewrite as an inline function as per this link inline UDFs
I like Pieter Geerkens answer and I hope it has resolved your problem. The scalar function will return results for each of the matching rows wherever it is being used. So it is something similar to cursors. A table valued function will accept a set and return a set.
To verify the I\O changes between both processes, management studio might not help but on your own machine you can put a small trace to see what magnitude of performance you get.
Cheers!!
I am trying to create a stored that will accept two values, a column name and a value. It will then check if there is a record that exists for the passed in column name with the passed in value. I've tried the following
CREATE PROCEDURE p_HasActiveReference
#pi_colName varchar(100)
,#pi_colValue varchar(100)
AS
BEGIN
SET NOCOUNT ON
declare #sql varchar(1000)
set #sql = 'IF EXISTS(SELECT TOP 1 p.PaymentId
FROM Payments p
WHERE ' + #pi_colName + ' = ' + #pi_colValue + 'AND Active = 1)
SELECT ''True'' AS RETVAL
ELSE
SELECT ''False'' AS RETVAL'
exec(#sql)
END
However, I always get this error
Conversion failed when converting the varchar value 'InternalOr' to data type int.
When I call the procedure with the following
p_HasActiveReference 'InternalOrgCode', '10110'
The internalOrgCode column is of value varchar(10)
I am not a SQL expert, so I am not even sure if what I need to achieve is even possible using that technique!
Thanks!
At least one issue: you should be surrounding your string value with single quotes, and to escape those inside a string you need to double them up:
WHERE ' + #pi_colName + ' = ''' + #pi_colValue + ''' AND ...
You also may want to declare your #sql variable as something bigger than 100 characters! Looks like your string is getting truncated.
If the possible values for #pi_colName are finite, the data type is always string, and the columns are collation compatible, you could do something like this and avoid dynamic SQL:
SELECT ...
WHERE CASE #pi_colName
WHEN 'col1' THEN col1
WHEN 'col2' THEN col2
END = #pi_ColValue;
Hey I have a name column in my table, suppose i get have name O'Neil, when ever i read this name and insert into a linked server throught dynamic SQL I get error...How to handle this...
One method is to check if my name has a quote(') but I am researching if someone coul
my dynamic SQL looks like
'name = '+#quote+#name+#quote
#quote is nothing but below...
set #quote = ''''
thanks,
Naga
I guess you problem could be demonstrated with this code:
declare #Name varchar(20)
set #Name = 'O''Neil'
declare #SQL nvarchar(100)
set #SQL = 'select '+#Name
exec (#SQL)
Result: Unclosed quotation mark after the character string 'Neil'.
You can use quotename to properly add quote characters:
set #SQL = 'select '+quotename(#Name, '''')
The better alternative is to use sp_executesql and pass #Name as a parameter.
set #SQL = 'select #Name'
exec sp_executesql #SQL, N'#Name varchar(20)', #Name
If you can use PreparedStatements then this is better way of inserting/updating many rows. Such code in Jython (using Javas JDBC) looks like:
db = DriverManager.getConnection(jdbc_url, usr, passwd)
pstm = db.prepareStatement("insert into test (lastname) values (?)")
pstm.setString(1, "O'Neil")
reccnt = pstm.execute()
You will find similar PreparedStatement in ODBC, .net etc. It should be much lighter for database for massive inserts then creating full SQL insert statement for every inserted value.
Such massive insert from CSV file can look like:
pstm = db.prepareStatement("insert into test (lastname) values (?)")
while not csvfile.eof():
lastname = csvfile.next_line()[0]
pstm.setString(1, lastname)
reccnt = pstm.execute()