BULK INSERT from comma delimited string - sql-server-2008

I have a table with the following data in one column:
abc,2,2,34,5,3,2,34,32,2,3,2,2
def,2,2,34,5,3,2,34,32,2,3,2,2
I want to take this data and insert it into another table, using the commas as delimiters, just like how you can specify the FIELDTERMINATOR in BULK INSERT statements.
Is there a way to do this using T-SQL?

I'm not sure if there is any direct way to do in the T-SQL , but if you want to use Bulk Insert you can use sqlcmd to export to CSV file and then Import the file back into server using Bulk Insert.
Create a dbo.Split Functionm, you can refer here split string into multiple record
There are tons of good examples.
if you want to execute as batch process, You can execute sqlcmd and 'Bulk Insert'
sqlcmd -S MyServer -d myDB -E -Q "select dbo.Split(col1) from SomeTable"
-o "MyData.csv" -h-1 -s"," -w 700
-s"," sets the column seperator to
bulk insert destTable
from "MyData.csv"
with
(
FIELDTERMINATOR = ',',
ROWTERMINATOR = '\n'
)
Otherwise, You can manipulate directly in the T-SQL, but given you have the same identify of columns definition.
INSERT INTO DestinationTable
SELECT dbo.Split(col1) FROM SomeTable

You need to use a Split function to split your string into a table variable, and then insert those values into your table.
There are tons of those split functions out there, with various pros and cons and various number of parameters and so forth.
Here is one that I quite like - very nicely done, clearly explained.
With that function, you should have no trouble converting your column into individual entries for your other table.

EDIT: allow multiple char separators
This is how I solved it, with two functions to do the splitting into columns (if you want a more complete solution with line splitting as well, see my other post here). It involves:
A scalar function (fSubstrNth) for extracting the n-th field of a line, given an separator
A scalar function (fPatIndexMulti) for finding the n-th index of the separator
(Optional) alternative Right function to accept negative values
Finally, some specific code to use in your solution, since SQL doesn't allow dynamic table-function definitions (in other words, you can't SELECT from a function with dynamic columns)
Now, for the code snippets:
fSubstrNth
-- =============================================
-- Author: Bernardo A. Dal Corno
-- Create date: 18/07/2017
-- Description: substring com 2 PatIndex limitando inicio e fim
-- =============================================
CREATE FUNCTION fSubstrNth
(
#Text varchar(max),
#Sep varchar(3),
#N int --Nth campo
)
RETURNS varchar(max)
AS
BEGIN
DECLARE #Result varchar(max)
IF #N<1 RETURN ''
IF #N=1
SET #Result = substring(#Text, 1, dbo.fPatIndexMulti(#Sep,#Text,1)-1)
ELSE
SET #Result = substring(#Text, dbo.fPatIndexMulti(#Sep,#Text,#N-1)+LEN(#Sep), CASE WHEN dbo.fPatIndexMulti(#Sep,#Text,#N)>0 THEN dbo.fPatIndexMulti(#Sep,#Text,#N)-dbo.fPatIndexMulti(#Sep,#Text,#N-1)-LEN(#Sep) ELSE LEN(#Text)+1 END)
RETURN #Result
END
fPatIndexMulti
-- =============================================
-- Author: Bernardo A. Dal Corno
-- Create date: 17/07/2017
-- Description: recursive patIndex
-- =============================================
CREATE FUNCTION [dbo].[fPatIndexMulti]
(
#Find varchar(max),
#In varchar(max),
#N tinyint
)
RETURNS int
AS
BEGIN
DECLARE #lenFind int, #Result int, #Texto varchar(max), #index int
DECLARE #i tinyint=1
SET #lenFind = LEN(#Find)-1
SET #Result = 0
SET #Texto = #In
WHILE (#i <= #N) BEGIN
SET #index = patindex('%'+#Find+'%',#Texto)
IF #index = 0 RETURN 0
SET #Result = #Result + #index
SET #Texto = dbo.xRight(#Texto, (#index + #lenFind)*-1)
SET #i = #i + 1
END
SET #Result = #Result + #lenFind*(#i-2)
RETURN #Result
END
xRight
-- =============================================
-- Author: Bernardo A. Dal Corno
-- Create date: 06/01/2015
-- Description: Right inverso (para nros < 0)
-- =============================================
CREATE FUNCTION [dbo].[xRight]
(
#Texto varchar(8000),
#Qntd int
)
RETURNS varchar(8000)
AS
BEGIN
DECLARE #Result varchar(8000)
IF (Len(#Texto) = 0) OR (#Qntd = 0)
SET #Result = ''
ELSE IF (#Qntd > 0)
SET #Result = Right(#Texto, #Qntd)
ELSE IF (#Qntd < 0)
SET #Result = Right(#Texto, Len(#Texto) + #Qntd)
RETURN #Result
END
Specific code
SELECT
acolumn = 'any value',
field1 = dbo.fSubstrNth(table.datacolumn,',',1),
field2 = dbo.fSubstrNth(table.datacolumn,',',2),
anothercolumn = 'set your query as you would normally do',
field3 = (CASE dbo.fSubstrNth(table.datacolumn,',',3) WHEN 'C' THEN 1 ELSE 0 END)
FROM table
Note that:
fSubstrNth receives the n-th field to extract from the 'datacolumn'
The query can be as any other. This means it can be stored in a procedure, tabled-function, view, etc. You can extract some or all fields, in any order you wish, and process however you want
If used in a stored procedure, you could create a generic way of creating a query and temp table that loads the string with dynamic columns, but you have to make a call to another procedure to use the data OR create a specific query like above in the same procedure (which would make it non-generic, just more reusable)

Related

Is it possible to search of a text in SSRS via Filters?

I would like to have a multiple values Paramter with text1, text2, text3
SSRS should only show me the rows that contain one or all of this Parameter values in a column.
I notice you can set a filter in a Dataset or the tablix. The problem is I do not have something a function that do both LIKE and IN
Do you someone have an idea?
I tried already the LIKE function and the VALUE =”*”+”Parameters!PAR.Value”+”*”.
It did work, but not on a multiple values parameter.
I don't think you will be able to do this using dataset filters using a multi-value parameter.
However, you should be able to change your dataset query to handle this.
CREATE PROC myProc (#myParameter varchar(1000))
AS
SELECT * FROM myTable t
JOIN string_split(#myParameter, ',') p on t.myColumn like '%' + p.value + '%'
When you pass the parameter to the stored proc you can use the following expression
=JOIN(Parameters!myParameter.Value, ",")
This will simply join each of your parameter values into a single comma separated string string ready to pass to the procedure.
If you do not have a version of SQL Server that support string_split
Hers's the code to create one. There are smaller versions around on the internet but this one was designed to handle special cases as well as the typical single character delimiters.
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE FUNCTION [Split](#sText varchar(8000), #sDelim varchar(20) = ' ')
RETURNS #retArray TABLE (idx smallint Primary Key, value varchar(8000))
AS
BEGIN
DECLARE #idx smallint,
#value varchar(8000),
#bcontinue bit,
#iStrike smallint,
#iDelimlength tinyint
IF #sDelim = 'Space'
BEGIN
SET #sDelim = ' '
END
SET #idx = 0
SET #sText = LTrim(RTrim(#sText))
SET #iDelimlength = DATALENGTH(#sDelim)
SET #bcontinue = 1
IF NOT ((#iDelimlength = 0) or (#sDelim = 'Empty'))
BEGIN
WHILE #bcontinue = 1
BEGIN
--If you can find the delimiter in the text, retrieve the first element and
--insert it with its index into the return table.
IF CHARINDEX(#sDelim, #sText)>0
BEGIN
SET #value = SUBSTRING(#sText,1, CHARINDEX(#sDelim,#sText)-1)
BEGIN
INSERT #retArray (idx, value)
VALUES (#idx, #value)
END
--Trim the element and its delimiter from the front of the string.
--Increment the index and loop.
SET #iStrike = DATALENGTH(#value) + #iDelimlength
SET #idx = #idx + 1
SET #sText = LTrim(Right(#sText,DATALENGTH(#sText) - #iStrike))
END
ELSE
BEGIN
--If you can't find the delimiter in the text, #sText is the last value in
--#retArray.
SET #value = #sText
BEGIN
INSERT #retArray (idx, value)
VALUES (#idx, #value)
END
--Exit the WHILE loop.
SET #bcontinue = 0
END
END
END
ELSE
BEGIN
WHILE #bcontinue=1
BEGIN
--If the delimiter is an empty string, check for remaining text
--instead of a delimiter. Insert the first character into the
--retArray table. Trim the character from the front of the string.
--Increment the index and loop.
IF DATALENGTH(#sText)>1
BEGIN
SET #value = SUBSTRING(#sText,1,1)
BEGIN
INSERT #retArray (idx, value)
VALUES (#idx, #value)
END
SET #idx = #idx+1
SET #sText = SUBSTRING(#sText,2,DATALENGTH(#sText)-1)
END
ELSE
BEGIN
--One character remains.
--Insert the character, and exit the WHILE loop.
INSERT #retArray (idx, value)
VALUES (#idx, #sText)
SET #bcontinue = 0
END
END
END
RETURN
END
GO
To test this try something like
select * from Split('abc,def,ghi', ',')
which will return

Mysql - Return table from stored procedure into a variable?

Thanks to this answer https://stackoverflow.com/a/8180159/16349298 , i'm able to translate
a string into a temporary table (usable for WHERE <id> IN <tmpTable>.<colomn>)
The only modification i made is at the end (The select) :
CREATE PROCEDURE stringToTmpTable(IN inputString VARCHAR(255), IN sep VARCHAR(255))
BEGIN
declare pos int; -- Keeping track of the next item's position
declare item varchar(100); -- A single item of the input
declare breaker int; -- Safeguard for while loop
-- The string must end with the delimiter
if right(inputString, 1) <> sep then
set inputString = concat(inputString, sep);
end if;
DROP TABLE IF EXISTS MyTemporaryTable;
CREATE TEMPORARY TABLE MyTemporaryTable ( columnName varchar(100) );
set breaker = 0;
while (breaker < 2000) && (length(inputString) > 1) do
-- Iterate looking for the delimiter, add rows to temporary table.
set breaker = breaker + 1;
set pos = INSTR(inputString, sep);
set item = LEFT(inputString, pos - 1);
set inputString = substring(inputString, pos + 1);
insert into MyTemporaryTable values(item);
end while;
SELECT * FROM MyTemporaryTable;
END
I would like to use this process in a function or procedure in order to call it in any procedure that needs it.
So here is the problem :
I don't know how to store the result of this procedure into a variable : i can't use the SELECT * INTO #p FROM ...; like CALL stringToTmpTable(<string>,<separator>) INTO #table;
An other way would be to add OUT parameter to stringToTmpTable() but it can't return multiple rows. Unfortunatly the amount of parameters in the string is variable so i can't define as much variable as there is parameters in the string.
Finally the FIND_IN_SET() isn't the solution i need.
In the worst case I could copy / past the stringToTmpTable() process in any other procedure that needs it, but that doesn't seem like the best way to me.
Any suggestions ?
"i'm able to translate a string into a temporary table" too, but I am using a different method:
SET #input = 'Banana, Apple, Orange, Pears';
WITH RECURSIVE cte1 as (
select
#input as s,
substring_index(substring_index(#input,',',1),',',-1) as w,
length(#input)-length(replace(#input,',','')) x
union all
select
substring_index(s,',',-x),
trim(substring_index(substring_index(substring_index(s,',',-x),',',1),',',-1)) as w,
x-1 x
from cte1 where s<>'' and x>0
)
select * from cte1
DBFIDDLE
But it's a bit of a problem to determine the real problem you have, which is causing you to ask this question. So this is not an answer, but just a different way of selecting all words from a comma-delimted string.

cursor logic in ssis

i have been strugling to convert the following t sql task into an automated ssis package.
i tried implementing using for loop conntainer but while creating the looping logic i am stucking off. i really appreciate of some one could help me out with logic
here is the t- sql code
-- SCRIPT TO BULK IMPORT MATERNAL HLA & INSERT CORD NIMA
IF OBJECT_ID('tempdb..#maternal_hla_from_solar') IS NOT NULL
BEGIN
drop table #maternal_hla_from_solar
END
CREATE TABLE #maternal_hla_from_solar(
[donor_id] [int] NOT NULL,
[antigen_id] [int] NOT NULL
)
-- bulk upload maternal HLA from file
BULK
INSERT #maternal_hla_from_solar
FROM 'C:\Users\zabeenp\Documents\SQL\populate_db\data\20111212_maternal_hla.txt'
WITH
(
FIRSTROW = 2,
MAXERRORS = 0,
FIELDTERMINATOR = '\t',
ROWTERMINATOR = '\n'
)
-- declare variables
DECLARE #donor_id int
DECLARE #maternal_hla varchar(5000)
DECLARE #definitive_type_date date = GETDATE()
DECLARE #modified_on datetime2(7) = GETDATE()
-- Cursor to process each row from #maternal_hla_from_solar
DECLARE c_insertCordNima CURSOR FAST_FORWARD
FOR select DISTINCT donor_id from #maternal_hla_from_solar -- important to use DISTINCT here
OPEN c_insertCordNima
FETCH NEXT FROM c_insertCordNima INTO #donor_id
WHILE ##FETCH_STATUS = 0
BEGIN
SET #maternal_hla = '' -- important to wipe string each time
-- concatenare maternal antigen_ids into one string
SELECT #maternal_hla = COALESCE(#maternal_hla + ',', '') + CONVERT(VARCHAR(10),antigen_id)
FROM #maternal_hla_from_solar
where donor_id = #donor_id
-- remove first comma
SET #maternal_hla = (select right(#maternal_hla,len(#maternal_hla)-1))
-- exec insertCordNIMA SP
EXECUTE insertCordNIMA
#donor_id
,#maternal_hla
,#definitive_type_date
,#modified_on
FETCH NEXT FROM c_insertCordNima INTO #donor_id
END
CLOSE c_insertCordNima
DEALLOCATE c_insertCordNima
maybe try to rewrite the cursor using a Recursive CTE?
http://msdn.microsoft.com/en-us/library/ms186243.aspx

Dynamic insert into variable table statement SQL Server

I have a variable table:
DECLARE #A_Table TABLE(ID INT, att1 VARCHAR(100), att2 nvarchar(200))
I want to make dynamic sql, so I insert into this table some data (all inside a loop):
WHILE (#i <= 100) BEGIN
SELECT #other_att = NAME FROM #other_Table where ID = #i;
SET #sql = 'INSERT ' + #A_Table+ '(ID,att1,att2) SELECT '+CAST(#i AS VARCHAR)+' , '''+ #other_att+''', SUM('+ #other_att') FROM '+ #EVEN_OTHER_Table;
EXEC (#sql);
END
sql every time would look like:
INSERT INTO #A_Table SELECT 1 , 'subject', SUM(subject)
INSERT INTO #A_Table SELECT 2 , 'age', SUM(age)
INSERT INTO #A_Table SELECT 3 , 'sex', SUM(sex)....
AND after executing this :
SO I will get:
#A_Table:
id att1 att2
1 subject 4.3
2 age 4.5
3 sex 4.1
but I get an error:
Msg 137, Level 16, State 1, Line 48
Must declare the scalar variable "#A_Table".
SO what is it the syntax to insert dynamically into a variable table?
Ok I have understood it.
You could use the INSERT ... EXEC syntax to insert the data returned by the dynamic SELECT. Of course, you would then need to remove the INSERT part from the dynamic statement.
WHILE (#i <= 100) BEGIN
SELECT #other_att = NAME FROM #other_Table where ID = #i;
SET #sql = 'SELECT '+CAST(#i AS VARCHAR)+' , ''' + #other_att+''', SUM('+ #other_att + ') FROM '+ #EVEN_OTHER_Table;
INSERT INTO #A_Table (ID,att1,att2)
EXEC (#sql);
END
You have a table variable, not a variable that contains the table name.
So you would need the following.
WHILE (#i <= 100) BEGIN
SELECT #other_att = NAME FROM #other_Table where ID = #i;
SET #sql = 'INSERT INTO #A_Table (ID,att1,att2) SELECT '+CAST(#i AS VARCHAR)+' , '''+ #other_att+''', SUM('+ #other_att') FROM #EVEN_OTHER_Table';
EXEC (#sql);
END
You would also need to declare the table variable as a statement inside the #sql variable, and execute your declare table and inserts together, or use a local/global temporary table.
With a local temporary table (stored in the tempdb) you could do something like this.
CREATE TABLE #testtbl (ID INT);
EXEC ('INSERT INTO #testtbl VALUES (1)');
SELECT * FROM #testtbl
DROP TABLE #testtbl
Some good info about temporary tables in BOL
http://msdn.microsoft.com/en-us/library/ms174979.aspx - quite far down the page
And the table type.
http://msdn.microsoft.com/en-us/library/ms175010.aspx
Your EXEC statement occurs in a different context and is therefore unaware of any variables created in your original context.
To create dynamic insert query it is really a task, I also struggle to find it ,finally I have tried in the following way and it's successfully working. Please find the code below.
CREATE PROCEDURE [dbo].[InsertTodaysData] (#tbl varchar(50),#Days int,
#MDate varchar(50), #EValue varchar(50), #Speed varchar(50),
#Totalreturn varchar(50),#Closingv varchar(50), #TotalReturnV varchar(50))
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;
DECLARE #SQLQuery varchar(2000)
-- Insert statements for procedure here
set #SQLQuery = 'INSERT INTO '+#tbl+' (ID,MDate,EValue,Speed,TotalReturnRatio,ClosingValue,
TotalReturnValue) VALUES ('+#Days+','''+#MDate+''', '+#EValue+', '+#Speed+',
'+#Totalreturn+', '+#Closingv+', '+#TotalReturnV+')'
EXECUTE(#SQLQuery)
END
Hope this will help you..

How can I pass an "array" of values to my stored procedure?

I want to be able to pass an "array" of values to my stored procedure, instead of calling "Add value" procedure serially.
Can anyone suggest a way to do it? am I missing something here?
Edit: I will be using PostgreSQL / MySQL, I haven't decided yet.
As Chris pointed, in PostgreSQL it's no problem - any base type (like int, text) has it's own array subtype, and you can also create custom types including composite ones. For example:
CREATE TYPE test as (
n int4,
m int4
);
Now you can easily create array of test:
select ARRAY[
row(1,2)::test,
row(3,4)::test,
row(5,6)::test
];
You can write a function that will multiply n*m for each item in array, and return sum of products:
CREATE OR REPLACE FUNCTION test_test(IN work_array test[]) RETURNS INT4 as $$
DECLARE
i INT4;
result INT4 := 0;
BEGIN
FOR i IN SELECT generate_subscripts( work_array, 1 ) LOOP
result := result + work_array[i].n * work_array[i].m;
END LOOP;
RETURN result;
END;
$$ language plpgsql;
and run it:
# SELECT test_test(
ARRAY[
row(1, 2)::test,
row(3,4)::test,
row(5,6)::test
]
);
test_test
-----------
44
(1 row)
If you plan to use MySQL 5.1, it is not possible to pass in an array.
See the MySQL 5.1 faq
If you plan to use PostgreSQL, it is possible look here
I don't know about passing an actual array into those engines (I work with sqlserver) but here's an idea for passing a delimited string and parsing it in your sproc with this function.
CREATE FUNCTION [dbo].[Split]
(
#ItemList NVARCHAR(4000),
#delimiter CHAR(1)
)
RETURNS #IDTable TABLE (Item VARCHAR(50))
AS
BEGIN
DECLARE #tempItemList NVARCHAR(4000)
SET #tempItemList = #ItemList
DECLARE #i INT
DECLARE #Item NVARCHAR(4000)
SET #tempItemList = REPLACE (#tempItemList, ' ', '')
SET #i = CHARINDEX(#delimiter, #tempItemList)
WHILE (LEN(#tempItemList) > 0)
BEGIN
IF #i = 0
SET #Item = #tempItemList
ELSE
SET #Item = LEFT(#tempItemList, #i - 1)
INSERT INTO #IDTable(Item) VALUES(#Item)
IF #i = 0
SET #tempItemList = ''
ELSE
SET #tempItemList = RIGHT(#tempItemList, LEN(#tempItemList) - #i)
SET #i = CHARINDEX(#delimiter, #tempItemList)
END
RETURN
END
You didn't indicate, but if you are referring to SQL server, here's one way.
And the MS support ref.
For PostgreSQL, you could do something like this:
CREATE OR REPLACE FUNCTION fnExplode(in_array anyarray) RETURNS SETOF ANYELEMENT AS
$$
SELECT ($1)[s] FROM generate_series(1,array_upper($1, 1)) AS s;
$$
LANGUAGE SQL IMMUTABLE;
Then, you could pass a delimited string to your stored procedure.
Say, param1 was an input param containing '1|2|3|4|5'
The statement:
SELECT CAST(fnExplode(string_to_array(param1, '|')) AS INTEGER);
results in a result set that can be joined or inserted.
Likewise, for MySQL, you could do something like this:
DELIMITER $$
CREATE PROCEDURE `spTest_Array`
(
v_id_arr TEXT
)
BEGIN
DECLARE v_cur_position INT;
DECLARE v_remainder TEXT;
DECLARE v_cur_string VARCHAR(255);
CREATE TEMPORARY TABLE tmp_test
(
id INT
) ENGINE=MEMORY;
SET v_remainder = v_id_arr;
SET v_cur_position = 1;
WHILE CHAR_LENGTH(v_remainder) > 0 AND v_cur_position > 0 DO
SET v_cur_position = INSTR(v_remainder, '|');
IF v_cur_position = 0 THEN
SET v_cur_string = v_remainder;
ELSE
SET v_cur_string = LEFT(v_remainder, v_cur_position - 1);
END IF;
IF TRIM(v_cur_string) != '' THEN
INSERT INTO tmp_test
(id)
VALUES
(v_cur_string);
END IF;
SET v_remainder = SUBSTRING(v_remainder, v_cur_position + 1);
END WHILE;
SELECT
id
FROM
tmp_test;
DROP TEMPORARY TABLE tmp_test;
END
$$
Then simply CALL spTest_Array('1|2|3|4|5') should produce the same result set as the above PostgreSQL query.
Thanks to JSON support in MySQL you now actually have the ability to pass an array to your MySQL stored procedure. Create a JSON_ARRAY and simply pass it as a JSON argument to your stored procedure.
Then in procedure, using MySQL's WHILE loop and MySQL's JSON "pathing" , access each of the elements in the JSON_ARRAY and do as you wish.
An example here https://gist.githubusercontent.com/jonathanvx/513066eea8cb5919b648b2453db47890/raw/22f33fdf64a2f292688edbc67392ba2ccf8da47c/json.sql
Incidently, here is how you would add the array to a function (stored-proc) call:
CallableStatement proc = null;
List<Integer> faultcd_array = Arrays.asList(1003, 1234, 5678);
//conn - your connection manager
conn = DriverManager.getConnection(connection string here);
proc = conn.prepareCall("{ ? = call procedureName(?) }");
proc.registerOutParameter(1, Types.OTHER);
//This sets-up the array
Integer[] dataFaults = faultcd_array.toArray(new Integer[faultcd_array.size()]);
java.sql.Array sqlFaultsArray = conn.createArrayOf("int4", dataFaults);
proc.setArray(2, sqlFaultsArray);
//:
//add code to retrieve cursor, use the data.
//: