Just wandering, is there any way that I can assign multiple rows data in a variable in SSIS?
example:
I have the following table (tableA) with the following data created by the data flow task:
DataRow
Jay,10,11 Happy St\n\n
David,12,13 Angel St\n\n
Tom,30,23 Betman St\n\n
How can I able to assign those records into a variable in SSIS as below:
Jay,10,11 Happy St\n\nDavid,12,13 Angel St\n\nTom,30,23 Betman St\n\n
Then pass it into the Web service task. At the moment I'm running the web service task into the loop, but I would like to compile all the data row and pass over to the web service task outside the loop
Any way that I can do it? Any example are link could share?
If you can, take the source data using an "Execute SQL Task". If that is not possible you can create a temporary table and then populate this table in your data flow task. The key is to store your result set in a varible of type "Object" and then you can iterate it into a loop task and send every row to the Web Service task as you want.
Here you could find a detailed tutorial about this:
Loop through ADO recordset in SSIS
If you need help with the temporary table just let me know.
Hope that helps,
Paul
Related
I have a requirement where I have execute a MYSQL procedure (procedure name is same across all the sites) for different sites, I have a For each loop which get the server details and database details and within the Foreach Loop container I have dataflow task, in the data flow task I created a source connection for a MySql connection and calling the stored procedure like below,
CALL spITProd( '2022-09-25 20:04:22.847000000', '2022-10-25 20:04:22.847000000' );
This gives different metadata for different sites, to be precise first 16 columns are same for all the sites and after that number of columns and column names can vary, my requirement is to store first 16 columns in a destination table and all other columns in a different table by unpivoting them, I was able to achieve this for one site as shown below SSIS data flow task,
I want to automate this for process for all the sites, I know I can create multiple data flow tasks but that's not what I want to do, when source connects to the new site it fails as it encounters new metadata (as I said number of columns and column name changes after first 16 columns), Please suggest any ideas on how to dynamically read the columns from the procedure. I don't have any permission son the MYSQL database it is managed by the vendor I just have permission to the call the procedure, I tried to store the data into System.Object variable but I am not sure how to use that later as I cant create a temp table on destination using that, Appreciate any ideas on how to handle this requirement.
Mentioned the details above
What is the major use of recordset destination in SSIS?I heard that it is an in-memory,so the variable which is holding the data is it in raw format? Can someone explain the explain me the real time project use of Recordset destination?
A recordset destination can be used for just about anything you can think of. Some common uses I hear is to use the recordset in a foreach loop. Say you want to export several "categories" from a transaction table. Perhaps you get a recordset of the categories that exist and then call a new dataflow to export that category as it's own file. Or perhaps date ranges, months, etc.
One way I use it is in a script task to perform an action on the data that SSIS cannot do natively. I was using a script component but this particular task ran into a concurrency issue. So by dumping to a recordset I was able to use the recordset in a script task to do the logic in a manner to avoid that issue.
Another script task use is to build and send HTML emails.
I suppose a use for it might be when you have 1 data flow to get 1 record set then do a bunch of non dataflow tasks and then use that as a source in another data flow task, but that is not something I have ever done.
I have got around 35 tables whose data need to be migrated from SQL Server to MySQL. I am using SSIS for this project and I have set up a control flow (using Load Multiple Tables) with a Script Task and a Foreach Loop Container that iterates through all the tables in my database. What I now need to do is convert the data type for some of the columns, in some of the tables, to 'Unicode String [DT_WSTR]' before I dump them in my destination tables. Is this something that can be done through SSIS? If so, any pointers or a set of instructions would be great.
Thanks,
Pratik Gandhi
Yes, this is a standard out-of-the-box task for SSIS.
Add a Data Flow Task.
Add a Data conversion component to the task
Add your source and destination servers
Map your columns, converting datatypes where required.
As always, MSDN provides further help.
I have read some blogs that give an idea about passing the full result set into a variable and then sending the variable to a for each loop container. I am stuck there. Basically I have two tables: 1. student_Class_timetable 2. students_exams_timetable.
I have a sql query that finds out the clashes between these two tables for. eg. say a student has to attend a class today between 10 to 12 but he also has to write an exam at the same time.
So I am doing an execute sql task and saving the full result set in a variable called studentlist. I now need to save the list of all these students in an audit log table before deleting them from student_Class_timetable.
ssis_screenshot
I don't know how to pass the Studentlist variable to a data flow task.
Could you please suggest a way to do that
If I were doing that, I would consider storing the data in a Recordset Destination with an Object Type VariableName and using a Foreach ADO Enumerator on that variable. A good example can be seen here.
Can somebody please help me to transfer around 15 tables from one database to another database. At present I can do this one by one using Data Flow task, but then I need to do this task 15 times which is very time consuming.
Why don't you just use a task? Maybe tasks->export is what you're looking for.
Otherwise you'll need to create separate blocks for each table or:
Create a variable of type object
Script Task: Add to your list all table names.
Iterate over this object variable with For each loop container
Inside the loop create a source from a variable. In this variable specify the connection dynamically depending on the current loop value.
you can use SSIS package, select Transfer SQL server objects from SSIS toolbox , in Object specify the source and destination servers and database. for copyAllObjects make it false . ObjectToCopy select CopyAllTables true or make it false and pick from the list the table you want to copy.