I have developed SSIS package where in I have script component and Data flow task. Script component take 2 input variables namely Db2Con, SQLCon which are DB2 connection string and SQL Connection String. In this script component, I am dynamically generating Query and saving its value to the output variable namely DB2Qry. Now, In Data flow task, I am using this variable for source component.But it is not allowing me to do so with an Error "SSIS Error Code DTS_E_OLEDBERROR". In script component, I have made this variable as 'ReadWrite Variable.' Please suggest if I am missing something.
SQL's EvaluateAsExpression should be set to FALSE. You're not putting an expression into the variable, As you're assigning a static value through your script.
and you need to manually set a "default" value for your SQL variable in the Variables pane, otherwise the OLE DB Source has nothing to use for the SQL statement at design time
Related
I have a parent child package situation where the parent is reading a SQL Server task table. Loops through the tasks and gets the connection string needed for the the task and passes it to the child package. I have created and set a variable in the parent package. I have also created and set a variable n the child package in a Script Task. The Variable is set to the Connection String of my OLE_DB connection in an expression (Package.DelayValidation = True). After the Script task runs(and works..I displayed a message box with the correct value) I run a Data Flow Task which trys to read the database using the connection string. This is where the "error code dts_e_cannotacquireconnectionfromconnectionmanager" happens. I know I have to be missing something, just can not put my finger on it.
Since both packages are available in a single project, try to select Project Reference ,in the parent package, as the ReferenceType and in the PackageNameFromProjectReference enter your child .
I can see and I can modify the values of the project variables, but my question is ... Can I modify the value of the parameters Project.params or Package Params? I do not find that option.
For example, I create a project parameter called StopExecution of type Boolean with its value in False, and when an error occurs in one of my SSIS steps, can I change the value of the Project.param StopExecution to True?, with this functionality in my other package called Package2.dtsx, before starting its execution when I finish my first package called Package1.dtsx (both are executed for a SQL job) Can I validate if my Package2.dtsx has to be executed?
Can you help me? I can do that?
No. Project Parameters are read-only; you can specify its value at package start.
You are looking for some global parameter which you can change in child package. File, SQL table or Web service could be such global storage; you write some value to an SQL table and read it later.
There is a guideline how to use SSIS parent package variables from child package and update parent package variable from child. You can try it, but this scenario relies on variable scope hack.
I have a data flow task in my foreach loop container. Every round, I reassign 3 varaiables: User::Date, User::Location, and User::Organization. I have another variable User:Query that has an expression using these variables in where clause.
I use this User:Query in the OLE DB Source of the data flow task, but in each foreach round, it uses the same default values for these variables. Why?
OLE DB Provider: OLE DB Provider for Analysis Services 10.0
And on break point, I can see the query variable is actually showing correct, but the source doesn't use the updated variable.
Finally figured out: I didn't change EvaluateAsExpression property to True for my User::Query variable.
I have a scenario where i need to pass an text file or excel file column as an parameter to my Sql Query in SSIS Package.
My Text or excel file has a column called Policy_no and it has more than 1000+ policy_no(EX: 12358685). i have an Sql script *select * from main_table where policy_no = ?*. And that that '?' has to be come from my package variable(txt or excel ).
Instead of manually writing script for each and every policy, how can we achieve this through SSIS.
Thanks
Assuming you want to loop through each row in your file and run the query against each individual value, you can use a Data Flow task to read your text file and load the policy numbers in an ADO Recordset (declared as a package variable). Next, you'd use a Foreach Loop Container to iterate through the recordset, loading each policy number in turn into a second variable and then executing your query and doing whatever other work is needed.
See Use a Recordset Destination in MSDN for an overview and example.
You can use EXECUTE SQL TASK (Connect Excel with OLE DB Connection) to get "Policy_no" data from Excel, then store the result into a variable, say:policyNoGroup, whose data type should be Object, then use For Each Loop to loop though variable policyNoGroup, see the example: http://www.codeproject.com/Articles/14341/Using-the-Foreach-ADO-Enumerator-in-SSIS
I am working with SSIS 2008. I have a select query name sqlquery1 that returns some rows:
aq
dr
tb
This query is not implemented on the SSIS at the moment.
I am calling a stored procedure from an OLE DB Source within a Data Flow Task. I would like to pass the data obtained from the query to the stored procedure parameter.
Example:
I would like to call the stored procedure by passing the first value aq
storedProdecure1 'aq'
then pass the second value dr
storedProdecure1 'dr'
I guess it would be something like a cycle. I need this because the data generated by the OLE DB Source through the stored procedure needs to be sent to another destination and this must be done for each record of the sqlquery1.
I would like to know how to call the query sqlquery1 and pass its output to call another stored procedure.
How do I need to do this in SSIS?
Conceptually, what your solution will look like is an execute your source query to generate your result set. Store that into a variable and then you'll need to do iterate through those results and for each row, you'll want to call your stored procedure with that row's value and send the results into a new Excel file.
I'd envision your package looking something like this
An Execute SQL Task, named "SQL Load Recordset", attached to a Foreach Loop Container, named "FELC Shred Recordset". Nested inside there I have a File System Task, named "FST Copy Template" which is a precedence for a Data Flow Task, named "DFT Generate Output".
Set up
As you're a beginner, I'm going to try and explain in detail. To save yourself some hassle, grab a copy of BIDSHelper. It's a free, open source tool that improves the design experience in BIDS/SSDT.
Variables
Click on the background of your Control Flow. With nothing selected, right-click and select Variables. In the new window that pops up, click the button that creates a New Variable 4 times. The reason for clicking on nothing is that until SQL Server 2012, the default behaviour of variable creation is to create them at the scope of the current object. This has resulted in many lost hairs for new and experienced developers alike. Variable names are case sensitive so be aware of that as well.
Rename Variable to RecordSet. Change the Data type from Int32 to Object
Rename Variable1 to ParameterValue. Change the data type from Int32 to String
Rename Variable2 to TemplateFile. Change the data type from Int32 to String. Set the value to the path of your output Excel File. I used C:\ssisdata\ShredRecordset.xlsx
Rename Variable 4 to OutputFileName. Change the data type from Int32 to String. Here we're going to do something slightly advanced. Click on the variable and hit F4 to bring up the Properties window. Change the value of EvaluateAsExpression to True. In Expression, set it to "C:\\ssisdata\\ShredRecordset." + #[User::ParameterValue] + ".xlsx" (or whatever your file and path are). What this does, is configures a variable to change as the value of ParameterValue changes. This helps ensure we get a unique file name. You're welcome to change naming convention as needed. Note that you need to escape the \ any time you are in an expression.
Connection Managers
I have made the assumption you are using an OLE DB connection manager. Mine is named FOO. If you are using ADO.NET the concepts will be similar but there will be nuances pertaining to parameters and such.
You will also need a second Connection Manager to handle Excel. If SSIS is temperamental about data types, Excel is flat out psychotic-stab-you-in-the-back-with-a-fork-while-you're-sleeping about data types. We're going to wait and let the data flow actually create this Connection Manager to ensure our types are good.
Source Query to Result Set
The SQL Load Recordset is an instance of the Execute SQL Task. Here I have a simple query to mimic your source.
SELECT 'aq' AS parameterValue
UNION ALL SELECT 'dr'
UNION ALL SELECT 'tb'
What's important to note on the General tab is that I have switched my ResultSet from None to Full result set. Doing this makes the Result Set tab go from being greyed out to usable.
You can observe that I have assigned the Variable Name to the variable we created above (User::RecordSet) and I the Result Name is 0. That is important as the default value, NewResultName doesn't work.
FELC Shred Recordset
Grab a Foreach Loop Container and we will use that to "shred" the results that were generated in the preceding step.
Configure the enumerator as a Foreach ADO Enumerator Use User::RecordSet as your ADO object source variable. Select rows in the first table as your Enumeration mode
On the Variable Mappings tab, you will need to select your variable User::ParameterValue and assign it the Index of 0. This will result in the zerotth element in your recordset object being assigned to the variable ParameterValue. It is important that you have data type agreement as SSIS won't do implicit conversions here.
FST Copy Template
This a File System Task. We are going to copy our template Excel File so that we have a well named output file (has the parameter name in it). Configure it as
IsDestinationPathVariable: True
DestinationVarible: User::OutputFileName
OverwriteDestination: True
Operation: Copy File
IsSourcePathVariable: True
SourceVariable: User::TemplateFile
DFT Generate Output
This is a Data Flow Task. I'm assuming you're just dumping results straight to a file so we'll just need an OLE DB Source and an Excel Destination
OLEDB dbo_storedProcedure1
This is where your data is pulled from your source system with the parameter we shredded in the Control Flow. I am going to write my query in here and use the ? to indicate it has a parameter.
Change your Data access mode to "SQL Command" and in the SQL command text that is available, put your query
EXECUTE dbo.storedProcedure1 ?
I click the Parameters... button and fill it out as shown
Parameters: #parameterValue
Variables: User::ParameterValue
Param direction: Input
Connect an Excel Destination to the OLE DB Source. Double click and in the Excel Connection Manager section, click New... Determine if you're needing 2003 or 2007 format (.xls vs .xlsx) and whether you want your file to have header rows. For you File Path, put in the same value you used for your #User::TemplatePath variable and click OK.
We now need to populate the name of the Excel Sheet. Click that New... button and it may bark that there is not sufficient information about mapping data types. Don't worry, that's semi-standard. It will then pop up a table definition something like
CREATE TABLE `Excel Destination` (
`name` NVARCHAR(35),
`number` INT,
`type` NVARCHAR(3),
`low` INT,
`high` INT,
`status` INT
)
The "table" name is going to be the worksheet name, or precisely, the named data set in the worksheet. I made mine Sheet1 and clicked OK. Now that the sheet exists, select it in the drop down. I went with the Sheet1$ as the target sheet name. Not sure if it makes a difference.
Click the Mappings tab and things should auto-map just fine so click OK.
Finally
At this point, if we ran the package it would overwrite the template file every time. The secret is we need to tell that Excel Connection Manager we just made that it needs to not have a hard coded name.
Click once on the Excel Connection Manager in the Connection Managers tab. In the Properties window, find the Expressions section and click the ellipses ... Here we will configure the Property ExcelFilePath and the Expression we will use is
#[User::OutputFileName]
If your icons and such look different, that's to be expected. This was documented using SSIS 2012. Your work flow will be the same in 2005 and 2008/2008R2 just the skin is different.
If you run this package and it doesn't even start and there is an error about the ACE 12 or Jet 4.0 something not available, then you are on a 64bit machine and need to tell BIDS/SSDT that you want to run in 32 bit mode.
Ensure the Run64BitRuntime value is False. This project setting can be found by right clicking on the project, expand the Configuration Properties and it will be an option under Debugging.
Further reading
A different example of shredding a recordset object can be found on How to automate the execution of a stored procedure with an SSIS package?