SSIS newbie question:
I have found different usages of a user variable and hence the confusion.
I have defined a user variable customerName.
In some places, (e.g. Script Task Editor's ReadOnlyVariables, it being referred as User::customerName since there is no option to select customerName.
Then in Script Task (Edit script), I see that its being referred as
string custName = Dts.Variables["customerName"].Value.ToString();
and not as
string custName = Dts.Variables["User::customerName"].Value.String();
Where and how does this work? Is there any rule?
Either syntax should be acceptable in a script task. To further complicate matters, you could also reference it as string custName = Dts.Variables[0].Value.ToString();
The User bit refers to the namespace of an SSIS Variable. By default, you can access Variables in the System and User namespace but nothing prevents you from creating variables in a different spaace.
You're also going to run into #[User::customerName] syntax.
Related
I want to insert values into database when the biml code is ran and the package has completed expansion is this possible using BIML or c#?
I have a table called BIML expansion created in my DB and I have test.biml which loads the package test.dtsx whenever the BIML expansion is completed a record should be inserted into my table that expansion has been completed.
Let me know if you have any questions or needs any additional info.
From comments
I tried your code
string connectionString = "Data Source=hq-dev-sqldw01;Initial Catalog=IM_Stage;Integrated Security=SSPI;Provider=SQLNCLI11.1";
string SrcTablequery=#"INSERT INTO BIML_audit (audit_id,Package,audit_Logtime) VALUES (#audit_id, #Package,#audit_Logtime)";
DataTable dt = ExternalDataAccess.GetDataTable(connectionString,SrcTablequery);
It has an error below must declare the scalar variable audit_id can you let me know the issue behind it?
In it's simplest form, you'd have content like this in your Biml script
// Define the connection string to our database
string connectionStringSource = #"Server=localhost\dev2012;Initial Catalog=AdventureWorksDW2012;Integrated Security=SSPI;Provider=SQLNCLI11.1";
// Define the query to be run after *ish* expansion
string SrcTableQuery = #"INSERT INTO dbo.MyTable (BuildDate) SELECT GETDATE()";
// Run our query, nothing populates the data table
DataTable dt = ExternalDataAccess.GetDataTable(connectionStringSource, SrcTableQuery);
Plenty of different ways to do this - you could have spun up your own OLE/ADO connection manager and used the class methods. You could have pulled the connection string from the Biml Connections collection (depending on the tier this is executed in), etc.
Caveats
Depending on the product (BimlStudio vs BimlExpress), there may be a background process compiling your BimlScript to ensure all the metadata is ready for intellisense to pick it up. You might need to stash that logic into a very high tiered Biml file to ensure it's only called when you're ready for it. e.g.
<## template tier="999" #>
<#
// Define the connection string to our database
string connectionStringSource = #"Server=localhost\dev2012;Initial Catalog=AdventureWorksDW2012;Integrated Security=SSPI;Provider=SQLNCLI11.1";
// Define the query to be run after *ish* expansion
string SrcTableQuery = #"INSERT INTO dbo.MyTable (BuildDate) SELECT GETDATE()";
// Run our query, nothing populates the data table
DataTable dt = ExternalDataAccess.GetDataTable(connectionStringSource, SrcTableQuery);
#>
Is that the problem you're trying to solve?
Addressing comment/questions
Given the query of
string SrcTablequery=#"INSERT INTO BIML_audit (audit_id,Package,audit_Logtime) VALUES (#audit_id, #Package,#audit_Logtime)";
it errors out due to #audit_id not being specified. Which makes sense - this query specifies it will provide three variables and none are provided.
Option 1 - the lazy way
The quickest resolution would be to redefine your query in a manner like this
string SrcTablequery=string.Format(#"INSERT INTO BIML_audit (audit_id,Package,audit_Logtime) VALUES ({0}, '{1}', '{2})'", 123, "MyPackageName", DateTime.Now);
I use the string library's Format method to inject the actual values into the placeholders. I assume that audit_id is a number and the other two are strings thus the tick marks surrounding 1 and 2 there. You'd need to define a value for your audit id but I stubbed in 123 as an example. If I were generating packages, I'd likely have a variable for my packageName so I'd reference that in my statement as well.
Option 2 - the better way
Replace the third line with .NET library usage much as you see in heikofritz on using parameters inserting data into access database.
1) Create a database Connection
2) Open connection
3) Create a command object and associate with the connection
4) Specify your statement (use ? as your ordinal marker instead of named parameters since this is oledb)
5) Create an Parameter list and associate with values
Many, many examples out there beyond the referenced but it was the first hit. Just ignore the Access connection string and use your original value.
My purpose is I want to change the value of this Application Scope Connection String using Code.
So, I have this dataset database for my vb.net project. and it's fully working
(fully rendered) when we're running the app.
If you open the project setting, it is stored on Connection String.
It is using Applicatio Scope. Not user Scope. ANd it's
saved with its value ; "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=C:\ProgramData\RK-Medis\db-used.mdb"
My questions are:
Can we change that variable value of Connection String before its
called / rendered on its GUI?
WHat are the code for changing it?
If no we can't change it, then what are the alternative for using
dataset + its Connection String that automatically used in almost
many tables (GUI) i have...?
Perhaps you need a tableConnectionString for example, to store the CS for each user. Then use a property of string(or maybe const variable) to store your connection string for that particular user during user login. Substitude the CS to your application. Then each of the user will be able to have different CS.
Hope its give you the idea.
Newbie question. I believe this is one of the most common errors. I found several of them on msdn forum itself but probably there are many ways on achieving this error? Please help.
I am trying to move and rename some images from one folder to another (and yes I have seen the blog by Rafael Salas and many others but none of them helps).
Like moving from \server1\images\123-456.jpg to \server2\images\123456.jpg
I am using a foreach.
Source variable is built dynamically. In the first iteration #imagePath = \server1\images\123-456.jpg ( i checked using messagebox.show)
I have defined #remoteImagePath = \server2\images\ (which never changes) and #revisedImageName = 123456.jpg (built dynamically in script task using string replace - also checked using messagebox.show)
In FileSystem Task, I am using SourceVariable as #imagePath and using Expressions to define Destination as in #[User::remoteImagePath] + "\" + #[User::revisedImageName]
Dont know for what reason, I am getting this error
Failed to lock variable "\server2\images\123456.jpg" for read access with error 0xC0010001 "The variable cannot be found. This occurs when an attempt is made to retrieve a variable from the Variables collection on a container during execution of the package, and the variable is not there. The variable name may have changed or the variable is not being created.".
Set the IsSourcePathVariable to True and set the SourceVariable to a SINGLE variable.
If you use the expression editor to set Source using multiple variables, or anything else apart from a SINGLE variable it will not work. If you want to concatenate hard-coded strings to variables, or multiple variables, do this by creating a new variable in the package, and use the Expressions tab in the variables tab to build the variable as a single variable.
I believe expressions editor needs syntax
#[User::remoteImagePath] + "\\" + #[User::revisedImageName]
I am working with SSIS 2008. I have a select query name sqlquery1 that returns some rows:
aq
dr
tb
This query is not implemented on the SSIS at the moment.
I am calling a stored procedure from an OLE DB Source within a Data Flow Task. I would like to pass the data obtained from the query to the stored procedure parameter.
Example:
I would like to call the stored procedure by passing the first value aq
storedProdecure1 'aq'
then pass the second value dr
storedProdecure1 'dr'
I guess it would be something like a cycle. I need this because the data generated by the OLE DB Source through the stored procedure needs to be sent to another destination and this must be done for each record of the sqlquery1.
I would like to know how to call the query sqlquery1 and pass its output to call another stored procedure.
How do I need to do this in SSIS?
Conceptually, what your solution will look like is an execute your source query to generate your result set. Store that into a variable and then you'll need to do iterate through those results and for each row, you'll want to call your stored procedure with that row's value and send the results into a new Excel file.
I'd envision your package looking something like this
An Execute SQL Task, named "SQL Load Recordset", attached to a Foreach Loop Container, named "FELC Shred Recordset". Nested inside there I have a File System Task, named "FST Copy Template" which is a precedence for a Data Flow Task, named "DFT Generate Output".
Set up
As you're a beginner, I'm going to try and explain in detail. To save yourself some hassle, grab a copy of BIDSHelper. It's a free, open source tool that improves the design experience in BIDS/SSDT.
Variables
Click on the background of your Control Flow. With nothing selected, right-click and select Variables. In the new window that pops up, click the button that creates a New Variable 4 times. The reason for clicking on nothing is that until SQL Server 2012, the default behaviour of variable creation is to create them at the scope of the current object. This has resulted in many lost hairs for new and experienced developers alike. Variable names are case sensitive so be aware of that as well.
Rename Variable to RecordSet. Change the Data type from Int32 to Object
Rename Variable1 to ParameterValue. Change the data type from Int32 to String
Rename Variable2 to TemplateFile. Change the data type from Int32 to String. Set the value to the path of your output Excel File. I used C:\ssisdata\ShredRecordset.xlsx
Rename Variable 4 to OutputFileName. Change the data type from Int32 to String. Here we're going to do something slightly advanced. Click on the variable and hit F4 to bring up the Properties window. Change the value of EvaluateAsExpression to True. In Expression, set it to "C:\\ssisdata\\ShredRecordset." + #[User::ParameterValue] + ".xlsx" (or whatever your file and path are). What this does, is configures a variable to change as the value of ParameterValue changes. This helps ensure we get a unique file name. You're welcome to change naming convention as needed. Note that you need to escape the \ any time you are in an expression.
Connection Managers
I have made the assumption you are using an OLE DB connection manager. Mine is named FOO. If you are using ADO.NET the concepts will be similar but there will be nuances pertaining to parameters and such.
You will also need a second Connection Manager to handle Excel. If SSIS is temperamental about data types, Excel is flat out psychotic-stab-you-in-the-back-with-a-fork-while-you're-sleeping about data types. We're going to wait and let the data flow actually create this Connection Manager to ensure our types are good.
Source Query to Result Set
The SQL Load Recordset is an instance of the Execute SQL Task. Here I have a simple query to mimic your source.
SELECT 'aq' AS parameterValue
UNION ALL SELECT 'dr'
UNION ALL SELECT 'tb'
What's important to note on the General tab is that I have switched my ResultSet from None to Full result set. Doing this makes the Result Set tab go from being greyed out to usable.
You can observe that I have assigned the Variable Name to the variable we created above (User::RecordSet) and I the Result Name is 0. That is important as the default value, NewResultName doesn't work.
FELC Shred Recordset
Grab a Foreach Loop Container and we will use that to "shred" the results that were generated in the preceding step.
Configure the enumerator as a Foreach ADO Enumerator Use User::RecordSet as your ADO object source variable. Select rows in the first table as your Enumeration mode
On the Variable Mappings tab, you will need to select your variable User::ParameterValue and assign it the Index of 0. This will result in the zerotth element in your recordset object being assigned to the variable ParameterValue. It is important that you have data type agreement as SSIS won't do implicit conversions here.
FST Copy Template
This a File System Task. We are going to copy our template Excel File so that we have a well named output file (has the parameter name in it). Configure it as
IsDestinationPathVariable: True
DestinationVarible: User::OutputFileName
OverwriteDestination: True
Operation: Copy File
IsSourcePathVariable: True
SourceVariable: User::TemplateFile
DFT Generate Output
This is a Data Flow Task. I'm assuming you're just dumping results straight to a file so we'll just need an OLE DB Source and an Excel Destination
OLEDB dbo_storedProcedure1
This is where your data is pulled from your source system with the parameter we shredded in the Control Flow. I am going to write my query in here and use the ? to indicate it has a parameter.
Change your Data access mode to "SQL Command" and in the SQL command text that is available, put your query
EXECUTE dbo.storedProcedure1 ?
I click the Parameters... button and fill it out as shown
Parameters: #parameterValue
Variables: User::ParameterValue
Param direction: Input
Connect an Excel Destination to the OLE DB Source. Double click and in the Excel Connection Manager section, click New... Determine if you're needing 2003 or 2007 format (.xls vs .xlsx) and whether you want your file to have header rows. For you File Path, put in the same value you used for your #User::TemplatePath variable and click OK.
We now need to populate the name of the Excel Sheet. Click that New... button and it may bark that there is not sufficient information about mapping data types. Don't worry, that's semi-standard. It will then pop up a table definition something like
CREATE TABLE `Excel Destination` (
`name` NVARCHAR(35),
`number` INT,
`type` NVARCHAR(3),
`low` INT,
`high` INT,
`status` INT
)
The "table" name is going to be the worksheet name, or precisely, the named data set in the worksheet. I made mine Sheet1 and clicked OK. Now that the sheet exists, select it in the drop down. I went with the Sheet1$ as the target sheet name. Not sure if it makes a difference.
Click the Mappings tab and things should auto-map just fine so click OK.
Finally
At this point, if we ran the package it would overwrite the template file every time. The secret is we need to tell that Excel Connection Manager we just made that it needs to not have a hard coded name.
Click once on the Excel Connection Manager in the Connection Managers tab. In the Properties window, find the Expressions section and click the ellipses ... Here we will configure the Property ExcelFilePath and the Expression we will use is
#[User::OutputFileName]
If your icons and such look different, that's to be expected. This was documented using SSIS 2012. Your work flow will be the same in 2005 and 2008/2008R2 just the skin is different.
If you run this package and it doesn't even start and there is an error about the ACE 12 or Jet 4.0 something not available, then you are on a 64bit machine and need to tell BIDS/SSDT that you want to run in 32 bit mode.
Ensure the Run64BitRuntime value is False. This project setting can be found by right clicking on the project, expand the Configuration Properties and it will be an option under Debugging.
Further reading
A different example of shredding a recordset object can be found on How to automate the execution of a stored procedure with an SSIS package?
In Reporting Services I would like to add a parameter that contains data from a custom code block. Ideally, I would be able to run the following code (this is a simple testing example):
Function GetPeriods() As String()
Dim values As System.Collections.ArrayList =
New System.Collections.ArrayList()
For i as integer = 1 to 24
values.Add(i)
Next
Return values.ToArray()
End Function
and put the following in the "Text Field" of the parameter:
=Code.GetPeriods()
However, when I run the report, the parameter I apply this to is disabled and empty. Is there a different technique that should be used? Or am I doing something wrong?
If you're using SQL 2008 Reporting Services then you can have a look at this page which introduces the concept of using custom assemblies.
If you're using SQL 2005 Reporting Services then this link is the one you want.
It's a mostly trivial thing, simply compile your code into a class library and follow the instructions provided to allow your report to reference it.
You are returning an array item (an array of strings) into a text field. Instead, try returning a plain string. That should work. If you would still like to return an array list, you must basically bind it to a list control in your RDL. You can definitely do that with dataset extensions. However, I am not sure if there is any other easy way. Check the proprties of the list control and see if it allows you to directly bind to an array list.
You can create the same stored procedure on SQL Server and load parameter values from that procedure.
To access your members/functions implemented in custom code of SSRS report you should set the access modifier to "Public":
Public Function GetPeriods() As String
...
see article Writing Custom Code in SQL Server Reporting Services
I've been trying to do this same thing, set a simple list of parameter values from report code. None of the links in any of these answers shows how to do this and after quite a bit of digging around I don't think it's even possible. Yes it is possible to get the values from a database query, from a web service, or from a custom assembly, but each of these creates a lot of overhead compared to getting the list from a simple function call like =Code.GetValues(), where the function uses a For loop to create the values.
msvcyc is correct in pointing out that the parameter is expecting a string value, but the function is returning an array. I changed the return type to Array as suggested by prashant sable, but the select list is still grayed out, it does not work. And coldice is correct in saying that the access modifier should be Public.
In my digging around I found an article by James Kovac from 2005 that pointed out why this is not possible. The Parameters class has a get method, but no set method. In the VS 2008 object browser for SSRS 2008 the object name has changed, but it still does not contain a set method (see Microsoft.ReportingServices.Interfaces.IParameter.Name or .Value).
My current workaround is to just hard code the list of values, but if your value list needs to be dynamic then your only choices are database queries, web services, or custom assemblies. I think the easiest workaround of these three is to get the values from the database engine, as suggested by oleksiy.t, as long as you can write a query to return the value list you want. Your list of integers, or my list of time intervals, would both be easy queries to write. Otherwise you will need to use one of the other two workarounds.
I checked your code. The only thing that's wrong is your function returns String(). When I changed your method signature to return Array, it worked fine, in my report.
Change the signature to Function GetPeriods() As Array
Everything I've seen requires parameters and their respective settings to be part of the RDL.
That being said, if you're going to "hardcode" the values, you could create a dataset just for the report, perhaps in XML, or if it needs to be programmatically driven, do it in a web service.