Using SSIS 2012 package parameters for connection properties - ssis

I am trying to write ETL that collects data from many identical server into a central repository. What I'm trying to do is write one package with source address, user id and password as parameters and execute the package once per server to be copied.
Is this doable? How can I use parameters to create a source?

I meant to ask how to parametrize the connection manager (is that even a real word?), not where to store the connection parameters. The answer is simple:
Create package parameters for Server, Database, User ID and Password
Create a connection manager as part of defining a data flow component
once a connection is defined, right-click on the connection manager at the bottom of the package design screen and select "Parametrize".
Select "ServerName" in the property drop-down
Select "Use existing parameter" or create new parameter if skipped step 1
If using existing parameter, select it from the drop down
Click OK to save (gotta do it after each parameter)
Repeat steps 4-7 for the rest of the parameters

You can store parameters in a table. Query the table with a sql task and store the results in a object variable. You can then use this variable in a for loop. Use expressions in SSIS to change values of your connection during each loop iteration.
Several books outline this method. Here is a code example.
Here are some steps - hopefully I didn't miss anything. You mention a server "Address", but I'm not sure exactly what you are trying to do. This example queries multiple sql servers.
You create the variables, SQL_RS with type of object, SRV_Conn with type of string. This holds my servername. In the execute SQL task, I have a query which returns the names of sql servers I want to query. Then set the following properties:
SELECT RTRIM(Server) AS servername
FROM ServerList_SSIS
WHERE (Server IS NOT NULL)
and coalesce(exclude,'0') <> 'True'
and IsOracle is Null
Execute SQL Task > General > ResultSet = "Full Result Set"
Execute SQL Task > Result Set Tab "Result Set Name = 0", Variable Name = "User::SQL_RS"
So we have a list of server names in the SQL_RS variable now.
ForEach > Collection > Enumerator = "Foreach ADO Enumerator"
ForEach > Collection > Enumerator Configuration > ADO Object source Variable = User::SQL_RS
This maps the first column of the SQL_RS object to the SRV_Conn variable, so each iteration of the loop will result in a new value in this variable.
ForEach > Variable Mappings > Variable = User::SRV_Conn, Index = 0
Inside the ForEach are some other sql execs, performing queries on sql databases, so I need to change the ServerName of my 'MultiServer' connection. I have another connection for the initial query that got me the list of servers to query. Making the connection dynamic is done in properties of the connection - right-click the connection > properties. Click the ellipses to the right of the expressions.
Connection > Properties > Expressions > Property = ServerName, Expression = "#[User::SRV_Conn]"
Note: The index value of 0 for the variable mapping works for Native OLEDB\SQL Server Native Client. If you're using another db provider, you may need to use other index types - this makes setup more confusing.
OLEDB = 0,1
ADO.NET = #Varname
ADO = #Param1, Param2
ODBC = 1,2
Full listing here.

Related

Data Flow Task - Set two User Date Variables as Parameters

I am creating an SSIS package that will run each month. This particular stored procedure needs to run for one week at a time since the data returned is very large.
I have set up my stored procedure to with two parameters: #StartDT and #EndDT. I created two SSIS variables: StartDT and Wk1EndDT (I'll create the other start and end dates for the weeks once I get this one working).
StartDT has this expression:
(DT_DATE)((DT_WSTR, 4)YEAR(DATEADD("mm", -1, GETDATE())) + "-" +RIGHT("0" + (DT_WSTR,2)MONTH(DATEADD("mm", -1, GETDATE())),2)+"-01")
Wk1EndDT has this expression:
DATEADD("DD",7, #[User::StartDT])
I'm using a DataFlow task with a SQL command text of:
EXECUTE dbo.uspUploadWk1 ?,?
When I go to preview the results, I receive the following error message:
There was an error displaying the preview.
No value given for one or more required parameters. (Microsoft SQL Server Native Client 11.0)
I have the parameters set like this:
I am not sure why this isn't working. I've searched all over and have not found an answer. I am using Visual Studio 2015.
Assuming an OLE DB Connection Manager, the Mappings tab should be using a zero based ordinal system on the Parameters column. Yes, it defaults to naming them as Parameter0, Parameter1, etc but for an OLE DB connection manager, you'll use the ordinal position of the question marks, ?, starting at zero.
For ODBC, it becomes a 1 based counting but still uses ? as the parameter place holder.
ADO.NET uses named parameters so we'd match EXECUTE dbo.uspUploadWk1 #Parameter0, #Parameter1 but the ADO.NET source component doesn't support parameterization
Reference on parameters and mapping for Execute SQL Task but the syntax remains the same for Data Flow Task components

"Must declare the scalar variable #Idx" when using a Dapper query on SQL server via OleDb

This code works when the connection is made to an accdb database:
Dim customer = connection.Query(Of Klantgegevens)("Select Actief,Onderhoudscontract From Klantgegevens Where Klantnummer=#Idx", New With {.Idx = customerId}).SingleOrDefault
But the code below gives the error about the Idx parameter when the connection is made to a SQL server database that has a table with the same structure:
Dim customer = connection.Query(Of Klantgegevens)("Select Actief,Onderhoudscontract From [dbo.Klantgegevens] Where Klantnummer=#Idx", New With {.Idx = customerId}).SingleOrDefault
What is going wrong here? I had hoped that by using Dapper I would be able to write database agnostic code. But it seems that is not the case!
If you are using an ODBC/OLEDB connection, then my first suggestion would be: move to SqlClient (SqlConnection). Everything should work fine with SqlConnection.
If you can't do that for some reason - i.e. you're stuck with a provider that doesn't have good support for named parameters - then you might need to tell dapper to use pseudo-positional parameters. Instead of #Idx, use ?Idx?. Dapper interprets this as an instruction to replace ?Idx? with the positional placeholder (simply: ?), using the value from the member Idx.
This is also a good fix for talking to accdb, which has very atypical parameter usage for an ADO.NET provider: it allows named parameter tokens, but all the tokens all replaced with ?, and given values from the positions of the added parameters (not via their names).

Get last cube processed date in SSIS

I need to get last processed date of SSAS cube in SSIS and save it into a variable.
I've tried a "Execute SQL task":
SELECT LAST_DATA_UPDATE as LAST_DT FROM $system.mdschema_cubes
WHERE CUBE_NAME = 'CubeName'
It works ok in MSSQL management studio MDX query window but in SSIS it says: Unsupported data type on result set binding.
Then I've tried:
WITH MEMBER [Measures].[LastProcessed] AS ASSP.GetCubeLastProcessedDate() SELECT [Measures].[LastProcessed] ON 0 FROM [CubeName]
And it says '[ASSP].[GetCubeLastProcessedDate]' function does not exist.
Any ideas how to do this?
Thank you
A linked server might be your best option;
Create the linked server with the following, changing as appropriate:
EXEC master.dbo.sp_addlinkedserver
#server = N'LINKED_SERVER_OLAP_TEST', --Change to a suitable name
#srvproduct='', --Creates the productname as blank
#provider=N'MSOLAP', --Analysis Services
#datasrc=N'localhost', --Change to your datasource
#catalog=N'TESTCUBE' --Change to set the default cube
Change the data source of your Execute SQL Task to make sure it is pointing to any of the databases where the linked server is hosted, I.E. don't use an analysis service datasource use a standard OLE DB. Then have the following in your execute SQL task (Changing as appropriate).
SELECT *
FROM OpenQuery(LINKED_SERVER_OLAP_TEST,'SELECT LAST_DATA_UPDATE as LAST_DT FROM $system.mdschema_cubes
WHERE CUBE_NAME = ''CUBENAME''')
Set the variable to be DATETIME and the result set to be single row.
There may well be other ways to do this, however I have always found this method the most straight forward.

Can't pass in variables into OLE DB source command

I am creating a SSIS package that involves a where clause as:
WHERE effectiveDate >= #effectiveDate
Therefore, I defined #effectiveDate at the package level so that I can pass in value from a sql command using a different connection.
At the OLE DB source, I selected 'SQL command', and put the code as
Select ... FROM ...
WHERE effectiveDate >= ?
When I click on the 'parameters', an error appears:
'Parameters cannot be extracted from the SQL command. blablabla... use "SQL command from variable"...'
I am trying to avoid using SQL command from variable and try to locate the problem since I should be able to pass in variable.
Can anyone help? Thanks a lot!!
When building an OleDbCommand, you should actually name the parameter as the package is expecting, such as
OleDbCommand cmd = new OleDbCommand( "select ... from... where... eff > #yourPkgPameterName");
cmd.Parameters.AddWithValue( "#yourPkgPameterName", whateverValueToSend );
It's been a while for SQL-based parameters and don't remember if the AddWithValue() method is expecting the "#" or not... may need to try without it if it fails.

Cannot switch a connection string in LLBL Gen Pro

I have two databases with the same schema inside a Sql 2008 R2 Server, of which names are Database1 and Database2. I connected and performed queries on the Database1, and then changed to Database2 to fetch my entities using the following code
this.ConnectionString = "Server=TestServer; Database=Database2;Trusted_Connection=true";
using (IDataAccessAdapter adapter = new DataAccessAdapter(this.ConnectionString))
{
var entities = new EntityCollection<T>();
adapter.FetchEntityCollection(entities, null);
return entities;
}
(The connection string was set before executing the code).
I debugged the application and looked at the value of the connection string, it pointed to the Database2.
However, when I executed the above code, the result was return from the Database1. And if I looked at SQL Profiler, the statement was executed against Database1.
So, could anyone know what was going on? Why the query was executed against the Database1, not Database2.
PS: If I used the above connection string with plain ADO.NET, I was able to retrieve data from Database2.
Thanks in advance.
I have figured out what was going on. The reason was: by default LLBL Gen Pro uses fully qualified names like [database1].[dbo].[Customer] to access database objects, and the catalog is specified when generating entities. So you can't access objects just by changing the connection string.
Hence, to change to another database you have to override the default catalogue by using following code
var adapter= new DataAccessAdapter(ConnectionString, false,
CatalogNameUsage.ForceName, DbName)
{CommandTimeOut = TenMinutesTimeOut};
More information can be found at the following link