Anyone have any idea how do we specify different connection strings in SSIS for different environments - like system integration test environment, user acceptance test environment and production environment?
Is it done by defining multiple connection managers or we can define multiple configuration files and point our connection string to?
You do not need to have multiple connections for the same database.
In SSIS 2008:
It's a good idea to have your connection strings defined using an expression. This is how you can do it:
Suppose you have added a anew OLEDB connection in your connection manager. Copy the value of the ConnectionString property. IT would look like this:
Data Source=(local);Initial Catalog=Learn;Provider=SQLNCLI10.1;Integrated Security=SSPI;Application Name=SSIS-RaggedFile-{03053F2E-8101-4985-9F2B-8C2DDE510065}(local).Learn;Auto Translate=False;
Remove the non-essentials:
Data Source=(local);Initial Catalog=Learn;Provider=SQLNCLI10.1;Integrated Security=SSPI;
Now create three new variables at the control surface level: sServer, sDb, sProvider. Type for all these three varaibles will be string. Their values - using this example - will be (local), Learn, and SQLNCLI10.1.
Go back to the ConnectionString property of your connection. and set its value to
"Data Source=(local);Initial Catalog=Learn;Provider=SQLNCLI10.1;Integrated Security=SSPI;"
Now, replace the server, db, and provider with the variables you have created to make your expression look like this:
"Data Source=" + #[User::sServer] + ";Initial Catalog=" + #[User::sDb] + ";Provider=" + #[User::sProvider] + ";Integrated Security=SSPI;"
When we move from one environment to another, we may encounter different versions of the databases - hence it is helpful to have a variable for provider as well.
Now, these values can be changed at the time of deployment.
In SSIS 2012 -
This method would still work with a little bit of change. Change these three variables to parameters and make them required. That way, at the time of deployment of deployment, you will be forced to change the values. This is just a starting point. Read up on server environment.
Note: I opine that a variable should have atomic value. This helps in avoiding any error due to mistyping. That's why I have created three separate variables as opposed to having one variable called sCnxn (for example) and have someone change a portion of that variable at the time of deployment.
Related
My purpose is I want to change the value of this Application Scope Connection String using Code.
So, I have this dataset database for my vb.net project. and it's fully working
(fully rendered) when we're running the app.
If you open the project setting, it is stored on Connection String.
It is using Applicatio Scope. Not user Scope. ANd it's
saved with its value ; "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=C:\ProgramData\RK-Medis\db-used.mdb"
My questions are:
Can we change that variable value of Connection String before its
called / rendered on its GUI?
WHat are the code for changing it?
If no we can't change it, then what are the alternative for using
dataset + its Connection String that automatically used in almost
many tables (GUI) i have...?
Perhaps you need a tableConnectionString for example, to store the CS for each user. Then use a property of string(or maybe const variable) to store your connection string for that particular user during user login. Substitude the CS to your application. Then each of the user will be able to have different CS.
Hope its give you the idea.
Is there a way in SSRS to create a snapshot for a report that uses a shared dataset? We are looking for a way to dynamically set the server and credentials in SSRS, but it seems when the shared dataset is used there is no way to cache a report.
Two things that I think may help you:
You can create a dynamic connection string from parameters you pass in. However you lose intellisense when creating this so generally I use an actual database first to do my dataset then change the connection string later:
A. Create a variable #Server, set it to text:
B. Create a local DataSource, this must be local as you cannot share a data source that is dynamic, to my knowledge as it has no input to go on thus a shared must have set inputs.
C. On the side of 'Connection string:' hit the 'fx' button to get a dynamic connection string. Build a connection string of text with your parameter being an input:
="Data Source=" & Parameters!Server.Value & ";Initial Catalog=(DBName)"
D. You NOW have to set up a dataset to bind to #Server variable or else someone just needs to do plain text to guess at a server. For this reason I usually create a dataset like
select 'Server1' as Server
union
select 'Server2'
union
select 'Server3'
You can handle the cache aspect COMPLETELY from the hosted end and not worry about the report stuff. Just go to a report once deployed and choose 'Manage'>'Snapshot Options'> Set your preferences.
EDIT: You probably want another variable for the database or else you will assume a same database structure all the time.
In an SSIS project, I know how to set up a variable to hold the connection string for a Connection Manager - but how do I go the other way? ie - I have an existing SSIS package and want to find out the name of the variable which provides the connection string for a particular Connection Manager.
I can find loads of references to how to set it up, and did expect that going to Connection manager>Properties>Expressions would show me, but it doesn't. I did manage to find it by going to Package Configuration Organizer, picking sensibly named parameters, going to edit them, going to the second dialog of the wizard to find the Exported property name. This can't be the only way, surely?
Regards, Stewart
EDIT - This is in Visual Studio 2008
First of all, grab BIDSHelper it's a free add-in for visual studio and at a minimum it helps identify when elements have Expressions and Configurations applied to them. One will have a teal highlight, the other a fuschia colored and yes, an object can have both.
The first scenario you described, look at the Properties, Expressions and identify the use of expressions. Other objects, you might need to look at an Expressions tab.
Configurations work differently. You can use an Environment Variable, Registry Value, Parent Package Value, XML file or SQL Server table. The first three provide a 1:1 mapping between a configuration value and a configured item (variable, connection manager, etc). XML and SQL Server can configure many items. The order configurations are applied is important as you could have 5 configuration entries and each one of them modify the same setting with different values. There is also a difference between how 2005 and 2008 applies configurations so take a peek at Understanding How Integration Services Applies Configurations
When a package loads, BIDS will indicate what configurations it is attempting to load (look in your output window). Beyond BIDS highlighting and those messages, those messages are your other clue that configurations exist and are being applied. That's also your opportunity for detecting missing configurations (I expected to find configuration X and didn't find it = the configuration resource doesn't exist) or (I expected to configure property X but could not find it = the thing being configured does not exist)
I have found the best approach is to define a common set of configurations (Sales connection, warehouse connection) that all the applications in an environment use and use a consistent configuration naming approach. We then use custom configurations for project level things (the path for input and output for the InsuranceProcessing packages is would apply across all the packages but would be different for Sales) and then a third set of configurations that is package specific. We use SQL Server tables for this as it makes inspecting values much easier than gloming through lots of ugly XML.
Lots of information, but nothing that directly answers your question. Sorry about that. You might be able to inspect the object model and look at what's configured but that's ugly.
Thanks to the above answers, I realised that it wasn't a variable, but a configuration. Once that became clear, more Googling led to an explanation of why the Target Object is blank in the Package Configuration Organizer (there can be more than one).
The answer is in the config database; the PackagePath entry holds \Package.Connections[connection manager name].Properties[ConnectionString], so to find out where a particular mapping comes from, use something like this.
SELECT TOP 1000 [ConfigurationFilter]
,[ConfiguredValue]
,[PackagePath]
,[ConfiguredValueType]
FROM [Database].[dbo].[Configurations_table]
where ConfiguredValue like '%object in which you are interested%'
(not sure why all that didn't go in the code block).
I still think it should be easier, but I hope this helps others.
Regards, Stewart
Great answer by billinkc. In addition to that answer (or rather fleshing out the "inspect the object model and look at what's configured" part), I run a script at the start of any package that adds the values of all connection manager connection strings to the output window, followed by the connection string expression for each manager. In addition it loops through all the variables that have been specifed for use in the script and outputs the value. Not so useful in production but very useful when developing/testing.
Just add a script task to the start of the package flow, specify any variables you want to debug then add the following code to the script:
'Report number of connections
Dts.Events.FireInformation(99, "debug", "number of connections = " & Dts.Connections.Count, "", 0, True)
'Loop through connection collection
For Each cConnection As Microsoft.SqlServer.Dts.Runtime.ConnectionManager In Dts.Connections
'Report connection string value
Try
Dts.Events.FireInformation(99, "debug", "connection """ & cConnection.Name & """ value = " & cConnection.ConnectionString, "", 0, True)
Catch
End Try
'Report connection string expression
Try
Dts.Events.FireInformation(99, "debug", "connection """ & cConnection.Name & """ constring expression = " & cConnection.GetExpression("ConnectionString"), "", 0, True)
Catch
End Try
Next
'Report number of variables
Dts.Events.FireInformation(99, "debug", "Number of Variables = " & Dts.Variables.Count, "", 0, True)
'Loop through variables collection
For Each vVariable As Microsoft.SqlServer.Dts.Runtime.Variable In Dts.Variables
'Report variable value
Try
Dts.Events.FireInformation(99, "debug", "Variable """ & vVariable.Name & """ value = " & vVariable.Value, "", 0, True)
Catch
End Try
Next
Visual Web Developer. Entity data sources model. I have it creating the new database fine. Example
creates SAMPLE1.MDF and SAMPLE1.LDF
When I run my app, it creates another SAMPLE1_LOG.lDF file.
When I run createdatabase, is there a place I can specify the _LOG.ldf for the log file? SQL 2008 r2.
It messes up when I run the DeleteDatabase functions... 2 log files...
How come it does not create the file SAMPLE1_Log.ldf to start with, if that is what it is looking for...
Thank you for your time,
Frank
// database or initial catalog produce same results...
// strip the .mdf off of newfile and see what happens?
// nope. this did not do anything... still not create the ldf file correctly!!!
// sample1.mdf, sample1.ldf... but when run, it creates sample1_log.LDF...
newfile = newfile.Substring(0, newfile.Length - 4);
String mfile = "Initial Catalog=" + newfile + ";data source=";
String connectionString = FT_EntityDataSource.ConnectionManager.GetConnectionString().Replace("data source=", mfile);
// String mexclude = #"attachdbfilename=" + "|" + "DataDirectory" + "|" + #"\" + newfile + ";";
// nope. must have attach to create the file in the app_data, otherwise if goes to documents & setting, etc sqlexpress.
// connectionString = connectionString.Replace(mexclude, "");
Labeldebug2.Text = connectionString;
using (FTMAIN_DataEntities1 context = new FTMAIN_DataEntities1(connectionString))
{
// try
// {
if (context.DatabaseExists())
{
Buttoncreatedb.Enabled = false;
box.Checked = true;
boxcreatedate.Text = DateTime.Now.ToString();
Session["zusermdf"] = Session["zusermdfsave"];
return;
// Make sure the database instance is closed.
// context.DeleteDatabase();
// i have entire diff section for deletedatabase.. not here.
}
// View the database creation script.
// Labeldebug.Text = Labeldebug.Text + " script ==> " + context.CreateDatabaseScript().ToString().Trim();
// Console.WriteLine(context.CreateDatabaseScript());
// Create the new database instance based on the storage (SSDL) section
// of the .edmx file.
context.CreateDatabaseScript();
context.CreateDatabase();
}
took out all the try, catch so i can see anything that might happen...
==========================================================================
Rough code while working out the kinks..
connection string it creates
metadata=res://*/FT_EDS1.csdl|res://*/FT_EDS1.ssdl|res://*/FT_EDS1.msl;provider=System.Data.SqlClient;provider connection string="Initial Catalog=data_bac100;data source=.\SQLEXPRESS;attachdbfilename=|DataDirectory|\data_bac100.mdf;integrated security=True;user instance=True;multipleactiveresultsets=True;App=EntityFramework"
in this example, the file to create is "data_bac100.mdf".
It creates the data_bac100.mdf and data_bac100.ldf
when I actually use this file and tables to run, it auto-creates data_bac100_log.LDF
1) was trying just not to create the ldf, so when the system runs, it just creates the single one off the bat...
2) the Initial Catalog, and/or Database keywords are ONLY added to the connection string to run the createdatabase().. the regular connection strings created in web config only have attachdbfilename stuff, and works fine.
I have 1 connection string for unlimited databases, with the main database in the web.config.. I use a initialize section based on the user roles, whether visitor, member, admin, anonymous, or not authenticated... which sets the database correctly with a expression builder, and function to parse the connection string with the correct values for the database to operate on. This all runs good.
The entity framework automatically generates the script. I have tried with and without the .mdf extensions, makes no difference... thought maybe there is a setup somewhere that holds naming conventions for ldf files...
Eventually all of this will be for naught when start trying to deploy where not using APP_Data folder anyways...
Here is an example of connection string created when running application
metadata=res://*/FT_EDS1.csdl|res://*/FT_EDS1.ssdl|res://*/FT_EDS1.msl;provider=System.Data.SqlClient;provider connection string="data source=.\SQLEXPRESS;attachdbfilename=|DataDirectory|\TDSLLC_Data.mdf;integrated security=True;user instance=True;multipleactiveresultsets=True;App=EntityFramework"
in this case, use the TDSLLCData.mdf file...
04/01/2012... followup...
Entity Framework
feature
Log files created by the ObjectContext.CreateDatabase method
change
When the CreateDatabase method is called either directly or by using Code First with the SqlClient provider and an AttachDBFilename value in the connection string, it creates a log file named filename_log.ldf instead of filename.ldf (where filename is the name of the file specified by the AttachDBFilename value).
impact.
This change improves debugging by providing a log file named according to SQL Server specifications. It should have no unexpected side effects.
http://msdn.microsoft.com/en-us/library/hh367887(v=vs.110).aspx
I am on a Windows XP with .net 4 (not .net 4.5)... will hunt some more.. but looks like a issue that cannot be changed.
4/1/2012, 4:30...
ok, more hunting and searching and some of the inconsistancies I have experienced with createdatabase and databaseexists... so .net 4.5 is supposed to add the _log.ldf, and not just .ldf files, so they must have addressed this for some reason....
found others with same issues, but different server....
MySQL has a connector for EF4, the current version is 6.3.5 and its main functionalities are working fine but it still has issues with a few methods, e.g.
•System.Data.Objects.ObjectContext.CreateDatabase()
•System.Data.Objects.ObjectContext.DatabaseExists()
which makes it difficult to fully use the model-first approach. It's possible by manually editing the MySQL script (available with the CreateDatabaseScript method). The MySQL team doesn't seem eager to solve those bugs, I'm not sure what the commitment level actually is from their part but it certainly is lower than it once was.
That being said, the same methods fail with SQL CE too (they are not implemented, and I don't see the MS team as likely to tackle that soon).
Ran out of space below... it just becomes a problem when create a database, and it does not create the _log.ldf file, but just the ldf file, then use the database, and it creates a _log.ldf file... now you have 2 ldf files.. one becomes invalid.. Then when done with the database, delete it, then try to create a new, and a ldf exists, it will not work....
it turns out this is just the way it is with EF4, and they changed with EF4.5 beta to create the _log.ldf file to match what is created when the database is used.
thanks for time.
I've never used this "mdf attachment" feature myself and I don't know much about it, but according to the xcopy deployment documentation, you should not create a log file yourself because it will be automatically created when you attach the mdf. The docs also mention naming and say that the new log filename ends in _log.ldf. In other words, this behaviour appears to be by design and you can't change it.
Perhaps a more important question is, why do you care what the log file is called? Does it actually cause any problems for your application? If so, you should give details of that problem and see if someone has a solution.
I have looked at other posts and questions but I couldn't find what I needed.
I am relatively new to SSIS Package creation so bear with me please.
Basically, I need the package to connect to multiple servers based on a list of IPs read from a table. I have a connection string that I am reading from a config table. the connection string is generic in that the datasource is simply 255.255.255.255 and I want to replace the datasource with the IPs read from the table as I loop through during package execution.
I am using IPs since the servers I am connecting to are not on our domain. I have set up the server name as a variable within the connection manager expressions. Thus what I am hoping is that the pkg config is read to obtain the entire connection string. Then as I loop through the IPs, the server name variable will be dynamically substituted into the data source value as it loops thorugh. I hope this makes sense.
So the connstring is: (generic within config table)
Data Source=255.255.255.255,65000;User ID=test;Password=test;Initial Catalog=myDB;Provider=SQLNCLI10.1;Integrated Security=SSPI;Auto Translate=False;Application Name=SSIS-myApp;
Then as I obtain the list of IPs I want it to change to and then obviously connect as:
Data Source=1.1.1.1,1000;User ID=test;Password=test;Initial Catalog=myDB;Provider=SQLNCLI10.1;Integrated Security=SSPI;Auto Translate=False;Application Name=SSIS-myApp;
Then the next IP and connect as:
Data Source=2.2.2.2,1000;User ID=test;Password=test;Initial Catalog=myDB;Provider=SQLNCLI10.1;Integrated Security=SSPI;Auto Translate=False;Application Name=SSIS-myApp;
How can I do this using SSIS?
Create a connection manager connection to a valid database.
Right click on the newly created connection manager connection, select to Properties and copy the value for the connection string
Create a string variable and paste the connection string in your clip board in the value for the newly created variable
Add an Execute SQL Task with a statement similar to this:
SELECT TOP 1
'Data Source=' + [IPAddress] +
';User ID='+[Username] +
...
FROM dbo.IPTable
Pass the result set to the string variable you previously created
Right click on your connection manager connection and click the ellipsis next to Expressions
In the dialog that pops out, under Property, select ConnectionString and click the ellipsis next to the blank value for Expression.
In the Expression Builder, add the variable name to the variable you created. Ex: #[User::CreatedVariableName]
And you're finished. This a basic concept and you can tweak from there..