I'm having this exact same problem I've been looking aroung but this is the only place I've seen the same issue and it is not resolved.
Does anyone knows what the problem might be?
I checked in Visual Studio and my SSIS version is 11.0.2100.60 (not a trial, not a beta).
EDIT: These are the steps I'm taking and the issue
First I choose a Connection Manager, right-click, select properties and click in Expressions option
Then in Property Expression Builder choose Connection String property and click in Expression option
Finally, in the Expression Builder dialog there is no option for variables, in every page I've read says that there should be a Variables node in there
Am I missing something?
SSIS 2012 has introduced the concept of Project level connection managers. What I see on the referenced post on the MSDN forums it the user has created a project level flat file connection manager and is unable to configure it with a local variable. Assuming that is the problem, my answer follows.
An SSIS project is generally more than one package. To simplify lives, the SSIS team now allows for the sharing of common resources across projects, connection managers being one of those resources.
Logically, if a thing is shared across a project, how can something that only exists in one file configure that resource? That configuration change would only work when Package1 is executing. When Package2 fires, unless the same variable and same expression was applied to the shared resource, you would experience different outcomes. That'd be a maintenance nightmare, which you might already experience if you don't have strong configuration practices.
If I create a Flat File Connection Manager at the project level, I can only reference variables that are also at the project level. Except there are not variables at the project level. Instead, they are called Parameters.
To that end, I created a Parameter called SomeProjectParameter
I then created a package, Package1.dtsx, and added 2 Flat File Connection Managers: FlatFileConnectionManagerLocal and FlatFileConnectionManagerProject
Instead that package, I also created a variable called SomeLocalVariable.
This screenshot shows me applying an expression to the ConnectionString property of FlatFileConnectionManagerLocal. There you can see that both the package variable, SomeLocalVariable is available as well as SomeProjectParameter
Now, if I try to apply an expression to the project's connection manager, you will only have project parameters available to you.
It's interesting to note that you can't apply an expression to a project level Connection Manager outside of the context of an SSIS package. There's simply no editor available to you until you have an open SSIS package. But, once applied, all the packages in the project will be similarly configured.
Quirk of the IDE I suppose. Also, don't be alarmed by the lack of color in these screenshots, I'm running with the 2012 version of SSDT.
i had the same problem and it was because the flat file source was set as a project source so i had to convert it into a package connection.
Right-click on the source and choose convert to package connection
Maybe this will help.
The Microsoft SSIS tutorial have these confusing statements:
In the Connection Managers pane, right-click Sample Flat File Source Data, and select Properties.
In the Properties window make sure the PackagePath starts with \Package.Connections. If not, in the
Connection Managers pane, right-click Sample Flat File Source Data, and select Convert to Package Connection.
\Package.Connections was initially displayed in the properties window, although I had not yet converted to Package Connection.
After reading this thread, I right clicked on Sample Flat File Source Data and then Convert to Package Connection and was able to complete the lesson. Thanks for the answers!
Related
I have a production SSIS project in SQL Server 2016 that creates and exports a flat file to another server. The destination server has reached end-of-life and I need to change the destination path to the new server so we can decommission the old server. Can I just edit the package or project in Visual Studio or do I need to recompile (redeploy? republish?)? I have never edited before, only created new projects however that was a few years ago and I am a little rusty.
Alternatively, I could copy the existing job, edit the copy, then run them in parallel first. Then I can disable the old project/package once I am confident the new one works. I'm not having much luck figuring out how to do this either.
Any help would be appreciated. Thanks!
If I'm understanding your question correctly, it sounds like your goal is to simply change the destination location for your report. You would need to update the connection manager for your flat file connection inside of your SSIS package.
You would need to edit your flat file connection manager's connection string (by loading up the connection manager properties and changing the value yourself or by changing the expressions, if it is parameterized). Once you have verified it is now pointing at the right server's location to save the report you are generating, you would rebuild the entire package and redeploy it, which would essentially overwrite the .ispac and .dtsx files on the deployment server with your updates.
Standard caveat of "it depends" but the most common case is that you can solve this through Configuration.
Right click on the Package (or Project) depending on how things are set up. In the "Connection Managers" tab, find the connection manager that corresponds to the flat file output (a strong naming standard helps). I have selected SO_61794511.dtsx and the Name is Flat File Conn... which then allows the right side menu to be populated.
Of interest here is ConnectionString. I am going to directly edit this to change from C:\ssisdata\input\so_61794511.txt to my new path D:\path\here\something\so_newthing.txt
Click OK 2x and the next time the package runs, it will use the configured value.
That's the easiest approach. You could accomplish a similar thing if you edit the job that runs the package to set the value at every execution but this just does it at a global scale.
Where this can go off the rails is if there's a expression applied to the ConnectionString property, e.g. the output file has a dynamic date in the file name. This is why I advocate for exposing Package or Project level parameters of a "base file path" concept. This allows me to change the path from C: (local development) to D: (server deployment) or even to a UNC path \server\share by setting a configuration instead of hard coding a path into the packages themselves.
I hesitate to post because I feel I'm missing something simple, but after reading a ton of articles, I need a second set of eyes because mine are glazed over.
SSIS 2012 Solution.
Goal: Import data from 47 MS Access Databases into corresponding SQL Server databases. This means 47 packages. I want to avoid 47 different sets of Connection Managers and parameterize out what makes sense on the project level so common data is easily configurable.
I added project-level Parameters to hold the SQL Server, SQL UserName, SQL Password, and the Path to the Access Databases. These will be mapped out to the environment to allow changing the destination SQL Server easily as well as the source directory for the Access DBs.
I added two project-level OLEDB Connection Managers - one for SQL 2012 (NativeClient 11.0), one for Access (MSJET 4.0) (they're access 2000 db's). They are set to use an initial database configuration that I hope will be overridden by the individual packages.
For each package, I want to specify the databases being used, so I create a Package Variable (SQLDatabase and AccessDatabase)
In the Connection Managers, I add Expressions to include the Package Variable - e.g. for InitialCatalog on the SQL Connection Manager = #[User::SQLDatabase] and the ServerName in the Access CM combines the project path + the #[User::AccessDatabase]
WHAT WORKS:
I can update the Package Variable values and see them reflected in the Properties of the Connection Managers.
I can go into the Data Flow and add an OLE DB Source, select the SQL Connection Manager, and successfully receive a list of tables from that database.
WHAT FAILS:
When I click Preview, or Columns, after selecting a table I get a big error that reads:
"Error at Package1: The variable 'User::SQLDatabase' was not found in the Variables collection. The variable might not exist in the correct scope."
After clicking OK to that error, then the Preview or Column window shows up with no further problems.
However, after clicking OK on the OLE DB Source Editor, there is a big red X sitting on the box indicating the same error as above.
MY CONFUSION:
If I open the Variables for the package, the variable is sitting right there, scoped to Package1. If I update the value it and save, the Connection Manager updates itself accordingly, so the Expression mapping seems to be okay.
It's as if this is only partially working, and I'm unsure where the flag is to make the rest of it work, or where I failed in my configuration. Can you not set an expression in a project-level CM to use a package-level variable? If that's the case, I'm at a loss on how to correctly set this up to avoid package-level CMs.
Any help appreciated. Please be gentle if I've been an idiot somewhere.
If the connection managers are at project level, then you cannot use package level variables to set their value. If you go to the expression builder for Initial Catalog then you can see that only your project level variables are available. Hence you can use your project level variable to set it instead. If you really want to use the package level variable then you would need to create a connection manager per package which would be a pain.
Been struggling with this for quite some time so any help is appreciated. Spent a lot of time looking over MSDN, Google, etc, and still no luck.
I have an SSIS project I intent to always execute from within BID. The project has multiple packages, so I created a Data Source to share amongst all the packages.
In each package, I create a Connection Manager based of that Data Source. Now when I go and edit this Data Source's connection string to point to a different server, the change is NOT reflected in the packages.
Now, MSDN says the opposite. In fact, one of the main purposes of Data Sources is to update connection strings easily from within BID; at least according to MSDN.
And for me its not working. This has got to be the most annoying thing I have seen thus far in BID.
Any one have any suggestions?
UPDATE
It looks like BID does not synchronize a Data Source change with Connection Managers if the data source's authentication is SQL server. If it is Windows authentication, it detects the change fine.
Very interesting indeed.
Just a guess. This kind of behavior happens when you have a "package configuration" configured.
Right click on any blank area of your control flow and select "package configuration". If the "Enable package configurations" check box is selected and you have a configuration on your connection string, it will override any value you had specified manually.
I figured it out for those that are interested. I am not sure if this is a bug in BID or not but I hope this helps someone else struggling with this problem.
All you have to do is simply open up all your packages at one time. This is as simple as highlighting all the packages in the solution, right clicking, and choose "open".
Once all the packages have been opened, go ahead and edit your Data Source's connection properties. BID will automatically apply the new changes to all the open packages that have a connection manager based on that Data source.
Hope this helps someone.
I found I could not change any of the data connection configuration values because there was a variable defined for the connection string. Enabling/disabling package configurations had no effect on the behavior.
To access the variables menu, right click anywhere on the DTSX designer background and select Variables. Make sure you do not have any values in the variables list that are being referenced by your data connection. If you can not find the references anywhere, try fixing the values for the variables that might be affecting it.
You need to change in respective dtsx package. Open it in notepad++ and search for the connection string, modify it and save it. You can see the connection change in solution now.
When I open up the solution that contains SSIS packages created by a colleague, I get this awkward error that tells me nothing about what I'm supposed to do to fix it.
He left instructions to take all the "variables" out of the connection string in the dtsx file manually before opening up the solution. I have done that, now when try to view the package in the designer I just get an image of a red x and this message.
EDIT: You cannot see any design elements, no tabs across the top to switch to errors or data flows. Just a gray center area on the screen with a red x, and the message, its like VisualStudio dies in the process of reading the dtsx file.
The question is rather unspecific so it’s of course difficult to get on the right track here. All of the given answers focus different issues. I would say that PeterX had the best guess. The reason for the error could be as simple as a modified data source.
I came across with a bug "error output has no corresponding output" quite often when adding a new column to a table that needs to be processed by an existing SSIS package. This bug came along with an error message saying that a "Value does not fall within the expected range".
A newly added column needed to be processed by an existing SSIS Package. The expected behavior is that SSIS will recognize that there is a new column and select this column on the columns page of the OLEDB Source Task SSIS to be processed. However, when opening the OLEDB Source Task for the first time after having modified the table I got twice the following error message: "Value does not fall within the expected range." The error message showed up when opening the editor and when opening the Columns page of the editor. Within the Advanced Editor of the OLEDB Source Task the new column showed up in the OLEDB Source Output Columns Tree, but not in the OLEDB Source Error Output Columns Tree. This is the actual underlying problem of the error message. Unfortunately, there seems to be no way to add the missing column manually.
To solve the problem, remove and re-add the newly added column on the Columns Page of the normal Editor as mentioned by Jeff.
It is worth to be mentioned that the data source of the OLEDB Source task was a modified MDS View. Microsoft CRM Dynamics – as mentioned in the related thread – is using views, too. That leads me to the conclusion, that using views as a data source may produce either of the above mentioned errors, when modifying datatypes or adding/removing columns.
Related Thread: Error" ...The OLE DB Source.Outputs[OLE DB Source Output].Columns[XXXXXXXX] on the non-error output has no corresponding output
The described workaround refers to Visual Studio 2008 Version 9.0.30729.4462 QFE with Mircorsoft.NET Framework 3.5 SP1. The database is SQL Server 2008 R2 (SP2).
I had to delete and recreate the OLE DB Data source in my Data Flow - this is where I got the error. I also noted I had to "re-select" the "OLE DB connection manager" in the drop-down-list to force it to recognise the new connection.
This was probably a combination of getting the solution from TFS (where I noticed the data-sources didn't come-across properly and it complaining about a missing connection GUID) and/or copying and pasting the elements from another package.
(For BIDS 2008).
I had this issue for my OLE DB Source component with an SQL command after adding new columns to the database, and it wouldn't let me select columns or anything else to add the new columns.
I'm working with an Oracle database, and the only way I could get it to update was to change the SQL query to select 1 from dual, and preview it. Then revert it back to my old query.
You get a similar message if someone uses EncryptAllWithUserKey as the ProtectionLevel. However, I believe the message is slightly different (even though you get a grey design surface with a red X).
Have you tried viewing the file in Notepad? Is it just a series of GUIDs or is there anything in it that is humanly readable? If it doesn't have any readable code, then it was probably encyrpted with the user key.
If the employee deployed the packages to a server and used SQL Server as the deployment destination (not File System or SSIS Pacakge Store) then you can download the packages to your machine. Just connect to the SQL Server Integration Services engine, expand Stored Packages, expand MSDB, expand the relevant folder, right-click on the package, and click Export Package. Save the file on your local machine and open it. The package will probably lose annotations and pretty formatting, but otherwise it should be identical to what the employee deployed.
I just struck the same issue. After flailing about for a bit, I found the solution was to edit the Solution Configuration.
The Solution Configuration appeared to have a matching Project configuration, as shown:
However clicking the drop-down arrow for that Project (SSIS-Advance in this example) revealed that there was no Project Configuration for that project called Production - Sub Reports. I'm not sure how that came about - this Solution has a 7-year history and many developers.
Anyway once I created a New Project configuration (using that same drop-down menu), it is all happy now.
If it has Oracle data sources, you may need to install the Microsoft Connectors v4.0 for Oracle by Attunity:
https://www.microsoft.com/en-us/download/details.aspx?id=52950
I also had to use VS 2015 - the version originally used to create the project and package.
I had this exact problem and installing these connectors and using VS 2015 fixed the issue.
I had this occur as well when I tried to call a stored procedure with OUTPUT parameters with OLE DB.
I found this: http://sqlsolutions.blogspot.com/2013/04/ssis-value-does-not-fall-within.html, which resolved my issue. The relevant action was to rename the SSIS parameter mappings to '0', '1', etc.
So for example, when calling dbo.StoredProc #variable0 = ?, #variable1 = ? OUTPUT, #variable2 = ?;, in the parameter mapping dialog, you would name the parameters '0', '1', 2' to correspond to those. Ah, SSIS <3
I get this when I do not follow the convention for parameter naming, e.g. not name parameters 0,1,2,... in the right order for OLE DB connections.
The details are documented here.
In your connection manager, convert your connections to package level instead of project level
Delete connection manager and re-create and setup ssis package solve the problem.
I got this issue after I Add Existing Connection Manager in a SSIS project. I was just importing a Project Connection Manager from a different project (.conmgr) to my project. My solution to fix the issue was:
Deleting the imported .conmgr
Recreating it from scratch
I am working on a winforms application using LINQ to SQL - and am building the app using a SQL Express instance on my workstation. The final installation of the project will be on a proper SQL Server 2005.
The database has the same name, and all tables are identical but the hostname is different.
The only way I have found to make my app work from one machine to the next is to re-open the code in Visual Studio, delete all of the objects referring to the SQL express instance from my .mdbl, save the project, connect to the other server, drag all of the references back on, and rebuild the application for release once more.
an answer here suggested that one can simply modify the app.config/web.config file - whih would work for asp.net, but this is a winforms application. What's the correct way of making LINQ to SQL apps use a new database without having to re-open the app in visual studio?
If I understand your problem correctly, you simply change the database's connection string in your app.config / web.config.
Edit, post clarification: You have the connection strings stored somewhere. They might be in the app.config of your server. Still, you get them from somewhere and that somewhere may be in an app.config. Use that then :)
One good solution is to add another connection to the dbml file itself. You can get to this by right-clicking on the field of the design surface and selecting properties. From there, you can add another connection string. Instead of deleting everything and redragging, just change the string and recompile.
But if you want to get fancy-schmancy, you can have the program auto-detect whether it is being run locally or not, using this neat utility function: detect local
And go from there to set the appropriate connection string based on the results.
A more useful answer...
app.config ends up as appname.exe.config when it has been built.
rather than opening Visual Studio and modifying app.config, you can simply edit the appname.exe.config file, and restart the app.
I believe you can store the connection information in an app.config file and retrieve it from there. Here is a post about doing that with LINQ to SQL. Once you deploy it to a production server, you can just edit the XML to change the data source.