We are creating SSIS packages for our data feeds. Our connection manager has our individual usernames and passwords. Is there a way we can create environmental variables?
Also when we deploy our packages to dev or test or prod environments, do we need service accounts?
Yes, you can protect credentials.
Create Parameter variable and then select sensitive. This will block out the credential with astericks '*'
Set both the project and package Protection Level as either EncryptSensitiveWithPassord, or EncryptSensitiveWithUserKey. Note: failure to set both will result in a compiler failure that says the consistency check of the Protection Level failed - this is required.
Deploy the package to the server. Note: if the package is encrypted then anyone else who tries to open the package will be unable to do so. This is an important point for deployment purposes. It's not a big deal if you are handing off the *.ispac file, but if your company deploys via the DBA opening the solution and deploying from there then they will be unable to do so. I think you can do this if you EncryptWithPassword and then share the password, but EncryptWithUserKey will not work.
Set the Environment variable value on the server and then mark it as sensitive as well.
I don't recommend storing user credentials in protected parameters, but SSIS has been designed with this in mind if necessary. I much prefer doing everything through Windows Authentication and I highly recommend you to do the same if that is available to you.
Regarding service accounts. Yes, you can use those. In fact, it is highly recommended that all production deployments use service accounts where the concept of least privilege is implemented. So, the service account should only be granted the bare minimum level of privileges necessary. Said differently: DO NOT GRANT THE SERVICE ACCOUNT ADMINISTRATOR PRIVILEGES. This means specifying each individual privilege on each object. For example, SELECT only on dimension tables and SELECT/INSERT/UPDATE/DELETE on fact tables.
The best way for determining what privileges to grant is to go through the package and identify all tables that are touched as well as the command that is used when touching them. So, you will need to look at the following: Execute SQL task, OLE DB Source, OLE DB Destination, OLE DB Command, Lookup, etc.
Related
I'm trying to do a POC using Sqlclr and C# to call .Net code (dll) from a stored procedure. I'm trying to write to EventLogs. I was able to create a safe assembly and then create a stored procedure wrapper around my dll. But while executing the code it failed - for Event Log access the Assembly needs to be created with an External_Access/ Unsafe permission set.
When I tried the same with an External Assembly it didn't work. So I went with the route of creating a Login from the signed dll.
I signed the dll using Visual Studio. Under project properties signing tab, I signed it without a password (snk file).
I then created an asymmetric key in sql with the dll.
Using this asymmetric key I have created a Login in Master.
I then granted this Login External Access Assembly.
Changed Database to use the particular one in which I am trying to create the wrapper procedure
In the next step I tried to create the Assembly with External_Access
permission set.
This failed with the Error Message -
Msg 300 -
External Access Assembly Permission was denied on object 'server', database 'master'
I then tried going through the route of setting Database Trustworthy
to ON just to check if the concept actually works.
This failed with the same exception.
My question is does my Windows authenticated login need to have more permissions to be able to set the External_Access permissions. Does it require an SA account?
First of all, did you mean to run this on master (as per the error message)?
To use TRUSTWORTHY, the user needs sysadmin as per here, but it is generally a bad idea to use TRUSTWORTHY.
To create an assembly with EXTERNAL_ACCESS the login needs EXTERNAL ACCESS ASSEMBLY permission on the server. To create the assembly as UNSAFE the login needs UNSAFE ASSEMBLY permission on the server. You can read more about that here.
Niels
It would help greatly to know what version of SQL Server you are using given that there was a rather significant change, security-wise, in SQL Server 2017 as it relates to creating Assemblies. I have written a series of articles explaining the overall issue in SQL Server 2017 and providing a few solutions that can be automated within Visual Studio / SSDT:
SQLCLR vs. SQL Server 2017, Part 1: “CLR strict security” – The Problem
Part 1 is mostly explanation while parts 2 and 3 are the two methods for working within these constraints.
If you are using SQL Server 2017, have you tried granting the Asymmetric Key-based Login the UNSAFE ASSEMBLY permission?
Did you do anything else besides GRANT EXTERNAL ACCESS ASSEMBLY for that Login? Did you DENY anything?
Another thing to try is to add the Asymmetric Key-based Login to the sysadmin fixed server role:
ALTER SERVER ROLE [sysadmin] ADD MEMBER [{LoginName}];
While this shouldn't be necessary, the older documentation (2008 R2 and before) stated that it was a requirement. I have never needed to do it, but it is worth testing as it if nothing else works, it will get us closer to the actual issue. Be sure to set TRUSTWORTHY back to OFF in the DB where you turned it ON.
Also, you might want to review the series of articles I wrote on SQL Server Central that explain various aspects of working with SQLCLR in general:
Stairway to SQLCLR
I am developing a multi-user-application windows desktop application.
For User Management i use the DMBS (Oracle, MySQL, MSSQL, Postgre).
So each user has an database user account.
I do not want the user to connect to the database directly and read / write data.
The data should only be accessible connection should only be valid if they use my application.
I found this website by SQLDUDE where he describes some techniques.
One way to access the data only through a specific application is a logon trigger that checks for application_name.
However he also describes that this method can easily be spoofed as the user can specify the application_name once he knows it in the connection string.
(detailed explanation here - see Solution for Scenario #2)
He also mentions Application Roles.
A more secure approach you could use for this is called "Application
Roles". When connecting from an application you assume a particular
role and only that role is granted privileges required in the
database.
So basically the user logs in with his login credentials but has no rights at all (only connect).
Then inside the application i call sp_setapprole with a password, once the connection is established so the application rules are granted.
Once this call succeeds then the connection gets the privileges of the
application role and loses privileges of the actual user, which is
what we want. So if someone tried to connect to the database from SSMS
or SQLCMD, they will access the DB using their credentials, which
won’t have the required permissions on the tables, since only the
application role has the rights on the tables. This is more secure &
reliable approach, but one that requires application code change and
as a DBA you will have to create the application role in SQL Server.
So Application Roles sounds like the way to go.
My question is:
Are application roles DMBS standard and available to most DBMS systems?
Is there way to trace the sp_setapprole login (e.g. with WireShark)?
Of course someone could reverse-engineer the application and get the credentials for the application role - but i guess that's unavoidable :)
I've seen one simpler solution in place and it worked fine. Here it is:
The application is responsible for creating users at the database
Before creating the user, the application hashed/encrypted their password, so if you chose password "123456" it would be created as "RRU2992191910" (just an example)
If the user tried to connect to the database with "123456" it wouldn't be successful
Only the application was able to connect to database, because it hashed/encrypted the password informed by the user
This is not the most secure solution in the world, but it's very simple and does what you want. And it could be ported to different RDBMS's with no extra cost.
I have an SSIS package that has run for months daily with no problem. Suddenly it is failing every day with the error above. No changes to the package or the stored procs it runs have been pushed, to my knowledge, no changes to the database environment have been made including permissions for the service account that owns the job.
This is a SQL Server 2008 package that connects to a SQL server 2008 R2 database. The configuration is through the ssisconfig table and it defines which server to use through an environment variable.
The component in question is using a stored proc. In the proc, all fields are explicitly cast to the same data type of the table the data is being staged too. The proc runs fine by itself. It is a relatively complex proc that has two table variables and a CTE.
I can't run it from my local on prod and dev is not failing. If I open the prod SSIS package while I am set to the environment variable that the job owner uses, I do not get a validation error on this component. ValidateExternalMetadata is set to true for this component. For the entire package though, Delay Validation is set to True. I do get a validation error that would be expected on a task using a variable that is set dynamically in an earlier task. This error, however, is many, many steps later than the step it is currently failing on.
Usually in the past when we have had this sort of unspecific error, I have been easily able to see the problem with the data that caused the sudden failure. But none of the data in the proc result set is even close to being incorrect for the datatypes defined for staging table for that field. None of the required fields are missing data either.
I am out of ideas for what else to look at. If the environment didn't change, the actual code didn't change and the data looks correct, what is left to try?
As I have commented for details error, you can try below link from Microsoft:
Debug SSIS Package while calling from Agent
Some info from link:
Reasons that the package may have failed are as follows:
The user account that is used to run the package under SQL Server Agent differs from the original package author.
The user account does not have the required permissions to make connections or to access resources outside the SSIS package.
The package may not run in the following scenarios:
The current user cannot decrypt secrets from the package. This scenario can occur if the current account or the execution account
differs from the original package author, and the package's
ProtectionLevel property setting does not let the current user
decrypt secrets in the package.
A SQL Server connection that uses integrated security fails because the current user does not have the required permissions.
File access fails because the current user does not have the required permissions to write to the file share that the connection
manager accesses. For example, this scenario can occur with text
log providers that do not use a login and a password. This scenario
can also occur with any task that depends on the file connection
manager, such as a SSIS file system task.
A registry-based SSIS package configuration uses the HKEY_CURRENT_USER registry keys. The HKEY_CURRENT_USER registry keys
are user-specific.
A task or a connection manager requires that the current user account has correct permissions.
Method 1: Use a SQL Server Agent proxy account Create a SQL Server Agent proxy account. This proxy account must use a credential that
lets SQL Server Agent run the job as the account that created the
package or as an account that has the required permissions.
This method works to decrypt secrets and satisfies the key
requirements by user. However, this method may have limited success
because the SSIS package user keys involve the current user and the
current computer. Therefore, if you move the package to another
computer, this method may still fail, even if the job step uses the
correct proxy account.
Method 2: Set the SSIS Package ProtectionLevel property to ServerStorage Change the SSIS Package ProtectionLevel property to
ServerStorage. This setting stores the package in a SQL Server
database and allows for access control through SQL Server database
roles.
Method 3: Set the SSIS Package ProtectionLevel property to EncryptSensitiveWithPassword Change the SSIS Package ProtectionLevel
property to EncryptSensitiveWithPassword. This setting uses a password
for encryption. You can then modify the SQL Server Agent job step
command line to include this password.
Method 4: Use SSIS Package configuration files Use SSIS Package configuration files to store sensitive information, and then store
these configuration files in a secured folder. You can then change the
ProtectionLevel property to DontSaveSensitive so that the package is
not encrypted and does not try to save secrets to the package. When
you run the SSIS package, the required information is loaded from the
configuration file. Make sure that the configuration files are
adequately protected if they contain sensitive information.
Method 5: Create a package template For a long-term resolution, create a package template that uses a protection level that differs
from the default setting. This problem will not occur in future
packages.
Above link will help you through logging and other possible scenarios of failure of your package from SQL Agent. I suspect that your user of Agent has been corrupted. But that is an assumption. You first enable the logging of SSIS by following link :
Enable SSIS Package Logging
Some info from link
To set the logging level for a package by using the Execute Package
dialog box
In SQL Server Management Studio, navigate to the package in Object Explorer.
Right-click the package and select Execute.
Select the Advanced tab in the Execute Package dialog box.
Under Logging level, select the logging level. See the table below for a description of available values.
Complete any other package configurations, then click OK to run the package.
Hope this will help you in the first place. If not please let me know.
I am working on an application which bulk-loads data into a SQL Server 2008 database. It writes a CSV file to a network share then calls a stored procedure which contains a BULK INSERT command.
I'm migrating the application to what amounts to a completely new network. In this new world bulk insertion fails with this error:
Msg 4861, Level 16, State 1, Line 1
Cannot bulk load because the file "\\myserver\share\subfolder\filename" could not be opened. Operating system error code 5(failed to retrieve text for this error. Reason: 15105).
I connect to the database using Windows Authentication, using the same account which wrote the file. The file, and the folder in which it resides, grant read and modify rights both to my user account and the database server's domain service account. That service account apparently has constrained delegation permitted, which is mentioned on MSDN. Still no good. If I connect using a SQL Server account then bulk insertion succeeds, but we are trying to stick exclusively to Windows Authentication.
Does anybody have a handle on what needs to be done to make this work? How exactly does SQL Server go about accessing data on network shares, hopping between its service account and that of the connected user? I know that I can bulk insert in a similar situation in our current infrastructure, but it is so crufty with age that it would be hard to track down what has been done to enable this in the past.
Recently we had this issue for a number of our Devs. I've come up with a number of ways to allow testing of bulk inserts.
Our preference was to use a SQL service account. We set the SQL server and SQL agent to run as a service account and then allowed the devs to trigger agent jobs. The service account was granted permission to the UNC shares and this all functioned correctly. Note that the service account will always been fine running these agent jobs (assuming UNC permissions are set). It's the Devs trying to test that will come across these issues.
Another method is to create a share on the SQL server itself and point the bulk insert path at the local directory. These errors seem to only occur when accessing UNC paths. Regardless of whether the UNC path has the correct permissions to allow you access. For example we create C:\test\ as a folder on the SQL server itself and permission it to allow a dev to drop test files there. These are then called via the bulk insert command.
A command may need to be run against master to allow a SQL login group permission to bulk insert. This is as below.
GRANT ADMINISTER BULK OPERATIONS TO "domain\usergroup"
Adam Saxton's blog (about Kerberos and bulk inserts from a share) should be read: http://blogs.msdn.com/b/psssql/archive/2012/09/07/bulk-insert-and-kerberos.aspx. Adam offers two approaches:
Enable constrained delegation for the machine (as opposed to the sqlservr.exe startup) account, or use a SQL Server login. Adam mentions two other approaches (which he does not recommend).
An aside, the latter half of the OP's message "(failed to retrieve text for this error. Reason: 15105)" may be related to a SQL Server startup account lacking rights documented within SQL Server's "Configure Windows Service Accounts and Permissions" topic, such as SeAssignPrimaryTokenPrivilege.
Did you ever resolve this issue? I recently had a similar problem and discovered that the best way to resolve it was to use a SQL login.
Initially, having read the notes here I thought if I just granted read permissions to the Windows account with which I was connecting to the SQL Server that would be okay, but even when I granted read access to Everyone, I still couldn't read in the file.
I believe the reason is something to do with SQL Server impersonating the windows user and attempting to access the UNC share, which is delegation and which is not allowed unless explicitly enabled. There are some notes here which may help. This is the constrained delegation of which you speak and I couldn't get it to work either!
Bottom Line: I just used a SQL Login and made sure the SQL Server Process account had read permissions on the share (by granting read to Everyone in my case) and it worked.
In order to bulk insert with AD users, the SQL service it self has to be running as a domain user and has to have the AD permission to be able to delegate authentication. Same if you wanted to run linked servers with ad users. Here is the link for AD and linked servers, but the permission are the same.
Linked Servers and Active Directory
The server must have an SPN registered by the domain administrator.
The account under which SQL Server is running must be trusted for delegation.
The server must be using TCP/IP or named pipes network connectivity.
In the web-application I'm developing I currently use a naive solution when connecting to the database:
Connection c = DriverManager.getConnection("url", "username", "password");
This is pretty unsafe. If an attacker gains access to the sourcecode he also gains access to the database itself. How can my web-application connect to the database without storing the database-password in plaintext in the sourcecode?
You can store the connection string in Web.config or App.config file and encrypt the section that holds it. Here's a very good article I used in a previous project to encrypt the connection string:
http://www.ondotnet.com/pub/a/dotnet/2005/02/15/encryptingconnstring.html
In .NET, the convention is to store connectionstrings in a separate config file.
Thereon, the config file can be encrypted.
If you are using Microsoft SQL Server, this all becomes irrelevant if you use a domain account to run the application, which then uses a trusted connection to the database. The connectionstring will not contain any usernames and passwords in that case.
I can recommend these techniques for .NET programmers:
Encrypt password\connection string in config file
Setup trusted connection between client and server (i.e. use windows auth, etc)
Here is useful articles from CodeProject:
Encrypt and Decrypt of ConnectionString in app.config and/or web.config
Unless I am missing the point the connection should be managed by the server via a connection pool, therefore the connection credentials are held by the server and not by the app.
Taking this further I generally build to a convention where the frontend web application (in a DMZ) only talks to the DB via a web service (in domain), therefore providing complete separation and enhanced DB security.
Also, never give priviliges to the db account over or above what is essentially needed.
An alternative approach is to perform all operations via stored procedures, and grant the application user access only to these procs.
Assuming that you are using MS SQL, you can take advantage of windows authentication which requires no ussername/pass anywhere in source code. Otherwise I would have to agree with the other posters recommending app.config + encryption.
Create an O/S user
Put the password in an O/S environment variable for that user
Run the program as that user
Advantages:
Only root or that user can view that user's O/S environment variables
Survives reboot
You never accidentally check password in to source control
You don't need to worry about screwing up file permissions
You don't need to worry about where you store an encryption key
Works x-platform