I am working on an application which bulk-loads data into a SQL Server 2008 database. It writes a CSV file to a network share then calls a stored procedure which contains a BULK INSERT command.
I'm migrating the application to what amounts to a completely new network. In this new world bulk insertion fails with this error:
Msg 4861, Level 16, State 1, Line 1
Cannot bulk load because the file "\\myserver\share\subfolder\filename" could not be opened. Operating system error code 5(failed to retrieve text for this error. Reason: 15105).
I connect to the database using Windows Authentication, using the same account which wrote the file. The file, and the folder in which it resides, grant read and modify rights both to my user account and the database server's domain service account. That service account apparently has constrained delegation permitted, which is mentioned on MSDN. Still no good. If I connect using a SQL Server account then bulk insertion succeeds, but we are trying to stick exclusively to Windows Authentication.
Does anybody have a handle on what needs to be done to make this work? How exactly does SQL Server go about accessing data on network shares, hopping between its service account and that of the connected user? I know that I can bulk insert in a similar situation in our current infrastructure, but it is so crufty with age that it would be hard to track down what has been done to enable this in the past.
Recently we had this issue for a number of our Devs. I've come up with a number of ways to allow testing of bulk inserts.
Our preference was to use a SQL service account. We set the SQL server and SQL agent to run as a service account and then allowed the devs to trigger agent jobs. The service account was granted permission to the UNC shares and this all functioned correctly. Note that the service account will always been fine running these agent jobs (assuming UNC permissions are set). It's the Devs trying to test that will come across these issues.
Another method is to create a share on the SQL server itself and point the bulk insert path at the local directory. These errors seem to only occur when accessing UNC paths. Regardless of whether the UNC path has the correct permissions to allow you access. For example we create C:\test\ as a folder on the SQL server itself and permission it to allow a dev to drop test files there. These are then called via the bulk insert command.
A command may need to be run against master to allow a SQL login group permission to bulk insert. This is as below.
GRANT ADMINISTER BULK OPERATIONS TO "domain\usergroup"
Adam Saxton's blog (about Kerberos and bulk inserts from a share) should be read: http://blogs.msdn.com/b/psssql/archive/2012/09/07/bulk-insert-and-kerberos.aspx. Adam offers two approaches:
Enable constrained delegation for the machine (as opposed to the sqlservr.exe startup) account, or use a SQL Server login. Adam mentions two other approaches (which he does not recommend).
An aside, the latter half of the OP's message "(failed to retrieve text for this error. Reason: 15105)" may be related to a SQL Server startup account lacking rights documented within SQL Server's "Configure Windows Service Accounts and Permissions" topic, such as SeAssignPrimaryTokenPrivilege.
Did you ever resolve this issue? I recently had a similar problem and discovered that the best way to resolve it was to use a SQL login.
Initially, having read the notes here I thought if I just granted read permissions to the Windows account with which I was connecting to the SQL Server that would be okay, but even when I granted read access to Everyone, I still couldn't read in the file.
I believe the reason is something to do with SQL Server impersonating the windows user and attempting to access the UNC share, which is delegation and which is not allowed unless explicitly enabled. There are some notes here which may help. This is the constrained delegation of which you speak and I couldn't get it to work either!
Bottom Line: I just used a SQL Login and made sure the SQL Server Process account had read permissions on the share (by granting read to Everyone in my case) and it worked.
In order to bulk insert with AD users, the SQL service it self has to be running as a domain user and has to have the AD permission to be able to delegate authentication. Same if you wanted to run linked servers with ad users. Here is the link for AD and linked servers, but the permission are the same.
Linked Servers and Active Directory
The server must have an SPN registered by the domain administrator.
The account under which SQL Server is running must be trusted for delegation.
The server must be using TCP/IP or named pipes network connectivity.
Related
I have a website on Heroku that uses JawsDB MySQL. The app is crashing with "ConnectionRefused." When I try to log into the database directly through MySQL Workbench, it returns "unable to connect to localhost," suggesting to ensure that MySQL is running on the amazonaws.com address, that port 3306 is open, and that the username has rights with the right password.
When I go to the JawsDB settings page, the Server Status indicates the "Status" is "BACKING-UP." It's been that way for hours.
Is this something that Heroku needs to fix? Amazon? How do I get it unstuck? I'd dump the whole thing and start a new database except I would need to get a backup of my data...which I can't get because I can't access the database.
HACKERS BEWARE
This issue could be related to recent Heroku security breach on April 7 - April 13, 2022 where many Heroku accounts and our applications were compromised! The following is statement from Salesforce (Heroku) Security Team (5/4/2022):
"... Our investigation also revealed that the same compromised token was leveraged to gain access to a database and exfiltrate the hashed and salted passwords for customers’ user accounts. For this reason, Salesforce is ensuring all Heroku user passwords are reset and potentially affected credentials are refreshed. We have rotated internal Heroku credentials and put additional detections in place. We are continuing to investigate the source of the token compromise."
I contacted Heroku who then contacted JawsDB. They did magic in the background to move my DB to an alternative location on Amazon, and it was fixed.
We are creating SSIS packages for our data feeds. Our connection manager has our individual usernames and passwords. Is there a way we can create environmental variables?
Also when we deploy our packages to dev or test or prod environments, do we need service accounts?
Yes, you can protect credentials.
Create Parameter variable and then select sensitive. This will block out the credential with astericks '*'
Set both the project and package Protection Level as either EncryptSensitiveWithPassord, or EncryptSensitiveWithUserKey. Note: failure to set both will result in a compiler failure that says the consistency check of the Protection Level failed - this is required.
Deploy the package to the server. Note: if the package is encrypted then anyone else who tries to open the package will be unable to do so. This is an important point for deployment purposes. It's not a big deal if you are handing off the *.ispac file, but if your company deploys via the DBA opening the solution and deploying from there then they will be unable to do so. I think you can do this if you EncryptWithPassword and then share the password, but EncryptWithUserKey will not work.
Set the Environment variable value on the server and then mark it as sensitive as well.
I don't recommend storing user credentials in protected parameters, but SSIS has been designed with this in mind if necessary. I much prefer doing everything through Windows Authentication and I highly recommend you to do the same if that is available to you.
Regarding service accounts. Yes, you can use those. In fact, it is highly recommended that all production deployments use service accounts where the concept of least privilege is implemented. So, the service account should only be granted the bare minimum level of privileges necessary. Said differently: DO NOT GRANT THE SERVICE ACCOUNT ADMINISTRATOR PRIVILEGES. This means specifying each individual privilege on each object. For example, SELECT only on dimension tables and SELECT/INSERT/UPDATE/DELETE on fact tables.
The best way for determining what privileges to grant is to go through the package and identify all tables that are touched as well as the command that is used when touching them. So, you will need to look at the following: Execute SQL task, OLE DB Source, OLE DB Destination, OLE DB Command, Lookup, etc.
I am unable to kill some SQL Server agent jobs. The task state continues to be running and the command stays in KILLED/ROLLBACK. The job executes queries against OSI's PI system via OLEDB linked server and Oracle. The only way I have found so far to kill these jobs is by restarting SQL server (not a preferred method).
I found following article
https://connect.microsoft.com/SQLServer/feedback/details/187192/openquery-to-linked-server-hangs-leaving-spid-with-open-tran-that-cannot-be-killed-then-templog-ldf-grows-without-limit-requires-sql-server-restart-on-production-servers
Apparently several people have this issue using openquery through a linked server that is not SQL Server. I'm reposting the work-around that BReuter posted on above article:
posted by BReuter on 1/30/2007 at 2:21 PM
*I have experianced the exact behavior and have found a combination of software which stablized our environment.
There were three key ingredients I found:
1) Make sure you do not have ANY linked servers using Microsoft OLEDB Provider for Oracle, instead use Oracle Provider for Oracle(version 9.2.0.4 is what I have in production).
2) Do not allow the linked server to run "in process". This took some research, but it is possible to run the linked server out of the SQL memory space by following the directions below.
3) I'm running SQL 2005 SP1 on W2K3, but I believe the OLEDB Provider is the key and not the OS or DB version.
The default security settings are too tight to run the Oracle OLEDB provider (OraOLEDB) out-of-process. Further, the default settings for MS DTC do not allow network communication.
Control Panel-> Administrative Tools-> Component Services
Drill to Component Services-> Computers
a. Right-click My Computer-> Properties
MSDTC tab -> Security Configuration button (screenshot below)
a. Network DTC Access – checked.
b. Allow Inbound / Outbound – checked.
c. No Authentication Required – This simulates the windows 2000 security settings.
d. Enable XA transactions – the type of transaction implemented by OraOLEDB provider.
Drill to Component Services-> Computers-> My Computer-> DCOM Config
a. Right-click MSDAINITALIZE-> Properties
Security tab (screenshot below)
a. Access Permissions -> Customize.
b. Press “Access Permissions” Edit button.
c. Give the SQL Server Service account “Local Access” permission.
d. Repeat for “Launch and Activation”.*
If they are large transactions, it might be that the server is actually still performing the rollback which might take some time.
This page
http://www.jaygeiger.com/index.php/2015/03/03/how-to-kill-a-frozen-linked-sql-server-connection/
provides a workaround.
It consist in manual TCP connection termination. It's not an ideal solution but it's the best one I know. It's better than having to restart the entire SQL Server.
Btw. I found that link at https://connect.microsoft.com/SQLServer/feedback/details/187192/openquery-to-linked-server-hangs-leaving-spid-with-open-tran-that-cannot-be-killed-then-templog-ldf-grows-without-limit-requires-sql-server-restart-on-production-servers page mentioned in Ahd's post
for me killing the OLEDB external resources did not worked and i unfortunately had to restart the SQL server instance to fix this issue always
i my cases it have select with OPENQUERY from oracle linked servers or SharePoint lists which simply has a simple error like bad password and it cannot resolve the error and goes and never come back until you restart the service
Transactions that get stuck in KILLED/ROLLBACK can be canceled by killing transactions on local server. If the query is cross-server and you don't want to wait for the rollback, you have to go to the remote server and kill the transaction as well as kill it on the local server.
This applies to any database system.
I have a website on a shared server . it uses Sql server as database ( also shared ) with an account limited to execute procedures . now I wonder if anyone who has access to sql server could read and manipulate my data without having my credentials ( by using thier own ) ? if so how can I track users logged into my database and more importantly stop them ?
There are two ways to connect to SQL using a SQL login or using windows authentication. These are your logins to the server. To then connect to the database you have to be granted rights to the database. So as long as your login is the only one that has been granted rights to the DB then you have no need to worry.
Depending on the rights you have on the DB could set up roles within the DB and allocate user(s) to the roles. This way the only way anyone can do anything in your db is if they are a member of the roles you have set up (or they have an SA account on the server or have also been set up with dbo rights on the database).
Again depending on what rights you have on the server you could set up a trace (google sp_trace_create and sp_trace_setevent) to capture logins to your database this writes to the harddrive of the server and then you can use fn_trace_gettable to query this data.
The problem goes beyond Execute permissions and logging access...
The bottom line is that you and your data are at the mercy of the the host. If they can be trusted and are competent, your data should be safe. However, if you are unsure, or if you are storing data that is too valuable to risk, your only choices are to either encrypt the data so that nothing useful can be gleamed from it and make sure you have a backup, or find a different host who will provide appropriate safeguards.
I am using SQL Server 2008 and I am getting an Access Denied error while trying to do a BULK INSERT which is accessing a network path, in the form: \\network_ip_address\Localdiskname$\filename_with_extenstion.
BULK
INSERT TableName
FROM '\\10.1.10.100\d$\Temp.csv'
How should I write the query correctly? How do I specify network authentication credentials such as username and password in the query?
A couple things here -
There may be a permissions issue on actually running the bulk insert command. Make sure that the account you are running the bulk insert operation from has the Fixed Server Role of bulkadmin or grant Administer Bulk Operations permissions which allows bulk insert operations to happen.
Also for the UNC path - You can't specify network authentication credentials in the query. This is going to access that path as the service account that SQL is running under.
I would verify:
SQL Server Is Running as an account OTHER than local system so it can access network resources
Verify that the permissions exist for that account that SQL is running under has rights on the network resource.
This article in SQL Server books online tells you all you need to know about bulk operations and has sections on permissions and network security.