A service running under domain account starts 20 ssis packages one by one to read 20 textfiles on a share. One of the packages (always the "same" filename) fails sporadically: one or more successfull reads, then one or more failed reads, then one or more successful reads and so on.
This behavior used to happen for file A.csv but not anymore and the file B.csv is the current victim. When the job fails I run succsessfully all the unfinished packages manually using the same domain account as the service uses.
"
Error during execution of package ...
Connection B failed validation.
The file name \share\folder\B.csv specified in the connection was not valid.
The file name property is not valid. The file name is a device or contains invalid characters.
"
Internet search provides comments like privileges ons the share, folders on the path and the file it self. But I think that is not the problem because the domain account does have these privileges and the reads (through the service) are sometimes succsessfull.
Any help will be well received
regards Bernódus
Try using path for shared folder with IP for example \\10.150.0.1\sharedFolder\B.csv
or without IP as \\server\sharedFolder\B.csv
If it's fail-pass then check whether that file is used by any other process\user, since it's on share so there are chances.
One of the workarounds is to have the network path mapped to a drive on the host server of the package.
You have make UNC paths to work on the calling machine, the server or the destination machine.
It could be a permissions issue.
Check permission on shared folder for -
Your personal Windows login account
Windows account which is owner of the SSIS Service
We use to have a sporadic copy file to shared drive failure. Everything would be executing fine for about a few weeks or a month or so and then failed task with error on "Network path" or "login failure".
We found that the job was taking too long with one of the tasks and some how it was giving errors.
We had to modify the packages to improve performance to fix the problem.
Check your SQL Agent Job history to see if the tasks were taking a long time when the failure occurred.
Is there anyway to improve on the package performance or separate the package into multiple packages to run in SQL Agent one package after another
Hope the information helps.
Related
I am having trouble moving a folder from an online drive, Drive :Z\zfilepath
to my local drive, Drive C:\cfilepath
So far, I have one 'file system task' that can copy a file from a source and move it to a destination.
I have correctly specified both filepaths and folder names into the task.
When I run the package, I get the following error:
[File System Task] Error: An error occurred with the following error message: "Access to the path ':Z\zfilepath' is denied.".
Is there some way for me to set up the package such that it will have the credentials to access the online :Z\ drive and run the task?
Z is likely a mapped drive. A drive is mapped per user account so the probable error is that the account that runs SQL Agent (or the credentialed user running a task) does not have that same drive mapped.
One maps a drive from the command line (Execute process task) with the following syntax
net use z: \\server\share PasswordInClearTextHere /user:domain\user /persistent:yes
I don't know enough about your environment. Maybe a one-time activity of mapping the drive will suffice /persistent:yes.
We had an issue where we always had to re-establish the mount when the job ran but we couldn't be certain the previous job succeeded (which had a final step of tear down the mount). Attempting to mount a drive when one already exists will throw an error so you might need another Execute Process Task to delete a mapped drive if it already exists. We ended up with a batch script that would check to see if the mount existed. If it did, it would delete it first. Then it established the mount.
This batch script approach allowed the DBAs to secure the file with file system permissions. The job could access the batch script but we could not so they didn't have to worry about us having access to yet-another-account.
The other, far simpler approach is to not deal with mounting drives and instead just use the UNC path. Assuming you don't have to worry about presenting different credentials than the current user, it makes life much easier.
\\server\share\MyFile.123.txt
I have psftp.exe installed on my server, and am using it to obtain files via sftp to be ingested into my sql server. I am using psftp.exe from the putty site to obtain it:
GET ZIP FILES:Error: In Executing "C:\Program Files\PuTTY\psftp.exe"
"XXXXXX#sftp1.XXXX.com -pw XXXXX -be -batch -b "D:\Code\XXXXX\XXXXXX.bat"" at
"D:\Data\ZIPFOLDER", The process exit code was "1" while the expected was "0".
The this works while in SSDT with no problems...however, when called by SQLSERVERAGENT on SQL SERVER, it fails.
to attempt to repair the issue, I have given NT SERVER\SQLSERVERAGENT full permissions on C:\Program Files\Putty\ and my Data drive D: where all of my code and data storage rests.
My problem is that while I think exit code 1 means a SFTP error (is it?) how do I troubleshoot?
Thanks.
UPDATE #1
as per instructions given by sandeep rawat, I added a windows user with administrative privileges and full control over the code and data sections of the server.
In addition, I created credentials associated with this use, and assigned a proxy user to those credentials.
Lastly, I reset the runas section in SQL SERVER AGENT's primary job to that proxy user.
Plus the setting in Internet Options.
I am still getting the same response.
THanks.
This type of issue generally happens when sql AGENT try to launch the DOS window which is the reason for the package to hang. and fails.
1 Change Window style property to hidden in process tab
2 Give the cmdExec permission to your SQL Agent account.
https://www.mssqltips.com/sqlservertip/2163/running-a-ssis-package-from-sql-server-agent-using-a-proxy-account/
3 Add the local drive to your trusted Intranet Sites by opening Internet Explorer and go to Internet Options > Security > Click Local Intranet > Sites and add your drive location as shown below
I am trying to create a simple SSIS package that can be deployed to a VM on another domain (same computer) that has SSIS on it. The package that I have created will transfer data if I simply run the data flow from SSDT on the source computer, but when I try to deploy and execute it, it appears to have 2 errors. The first being that it cannot open the datafile "C:\SSIS\Product Data.txt". The second being that the Flat Files Source 1 failed the pre-execute phase. I think that the second error is caused from the first. Does anyone know how to create a package that can transfer data to another computer without the flat file source?
Thanks in advance!
C:\SSIS.... will refer to the local C drive of the computer that is executing the package. So if you have it on the VM it would be at the \VM\C$. Typically it means 1 of 2 things.
1) The file path is not correct and you are not referencing where the file actually is.
2) the user you are executing the package as does not have permission to that file.
So where is the file is it on the C drive of the machine you are executing the SSIS package on? The answer to that wasn't clear in your question.
Second how is the file being executed? manual execution such as DTS_Exec? SQL Agent? T-SQL? If manual then the user logged in executing the task must have permission, if SQL Agent then the sql agent service account and possible the sql server service account have to have permission to the file, and T-sql sql server service account needs to have permission.
When it is local file path then a local user or domain user will work for the service accounts. If network path then the local user used for service account won't work and you will need to setup a way to run as a different credential.
I have an SSIS package that has run for months daily with no problem. Suddenly it is failing every day with the error above. No changes to the package or the stored procs it runs have been pushed, to my knowledge, no changes to the database environment have been made including permissions for the service account that owns the job.
This is a SQL Server 2008 package that connects to a SQL server 2008 R2 database. The configuration is through the ssisconfig table and it defines which server to use through an environment variable.
The component in question is using a stored proc. In the proc, all fields are explicitly cast to the same data type of the table the data is being staged too. The proc runs fine by itself. It is a relatively complex proc that has two table variables and a CTE.
I can't run it from my local on prod and dev is not failing. If I open the prod SSIS package while I am set to the environment variable that the job owner uses, I do not get a validation error on this component. ValidateExternalMetadata is set to true for this component. For the entire package though, Delay Validation is set to True. I do get a validation error that would be expected on a task using a variable that is set dynamically in an earlier task. This error, however, is many, many steps later than the step it is currently failing on.
Usually in the past when we have had this sort of unspecific error, I have been easily able to see the problem with the data that caused the sudden failure. But none of the data in the proc result set is even close to being incorrect for the datatypes defined for staging table for that field. None of the required fields are missing data either.
I am out of ideas for what else to look at. If the environment didn't change, the actual code didn't change and the data looks correct, what is left to try?
As I have commented for details error, you can try below link from Microsoft:
Debug SSIS Package while calling from Agent
Some info from link:
Reasons that the package may have failed are as follows:
The user account that is used to run the package under SQL Server Agent differs from the original package author.
The user account does not have the required permissions to make connections or to access resources outside the SSIS package.
The package may not run in the following scenarios:
The current user cannot decrypt secrets from the package. This scenario can occur if the current account or the execution account
differs from the original package author, and the package's
ProtectionLevel property setting does not let the current user
decrypt secrets in the package.
A SQL Server connection that uses integrated security fails because the current user does not have the required permissions.
File access fails because the current user does not have the required permissions to write to the file share that the connection
manager accesses. For example, this scenario can occur with text
log providers that do not use a login and a password. This scenario
can also occur with any task that depends on the file connection
manager, such as a SSIS file system task.
A registry-based SSIS package configuration uses the HKEY_CURRENT_USER registry keys. The HKEY_CURRENT_USER registry keys
are user-specific.
A task or a connection manager requires that the current user account has correct permissions.
Method 1: Use a SQL Server Agent proxy account Create a SQL Server Agent proxy account. This proxy account must use a credential that
lets SQL Server Agent run the job as the account that created the
package or as an account that has the required permissions.
This method works to decrypt secrets and satisfies the key
requirements by user. However, this method may have limited success
because the SSIS package user keys involve the current user and the
current computer. Therefore, if you move the package to another
computer, this method may still fail, even if the job step uses the
correct proxy account.
Method 2: Set the SSIS Package ProtectionLevel property to ServerStorage Change the SSIS Package ProtectionLevel property to
ServerStorage. This setting stores the package in a SQL Server
database and allows for access control through SQL Server database
roles.
Method 3: Set the SSIS Package ProtectionLevel property to EncryptSensitiveWithPassword Change the SSIS Package ProtectionLevel
property to EncryptSensitiveWithPassword. This setting uses a password
for encryption. You can then modify the SQL Server Agent job step
command line to include this password.
Method 4: Use SSIS Package configuration files Use SSIS Package configuration files to store sensitive information, and then store
these configuration files in a secured folder. You can then change the
ProtectionLevel property to DontSaveSensitive so that the package is
not encrypted and does not try to save secrets to the package. When
you run the SSIS package, the required information is loaded from the
configuration file. Make sure that the configuration files are
adequately protected if they contain sensitive information.
Method 5: Create a package template For a long-term resolution, create a package template that uses a protection level that differs
from the default setting. This problem will not occur in future
packages.
Above link will help you through logging and other possible scenarios of failure of your package from SQL Agent. I suspect that your user of Agent has been corrupted. But that is an assumption. You first enable the logging of SSIS by following link :
Enable SSIS Package Logging
Some info from link
To set the logging level for a package by using the Execute Package
dialog box
In SQL Server Management Studio, navigate to the package in Object Explorer.
Right-click the package and select Execute.
Select the Advanced tab in the Execute Package dialog box.
Under Logging level, select the logging level. See the table below for a description of available values.
Complete any other package configurations, then click OK to run the package.
Hope this will help you in the first place. If not please let me know.
I have an File System Task that copies a file from one directory to another. But i executing package with different machine very first time I got device is not ready error message from file system tack component.
Any one guide me.
If you are executing package from SQL Server Job check credentials from which SQL Agent Service is running. If credentials is OK then it maybe network (DNS) or Active Directory issue.