I have a SQL job that I would like to copy files in a directory out to a directory on another server. Here is the CMD that I am running
xcopy \DB1\TEST*.* \DB2\foo /S /Y
And I get this error
Executed as user: DOMAIN\SA$. File not found - . 0 File(s) copied. Process Exit Code 0. The step succeeded.
I have also tried the copying to the same server with same results as well as just trying to grab one file and copying
I have added the DOMAIN\SA account as an admin on the servers and still getting this. Is this just a bug or am I missing some permissions setting?
Related
This question already has answers here:
If there is an error, how can I exit from SSIS package
(1 answer)
Fail SSIS task when the mask to select files to be uploaded to FTP server using WinSCP matches no files
(1 answer)
Closed 3 years ago.
I have a SSIS package that executes the WinSCP.exe via execute process task and download the set of files to some common folder location.
Files names are:
Text1.csv
Test2.csv
and so on..
After that I am running another set of Execute process tasks which will use the C:\Windows\System32\cmd.exe which start copying downloaded files to different respective locations.
Below is my command line arguments to first download the files at some common location:
/log=G:\USER_DATA\USER_USER_SYNC\SFTP_LOG\user_sync_winscp.log /command "open sftp://bisftp:*UFVy2u6jnJ]#hU0Zer5AjvDU4#K3m#stransfer.host.com/ -hostkey=""ssh-rsa 2048 9b:63:5e:c4:26:bb:35:0d:49:e6:74:5e:5a:48:c0:8a""" "cd /DATA" "get -filemask=">=today" Test1.csv Test2.csv G:\USER_DATA\USER_USER_SYNC\Download\" "exit"
Below are my command line arguments to copy the files to different locations:
/C copy /b G:\USER_DATA\USER_USER_SYNC\Download\Test1.csv G:\USER_DATA\USER_USER_SYNC\Testing1
/C copy /b G:\USER_DATA\USER_USER_SYNC\Download\Test1.csv G:\USER_DATA\USER_USER_SYNC\Testing2
I have few questions here.
Since I am using filemask to download the latest updated files only.
If first Execute process task run successfully then it will set the exit code returned as '0' and then next Execute process task gets executed to copy the files to different locations.
Everything runs fine till this end.
However there can be a case when first execute task is not able to download the files to common folder location.
Reasons of failure could be that other party has not put the latest files on SFTP server and due to date mismatch, it was not able to download the files in first place or could be some other network issues as well.
So it throws an exception that like "The process exit code was "1" while the expected was "0"." due to when my next subsequent execute process task run and it's getting failed as it has nothing to copy or move.
Can we capture the exit code returned to some variable, so that we can use it further in some expression to decide whether to execute another task or not?
Also I am using different execute process task to copy the files. Can we combine separate copy steps into one and use one Execute process task?
I have a folder 'DATA' at SFTP location from where I need to download the set of files to some common location and then copy the respective files to different folder location.
File Names are:
Test1.csv
Test2.csv
Test3.csv
Test4.csv
Test5.csv
I want that files first gets downloaded to below location:
G:\USER_DATA\USER_USER_SYNC\Download
Since these files are related to different schema and have to processed separately by each different ssis packages for further transformations and loading.
For some reasons we have to first keep it at some common location and then move or copy afterwards.
Here's my command line argument.
/log=G:\USER_DATA\USER_USER_SYNC\SFTP_LOG\user_sync_winscp.log /command "open sftp://username:password#stransfer.host.com/" -hostkey=""ssh-rsa 2048 9b:63:5e:c4:26:bb:35:0d:49:e6:74:5e:5a:48:c0:8a""" "get /DATA/Test1.csv G:\USER_DATA\USER_USER_SYNC\Download\" "exit"
Using above, I am able to download a given file one file at a time.
Since, I need to have first it at some common folder location. Hence I am planning to add another Execute process task to copy the files.
/C copy /b G:\USER_DATA\USER_USER_SYNC\Download\Test1.csv G:\USER_DATA\USER_USER_SYNC\Testing1
/C copy /b G:\USER_DATA\USER_USER_SYNC\Download\Test1.csv G:\USER_DATA\USER_USER_SYNC\Testing2
and so on...
I am looking for some way, using which we can download all the available files to some common folder location and then move or copy to different folder locations.
I have changed the design and followed a new approach. Thanks to Martin for fixing the sftp related issues and continuous support.
New SSIS package has below tasks:
Step1. It will look for latest updated files on sftp server and download the given files Test1.csv and Test2.csv to location G:\USER_DATA\USER_USER_SYNC\Download\
Here's my command line arguments:
/log=G:\USER_DATA\USER_USER_SYNC\SFTP_LOG\user_sync_winscp.log /command "open sftp://bisftp:*UFVy2u6jnJ]#hU0Zer5AjvDU4#K3m#stransfer.host.com/ -hostkey=""ssh-rsa 2048 9b:63:5e:c4:26:bb:35:0d:49:e6:74:5e:5a:48:c0:8a""" "cd /DATA" "get -filemask=">=today" Test1.csv Test2.csv G:\USER_DATA\USER_USER_SYNC\Download\" "exit"
Step-2. Since my requirement was to further copy each file to different folder location, so that respective process can pick corresponding file and start transformation and loading it into sql server.
This step will execute the Window cmd process and copy Test1.csv to new location as
G:\USER_DATA\USER_USER_SYNC\Testing1
command line arguments as:
/C copy /b G:\USER_DATA\USER_USER_SYNC\Download\Test1.csv G:\USER_DATA\USER_USER_SYNC\Testing1
Like wise I have another Execute process task to copy Test2.csv to new location as
G:\USER_DATA\USER_USER_SYNC\Testing2
command line arguments as:
/C copy /b G:\USER_DATA\USER_USER_SYNC\Download\Test2.csv G:\USER_DATA\USER_USER_SYNC\Testing2
The given solution is working fine, However there are couple of things which still needs to be handle.
Since I am downloading the latest file only using -filemask=">=today". Everything runs fine if execute process task is able to find the latest files on sftp server. If it's not there, than the next subsequent execute process task is failing with below error message.
The returning The process exit code was "1" while the expected was "0"
Here what I understand is that it's failing as it has nothing to copy or move.
Is there any way by which we can capture the exit code returned from first execute process task and store it into some variable, so that we can use expression to decide that whether to start next task or not.
Second, as you can see that I am using two execute process task to copy files from one location to another. Can we do anything to combine both these two commands into one execute process task?
Any suggestion most welcome and also i think that this issue needs to be addressed as a separate question.
I am using SSIS 2005 to do some SFTPtasks using WinSCP. I googled about the error but couldn't make it work. I got this error:
[Execute Process Task] Error: In Executing "C:\Program Files\WinSCP\WinSCP.exe" "-script=C:\Documents and Settings\nian_z\Desktop\temp\SSISMovingSOA\removeSOA8.txt" at "C:\Program Files\WinSCP", The process exit code was "1" while the expected was "0".
Here is my WinSCP script that I use:
option batch abort
option confirm off
#open sftp://user:password#server:22
#cd /m/vo/Cont/fileftp
get OrderOutbound*
close
exit
For above script, I even tried leaving only one statement at a time, but still got error.
Here is the setup of the execute process task.
This is not an answer but an attempt to help you find a solution to your problem.
Try this:
Try the following steps to find out if your script is actually working outside of SSIS or not.
Click Windows Start and click Run...
Type cmd to open Command Prompt.
On the command prompt window, type the following command at the prompt to switch to the WinSCP installation directory. I have the WinSCP installed in the following directory. Change the path according to your environment settings.
cd "C:\Program Files (x86)\WinSCP"
If your WinSCP script file removeSOA8.txt is located in the path C:\Documents and Settings\userid\Desktop\temp\SSISMovingSOA\ with spaces, then type the following command by enclosing the script path in double quotes to run the script and also use the /log option to capture all the status messages.
WinSCP.exe "/script=C:\Documents and Settings\userid\Desktop\temp\SSISMovingSOA\removeSOA8.txt" /log=C:\temp\WinSCP_log.txt
After the script executes, your will find that a log file named WinSCP_log.txt will be created in the path C:\temp. Read through the file to identify if there are any error messages.
Attempt to run FTP in SSIS:
I tried downloading a file from FTP using WinSCP with the following script:
option batch abort
option confirm off
open ftp://myuserid:mypassword#ftp.myftpsite.com:21 -passive=on
cd /root/somefolder/
option transfer binary
get SomeFileOnFTP.txt c:\temp\
close
exit
Here are the settings how I have configured the Execute Process Task within the SSIS package.
The process ran successfully in BIDS.
Hope that gives you an idea.
If you are running the script from SQL Server job agent then mostly you will get this exit error.
Scenario: In my case I faced when i m trying to download a file from sFTP server.
Solution :
Step 1 - Create a batch file
#echo off
CD "C:\Program Files (x86)\WinSCP"
winscp.com /ini=nul /script=C:\path\download_script_pm.txt
Exit
Note: When calling using SQL Server Job agent make sure to NULL out the “ini” configuration else it will throw up this error
Step2 – create a txt file (winscp script)
option echo on
option batch on
option confirm off
open sftp://user:password#sftp.xxxx.com/ -hostkey="ssh-edXXXX-xx-xx-xx-xx-xx-xx-xx-xx-xx-xx-xx-xx-xx-xx-xx-xx"
option confirm off
get -filemask=">=1D" /sftp_Path/File_Name_.csv
C:\local_path_for_downloaded_file\"
exit
Note: make sure to use hostkey. To obtain host key you need to do the following :
1. Open winscp app tool and login to the server
Click Session -> Generate Session / URL code option
Check SSH Host key and copy the finger print value and use in the winscp hostkey value
Earlier I was getting the same error but now is working fine for me.
SSIS Execute Process Task :
Executable - C:\SSIS\SSIS_2008_Projects\HRIS_RepomanFusion_Load\WinSCP\WinSCP.com
Arguments - "/script=wscpBatchTest.txt" /log=C:\SSIS\SSIS_2008_Projects\FTP_Load\WinSCP\WinSCP_log.txt
Working Directory - C:\SSIS\SSIS_2008_Projects\HRIS_RepomanFusion_Load\WinSCP
Here is the WinSCP script that I use: wscpBatchTest.txt
option batch
option confirm off
open sftp://username:password#ftp.test.com
option transfer binary
cd /home/SAFAA
get employeedetails.csv C:\SSIS\SSIS_2008_Projects\SAFAA\InputDirectory\
close
exit
I have a .dtsx file (an SSIS package) that downloads files from an FTP server and imports data. It runs fine whenever I run it manually. However, when I schedule calling the package as a step in a SQL server agent job, it fails. The step it fails at is the one where I call a .bat file. The error in the job history viewer says this:
Error: 2009-05-26 12:52:25.64
Code: 0xC0029151 Source: Execute
batch file Execute Process Task
Description: In Executing
"D:\xxx\import.bat" "" at "", The
process exit code was "1" while the
expected was "0". End Error DTExec:
The package execution returned
DTSER_FAILURE (1).
I think it's a permissions issue, but I'm not sure how to resolve this. The job owner is an admin user, so I've verified they have permissions to the directory where the .bat file is located. I've tried going into Services and changing the "Log On As" option for SQL Server Agent, and neither option works (Local System Account and This Account). Does anyone have ideas as to what other permissions need to be adjusted in order to get this to work?
I tried executing just the batch file as a SQL Job step, and it gave more specifics. It showed that it failed when I was trying to call an executable, which was in the same directory as my .bat file, but not in the windows/system32 directory, which is where it was executing from.
I moved the executable to the system32 directory, but then I had no clue where my files were being downloaded to. Then I found that there's a property for the Execute Process Task (the one that executes the .bat) called WorkingDirectory. I set this to be the directory where the bat is located, moved the executable back into the same one as the .bat file, and it's now working as expected.
For me it was a permissions issue. Go to Environment --> Directories, then change Local directory to something the SQLAgentUser can access. I used C:\temp. Click the dropdown for Save, and choose "Set defaults".
Are you executing the SSIS job in the batch file, or is the batch file a step in the SSIS control flow?
I'm assuming the latter for this answer. What task are you using to execute the batch file (e.g. simple execute program task or a script task). If the latter, it looks like your batch file is actually failing on some step, not the SSIS script. I'd check the permissions of what your batch file is trying to access
In fact, it might be a better idea to rewrite the batch file as a script task in SSIS, because you'll get much better error reporting (it'll tell you which step in the script fails).
You could try executing the batch file using the runas command in a command window. If you try and execute it under the local system or network system account, it should give you a better error. If it does error, you can check the error level by going "echo %ERRORLEVEL%".
If it wasn't the latter, and you're executing the SSIS package via a batch file, why?
Are you possibly accessing a mapped drive in your .bat file? If so, you can't rely on the mapped drive from within the service, so you'd have to use UNC path.
I had the same error and I resolved it by logging on to the user account that runs the job, opened Coreftp site in question there, test the site access, made the change there (in my case, I had to reenter the new password) and now it works.
So yes, it is an issue of file access. This one is file access to the coreftp site in question.
I am creating a script on the fly to ftp some files from a remote computer. I create a file which is then called from the command line with
ftp -s:filename proxy
where filename is the file I just created. The file has code similar to the following:
anonymous#ip address
username
prompt off
binary
cd c:\destination directory
mget c:\source directory\*.*
quit
That doesn't work. Neither does the following:
anonymous#ip address
username
prompt off
binary
cd c:\source directory
mput c:\destination directory
quit
Obviously, I'm not so good at ftp. How, in what order, where in my file do I specify the place where I want the files to be put (destination directory, and also from where the ftp process is running), and where I want the files to come from (ip address computer which has files I want). Do I need to set the directory before starting the ftp process?
I'm running this in an SSIS package, and I'm not using the SSIS ftp task, because I don't want a failure if no files are found. If there's nothing there, that's cool. If there is something there, I want a copy.
(It was working in my development area, and now, when I'm trying to get files from a server that I truely have no access to except ftp, I'm not getting anything. See How to avoid SSIS FTP task from failing when there are no files to download? for an earlier, related question.)
Update:
Both of the answers below, listing lcd and cd, are correct. However, my example still failed, until I replaced the backslashes with forward slashes. In other words, my final, working result is as follows:
anonymous#ip address
username
prompt off
binary
lcd /destination directory
cd /source directory
mget *.*
quit
Are you looking for LCD and CD where LCD changes directory on the local machine? EG:
LCD c:\destination directory
mget c:\source directory\*.*
In most ftp clients you can set the working directory on the server with the command cd, and you set the working directory on the client with the command lcd.
But it is not clear to me what you are trying to do.
Are you trying to move or copy files that are on the ftp server to another location on the ftp server? As far as I know you cannot do that with ftp. If you wished to copy files from one folder on the ftp server to another, then I believe you would get a copy to the local system, and then reupload them to a new folder. If you wish to move files you can use the rename command.