I have SSIS package, which uses winscp to execute some commands in a text file to upload files to a FTP site. I use DTExec.exe to execute this SSIS package. Right now, I need to show winscp FTP log including display file name uploaded on FTP site. Does someone know how to capture the log? I use '>' and it does not work.
WinSCP can generate its own log using /log command-line switch (or /xmllog):
winscp.com /script=script.txt /log=winscp.log
If you want to just capture WinSCP console output:
Make sure you are using winscp.com, not winscp.exe.
Run winscp.com via cmd.exe, as that is what understands the >.
cmd.exe /C winscp.com /script=script.txt > winscp.log
In both cases, you will typically need to use full paths to all files (winscp.com, script.txt, winscp.log).
Related
I am having couple of files present at WinSCP folder location USERDATA.
Files present are staring with following names: ABC_XXXX.txt and XYZ_XXXX.txt.
I am interested in downloading all the files which are starting with ABC_ prefix only.
Below is my Execute Process Task configuration:
Executable- C:\Program Files (x86)\WinSCP\WinSCP.exe
/log=c:\path\to\log\winscp.log /command "open sftp://username:password#example.com/" "get /USERDATA/User_file.txt C:\User\Local\" "exit"
It's running fine when I am downloading a single file. How can we parametrize it to fetch the specific files which starts with prefix ABC_ only. Is there any way we can set the expression for this?
Use file mask ABC_*:
... "get /USERDATA/ABC_* C:\User\Local\" ...
I am running Jenkins on a script, that generates a junit.xml report file and other files. However, all those files are zipped by the script, hence Jenkins cannot find it.
Is there a way to make Jenkins unzip the .zip file , find my particular junit file and generate the run results ?
All this is in Linux.
Thanks
Jenkins has the ability to execute arbitrary shell commands as a build step, just add the 'Execute Shell' step to your build and put in the commands you want (presumably 'unzip' would be among them).
Once you've extracted the xml, provided your internal tool generates it using this schema the JUnit plugin just needs the path you extracted to and it will show the results on the build page.
If you have the option, I would really suggest executing your tests via gradle or maven, as outputs from those tasks will produce a report that Jenkins (and other tools) can already read, and can streamline the job setup process for your users. But, if you can't get away with that, the above should work for you.
Kindly see this screen cast to get better idea about our requirement:
https://www.screenr.com/QmDN
We want to automate the Text Datasource Generation and connection to MS Excel in order to make it easier to the end-user to connect to the Text Datasource (CSV) to MS Excel so that they can generate their own reports.
The steps I have in mind:
Use WinSCP FTP Client with Scripting
Write script to get the most recent updated file from FTP Folder
Or instead of step 2, download all generated files from FTP to a Shared Folder on the Network.
Get the most recent version of the Generated CSV File
Rename the file to the Standard Naming Convention. This must be the name used in MS Excel as the CSV Text Datasource.
Delete all other files
I developed sample script that can be used by WinSCP to download the files from FTP folder:
# Automatically abort script on errors
option batch abort
# Disable overwrite confirmations that conflict with the previous
option confirm off
# Connect
open CSOD
# Change remote directory
cd /Reports/CAD
# Force binary mode transfer
option transfer binary
# Download file to the local directory d:\
#get "Training Attendance Data - Tarek_22_10_21_2014_05_05.CSV" "D:\MyData\Business\Talent Management System\Reports\WinCSP\"
get "*.CSV" "D:\MyData\Business\Talent Management System\Reports\WinCSP\Files\"
# Disconnect
close
exit
Then, I can schedule the above code to run periodically using this command:
winscp.com /script=example.txt
The above sample is working fine, but the main problem is how to identify the most recent file, so that I can rename it, and delete all the other files.
Appreciate your help.
Tarek
Just add the -latest switch to the get command:
get -latest "*.CSV" "D:\MyData\Business\Talent Management System\Reports\WinCSP\Files\"
For more details, see WinSCP article Downloading the most recent file.
You don't specify the language you use, here a Ruby script that downloads the most recent file of an FTP path. Just to demonstrate how easy and terse this can be done with a scripting language like Ruby.
require 'net/ftp'
Net::FTP.open('url of ftpsite') do |ftp|
ftp.login("username", "password")
path = "/private/transfer/*.*"
# file[55..-1] gives the filename part of the returned string
most_recent_file = ftp.list(path)[2..-1].sort_by {|file|ftp.mtime(file[55..-1])}.reverse.first[55..-1]
puts "downloading #{most_recent_file}"
ftp.getbinaryfile(most_recent_file, File.basename(most_recent_file))
puts "done"
end
I am using SSIS 2005 to do some SFTPtasks using WinSCP. I googled about the error but couldn't make it work. I got this error:
[Execute Process Task] Error: In Executing "C:\Program Files\WinSCP\WinSCP.exe" "-script=C:\Documents and Settings\nian_z\Desktop\temp\SSISMovingSOA\removeSOA8.txt" at "C:\Program Files\WinSCP", The process exit code was "1" while the expected was "0".
Here is my WinSCP script that I use:
option batch abort
option confirm off
#open sftp://user:password#server:22
#cd /m/vo/Cont/fileftp
get OrderOutbound*
close
exit
For above script, I even tried leaving only one statement at a time, but still got error.
Here is the setup of the execute process task.
This is not an answer but an attempt to help you find a solution to your problem.
Try this:
Try the following steps to find out if your script is actually working outside of SSIS or not.
Click Windows Start and click Run...
Type cmd to open Command Prompt.
On the command prompt window, type the following command at the prompt to switch to the WinSCP installation directory. I have the WinSCP installed in the following directory. Change the path according to your environment settings.
cd "C:\Program Files (x86)\WinSCP"
If your WinSCP script file removeSOA8.txt is located in the path C:\Documents and Settings\userid\Desktop\temp\SSISMovingSOA\ with spaces, then type the following command by enclosing the script path in double quotes to run the script and also use the /log option to capture all the status messages.
WinSCP.exe "/script=C:\Documents and Settings\userid\Desktop\temp\SSISMovingSOA\removeSOA8.txt" /log=C:\temp\WinSCP_log.txt
After the script executes, your will find that a log file named WinSCP_log.txt will be created in the path C:\temp. Read through the file to identify if there are any error messages.
Attempt to run FTP in SSIS:
I tried downloading a file from FTP using WinSCP with the following script:
option batch abort
option confirm off
open ftp://myuserid:mypassword#ftp.myftpsite.com:21 -passive=on
cd /root/somefolder/
option transfer binary
get SomeFileOnFTP.txt c:\temp\
close
exit
Here are the settings how I have configured the Execute Process Task within the SSIS package.
The process ran successfully in BIDS.
Hope that gives you an idea.
If you are running the script from SQL Server job agent then mostly you will get this exit error.
Scenario: In my case I faced when i m trying to download a file from sFTP server.
Solution :
Step 1 - Create a batch file
#echo off
CD "C:\Program Files (x86)\WinSCP"
winscp.com /ini=nul /script=C:\path\download_script_pm.txt
Exit
Note: When calling using SQL Server Job agent make sure to NULL out the “ini” configuration else it will throw up this error
Step2 – create a txt file (winscp script)
option echo on
option batch on
option confirm off
open sftp://user:password#sftp.xxxx.com/ -hostkey="ssh-edXXXX-xx-xx-xx-xx-xx-xx-xx-xx-xx-xx-xx-xx-xx-xx-xx-xx"
option confirm off
get -filemask=">=1D" /sftp_Path/File_Name_.csv
C:\local_path_for_downloaded_file\"
exit
Note: make sure to use hostkey. To obtain host key you need to do the following :
1. Open winscp app tool and login to the server
Click Session -> Generate Session / URL code option
Check SSH Host key and copy the finger print value and use in the winscp hostkey value
Earlier I was getting the same error but now is working fine for me.
SSIS Execute Process Task :
Executable - C:\SSIS\SSIS_2008_Projects\HRIS_RepomanFusion_Load\WinSCP\WinSCP.com
Arguments - "/script=wscpBatchTest.txt" /log=C:\SSIS\SSIS_2008_Projects\FTP_Load\WinSCP\WinSCP_log.txt
Working Directory - C:\SSIS\SSIS_2008_Projects\HRIS_RepomanFusion_Load\WinSCP
Here is the WinSCP script that I use: wscpBatchTest.txt
option batch
option confirm off
open sftp://username:password#ftp.test.com
option transfer binary
cd /home/SAFAA
get employeedetails.csv C:\SSIS\SSIS_2008_Projects\SAFAA\InputDirectory\
close
exit
I tried to do this in Hudson but:
I have a script in Perl on my server (windows): lets say: d\util\demo.pl I want to have it running in Hudson. so I go to Hudson, create new job, go to: Build Execute Windows batch command and add: perl.exe d\util\demo.pl
I got this error: 'perl.exe' is not recognized as an internal or external command, operable program or batch file.
please help!
It can't find the perl.exe in the path of the agent that is running the task. Verify that perl is properly installed AND that the path where perl.exe was in stalled to is in you system path on EVERY agent that will run this task.
Can you run that command from any folder of the server?
If yes, than the Hudson server runs definitely under a different user account. Make sure that the user account Hudson is running under has all necessary environment variables set.
If not, than add the full qualified path to the perl.exe (e.g. C:\program files\perl\bin\perl.exe d:\util\demo.pl). If this doesn't help, you have to also set all environment variables (see "if yes").