Can we read a csv file from SFTP folder location without downloading it anywhere on local?
I can use FTP task component but it will download file locally, i want to read file from sftp itself.
Is it possible? I am using SSIS 2008.
Thanks,
Chaitanya
In general, FTP does not allow to read file directly from remote ftp server without downloading it to local folder. You have to download/copy this file to local folder for further processing.
you can delete this locally downloaded file after processing it.
But I would like to know why you decided to read it directly? Any specific reason.
There is another approach using script task. you need to write code to download the file into filestream using FTPWebRequest class. in ssis script editor.
FtpWebRequest request = (FtpWebRequest)WebRequest.Create("ftp://www.contoso.com/test.htm");
request.Method = WebRequestMethods.Ftp.DownloadFile;
// This example assumes the FTP site uses anonymous logon.
request.Credentials = new NetworkCredential ("anonymous","janeDoe#contoso.com");
FtpWebResponse response = (FtpWebResponse)request.GetResponse();
Stream responseStream = response.GetResponseStream();
StreamReader reader = new StreamReader(responseStream);
Console.WriteLine(reader.ReadToEnd());
Console.WriteLine("Download Complete, status {0}", response.StatusDescription);
reader.Close();
response.Close();
Related
I am currently trying to send a single file through to an FTP site, using an SSIS FTP task to which it is failing and I'm receiving the error:
[FTP Task] Error: Unable to send files using "FTP Connection Manager".
Using the SSIS FTP Task, I am trying to send a file from a local drive to an FTP root folder. I have all the permissions to read and write. I have tried to delete the file on the FTP server (successful), move the file using WinSCP (successful), delete the file from the FTP folder (successful) as well as create a directory on the FTP site. However, when I try and send from a local folder to the root drive of the FTP site, I get the above-mentioned error. My local path includes the file I am wanting to send as well
C:\WebSites\Files\Import\SFTP\Extract\CE_CS_Shipment.xlsx
Is there anything else I can try because I am at my wits ends.
I seemed to have found a workaround because using the FTP Task in SSIS just wasn't budging.
I created the file on the FTP site, followed by copying the stream and that works perfectly.
FtpWebRequest request = (FtpWebRequest)WebRequest.Create(sftpServer + result);
request.Credentials = new NetworkCredential(sftpUser, strFTPPassword);
request.Method = WebRequestMethods.Ftp.UploadFile;
using (Stream fileStream = File.OpenRead(archiveFolder.Replace("//", "/") + #"/" + result))
using (Stream ftpStream = request.GetRequestStream())
{
fileStream.CopyTo(ftpStream);
}
This now works perfectly in the event that someone else is looking for or struggling with the same problem.
I'm using the FTP Task in SSIS to send files. They task succeeds but the files get uploaded to the wrong folder. Instead of being sent to the cg268301 folder they are being sent to the cg268300 despite selecting /cg268301 from the remote path field in the FTP task editor.
I've tried uploading this file to the /cg268301 file using an execute process task and it works fine. I just don't know why it won't work with the FTP task.
Kindly see this screen cast to get better idea about our requirement:
https://www.screenr.com/QmDN
We want to automate the Text Datasource Generation and connection to MS Excel in order to make it easier to the end-user to connect to the Text Datasource (CSV) to MS Excel so that they can generate their own reports.
The steps I have in mind:
Use WinSCP FTP Client with Scripting
Write script to get the most recent updated file from FTP Folder
Or instead of step 2, download all generated files from FTP to a Shared Folder on the Network.
Get the most recent version of the Generated CSV File
Rename the file to the Standard Naming Convention. This must be the name used in MS Excel as the CSV Text Datasource.
Delete all other files
I developed sample script that can be used by WinSCP to download the files from FTP folder:
# Automatically abort script on errors
option batch abort
# Disable overwrite confirmations that conflict with the previous
option confirm off
# Connect
open CSOD
# Change remote directory
cd /Reports/CAD
# Force binary mode transfer
option transfer binary
# Download file to the local directory d:\
#get "Training Attendance Data - Tarek_22_10_21_2014_05_05.CSV" "D:\MyData\Business\Talent Management System\Reports\WinCSP\"
get "*.CSV" "D:\MyData\Business\Talent Management System\Reports\WinCSP\Files\"
# Disconnect
close
exit
Then, I can schedule the above code to run periodically using this command:
winscp.com /script=example.txt
The above sample is working fine, but the main problem is how to identify the most recent file, so that I can rename it, and delete all the other files.
Appreciate your help.
Tarek
Just add the -latest switch to the get command:
get -latest "*.CSV" "D:\MyData\Business\Talent Management System\Reports\WinCSP\Files\"
For more details, see WinSCP article Downloading the most recent file.
You don't specify the language you use, here a Ruby script that downloads the most recent file of an FTP path. Just to demonstrate how easy and terse this can be done with a scripting language like Ruby.
require 'net/ftp'
Net::FTP.open('url of ftpsite') do |ftp|
ftp.login("username", "password")
path = "/private/transfer/*.*"
# file[55..-1] gives the filename part of the returned string
most_recent_file = ftp.list(path)[2..-1].sort_by {|file|ftp.mtime(file[55..-1])}.reverse.first[55..-1]
puts "downloading #{most_recent_file}"
ftp.getbinaryfile(most_recent_file, File.basename(most_recent_file))
puts "done"
end
I am trying to open local hard drive files inside AS3. I'm getting a security error with this code:
var soundFile:URLRequest = new URLRequest("c:\slushy.mp3");
Is it possible to URLload an absolute path on the hard drive?
Edit: so this appears to be a sandboxing issue. Drat. Is it possible to load the local file via PHP and send to flash?
PHP is a server language, so the final protocol is http.
The you must to acces by file:/// to the local file, but if you want to share the resources over Internet, you must upload your files to folder in the root of site.
By example: http://www.mysite.com/music
Then you can load the file:
var soundFile:URLRequest = new URLRequest("http://www.mysite.com/music/slushy.mp3");
Requisite: you must to create the directory "music" in server web application directory and upload the file.
I was wondering if it would be possible to write a file, not to the local system, but to a connected server or other network path.
Would I have to use an external interface for this or can I entrust this to the AIR framework?
I'm pretty sure that AIR has certain security precautions to prevent this.
What kind of network path? SMB, FTP, HTTP, WebDav, .. ?
If it's mounted on your local PC, then you should be able to write to it just like writing to any other drive or filesystem.
to write a new file to a folder on a networked drive in OSX:
var targetDir:File = new File("/Volumes/Data/SomeFolder");
targetDir.createDirectory(); // ensure directory exists, create if not
var targetFile:File = targetDir.resolvePath("newFile.ext");
var fileStream:FileStream = new FileStream();
try {
fileStream.open(targetFile, FileMode.WRITE);
fileStream.writeBytes(byteArray); // or what/however you want to write
fileStream.close();
} catch (e:Error) {
trace(e);
}
i assume for Windows you would just swap the network drive path in the first line. you should not specify a protocol (e.g. file://, smb://, etc); the format of the parameter for the File constructor is the native path (as it would appear in a terminal / command shell).
to get the path in OSX, use the Finder to navigate to the target networked folder. begin dragging the folder somewhere else, hit apple+space to open Spotlight, and continue dragging the folder into Spotlight. alternately, open a Terminal window and drag the folder into the Terminal window.