SSIS Task - FTP Conecction [duplicate] - ssis

This question already has answers here:
Upload file to FTP site using VB.NET
(5 answers)
Closed 1 year ago.
All,
SSIS - trying to make an ftp connection to upload files to an ftp server using a script task in visual basic programming language, I have run into a problem. I have not found something similar in my searches so I would appreciate your help.
[Connection manager "FTP"] Error: An error occurred in the requested FTP operation. Detailed error description: 220
550 SSL/TLS required on the control channel
CODE:
Public Sub Main()
'
' Add your code here
'
Try
Dim cm As ConnectionManager
cm = Dts.Connections("FTP")
'Set the properties like username & password
cm.Properties("ServerName").SetValue(cm, "ftps.example.com")
cm.Properties("ServerUserName").SetValue(cm, "username")
cm.Properties("ServerPassword").SetValue(cm, "password")
cm.Properties("ServerPort").SetValue(cm, "port")
cm.Properties("Timeout").SetValue(cm, "0") 'The 0 setting will make it not timeout
cm.Properties("ChunkSize").SetValue(cm, "1000") '1000 kb
cm.Properties("Retries").SetValue(cm, "1")
'create the FTP object that sends the files and pass it the connection created above.
Dim ftp As FtpClientConnection = New FtpClientConnection(cm.AcquireConnection(Nothing))
'Connects to the ftp server
ftp.Connect()
'Build a array of all the file names that is going to be FTP'ed (in this case only one file)
Dim files(0) As String
files(0) = "C:\ local path file"
'ftp the file
ftp.SendFiles(files, "/remote path", True, False) ' the True makes it overwrite existing file and False is saying that it is not transferring ASCII
ftp.Close()
Catch ex As Exception
Dts.TaskResult = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Failure
End Try

...... I finally solved it. I changed the language script task to C# (it works in VB you just have to adapt the code) and used the code from this comment.
note: if the server is on a port other than 21 you have to specify it in the remote path. thanks.
public void Main()
{
// TODO: Add your code here
FtpWebRequest request =
(FtpWebRequest)WebRequest.Create("ftp://ftps.example.com:port/remote/path/file.zip");
request.Credentials = new NetworkCredential("username", "password");
request.EnableSsl = true;
request.Method = WebRequestMethods.Ftp.UploadFile;
using (Stream fileStream = File.OpenRead(#"C:\local\path\file.zip"))
using (Stream ftpStream = request.GetRequestStream())
{
fileStream.CopyTo(ftpStream);
}
}

Related

Download SFTP file using SSIS package

I want to create a SSIS package which need to download a file automatically and place it in our local.
Note: using process executive task and batch script files only.
In a new SSIS project, create a new package. Navigate to the Parameters tab, where we’ll create a handful of runtime values that will make the DownloadSFTP package more reusable.
pFilename: This is the file name to download from the server. Note
that we can also use wildcards (assuming they are supported by the
target server) – in the example above, we’ll be downloading all files
ending in “.TXT”.
pServerHostKey: This is used to satisfy a security mechanism built
into the WinSCP process. By default, WinSCP will prompt the user to
verify and add to local cache the host key when connecting to an SFTP
server for the first time. Because this will be done in an automated,
non-interactive process, getting that prompt would cause an error in
our script. To prevent this, the script is built to supply the server
host key to avoid the error, and also has the added benefit of
ensuring we’re actually connecting to the correct SFTP server. This
brief article on the WinSCP documentation site describes how to
retrieve the server host key for the target server.
pServerUserPassword: This is marked as sensitive to mask the
password. As part of the script logic, this password will be
decrypted before it is sent to the server.
Create a new script task in the control flow, and add all 7 of the parameters shown above to the list of ReadOnlyVariables.
Using the Main() function (which is created automatically in a new script task), create the Process object and configure a few of the runtime options, including the name of the executable and the download directory.
public void Main()
{
// Create a new Process object to execute WinSCP
Process winscp = new Process();
// Set the executable path and download directory
winscp.StartInfo.FileName = Dts.Variables["$Package::pWinSCPLocation"].Value.ToString();
winscp.StartInfo.WorkingDirectory = Dts.Variables["$Package::pDownloadDir"].Value.ToString();
// Set static execution options (these should not need to change)
winscp.StartInfo.UseShellExecute = false;
winscp.StartInfo.RedirectStandardInput = true;
winscp.StartInfo.RedirectStandardOutput = true;
winscp.StartInfo.CreateNoWindow = true;
// Set session options
string sessionOptionString = "option batch abort" + System.Environment.NewLine + "option confirm off";
The next step is to create the input strings that will make the connection and download the file. At the bottom of this snippet, there are 3 variables that will capture output messages, error messages, and the return value, all of which will be used to log runtime information.
// Build the connect string (<user>:<password>#<hostname>)
string connectString = #"open " + Dts.Variables["$Package::pServerUserName"].Value.ToString()
+ ":"
+ Dts.Variables["$Package::pServerUserPassword"].GetSensitiveValue().ToString()
+ "#"
+ Dts.Variables["$Package::pServerName"].Value.ToString();
// Supplying the host key adds an extra level of security, and avoids getting the prompt to trust the server.
string hostKeyString = Dts.Variables["$Package::pServerHostKey"].Value.ToString();
// If hostkey was specified, include it
if (hostKeyString != null && hostKeyString.Length > 0)
connectString += " -hostkey=\"" + hostKeyString + "\"";
// Build the get command string
string getString = "get " + Dts.Variables["$Package::pFilename"].Value.ToString();
// Create output variables to capture execution info
string outStr = "", errStr = "";
int returnVal = 1;
With all of the options configured, it’s time to invoke WinSCP.com. The try/catch block below will attempt to connect and download the specified file from the server.
// This try/catch block will capture catastrophic failures (such as specifying the wrong path to winscp).
try
{
winscp.Start();
winscp.StandardInput.WriteLine(sessionOptionString);
winscp.StandardInput.WriteLine(connectString);
winscp.StandardInput.WriteLine(getString);
winscp.StandardInput.Close();
winscp.WaitForExit();
// Set the outStr to the output value, obfuscating the password
outStr = winscp.StandardOutput.ReadToEnd().Replace(":" + Dts.Variables["$Package::pServerUserPassword"].GetSensitiveValue().ToString() + "#", ":*******#");
returnVal = winscp.ExitCode;
}
catch (Exception ex)
{
errStr = "An error occurred when attempting to execute winscp.com: " + ex.Message.Replace("'", "\"").Replace("--", " - ");
}
The package is ready to be executed. Assuming everything is configured properly, running the package on the system should download exactly two text files (remember, we used the wildcard “*.txt” to get all text files).

Visual Basic - HTTPClient and API Request Issue

Good morning,
As a returning developer (I've been in management for a long time!), I've been tasked with developing a module for our proprietary financial system (Visual Basic/ SQL Server) that will read large .CSV files containing customer information, convert them into JSON format, then submit them to an external party for processing via that company's API.
The conversion part was easy and I'm almost ready to go with it, but I can't establish connectivity to the external API.
There are two parts to the process: -
Submit login details (obfuscated here) {"username": "MadeUpUser","password": "Y66***uYj6%%YY"} and receive a Bearer Token from the API
Submit JSON format customer info to API endpoint, using Bearer Token, receive confirmation
I've submitted both the login creds and my JSON-format data to the API via Postman and it works perfectly, however, when I try to login via my VB app using the HTTPClient class, I'm getting the follow exception (copied from the Exception class properties) :
**HResult = -2146233088
StackTrace = " at System.Net.HttpWebRequest.EndGetRequestStream(IAsyncResult asyncResult, TransportContext& context)" & vbCrLf & " at System.Net.Http.HttpClientHandler.GetRequestStreamCallback(IAsyncResult ar)"
InnerException = {"The request was aborted: Could not create SSL/TLS secure channel."}**
It's clearly something I'm doing wrong programmatically or something I'm not doing and any help or advice offered will be greatly appreciated.
The code: (AddLogEntry() is a Sub I created to log events to a text file)
Private Async Function PostAsync(ByVal JSONString As String, EndPoint As String) As Task
Dim APIuri As New Uri(EndPoint)
Try
With ZincClient
.BaseAddress = APIuri
.DefaultRequestHeaders.Accept.Clear()
.DefaultRequestHeaders.Accept.Add(New Net.Http.Headers.MediaTypeWithQualityHeaderValue("application/json"))
End With
Dim content As New Net.Http.StringContent(JSONString, System.Text.Encoding.UTF8, "application/json")
Dim response As Net.Http.HttpResponseMessage = Await ZincClient.PostAsync(APIuri, content)
Dim result As String = Await response.Content.ReadAsStringAsync()
Catch ex As Exception
AddLogEntry("ERROR! - " & ex.Message)
End Try
Return
End Function

HTTP-GET via SSIS

I have some ethernet device which collect data and it's possible to download it via data export interface: HTTP-GET query returns the data in [Content-Type: text/plain Charset: utf-8]
I saw this: How to make an HTTP request from SSIS? - it rather doesn't work for me (C# is a little Chinese for me) and it's about how to fetch this data to variable into SSIS
In your SSIS package add a C# Script Task
Edit the Script Task
At the top with the other using statements add using System.Net;
in Main use the following code snippet to make a GET request (Note: Change "https://somewhere.com/contacts/get" to your actual endpoint.)
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("https://somewhere.com/contacts/get");
request.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
using(HttpWebResponse response = (HttpWebResponse)request.GetResponse())
using(Stream stream = response.GetResponseStream())
using(StreamReader reader = new StreamReader(stream))
{
return reader.ReadToEnd();
}

C# Box-API [net_WebHeaderInvalidControlChars] received when downloading a file with Korean filename or when the file is corrupted

I received the following error when downloading from Box server:
InnerException {System.ArgumentException: [net_WebHeaderInvalidControlChars]
Arguments:
Debugging resource strings are unavailable. Often the key and arguments provide sufficient information to diagnose the problem. See http://go.microsoft.com/fwlink/?linkid=106663&Version=4.7.60408.0&File=System.Net.dll&Key=net_WebHeaderInvalidControlChars
Parameter name: name
at System.Net.Browser.ClientHttpWebRequest.InternalEndGetResponse(IAsyncResult asyncResult)
at System.Net.Browser.ClientHttpWebRequest.<>c__DisplayClasse.<EndGetResponse>b__d(Object sendState)
at System.Net.Browser.AsyncHelper.<>c__DisplayClass1.<BeginOnUI>b__0(Object sendState)} System.Exception {System.ArgumentException}
Shown below is the code snippet.
using (HttpClient client = new HttpClient(handler) { MaxResponseContentBufferSize = Int32.MaxValue })
{
client.DefaultRequestHeaders.Add("Authorization", "Bearer " + accessToken);
try
{
var fileResponse = await client.GetAsync(requestUrl, HttpCompletionOption.ResponseHeadersRead);
if (fileResponse != null && fileResponse.StatusCode == System.Net.HttpStatusCode.OK)
return await fileResponse.Content.ReadAsByteArrayAsync();
else return null;
}
catch(Exception e)
{
System.Diagnostics.Debug.WriteLine("Error in getAsync, " + e.StackTrace);
return null;
}
}
Note that this only happens for files with Korean (non-English) filenames and corrupted files. For image files and non-corrupted files, I was able to download successfully. (Example of corrupted file is a word or ppt file that shows an error msg when opened).
Having the same issue in windows phone 8.
I tried
HttpResponseMessage response = await httpClient.SendAsync(requestMessage);
I tried
WebClient client = new WebClient();
client.OpenReadAsync(new Uri(url,UriKind.RelativeOrAbsolute));
I tried IRestResponse from RestClient
All resulting in exception with empty value and inner exception net_WebHeaderInvalidControlChars.
The problem was that the user agent passed from windows client was not recognized server side. As a result, one from the returned response headers, was making .Net to crash in a lower level and the response could not be processed.
Adding NativeHost as the recognized UserAgent fixed the exception

SSIS script task error retrieving variable values

In a Script Task, I am trying to retrieve a file from a networked location and FTP that file to an offsite location
In SSIS I created the FTP Connection and tested that it is setup and works
Created three variables
variable 1. FullPathName = \ftpservercsc\\\Filename.txt
variable 2 FTPFilePath = \ftpservercsc\\\
variable 3 FTPFileName = Filename.txt
Created a Script Task and added the vb code as such ...
'Get instance of the connection manager.
Dim cm As ConnectionManager = Dts.Connections("FTP Connection Manager")
Dim remotePath As String = Dts.Variables("FTPFilePath").Value.ToString
'create the FTP object that sends the files and pass it the connection
'created above.
Dim ftp As FtpClientConnection = New FtpClientConnection
(cm.AcquireConnection(Nothing))
'Connect to the ftp server
ftp.Connect()
'Set the path on the FTP server where dropping files
'ftp.SetWorkingDirectory("/Prequalify") 'set the remote directory
Dim files(0) As String
files(0) = Dts.Variables("FTPFileName").Value.ToString 'eg. File1.trg
'Send File
ftp.SendFiles(files, remotePath, True, True)
' Close the ftp connection
ftp.Close()
'Dts.Events.FireInformation(0, context, "File " + fileToGet
' + " retrieved successfully.", Nothing, Nothing, True)
Dts.TaskResult = Dts.Results.Success
Error: The element cannot be found in a collection. This error happens when you try to retrieve an element from a collection on a container during execution of the package and the element is not there.
So I have commented out and found the error is generating on retrieving the variable value but I do not know what is incorrect here
Dim remotePath As String = Dts.Variables("FTPFilePath").Value.ToString
I have tried multiple variable retrievals and all get the same error. Anyone see anything wrong?
Two things:
Make sure you config the Script Task to have Read Access to the variable. To do this right-click on the Script Task and select Edit. Click the ... under ReadOnlyVariables.
Fully qualify your variables such as Dts.Variables["User::RemotePath"].Value