"425 Unable to build data connection" when attempting to download files from a FTPS server using .Net with TLS 1.3 [duplicate] - .net-4.8

I am trying to open an FTP connection over SSL in my code. I'm able to connect and list a directory using FileZilla of WinSCP. But when listing the directory through .NET code using FtpWebClient, I get the error
(425) Can't open data connection
Since I'm able to connect using FileZilla from the same computer, I'm not sure how to go about troubleshooting this.
Here's my code
public void FtpStuff()
{
string url = "ftp://my.server.com";
FtpWebRequest request = (FtpWebRequest)WebRequest.Create(url);
request.Credentials = new NetworkCredential("myname", "password");
request.EnableSsl = true;
request.Method = WebRequestMethods.Ftp.ListDirectory;
FtpWebResponse response = (FtpWebResponse)request.GetResponse();
StreamReader streamReader = new StreamReader(response.GetResponseStream());
// This is the line that throws the exception
string line = streamReader.ReadLine();
}
I also tried FluentFTP. Here's my code for that. I get the exception
Unable to build data connection: Operation not permitted.
public void FtpStuff()
{
FtpClient client = new FtpClient();
client.Host = "my.server.com";
client.Credentials = new NetworkCredential("myname", "password");
client.EncryptionMode = FtpEncryptionMode.Explicit;
client.Connect();
// This line gives me an exception.
var files = client.GetListing();
}
Here is the logging information from FluentFTP. I changed the real user name and IP, but the rest of the data (including the port) is the real data. My FTP service provider specifies that I have to connect on port 21. The problem seems to happen towards the end after the EPSV command is issued and a connection on a new port is established.
# Connect()
The thread 0x5514 has exited with code 0 (0x0).
The thread 0xc80 has exited with code 0 (0x0).
The thread 0x89d4 has exited with code 0 (0x0).
Status: Connecting to 123.123.123.123:21
Response: 220 FTP Server Ready
Command: AUTH TLS
Response: 234 AUTH TLS successful
Status: FTPS Authentication Successful
Status: Time to activate encryption: 0h 0m 0s. Total Seconds: 0.1339995.
Command: USER me#mysite.com
The thread 0x6ddc has exited with code 0 (0x0).
Response: 331 Password required for me#mysite.com
Status: Testing connectivity using Socket.Poll()...
Command: PASS ***
Response: 230-***************************************************************************
Response: NOTICE TO USERS
Response: This computer system is private property. It is for authorized use only.
Response: Users (authorized or unauthorized) have no explicit or implicit
Response: expectation of privacy.
Response:
Response: Any or all uses of this system and all files on this system may be
Response: intercepted, monitored, recorded, copied, audited and inspected by
Response: using this system, the user consents to such interception, monitoring,
Response: recording, copying, auditing, inspection, and disclosure at the
Response: discretion of such personnel or officials. Unauthorized or improper use
Response: of this system may result in civil and criminal penalties and
Response: administrative or disciplinary action, as appropriate. By continuing to
Response: use this system you indicate your awareness of and consent to these terms
Response: and conditions of use. LOG OFF IMMEDIATELY if you do not agree to the
Response: conditions stated in this warning.
Response: ****************************************************************************
Response: 230 User me#mysite.com logged in
Command: PBSZ 0
Response: 200 PBSZ 0 successful
Command: PROT P
Response: 200 Protection set to Private
Command: FEAT
Response: 211-Features:
Response: AUTH TLS
Response: CCC
Response: CLNT
Response: EPRT
Response: EPSV
Response: HOST
Response: MDTM
Response: MFF modify;UNIX.group;UNIX.mode;
Response: MFMT
Response: MLST modify*;perm*;size*;type*;unique*;UNIX.group*;UNIX.groupname*;UNIX.mode*;UNIX.owner*;UNIX.ownername*;
Response: PBSZ
Response: PROT
Response: REST STREAM
Response: SIZE
Response: SSCN
Response: TVFS
Response: 211 End
Status: Text encoding: System.Text.ASCIIEncoding
Command: SYST
Response: 215 UNIX Type: L8
# GetListing(null, Auto)
# GetWorkingDirectory()
Command: PWD
Response: 257 "/" is the current directory
Command: TYPE I
Response: 200 Type set to I
# OpenPassiveDataStream(AutoPassive, "MLSD /", 0)
Command: EPSV
Response: 229 Entering Extended Passive Mode (|||50304|)
Status: Connecting to 123.123.123.123:50304
Command: MLSD /
Response: 150 Opening BINARY mode data connection for MLSD
Status: FTPS Authentication Successful
Status: Time to activate encryption: 0h 0m 0s. Total Seconds: 0.1210002.
+---------------------------------------+
-----------------------------------------
Status: Disposing FtpSocketStream...
# CloseDataStream()
Response: 425 Unable to build data connection: Operation not permitted
Status: Disposing FtpSocketStream...
Exception thrown: 'FluentFTP.FtpCommandException' in FluentFTP.dll
Here are my FileZilla logs.
Status: Resolving address of mysite.com
Status: Connecting to 123.123.123.123:21...
Status: Connection established, waiting for welcome message...
Response: 220 FTP Server Ready
Command: AUTH TLS
Response: 234 AUTH TLS successful
Status: Initializing TLS...
Status: Verifying certificate...
Status: TLS connection established.
Command: USER me#mysite.com
Response: 331 Password required for me#mysite.com
Command: PASS ************
Response: 230-***************************************************************************
Response: NOTICE TO USERS
Response: This computer system is private property. It is for authorized use only.
Response: Users (authorized or unauthorized) have no explicit or implicit
Response: expectation of privacy.
Response:
Response: Any or all uses of this system and all files on this system may be
Response: intercepted, monitored, recorded, copied, audited and inspected by
Response: using this system, the user consents to such interception, monitoring,
Response: recording, copying, auditing, inspection, and disclosure at the
Response: discretion of such personnel or officials. Unauthorized or improper use
Response: of this system may result in civil and criminal penalties and
Response: administrative or disciplinary action, as appropriate. By continuing to
Response: use this system you indicate your awareness of and consent to these terms
Response: and conditions of use. LOG OFF IMMEDIATELY if you do not agree to the
Response: conditions stated in this warning.
Response: ****************************************************************************
Response: 230 User me#mysite.com logged in
Command: SYST
Response: 215 UNIX Type: L8
Command: FEAT
Response: 211-Features:
Response: AUTH TLS
Response: CCC
Response: CLNT
Response: EPRT
Response: EPSV
Response: HOST
Response: MDTM
Response: MFF modify;UNIX.group;UNIX.mode;
Response: MFMT
Response: MLST modify*;perm*;size*;type*;unique*;UNIX.group*;UNIX.groupname*;UNIX.mode*;UNIX.owner*;UNIX.ownername*;
Response: PBSZ
Response: PROT
Response: REST STREAM
Response: SIZE
Response: SSCN
Response: TVFS
Response: 211 End
Status: Server does not support non-ASCII characters.
Command: PBSZ 0
Response: 200 PBSZ 0 successful
Command: PROT P
Response: 200 Protection set to Private
Status: Logged in
Status: Retrieving directory listing...
Command: PWD
Response: 257 "/" is the current directory
Command: TYPE I
Response: 200 Type set to I
Command: PASV
Response: 227 Entering Passive Mode (123,123,123,123,197,68).
Command: MLSD
Response: 150 Opening BINARY mode data connection for MLSD
Response: 226 Transfer complete
Status: Directory listing of "/" successful
I can also connect using WinSCP. As suggested in comments, I did check if TLS/SSL session ID is reused when opening the data connection. It seems that it is.
227 Entering Passive Mode (???)
MLSD
Connecting to ??? ...
Connection pending
Data connection opened
Trying reuse main TLS session ID
Session ID reused
150 Opening data channel for directory listing of "/"

.NET framework does not support TLS/SSL session reuse. If your server requires it (what it looks it does and what is quite common nowadays and what is good thing for security), you cannot use FtpWebRequest nor FluentFTP. Both use the .NET implementation of TLS/SSL.
You will have to use FTP library that uses own TLS/SSL implementation.
You can use my WinSCP .NET assembly. Though contrary to FluentFTP, it's not a native .NET library, it has dependencies on an external binary. But that's what makes it working.
Some references:
https://github.com/robinrodricks/FluentFTP/issues/347
https://github.com/dotnet/runtime/issues/27916
"Authentication failed because the remote party has closed the transport stream" when transferring to/from FTP server over TLS/SSL using FluentFTP
Upload file to implicit FTPS server in C# with TLS session reuse
Suddenly getting "150 Opening Data channel for file download from server" after the FTP downloads was working for years – According to this post and other references elsewhere, the TLS/SSL session reuse was supported earlier with .NET Framework, but some update broke it. In .NET Core it was never working (see also the dotnet GitHub link above).

Related

CRITICAL keystonemiddleware.auth_token [-] Unable to validate token: Failed to fetch token data from identity server

I am building openstack (Yoga version on Ubuntu 22.04) high availability using ssl configuration. I was able to get other services to work using https (except neutron, cinder and dashboard), but Nova throws the error in /var/log/nova/nova-api.log below:
CRITICAL keystonemiddleware.auth_token [-] Unable to validate token: Failed to fetch token data from identity server: keystonemiddleware.auth_token._exceptions.ServiceError: Failed to fetch token data from identity server
When I run the command below to get token for user "nova" I am able to get a token:
openstack --os-auth-url https://controller:5000/v3 --os-project-domain-name Default --os-user-domain-name Default --os-project-name service --os-username nova --os-password token issue
Controller is the virtual hostname for all controllers (x3). I have all nodes (controller and Compute nodes) configured in /etc/hosts file.
My configuration is as follows:
admin-openrc
export OS_USER_DOMAIN_NAME=default
export OS_PROJECT_NAME=admin
export OS_USERNAME=admin
export OS_PASSWORD=<admin-password>
export OS_AUTH_URL=https://controller:5000/v3
#export OS_SERVICE_TOKEN=
export OS_IDENTITY_API_VERSION=3
export OS_IMAGE_API_VERSION=2
/etc/nova/nova.conf
[keystone_authtoken]
www_authenticate_uri = https://controller:5000
auth_url = https://controller:5000
memcached_servers = 192.168.120.11:11211,192.168.120.12:11211,192.168.120.13:11211
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username = nova
password = <nova-passwd>
Your assistance is highly appreciated. Please let me know if you require more info.
Thank you
Thank you for your assistance in advance. I discovered that when I use the config below, everything works fine
frontend glance-api-front
bind 192.168.100.10:9292
default_backend glance-api-back
backend glance-api-back
balance source
option tcpka
option httpchk
# option tcplog
server controller1 192.168.100.11:9292 check inter 2000 rise 2 fall 5
server controller2 192.168.100.12:9292 check backup inter 2000 rise 2 fall 5
server controller3 192.168.100.13:9292 check backup inter 2000 rise 2 fall 5
but when I try to simulate failure of active controller node, I get the error below:
"An unexpected error prevented the server from fulfilling your request. (HTTP 500) (Request-ID: req-8d4979ac-c0f0-4900-94b8-814b855c5853)"
not sure how to configure HA to failover to backup controller nodes
Thank you

HAProxy 1.5 - Serving static json file on 504 error

I'm trying to set up HAProxy to server a static JSON file on 504 errors. To test, we've set up the configuration file to timeout after 10 seconds, and to use the errorfile option:
defaults
log global
mode http
retries 3
timeout client 10s
timeout connect 10s
timeout server 10s
option tcplog
balance roundrobin
frontend https
maxconn 2000
bind 0.0.0.0:9000
errorfile 504 /home/user1/test/error.json
acl employee-api-service path_reg /employee/api.*
use_backend servers-employee-api if employee-api-service
backend servers-employee-api
server www.server.com 127.0.0.1:8000
Effectively, I'm trying to serve JSON instead of HTML on a timeout, so the backend service can fail gracefully. However, on testing, we could not get anything, neither HTML or JSON. On looking at the response, it simply says it failed, with no status code. Is my setup correct for errorfile? Does HAProxy 1.5 support this?
According to the documentation of errorfile:
<file> designates a file containing the full HTTP response. It is
recommended to follow the common practice of appending ".http" to
the filename so that people do not confuse the response with HTML
error pages, and to use absolute paths, since files are read
before any chroot is performed.
So, the file should contain a complete HTTP response but you're trying to serve JSON only.
The documentation further says that:
For better HTTP compliance, it is
recommended that all header lines end with CR-LF and not LF alone.
The example configuration, for example,
errorfile 503 /etc/haproxy/errorfiles/503sorry.http
shows the common practice of .http extension for the error file.
You can find samples of some default error files here.
Sample (504.http):
HTTP/1.0 504 Gateway Time-out
Cache-Control: no-cache
Connection: close
Content-Type: text/html
<html><body><h1>504 Gateway Time-out</h1>
The server didn't respond in time.
</body></html>
So, in your scenario, 504.http would be like this:
HTTP/1.0 504 Gateway Time-out
Cache-Control: no-cache
Connection: close
Content-Type: application/json
{
"message": "Gateway Timeout"
}
Also, you need to keep the file size under limit i.e. BUFSIZE (8 or 16 KB) as described in the documentation.
There might be some error logs for not serving your JSON file. You might want to look at HAProxy's logs again thoroughly. Just to be sure.

fiware POI: cannot add poi: always unauthorized

I have two instances:
keyrock
Poi with wilma-proxy
Trying to create a poi in instance, allways the response is 401 Permission denied. But token is correct as said proxy log:
2016-10-14 09:40:30.132 - INFO: IDM-Client - Token in cache, checking timestamp...
2016-10-14 09:40:30.135 - INFO: IDM-Client - Token in cache expired
2016-10-14 09:40:30.136 - INFO: IDM-Client - Checking token with IDM...
2016-10-14 09:40:30.342 - INFO: Root - Access-token OK. Redirecting to app...
Refused to set unsafe header "content-length"
2016-10-14 09:40:30.366 - ERROR: HTTP-Client - Error: 401 Permission denied.
So, i think the problem is the authenticate.html file. The only lines i changed are:
<meta name="fiware_lab-signin-client_id"
content="8dc5826cdaea4729a4f43a01d01cb32e">
<meta name="fiware_lab-signin-host" content="http://myserver.com:8000">
fiware_lab-signin-client_id has the client_id of application POI created in my keyrock instance.
the fiware_lab-signin-host has the server AND port of horizon.
Is that correct? I try with 5000 port (keystone) without results.
Also, I found this in add_poi.php:
$session = get_session();
$user_id = $session['user'];
$add_permission = $session['permissions']['add'];
if(!$add_permission) {
header("HTTP/1.0 401 Unauthorized");
die("Permission denied.");
}
If comment these lines, all proccess seems to be correct until the sql insert. (needs the $session['user'] and other parameters that are null)
I dont understand why these lines are there. also, the function get_session() returns a hardcoded array and not the session (which may not have)
I'm so confused how to use this enabler. Anyone knows how to use?
Have you configured the auth_conf.json file with the root user(s) of the POI-DP? Configuring hard users The POI-DP considers the Keystone as a general identity provider (as Google+). You have to separately give permissions to the POI-DP users, first configuring the root users with all privileges and then they can call other users using user_management.html . Site Administration
Please, use the tag fiware-poi for quicker response to POI-DP questions.

Using outbound Node-RED email to local SMTP server

I have a local SMTP email server I use for testing purposes running on my machine. It listens for SMTP on port 25. I am able to send and receive emails to it using a regular email client.
When I build a Node-RED flow that contains an e-mail output node and configure its properties with:
to: <email address>
server: localhost
port: 25
and submit a flow, I get the error:
25 Feb 16:43:24 - [error] [e-mail:<email address>] Error: 101057795:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:openssl\ssl\s23_clnt.c:794:
I am at a loss on how to proceed. Looking at the messages, it almost appears that there is some form of SSL negotiation/test at play here. Switching on trace on my SMTP server, I find the following logs each time I try and run a flow:
"TCPIP" 10708 "2016-02-25 16:43:08.294" "TCP - 127.0.0.1 connected to 127.0.0.1:25."
"DEBUG" 10708 "2016-02-25 16:43:08.298" "Creating session 22"
"SMTPD" 10708 22 "2016-02-25 16:43:08.298" "127.0.0.1" "SENT: 220 WIN7-X64 ESMTP"
"DEBUG" 9772 "2016-02-25 16:43:08.299" "Ending session 22"
It appears that the Node-RED node is sending a connection request, getting back the SMTP 220 response and then failing immediately after that.
I came across the same problem and have a nasty hack that will enable mail to go via my local exchange server's plain SMTP, with no auth.
Edit the .../61-email.js file and change it thusly:
var smtpTransport = nodemailer.createTransport({
host: node.outserver,
port: node.outport,
secure: false,
ignoreTLS: true //,
// auth: {
// user: node.userid,
// pass: node.password
// }
});
I see Dave has replied to the github issue but just to close the loop on this question.
At this time (Feb 2016) the node assumes SSL is always available and enabled, at some point we need to go back to the email node and find a simple way to expose a lot more of the nodemailer options to allow connections to a wider range of email providers both public and private.

Worklight QA Send mail

I'm using Worklight QA and I got an error trying to send mails using SendGrid.
The error was on the Send activation link for the user.
This is part of the error on celeryd.log
HTTPError: HTTP Error 429: UNKNOWN STATUS CODE
[2014-09-29 13:29:55,549: WARNING/Worker-3] Unable to reach Sentry log server: HTTP
Error 429: UNKNOWN STATUS CODE (url: https://app.getsentry.
com/api/13389/store/, body: Creation of this event was
denied due to rate limiting.)
[2014-09-29 13:29:55,555: ERROR/MainProcess] Failed to submit message: u'error:
[Errno 111] Connection refused'
[2014-09-29 13:29:55,556: WARNING/Worker-3] Failed to submit message: u'error:
[Errno 111] Connection refused'
[2014-09-29 13:29:55,558: ERROR/MainProcess] Task notifications.email.ActivationEmail
[88c97bed-812a-427f-98a1-9bc77ff38876] raised exception:
error(111, 'Connection refused')
I've configured local_settings.py with the SendGrid information, the SendGrid account is provisioned and ready to send mails.
EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
EMAIL_HOST = 'smtp.sendgrid.net'
EMAIL_PORT = 587
EMAIL_HOST_USER = '******'
EMAIL_HOST_PASSWORD = '******'
EMAIL_USE_TLS = False
I've also tried to disable iptables on the server thinking on local firewall issue, but It was getting the same error.
I don't know if this rate limiting error from Sentry has something to do with it.
This could possibly be some kind of SMTP integration issue on your end. Not sure Sentry has anything to do with it.
Suggest changing EMAIL_USE_TLS to True and see if that works. It is possible that SendGrid is enforcing that.