SSL messages encoding - json

I am trying to build an SSL Server under Python 3.4. The point is to communicate and exchange data with a programme through a defined protocol based on JSON data format.
So I used a basic "echo server" and client in SSL Protocol and modified those to see if I could exchange data. It worked and sending "hello" one side comes as b"hello" on the other side and it works both ways.
I start the server side, connect the program, it communicates succesfully, but:
I am expecting something like : LOGIN:n::{“user”:”XXXXX”, , ”password”:”YYYYY ”, app”:”ZZZZZ”, “app_ver”:”zzz”, ”protocol”:”xxx”,”protocol_ver”:”xxxx”} arriving from the client (program)
But instead I am getting something like this b"\x16\x03\x03\x00\x8e\x01\x00\x00\x8a\x03\x03^\x9e\xeb\xd8\x8f\xd9 \x05v\xbbF:}\xda\x17\xf7\x13\xff\xa9\xde=5\xfb_\xbco\x16\x96EL#\x00\x00*\xc0,\xc0+\xc00\xc0/\x00\x9f\x00\x9e\xc0$\xc0#\xc0(\xc0'\xc0\n\xc0\t\xc0\x14\xc0\x13\x00\x9d\x00\x9c\x00=\x00<\x005\x00/\x00\n\x01\x00\x007\x00\n\x00\x08\x00\x06\x00\x1d\x00\x17\x00\x18\x00\x0b\x00\x02\x01\x00\x00\r\x00\x14\x00\x12\x06\x01\x06\x03\x04\x01\x05\x01\x02\x01\x04\x03\x05\x03\x02\x03\x02\x02\x00#\x00\x00\x00\x17\x00\x00\xff\x01\x00\x01\x00"
I thought it was simply encoded, but I have tried the bytemessage.decode()method, with utf-8, cp437, cp1250, cp1252, latin-1, etc. I have also tried codecs.decode() with hex. No success, I Don't understand what language is this.
I am new to SSL so I suppose I am missing something obvious here, but I have no idea what …
Any help would be greatly appreciated.
Thanks in advance !
---- Edit here is the code of my server-----
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_address = ('localhost', 5000)
print ('starting up on %s port %s' % server_address)
sock.bind(server_address)
sock.listen(1)
while True:
print ( 'waiting for a connection')
connection, client_address = sock.accept();
try:
print( 'connection from', client_address)
while True:
data = connection.recv(16)
print ( 'received "%s"' % data)
if True:
#data2=b'{"timing":{"liveEvents": {"sector": {"dayTime": 1483523892618,"driver": 1,"isValid": false,"participant": "0","sector": 3,"time": -1}}}}'
print ('sending data to the client')
#connection.sendall(data2)
else:
print ( 'no more data from', client_address)
break
finally:
connection.close()

b"\x16\x03\x03...
This is a TLS message. Looks like your client tries to speak TLS to your server but your server cannot properly handle it. Instead of treating the data as TLS it will assume that the TLS is the actual application data.
Looking at your server code the reason is clear: you are not doing any SSL there, i.e. you are doing a plain TCP socket. SSL will not magically appear just because a clients tries to talk SSL with the server but you need to use the ssl module, properly wrap_socket and provide the necessary server certificate and key. For some simple example see the documentation.

As #Steffen mentioned , I wasn't handling SSL at all, which I now do with ssl.wrap_socket(sock,certfile='certificat.pem', keyfile='cle.pem', server_side=True)
Operation on server side requires certificates and key files in pem, which I generated with SelfSSL7 and then split the pfx into 2 pem key and certificate files with OpenSSL
openssl pkcs12 -in yourpfxfile.pfx -nocerts -out privatekey.pem -nodes
openssl pkcs12 -in yourpfxfile.pfx -nokeys -out publiccert.pem -nodes
Maybe not the fastest solution for a self signed certificate since I now have OpenSSL installed but …
Finally, the expected message !!
starting up on localhost port 11000
waiting for a connection
connection from ('127.0.0.1', 60488)
received "b'PING:0::\r\n'"
sending data to the client
received "b'LOGIN:::{"user":"test","password":"test","app":"AppName","app_ver":"1.0.0","protocol":" ","protocol_ver":"1.0.0"}\r\n'"
sending data to the client
Again thank you very much #SteffenUllrich

Related

How to get InfluxDB to accept a self-signed certificate?

I've been trying to get InfluxDB to accept a self-signed certificate, but so for, no luck. I've been following instructions from here:
https://docs.influxdata.com/influxdb/v2.3/security/enable-tls/#configure-influxdb-to-use-tls
I created the cert and key with this command:
openssl req -x509 -nodes -newkey rsa:2048 -keyout influxdb-selfsigned.key -out influxdb-selfsigned.crt -days 9999 -config "C:\OpenSSL\openssl.cnf"
The config.yml file is as follows:
http-bind-address: ":8087"
tls-cert: influxdb-selfsigned.crt
tls-key: influxdb-selfsigned.key
Note, I made the bind port 8087 to ensure it was reading the configuration.
When I start influx from the command line, there are no error messages. Initially there were some TLS handshake errors, but those disappeared, I think when I added the crt and key to the configuration.
However, when I access the URL https://localhost:8087, chrome shows a "not secure" message and I have to click through warnings to get to the site.
To try to get Chrome to trust the certificate, - I followed the instructions from this site:
https://www.pico.net/kb/how-do-you-get-chrome-to-accept-a-self-signed-certificate
I exported the cert, then re-imported it as trusted.
However, I still get the "not secure" message in Chrome.
Also, the InfluxDB console shows this message:
info http: TLS handshake error from [::1]:63065: remote error: tls: unknown certificate {"log_id": "0cKnmWB0000", "service": "http"}
Any ideas how to get the cert working?
Currently it seems this is no easy way or workaround in open source version. The community has been asking for this feature but no progress yet. See more details here.
However, in Enterprise version, you could configure the server to know you are using self-signed certificates by setting this configuration in influxdb-meta.conf file:
# If using a self-signed certificate:
https-insecure-tls = true

ejabberd: How to use "ldap_tls_certfile"

According to the ejabberd docs, you can use ldap_tls_certfile in order to verify the TLS connection to the LDAP server. But which certificate is expected here?
Quoting the docs:
A path to a file containing PEM encoded certificate along with PEM
encoded private key. This certificate will be provided by ejabberd
when TLS enabled for LDAP connections. There is no default value,
which means no client certificate will be sent.
Sooo.... I tried to use a concatenated PEM file containing first the host certificate of the ejabberd server, then second the host key. But this leads to the following errors:
<0.471.0>#eldap:connect_bind:1073 LDAP connection to
ldap1.example.com:636 failed: received CLIENT ALERT: Fatal - Handshake
Failure - {bad_cert,hostname_check_failed}
<0.1975.0> TLS client: In state certify at ssl_handshake.erl:1372 generated CLIENT ALERT: Fatal - Handshake Failure - {bad_cert,hostname_check_failed}
This obviously is not what is expected. Is it the public certificate of the LDAP server? But then, what private key is expected?
I'm a bit lost here. Anyone mind to lend me a hand?
Disclaimer: I never used LDAP TLS.
Looking at the ejabberd source code, the value of ejabberd's option ldap_tls_certfile
is copied into eldap's option tls_certfile
https://github.com/processone/ejabberd/blob/e4d600729396a8539e48ac0cbd97ea1b210941cd/include/eldap.hrl#L72
And later the value of eldap's tls_certfile is copied into ssl's option certfile
https://github.com/processone/ejabberd/blob/e4d600729396a8539e48ac0cbd97ea1b210941cd/src/eldap.erl#L580
That option, among others, is provided as an argument when calling ssl:connect/4
https://github.com/processone/ejabberd/blob/e4d600729396a8539e48ac0cbd97ea1b210941cd/src/eldap.erl#L1140
So, the option that you set in ejabberd is named 'certfile' in ssl:connect, you can see here its documentation:
https://erlang.org/doc/man/ssl.html#connect-4
Searching for certfile in that page, it shows this description:
Path to a file containing the user certificate on PEM format.
Is it the public certificate of the LDAP server?
Try that one and comment here.
But then, what private key is expected?
Try not putting any private key. In any case, when the LDAP certificate was created, it produced a private key file, too.

CakePHP 3 - Enable SSL on development server [duplicate]

OS: Ubuntu 12.04 64-bit
PHP version: 5.4.6-2~precise+1
When I test an https page I am writing through the built-in webserver (php5 -S localhost:8000), Firefox (16.0.1) says "Problem loading: The connection was interrupted", while the terminal tells me "::1:37026 Invalid request (Unsupported SSL request)".
phpinfo() tells me:
Registered Stream Socket Transports: tcp, udp, unix, udg, ssl, sslv3,
tls
[curl] SSL: Yes
SSL Version: OpenSSL/1.0.1
openssl:
OpenSSL support: enabled
OpenSSL Library Version OpenSSL 1.0.1 14 Mar 2012
OpenSSL Header Version OpenSSL 1.0.1 14 Mar 2012
Yes, http pages work just fine.
Any ideas?
See the manual section on the built-in webserver shim:
http://php.net/manual/en/features.commandline.webserver.php
It doesn't support SSL encryption. It's for plain HTTP requests. The openssl extension and function support is unrelated. It does not accept requests or send responses over the stream wrappers.
If you want SSL to run over it, try a stunnel wrapper:
php -S localhost:8000 &
stunnel3 -d 443 -r 8080
It's just for toying anyway.
It's been three years since the last update; here's how I got it working in 2021 on macOS (as an extension to mario's answer):
# Install stunnel
brew install stunnel
# Find the configuration directory
cd /usr/local/etc/stunnel
# Copy the sample conf file to actual conf file
cp stunnel.conf-sample stunnel.conf
# Edit conf
vim stunnel.conf
Modify stunnel.conf so it looks like this:
(all other options can be deleted)
; **************************************************************************
; * Global options *
; **************************************************************************
; Debugging stuff (may be useful for troubleshooting)
; Enable foreground = yes to make stunnel work with Homebrew services
foreground = yes
debug = info
output = /usr/local/var/log/stunnel.log
; **************************************************************************
; * Service definitions (remove all services for inetd mode) *
; **************************************************************************
; ***************************************** Example TLS server mode services
; TLS front-end to a web server
[https]
accept = 443
connect = 8000
cert = /usr/local/etc/stunnel/stunnel.pem
; "TIMEOUTclose = 0" is a workaround for a design flaw in Microsoft SChannel
; Microsoft implementations do not use TLS close-notify alert and thus they
; are vulnerable to truncation attacks
;TIMEOUTclose = 0
This accepts HTTPS / SSL at port 443 and connects to a local webserver running at port 8000, using stunnel's default bogus cert at /usr/local/etc/stunnel/stunnel.pem. Log level is info and log outputs are written to /usr/local/var/log/stunnel.log.
Start stunnel:
brew services start stunnel # Different for Linux
Start the webserver:
php -S localhost:8000
Now you can visit https://localhost:443 to visit your webserver: screenshot
There should be a cert error and you'll have to click through a browser warning but that gets you to the point where you can hit your localhost with HTTPS requests, for development.
I've been learning nginx and Laravel recently, and this error has came up many times. It's hard to diagnose because you need to align nginx with Laravel and also the SSL settings in your operating system at the same time (assuming you are making a self-signed cert).
If you are on Windows, it is even more difficult because you have to fight unix carriage returns when dealing with SSL certs. Sometimes you can go through the steps correctly, but you get ruined by cert validation issues. I find the trick is to make the certs in Ubuntu or Mac and email them to yourself, or use the linux subsystem.
In my case, I kept running into an issue where I declare HTTPS somewhere but php artisan serve only works on HTTP.
I just caused this Invalid request (Unsupported SSL request) error again after SSL was hooked up fine. It turned out to be that I was using Axios to make a POST request to https://. Changing it to POST http:// fixed it.
My recommendation to anyone would be to take a look at where and how HTTP/HTTPS is being used.
The textbook definition is probably something like php artisan serve only works over HTTP but requires underlying SSL layer.
Use Ngrok
Expose your server's port like so:
ngrok http <server port>
Browse with the ngrok's secure public address (the one with https).
Note: Though it works like a charm, it seems an overkill since it requires internet and would appreciate better recommendations.

Ubuntu/Thunderbird: ssl_error_rx_record_too_long

I installed Postfix, Courier and a bunch of other e-mail related services on my Ubuntu server. The data is fetched from a MySQL database. But when I try to connect through Thunderbird, I get this error:
ssl_error_rx_record_too_long
Why :(? How can this be fixed??
How many bits in your SSL key? IIRC, you'll get this error if you use larger than a 2048 bit key.
This error also happens in Thunderbird, however against a Courier imapd ssl installed on Gentoo.
Turns out there were a bunch of broken SSL certificate symlinks in /etc/ssl/certs. Removing them fixed the problem.
find . -type l | (while read FN ; do test -e "$FN" || ls -ld "$FN"; done)

How to make my PHP Socket Server send a policy file to flash clients?

My flash game needs to connect to my PHP Socket Server. Because of security things, a policy file has to be send to the flash client when it tries to connect.
The following is what I've done.
In Actionscript / Flex 3 / Flash:
Security.loadPolicyFile("http://[SERVER.IP]:9000/crossdomain.xml");
socket.connect(hostName, port); //connect to the socket
[rest of original code]
To make the socket server respond on the request, I added the following to the server:
elseif (preg_match("/policy-file-request/i", $buffer) or preg_match("/crossdomain/i", $buffer)) {
socket_write($socket, '<?xml version="1.0"?><cross-domain-policy><site-control permitted-cross-domain-policies="all"/><allow-access-from domain="*" to-ports="9000" /></cross-domain-policy>');
unset($read_sockets[array_search($socket, $read_sockets)]);
socket_shutdown($socket, 2);
socket_close($socket);
I however get the following error: "Ignoring policy file at (URL) due to missing Content-Type." So, I tried to fix this by adding a header right above my xml code:
socket_write($socket, "Content-Type: text/xml\n");
Unfortunately, I still get the same error. Am I giving the content type in a wrong way?
You need to return a valid HTTP response if you are going to use:
Security.loadPolicyFile("http://[SERVER.IP]:9000/crossdomain.xml");
If the flash is going to connect to your PHP socket server, just skip the above line and it will try the port itself and expect raw data instead of a HTTP response.
Try sending this:
HTTP/1.1 200 OK\r\nContent-Type: text/xml\r\n\r\n
Make sure nothing is sent before this.
Also, send a \r\n before the socket is closed.
you can load the policyfile from any port of the server using Security.loadPolicyFile() ... maybe you should simply serve it per http on port 80, load it from there and then connect to the server ...
also, by default, i think flashplayer 9 (upwards from some minor version) sends a policyfile request to port 943 by default ... so you might put a server there, to do that ...
a little side note: PHP was never designed for socket servers and is not very good at that ... if you can, try using Java, or NekoVM, that you can use with Haxe ... also Haxe remoting, as well as ThreadedRemotingServer might be of interest to you ... there's some good and clear tutorials on the Haxe site ...
Try it with \r\n after the Content-Type.
socket_write($socket, "Content-Type: text/xml\r\n");
Shouldn't you be using a xmlsocket on port 843?
if(trim($buffer) == '<policy-file-request/>') {
$policy_file =
'<'.'?xml version="1.0" encoding="UTF-8"?'.'>'.
'<cross-domain-policy xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="http://www.adobe.com/xml/schemas/PolicyFileSocket.xsd">'.
'<allow-access-from domain="*" to-ports="*" secure="false" />'.
'<site-control permitted-cross-domain-policies="master-only" />'.
'</cross-domain-policy>';
socket_write($socket, $policy_file.chr(0));
}
works fine for me, the client will request a policy file, disconnect, then reconnect after receiving the policy file