I use Chrome 14. This is my python websocket server code snippet:
global guid
key = hashlib.sha1(headers['Sec-WebSocket-Key']+guid).digest()
key.encode('iso-')
headers['Sec-WebSotycket-Accept'] = base64.b64encode(key)
print headers['Sec-WebSocket-Accept']
handshake = '\
HTTP/1.1 101 Switching Protocols\r\n\
Upgrade: %s\r\
Connection: %s\r\
Sec-WebSocket-Accept: %s\r\
Sec-WebSocket-Protocol: base64\r\
' %(headers['Upgrade'],headers['Connection'],headers['Sec-WebSocket-Accept'])
try:
self.conn.send(handshake)
except Exception as e:
print e
Why does the Chrome client not accept this server send handshake?
Request URL:ws://127.0.0.1:1234/
Request Headers
Connection:Upgrade
Host:127.0.0.1:1234
Sec-WebSocket-Key:xuV2xuiXxqL4Hwcxjg9dJA==
Sec-WebSocket-Origin:null
Sec-WebSocket-Version:8
Upgrade:websocket
(Key3):00:00:00:00:00:00:00:00
Could it be line 4 of your server code? Your key "Sec-WebSotycket-Accept" looks misspelled.
Related
I'm trying to get some JSON data from this API - https://ped.uspto.gov/api/queries
This cURL request works fine and returns what is expected:
curl -X POST "https://ped.uspto.gov/api/queries" -H "accept: application/json" -H "Content-Type: application/json" -d "{ \"searchText\":\"*:*\", \"fq\":[ \"totalPtoDays:[1 TO 99999]\", \"appFilingDate:[2005-01-01T00:00:00Z TO 2005-12-31T23:59:59Z]\" ], \"fl\":\"*\", \"mm\":\"100%\", \"df\":\"patentTitle\", \"facet\":\"true\", \"sort\":\"applId asc\", \"start\":\"0\"}"
I have this python script to do the same thing:
from requests.structures import CaseInsensitiveDict
import json
url = "https://ped.uspto.gov/api/queries"
headers = CaseInsensitiveDict()
headers["accept"] = "application/json"
headers["Content-Type"] = "application/json"
data = json.dumps({
"searchText":"*:*",
"fq":[
"totalPtoDays:[1 TO 99999]",
"appFilingDate:[2005-01-01T00:00:00Z TO 2005-12-31T23:59:59Z]"
],
"fl":"*",
"mm":"100%",
"df":"patentTitle",
"facet":"true",
"sort":"applId asc",
"start":"0"
})
resp = requests.post(url, headers=headers, data=data)
print(resp.status_code)
but it returns a 403 error code and the following response header:
"Date":"Mon, 24 Oct 2022 16:13:58 GMT",
"Content-Type":"text/html",
"Content-Length":"919",
"Connection":"keep-alive",
"X-Cache":"Error from cloudfront",
"Via":"1.1 d387fec28536c5aa92926c56363afe9a.cloudfront.net (CloudFront)",
"X-Amz-Cf-Pop":"LHR50-P8",
"X-Amz-Cf-Id":"RMd69prehvXNAl97mo0qyFtuBIiY8r9liIxcQEmbdoBV1zwXLhirXA=="
I'm at quite a loss at what to do, because I really don't understand what my Python is missing to replicate the cURL request.
Thanks very much.
I was interested in this. I got an account with uspto.gov and acquired an access key. Their other API's work well. But the PEDS API? I kept getting the Cloudflare Gateway Timeout 503 error. While I was on their website, I looked into the PEDS API, I could not load any link to a https://ped.uspto.gov page.
I called them and they gave me an email address. I got this reply:
The PEDS API was taken down, because repeated data mining was bringing the entire PEDS System down.
The PEDS Team is working on a solution to fix the PEDS API, so that it can be re-enabled.
I tried it using PHP.
Cloudflare has been causing a lot of problems for curl.
I got a timeout.
I may have gotten past the 403 Forbidden, but did not have credentials and so the server dropped the connection.
An HTTP 504 status code (Gateway Timeout) indicates that when
CloudFront forwarded a request to the origin (because the requested
object wasn't in the edge cache), one of the following happened: The
origin returned an HTTP 504 status code to CloudFront. The origin
didn't respond before the request expired.
AWS Cloudflare Curl Issues
bypassing CloudFlare 403
How to Fix Error 403 Forbidden on Cloudflare
403 Forbidden cloudflare
██████████████████████████████████████████████████████████████
This is a conversion from you curl.
The Content-Type:application/data is added by default when you send JSON data.
I do not know about your json_data.dump or you putting the JSON in parentheses.
import requests
headers = {
'accept': 'application/json',
}
json_data = {
'searchText': '*:*',
'fq': [
'totalPtoDays:[1 TO 99999]',
'appFilingDate:[2005-01-01T00:00:00Z TO 2005-12-31T23:59:59Z]',
],
'fl': '*',
'mm': '100%',
'df': 'patentTitle',
'facet': 'true',
'sort': 'applId asc',
'start': '0',
}
response = requests.post('https://ped.uspto.gov/api/queries', headers=headers, json=json_data)
When using hard-coded username / email / password I have no problem getting a message sent with phpmailer. But when I use $_ENV to hide the credentials I get the smtp error as shown here:
2020-09-08 15:50:51 SERVER -> CLIENT: 220 dd45234.kasserver.com ESMTP
2020-09-08 15:50:51 CLIENT -> SERVER: EHLO browsegenres-f3.loc
2020-09-08 15:50:51 SERVER -> CLIENT: 250-dd45234.kasserver.com250-PIPELINING250-SIZE 102400000250-VRFY250-ETRN250-STARTTLS250-AUTH PLAIN LOGIN250-AUTH=PLAIN LOGIN250-ENHANCEDSTATUSCODES250-8BITMIME250 DSN
2020-09-08 15:50:51 CLIENT -> SERVER: STARTTLS
2020-09-08 15:50:51 SERVER -> CLIENT: 220 2.0.0 Ready to start TLS
2020-09-08 15:50:51 CLIENT -> SERVER: EHLO xxxxxxxxxxxxxxxxxxxx.loc
2020-09-08 15:50:51 SERVER -> CLIENT: 250-xxxxxxxx.[SERVER].com250-PIPELINING250-SIZE 102400000250-VRFY250-ETRN250-AUTH PLAIN LOGIN250-AUTH=PLAIN LOGIN250-ENHANCEDSTATUSCODES250-8BITMIME250 DSN
2020-09-08 15:50:51 CLIENT -> SERVER: AUTH LOGIN
2020-09-08 15:50:51 SERVER -> CLIENT: 334 VXNlcm5hbWU6
2020-09-08 15:50:51 CLIENT -> SERVER: [credentials hidden]
2020-09-08 15:50:53 SERVER -> CLIENT: 535 5.7.8 Error: authentication failed: VXNlcm5hbWU6
2020-09-08 15:50:53 SMTP ERROR: Username command failed: 535 5.7.8 Error: authentication failed: VXNlcm5hbWU6
SMTP Error: Could not authenticate.
2020-09-08 15:50:53 CLIENT -> SERVER: QUIT
2020-09-08 15:50:53 SERVER -> CLIENT: 221 2.0.0 Bye
SMTP Error: Could not authenticate.
Message could not be sent. Mailer Error: SMTP Error: Could not authenticate.
I don't wan to hardcode the credentials. Any idea how to get rid of this error?
Here's the code:
// initiate phpMailer
$mail = new PHPMailer(true);
// see config file
$mailSenderName = $_ENV['MAILER_CONTACT_USERNAME'];
$masterPassword = $_ENV['MAILER_CONTACT_PASSWORD'];
$masterEmail = $_ENV['MAILER_CONTACT_EMAIL'];
$recipient = $_ENV['MAILER_CONTACT_RECIPIENT'];
try {
//Server settings
$mail->SMTPDebug = SMTP::DEBUG_SERVER;
$mail->isSMTP();
$mail->Host = 'xxxxxxx.[SERVER].com';
$mail->SMTPAuth = true;
$mail->Username = $masterEmail;
$mail->Password = $masterPassword;
$mail->SMTPSecure = PHPMailer::ENCRYPTION_STARTTLS;
$mail->Port = 25;
//Recipients
$mail->setFrom('aaa#bbbbbbbbbbb.com', 'aabbcc');
$mail->addAddress('mmmmmmmmm#bbbbbbbbbbb.com');
// Content
$mail->isHTML(true);
$mail->Subject = 'Message Received (Contact Page)';
$emailbody =
'There is a new message from: <br>' .
'==================================== <br>' .
$senderName . '<br>' .
$senderEmail . '<br' .
'====================================' .
$message . '<br>' .
'====================================';
$mail->Body = $emailbody;
$mail->send();
// success, show thank you
$f3->reroute('/contact/thankyou'); //todo
} catch (\Exception $e) {
echo "Message could not be sent. Mailer Error: {$mail->ErrorInfo}";
}
Thanks!
Debug one thing at a time. There's no point in looking at error in your email when you know you know you have a problem before it ever gets that far. PHPMailer uses whatever you give it, so you need to be sure you're giving it the right thing.
You could reduce the code to debug in this case by cutting it back to:
var_dump($_ENV);
Once you know that you're setting the contents of $_ENV correctly (whether from real env vars, from a dotenv script, your php.ini config, etc), you can then start using the values in your email code.
After installing dotenv (vlucas) I simply didn't include it correctly in my ContactController. So that's why var_dump($_ENV) always resulted in NULL. I compared my settings with the other route, NewsletterController. The difference is that in this route I query the database and in the models constructor (where the db connection is set) I 'use' the dotenv class correctly, and that's why the $_ENV is filled with data. I simply didn't see it.
So, in ContactController I set:
use \Dotenv;
and after initialising phpmailer I added:
$mail = new PHPMailer(true);
$dotenv = Dotenv\Dotenv::createImmutable($_SERVER['DOCUMENT_ROOT']);
$dotenv->load();
Difference to Models class (database connection):
namespace Models;
use \Dotenv;
abstract class Model
{
protected $db;
public function __construct()
{
$dotenv = Dotenv\Dotenv::createImmutable($_SERVER['DOCUMENT_ROOT']);
$dotenv->load();
$this->db = new \DB\SQL(
'mysql:host='. $_ENV['DB_HOST'] .';port='.$_ENV['DB_PORT'].';dbname='.$_ENV['DB_NAME'],
$_ENV['DB_USERNAME'],
$_ENV['DB_PASSWORD']
);
}
}
I am sending a Post request with a json body to a server but can not extract the json file when it arrives. I have does exhaustive searches but to no avail. I have provided both client and server scripts to illustrate what is happening.
All I need is to extract the json portion at the end of the received string so I can analyze the request and return the appropriate data.
I'm sure it's simple but I can't seem to find the answer. Any direction would be appreciated
***
CLIENT: script to test Server
import json
import requests
def info_send():
url = 'http:1234abcd.ngrok.io'
payload = {
'command': '["command", "status", "off", None]',
'userID': 'userID string',
'status': 'current status',
}
requests.post(url, data=json.dumps(payload))
info_send()
***
SERVER: receives json POST request
HOST, PORT = '', 5000
listen_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
listen_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
listen_socket.bind((HOST, PORT))
listen_socket.listen(1)
print('Listening on port %s' % PORT)
while True:
client_connection, client_address = listen_socket.accept()
request = client_connection.recv(1024).decode('utf-8')
print(request)
***
This is what is printed at the server
POST / HTTP/1.1
Host: 1234abcd.ngrok.io
User-Agent: python-requests/2.18.4
Accept-Encoding: gzip, deflate
Accept: /
Content-Length: 112
X-Forwarded-For: 112.162.214.265
{"command": "[\"command\", \"status\", \"off\", None]", "userID": "userID string", "deviceID": "current status"}
I am trying to access a rest service that I am hosting on an amazon AWS server from a firefox WebExtension.
I have registered a background script in the manifest.json which then tries to access the service.
"background": {
"scripts": ["OwnerLangBackground.js"]
},
"permissions": [
"*://ec2-35-158-91-62.eu-central-1.compute.amazonaws.com:9000/*"
]
However, the XMLHttpRequest just returns an error but I don't see what goes wrong. While researching this issue, I stumbled across the following page:
https://mathiasbynens.be/notes/xhr-responsetype-json
Replacing my own code with a (slightly modifed) copy of the code from the above link I now have:
// OwnerLangBackground.js
console.log("OwnerLangBackground.js loaded");
var getJSON = function(url, successHandler, errorHandler) {
var xhr = new XMLHttpRequest();
xhr.open('get', url, true);
xhr.onreadystatechange = function() {
var status;
var data;
// https://xhr.spec.whatwg.org/#dom-xmlhttprequest-readystate
if (xhr.readyState == 4) { // `DONE`
status = xhr.status;
if (status == 200) {
data = JSON.parse(xhr.responseText);
successHandler && successHandler(data);
} else {
errorHandler && errorHandler(status, xhr.responseText);
}
}
};
xhr.send();
};
/* BLOCK 1: removing the comments for this block works
getJSON('https://mathiasbynens.be/demo/ip', function(data) {
console.log('Your public IP address is: ' + data.ip);
console.log('Your response is: ', data);
}, function(status) {
console.warn('Something went wrong.', status);
});
*/
/* BLOCK 2: removing the comments for this block, does not work
getJSON('http://ec2-35-158-91-62.eu-central-1.compute.amazonaws.com:9000/get-languages', function(data) {
console.log('Your response is: ', data);
}, function(status) {
console.warn('Something went wrong.', status);
});
*/
Strangely enough, activating BLOCK 1 works as expected (ip address obscured on purpose).
OwnerLangBackground.js loaded
Your public IP address is: xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xx
Your response is: Object { ip: "xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:…" }
Activating BLOCK 2 results in the error response.
OwnerLangBackground.js loaded
Something went wrong. 0
However, if I call the two URLs using curl directly, they both return valid JSON:
> curl https://mathiasbynens.be/demo/ip
{"ip":"xxxx:xxxx:xxxx::xxx"}
> curl http://ec2-35-158-91-62.eu-central-1.compute.amazonaws.com:9000/get-languages
[{"language":"??"},{"language":"de"},{"language":"en"},{"language":"fr"},{"language":"it"}]
I have added debugging output to my rest service on the AWS server and I see that it gets called. I also traced the WebExtension call to the rest service using Wireshark on my local machine on which the WebExtension is running and I can see the JSON string being returned, so I am guessing that the error occurs somewhere within firefox/the webextension, but I am at a total loss.
Things I have considered:
Permissions in the manifest: as far as I can tell the URL pattern for my aws-url is correctly added. However, the call to mathiasbynens.be works even though I have not added the url to the permissions
the call that works uses https while the call that does not work uses http. Could this be the reason?
Can anyone point me in the right direction to get more feedback on what goes wrong? I've tried adding a onerror callback to the xhr request. It is called but as far as I can see doesn't provide more information.
UPDATE:
I've come up with two more ideas. Using curl -v provided me with the headers:
> curl -v http://ec2-35-158-91-62.eu-central-1.compute.amazonaws.com:9000/get-languages
* Hostname was NOT found in DNS cache
* Trying 35.158.91.62...
* Connected to ec2-35-158-91-62.eu-central-1.compute.amazonaws.com (35.158.91.62) port 9000 (#0)
> GET /get-languages HTTP/1.1
> User-Agent: curl/7.38.0
> Host: ec2-35-158-91-62.eu-central-1.compute.amazonaws.com:9000
> Accept: */*
>
< HTTP/1.1 200
< Content-Type: application/json;charset=UTF-8
< Transfer-Encoding: chunked
< Date: Sun, 23 Apr 2017 06:43:42 GMT
<
* Connection #0 to host ec2-35-158-91-62.eu-central-1.compute.amazonaws.com left intact
[{"language":"??"},{"language":"de"},{"language":"en"},{"language":"fr"},{"language":"it"}]
> curl -v https://mathiasbynens.be/demo/ip
* Hostname was NOT found in DNS cache
* Trying 2a01:1b0:7999:402::144...
* Connected to mathiasbynens.be (2a01:1b0:7999:402::144) port 443 (#0)
* successfully set certificate verify locations:
* CAfile: none
CApath: /etc/ssl/certs
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Server key exchange (12):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* Server certificate:
* subject: OU=Domain Control Validated; OU=PositiveSSL Wildcard; CN=*.mathiasbynens.be
* start date: 2015-07-28 00:00:00 GMT
* expire date: 2018-08-12 23:59:59 GMT
* subjectAltName: mathiasbynens.be matched
* issuer: C=GB; ST=Greater Manchester; L=Salford; O=COMODO CA Limited; CN=COMODO RSA Domain Validation Secure Server CA
* SSL certificate verify ok.
> GET /demo/ip HTTP/1.1
> User-Agent: curl/7.38.0
> Host: mathiasbynens.be
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Sun, 23 Apr 2017 06:44:16 GMT
* Server Apache is not blacklisted
< Server: Apache
< Access-Control-Allow-Origin: *
< Strict-Transport-Security: max-age=15768000; includeSubDomains
< Vary: Accept-Encoding
< Cache-Control: max-age=0
< Expires: Sun, 23 Apr 2017 06:44:16 GMT
< X-UA-Compatible: IE=edge
< X-Content-Type-Options: nosniff
< X-Frame-Options: DENY
< X-XSS-Protection: 1; mode=block
< Transfer-Encoding: chunked
< Content-Type: application/json;charset=UTF-8
<
* Connection #0 to host mathiasbynens.be left intact
{"ip":"xxxx:xxxx:xxxx::xxx"}
The one difference that stuck out was that my rest service's response lacks the Transfer-Encoding and Access-Control-Allow-Origin? headers, so I'll look into adding those.
Still, if anyone has a hint on how to get more error information for what goes wrong with XmlHttpRequest I'd be glad to hear it.
Ok, it seems the missing Access-Control-Allow-Origin? header was the root of my problems.
I have now changed all methods in my Spring-RestControllers by adding another method parameter HttpServletResponse response and then calling setHeader() on that parameter.
#RequestMapping("/get-languages")
public #ResponseBody List<Language> getLanguages(HttpServletResponse response) {
response.setHeader("Content-Type", "application/json;charset=UTF-8");
response.setHeader("Access-Control-Allow-Origin", "*");
return languageRepository.findAll();
}
Now my WebExtension can use this rest service sucessfully using XmlHttpRequest.
It would have been helpful if this information (that the CORS header was missing) had been visible somewhere in firefox's debugging or js console, so if anyone can tell me how I could have seen this, I'd still appreciate a hint.
A Mule flow I have deployed to EC2 cannot send emails to a nodejs "simplesmtp" server running on another EC2 instance. Here are the authentication details for the SMPT server
user: joseph
pass: josephk
port: 2525 (open in security group)
I can send emails to this SMTP server using various clients on other hosts, here's the auth part of a nodejs client:
var smtpTransport = nodemailer.createTransport("SMTP",{
host: 'ec2-54-211-220-210.compute-1.amazonaws.com',
secureConnection: false,
port: 2525,
auth: {
user: "joseph",
pass: "joseph12"
}
});
My Mule smpt endpoint looks like this
<smtp:outbound-endpoint
host="ec2-54-211-220-210.compute-1.amazonaws.com"
port="2525"
user="joseph"
password="joseph12"
to="cleardot#aol.com"
from="joseph#ec2-54-211-220-210.compute-1.amazonaws.com"
subject="Email from Mule ESB"
doc:name="send notification"
/>
My log error is (I hate dumping log files on here but in this case have to)
INFO 2014-11-02 12:03:08,734 [[q2].connector.smtp.mule.default.dispatcher.01] org.mule.lifecycle.AbstractLifecycleManager: Starting: 'connector.smtp.mule.default.dispatcher.1535039248'. Object is: SmtpMessageDispatcher
ERROR 2014-11-02 12:03:08,815 [[q2].connector.smtp.mule.default.dispatcher.01] org.mule.exception.DefaultMessagingExceptionStrategy:
********************************************************************************
Message : Failed to route event via endpoint: DefaultOutboundEndpoint{endpointUri=smtp://joseph:<password>#ec2-54-211-220-210.compute- 1.amazonaws.com, connector=SmtpConnector
{
name=connector.smtp.mule.default
lifecycle=start
this=54d8fd1a
numberOfConcurrentTransactedReceivers=4
createMultipleTransactedReceivers=true
connected=true
supportedProtocols=[smtp]
serviceOverrides=<none>
, name='endpoint.smtp.joseph.compute.1.amazonaws.com.2525', mep=ONE_WAY, properties={toAddresses=cleardot#aol.com, subject=Email from Mule ESB, fromAddress=joseph#ec2-54-211-220-210.compute-1.amazonaws.com}, transactionConfig=Transaction{factory=null, action=INDIFFERENT, timeout=0}, deleteUnacceptedMessages=false, initialState=started, responseTimeout=10000, endpointEncoding=UTF-8, disableTransportTransformer=false}. Message payload is of type: MimeMessage
Code : MULE_ERROR--2
--------------------------------------------------------------------------------
Exception stack is:
1. 530 5.5.1 Authentication Required
(com.sun.mail.smtp.SMTPSendFailedException)
com.sun.mail.smtp.SMTPTransport:1829 (null)
2. Failed to route event via endpoint: DefaultOutboundEndpoint{endpointUri=smtp://joseph:<password>#ec2-54-211-220-210.compute-1.amazonaws.com, connector=SmtpConnector
{
name=connector.smtp.mule.default
lifecycle=start
this=54d8fd1a
numberOfConcurrentTransactedReceivers=4
createMultipleTransactedReceivers=true
connected=true
supportedProtocols=[smtp]
serviceOverrides=<none>
}
, name='endpoint.smtp.joseph.compute.1.amazonaws.com.2525', mep=ONE_WAY, properties={toAddresses=cleardot#aol.com, subject=Email from Mule ESB, fromAddress=joseph#ec2-54-211-220-210.compute-1.amazonaws.com}, transactionConfig=Transaction{factory=null, action=INDIFFERENT, timeout=0}, deleteUnacceptedMessages=false, initialState=started, responseTimeout=10000, endpointEncoding=UTF-8, disableTransportTransformer=false}. Message payload is of type: MimeMessage (org.mule.api.transport.DispatchException)
org.mule.transport.AbstractMessageDispatcher:117 (http://www.mulesoft.org/docs/site/current3/apidocs/org/mule/api/transport/DispatchException.html)
--------------------------------------------------------------------------------
Root Exception stack trace:
com.sun.mail.smtp.SMTPSendFailedException: 530 5.5.1 Authentication Required
at com.sun.mail.smtp.SMTPTransport.issueSendCommand(SMTPTransport.java:1829)
at com.sun.mail.smtp.SMTPTransport.mailFrom(SMTPTransport.java:1368)
at com.sun.mail.smtp.SMTPTransport.sendMessage(SMTPTransport.java:886)
+ 3 more (set debug level logging or '-Dmule.verbose.exceptions=true' for everything)
********************************************************************************
INFO 2014-11-02 12:03:09,697 [[q2].processArcMessage.stage1.02] org.mule.api.processor.LoggerMessageProcessor: send alert [B#3c32fb80
ERROR 2014-11-02 12:03:09,706 [[q2].connector.smtp.mule.default.dispatcher.01] org.mule.exception.DefaultMessagingExceptionStrategy:
Other comments:
I tried using joseph#ec2-54-211-220-210.compute-1.amazonaws.com as the user param with no luck.
Am wondering if Mule smpt requires that I use a standard port, or if I must use smpts not smpt.
My SMPT server logs attempts at authentication but sees nothing coming from Mule
Any help appreciated on this and the general issue of sending email from Mule on EC2
Clearly Mule can connect to your server because its response is visible in the logs:
530 5.5.1 Authentication Required
While your server was open to the web (see my comment above), I've noticed that it issued a 250 STARTTLS so I'm thinking that you should probably use the SMTPS transport for Mule, instead of the SMTP one in order to have proper support for TLS.