I am sending a GET request for JSON data to my proxy server via lighttpd:
Firefox -> lighttpd -> proxy server
Lighttpd is configured for gzipped json via the following line in lighttpd.conf:
compress.filetype = ( "application/json", "application/javascript", …)
The proxy server gzips the JSON, adds
Content-Type: application/json
Content-Encoding: "gzip"
to the HTTP response header, and sends the response to lighttpd, which forwards it to the client.
When Firefox receives the response, I get the following error:
SyntaxError: JSON.parse: unexpected character at line 1 column 1 of the JSON data
So there is something wrong with the compressed JSON (or its interpretation). I checked that the compressed JSON data has the same hash as if I save the uncompressed JSON to file and gzip it on the command line.
Can anyone offer any insight as to why Firefox won't interpret the compressed JSON?
Related
I'm trying to debug a call made from a Java desktop application to a Java server, which goes via Apache http server. The call is an http json call, I want to log the request and response on Apache. I have enabled the dump io module and can now see the request and response, but the response is formatted as binary:
16:58:10 GMT\r\nX-XSS-Protection: 1; mode=block\r\nX-Content-Type-Options:
nosniff\r\nReferrer-Policy: no-referrer\r\nbreadcrumbId: ID-machine-name-here-33747-
1626280202382-10-673\r\nContent-Type: application/json;charset=UTF-8\r\nSet-Cookie:
JSESSIONID=88F411906EC0D; Path=/; HttpOnly\r\nVary: Accept-Encoding\r\nContent-
Encoding: gzip\r\nSet-Cookie: ROUTEID=.1; path=/app-path\r\nContent-Length:
61\r\nConnection: close\r\n\r\n
[Fri Jul 16 17:58:07 2021] [debug] mod_dumpio.c(74): mod_dumpio: dumpio_out (data-
HEAP): \xabV\xaa\xc8\xcdQ\xb2R\xb2)JMK-
J\xcdKNuI,I\x0cIL\xcaIuIM\xcb\xcc\xcb,\xc9\xcc\xcf\xd3\xb7S\xaa\x05
Why is this showing as binary? Is there a way I can get this logged as text?
Why is this showing as binary?
Everything is binary, but the reason it's not showing as plain text is because this is the actual body that's sent, and the body is gzipped.
Is there a way I can get this logged as text?
You could turn off gzipping.
I'm trying to set up HAProxy to server a static JSON file on 504 errors. To test, we've set up the configuration file to timeout after 10 seconds, and to use the errorfile option:
defaults
log global
mode http
retries 3
timeout client 10s
timeout connect 10s
timeout server 10s
option tcplog
balance roundrobin
frontend https
maxconn 2000
bind 0.0.0.0:9000
errorfile 504 /home/user1/test/error.json
acl employee-api-service path_reg /employee/api.*
use_backend servers-employee-api if employee-api-service
backend servers-employee-api
server www.server.com 127.0.0.1:8000
Effectively, I'm trying to serve JSON instead of HTML on a timeout, so the backend service can fail gracefully. However, on testing, we could not get anything, neither HTML or JSON. On looking at the response, it simply says it failed, with no status code. Is my setup correct for errorfile? Does HAProxy 1.5 support this?
According to the documentation of errorfile:
<file> designates a file containing the full HTTP response. It is
recommended to follow the common practice of appending ".http" to
the filename so that people do not confuse the response with HTML
error pages, and to use absolute paths, since files are read
before any chroot is performed.
So, the file should contain a complete HTTP response but you're trying to serve JSON only.
The documentation further says that:
For better HTTP compliance, it is
recommended that all header lines end with CR-LF and not LF alone.
The example configuration, for example,
errorfile 503 /etc/haproxy/errorfiles/503sorry.http
shows the common practice of .http extension for the error file.
You can find samples of some default error files here.
Sample (504.http):
HTTP/1.0 504 Gateway Time-out
Cache-Control: no-cache
Connection: close
Content-Type: text/html
<html><body><h1>504 Gateway Time-out</h1>
The server didn't respond in time.
</body></html>
So, in your scenario, 504.http would be like this:
HTTP/1.0 504 Gateway Time-out
Cache-Control: no-cache
Connection: close
Content-Type: application/json
{
"message": "Gateway Timeout"
}
Also, you need to keep the file size under limit i.e. BUFSIZE (8 or 16 KB) as described in the documentation.
There might be some error logs for not serving your JSON file. You might want to look at HAProxy's logs again thoroughly. Just to be sure.
The response from the IntelliJ rest client is mangled and isn't parsed as JSON. The 401 unauthorized code is expected.
GET https://circleci.com/api/v1.1/me
# Response
HTTP/1.1 401 Unauthorized
X-route: /api/v1.1/me
Content-Length: 35
## Malformed JSON
{
:message "You must log in first."}
However, cURL with the same URL has JSON output:
$ curl https://circleci.com/api/v1.1/me
{
"message" : "You must log in first."
}
How do I get JSON from the IntelliJ rest client?
I have such a route:
#app.route('/wikidata/api/v1.0/ask', methods=['POST'])
def get_tasks():
print(request.data)
print(request.json)
return jsonify(1)
I send a request:
curl -i -H "Content-Type:application/json" -X POST -d "{\"название\": \"значение?\",\"param1\": \"Q29424\"}" http://localhost:8529/wikidata/api/v1.0/ask
and get the error:
HTTP/1.0 400 BAD REQUEST
Content-Type: text/html
Content-Length: 223
Server: Werkzeug/0.14.1 Python/3.6.5
Date: Fri, 15 Feb 2019 08:34:27 GMT
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>400 Bad Request</title>
<h1>Bad Request</h1>
<p>Failed to decode JSON object: 'utf-8' codec can't decode byte 0xed in position 2: invalid continuation byte</p>
Meanwhile print(request.data) shows that request.data is b'{"\xed\xe0\xe7\xe2\xe0\xed\xe8\xe5": "\xe7\xed\xe0\xf7\xe5\xed\xe8\xe5?","param1": "Q29424"}'
The only thing that helped so far is
decoded_data = request.data.decode('windows-1251')
question = json.loads(decoded_data)
I'm looking a way to send a request properly (or configure server) so that I can use request.json without errors.
Thank you.
That's a Windows specific issue likely due to the default charset in the Windows console prompt. The cyrillic characters in your command line are misinterpreted with a non UTF-8 compatible charset.
Since you already use Python, the easiest way IMHO to send your request is to use the requests module (that you can install using pip install requests). Then type this command in a python file, using UTF-8 as charset:
import requests
requests.post("http://localhost:8529/wikidata/api/v1.0/ask", json={"название": "значение?","param1": "Q29424"})
Run it and it will have the same effect as your curl command line, only with proper cyrillic characters handling.
I'm running a tomcat server with a REST service on localhost.
This url typed in the browser gives me a json response: http://localhost:8080/a/rest/dataset/list
But when I use curl: curl http://localhost:8080/a/rest/dataset/list the response is HTML code with a 404 error from the server.
What am I missing in the curl command?
Tomcat may be waiting for a specially formatted HTTP request from the user. As you are expecting a response in json, you should probably forward the following argument to curl :
-H 'Content-Type: application/json'
Also, you may always specify to whatever HTTP server the response you are expecting him to return. A lot of back-end application based on HTTP are issuing a response given the HTTP request.