I'm receiving invalid characters on try to receiving JSON by http protocol in C.
When I send
GET /<query> http/1.1\r\nHost:<host>\r\n\r\n
then the result displays as follow:
HTTP/1.0 200 OK
Cache-Control: private
Content-Type: application/json; charset=utf-8
Content-Encoding: gzip
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET, POST
Access-Control-Allow-Credentials: false
X-Content-Type-Options: nosniff
Date: Sun, 07 Dec 2014 13:14:29 GMT
Content-Length: 11410
�
I don't know why I don't receiving JSON? However, in browser's address bar I type the same query & I received the json as well.
[UPDATE # 1]
I found it useful to work on built-in library, as the error is related to network byte compressions so I used Qt's library QNetworkManager specifically to done this job.
Please use a HTTP library or make yourself comfortable with the HTTP protocol and implement it properly.
Content-Encoding: gzip
At least this line means trouble for your simple parsing. But interestingly, based on your request the server should not sent this line if a non-compressed version is available. This means not only your code is buggy, but that the server might be buggy too (it actually assume that you understand any compressions because you did not say different).
GET / http/1.1\r\nHost:\r\n\r\n
And your request is wrong too. It must be HTTP/1.1 instead of http/1.1. And it must contain a proper host header.
Qt's QNetworkManager is best for this purpose, as it do all the intricate stuff itself & provide elegant interface to user.
Related
By default, Servant returns plain string requests even if the requested endpoint returns JSON
$ http $S/signup email=mail#domain.com
HTTP/1.1 400 Bad Request
Connection: keep-alive
Date: Tue, 14 Apr 2020 15:59:32 GMT
Server: nginx/1.17.9 (Ubuntu)
Transfer-Encoding: chunked
Error in $: parsing Credentials.Credentials(Credentials) failed, key "password" not found
I am trying to wrap such strings into simple JSON dictionaries:
$ http $S/signup email=mail#domain.com
HTTP/1.1 400 Bad Request
Connection: keep-alive
Date: Tue, 14 Apr 2020 15:59:32 GMT
Server: nginx/1.17.9 (Ubuntu)
Transfer-Encoding: chunked
{"error": "Error in $: parsing Credentials.Credentials(Credentials) failed, key \"password\" not found"}
But it looks like it's not that easy.
This question states possible solutions but I can't make them work today Custom JSON errors for Servant-server
Another approach is discussed in this thread https://github.com/haskell-servant/servant/issues/732 but it looks like overkill to such a simple task.
I wonder if there is a simple and robust solution in 2020?
There is a library called servant-errors. It provides a middleware that does exactly what you are looking for – transforms error responses to have a uniform structure of your choice, JSON being one of the built-in options.
See the documentation for details, but the basic usage is as straightforward as wrapping
errorMw #JSON #["error", "status"]
around your appilcation.
I am working with rails, and returning a json response with the below method
def return_json
render json: params
end
When i am viewing the response on chrome developers tools, everything seems to be right. But when i trace the HTTP response on wireshark, on HTTP response body it seems that some extra characters exists.
HTTP/1.1 200 OK
Server: nginx/1.10.3
Date: Wed, 05 Jul 2017 17:07:48 GMT
Content-Type: application/json; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
Cache-Control: max-age=0, private, must-revalidate
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
X-Runtime: 0.433854
X-Request-Id: f46a0e87-6969-4285-9b80-da0223edac01
X-Powered-By: Phusion Passenger 5.1.5
Status: 200 OK
49
{"device_attributes":[{"id":"85","value":"35"},{"id":"80","value":"65"}]}
0
(extra empty line)
I'm talking about the number 49 which is in hex and it seems to be the length of the JSON string. And after that it follows a 0 with an empty line.
Wireshark screenshot which shows the response
First of all, i would like to ask, what a valid HTTP response look like?
I think that after headers, follows an empty line and then the response body and after that nothing.
And second why rails do that and if there is a way to change that. I think that rails do that, because i get the same response from apache + phusion passenger and also puma. Also i tried this from some other code, not related to rails, and the HTTP response it was as i explained earlier and not as rails does.
I did not found out the answer i was looking for, but a workaround in order for the extra info to be removed, is to render as follows:
render plain: ActiveSupport::JSON.encode({ result: :ok })
This work around does set content-type as 'text/plain' and not 'application/json'. If you set content-type in render options as 'application/json' this extra info are being displayed again.
So i will assume that has something to do with the json renderer module, but i can't research it more at this time.
How should an HTTP Agent make decisions about using cached response when a request has the same path but different headers?
Take for example this HTTP request/response:
GET /resource HTTP/1.1
Host: example.org
X-Filter: foo=bar
HTTP/1.1 200 OK
Cache-Control: max-age=3600
Content-Type: application/json
Content-Length: 13
{"foo":"bar"}
Should the agent consider the response valid for a second request with a different X-Filter header? For example:
GET /resource HTTP/1.1
Host: example.org
X-Filter: foo=baz
then within an hour from the first request, should the agent request a fresh response since the request header differs, or should use the cached response from the first request, ignoring the header?
I'm asking this because I noticed that Google Chrome makes a new request, Microsoft Edge instead use the cached response.
You should use the cached version unless changed header appears in the list provided by the (optional) Vary response header.
For example, a response that contains
Vary: accept-encoding, accept-language
indicates that the origin server might have used the request's
Accept-Encoding and Accept-Language fields (or lack thereof) as
determining factors while choosing the content for this response.
I have a web page that returns the following header when I access material:
HTTP/1.1 200 OK
Date: Sat, 29 Jun 2013 15:57:25 GMT
Server: Apache
Content-Length: 2247515
Cache-Control: no-cache, no-store, must-revalidate, max-age=-1
Pragma: no-cache, no-store
Expires: -1
Connection: close
Using a chrome extension, I want to modify this response header so that the material is actually cached instead of wasting bandwidth.
I have the following sample code:
chrome.webRequest.onHeadersReceived.addListener(function(details)
{
// Delete the required elements
removeHeader(details.responseHeaders, 'pragma');
removeHeader(details.responseHeaders, 'expires');
// Modify cache-control
updateHeader(details.responseHeaders, 'cache-control', 'max-age=3600;')
console.log(details.url);
console.log(details.responseHeaders);
return{responseHeaders: details.responseHeaders};
},
{urls: ["<all_urls>"]}, ['blocking', 'responseHeaders']
);
Which correctly modifies the header to something like this (based on the console.log() output):
HTTP/1.1 200 OK
Date: Sat, 29 Jun 2013 15:57:25 GMT
Server: Apache
Content-Length: 2247515
Cache-Control: max-age=3600
Connection: close
But based on everything I have tried to check this, I cannot see any evidence whatsoever that this has actually happened:
The cache does not contain an entry for this file
The Network tab in the Developer Console shows no change at all to the HTTP response (I have tried changing it to even trivial modifications just for the sake of ensuring that its not a error, but still no change).
The only real hints I can find are this question which suggests that my approach still works and this paragraph on the webRequest API documentation which suggests that this won't work (but doesn't explain why I can't get any changes whatsoever):
Note that the web request API presents an abstraction of the network
stack to the extension. Internally, one URL request can be split into
several HTTP requests (for example to fetch individual byte ranges
from a large file) or can be handled by the network stack without
communicating with the network. For this reason, the API does not
provide the final HTTP headers that are sent to the network. For
example, all headers that are related to caching are invisible to the
extension.
Nothing is working whatsoever (I can't modify the HTTP response header at all) so I think that's my first concern.
Any suggestions at where I could be going wrong or how to go about finding what is going wrong here?
If its not possible, are there any other ways to achieve what I am trying to achieve?
I have recently spent some hours on trying to get a file cached, and discovered that the chrome.webRequest and chrome.declarativeWebRequest APIs cannot force resources to be cached. In no way.
The Cache-Control (and other) response headers can be changed, but it will only be visible in the getResponseHeader method. Not in the caching behaviour.
Im starting experimenting with Wikimedia, but I somehow can't get the login request working with a HTTP Client (RESTClient Firefox and others). This should be fairly simple, but it seems to fail or I have overlooked something evident.
I am using the instructions from this site.
This is what I insert in RESTClient:
I don't get the MediaWiki API Result back, but the help page (see below).
What am I doing wrong here? Thanks for any input.
Status Code: 200 OK
Cache-Control: private
Connection: keep-alive
Content-Encoding: gzip
Content-Length: 38052
Content-Type: text/html; charset=utf-8
Date: Mon, 09 Jul 2012 11:50:51 GMT
MediaWiki-API-Error: help
Server: Apache
Vary: Accept-Encoding
X-Cache: MISS from sq33.wikimedia.org, MISS from amssq35.esams.wikimedia.org, MISS from amssq39.esams.wikimedia.org
X-Cache-Lookup: MISS from sq33.wikimedia.org:3128, MISS from amssq35.esams.wikimedia.org:3128, MISS from amssq39.esams.wikimedia.org:80
X-Content-Type-Options: nosniff
<!DOCTYPE HTML>
<html>
<head>
<title>MediaWiki API</title>
</head>
<body>
<pre>
<span style="color:blue;"><?xml version="1.0"?></span>
<span style="color:blue;"><api servedby="mw67"></span>
<span style="color:blue;"><error code="help" info=""
xml:space="preserve"></span>
The are two problems with your request:
You're using the wrong URL. The correct domain is en.wikipedia.org, not www.wikipedia.org.
It seems RESTClient is using the Content-Type of text/plain by default, but the API expects application/x-www-form-urlencoded.
If you correct those two problems, you will get the correct response.
You also might want to indicate in what format do you want the response, by adding format=xml or format=json to the request. The default is HTML-formatted XML, which is useful for showing in a browser, but not for consumption by your application.