I try to create an application in web2py framework. By default web2py server has Transfer-Encoding: Chunked header for response, but in that case when target remote web application sends GET request to my app it could get only first string of text from requested page (from file's content that displayed on page). If to use Content-Length instead, for example, with value of 1000 it will get 1000 bytes of data from page... But if I expect to response with huge range of data, how to set Content-Length parameter to infinity or by file (like here but with web2py syntax instead of php )?
If you are serving web2py via the built-in development server and serving files via the Expose functionality, then the files will be served via chunked transfer encoding.
However, if you instead use response.stream to serve files, the Content-Length header will be set automatically.
Related
My webapp running on Payara-Micro is a tool to listen to audio files and navigate freely through them using the javascript currentTime property.
So the browser has <audio src="..."> tags and to get the audio file, it sends http GET request to the server with the header Range: bytes=0-
Unfortunately Payara in response doesn't returns 206 code and Content-range: bytes 0-881403 but it returns 200 and this has the effect that when I use currentTime=10 for exemple, the currentTime becomes equal to 0!
Previously this app was running in PHP with an apache server and apache was supporting the range header.
Is it possible to configure PayaraMicro or Grizzly to support range request ? Or If I put an Apache server in front of PayaraMicro it will work ?
Thank you for your help!
Payara supports RFC-7233. I did a mistake: there was a filter in my code that removed the default response header for media files.
Here is the devil filter:
public void doFilter(...
(...)
httpServletResponse.setHeader("Content-Type", "text/html; charset=UTF-8");
filterChain.doFilter(servletRequest,servletResponse);
}
Subject closed!
I'm using Apache HttpClient 4.3.1 and I'm trying to integrate etag validation cache.
I've tried to "drop in" httpclient-cache CachingHttpClientBuilder instead of my usual HttpClientBuilder using instructions in here, but that didn't seem to do any good. While tracing the execution, it seems like a response that has "etag" header (weak etag) isn't considered cache-able - and so isn't retained for the next cycle.
Has anyone managed to use etag validation based cache with Apache HttpClient? I'm also open for alternative implementations.
Notes:
The server returns the first request with a weak etag header (W/"1234"). If the second request to the same URL has "If-None-Match=1234", the server returns 304. This is checked and working.
The server does not send any other cache header (expires, etc).
The whole setup works wonderfully when using a modern browser.
Whether a response is considered as cacheable or not is decided in
ResponseCachingPolicy#isResponseCacheable(org.apache.http.HttpRequest, org.apache.http.HttpResponse)
which checks for some headers using
ResponseCachingPolicy#isExplicitlyCacheable
when
header 'Expires' is set or the header 'Cache-Control:' has one of the values "max-age" "s-maxage" "must-revalidate" "proxy-revalidate" or "public" the response is considered cacheable.
For us, it worked to add "Cache-Control: 'must-revalidate' to the response on the server, along with the 'Etag' header.
With this settings the apache http client
stores the response of the first request in the cache
on the second request, sends a request to the server and if this responds with a HttpStatus 304 (Not Modified) returns a HttpStatus 200 (ok) and the original content to the caller
That is how it should be.
We are using release 4.5.2 of apache http client cache.
I'm using Dev HTTP Client Chrome extension to verify restful URL so i can build C# application that can consume it. I have a trouble with encoding meaning when response is shown inside plugin/browser encoding is not proper, but when i download it with that same plugin and open file with Notepad++ encoding is fine. I'm having same problem with my C# application when reading JSON response from that web service. I also used restclient-ui-3.1 to check data but it behaves same as Chrome plugin, meaning it displays wrong characters in its response body tab.
Obviously web service is sending properly encoded data but i can't manage to read it accordingly on client side. Any hints?
Dev HTTP Client uses Content-Encoding and Content-Type HTTP headers to determine how is response body encoded (e.g. Content-Encoding: gzip) and what representation contains (e.g. Content-Type: application/json; charset=utf-8).
It seems that your webservice is not sending proper HTTP headers. Check them.
I have a client that will be posting large JSON files to an API server. Since the files are so compressable, I would like to gzip them and send the compressed data.
What I would like to know is: What is the best way to signal my intent to the server?
Basically, I want the reverse of Accept-encoding, such that the server would treat the data as only compressed for transport purposes, and automatically decompress the data before interpreting it according to the Content-Type.
This means that I can't set the Content-Type field to application/gzip because it needs to be application/json for the server to understand what the true uncompressed data encoding is.
Content-Transfer-Encoding looks like it would serve my purposes, but it was built with email in mind, and only supports 7bit, quoted-printable, base64, 8bit, and binary.
Does transparent compression/decompression for HTTP transport from client to server exist? And if not, what are the best practices for what I'm trying to achieve?
The "reverse" of the Accept-encoding header is Content-encoding. This signals to the server that the content is gzipped:
Content-encoding: gzip
You're correct that you shouldn't use the Content-type header for this, since the gzip compression is purely a matter of how the request is encoded, not what it represents.
I'd like to know if the POST method on HTTP sends data as a QueryString, or if it use a special structure to pass the data to the server.
In fact, when I analyze the communication with POST method from client to server (with Fiddler for example), I don't see any QueryString, but a Form Body context with the name/value pairs.
The best way to visualize this is to use a packet analyzer like Wireshark and follow the TCP stream. HTTP simply uses TCP to send a stream of data starting with a few lines of HTTP headers. Often this data is easy to read because it consists of HTML, CSS, or XML, but it can be any type of data that gets transfered over the internet (Executables, Images, Video, etc).
For a GET request, your computer requests a specific URL and the web server usually responds with a 200 status code and the the content of the webpage is sent directly after the HTTP response headers. This content is the same content you would see if you viewed the source of the webpage in your browser. The query string you mentioned is just part of the URL and gets included in the HTTP GET request header that your computer sends to the web server. Below is an example of an HTTP GET request to http://accel91.citrix.com:8000/OA_HTML/OALogout.jsp?menu=Y, followed by a 302 redirect response from the server. Some of the HTTP Headers are wrapped due to the size of the viewing window (these really only take one line each), and the 302 redirect includes a simple HTML webpage with a link to the redirected webpage (Most browsers will automatically redirect any 302 response to the URL listed in the Location header instead of displaying the HTML response):
For a POST request, you may still have a query string, but this is uncommon and does not have anything to do with the data that you are POSTing. Instead, the data is included directly after the HTTP headers that your browser sends to the server, similar to the 200 response that the web server uses to respond to a GET request. In the case of POSTing a simple web form this data is encoded using the same URL encoding that a query string uses, but if you are using a SOAP web service it could also be encoded using a multi-part MIME format and XML data.
For example here is what an HTTP POST to an XML based SOAP web service located at http://192.168.24.23:8090/msh looks like in Wireshark Follow TCP Stream:
Post uses the message body to send the information back to the server, as opposed to Get, which uses the query string (everything after the question mark). It is possible to send both a Get query string and a Post message body in the same request, but that can get a bit confusing so is best avoided.
Generally, best practice dictates that you use Get when you want to retrieve data, and Post when you want to alter it. (These rules aren't set in stone, the specs don't forbid altering data with Get, but it's generally avoided on the grounds that you don't want people making changes just by clicking a link or typing a URL)
Conversely, you can use Post to retrieve data without changing it, but using Get means you can bookmark the page, or share the URL with other people, things you couldn't do if you'd used Post.
http://en.wikipedia.org/wiki/GET_%28HTTP%29
http://en.wikipedia.org/wiki/POST_%28HTTP%29
As for the actual format of the data sent in the message body, that's entirely up to the sender and is specified with the Content-Type header. If not specified, the default content-type for HTML forms is application/x-www-form-urlencoded, which means the server will expect the post body to be a string encoded in a similar manner to a GET query string. However this can't be depended on in all cases. RFC2616 says the following on the Content-Type header:
Any HTTP/1.1 message containing an entity-body SHOULD include a
Content-Type header field defining the media type of that body. If
and only if the media type is not given by a Content-Type field, the
recipient MAY attempt to guess the media type via inspection of its
content and/or the name extension(s) of the URI used to identify the
resource. If the media type remains unknown, the recipient SHOULD
treat it as type "application/octet-stream".
A POST request can include a query string, however normally it doesn't - a standard HTML form with a POST action will not normally include a query string for example.
GET will send the data as a querystring, but POST will not. Rather it will send it in the body of the request.
If your post try to reach the following URL
mypage.php?id=1
you will have the POST data but also GET data.