I am trying to study how the http2.0 request and response are encoded in binary. Postman doesn't support http2.0 I have tried to use curl --http2 to send a request and capture. But it is not working as http request package cannot be captured if there is no connection established. So I have to create an http2 server using python but it still failed for some reason.
Is there anyway to quickly view different http2 request packages in tools such as Wireshark so that I can view how it is encoded?
Note: It will be better that the request payload contains json payload or image payload. It will also be better if I can set up an easy RESTful http2 server with get and post method. It will be even better if a http2 request can be simulated directly in local machine or there is any public REST server with public documentation using http2 protocol.
Related
I have a .net WebApi service which is hosted on IIS server, it returns JSON response. when i hit the .net service via postman i get proper complete data as expected.
When i call the same service via Kong Api Gateway (Configured on linux machine with Nginx) from postman, i get truncated JSON response from postman and error calling from c# code.
Received an unexpected EOF or 0 bytes from the transport stream.
Response from postman as below
Can someone please provide me some pointers to proceed with.
This sounds very absurd but I opened up an old React project of mine and the view does not render. Upon inspection, I found it is receiving a HTML response from the Flask server. However, I am sending valid JSON response to the frontend. This is evident from the POSTMAN client too which throws a JSON response.
So to summarise, Postman Client is receiving expected JSON response from Flask server BUT the Chrome browser is not ! How /Why is this happening ?
Attaching the screenshot below
The server and client are running on different ports. The Postman requests are sent to the port 5000 and any endpoint is relative to localhost:5000 . Hence the response on Postman is as expected. However, for the React frontend to redirect its calls to the backend running on port 5000 , we need a way to tell the frontend to do so since endpoints are relative to the address on which they are running. Hence, an endpoint /questions for the server running on 5000 means localhost:5000/questions but for the frontend means localhost:3000/questions . This connection is provided by setting proxy on the package.json of your project. The value of the proxy would be the server address .
The issue: Webmethods HTTP client is calling the wrong endpoint on my Apache server configured with multiple virtual hosts, based on DNS.
What I think is happening: I think Webmethods HTTP client may be looking up the IP address and using that to perform HTTP operations instead of using the DNS name, which is causing the Apache server to identify it as a request to the main virtual server, not the desired one.
Question: So, how can I make webmethods use the DNS name instead of the IP? Is my theory about the Webmethods HTTP client correct? As far as I can tell this is a very non-standard approach to HTTP Client design.
Here is how it is configured to help you better understand:
Apache ->
host.example.com => /var/www/host/html
host2.example.com => /var/www/host2/html
curl -v http://host.example.com and curl -v http://host2.example.com appropriately return documents from their respective directories.
Configuring pub.client:http with http://host2.example.com causes the webmethods IS server to request http://host.example.com documents (obviously leading to a 404: Not Found).
Note that obviously the system is not returning documents like HTML but rather serving dynamic content.
The comment from Progman is the clue here - basically in order to direct Apache to call your virtual server, the Host header must be given with the expected value. In my example, that would be Host: host2.example.com. I was having webmethods IS copy over the headers exactly like I was posting them from curl, and it was sending Host: localhost:5555 over to my proxied server. I simply created a pipeline Map operation and hardcoded it and it is working fine now.
The oddity to me seems to be that pub.client:http didn't auto-set the Host header for me based on the 'url' value, which is what I would have expected.
I am new to http protocol. When we are sending json message over http to server, How we need to send ?
we need to send the data from different port each time
OR
we can send data form a single port in each time.
If I want to use existing connection to send data in future then whether it is possible or not ?
There is no reason why you would create a TCP socket for each piece of data you want to send — and this has nothing to do with HTTP — and particularly not through a different port each time. In fact, once you hace the socket created and you have connected to the server you should in principle always talk to the server through that socket.
Also, the HTTP protocol uses the port 80, and HTTPS uses 443. That number does not change on demand. Of course you can send HTTP requests through any available port you want and some services even run on special ports using HTTP as the communication protocol but normaly HTTP is 80. See the /etc/services file on linux and read about getaddrinfo().
As I understand:
A port designates a program on the server.
When we say to share a port, it actually means to have the requests processed by the same program listening on that port.
The WebSocket handshake resembles the HTTP format, so it can be understood by the server program that handles HTTP protocol. So it's OK to send the handshake request to port 80.
But after the handshake, the WebSocket data format is totally different from HTTP format, how could it still be sent to port 80? Such as via URL like below:
ws://somehost:80/chat
How does it work out?
My guess:
Does the HTTP program see that the incoming request on port 80 cannot be handled as HTTP, and then it will pass it to WebSocket program to process it. If so, what if there's some other protocol that wants to share port 80, say WebSocket2, how could HTTP program know which protocol to pass on to if there's not a way to identify the protocol being used.
ADD 1
Based on jfriend00's reply, I draw the following diagram:
So WebSocket and HTTP traffic in the same browser are actually carried out through different socket connections. Though they both start by connecting to server's port 80.
I think if the word WebSocket doesn't contain a socket in it, it will be easier to understand it as just another application level protocol over TCP protocol.
ADD 2
I refined the above diagram to below based on jfriend00's further comments.
What I want to show is how WebSocket communication and HTTP communication to the same server coexist in a browser.
ADD 3
After reading this thread, I recalled that the server port doesn't change when server accept a connection: Does the port change when a TCP connection is accepted by a server?
So the diagram should be like this:
The TCP connection for HTTP and the TCP connection for WebSocket should be using different client ports.
When a server listens on a given port, it is listening for incoming connections. When a new incoming connection arrives, it is given its own socket to run on. That socket provides the connection between the two endpoints. From then on, that socket runs completely independently from all other sockets that might also be connected.
So, one incoming http request can specify the "upgrade" header and get upgraded to webSocket and then both ends agree to talk the webSocket protocol from then on. Meanwhile, other incoming http requests without that upgrade header are treated only as normal http requests.
In case you don't quite understand how the webSocket protocol works, you can get a full look at how it connects here.
Here are the main steps:
The client requesting a webSocket connection, sends an HTTP request to the server on port 80.
That HTTP request is a perfectly legal HTTP request, but it has a header included on it Upgrade: websocket.
If the server supports the webSocket protocol, then it responds with a legal HTTP response with a 101 status code that includes a header Connection: Upgrade.
At that point, both sides then switch protocols to the webSocket protocol and all future communication on that socket is done using the data format for the webSocket frame.
Any other incoming HTTP requests that do not contain the upgrade request header are treated as normal HTTP requests.
Does the HTTP program see that the incoming request on port 80 cannot
be handled as HTTP, and then it will pass it to WebSocket program to
process it.
No, the first request IS a legal HTTP request (just with a special header in it) and the response sent back is a legal HTTP response. But, after that response, both sides switch protocols to webSocket. So a custom header is used to tell the web server that this incoming HTTP request is meant to be the first step in establishing a webSocket connection.
If so, what if there's some other protocol that wants to share port
80, say WebSocket2, how could HTTP program know which protocol to pass
on to if there's not a way to identify the protocol being used.
This upgrade mechanism could be used to support other protocols too by just specifying a different protocol name Upgrade: someOtherProtocol though I'm not aware of any others that have been standardized.
Because the browser use a new port to connect and send/receive messages to/from the server.