I am looking for a way to dump http request & reaponse body (json format) in resteasy on wildfly 8.2.
I've checked this answer Dump HTTP requests in WildFly 8 but it just dumps headers.
I want to see the incoming json message and outgoing one as well.
Can configuration do it without filter or any coding?
Logging HTTP bodies is not something frequently done. That's probably the primary reason for RequestDumpingHandler in Undertow only logging the header values. Also keep in mind that the request body is not always very interesting to log. Think for example of using WebSockets or transmitting big files. You can write your own MessageBodyReader/Writer for JAX-RS, and write to a ByteArrayOutputStream first, then log the captured content before passing it on. However, given the proven infeasibility of this in production, I think your mostly interested in how to do this during development.
You can capture HTTP traffic (and in fact any network traffic) using tcpflow or Wireshark. Sometimes people use tools such as netcat to quickly write traffic to a file. You can use for example the Chrome debugger to read HTTP requests/responses (with their contents).
Related
I would need to test the "resistance" of the server receiving an increasing number of websocket connections, through which data is sent over MQTT.
To that purpose, I created a simple webpage which has an "onLoad" directive so that when it loads, it connects to the server over websocket and starts receiving the data. In order to simulate (and increase the number of websocket connections), I thought about JMeter with a Loop controller that opens the page. However, I note that when the page is loaded by JMeter, it does not establish the websocket (in other words, it discards the "onLoad") javascript directive in the page. Is there a special JMeter configuration I'm missing? Is there another way to achieved what I would like?
Many thanks - Christian
As per JMeter project main page:
JMeter is not a browser, it works at protocol level. As far as web-services and remote services are concerned, JMeter looks like a browser (or rather, multiple browsers); however JMeter does not perform all the actions supported by browsers. In particular, JMeter does not execute the Javascript found in HTML pages. Nor does it render the HTML pages as a browser does (it's possible to view the response as HTML etc., but the timings are not included in any samples, and only one sample in one thread is ever displayed at a time).
therefore your onload event is being absolutely ignored by JMeter's HTTP Request sampler as it doesn't execute any functions.
If you want to load test your websocket backend you will need to go for JMeter WebSocket Samplers and establish connection/send payload using the relevant samplers. The plugin can be installed using JMeter Plugins Manager, there are JMeter WebSocket Samplers - A Practical Guide article and some example JMeter scripts which you can use as the reference
I've run into "application/x-amz-json-1.1" in making requests to AWS resources. Most recently, it became a problem that an API Gateway I was communicating with didn't like handling it (for whatever reason). This got me wondering what the benefit to using application/x-amz-json-1.1 instead of application/json for my requests is. And to my disappointment, AWS doesn't seem to have any documentation on this odd content type.
So I turn to SO: what is "application/x-amz-json" and how is it different from "application/json"?
Amazon does not specifically document what application/x-amz-json Content-Type is for, however there are protocol documentations on Smithy (an open source language for defining services and SDKs published by AWS):
AWS JSON 1.1 protocol
AWS JSON 1.0 protocol
Considering the question relates to the difference when used as Content-Type1 header to make requests, I think we can tell the difference is:
application/json is to request/receive JSON data without anything more specific
application/x-amz-json-1.1 (or other version) is also to request/receive JSON data and expect additional behaviors described in the docs above. (i.e. tell the server/client this is JSON plus additional elements)
I think application/x-amz-json can be thought as a sort of extension or a more specific way of doing application/json requests.
it became a problem that an API Gateway I was communicating with didn't like handling it (for whatever reason)
In the specific case of making PATCH, PUT and POST requests to AWS Amazon API Gateway, specifying Content-Type header application/x-amz-json-1.1 or other version seems to be required. As per related docs:
Content-Type (Conditional)
Specifies JSON and the version, for example, Content-Type: application/x-amz-json-1.0.
Condition: Required for PATCH, PUT and POST requests.
Maybe the server understands application/json as basic JSON but requires application/x-amz-json-1.1 to perform specific requests.
1 Content-Tye header being used to tell the server/client how to process our request
Protocol: A standard to define a method of exchanging data over a network.
If a browser wants to communicate with a server, it has to create an HTTP request and send that HTTP request to the server to convey its request of resources and options. The server receives the request and process it and do the needful and create an HTTP response to send to the browser. The browser has to follow the HTTP specification in creating the HTTP request. The server also has to follow the HTTP specification in creating the HTTP response. This is how the communication between the browser and the server happens in a standard way to avoid conflicts by following the HTTP protocol.
Json Wire Protocol: A client has an object that has to be sent to a server. The client converts this object into a JSON object and sends it to the server. The server parses the JSON object and converts it back to object for use. The server converts the response object into a JSON object and sends it back to the client. The client then converts the JSON object to object for use.
Why the later is called as Json Wire Protocol?
You are pretty correct both about Protocol and JsonWireProtocol. At this point it is worth to mention that, earlier all implementations of WebDriver that communicated with the browser, or a RemoteWebDriver server shall use a common wire protocol. This wire protocol defines a RESTful web service using JSON over HTTP.
JSON Wire Protocol is an abstract specification of how automation behavior like clicking or typing or whatever you actually want to do with your automation script is mapped to selenium or appium or HTTP requests and response. The protocol will assume that the WebDriver API has been "flattened", but there is an expectation that client implementations will take a more Object-Oriented approach, as demonstrated in the existing Java API. The wire protocol is implemented in request/response pairs of "commands" and "responses".
What is JSON Wire protocol?
JSON (JavaScript Object Notation) is a lightweight format for data exchange between client and server. Applications use JSON objects to send and receive data between each other in the web world. JSON data structure is industry standard and can be used for sending and receiving data as Key & Value pair. Some people say its a very nice alternative for XML. We can save JSON files as .json extension.
How JSON looks like?
A simple json file looks like below and there are many online editors which can be used to edit and verify JSON structure.
{
"Student":{
"FirstName":"Pawan",
"LastName":"Garia",
"IdNumber":"12345",
"City" : "New Delhi",
"EmailID" : "email#gmail.com" }
}
Why JSON Wire Protocol was used in first place?
To implement a client-server architecture which can give us the following benefits.
You write test in any programming language.
You can perform or run test on cloud services like SauceLabs, BrowserStack or Selenium Grid setup.
You are not bound to run test only on the local machine.
Different Drivers(FirefoxDriver, ChromeDriver) can be crated for browsers and separate implementation by using the same standards.
So client-server implementation requires a standard set of the specification beforehand so that Server and Client should be in sync with each other in term of what is coming and going on request and response. It’s something like a language of communication with each other. So we need some common specification to solve this kind of requirement and the solution was HTTP.
Why HTTP is the solution?
HTTP is a standard for web and can be a good base for the specification. Every programming language has a good HTTP libraries which can be used for creating client and server for request and response calls.
How JSON Wire protocol worked with HTTP?
HTTP request and response are generally made of GET and POST requests which is out of scope for this discussion.
Current status
From Selenium perspective, JSON Wire Protocol is obsolete now and the WebDriver W3C Living Document is the new implementation.
WebDriver Communication
The WebDriver protocol is organised into commands. Each HTTP request with a method and template defined in the specification represents a single command and hence each command produces a single HTTP response. In response to a command, the remote end will run a series of actions known as remote end steps. These provide the sequences of actions that a remote end takes when it receives a particular command.
Command Processing
The remote end is an HTTP server reading requests from the client and writing responses typically over a TCP socket. In the specification the communication is modeled as the data transmission between a particular local end and remote end with a connection to which the remote end may write bytes and read bytes. The exact details of how this connection works and how it is established is a bigger topic and out of scope for this question. After a connection has been established, the remote end must read bytes from the connection until a complete HTTP request can be constructed from the data. If it is not possible to construct a complete HTTP request, the remote end must either close the connection, return an HTTP response with status code 500, or return an error with error code unknown error.
Outro
Difference between JsonWireProtocol mechanisms and the new standards in W3C Living Document when using Selenium
I am writing a web service in node, and testing it with Postman. I spent a long timing looking for an error. When I finally found it, it turned out to be a simple error formatting the response body, which is json.
If I leave off the final brace in the response body, Postman waits for two minutes, and then reports that it received everything, just fine.
If I leave off the closing quote in the last value in the json, Postman says the server didn't respond, perhaps I should check my security certificates.
I would much rather Postman said "Hey, Buddy, you left off a quote!"
If there some validation service I can talk to? Or a plugin in Postman?
Here there are some validation javascript libraries, you can use:
Validator provides a declarative way of validating javascript objects.
Express-validator acts as an express.js middleware for node-validator.
Meanwhile, Postman got API testing and Collection Runner that can help you through this; which you can write some pre-request script as well as test script for each request.
Also, they got Newman which is a command-line collection runner. It allows you to effortlessly run and test a Postman collection directly from the command-line. It is built with extensibility in mind so that you can easily integrate it with your continuous integration servers and build systems.
I found that Paw worked (https://paw.cloud/). And so far I haven't paid for it.
Where Postman said "check your security certificates," Paw said "we were expecting 376 bytes but you only sent us 312."
Cuts down my time solving the problem a lot!
I use Fiddler for this. It is very good at identifying (with an error message that pops up) problems and bad implementations of the HTTP protocol. Browse the web with it running, and within a few minutes you'll undoubtedly hit a poorly implemented server.
Postman won't be able to handle these cases since it's insulated from poor behavior by the browser's framework.
That's not your problem though.
When I finally found it, it turned out to be a simple error formatting the response body, which is json.
That has absolutely nothing to do with HTTP. HTTP doesn't know or care what your request/response bodies are.
The problem you face is that your API endpoint could be returning whatever it wants. You need a custom solution to your problem, as there is no standard API server in this case.
Most folks will run unit tests that hit common endpoints of your service to ensure they're alive and well.
I should also point out that it should be all but impossible for you to break the JSON response if you're doing it correctly. Sounds like you're serializing JSON manually... never do that, we have JSON serializers for this purpose. Send in an object and let it worry about building the JSON output for you. Otherwise, you'll waste a lot of time on problems like these.
I am using Jetty's proxy servlet to route requests from a front-end client to Couch database. In addition to proxying, I am injecting an additional details, to the incoming request that is used by the CouchDB. HTTP request is modified by extending HTTPServletRequestWrapper. When I override the getContentLength method and recalculate the size of the inputStream, only the content of size of the original request is sent to CouchDB and since the input JSON is invalid Couch DB issues a Bad Request(400 HTTP) code. In the Jetty logger, I see that contentWritten != contentLength message. Once I remove Content-Length header altogether, Jetty sends the request as expected copying all the content to the proxied request. Even if this approach works, I am wondering what really causes Jetty to not send the modified request content even when the correct content length is set by overriding the getContentLength method of ServletInputStream.
I am using Jetty 8.1.3 version and all requests are JSON requests/responses. I have also subclassed the ServletInputStream class and read() method, so I don't think there is any problem with that.
This is a chunked encoding bug fixed recently in Jetty 7.6.10-SNAPSHOT and 8.1.10-SNAPSHOT.
https://bugs.eclipse.org/bugs/show_bug.cgi?id=401382
Fixed in Jetty-7 (and then merged into Jetty-8)
Commit #1
Commit #2
You can use Jetty 7.6.10-SNAPSHOT or 8.1.10-SNAPSHOT from the oss.sonatype.org SNAPSHOTS repository.
The 7.6.10 and 8.1.10 releases are likely going to occur in the first week of March 2013.
Yes, Jetty-7 and Jetty-8 are the same codebase, developed in parallel, and even released in parallel, with the only difference being the servlet spec supported by each. (Jetty-7 is Servlet 2.5, Jetty-8 is Servlet 3.0)