I am using AppMon 6.5 to monitor an application running in Oracle Weblogic with an Apache server as web server.
The clients perform a lot of HTTP POST requests with json payload and I would like to capture the sent data.
I have configured Web Server sensor (Apache agent) to capture all request headers and request parameters and the Servlet sensor (Weblogic agent) to capture all request headers, request parameters, session attributes...
But I am able to capture only the POST parameters when the clients send the parameters as form-data (e.g. login request consists in two post parameters sent as form-data)
The documentation of Web server sensor states this
Request Parameter Capturing
Capturing the request parameter has some limitations on the web server:
Only the first 20.000 characters of the combined string of query and POST body will be analyzed. Parameters which are beyond that limit cannot be captured.
Reading request parameters from the POST body is only supported for IIS7+ and Apache based web servers.
https://community.dynatrace.com/community/display/DOCDT65/Web+Server+Sensor
Is it not possible to capture the POST data sent as body (e.g json content)??
Thanks in advance
Regards.
Dynatrace AppMon does not capture Payload data, it would cause a high overhead. For almost all situations you can get the information you want another way though, e.g. via method arguments.
Related
Protocol: A standard to define a method of exchanging data over a network.
If a browser wants to communicate with a server, it has to create an HTTP request and send that HTTP request to the server to convey its request of resources and options. The server receives the request and process it and do the needful and create an HTTP response to send to the browser. The browser has to follow the HTTP specification in creating the HTTP request. The server also has to follow the HTTP specification in creating the HTTP response. This is how the communication between the browser and the server happens in a standard way to avoid conflicts by following the HTTP protocol.
Json Wire Protocol: A client has an object that has to be sent to a server. The client converts this object into a JSON object and sends it to the server. The server parses the JSON object and converts it back to object for use. The server converts the response object into a JSON object and sends it back to the client. The client then converts the JSON object to object for use.
Why the later is called as Json Wire Protocol?
You are pretty correct both about Protocol and JsonWireProtocol. At this point it is worth to mention that, earlier all implementations of WebDriver that communicated with the browser, or a RemoteWebDriver server shall use a common wire protocol. This wire protocol defines a RESTful web service using JSON over HTTP.
JSON Wire Protocol is an abstract specification of how automation behavior like clicking or typing or whatever you actually want to do with your automation script is mapped to selenium or appium or HTTP requests and response. The protocol will assume that the WebDriver API has been "flattened", but there is an expectation that client implementations will take a more Object-Oriented approach, as demonstrated in the existing Java API. The wire protocol is implemented in request/response pairs of "commands" and "responses".
What is JSON Wire protocol?
JSON (JavaScript Object Notation) is a lightweight format for data exchange between client and server. Applications use JSON objects to send and receive data between each other in the web world. JSON data structure is industry standard and can be used for sending and receiving data as Key & Value pair. Some people say its a very nice alternative for XML. We can save JSON files as .json extension.
How JSON looks like?
A simple json file looks like below and there are many online editors which can be used to edit and verify JSON structure.
{
"Student":{
"FirstName":"Pawan",
"LastName":"Garia",
"IdNumber":"12345",
"City" : "New Delhi",
"EmailID" : "email#gmail.com" }
}
Why JSON Wire Protocol was used in first place?
To implement a client-server architecture which can give us the following benefits.
You write test in any programming language.
You can perform or run test on cloud services like SauceLabs, BrowserStack or Selenium Grid setup.
You are not bound to run test only on the local machine.
Different Drivers(FirefoxDriver, ChromeDriver) can be crated for browsers and separate implementation by using the same standards.
So client-server implementation requires a standard set of the specification beforehand so that Server and Client should be in sync with each other in term of what is coming and going on request and response. It’s something like a language of communication with each other. So we need some common specification to solve this kind of requirement and the solution was HTTP.
Why HTTP is the solution?
HTTP is a standard for web and can be a good base for the specification. Every programming language has a good HTTP libraries which can be used for creating client and server for request and response calls.
How JSON Wire protocol worked with HTTP?
HTTP request and response are generally made of GET and POST requests which is out of scope for this discussion.
Current status
From Selenium perspective, JSON Wire Protocol is obsolete now and the WebDriver W3C Living Document is the new implementation.
WebDriver Communication
The WebDriver protocol is organised into commands. Each HTTP request with a method and template defined in the specification represents a single command and hence each command produces a single HTTP response. In response to a command, the remote end will run a series of actions known as remote end steps. These provide the sequences of actions that a remote end takes when it receives a particular command.
Command Processing
The remote end is an HTTP server reading requests from the client and writing responses typically over a TCP socket. In the specification the communication is modeled as the data transmission between a particular local end and remote end with a connection to which the remote end may write bytes and read bytes. The exact details of how this connection works and how it is established is a bigger topic and out of scope for this question. After a connection has been established, the remote end must read bytes from the connection until a complete HTTP request can be constructed from the data. If it is not possible to construct a complete HTTP request, the remote end must either close the connection, return an HTTP response with status code 500, or return an error with error code unknown error.
Outro
Difference between JsonWireProtocol mechanisms and the new standards in W3C Living Document when using Selenium
I have an Oracle Business Intelligence Publisher 12.2.1.2.0 server running and I am trying to call the REST API services in order to access a report (i.e, TestReport) present over the server by using Postman tool. I am using various methods (GET, POST, PUT and DELETE) available over the Oracle documentation to get the relevant data from the server. But, only three of the methods (Get report definition, Get report sample data and Get XDO schema) are working fine and the rest are giving either of the following errors: 400 Bad Request or 415 Unsupported Media Type or 404 Not Found or 405 Method Not Allowed or Could not get any response. Most of the errors are associated with the POST and PUT methods so there may be some data format issue with the request. The request URL is: http://localhost:port/xmlpserver/services/rest/v1/reports/Components%2FTestReport/
Please help.
I have attached one of the responses for "Run Report".
Post Method:: Header
Post Method:: Body
SOLVED::
Add the associated headers and body in the proper format and then send the POST request.
For Example- For running a report
In Postman tool:
URL: http://localhost:port/xmlpserver/services/rest/v1/reports/Components%2FTestReport/run
Authorization:: username/password --> Preview Request
Headers:: (Key/Value)
1. Authorization/Basic...
2. Content-Type/multipart/form-data; boundary="Boundary_1_1153447573_1465550731355"
3. Accept/multipart/form-data
Body::
--Boundary_1_1153447573_1465550731355
Content-Type: application/json
Content-Disposition: form-data; name="ReportRequest"
{"byPassCache":true,"flattenXML":false,"attributeFormat":"pdf"}
--Boundary_1_1153447573_1465550731355--
It will generate the report in PDF format
Refer: https://docs.oracle.com/middleware/12211/bip/BIPAP/op-v1-reports-%7BreportPath%7D-run-post.html
How can I send my mysql table parameters to a link through GET?
e.x. : http:/example.com/Send.ashx?id=[member.id]&name=[class.name]
You cannot access MySQL via http directly. You will need to set up a rest API server that handles http requests, makes the corresponding SQL queries, formats the result (typically as json or XML) and sends it back to the client.
In case you already have an API / web server, we need more detailed information on your setup
My application is based on AngularJS, totally client-side; the server is based on Express JS. For data communication I am using the http post method. When I send a http request, the server responds with data in JSON format, but all the JSON data shows in the client browser. I don't want to show JSON data in client browser.
Is there any way to hide or secure json data in client browser?
What ever the response you will send, will be shown at the client-end. If you want to hide some data, you can always encrypt them and send it. One of the useful tool for such is Crypto-JS.
User will still see the data but as it will be encrypted, he cannot understand it.
But, still it is safer not to send user-sensitive data to client-side.
The answer to this little question will clear everything up for me.
If have a form tag that has a Get method and an action of some random script.
When I hit the submit button on the page, the Get Method is sent to HTTP and HTTP is what appends the query string to the url, the HTTP then returns a 20X status if the response is good and a 40X is a bad response? And our action goes to our webserver to run the script?
HTTP is transport and HTML is content. The Form submit calls a GET or POST request on the server depending on the action defined for the HTML form. The Form's arguments are appended by the Browser's form logic to the HTTP request, depending whehter GET or POST is used, they are attached to the request URL or put into the request body.
Then the request is handled on the server and the result is returned by the server logic (which can be a CGI, some perl script, a J2EE application etc.).
The server seponds with a HTTP status code (where everything below 300 is a success, and everything above 399 is an error - see here:HTTP staus codes ).
You are sending your form's data via HTTP using the "get" request. HTTP is a protocol and not a server. Your request is handled by a server who knows how to handle the HTTP protocol, eg. Apache.
The server processes the data and sends back a response. As you mention there are different kind of responses. 404 is best known (document not found).
The script is not run on the server, it is run on the client (the browser).
HTML is the markup code that describes the structure of the page. Browsers interpet the HTML code they receive and construct your page from it. Check here for more details: Wikipedia: HTML
The HTTP is the protocol used by the browser to talk to the server. Check this for more details: Wikipedia again: HTTP