Modify the rendered string of a response in Grails 3 before sending to the client - html

I need to create a benchmark report regarding whether in the grand scheme of things: minifying + GZIP dynamic HTML responses (generated through GSPs) on every request, which will lead to an additional overhead due to parsing of the generated dynamic HTML string then compressing using a Java library (which results to a smaller response size) is actually better than GZIP without minifying (which results to faster response time but a little larger response size). I got the feeling that this "improvement" maybe is insignificant, but I need the benchmark report to back it up to the team.
To do that, I modify controller actions like so:
// import ...MinifyPlugin
class HomeController {
def get() {
Map model = [:]
String htmlBody = groovyPageRenderer.render(view: "/get", model: model)
// This adds a few milliseconds and reduce few characters.
htmlBody = MinifyPlugin.minifyHtmlString(htmlBody)
render htmlBody
}
}
But the Grails project has almost a hundred actions and doing this on every existing action is impractical and not maintainable, especially that after the benchmarking, we may decide to not minify the HTML response. So I was thinking of doing this inside an Interceptor instead:
void afterView() {
if(response.getContentType().contains("text/html")) {
// This throws IllegalStateException: getWriter() has already been called for this response
OutputStream servletOutputStream = response.getOutputStream()
String htmlBody = new String(servletOutputStream.toByteArray())
htmlBody = MinifyingPlugin.minifyHtmlString(htmlBody)
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream()
byteArrayOutputStream.write(htmlBody.getBytes())
response.setCharacterEncoding("UTF-8")
response.setContentType("text/html")
response.outputStream << byteArrayOutputStream
}
}
But it seems that modification of the response body is impossible once it enters the afterView interceptor...? So is any other way to do this using Grails 3 Interceptors, or should I update every controller action we have manually and perform the modification there instead?

This is what I like to use Interceptors for.
The after() part of the interceptor can act on the model after it is returned from the controller (wherein 'before()' acts on the request before it is sent to the controller)
This allows you to manipulate all data for a set of endpoints (or one specific endpoint) prior to return to client
If you are wanting to render to a view, you do that in the interceptor rather than in the controller; you merely return data from the controller

Related

JsonDocument incomplete parsing with larger payloads

So basically, I have a HttpClient that attempts to obtain any form of JSON data from an endpoint. I previously utilized Newtonsoft.Json to achieve this easily but after migrating all of the functions to STJ, I started to notice improper parsing.
Platforms tested: macOS & Linux (Google Kubernetes Engine)
Framework: .NET Core 3.1 LTS
The code screenshots below show an API that returns a JSON Array. I simply stream it, load it into a JsonDocument, and then attempt to peek into it. Nothing comes out as expected. Code below is provided along with the step debug var results.
using System;
using System.ComponentModel;
using System.IO;
using System.Linq;
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Text;
using System.Text.Json;
using System.Threading.Tasks;
using System.Web;
using System.Xml;
namespace HttpCallDemo
{
class Program
{
static async Task Main(string[] args)
{
using (var httpClient = new HttpClient())
{
// FLUSH
httpClient.DefaultRequestHeaders.Clear();
httpClient.MaxResponseContentBufferSize = 4096;
string body = string.Empty, customMediaType = string.Empty; // For POST/PUT
// Setup the url
var uri = new UriBuilder("https://api-pub.bitfinex.com/v2/tickers?symbols=ALL");
uri.Port = -1;
// Pull in the payload
var requestPayload = new HttpRequestMessage(HttpMethod.Get, uri.ToString());
HttpResponseMessage responsePayload;
responsePayload = await httpClient.SendAsync(requestPayload,
HttpCompletionOption.ResponseHeadersRead);
var byteArr = await responsePayload.Content.ReadAsByteArrayAsync();
if (byteArr.LongCount() > 4194304) // 4MB
return; // Too big.
// Pull the content
var contentFromBytes = Encoding.Default.GetString(byteArr);
JsonDocument payload;
switch (responsePayload.StatusCode)
{
case HttpStatusCode.OK:
// Return the payload distinctively
payload = JsonDocument.Parse(contentFromBytes);
#if DEBUG
var testJsonRes = Encoding.UTF8.GetString(
Utf8Json.JsonSerializer.Serialize(payload.RootElement));
// var testRawRes = contentStream.read
var testJsonResEl = payload.RootElement.GetRawText();
#endif
break;
default:
throw new InvalidDataException("Invalid HTTP response.");
}
}
}
}
}
Simply execute the above Minimal code, notice that the payload is different from its original after parsing? I'm sure there's something wrong with the options for STJ. Seems like we have to optimise or explicitly define its limits to allow it to process that JSON payload.
Diving deeper into the debug content made things even weirder. When the HttpClient obtains the payload, reads it to a string, it gives me the entire JSON string as is. However, once we attempt to parse it into a JsonDocument and the further invoking RootElement.Clone(), we'll end up with a JsonElement with much lesser data and while carrying an invalid JSON struct (Below).
ValueKind = Array : "[["tBTCUSD",11418,70.31212518,11419,161.93475693,258.02141213,0.0231,11418,2980.0289306,11438,11003],["tLTCUSD",58.919,2236.00823543,58.95,2884.6718013699997,1.258,0.0218,58.998,63147.48344762,59.261,56.334],["tLTCBTC",0.0051609,962.80334198,0.005166,1170.07399991,-0.000012,-0.0023,0.0051609,4178.13148459,0.0051852,0.0051],["tETHUSD",396.54,336.52151165,396.55,384.37623341,8.26964946,0.0213,396.50930256,69499.5382821,397.77,380.5],["tETHBTC",0.034731,166.67781664000003,0.034751,356.03450125999996,-0.000054,-0.0016,0.034747,5855.04978836,0.035109,0.0343],["tETCBTC",0.00063087,15536.813429530002,0.00063197,16238.600279749999,-0.00000838,-0.0131,0.00063085,73137.62192801,0.00064135,0.00062819],["tETCUSD",7.2059,9527.40221867,7.2176,8805.54677899,0.0517,0.0072,7.2203,49618.78868196,7.2263,7],["tRRTUSD",0.057476,33577.52064154,0.058614,20946.501210000002,0.023114,0.6511,0.058614,210741.23592011,0.06443,0.0355],["tZECUSD",88.131,821.28048322,88.332,880.37484662,5.925,0.0
And of course, attempting to read its contents would result in:
System.InvalidOperationException: Operation is not valid due to the current state of the object.
at System.Text.Json.JsonElement.get_Item(Int32 index)
at Nozomi.Preprocessing.Abstracts.BaseProcessingService`1.ProcessIdentifier(JsonElement jsonDoc, String identifier) in /Users/nicholaschen/Projects/nozomi/Nozomi.Infra.Preprocessing/Abstracts/BaseProcessingService.cs:line 255
Here's proof that there is a proper 38KBs worth of data coming in from the endpoint.
UPDATE
Further testing with this
if (payload.RootElement.ValueKind.Equals(JsonValueKind.Array))
{
string testJsonArr;
testJsonArr = Encoding.UTF8.GetString(
Utf8Json.JsonSerializer.Serialize(
payload.RootElement.EnumerateArray()));
}
show that a larger array of arrays (exceeding 9 elements each with 11 elements) would result in an incomplete JSON struct, causing the issue i'm facing.
For those who are working with JsonDocument and JsonElement, take note that the step debug variables are not accurate. It is not advisable to inspect the variables during runtime as they do not display themselves entirely.
#dbc has proven that re-serializing the deserialized data will produce the complete dataset. I strongly suggest you wrap the serializers for debugging in a DEBUG preprocessor to make sure these redundant lines don't end up being executed out of development.
To interact with these entities, ensure you .clone() whenever you can to prevent disposals and ensure that you're accessing the RootElement and then subsequently traversing into it before viewing its value in step debug mode because large values will not be displayed.

Dynamic parameter as part of request URI with Apache HttpCore

I am looking for existing solutions to match dynamic parameters with HttpCore. What I have in mind is something similar to constraints in ruby on rails, or dynamic parameters with sails (see here for example).
My objective is to define a REST API where I could easily match requests like GET /objects/<object_id>.
To give a little bit of context, I have an application that creates an HttpServer using the following code
server = ServerBootstrap.bootstrap()
.setListenerPort(port)
.setServerInfo("MyAppServer/1.1")
.setSocketConfig(socketConfig)
.registerHandler("*", new HttpHandler(this))
.create();
And the HttpHandler class that matches the requested URI and dispatches it to the corresponding backend method:
public void handle(final HttpRequest request, final HttpResponse response, final HttpContext context) {
String method = request.getRequestLine().getMethod().toUpperCase(Locale.ROOT);
// Parameters are ignored for the example
String path = request.getRequestLine().getUri();
if(method.equals("POST") && path.equals("/object/add") {
if(request instanceof HttpEntityEnclosingRequest) {
addObject(((HttpEntityEnclosingRequest)request).getEntity())
}
[...]
For sure I can replace path.equals("/object/add") by something more sophisticated with RegEx to match these dynamic parameters, but before doing so I'd like to know if I am not reinventing the wheel, or if there is an existing lib/class I didn't see in the docs that could help me.
Using HttpCore is a requirement (it is already integrated in the application I am working on), I know some other libraries provide high-level routing mechanisms that support these dynamic parameters, but I can't really afford switching the entire server code to another library.
I am currently using httpcore 4.4.10, but I can upgrade to a newer version of this might help me.
At present HttpCore does not have a fully featured request routing layer. (The reasons for that are more political than technical).
Consider using a custom HttpRequestHandlerMapper to implement your application specific request routing logic.
final HttpServer server = ServerBootstrap.bootstrap()
.setListenerPort(port)
.setServerInfo("Test/1.1")
.setSocketConfig(socketConfig)
.setSslContext(sslContext)
.setHandlerMapper(new HttpRequestHandlerMapper() {
#Override
public HttpRequestHandler lookup(HttpRequest request) {
try {
URI uri = new URI(request.getRequestLine().getUri());
String path = uri.getPath();
// do request routing based on the request path
return new HttpFileHandler(docRoot);
} catch (URISyntaxException e) {
// Provide a more reasonable error handler here
return null;
}
}
})
.setExceptionLogger(new StdErrorExceptionLogger())
.create();

Pass object as parameter in GET request using Google Http Client

I'm using Google Http Client and Jackson to query data to backend (JSON API).
I need to pass parameters (one Java bean object). The object might have few or lot of field. Initially I attempt to pass it as content as follow:
HttpRequest request = requestFactory.buildGetRequest(getUrl(api)).setContent(new JsonCContent(jsonFactory, params));
However, I'm not allowed to set the HTTP content in GET operation.
Any suggestion how can I pass these parameters?
Under one condition:
I don't want to write a util method to convert this object into string of URL parameters. But if there's already reusable API to do it, that would be fine.
I need generic solution if possible. Because I'm going to apply this to 600 JSON API calls.
My last alternative would be to change backend to expect POST request instead of GET, then I perform POST operation on the client side.
Thanks
Instead of extends GenericUrl, you can use GenericUrl.put (inherit from GenericData) to set query parameters. For example:
GenericUrl genericUrl = new GenericUrl("http://yourapi.com/request");
genericUrl.put("user", "user name");
genericUrl.put("token", "token values");
HttpRequest request = requestFactory.buildGetRequest(genericUrl);
It seems like the expected usage is to extend the URL class you are using for your buildGetRequest() call. For instance, let's say you wanted to provide two extra query parameters called "user" and "token". You could do this with the following:
HttpRequest request = requestFactory.buildGetRequest(
new CustomUrl("http://www.yourserver.com").setUser(userId).setToken(token));
where the CustomUrl class is defined as:
public class CustomUrl extends GenericUrl {
public CustomUrl(String encodedUrl) {
super(encodedUrl);
}
#Key("user")
private String mUserId;
#Key("token")
private String mToken;
public CustomUrl setUser(String userId) {
mUserId = userId;
return this;
}
public CustomUrl setToken(String token) {
mToken = token;
return this;
}
}
The values are not necessary for the #Key annotations, but will be used as the name of the respective query parameters if provided. If omitted, the name of the variable will be used instead (see example)
Check google-http-client's javadoc for more info.

Restlet POST Form to send to client resource via JSON

I am getting a 415 Error when sending a form entry to another client resource via JSON. The target URI in my code below ("/message") works when not using the form (i.e. hit "/message" with a test mock object).
Here is my code to get the values of the form and do the post to the target resource. Am I missing something that needs to be done?
I am using the following:
Restlet: 2.1 RC5
GAE: 1.6.1
Form Restlet:
#Post
public void handlePost(Representation entity) {
final Form webForm = new Form(entity);
MessageEntity newMessage = new MessageEntity();
String subject = webForm.getFirstValue("subject");
String sendto = webForm.getFirstValue("email");
String message = webForm.getFirstValue("message");
newMessage.setCategoryID(subject);
newMessage.setAccountID(sendto);
newMessage.setMessageText(message);
ClientResource cr = new ClientResource(getRootRef()+ "/message");
cr.post(newMessage, MediaType.APPLICATION_JSON);
}
Target Resource ("/message")
#Post("json")
public void HandleRequest(MessageEntity messageEntity) {
// Logic here
}
Please let me know if you need more information
Thanks!
I have code that is very similar to yours that works fine. I am also running similar versions of Restlet and GAE. First question I have is are there other #Post methods in your Target Resource as sometimes the ordering matters.
Here are two versions of code that I have that work....
1)
public Representation postHandler() {
Reference commitsRef = new Reference(Consts.RESOURCE_BASE + "commitments/");
ClientResource commitsResource = new ClientResource(getContext(), commitsRef);
....
Representation commitsRep = commitsResource.post(commitForm);
That is posting a form to a Target resource that handles both #Post("json") and #Post("form")
2)
public Representation doPostFromGet() {
Reference takeActRef = new Reference(Consts.RESOURCE_BASE + "commitment/"
+ commitmentId + "/userActs/");
ClientResource takeActResource = new ClientResource(getContext(), takeActRef);
...
Representation takeActRep = takeActResource.post(newAct);
That is posting a Java object to a form that uses what I call the "Peierls magic". See:
http://tembrel.blogspot.com/2012/03/converting-forms-in-restlet-to-pojos.html
It allows you to have one post() in the Target and accept both forms and pojos.
On a minor note, if you are doing a post to add a new message, should the url be "/messages/" (plural) - and perhaps there is a typo somewhere? (An unlikely possibility, but I thought I would mention it).
Good luck,
RB

Angular & Spring: How to load an external page, without exposing the URL?

This might be an odd question, so I'm trying to explain my issue as well as I can.
I have a frontend application, that has to load an iframe-content from another page. However, the frontend application isn't allowed to simply call URL of the to-be integrated application but instead by a sort of proxy address. Even URL Parameters that allow changes on the iframe-content are supposed to be run through the Spring Backend first, which calls the particular page and then returns it to the frontend.
What did I do? At first, I tried to create a pretty barebones REST call. Keep in mind, I'm not very knowledgable when it comes to Spring
#RequestMapping(value = "/api/include-content", method = RequestMethod.GET)
#CrossOrigin(origins = "http://localhost:4200")
public ResponseEntity<String> getIFrameContent() {
String server1 = <page_in_frame>;
RestTemplate rest;
HttpHeaders headers;
rest = new RestTemplate();
headers = new HttpHeaders();
headers.add("Content-Type", "text/html");
headers.add("Accept", "*/*");
HttpEntity<String> requestEntity = new HttpEntity<String>("", headers);
ResponseEntity<String> responseEntity = rest.exchange(server1, HttpMethod.GET, requestEntity, String.class);
return responseEntity;
}
Getting a response from it looked fine, at least with a REST-Client for a quick test. So I went on with my frontend. I kept it as basic as possible there as well.
For the HTML Template
<div id="main-content" class="container-fluid">
<iframe id="iframe-content" frameborder="0" style="width: 90%; height: 700px">
</iframe>
</div>
And for the ts I created a method that works like this...
redirectTo() {
this.httpClient.get("http://localhost:8080/api/include-content", {responseType: "text"}).subscribe(data => {
this.redirectData = data;
}, (error) => {
this.errorService.showError(error)
}, () => {
let iFrame = document.getElementById("powerBIFrame");
iFrame['src'] = 'data:text/html,' + encodeURIComponent(this.redirectData);
});
}
What I didn't consider in this case is, that I don't get any resources back. I just get a static page without styling and/or functionality. Clicking any on the links just creates an empty page. I could only conclude that my approach is inherently flawed and that I require (perhaps?) some sort of proxy functionality on my backend?
You will have to expose a url to the frontend if you're using an iframe as it needs a source to load in this case.
If you're wanting to use a backend generated output (spring) you can simply point to the localhost/whatever (like you have now) and set it as the src. This will work and is trivial.
I would inject the response payload into the DOM rather than passing it as the src attribute especially if it's going to potentially be a large payload (as is the case with power bi views).