I am trying to access data from RESTAPI using groovy code where i am getting error as below:
groovy.lang.MissingMethodException: No signature of method: java.lang.String.call() is applicable for argument types: () values: []
Possible solutions: wait(), chars(), any(), wait(long), take(int), tap(groovy.lang.Closure)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.unwrap(ScriptBytecodeAdapter.java:70)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodN(ScriptBytecodeAdapter.java:182)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeClosure(ScriptBytecodeAdapter.java:586)
The error is coming mostly on the below part of the lines from code :
String requestString = getRequestStringPrefix() + sb.toString()
readHistory(authToken,ricMap,outFile)
writeInstFile(outFile)
I am really new in the groovy coding and not understanding exactly the cause of the issue and how to resolve this issue in the code.
With this getRequestStringPrefix() you are calling a method with that name or as a shortcut a method call() on the underlying object, then it looks like getRequestStringPrefix.call().
I'm not sure what your intention was, but the line:
String requestString = getRequestStringPrefix() + sb.toString()
should look like
String requestString = getRequestStringPrefix + sb.toString()
because the variable getRequestStringPrefix (a strange name for a var) is defined as String further down:
String getRequestStringPrefix = """{
"ExtractionRequest": {..."""
Related
I'm using Flink to process the data coming from some data source (such as Kafka, Pravega etc).
In my case, the data source is Pravega, which provided me a flink connector.
My data source is sending me some JSON data as below:
{"key": "value"}
{"key": "value2"}
{"key": "value3"}
...
...
Here is my piece of code:
PravegaDeserializationSchema<ObjectNode> adapter = new PravegaDeserializationSchema<>(ObjectNode.class, new JavaSerializer<>());
FlinkPravegaReader<ObjectNode> source = FlinkPravegaReader.<ObjectNode>builder()
.withPravegaConfig(pravegaConfig)
.forStream(stream)
.withDeserializationSchema(adapter)
.build();
final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
DataStream<ObjectNode> dataStream = env.addSource(source).name("Pravega Stream");
dataStream.map(new MapFunction<ObjectNode, String>() {
#Override
public String map(ObjectNode node) throws Exception {
return node.toString();
}
})
.keyBy("word") // ERROR
.timeWindow(Time.seconds(10))
.sum("count");
As you see, I used the FlinkPravegaReader and a proper deserializer to get the JSON stream coming from Pravega.
Then I try to transform the JSON data into a String, KeyBy them and count them.
However, I get an error:
The program finished with the following exception:
Field expression must be equal to '*' or '_' for non-composite types.
org.apache.flink.api.common.operators.Keys$ExpressionKeys.<init>(Keys.java:342)
org.apache.flink.streaming.api.datastream.DataStream.keyBy(DataStream.java:340)
myflink.StreamingJob.main(StreamingJob.java:114)
It seems that KeyBy threw this exception.
Well, I'm not a Flink expert so I don't know why. I've read the source code of the official example WordCount. In that example, there is a custtom splitter, which is used to split the String data into words.
So I'm thinking if I need to use some kind of splitter in this case too? If so, what kind of splitter should I use? Can you show me an example? If not, why did I get such an error and how to solve it?
I guess you have read the document about how to specify keys
Specify keys
The example codes use keyby("word") because word is a field of POJO type WC.
// some ordinary POJO (Plain old Java Object)
public class WC {
public String word;
public int count;
}
DataStream<WC> words = // [...]
DataStream<WC> wordCounts = words.keyBy("word").window(/*window specification*/);
In your case, you put a map operator before keyBy, and the output of this map operator is a string. So there is obviously no word field in your case. If you actually want to group this string stream, you need to write it like this .keyBy(String::toString)
Or you can even implement a customized keySelector to generate your own key.
Customized Key Selector
A third party NuGet package throws an exception and in its exception message there is a an error and a JSON object:
Request failed, Message: {"Message":"Some error message"}
How can I extract the JSON from string and get the Message property?
I know that I could use Regex to format the string before passing it to deserializer or even trim the text until the first {
Is there a cleaner way to do it using Json.NET?
No.
Json.Net is built to parse JSON. If you have extra text in the string that is not JSON, the parser will not be able to make sense of it. Your best bet is to strip off the text before the first brace (and after the last brace), like you suggested in your question. You can make a helper method to do this easily:
public static string ExtractJson(string text)
{
int i = text.IndexOf('{');
int j = text.LastIndexOf('}');
return i > -1 && j > i ? text.Substring(i, j - i + 1) : null;
}
Once you've extracted the JSON, you can use Json.Net like you normally would.
Fiddle: https://dotnetfiddle.net/WoflVv
Been asked to modify a project to save space in our console and log files that has the following Spring application property which I've been asked to leave in place:
spring.jackson.serialization.indent_output=true
Obviously, that's going to pretty print all of our JSON and take up a ton of space in the console and log files.
The application has a groovy class that extends Spring's OncePerRequestFilter class that grabs HTTP requests and responses and sends them through a groovy class that masks sensitive data. This class has a method that takes in a string which is essentially the stringified version of the request's or response's body.
Once the string has been masked, it is run through Jackson's ObjectMapper to basically undo the spring application property to make every print pretty:
Code #1:
return objectMapper.writer().without(SerializationFeature.INDENT_OUTPUT).writeValueAsString(stringToWrite);
Here's some example input that will be going through the code:
{
"person" : {
"personName" : "BAR, FOO",
}
}
... and the result ends up being ...
Result #1:
"{\r\n \"person\" : {\r\n \"personName\" : \"BAR, FOO\",\r\n }\r\n }"
... trying to take the easy way out, I figured applying a simple .replaceAll() on the string would knock out the explicit quote escapes and carriage returns, but I found the carriage returns disappeared and that the quote escapes just come right back...
Code #2:
return objectMapper.writer().without(SerializationFeature.INDENT_OUTPUT).writeValueAsString(stringToWrite.replaceAll("\\\"", '"').replaceAll("\\r", "").replaceAll("\\n", ""));
Result #2:
"{ \"person\" : { \"personName\" : \"BAR, FOO\" } }"
... and finally, if I just use
Code #3
.replaceAll("\\\"", "")
... instead of what I used in Code #2 which was ...
.replaceAll("\\\"", '"')
... then I get ...
Result #3
"{ person : { personName : BAR, FOO } }"
... but what is asked of me is ...
Desired Result:
"{ "person" : { "personName" : "BAR, FOO" } }"
It appears to be a result of something the objectMapper is using, but I'm a little at a loss at this point.
Turns out you can set this, but it isn't obvious.
There is probably a better way, but I found this works:
DefaultPrettyPrinter p = new DefaultPrettyPrinter();
DefaultPrettyPrinter.Indenter i = new DefaultIndenter(" ", "\n");
p.indentArraysWith(i);
p.indentObjectsWith(i);
mapper.setDefaultPrettyPrinter(p);
String jsonString = mapper.writerWithDefaultPrettyPrinter().writeValueAsString(myObject);
As it turns out, I was complicating this whole situation by thinking it was an issue with objectMapper. Instead, my custom OncePerRequestFilter is logging my custom javax.servlet.http.HttpServletRequestWrapper's requestBody which is a byte[] converted to a String so all I needed was to replace the explicit return carriages and line feeds for UTF-8 encoding:
stringToWrite.replaceAll("\\r", "").replaceAll("\\n", "")
Please pardon me if this is a repeat question. I have been through some of the questions/answers with a similar requirement but somehow got a bit overwhelmed and confused at the same time. My requirement is:
I get a JSON string/object as a request parameter. ( eg: params.timesheetJSON )
I then have to parse/iterate through it.
Here is the JSON that my grails controller will be receiving:
{
"loginName":"user1",
"timesheetList":
[
{
"periodBegin":"2014/10/12",
"periodEnd":"2014/10/18",
"timesheetRows":[
{
"task":"Cleaning",
"description":"cleaning description",
"paycode":"payCode1"
},
{
"task":"painting",
"activityDescription":"painting description",
"paycode":"payCode2"
}
]
}
],
"overallStatus":"SUCCESS"
}
Questions:
How can I retrieve the whole JSON string from the request? Does request.JSON be fine here? If so, will request.JSON.timesheetJSON yield me the actual JSON that I want as a JSONObject?
What is the best way to parse through the JSON object that I got from the request? Is it grails.converters.JSON? Or is there any other easy way of parsing through? Like some API which will return the JSON as a collection of objects by automatically taking care of parsing. Or is programatically parsing through the JSON object the only way?
Like I said, please pardon me if the question is sounding vague. Any good references JSON parsing with grails might also be helpful here.
Edit: There's a change in the way I get the JSON string now. I get the JSON string as a request paramter.
String saveJSON // This holds the above JSON string.
def jsonObject = grails.converters.JSON.parse(saveJSON) // No problem here. Returns a JSONObject. I checked the class type.
def jsonArray = jsonArray.timesheetList // No problem here. Returns a JSONArray. I checked the class type.
println "*** Size of jsonArray1: " + jsonArray1.size() // Returns size 1. It seemed fine as the above JSON string had only one timesheet in timesheetList
def object1 = jsonArray[1] // This throws the JSONException, JSONArray[1] not found. I tried jsonArray.getJSONObject(1) and that throws the same exception.
Basically, I am looking to seamlessly iterate through the JSON string now.
I have wrote some code that explains how this can be done, that you can see below, but to be clear, first the answers to your questions:
Your JSON String as you wrote above will be the contents of your POST payload to the rest controller. Grails will use its data binding mechanism to bind the incomming data to a Command object that your should prepare. It has to have fields corresponding to the parameters in your JSON String (see below). After you bind your command object to your actual domain object, you can get all the data you want, by simply operating on fields and lists
The way to parse thru the JSON object is shown in my example below. The incomming request is esentially a nested map, with can be simply accessed with a dot
Now some code that illustrates how to do it.
In your controller create a method that accepts "YourCommand" object as input parameter:
def yourRestServiceMethod (YourCommand comm){
YourClass yourClass = new YourClass()
comm.bindTo(yourClass)
// do something with yourClass
// println yourClass.timeSheetList
}
The command looks like this:
class YourCommand {
String loginName
List<Map> timesheetList = []
String overallStatus
void bindTo(YourClass yourClass){
yourClass.loginName=loginName
yourClass.overallStatus=overallStatus
timesheetList.each { sheet ->
TimeSheet timeSheet = new TimeSheet()
timeSheet.periodBegin = sheet.periodBegin
timeSheet.periodEnd = sheet.periodEnd
sheet.timesheetRows.each { row ->
TimeSheetRow timeSheetRow = new TimeSheetRow()
timeSheetRow.task = row.task
timeSheetRow.description = row.description
timeSheetRow.paycode = row.paycode
timeSheet.timesheetRows.add(timeSheetRow)
}
yourClass.timeSheetList.add(timeSheet)
}
}
}
Its "bindTo" method is the key piece of logic that understands how to get parameters from the incomming request and map it to a regular object. That object is of type "YourClass" and it looks like this:
class YourClass {
String loginName
Collection<TimeSheet> timeSheetList = []
String overallStatus
}
all other classes that are part of that class:
class TimeSheet {
String periodBegin
String periodEnd
Collection<TimeSheetRow> timesheetRows = []
}
and the last one:
class TimeSheetRow {
String task
String description
String paycode
}
Hope this example is clear enough for you and answers your question
Edit: Extending the answer according to the new requirements
Looking at your new code, I see that you probably did some typos when writting that post
def jsonArray = jsonArray.timesheetList
should be:
def jsonArray = jsonObject.timesheetList
but you obviously have it properly in your code since otherwise it would not work, then the same with that line with "println":
jsonArray1.size()
shuold be:
jsonArray.size()
and the essential fix:
def object1 = jsonArray[1]
shuold be
def object1 = jsonArray[0]
your array is of size==1, the indexing starts with 0. // Can it be that easy? ;)
Then "object1" is again a JSONObject, so you can access the fields with a "." or as a map, for example like this:
object1.get('periodEnd')
I see your example contains errors, which lead you to implement more complex JSON parsing solutions.
I rewrite your sample to the working version. (At least now for Grails 3.x)
String saveJSON // This holds the above JSON string.
def jsonObject = grails.converters.JSON.parse(saveJSON)
println jsonObject.timesheetList // output timesheetList structure
println jsonObject.timesheetList[0].timesheetRows[1] // output second element of timesheetRows array: [paycode:payCode2, task:painting, activityDescription:painting description]
I have the following NancyFX unit test. I use the Shouldly assertion library to give the set of extensions methods that start .Should---
[Fact]
public void Assessment__Should_return_assessment_state_for_specified_user()
{
const AssessmentState assessmentState = AssessmentState.Passed;
var user = Fake.Mentor();
using (var db = Fake.Db())
{
db.Save(user);
Fake.Assessment(user.Id, db, assessmentState);
db.ClearStaleIndexes();
}
var response = Fake.Browser(user.UserName, user.Password)
.Get("/assessment/state/" + user.Id, with => with.HttpRequest());
//var result = (dynamic)body.DeserializeJson<ExpandoObject>();
var result = (dynamic) JsonConvert.DeserializeObject<ExpandoObject>(response.Body.AsString());
result.ShouldNotBe(null);
((AssessmentState) result.State).ShouldBe(assessmentState);
}
This test calls a AssessmentService uri defined as /assessment/state/" + user.Id which returns a simple JSON object definition that has a single property State of type (enum) AssessmentState, either Passed, Failed or NotStarted.
Here is the service handler so you can see there are no tricks.
Get["/assessment/state/{userid}"] = parameters =>
{
var assessment = AssessmentService.GetByUserId(Db, (string)parameters.userid);
return assessment == null ? HttpStatusCode.NotFound : Response.AsJson(new
{
assessment.State
});
};
And here is an example the JSON this service call returns:
{"State":1}
Everything works fine until I try to Deserialize the JSON returned by the fake Nancy browser. First I tried to use the built in method provided by Nancy's BrowserResponse.Body object:
var result = (dynamic)response.Body.DeserializeJson<ExpandoObject>();
This deserializes to an empty object. Which is no good. However, if we use the Newtonsoft equivalent then everything is fine (almost).
var result = (dynamic) JsonConvert.DeserializeObject<ExpandoObject>(response.Body.AsString());
The JSON deserialization now works and so the following Shouldly assertion passes with flying colours:
((AssessmentState) result.State).ShouldBe(assessmentState);
However, for reasons that I suspect have to do with anonymous types, the following line fails at run-time (it compiles fine).
result.ShouldNotBe(null);
That is quite a lot of information. Let me distil it down to two questions:
Why does Nancy's built in JSON deserializer not work given that the Newtonsoft version does?
How do I work with the dynamic types generated by the JSON de-serialisation so that the Shouldly extension methods do not cause a run-time exception?
Thanks
I can't answer the first question, but WRT Shouldly and dynamic types, Shouldly's ShouldNotBe method is an extension method on object. The DLR doesn't allow you to call extension methods on objects typed as dynamic (hence the runtime binder exception you're seeing)
I'd suggest that if you want to call ShouldNotBe(null) on result, you'd have to cast it to an object first (ie: ((object)result).ShouldNotBe(null))
-x