StackOverflowException in JsonConvert.DeserializeXmlNode - json

We recently upgraded to Json.NET 10.0r2 from 6.0.1 and since upgrading I noticed that one of our unit tests throws a Stack Overflow Exception when trying to deserialize invalid Json. The purpose of the test was to ensure handling of invalid Json. This same test used to throw a JsonSerializationException, but now is bringing down nUnit with the StackOverflow.
I've replicated it in Json.NET's own unit testing project with this test:
[Test]
public void FailOnInvalidJSON( )
{
string json = #"{'Row' : ";
Assert.Throws<JsonSerializationException>(()=>JsonConvert.DeserializeXmlNode(json, "ROOT"));
}
Any ideas on work a work-around?
Thanks!

Update
And promptly fixed in change set 822c3f0. Should be in the next release after 10.0.2.
Original Answer
It looks like a change to JsonTextReader in version 8.0.1 may have uncovered a bug in XmlNodeConverter.
In 7.0.1, when the unexpected end of file is reached, JsonReader.TokenType becomes JsonToken.None after the next attempt to Read(), which causes DeserializeNode() to throw an Unexpected JsonToken when deserializing node: None exception. But in 8.0.1 and later the TokenType appears to stay stuck at the type of the last encountered token, namely JsonToken.PropertyName, which causes the infinite recursion.
The correct fix would be, in XmlNodeConverter.DeserializeNode() around line 2171, to check the return from reader.Read():
case JsonToken.PropertyName:
if (currentNode.NodeType == XmlNodeType.Document && document.DocumentElement != null)
{
throw JsonSerializationException.Create(reader, "JSON root object has multiple properties. The root object must have a single property in order to create a valid XML document. Consider specifying a DeserializeRootElementName.");
}
string propertyName = reader.Value.ToString();
// Need to check the return from reader.Read() here:
if (!reader.Read())
{
throw JsonSerializationException.Create(reader, "Unexpected end of file when deserializing property: " + propertyName );
}
... And it appears there are a few more places in XmlNodeConverter.cs where the return from reader.Read() needs to be checked, for instance in ReadAttributeElements(JsonReader reader, XmlNamespaceManager manager) around line 1942.
You could report an issue if you want.
In the meantime, your options for a workaround would be:
Corrupt the JSON in a different manner, for instance like so:
string json = #"{'Row' : }";
And check for the more general exception JsonException.
Pre-parse the JSON into a JToken:
Assert.Throws<JsonException>(()=>JsonConvert.DeserializeXmlNode(JToken.Parse(json).ToString(), "ROOT"));

Related

Get object as JSON in IntelliJ Idea from debugger

Is it possible to get the whole object from debugger as Json?
There is an option View text but can I somehow View JSON?
EDIT: as noted in the comments, this is not perfect, as for some variables you will get a "stackoverflow" response
As suggested by #Mr Han's answer, here's how you can do this:
Add a new way to view objects in Intellij debugger as json by
Going to File | Settings | Build, Execution, Deployment | Debugger | Data Views | Java Type Renderers
Click + to add new renderer
Call it JSON renderer
Supply java.lang.Object for Apply renderer to objects of type
Choose Use following expression: and supply an expression like so:
if (null == this || this instanceof String)
return this;
new com.google.gson.GsonBuilder().setPrettyPrinting().create().toJson(this);
Click OK
Now when you choose Copy Value on a variable, it will copy as json.
Alternatively, as seen here, you can use the following piece of code in your debug watcher:
new ObjectMapper()
.setSerializationInclusion(JsonInclude.Include.NON_NULL)
.writerWithDefaultPrettyPrinter()
.writeValueAsString( myObject )
You can try this code fragment into the Evaluate Expression(Alt + F8) on IntelliJ :
new com.fasterxml.jackson.databind.ObjectMapper() .registerModule(new com.fasterxml.jackson.datatype.jsr310.JavaTimeModule()) .disable(com.fasterxml.jackson.databind.SerializationFeature.WRITE_DATES_AS_TIMESTAMPS) .writerWithDefaultPrettyPrinter() .writeValueAsString( myObject );
image IntelliJ
You could use the Show as ... plugin for IntelliJ.
A small plugin to display formatted data out of the debugger and console.
Uses IntelliJ's build-in formatting capabilities.
No more need to copy values from debugger or console to a file to format them there. Following formats are supported: JSON, SQL, XML, Base64 encoded JSON, Base64 encoded text
If you have gson dependency in your project you can create a watch variable
new GsonBuilder().setPrettyPrinting().create().gson.toJson(myObject)
where myObject is your object.
Just follow it : File | Settings | Build, Execution, Deployment | Debugger | Data Views | Java Type Renderers, click + to add new render ,
copy is OK :) u can choose another jar to format it
And now , Apply, join it ~
Follow the instructions of #BradParks, and use the following expression.
For me it did not work without fully-qualified class names. I also added some modifications to the ObjectMapper. For some reason which I don't understand, even if I have Apply renderers to object of type set to java.lang.Object, I needed to typecast this as (Object)this when used as a parameter of the writeValueAsString() method.
if (this == null
|| this instanceof CharSequence
|| this instanceof Number
|| this instanceof Character
|| this instanceof Boolean
|| this instanceof Enum) {
// Here you may add more sophisticated test which types you want to exclude from the JSON conversion.
return this;
}
new com.fasterxml.jackson.databind.ObjectMapper()
.registerModule(new com.fasterxml.jackson.datatype.jsr310.JavaTimeModule())
.disable(com.fasterxml.jackson.databind.SerializationFeature.WRITE_DATES_AS_TIMESTAMPS)
.setVisibility(
com.fasterxml.jackson.annotation.PropertyAccessor.FIELD,
JsonAutoDetect.Visibility.ANY)
.setSerializationInclusion(com.fasterxml.jackson.annotation.JsonInclude.Include.NON_NULL)
.writerWithDefaultPrettyPrinter()
.writeValueAsString((Object)this);
In case someone is having hard time to make the renderers work for more complex objects - below you can find a combined JSON renderer from:
https://stackoverflow.com/a/60189764/3837638
https://stackoverflow.com/a/69802068/3837638
https://stackoverflow.com/a/63060407/3837638
The following renderer helped me to identify multiple fields with the same name in the class hierarchy, so I could change that.
Initially I was having IllegalArgumentException for serialization of a nested object that I wasn't able to analyse.
If there is an issue during serialization, with this renderer you can find the stack trace from the exception that you need to fix in the console.
Good luck!
if (null == this)
return "null";
if (this instanceof CharSequence
|| this instanceof Number
|| this instanceof Character
|| this instanceof Boolean
|| this instanceof Enum) {
// Here you may add more sophisticated test which types you want to exclude from the JSON conversion.
return this;
}
try {
String json = new GsonBuilder().setPrettyPrinting().create().toJson(this);
return json;
} catch (Exception e) {
e.printStackTrace();
}
Use Intellij plugin Debug Variable Extractor
More information -
https://plugins.jetbrains.com/plugin/16362-debug-variable-extractor

Xstream ConversionException for Wrong Input String

I'm using xStream to some JSON. I've used xstream quite extensively over the years. However this issue has me stumped.
I'm getting the following ConversionException...
com.thoughtworks.xstream.converters.ConversionException: For input string: ".232017E.232017E44"
---- Debugging information ----
message : For input string: ".232017E.232017E44"
cause-exception : java.lang.NumberFormatException
cause-message : For input string: ".232017E.232017E44"
class : java.sql.Timestamp
required-type : java.sql.Timestamp
converter-type : com.etepstudios.xstream.XStreamTimestampConverter
line number : -1
class[1] : com.pbp.bookacall.dataobjects.AppleReceipt
converter-type[1] : com.thoughtworks.xstream.converters.reflection.ReflectionConverter
class[2] : com.pbp.bookacall.dataobjects.AppleReceiptCollection
version : 1.4.10
-------------------------------
at com.etepstudios.xstream.XStreamTimestampConverter.unmarshal(XStreamTimestampConverter.java:87)
In my XStreamTimestampConverter class I print out the value that is attempting to be converted.. Which turns out to be the following...
XStreamTimestampConverter value = 2017-08-05 23:44:23.GMT
Here is the unmarshal function in my converter...
public Object unmarshal(HierarchicalStreamReader reader, UnmarshallingContext context)
{
Timestamp theTimestamp;
Date theDate;
String value = reader.getValue ();
try
{
SimpleDateFormat formatter = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss.Z");
theDate = formatter.parse(value);
theTimestamp = new Timestamp (theDate.getTime());
}
catch (Exception e)
{
System.out.println ("XStreamTimestampConverter value = " + value);
throw new ConversionException(e.getMessage(), e);
}
return theTimestamp;
}
Any idea where this odd string is coming from? It does not exist anywhere in my JSON. Does xstream have some odd .[num]E.[num]E[num] notation for something? These numbers can change as I run this each time. Also I get an For input string: "" on occasion too. Yet the value is similar to the what is above. It's like it's randomly getting odd values for somewhere.
The data source is from Apple's In-App Purchase /VerifyReceipt web call. The system works just fine some times but then others it does not. It's also important to note that in this very case it parsed 100s of other Date/Timestamp strings using this converter. It just get's confused. Perhaps due to the size of the data?
So I figured out what was going on here. The unmarshal function above is not exactly as I have it in code...
The SimpleDateFormat formatter is actually set in the class rather than in the unmarshal method. Therefore if Xstream holds on to an instance of my converter and the unmarshal is called across multiple threads then it is possible that the formatter can get confused since it is the same object.
That's my only guess at this point as moving the formatter initialization into the method solved the issue. I would say SimpleDateFormatter is not thread safe?
It was the just the sheer about of data and the number of times it was concurrently being called that exposed this issue. Just a tip for anyone else in case this happens to them.

How to throw a string literal in java

Is there a similar way of throwing a string literal in java than you would in c++?
In my assignment we have to convert our c++ code to java and I cannot get java to throw a string literal.
if (!head)
{
if (index == 0)
head = new Node<T>(element);
else
throw "Cannot add to empty list";
}
else if (index < 0 || index > size())
throw "Invalid index";
How would I implement this code snippet in Java? The main function will catch our exceptions so we do not need to catch it outself.
You can only throw something that implements Throwable. So no, you can't throw a String. You would have to make something that implements throwable, then throw it. Probably better to have it implement exception though and have your exception message be the String.
In your specific case, it looks like you are implementing some sort of data structure. You should look at the closest data structure in the Java API and see what specific exceptions it throws in similar cases to get an idea.

JSON.NET Classifying JsonReaderException's to their specific error

I'm using the Newtonsoft json.NET parser for JSON parsing. In my deserialization, I have the following code so that errors when converting from String to Int will not force me to throw away the entire object:
var param2 = new JsonSerializerSettings
{
Error = delegate(object sender, Newtonsoft.Json.Serialization.ErrorEventArgs args)
{
args.ErrorContext.Handled = true;
}
};
bcontent = JsonConvert.DeserializeObject<BContent>(json, param2);
I do not have control of the input data and the parsing errors are very common so I need to be versatile enough to handle them. Unfortunately, marking all errors as handled causes the deserialization to not terminate when it runs into a different error in a constrained environment.
What I want to do is to mark the errors as handled when they are of a type with a similar message as:
Could not convert string to integer....
But not when they are something different, such as this error which causes the hang:
Unterminated string. Expected delimiter...
What I can do is something like this:
Error = delegate(object sender, Newtonsoft.Json.Serialization.ErrorEventArgs args)
{
if (args.ErrorContext.Error.Message.Contains("convert string to integer"))
args.ErrorContext.Handled = true;
}
But it seems like there's no other way to determine a more specific error than JsonReaderException. Has anyone encountered this issue before and found a better workaround than a String.Contains()?

Groovy end exception different from exception thrown

I am running into an extremely strange behavior in Groovy. When I throw an exception from a closure in a Script, the end exception that was thrown was different.
Here are the code and the details:
public class TestDelegate {
def method(Closure closure) {
closure.setResolveStrategy(Closure.DELEGATE_FIRST);
closure.delegate = this;
closure.call();
}
public static void main(String[] args) {
// Make Script from File
File dslFile = new File("src/Script.dsl");
GroovyShell shell = new GroovyShell();
Script dslScript = shell.parse(dslFile);
TestDelegate myDelegate = new TestDelegate();
dslScript.metaClass.methodMissing = {
// will run method(closure)
String name, arguments ->
myDelegate.invokeMethod(name, arguments);
}
dslScript.metaClass.propertyMissing = {
String name ->
println "Will throw error now!"
throw new MyOwnException("errrrror");
}
dslScript.run();
}
}
class MyOwnException extends Exception {
public MyOwnException(String message) {
super(message);
}
}
Script.dsl:
method {
println a;
}
So the plan is that when I run the main() method in TestDelegate, it will run the DSL script, which calls for the method method(). Not finding it in the script, it will invoke methodMissing, which then invokes method() from myDelegate, which in turns invoke the closure, setting the delegate to the testDelegate. So far, so good. Then the closure is supposed to try printing out "a", which is not defined and will thus set off propertyMissing, which will will throw MyOwnException.
When I run the code, however, I get the following output:
Will throw error now!
Exception in thread "main" groovy.lang.MissingPropertyException: No such property: a for class: TestDelegate
Now, it must have reached that catch block, since it printed "Will throw error now!" It must have thrown MyOwnException too! But somewhere along the lines, MyOwnException was converted to MissingPropertyException, and I have no idea why. Does anyone have any idea?
P.S. if I remove closure.setResolveStrategy(Closure.DELEGATE_FIRST) from TestDelegate#method(), the code acts as expected and throws MyOwnException. But I really need the setResolveStrategy(Closure.DELEGATE_FIRST) for my DSL project. And I would prefer to know the root cause of this rather than just removing a line or two and see that it works without understanding why.
I think this is what essentially happens: With a delegate-first resolve strategy, the Groovy runtime first tries to access property a on myDelegate, which results in a MissingPropertyException because no such property exists. Then it tries propertyMissing, which causes a MyOwnException to be thrown. Eventually the runtime gives up and rethrows the first exception encountered (a design decision), which happens to be the MissingPropertyException.
With an owner-first resolve strategy, propertyMissing is consulted first, and hence MyOwnException is eventually rethrown.
Looking at the stack trace and source code underneath should provide more evidence.