How did I solve the Json serializing circular reference error? - json

There is post here that asks how to solve the circular reference error when returning a serialized object via EF4 CTP5. I ran into this same problem with a WCF web forms project a while back.
I was able to "solve" this problem in my WCF/web forms project and in my MVC3 project. I don't think it matters what type of project as this appears to be a EF serialization "thing".
I solved the problem by disabling ProxyCreation in my ObjectContext constructor like this:
public class MyObjectContext : DbContext, IDbContext
{
public MyObjectContext(string connectionStringName) : base(connectionStringName)
{
((IObjectContextAdapter)this).ObjectContext.ContextOptions.ProxyCreationEnabled = false;
}
public DbSet<Product> Products {get;set;}
//etc.
}
My question is: Could someone explain why this would seemingly solve the problem?
I think the problem has to do with navigation properties in my POCO's but after that I am stumped. Thanks.

If you turn off proxy creation you also turn off lazy loading. When serialization of entity occures it visits all navigation properties. If lazy loading is enabled it loads all related objects and tries to serialize them as well. Again it visits all their properties including navigation properties pointing back to parent object. At this point you have to say serialization that this property is circular reference or it will serialize the object again and continue in infinite loop.
The trick here could be to annotate your circular navigation property in child entity with the ScriptIgnore attribute.

The circular reference happens because you use eager loading on the object.
You have a couple of methods:
Turn off eager loading when your loading your Query (linq or lambda)
DbContext.Configuration.ProxyCreationEnabled = false;
Remove the virtual keyword from the Domainmodel
Include them while loading the objects
Detach the objects (= no eager loading functionality & no proxy)
Repository.Detach(entityObject)
DbContext.Entry(entityObject).EntityState = EntityState.Detached
Clone the properties
You could use something like AutoMapper to clone the object, don't use the ICloneable interface, because it also clones the ProxyProperties in the object, so that won't work.
In case you are building an API, try using a separte project with a different configuration (that doesn't return proxies)
PS. Proxies is the object that's created by EF when you load it from the Entity Framework. In short: It means that it holds the original values and updated values so they can be updated later. It handles other things to ;-)

Quick note: if you still face the exception remember getting rid of
.Include("NestedObject")
This way parent-child relationship will be gone as well as the exception

Related

Unexpected behaviour from Gson

I developed a small application that stores data coming from a device: I chose to store data in JSON format, and the serialization/deserialization of the data works just fine, even if it involves some custom types created by me...but only I work in the IDE (Eclipse, for that matter).
When I export a runnable JAR file though, the deserialization of the data encounters some kind of problem, because the software always throws this exception:
Caused by: java.lang.UnsupportedOperationException: Cannot allocate class LocalDateTime
at com.google.gson.internal.UnsafeAllocator$4.newInstance(UnsafeAllocator.java:104)
at com.google.gson.internal.ConstructorConstructor$14.construct(ConstructorConstructor.java:225)
... 88 common frames omitted
I thought I'd encounter problems with custom types, not a built-in one. At this point, I discovered two things:
if I use a full JRE 9 to run the JAR file, the exception is not thrown: I double checked the modules included in the custom JRE I created with Jlink.exe, and everything is included correctly. I still want to use a smaller JRE, so I did not investigate further yet (I guess this explains why in the IDE it works perfectly)
I added a custom deserializer to the Gson object (see below), with which I simply manually converted the JSON string into a valid data, and that avoided the exception on the LocalDateTime class...but the exception reappeared simply on another class, this time a custom-made one.
At this point, I guess I can simply add a deserializer for each data type that causes problem, but I'm wondering why the issue won't happen with a full JRE, and why a smaller JRE causes this, even if all the modules required are included. Maybe it's worth mentioning also that I added no custom serializer to the Gson object that saves the data, it is all serialized as per Gson default.
LocalDateTime deserializer:
#Override
public LocalDateTime deserialize(JsonElement json, java.lang.reflect.Type type,
JsonDeserializationContext jsonDeserializationContext) throws JsonParseException {
JsonObject joDate = json.getAsJsonObject().get("date").getAsJsonObject();
JsonObject joTime = json.getAsJsonObject().get("time").getAsJsonObject();
//JSON example: {"date":{"year":2019,"month":1,"day":9},"time":{"hour":6,"minute":14,"second":1,"nano":0}
return LocalDateTime.of(joDate.get("year").getAsInt(),
joDate.get("month").getAsInt(),
joDate.get("day").getAsInt(),
joTime.get("hour").getAsInt(),
joTime.get("minute").getAsInt(),
joTime.get("second").getAsInt(),
joTime.get("nano").getAsInt());
}
}
Jdeps.deps modules list:
com.google.gson
java.base
javafx.base
javafx.controls
javafx.fxml
javafx.graphics
org.slf4j
After the answer I received, I opened an issue here.
TL;DR
You need a runtime image (e.g. full JDK or something built with jlink) that includes the module jdk.unsupported.
Full Answer
GSON wants to create instances of classes it deserializes without calling any constructors (so nothing gets initialized without GSON saying so). This can't normally be done, but sun.misc.Unsafe offers a way to do this with the method allocateInstance. To that end, GSON needs an instance of sun.misc.Unsafe. The topmost frame in the call stack is from UnsafeAllocator, which uses common trickery to get Unsafe.
The problem is, sun.misc.Unsafe is in module jdk.unsupported, which is present in a full JDK but you won't usually find in runtime images.
When creating your runtime image with jlink, make sure to include the option --add-modules jdk.unsupported and you should be good to go.
Arguably, GSON should declare an optional dependency on jdk.unsupported with requires static.
I have faced the same issue when packing compose a desktop application.
update build.gradle file, add an unsupported module.
compose.desktop {
application {
mainClass = "MainKt"
nativeDistributions {
targetFormats(TargetFormat.Dmg, TargetFormat.Msi, TargetFormat.Deb)
packageName = "admin"
packageVersion = "1.0.0"
modules("java.sql")
modules("jdk.unsupported")
}
}
}

Grails JSON marhsaling using introspection causes severe bottleneck on Classloader.loadClass()

I am using Grails 2.2.4 and have a controller endpoint which converts a domain object list to JSON. Under load (as little as 5 concurrent requests) the marshaling performance is very poor. Taking thread dumps the threads are blocked on:
java.lang.ClassLoader.loadClass(ClassLoader.java:291)
There is a single marhsaler registered to marshal all domain objects using reflection and introspection. Realizing that reflection and introspection is slower than direct method calls, I am still seeing unexpected behavior in that the class loader is caller every time and in turn blocking occurs. An example stacktrace is as follows:
java.lang.Thread.State: BLOCKED (on object monitor)
at java.lang.ClassLoader.loadClass(ClassLoader.java:291)
- waiting to lock <785e31830> (a org.grails.plugins.tomcat.ParentDelegatingClassLoader)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at java.beans.Introspector.instantiate(Introspector.java:1470)
at java.beans.Introspector.findExplicitBeanInfo(Introspector.java:431)
at java.beans.Introspector.<init>(Introspector.java:380)
at java.beans.Introspector.getBeanInfo(Introspector.java:167)
at java.beans.Introspector.getBeanInfo(Introspector.java:230)
at java.beans.Introspector.<init>(Introspector.java:389)
at java.beans.Introspector.getBeanInfo(Introspector.java:167)
at java.beans.Introspector.getBeanInfo(Introspector.java:230)
at java.beans.Introspector.<init>(Introspector.java:389)
at java.beans.Introspector.getBeanInfo(Introspector.java:167)
at java.beans.Introspector.getBeanInfo(Introspector.java:230)
at java.beans.Introspector.<init>(Introspector.java:389)
at java.beans.Introspector.getBeanInfo(Introspector.java:167)
at org.springframework.beans.CachedIntrospectionResults.<init>(CachedIntrospectionResults.java:217)
at org.springframework.beans.CachedIntrospectionResults.forClass(CachedIntrospectionResults.java:149)
at org.springframework.beans.BeanWrapperImpl.getCachedIntrospectionResults(BeanWrapperImpl.java:324)
at org.springframework.beans.BeanWrapperImpl.getPropertyValue(BeanWrapperImpl.java:727)
at org.springframework.beans.BeanWrapperImpl.getPropertyValue(BeanWrapperImpl.java:721)
at org.springframework.beans.PropertyAccessor$getPropertyValue.call(Unknown Source)
at com.ngs.id.RestDomainClassMarshaller.extractValue(RestDomainClassMarshaller.groovy:203)
...
...
A simple benchmark loading the same endpoint with the same parameters results in the loadClass call.
I was under the impression the classes would be at least cached by the class loader and not loaded on every method call to get the property to be marshaled.
The code to retrieve the property value is as follows:
BeanWrapper beanWrapper = PropertyAccessorFactory.forBeanPropertyAccess(domainObject);
return beanWrapper.getPropertyValue(property.getName());
Is there a configuration setting that is needed to ensure the classes are only loaded once? or perhaps a different way to get the property that doesn't result in class loading every time? Or perhaps a more performant way to achieve this?
Writing a custom marshaler per domain class would avoid the reflection and introspection but is going to be a lot of repeat code.
Appreciate any input.
So after much digging this is what I found out.
Using the BeanUtils.getPropertyDescriptors and getValue will always try and find a BeanInfo class describing the bean using the class loader. In this case we don't provide BeanInfo classes for our grails domain classes so this call is redundant. I found some information where you can provide a custom BeanInfoFactory to bypass this and exclude your packages but I couldn't find how to configure it with Grails.
Also searching the springframework documentation there is a configuration option you can pass Introspector.IGNORE_ALL_BEANINFO that will tell CachedIntorspectionResults to never look up the bean classes. However this was not available in version 3.1.4 of springframework which was current for grails 2.2.4. The newer versions do appear to have this option.
So, if using BeanUtils you can't by pass this initial lookup on the class loader. However subsequent loaders should be cached by CachedIntrospectionResults. Unfortunately this doesn't happen in our scenario. There looks to be a bug in the test to see if the lookup is cacheable. See more info on this below.
The fix was ultimately to fall back to use pure reflection. Rather than use:
beanWrapper.getPropertyValue(property.getName());
To use:
PropertyDescription pd = BeanUtils.getPropertyDescriptor(domainObject.getClass(), property.getName())
pd.readMethod.invoke(domainObject)
Where the pd is cached.
After fixing this the profiler still showed a lack of caching on CachedIntorspectionResults for the out of the box grails marshaller. This was due to the bad caching implementation in CachedIntrospectionResults. The work around for this was to add the correct class loader to the acceptedClassLoaders in the CachedIntrospectionResults.
public class EnhanceCachedIntrospectionResultsAcceptedClassLoadersListener implements ServletContextListener {
public void contextInitialized(ServletContextEvent event) {
CachedIntrospectionResults.acceptClassLoader(Thread.currentThread().getContextClassLoader().getParent());
}
public void contextDestroyed(ServletContextEvent event) {
CachedIntrospectionResults.clearClassLoader(Thread.currentThread().getContextClassLoader().getParent());
Introspector.flushCaches();
}
}
Note that it was required to add the parent to the accepted class loader list rather than the current class loader. Not sure if this is specific to grails or not but this fixed the issue. I'm not sure if there may be a side effect to this fix.
In summary we went from 10 requests/sec in the original setup to 120 requests/sec after using direct reflection and fixing the CachedIntrospectionResults cache.
However the real eye opened was that if we use a 1-1 marshaller per domain class we were seeing another x2 improvement in performance over the generic marshaller where we test objects for whether they're instances of class etc. We're saving a lot of code with the generic marshaller but there's a lot more work to do to get comparable performance to writing a 1-1 marshaller.
Hopefully this will be useful to someone else who runs into this ...

Grails JSON Marshalling Works after first running compile

I'm experiencing a discrepancy between the first compilation of a Grails app and the compilation that happens when a file changes while the app is running.
Background:
My app creates some spring beans from Spring LDAP (docs) using conf/spring/resources.groovy.
I have an LdapUser.groovy class in src/groovy (I'm using it similarly to a domain class, except it isn't in grails-app/domain as it doesn't map to a database table).
In BootStrap.groovy I register a JSON marshaller for LdapUser (using JSON.registerObjectMarshaller).
I have a controller with an index method that responds a list of LdapUser objects. This renders correctly in JSON (according to the marshaller).
With that background, here are the pieces of the problem:
When the show method, which responds a single LdapUser, gets called, I get an exception that LdapUser cannot be converted to grails.converters.JSON. (fair enough)
But, if I save the LdapUser.groovy file, thus invoking a recompile on the file while the app is running, the JSON marshaller suddenly works fine.
Before saving the LdapUser.groovy, my controller has a to an LdapUserRepo (a class instantiated via an #EnableLdapRepositories annotation on the controller), but this reference becomes null after I save LdapUser.groovy. I'm not sure how this relates to the problem, as I was also able to reproduce the problem in a controller lacking an injected LdapUserRepo (but with the annotated controllers still in the app).
I also at one point was setting an asType method on the LdapUser class, which was called as expected before the save-invoked recompile. After the recompile, however, my asType method was no longer called and the JSON marshaller was taking over. ( I was doing exception-worthy things in the asType that were throwing before recompile and not throwing after... )
My understanding of the problem is therefore:
Somehow the asType method of the LdapUser.groovy class is not being automatically generated on first compile when running the app, but is being generated on subsequent compiles.
The LdapUser class is tied to the LdapUserRepo in more ways than merely being a type the Repo uses, and the recompile is not reflecting that connection correctly.
Methods rendering lists of objects are somehow unaffected by the asType method. This leads me to believe that the JSON marshaller gets called directly on list elements (instead of via asType) when the list asType has been called (whether or not the "as" operation is implicit...).
My question then is:
what is the Grails compiler doing differently on run-app vs on compile while app is running that could be causing this behavior?
how can I restructure things to ensure it works properly out of the box?
If I need to RTFM, what would be the FM section? (My google-fu is sadly quite weak).
Note: this question is vaguely similar, but doesn't have any meaningfulness to the answer:
Grails: Defining a JSON custom marshaller as static method in domain

Resolving a dependency while supplying values for downstream dependencies

I've been running into endless problems attempting to use Windsor with Web API and injecting HttpRequestMessage into downstream dependencies of a controller. Since I've tried all the matching answers on Stackoverflow, I'd like to ask the question in a different way:
In Castle Windsor, how can I resolve a component instance while supplying a value for a downstream dependency? That is, the supplied value is required by a component that is required by the component being resolved.
For context, I'm trying to inject HttpRequestMessage so that I can use it to resolve the request context (primarily to resolve an absolute URL).
Edit I'd also like to point out that I don't currently have a dependency on Web Host / System.Web and I'd rather not change that.
A proper approach is to
Create IMyDesiredRouteParameterProvider
Implement it. Get the current request inside it and get the url
Register it and inject it in the desired dependent class via constructor.
I made myself such an implementation and I can say that this way it works fine. You can make Web.Infrastructure assembly and put the implementation there. Or put both the interface and the implementation there if you are going to reference it from another web module.
using System;
using System.Web;
namespace RouteParameterProvider
{
interface IMyRouteParameterProvider
{
string GetRouteParameter();
}
public class ControllerActionMethodRouteParameterProvider : IMyRouteParameterProvider
{
public string GetRouteParameter()
{
string Parameter = HttpContext.Current.Request.RequestContext.RouteData.Values["controller"] as string;
if (string.IsNullOrEmpty(Parameter))
{
throw new InvalidOperationException();
}
return Parameter;
}
}
}
You can get every possible thing that the Request Context contains from :
HttpContext.Current.Request.RequestContext
And it will be better if you rethink your design decision :
I need HttpRequestMessage to be regstered prior to creating each
instance of SomethingController so that it will be available down at
the LinkGenerator layer.
Containers are to be initialized at runtime and then used to resolve.
I need HttpRequestMessage to be regstered prior to creating each
instance of SomethingController so that it will be available down at
the LinkGenerator layer.
It sounds like you want to register an item with the container at runtime, post-startup. In general, this is not a good practice--registration should be a discrete event that happens when the app is fired up, and the container's state should not be changed during runtime.
Dependency Injection is about resolving service components, not runtime state--state is generally passed via methods (method injection). In this case it sounds like your LinkGenerator component needs access to the ambient state of the request.
I'm not that familiar with HttpRequestMessage, but this answer seems to show that it is possible to retreive it from HttpContext.Current. You could make this a method on your LinkGenerator class, or wrap this call in a separate component that gets injected into LinkGenerator (HttpRequestMessageProvider?). The latter would be my preferred method, as it allows LinkGenerator to be more testable.
Given the lack of a clean way of doing this and Web API not providing information as to the hosted endpoint beyond per-request context objects, I ended up injecting the base url from configuration.
Is this library by Mark Seemann the answer? In the description he writes explicitly :
This approach enables the use of Dependency Injection (DI) because the
request can be injected into the services which require it.
Then gives an example :
// Inside an ApiController
var uri = this.Url.GetLink(a=> a.GetById(1337));
By which you can then pass the URL down the road in the service that you have injected in the controller.
UPDATE :
Mark Seemann wrote about the same exact problem here:
"Because HttpRequestMessage provides the context you may need to
compose dependency graphs, the best extensibility point is the
extensibility point which provides an HttpRequestMessage every time a
graph should be composed. This extensibility point is the
IHttpControllerActivator interface:..."
This way you can pass request context information to a component deep in the object graph by getting from the HttpRequestMessage and passing it to the DI container.
Just take a look at the interface of IHttpControllerActivator.
The WEB API framework gets the IHttpControllerActivator through DependencyResolver. You probably already replaced it by your CastleWindsorDependencyResolver. Now you have to implement and register your HttpControllerActivator and register it.
When the WEB API framework gets IHttpControllerActivator from DependencyResolver (your Castle Windsor DR) and calls IHttpControllerActivator.Create() it will pass you the HttpRequestMessage. You can get your info from there and pass it to the your CastleDR before you call Resolve(typeof(MyController)) which will resolve the whole object graph - that means you will have MyHttpContextInfo to inject in your XYZComponent deep in the resolution stack.
This way tou are passing the arguments in the last possible moment but it is still possible. In Castle Windsor I make such passing of arguments though CreationContext.AdditionalArguments["myArgument"];.

Use javaJVMLocalObjectMimeType for local DnD and serialization for external drop

I'm working on an component in which smaller components are stacked. The user should be able to change the order of these components using drag and drop. I've made this work by implementing an TransferHandler that accepts a local reference DataFlavor (javaJVMLocalObjectMimeType) of the underlying data models. This works fine.
Now I would like to also run my application a second time and be able to drag on of my components from one application into the other. In this case I want to bundle the necessary data of the drag source into a serializable object to reconstruct the object at the drop application and use a serializable DataFlavor for that. I don't want to use object serialization in both situations.
How do I decide if my drag operation is originated in the same JVM so that I can decide to use the object reference or the serialized version of the data. The official swing DnD documentation mentions that it is possible to mix local and serialization flavors but it doesn't tell how to make best use of this.
Edit
Here's how I create the flavor in my DataModelTransferable
public static DataFlavor localFlavor;
static {
try {
localFlavor = new DataFlavor(DataFlavor.javaJVMLocalObjectMimeType + ";class=\"" + ArrayList.class.getName() + "\"");
} catch (ClassNotFoundException e) {
e.printStackTrace();
}
}
...
#Override
public DataFlavor[] getTransferDataFlavors() {
return new DataFlavor[] { localFlavor };
}
And in my TransferHandler I do this
#Override
public boolean canImport(TransferSupport support) {
return support.isDataFlavorSupported(DataModelTransferable.localFlavor);
}
As I said, this works fine locally, but if I drag from one to another instance the drag is accepted which leads to a java.io.IOException: Owner failed to convert data on the drop application and a java.io.NotSerializableException: alignment.model.DataModel on the drag source application. This is ok but the drag shouldn't be accepted on another app in the first place.
I'm using a ArrayList as I also want to be able to drag several objects at once, fyi.
I just found out myself what the problem was. I wrote a wrapper class DataModelListfor my ArrayList of DataModels that doesn't implement Serializable and modified the declaration of my data flavor to this:
localFlavor = new DataFlavor(DataFlavor.javaJVMLocalObjectMimeType + ";class=\"" + DataModelList.class.getName() + "\"");
After that, this flavor is not considered to be equal between the drag source and the drop target if they are not within the same JVM.
I conclude that it's not possible to use local object reference flavors directly with classes that implement Serializable. If someone knows where this is documented I would be glad to hear about it.
Your object reference flavor will normally be different for each running JVM. So you first check whether the Transferable supports your 'object reference flavor' before you ask for the 'serialized dataflavor version'.