Creating Spark ML object we just need to know:
The type of model
The parameters for the model
I am just brainstorming a way to pass this information using json and instantiate a Spark ML object from it.
For example, with this json
{
"model": RandomForestClassifier,
"numTrees": 10,
"featuresCol": "binaryFeatures"
}
It will instantiate a Random Forest model.
val rf = new RandomForestClassifier().setNumTrees(10).setFeaturesCol("binaryFeatures")
It is fairly straightforward to write a custom json serializer/deserializer by my own. Scala's pattern match seems good use case to dynamically instantiate an object from the name in string. However, when the object gets more complex (i.e. supporting pipeline), it is hard to maintain the custom serializer.
Is there any existing implementation for this? If not, what should the json structure look like?
Related
In django view I want to be able to serialize whole context, that is usually used to send to template (typically by calling render and passing locals).
I want to experiment with SPA+API and possibilities to go forward with and I'd like to create function, that would serialize locals to json and return it as json response.
Now problem is, that locals is typically mix of lists, dists and querysets of models.
I can serialize models using django.core.serializers or using django-rest-framework. I can serialize dict with primitive types using json library, but I don't know any simple way how to do mix of those.
Ideal would be way to go through locals dictionary and replace all found models with their serialized representations and then put it all together, maybe even specify before what serializer (in sense of drf) to use for which model. But I really don't want to reinvent wheel in case it already exists.
Anoher question is - is this even a good idea to try to do this? Return json context as alternative to server side rendering? I am in prototyping stage so I am still thinking of how to move forward and any input in the area is appreciated.
I would recommand to go with DRF
ModelSerializer will return a Json encoded array of model
Serializer with DictField will return a Json encoded dict
Serializer with ListField will return a Json encoded list
You can create Serializer with field is another Serializer for nesting purpose.
https://www.django-rest-framework.org/api-guide/fields/#composite-fields
https://www.django-rest-framework.org/api-guide/serializers/#dealing-with-nested-objects
For your' question is this a good idea, i would said :
If you push data to an external source (not django) it's fine
If you push data to django template it's a bad idea, you loose a lot of django power :(
When I naively use Jackson to convert to JSON i receive this exception:
Caused by: com.fasterxml.jackson.databind.exc.InvalidDefinitionException: No serializer found for class org.apache.cayenne.access.DefaultDataRowStoreFactory and no properties discovered to create BeanSerializer
Edit: I'd like to do something like this:
ObjectContext context = cayenneRuntime.newContext();
List<User> users = ObjectSelect.query(User.class).select(context);
JsonObject json = Json.mapper.convertValue(obj, Map.class)
Are there any existing solutions? Thanks
Considering that in a general case Cayenne gives you not just objects, but a virtual graph of objects, serialization to JSON becomes a more quirky topic than it initially appears.
The short answer: you'd have manually build JSON for whatever subgraph of your object graph.
While not a direct answer, it may be worth mentioning that Agrest framework (ex. LinkRest) supports rule-based serialization of Cayenne object graphs to JSON. But it is not a standalone component. I.e. it will only work if you use it for your REST services.
I need to parse the jax-ws rest response and I tried the following two ways of parsing the response.Both works good.But I am in need to know the best efficient way of implementation.Please provide me your view.
First Approach:
Use getEntity Object and get the response as Input Stream.
Using Jackson ObjectMapper readValue() -covert the inputstream to java
object.
Using getters and setters of nested java class get the response objects member values.
Second Approach:
Use getEntity Object and get the response as Input Stream and and
convert the Input Stream to String.
Using Google Json API,convert the string to json object.
Using Json parser and get the nested objects member values.
I would say the first approach is better for two reasons:
You don't go through the intermediate process of reading the response payload into String
The setter methods that are called during Jackson deserialization may perform validation on input and throw appropriate exceptions, so you do validation during deserialization.
Maybe not a general answer to this question but another variant of what you're describing under "First approach". I would start with a generic data structure and would only introduce an extra bean if necessary. I wouldn't use String to pass structured data around.
Use jackson to convert the JSON response to a
Map<String,Object> or JsonNode.
Advantage:
You don't need to implement a specialized bean class. Even a very simple bean can become unhandy over time (if format changes or new nested structures are added to the json response, etc.). It also introduces some kind of metaphor to your code which sometimes helps but also can be misleading.
Map<String,Object> is in the JDK and offers a good interface to access data. You don't have to change any interfaces even if the JSON format changes.
You can always pass your data in form of a Map<String,Object>
Disadvantage
Data Encapsulation. The map is a very close representation of the input data and therefore offers not same level of abstraction like a bean.
I want to parse JSON data from a RESTful service.
Unlike a SOAP-based service, where a service consumer can create stubs and skeleton from WSDL, in the case of the RESTful service, the service consumer gets a raw JSON string.
Since the service consumer does not have a Java object matching the JSON structure, we are not able to use the JSON to Java Mappers like GSON, Jackson etc.
One another way is to use parsers like JsonPath, minimal-json, etc which help traversing the JSON structure and read the data.
Is there any better way of reading JSON data?
The official docs for Jackson mention 3 different ways to parse a JSON doc from Java. The first 2 do not require "Java object matching the JSON structure". In Summary :
Streaming API (aka "Incremental parsing/generation") reads and writes JSON content as discrete events.
Tree Model provides a mutable in-memory tree representation of a JSON document. ObjectMapper can build trees that consist of JsonNode nodes.
Data Binding converts JSON to and from POJOs based either on property accessor conventions or annotations.
With simple data binding you convert to and from Java Maps, Lists, Strings, Numbers, Booleans and nulls
With full data binding you convert to and from any Java bean type (as well as "simple" types mentioned above)
Another option is to generate Java Beans from JSON documents. You mileage may vary and you may/probably will have to modify the generated files. There are at least 5 online tools for that purpose that you can try:
http://www.jsonschema2pojo.org/
http://pojo.sodhanalibrary.com/
https://timboudreau.com/blog/json/read
http://jsongen.byingtondesign.com/
http://json2java.azurewebsites.net/
There are also IDE plugins that you can use. For instance this one for Intellij https://plugins.jetbrains.com/idea/plugin/7678-jackson-generator-plugin
The GSON supports work without objects, too. Something as this:
JsonObject propertiesWrapper = new JsonParser().parse(responseContent).getAsJsonObject();
assertNotNull(propertiesWrapper);
propertiesWrapper = propertiesWrapper.getAsJsonObject("properties");
assertNotNull(propertiesWrapper);
JsonArray propertiesArray = propertiesWrapper.getAsJsonArray("property");
assertNotNull(propertiesArray);
assertTrue(propertiesArray.size()>0, "The list of properties should not be empty. ");
The problem is that the work this way is so inconvenient that it is really better to create objects instead.
Jackson has absolutely the same problems, and to greater extent - extremal inconvenient for direct json reading/creation. All its tutorials advice to use POJOs instead, too.
The only really convenient way is use Groovy. Groovy works as an envelope on Java, you can simply write Java code and use Groovy operators at need. And in JSON or XML reading and creation Groovy is incomparably more powerful that Java with all its libraries multiplied on each other! It is even much more convenient than already prepared by somebody else tree structure of ready POJOs.
We're in a process of switching from Json.NET to ServiceStack.Text and I came across an issue with serialization of polymorphic collections.
In JSON.NET I used to create a custom JsonCreationConverter and overriding the Create() method to select the type I want to create - as outlined here:
http://dotnetbyexample.blogspot.co.uk/2012/02/json-deserialization-with-jsonnet-class.html
The collection in question is List<ItemBase> which can contain FlightItem or CarHireItem objects.
This is my version of the Create() method for JSON.NET:
protected override ItemBase Create(Type objectType, JObject jsonObject)
{
var itemType = jsonObject["Type"].ToString();
switch (itemType)
{
case "flight":
return new FlightItem();
case "carhire":
return new CarHireItem();
default:
return null;
}
}
Is that possible with ServiceStack?
Serialization and deserialization in ServiceStack for polymorphic collections works, however, it appends the object type to the JSON output, e.g.
"__type" : "TestProject.Model.FlightItem, TestProject"
This means that I need to supply the type when posting JSON and I'm not too keen on having the .NET type visible for anyone within the API calls.
Any suggestions? If it's possible to do this in a different way, can you point me to some examples?
Firstly, Interfaces or abstract types in DTOs are a bad idea.
You're now in the strange position where you're trying to support polymorphic types in DTOs, but don't want to provide JSON Serializer-specific info? The reason why you need bespoke code to support polymorphic DTO's is because it's impossible to tell what concrete type should be used based on the wire format, hence why ServiceStack emits the __type property for this purpose.
To avoid these hacks, and have it work equally well in all JSON Serializers, you're better off "flattening" your polymorphic types into a single "flat" DTO and send that across the wire instead. Once you're back in C# you can use code to project it into the ideal types.
ServiceStack does provide some JsConfig<ItemBase>.RawDeserializeFn that will let you do something similar, see the CustomSerializerTests for an example. There's also the JsConfig<ItemBase>.OnDeserializedFn hook that can potentially help, but it's dependent on whether ItemBase contains a complete property list of both concrete types.