I'm new to Spring development and am currently looking for best practices for creating objects based on JSON data passed to a rest service, which are then to be stored using JPA.
As an example, let's assume a simplified file upload service where clients can post the following JSON object to:
{"filename":"myfile.pdf", "data":someByteArray, "extension":"pdf"}
On the server, I don't want to store the file's raw data in the database, but upload it to some cloud storage and just store a link to it. So once the "data" parameter content is stored and the link is received, it is obsolete.
The options I've come up with is:
a) define one 'union' class to deserialize the request body that comprises all fields ("filename", "link", "extension" AND "data"), marking 'intermediate' data as #Transient, so it won't be stored in the database,
b) define two classes: one solely for deserializing the request body ("filename", "data", "extension") which is then used to construct the desired entity class ("filename", "link", "extension") that is to be stored,
c) define a custom JsonDeserializer that parses the request body and constructs the entity class ("filename", "link", "extension").
While all of the approaches will probably work, in a more complex scenario, option a) will clutter the entity's class with members that are only required for deserializing once, while option b) and c) may result in a lot of additional classes.
Is there any best practice for this? Btw, thanks for reading this far :)
Related
I have a producer that writes a json file to the topic to be read by a kafka consumer stream. Its simple key-value pair.
I want to stream the topic and enrich the event by adding/concatenating more JSON key-value rows and publish to another topic.
None of the values or keys have anything in common by the way.
I am probably overthinking this, but how would I get around this logic?
I suppose you want to decode JSON message at the consumer side.
If you are not concerned about schema and but just want to deal with JSON as a Map, you can use Jackson library to read the JSON string as a Map<String,Object>. For this you can add the fields that you want, convert it back to a JSON string and push it to the new topic.
If you want to have a schema, you need to store the information as to which class it is mapping to or the JSON schema or some id that maps to this, then the following could work.
Store the schema info in headers
For example, you can store the JSON schema or Java class name in the headers of the message while producing and write a deserializer to extract that information from the headers and decode it.
The Deserializer#deserialize() has the Headers argument.
default T deserialize(java.lang.String topic,
Headers headers,
byte[] data)
and you can do something like..
objectMapper.readValue(data,
new Class.forName(
new String(headers.lastHeader("classname").value()
))
Use schema registry
Apart from these, there is also a schema registry from Confluent which can maintain different versions of the schema. You would need to run another process for that, though. If you are going to use this, you may want to look at the subject naming strategy and set it to RecordNameStrategy since you have multiple schemas in the same topic.
I have to implement REST endpoints to update a resource.
I will use the methods PUT and PATCH (the latter used to send a json that has only the attributes to modify).
The payload of the calls will be a json that is parsed by Jackson.
Using custom deserializers and converter,s the parser will create the correct instances of java beans.
I know that usually these endpoints will have in their URL the ID of the resource to update.
For the PATCH endpoint, I would prefer to send the ID of the resource in the json, together with all the other resources.
I can see PROS and CONS.
CONS: the URL looks like something that is updating the collection of resources and not a single resource,
PROS: the json contains all the necessary information and the parser can find the resource in the database and add the information that wasn't sent with the request.
This would simplify the code because the parser will return an object ready to use.
As you pointed out, it would be better to pass the ID in the URL.
One of the main constraint of a REST API is the "Resource identification in requests": it should be possible to easily identify the resource modified looking the call.
It is not enough that the representation passed has the information necessary to identify the resource: it is also important that the identification is easy and does not require the knowlodege of internal details.
I will give an example to better explain my question
my request:
POST
http://localhost:8080/users/
request body: (This gets posted)
{"name":"Matt",
"salary":10000,
"blog_url":"www.myblog.com",
"dept_name":"ENG"
}
class CustomRequest(object):
def __init__(self,name,salary,blog_url,dept_name):
self.name=name
self.salary=10000
self.blog_url=blog_url
self.dept_name=dept_name
models.py
class myUser(models.Model):
//fields -- username, salary
class myUserProfile(models.Model):
User=models.OneToOneField(user)
blog_url=models.URLfield()
dept_name=models.ForeignKey(Department)
#apiview(['POST'])
def createUser(customrequest):
myuser=user(customrequest.name, customrequest.salary)
myuser.save()
myuser.userprofile.blog_url(customrequest.blog_url)
myuser.userprofile.dept_name(customrequest.dept_name)
myuser.save()
I have been most of REST services using Java JAX-RS API. In this framework,
POST request body is automatically deserialized to the object that the method takes in( in the above example, it is customrequest). A developer can define an object with attributes that he is looking for in the POST request and then perform the business logic.
Now that we are thinking of migrating to Django, I am wondering if Django Rest Framework provides this kind of behavior out of box. If so, how would I do that?
Please note, in the JAX-RS world, there is no need for a developer to write a serializer. All that is needed is the transfer object where the incoming JSON gets deserailzed into.
I assume in Django, both a serializer and a transfer object is needed to achieve the same purpose.
In DRF, you have two options:
either you use ModelSerializer and you get the model instance automatically
or you use Serializer and you get the validated_data and do whatever you like with it
The serializer is the same as what you call transfer object, in the sense that both define the data structure and that both will hold the deserialized values.
For models, it is required for persistence, but that is also required in JAX-RS, unless you use the same classes as ORM entities(which is a bad design). So in JAX-RS you will have, for example, CustomRequest and JPA CustomRequestEntity
I'm developing a RESTful interface which is used to provide JSON data for a JavaScript application.
On the server side I use Grails 1.3.7 and use GORM Domain Objects for persistence. I implemented a custom JSON Marshaller to support marshalling the nested domain objects
Here are sample domain objects:
class SampleDomain {
static mapping = { nest2 cascade: 'all' }
String someString
SampleDomainNested nest2
}
and
class SampleDomainNested {
String someField
}
The SampleDomain resource is published under the URL /rs/sample/ so /rs/sample/1 points to the SampleDomain object with ID 1
When I render the resource using my custom json marshaller (GET on /rs/sample/1), I get the following data:
{
"someString" : "somevalue1",
"nest2" : {
"someField" : "someothervalue"
}
}
which is exactly what I want.
Now comes the problem: I try to send the same data to the resource /rs/sample/1 via PUT.
To bind the json data to the Domain Object, the controller handling the request calls def domain = SampleDomain.get(id) and domain.properties = data where data is the unmarshalled object.
The binding for the "someString" field is working just fine, but the nested object is not populated using the nested data so I get an error that the property "nest2" is null, which is not allowed.
I already tried implementing a custom PropertyEditorSupport as well as a StructuredPropertyEditor and register the editor for the class.
Strangely, the editor only gets called when I supply non-nested values. So when I send the following to the server via PUT (which doesn't make any sense ;) )
{
"someString" : "somevalue1",
"nest2" : "test"
}
at least the property editor gets called.
I looked at the code of the GrailsDataBinder. I found out that setting properties of an association seems to work by specifying the path of the association instead of providing a map, so the following works as well:
{
"someString" : "somevalue1",
"nest2.somefield" : "someothervalue"
}
but this doesn't help me since I don't want to implement a custom JavaScript to JSON object serializer.
Is it possible to use Grails data binding using nested maps? Or do I really heave to implement that by hand for each domain class?
Thanks a lot,
Martin
Since this question got upvoted several times I would like to share what I did in the end:
Since I had some more requirements to be implemented like security etc. I implemented a service layer which hides the domain objects from the controllers. I introduced a "dynamic DTO layer" which translates Domain Objects to Groovy Maps which can be serialized easily using the standard serializers and which implements the updates manually. All the semi-automatic/meta-programming/command pattern/... based solutions I tried to implement failed at some point, mostly resulting in strange GORM errors or a lot of configuration code (and a lot of frustration). The update and serialization methods for the DTOs are fairly straightforward and could be implemented very quickly. It does not introduce a lot of duplicate code as well since you have to specify how your domain objects are serialized anyway if you don't want to publish your internal domain object structure. Maybe it's not the most elegant solution but it was the only solution which really worked for me. It also allows me to implement batch updates since the update logic is not connected to the http requests any more.
However I must say that I don't think that grails is the appropriate tech stack best suited for this kind of application, since it makes your application very heavy-weight and inflexbile. My experience is that once you start doing things which are not supported by the framework by default, it starts getting messy. Furthermore, I don't like the fact that the "repository" layer in grails essentially only exists as a part of the domain objects which introduced a lot of problems and resulted in several "proxy services" emulating a repository layer. If you start building an application using a json rest interface, I would suggest to either go for a very light-weight technology like node.js or, if you want to/have to stick to a java based stack, use standard spring framework + spring mvc + spring data with a nice and clean dto layer (this is what I've migrated to and it works like a charm). You don't have to write a lot of boilerplate code and you are completely in control of what's actually happening. Furthermore you get strong typing which increases developer productivity as well as maintainability and which legitimates the additional LOCs. And of course strong typing means strong tooling!
I started writing a blog entry describing the architecture I came up with (with a sample project of course), however I don't have a lot of time right now to finish it. When it's done I'm going to link to it here for reference.
Hope this can serve as inspiration for people experiencing similar problems.
Cheers!
It requires you to provide teh class name:
{ class:"SampleDomain", someString: "abc",
nest2: { class: "SampleDomainNested", someField:"def" }
}
I know, it requires different input that the output it produces.
As I mentioned in the comment earlier, you might be better off using the gson library.
Not sure why you wrote your own json marshaller, with xstream around.
See http://x-stream.github.io/json-tutorial.html
We have been very happy with xstream for our back end (grails based) services and this way you can render marshall in xml or json, or override the default marshalling for a specific object if you like.
Jettison seems to produce a more compact less human readable JSON and you can run into some library collision stuff, but the default internal json stream renderer is decent.
If you are going to publish the service to the public, you will want to take the time to return appropriate HTTP protocol responses for errors etc... ($.02)
How do I use JSON in java servlet? Using the URL, to pass parameters from a POST servlet, the JSON would respond based on the URL parameters.
You need a JSON library in Java. With that library, you will be able to serialize Java Objects into JSON objects, and send them through HttpServletResponse instance in your Servlet.
"How do you use it"?
JSON is simply one way of representing hierarchical data (such an an object model) in a flat text format (such as an HTTP body, or filesystem file). So you'd use it to represent hierarchical data in these situations.
If you mean how do you parse/create it, there are many mature libraries for JSON handling.
Perhaps a specific situation or use case would assist in pinning down your question, if the above is not the answer you were looking for.