Given an AVRO schema, I create a JSON string which conforms to this schema.
How can I serialize the JSON string using AVRO to pass it to a Kafka producer which expects an AVRO-encoded message?
All examples I find don't have JSON as input.
BTW, the receiver will then deserialize the message to a POJO - we are working in different tech stacks. So, basically it's JSON -> Kafka -> POJO.
All examples I find don't have JSON as input
Unclear what examples you've found, but kafka-avro-console-producer does exactly what you want (assuming you're using the Confluent Schema Registry).
Otherwise, you want a jsonDecoder, which is what Confluent uses
can Kafka connect value conveter (JSONConverter) can be used to convert GPB ?
I am using kafka connect as sink and writing all topic messages (GPB) into database
I am using default JSONConverter as valueconverter in value.converter field in property file, can this be used to convert GPB object ?
If not can I use the deserializer class used to this deserialize this object or I need to write some other custom class? could you please share some example of the same
No, the JSONConverter expects strictly-formatted JSON, Protocol Buffers (I assume that's what you mean by GPB?) are binary records and need an appropriate converter.
Fortunately the community has one available here: https://github.com/blueapron/kafka-connect-protobuf-converter/blob/master/README.md
I am writing a REST Client for one of the Vendor REST Service. I use jersey 2.x and JSON-P, below are dependencies I add.
<dependencies>
<dependency>
<groupId>org.glassfish.jersey.media</groupId>
<artifactId>jersey-media-json-processing</artifactId>
<version>2.26</version>
</dependency>
<dependency>
<groupId>org.glassfish.jersey.core</groupId>
<artifactId>jersey-client</artifactId>
<version>2.26</version>
</dependency>
</dependencies>
I successfully write code for GET request and received JSON output. I saved it to a file and used JSON-P to interpret and do my own logic without any issues.
But now I need to write a POST request. When I use CURL as below I am able do it. and want to implement the same using Jeresey 2.x and JSON-P.
curl -v -H "Accept:application/json" -H "Content-Type:application/json" -u user:password -X POST --databinary #./InputParameters.json https://<IP>:<PORT>/Configuration/server
InputParameters.json contents
{
"ip": "10.197.70.16",
"partNumber": 202067,
"model": "IBM P7"
}
When I tried to pass response body as String in JSON format ({"ip": "10.197.70.16", "partNumber": 202067, "model": "IBM P7"}), but didn't work. So tried as JsonObject as below still didn't work.
JsonObject jsonObj = Json.createObjectBuilder()
.add("ip", "10.197.70.16")
.add("partNumber", 202067)
.add("model", "IBM P7")
.build();
response = invocationBuilder.post(Entity.json(jsonObj));
I know core java, based on that experience I jumped into writing this program and got success with GET but not POST. I doubt I am doing something fundamentally wrong with POST.
Let's unpack what you're doing for a bit. First there's this part:
JsonObject jsonObj = Json.createObjectBuilder()
.add("ip", "10.197.70.16")
.add("partNumber", 202067)
.add("model", "IBM P7")
.build();
This creates a javax.json.JsonObject instance. JsonObject, which is part of the JSON-P Java API, is pretty much what it says: a class to represent a JSON object. A JsonObject instance contains a hierarchy of javax.json.JsonValue instances, which conform to more specific types like JsonArray, JsonString, other JsonObjects and so on. In this regard it's not unlike the classes of the DOM API for representing XML documents (let's hope Oracle keeps the API docs at that URL for a while). But JSON is fortunately a lot more straightforward than XML and DOM.
Your jsonObj instance would contain a JsonString instance with value "10.197.70.16" mapped to name "ip", a JsonNumber with value 202067 (probably represented as BigDecimal) mapped to name "partNumber" and so on.
Next your code executes this:
Entity.json(jsonObj)
javax.ws.rs.client.Entity.json(something) basically states that you want to create an entity that will provide the payload for a JAX-RS client invocation with as Content-Type application/json. In other words, the something you create it for must be transformed to a JSON representation when it's sent to the API, which should expect a JSON payload and know how to handle it. Note that Entity.json(...) has a generic type parameter. The method signature is static <T> Entity<t> json(T entity). So you're creating an instance of Entity<JsonObject> with the payload entity jsonObj.
When this is handed over to the post method of a javax.ws.rs.client.Invocation.Builder instance (the post method is actually defined in its parent interface SyncInvoker) the client implementation goes to work.
response = invocationBuilder.post(Entity.json(jsonObj));
It takes the provided Entity instance and its content (our jsonObj), checks what the desired output is of the Entity (this is application/json) and seeks a provider that can turn objects of the given type into that output. In other words, some component must be located that can be given a javax.json.JsonObject and write a representation of it as JSON to an OutputStream. The component handling this could be a javax.ws.rs.ext.MessageBodyWriter that claims it can perform this transformation and was registered to the JAX-RS runtime. Various libraries supply such providers and you can also write your own. This makes JAX-RS extensible to deal with various scenarios, handle non-standard input and output or lets you tune its behaviour. When multiple providers are capable of handling the given entity type and producing the desired output, there are rules to determine which one takes on the job. Note that this can depend on what is on your classpath. There are ways of forcing this explicitly.
The client puts together the invocation through its configuration, using the proper URL, query parameters, HTTP method, headers and so on. The payload is created by writing the entity to an OutputStream in the required format. In your example this results in a POST to the server. When the invocation has been completed you receive a javax.ws.rs.core.Response that you can use to determine the HTTP result code and retrieve a response payload, if any. The readEntity(Class<T> entityType) method of Response works like the reverse of turning an Entity into a payload. It searches for a MessageBodyReader that can interpret the response stream according to the value returned from response.getMediaType() and can create an instance of Class entityType from it.
So with all of that explained, what exactly is going wrong in your approach? Well, the issue is that the default implementations available to your JAX-RS runtime probably don't have a writer specifically for an input of type JsonObject and with expected output application/json. It may seem very logical if the server expects JSON, that you should be able to supply a JsonObject as payload. But if the JAX-RS implementation can't find something to handle that class, then at best it can just use some default approach. In that case it may try to interpret the object as a POJO and serialize it to JSON in a default manner, which could lead to weirdness like this:
{
"valueMap": {
"ip": {
"value": "10.197.70.16"
},
"partNumber": {
"num": 202067,
"integral": TRUE
},
...
}
}
That's what a literal interpretation of the JsonObject instance could look like, depending on which implementation it uses and what is used by JAX-RS to turn it into JSON output. Of course it's possible that the object can't be serialized to JSON at all, either because no suitable MessageBodyWriter can be found or it runs into an error when creating the output.
A first solution would be a very simple one. Just turn the JsonObject into a String and simply provide that as the entity:
StringWriter stringWriter = new StringWriter();
try (JsonWriter jsonWriter = Json.createWriter(stringWriter);) {
jsonWriter.writeObject(jsonObject);
} // some error handling would be needed here
String jsonPayload = stringWriter.toString();
response = invocationBuilder.post(Entity.json(jsonPayload));
It seems you had already tried that. A possible problem with this is that the MessageBodyWriter that gets used needs to just output the String's bytes in a suitable encoding (probably UTF-8) when presented with a String as output and application/json as the required content type. Some may not do that. You could try Entity.entity(jsonPayload, MediaType.TEXT_PLAIN_TYPE.withCharset("UTF-8")) but then the server might reject the call.
Other possible solutions are
Writing your own MessageBodyWriter for String objects with an #javax.ws.rs.Produces(MediaType.APPLICATION_JSON) annotation on it.
Better yet, writing such a class that accepts JsonObject instances.
Creating POJO classes for your JSON structure and letting those get used for generating JSON from instances or deserializing JSON responses to instances.
Finding an extension library that contains suitable providers for dealing with javax.json classes.
The addition of the com.owlike:genson dependency to your project is exactly the application of that last suggestion. The Genson library provides conversions between Java objects and JSON in both directions, as well as data binding. Part of its code base is dedicated to JAX-RS providers, among which a class suitable for accepting JsonObject as input.
Issue is resolved after adding below dependency. At this point I am not sure on what does it do.
Thanks to Swamy (TCS) for his support to resolve this.
<dependency>
<groupId>com.owlike</groupId>
<artifactId>genson</artifactId>
<version>1.4</version>
</dependency>
Example using genson
String serialize = new Genson().serialize(
Json.createObjectBuilder()
.add("ip", "10.197.70.16")
.add("partNumber", 202067)
.add("model", "IBM P7")
.build()
);
response = invocationBuilder.post(Entity.json(serialize));
I'm looking for a way to generate a set of Angular2 form templates from a Swagger API definition file. I want a result that will allow me to test my POST/PUT requests, and even use it in my app.
After some research I found this Angular2 form library that takes a JSON schema as input: https://github.com/makinacorpus/angular2-schema-form
So if you know of a Swagger -> JSON Schema converter that will work too.
Cheers!
So if you know of a Swagger -> JSON Schema converter that will work
too.
Swagger 2.0 supports a subset of JSON schema draft 4. This is what swagger's Schema object is. From the docs:
The following properties are taken directly from the JSON Schema
definition and follow the same specifications:
$ref - As a JSON Reference
format (See Data Type Formats for further details)
title
description (GFM syntax can be used for rich text representation)
default (Unlike JSON Schema, the value MUST conform to the defined type for the Schema Object)
multipleOf
...
The following properties are taken from the JSON Schema definition but
their definitions were adjusted to the Swagger Specification.
items
allOf
properties
additionalProperties
It should be a fairly simple exercise to manually extract the schema from your swagger, but I don't know of any automated tool to do this. I think the fact some of the JSON schema properties have been modified by swagger may make auto conversion problematic in certain circumstances.
I have a schema defined in yaml format. At application startup, I just need to verify that this schema has not been changed. So for that, in code I need to write a bean and validate that schema against that bean. Is this possible? It may sound strange because usually payloads are validated against schema, but I need to validate schema against a bean. Any JAVA based solution?
I was looking at Jackson-module-jsonSchema but it generates Schema from a bean, not validate a bean against a schema AND its json not yaml.
P.S. I can convert yaml to json if need be (if an api is available to do this in json)