I created a kafka consumer service using spring boot. I need to get the message from the kafka and assign it to the java object and do some calculations according to the data in the object and store it in MongoDB database.
Here is my kafka listner.
#KafkaListener(topics = "neworder", groupId = "group_json",
containerFactory = "orderKafkaListenerFactory")
public void consumeJson(List<Order> order) {
System.out.println("Consumed JSON Message: " + order);
}
Here is the received kafka message.
Consumed JSON Message: [{Id=11, FualType=Petrol 92, Capacity=33000}]
Here is my Java class.
import org.springframework.data.annotation.Id;
import org.springframework.data.mongodb.core.mapping.Document;
#Document("Order")
public class Order {
#Id
private int id;
private String type;
private int capacity;
public Order() {
}
public Order(int id, String type, int capacity) {
this.id = id;
this.type = type;
this.capacity = capacity;
}
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public String getType() {
return type;
}
public void setType(String type) {
this.type = type;
}
public int getCapacity() {
return capacity;
}
public void setCapacity(int capacity) {
this.capacity = capacity;
}
}
I need to assign the received message data from kafka to this java class.
Anyone can help me?
I assume you are getting some errors at the moment. It would help if you could add them to your question.
First, since the fields in the json of the Kafka message do not match those of your java class, you have to annotate them:
#Id
#JsonProperty("Id")
private int id;
#JsonProperty("FualType")
private String type;
#JsonProperty("Capacity")
private int capacity;
In terms of configuration of the consumer, normally it is enough to configure the JsonDeserializer for the value of the kafka message and use it in the ConsumerFactory:
#Bean
public ConsumerFactory<String, List<Order>> consumerFactory() {
return new DefaultKafkaConsumerFactory<>(consumerConfigs());
}
and
#Bean
public Map<String, Object> consumerConfigs() {
Map<String, Object> props = new HashMap<>();
...
props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, ErrorHandlingDeserializer.class);
props.put(ErrorHandlingDeserializer.VALUE_DESERIALIZER_CLASS, JsonDeserializer.class.getName());
...
return props;
}
Related
I am using SpringBoot and trying to deserialize JSON like:
{
"userId": "Dave",
"queryResults": {
"id": "ABC",
"carData": {.....},
"carId": "Honda",
"status": 0,
"model": "X"
}
}
, into MyRequestModel clas:
public class MyRequestModel {
private String userId;
private String: queryResults;
}
, that is received as #RequestBody parameter in my #PostMapping method that looks like:
#PostMapping
public String postDate(#RequestBody MyRequestModel data) {
...
return "posted";
}
The above queryResults field is supposed to be stored as a CLOB in a database.
Problem I am having is that if I send this JSON to hit my endpoint (PostMapping) method, it cannot deserialize it into MyRequestModel and I get this error:
Cannot deserialize instance of java.lang.String out of START_OBJECT token
at [Source: (PushbackInputStream); line: 3, column: 18] (through reference chain: MyRequestModel["queryResults"])]
I guess the real answer to your question is: if you NEED the queryResults property to be a String, then implement a custom deserializer.
If not, then, use one of the alternatives that Jonatan and Montaser proposed in the other answers.
Implementing a custom deserializer within Spring Boot is fairly straightforward, since Jackson is its default serializer / deserializer and it provides a easy way to write our own deserializer.
First, create a class that implements the StdDeserializer<T>:
MyRequestModelDeserializer.java
public class MyRequestModelDeserializer extends StdDeserializer<MyRequestModel> {
public MyRequestModelDeserializer() {
this(null);
}
public MyRequestModelDeserializer(Class<?> vc) {
super(vc);
}
#Override
public MyRequestModel deserialize(JsonParser p, DeserializationContext ctxt)
throws IOException, JsonProcessingException {
JsonNode node = p.getCodec().readTree(p);
String userId = node.get("userId").asText();
String queryResults = node.get("queryResults").toString();
MyRequestModel model = new MyRequestModel();
model.setQueryResults(queryResults);
model.setUserId(userId);
return model;
}
}
Second, mark your class to be deserialized using your custom deserializer by using the #JsonDeserialize annotation:
MyRequestModel.java
#JsonDeserialize(using = MyRequestModelDeserializer.class)
public class MyRequestModel {
private String userId;
private String queryResults;
}
It's done.
queryResults is a String on Java side but it is an Object on JSON side.
You will be able to deserialize it if you send it in as a String:
{
"userId": "Dave",
"queryResults": "foo"
}
or if you create classes that maps to the fields:
public class MyRequestModel {
private String userId;
private QueryResults queryResults;
}
public class QueryResults {
private String id;
private CarData carData;
private String carId;
private Integer status;
private String model;
}
or if you serialize it into something generic (not recommended):
public class MyRequestModel {
private String userId;
private Object queryResults;
}
public class MyRequestModel {
private String userId;
private Map<String, Object> queryResults;
}
public class MyRequestModel {
private String userId;
private JsonNode queryResults;
}
You have two options to deserialize this request:-
change the type of queryResults to Map<String, Object>, it will accepts everything as an object of key and value. (Not recommended)
public class MyRequestModel {
private String userId;
private Map<String, Object> queryResults;
}
You have to create a class that wraps the results of queryResults as an object.
class QueryResult {
private String id;
private Map<String, Object> carData;
private String carId;
private Integer status;
private String model;
public QueryResult() {}
public QueryResult(String id, Map<String, Object> carData, String carId, Integer status, String model) {
this.id = id;
this.carData = carData;
this.carId = carId;
this.status = status;
this.model = model;
}
public String getId() {
return id;
}
public void setId(String id) {
this.id = id;
}
public Map<String, Object> getCarData() {
return carData;
}
public void setCarData(Map<String, Object> carData) {
this.carData = carData;
}
public String getCarId() {
return carId;
}
public void setCarId(String carId) {
this.carId = carId;
}
public Integer getStatus() {
return status;
}
public void setStatus(Integer status) {
this.status = status;
}
public String getModel() {
return model;
}
public void setModel(String model) {
this.model = model;
}
}
and make the type of queryResult as shown:-
public class MyRequestModel {
private String userId;
private QueryResult queryResults;
}
I am new to springboot, i am getting a response as below in my json response:
"Number": "08002050"
I have defined it as String in my spring boot app.
I want to get a response as below:
"Number": 08002050
How do i accomplish this. please help
You can manage it in server side with a tricky way.
public class User {
private int id;
private String name;
#JsonIgnore // ignore this field when serialize
private String number;
#JsonProperty(value = "number") // change name of field when serialize
private int intValueOfNumber;
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public String getNumber() {
return number;
}
public void setNumber(String number) {
this.number = number;
}
public int getIntValueOfNumber() {
return Integer.parseInt(number); // parse number string to int
}
public void setIntValueOfNumber(int intValueOfNumber) {
this.intValueOfNumber = intValueOfNumber;
}
}
In this entity #JsonIgnore annotation is ignore your field for JSON serialization and pass intValueOfNumber as int to JSON. Your json will be following:
{"id":1,"name":"Java","number":44124}
You may lost zero suffix of number string when you parse it to int.
I'm using JsonNode for getting data from any kind of jason format and storing it to mongoDb
But while fetching data from mongoDB it is throwing error as below.
Failed to instantiate com.fasterxml.jackson.databind.node.ObjectNode using constructor NO_CONSTRUCTOR with arguments
Below is my domain class
public class Profiler {
#Id
private String id;
#Field("email")
private String email;
#Field("profiler")
private Map<String,JsonNode> profiler;
public String getEmail() {
return email;
}
public void setEmail(String email) {
this.email = email;
}
public Map<String, JsonNode> getProfiler() {
return profiler;
}
public void setProfiler(Map<String, JsonNode> profiler) {
this.profiler = profiler;
}
public Profiler(String email,Map<String,JsonNode> profiler){
this.email=email;
this.profiler = profiler;
}
#JsonCreator
public Profiler(#JsonProperty("_id")String id,#JsonProperty("email")String email,#JsonProperty("profiler")Map<String,JsonNode> profiler){
this.id=id;
this.email=email;
this.profiler = profiler;
}
public Profiler(String id){
this.id=id;
}
public Profiler(Map<String,JsonNode> profiler){
this.profiler = profiler;
}
public Profiler(){
}
}
public interface ProfilerRepository extends MongoRepository<Profiler, String>{
public Profiler findOneByEmail(String email);
}
And my controller call is as below and I'm getting the error on this line.
Profiler profile=profileService.findOneByEmail(email);
I have made this changes and work as expected.
Map<String, Object> profile;
This problem occurs because com.fasterxml.jackson.databind.node.ObjectNode class doesn't have default constructor (no argument constructor) and Jackson expects the default constructor.
Related Post refer azerafati's answer
The problem can be resolved if you define the profiler field as static in domain class.
private static Map<String, JsonNode> profiler;
Please note that static fields have its own limitations and issues. I can assure that this would resolve the above exception. However, it may not be the most appropriate solution.
in my case problem solved . i had entity that i defined :
private JsonNode data;
i changed it to:
private Map<String,String> data;
or this also work :
private Map<Object,String> data;
please let me know if you had any question
I would like to unmarshal a json string to a pojo class.
I am reading it from an existing url:
https://builds.apache.org/job/Accumulo-1.5/api/json
I am using apache camel to unmarshal the url
#Component
public class RouteBuilder extends SpringRouteBuilder {
private Logger logger = LoggerFactory.getLogger(RouteBuilder.class);
#Override
public void configure() throws Exception {
logger.info("Configuring route");
//Properties die hij niet vindt in de klasse negeren
ObjectMapper objectMapper = new ObjectMapper();
objectMapper.configure(DeserializationConfig.Feature.FAIL_ON_UNKNOWN_PROPERTIES, false);
DataFormat reportFormat = new JacksonDataFormat(objectMapper, HealthReport.class);
from("timer://foo?fixedRate=true&delay=0&period=2000&repeatCount=1")
.routeId("accumoloToJsonRoute")
.setHeader(Exchange.HTTP_METHOD, constant("GET"))
.to("https://builds.apache.org:443/job/Accumulo-1.5/api/json")
.convertBodyTo(String.class)
.unmarshal(reportFormat) //instance van Build
.log(LoggingLevel.DEBUG, "be.kdg.teamf", "Project: ${body}")
.to("hibernate:be.kdg.teamf.model.HealthReport");
}
}
So far so good. I would like to only insert the 'healthReport' node using hibernate annotations.
#XmlRootElement(name = "healthReport")
#JsonRootName(value = "healthReport")
#Entity(name = "healthreport")
public class HealthReport implements Serializable {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private int Id;
#Column
#JsonProperty("description")
private String description;
#Column
#JsonProperty("iconUrl")
private String iconUrl;
#Column
#JsonProperty("score")
private int score;
public HealthReport() {
}
public HealthReport(int score, String iconUrl, String description) {
this.score = score;
this.iconUrl = iconUrl;
this.description = description;
}
public String getDescription() {
return description;
}
public String getIconUrl() {
return iconUrl;
}
public int getId() {
return Id;
}
public int getScore() {
return score;
}
public void setDescription(String description) {
this.description = description;
}
public void setIconUrl(String iconUrl) {
this.iconUrl = iconUrl;
}
public void setId(int id) {
Id = id;
}
public void setScore(int score) {
this.score = score;
}
}
This is where the problem is. It does not recognize the annotations
and only null values are inserted in my database
#XmlRootElement(name = "healthReport")
#JsonRootName(value = "healthReport")
Does anybody know how to fix this?
Thanks
Fixed it using a Processor for my Route
public class HealthReportProcessor implements Processor {
#Autowired
private ConfigurationService configurationService;
#Override
public void process(Exchange exchange) throws Exception {
ObjectMapper mapper = new ObjectMapper();
JsonNode root = mapper.readTree(exchange.getIn().getBody().toString());
ArrayNode report = (ArrayNode) root.get("healthReport");
int configId = configurationService.findJenkinsConfigurationByName(root.get("displayName").asText()).getId();
for (JsonNode node : report) {
JsonObject obj = new JsonObject();
obj.addProperty("description", node.get("description").asText());
obj.addProperty("iconUrl", node.get("iconUrl").asText());
obj.addProperty("score", node.get("score").asInt());
obj.addProperty("jenkinsConfig", configId);
exchange.getIn().setBody(obj.toString());
}
}
}
It is working but I think there is a better solution.
If you have a better solution please let me know ;)
Can you try this,
from("timer://foo?fixedRate=true&delay=0&period=2000&repeatCount=1")
.routeId("accumoloToJsonRoute")
.setHeader(Exchange.HTTP_METHOD,constant("GET"))
.to("https://builds.apache.org:443/job/Accumulo-1.5/apijson")
.unmarshal().json(JsonLibrary.Jackson, HealthReport.class)
And make sure the response params match the POJO fields.
Let me know if it works.
I have a simple jersey web service and I'd like to consume / produce objects that contain map fields, like
#XmlElement
private Map<String,String> properties;
if this string goes into the web service,
{ properties: { key1: val1, key2: val2 )}
the properties field is deserialized as null with no errors. the same JSON goes in and out of GSON no problems, and in the short term I solved this by having jersey consume produce strings and using GSON to serialize / deserialize the JSON.
any ideas?
One option is to use annotated classes. So for instance a user might be represented by the following data.
import javax.xml.bind.annotation.XmlRootElement;
#XmlRootElement(name = "user")
public class User {
private int uid;
public int user_id;
public String user_name;
public String email;
public URI image_url;
public List<User> friends;
public boolean admin;
public User() {
...
}
public User(final int userid) {
// Find user by id
}
}
If you return the User object as in the following piece of code, then jaxb will automatically serialize the List as a JSON list etc etc....
#GET
#Path("/{userid}")
#Produces("application/json", "application/xml")
public User showUser(#PathParam("userid") final int userid) {
return new User(userid);
}
Jersey uses JAXB for serialization. JAXB can not serialize a Map as there is no XML type for Java type Map. Also, Map is an interface and JAXB does not like interfaces.
If you are using JAXBJackson bridge to marshal, you will run into issue.
You will need to create an adapter like below and annotate your Map property with
#XmlJavaTypeAdapter(MapAdapter.class)
private Map<String,String> properties;
#XmlSeeAlso({ Adapter.class, MapElement.class })
public class MapAdapter<K,V> extends XmlAdapter<Adapter<K,V>, Map<K,V>>{
#Override
public Adapter<K,V> marshal(Map<K,V> map) throws Exception {
if ( map == null )
return null;
return new Adapter<K,V>(map);
}
#Override
public Map<K,V> unmarshal(Adapter<K,V> adapter) throws Exception {
throw new UnsupportedOperationException("Unmarshalling a list into a map is not supported");
}
#XmlAccessorType(XmlAccessType.FIELD)
#XmlType(name="Adapter", namespace="MapAdapter")
public static final class Adapter<K,V>{
List<MapElement<K,V>> item;
public Adapter(){}
public Adapter(Map<K,V> map){
item = new ArrayList<MapElement<K,V>>(map.size());
for (Map.Entry<K, V> entry : map.entrySet()) {
item.add(new MapElement<K,V>(entry));
}
}
}
#XmlAccessorType(XmlAccessType.FIELD)
#XmlType(name="MapElement", namespace="MapAdapter")
public static final class MapElement<K,V>{
#XmlAnyElement
private K key;
#XmlAnyElement
private V value;
public MapElement(){};
public MapElement(K key, V value){
this.key = key;
this.value = value;
}
public MapElement(Map.Entry<K, V> entry){
key = entry.getKey();
value = entry.getValue();
}
public K getKey() {
return key;
}
public void setKey(K key) {
this.key = key;
}
public V getValue() {
return value;
}
public void setValue(V value) {
this.value = value;
}
}
}