Elasticsearch java client stopped supporting custom query type values, such as enum - elasticsearch-7

Looks like there is a regression\change in Elasticsearch java client, it stopped supporting custom query type values, such as enum:
String query = QueryBuilders.boolQuery().
must(QueryBuilders.termsQuery("status",
List.of(MyEnumStatus.ACTIVE, MyEnumStatus.UNDER_TEST))).toString();
The above example produced a json string query with java client 6.8.14 but gives below exception with 7.15.2, any workaround?
java.lang.IllegalArgumentException: can not write type [class ...$MyEnumStatus]
at org.elasticsearch.common.io.stream.StreamOutput.writeGenericValue(StreamOutput.java:845)
at org.elasticsearch.common.io.stream.StreamOutput.lambda$static$12(StreamOutput.java:699)
at org.elasticsearch.common.io.stream.StreamOutput.writeGenericValue(StreamOutput.java:843)
at org.elasticsearch.index.query.TermsQueryBuilder$Values.serialize(TermsQueryBuilder.java:435)
at org.elasticsearch.index.query.TermsQueryBuilder$BinaryValues.<init>(TermsQueryBuilder.java:495)
at org.elasticsearch.index.query.TermsQueryBuilder$BinaryValues.<init>(TermsQueryBuilder.java:485)
at org.elasticsearch.index.query.TermsQueryBuilder.<init>(TermsQueryBuilder.java:161)
at org.elasticsearch.index.query.QueryBuilders.termsQuery(QueryBuilders.java:590)

As per my understanding, it is JSON serialization issue. You can add toString() method to both the ENUM and it will resolve issue.
String query = QueryBuilders.boolQuery().
must(QueryBuilders.termsQuery("status",
List.of(MyEnumStatus.ACTIVE.toString(), MyEnumStatus.UNDER_TEST.toString()))).toString();

Related

Failed to convert argument [obj] for value [null] due to: Cannot deserialize value of type `java.time.LocalDate` from String

Micronaut 1.3
Jackson 2.13.1
I am passing the date in the format 'dd.MM.yyyy'
My class field looks like this
#JsonInclude
#Column(name = "moving_date")
#JsonView(JsonViewSimple.class)
#JsonFormat(shape = JsonFormat.Shape.STRING, pattern = "dd.MM.yyyy")
private LocalDate moving_date;
After I make a post request with '02.06.2022', I get an error
Failed to convert argument [obj] for value [null] due to: Cannot deserialize value of type `java.time.LocalDate` from String \"02.06.2022\": Failed to deserialize java.time.LocalDate: (java.time.format.DateTimeParseException) Text '02.06.2022' could not be parsed at index 0\n at [Source: UNKNOWN; byte offset: #UNKNOWN]
In other classes, where I have similar date fields with a pattern and the same set of annotations, there is no such error.
Can anyone help?
I also wrote custom and date serializer and deserializer,
but when I enable debugging - the program doesn't hit the breakpoints in any of them and an error occurs immediately.
But if I pass the date in the format
yyyy-MM-dd
then there is no error and all is well.
As far as I understand, this bug occurs somewhere in the depths of the Micronaut.

How to insert/update a row in MySQL for JSON column using Java SprinBoot CrudRepository

I am trying to insert a row in MySQL JSON column using Java SpringBoot CrudRepository. I already skimmed through multiple posts but couldn't get answer for this.
I defined a custom query as:
#Modifying
#Transactional
#Query(value = "insert into cs_test_json(config) values (':data')", nativeQuery = true)
public void setData(#Param("data") String string);
and MySQL table(cs_test_json) is :
id int
config JSON
In my Controller I wrote
JSONObject data = new JSONObject();
data.put("name", "kc");
data.put("age", 28);
data.put("city", "Abad");
repository.setData(data.toString());
When I run this application I always get (on calling setData method)
com.mysql.cj.jdbc.exceptions.MysqlDataTruncation: Data truncation: Invalid JSON text: "Invalid value." at position 0 in value for column 'cs_test_json.config'
Please suggest me what I am missing.
I am using MySQL 8
Thank you very much.
I spotted the ':data'. Can you give a try with :data.? It's a string that you pass and it doesn't need a quotation to further qualify. Spring data takes care of it.
I'm replying from a mobile. Kindly excuse typos and formatting.

How to handle JSON in a Kafka topic?

I'm building a Java application in which there are some JSON objects (in particular AnyJson objects from com.satori.rtm.model.AnyJson) and I want to send these objects in a Kafka Topic. Should I send them in AnyJson o String type? I'm asking this because the constructor KafkaProducer<K, V> seems to have some problem to handle JSON values when (de)serialization.
This in my producer configuration
Properties props= new Properties();
props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, kafkaBrokerEndpoint);
props.put(ProducerConfig.CLIENT_ID_CONFIG, "KafkaProducer");
props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, IntegerSerializer.class);
props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, JsonSerializer.class);
kafkaProducer= new KafkaProducer<Integer, AnyJson>(props);
where JsonSerializer.class is from org.springframework.kafka.support.serializer.JsonSerializer;
I have seen there are multiple packages that handle the JSON objects (as kafka.utils.json, com.google.gson.JsonObject;, etc).
Then when running
ProducerRecord<Integer, AnyJson> record= new ProducerRecord<Integer, AnyJson>(topic, json);
kafkaProducer.send(record);
I have the following exception
No serializer found for class com.satori.rtm.connection.GsonSerializer$JsonElementWrapper
Any help?
You may want to use org.apache.kafka.common.serialization.ByteArraySerializer for key and value serializations.
Now you should configure your producer record with byte[].
Then use ObjectMapper from Jackson (http://www.baeldung.com/jackson-object-mapper-tutorial ) to convert any json object to byte array.

Put/Post json not working with ODataController if Model has Int64

I have this Data Object with an Int64 column:
[TableAttribute(Name="dbo.vw_RelationLineOfBusiness")]
[DataServiceKey("ProviderRelationLobId")]
public partial class RelationLineOfBusiness
{
#region Column Mappings
private System.Guid _Lineofbusiness;
private System.String _ContractNumber;
private System.Nullable<System.Int32> _ProviderType;
private System.String _InsuredProviderType;
private System.Guid _ProviderRelationLobId;
private System.String _LineOfBusinessDesc;
private System.String _CultureCode;
private System.String _ContractDesc;
private System.Nullable<System.Guid> _ProviderRelationKey;
private System.String _ProviderRelationNbr;
**private System.Int64 _AssignedNbr;**
When I post/Put object through my OData controller using HttpClient and NewtsonSoft:
partial class RelationLineOfBusinessController : ODataController
{
public HttpResponseMessage PutRelationLineOfBusiness([FromODataUri] System.Guid key, Invidasys.VidaPro.Model.RelationLineOfBusiness entity)
the entity object is null and the error in my modelstate :
"Cannot convert a primitive value to the expected type 'Edm.Int64'. See the inner exception for more details."
I noticed when I do a get on my object using the below URL:
Invidasys.Rest.Service/VidaPro/RelationLineOfBusiness(guid'c6824edc-23b4-4f76-a777-108d482c0fee')
my json looks like the following - I noticed that the AssignedNbr is treated as a string.
{
"odata.metadata":"Invidasys.Rest.Service/VIDAPro/$metadata#RelationLineOfBusiness/#Element",
"Lineofbusiness":"ba129c95-c5bb-4e40-993e-c28ca86fffe4","ContractNumber":null,"ProviderType":null,
"InsuredProviderType":"PCP","ProviderRelationLobId":"c6824edc-23b4-4f76-a777-108d482c0fee",
"LineOfBusinessDesc":"MEDICAID","CultureCode":"en-US","ContractDesc":null,
"ProviderRelationKey":"a2d3b61f-3d76-46f4-9887-f2b0c8966914","ProviderRelationNbr":"4565454645",
"AssignedNbr":"1000000045","Ispar":true,"ProviderTypeDesc":null,"InsuredProviderTypeDesc":"Primary Care Physician",
"StartDate":"2012-01-01T00:00:00","EndDate":"2014-01-01T00:00:00","Created":"2014-06-13T10:59:33.567",
"CreatedBy":"Michael","Updated":"2014-06-13T10:59:33.567","UpdatedBy":"Michael"
}
When I do a PUT with httpclient the JSON is showing up in my restful services as the following and the json for the AssignedNbr column is not in quotes which results in the restful services failing to build the JSON back to an object. I played with the JSON and put the AssignedNbr in quotes and the request goes through correctly.
{"AssignedNbr":1000000045,"ContractDesc":null,"ContractNumber":null,"Created":"/Date(1402682373567-0700)/",
"CreatedBy":"Michael","CultureCode":"en-US","EndDate":"/Date(1388559600000-0700)/","InsuredProviderType":"PCP",
"InsuredProviderTypeDesc":"Primary Care Physician","Ispar":true,"LineOfBusinessDesc":"MEDICAID",
"Lineofbusiness":"ba129c95-c5bb-4e40-993e-c28ca86fffe4","ProviderRelationKey":"a2d3b61f-3d76-46f4-9887-f2b0c8966914",
"ProviderRelationLobId":"c6824edc-23b4-4f76-a777-108d482c0fee","ProviderRelationNbr":"4565454645","ProviderType":null,
"ProviderTypeDesc":null,"StartDate":"/Date(1325401200000-0700)/","Updated":"/Date(1408374995760-0700)/","UpdatedBy":"ED"}
The reason we wanted to expose our business model as restful services was to hide any data validation and expose all our databases in format that is easy to develop against. I looked at the DataServiceContext to see if it would work and it does but it uses XML to communicate between the restful services and the client. Which would work but DataServiceContext does not give the level of messaging that HttpRequestMessage/HttpResponseMessage gives me for informing users on the errors/missing information with their post.
We are planning on supporting multiple devices from our restful services platform but that requires that I can use NewtonSoft Json as well as Microsoft's DataContractJsonSerializer if need be.
My question is for a restful service standpoint - is there a way I can configure/code the restful services to take in the AssignedNbr as in JSON as without the quotes.
Or from a JSON standpoint is their a way I can get the JSON built without getting into the serializing business nor do I want our clients to have deal with custom serializers if they want to write their own apps against our restful services.
Any suggestions?
Thanks.
I think you can migrate to Web API 2.2 for OData V4. Here's the information:
Announcing the Release of ASP.NET MVC 5.2, Web API 2.2 and Web Pages 3.2
OData V4 Spec says:
3.2 Controlling the Representation of Numbers
The IEEE754Compatible=true format parameter indicates that the service MUST serialize Edm.Int64 and Edm.Decimal numbers (including the odata.count, if requested) as strings. If not specified, or specified as IEEE754Compatible=false, all numbers MUST be serialized as JSON numbers.
This enables support for JavaScript numbers that are defined to be 64-bit binary format IEEE 754 values [ECMAScript] (see section 4.3.1.9) resulting in integers losing precision past 15 digits, and decimals losing precision due to the conversion from base 10 to base 2.
OData JSON payloads that format Edm.Int64 and Edm.Decimal values as strings MUST specify this format parameter in the media type returned in the Content-Type header.
So, for payload as:
#"{
""Lineofbusiness"": ""ba129c95-c5bb-4e40-993e-c28ca86fffe4"",
""AssignedNbr"": ""1000000045""
}";
you should set:
request.Content.Headers.ContentType = MediaTypeHeaderValue.Parse("application/json;IEEE754Compatible=true");
Otherwise, you shouldn't.
Sam Xu is exactly right and should be marked as the answer.
However, I wanted to add exactly what you need to do to add this to the pipeline.
First, you can set this global, per route etc. You can find that information here:
http://www.asp.net/web-api/overview/advanced/http-message-handlers
Below you'll find an example that will work.
public static void Configuration(IAppBuilder builder)
{
HttpConfiguration config = new HttpConfiguration();
config.MessageHandlers.Add(new MethodOverrideHandler());
}
public class MethodOverrideHandler : DelegatingHandler
{
protected override Task<HttpResponseMessage> SendAsync(
HttpRequestMessage request, CancellationToken cancellationToken)
{
request.Content.Headers.ContentType = MediaTypeHeaderValue.Parse("application/json;IEEE754Compatible=true");
return base.SendAsync(request, cancellationToken);
}
}
Alternatively, try changing the type you send to your web api to Number instead of string
Also, check the Type of the decimal that you are sending. If it's type 'string' you can change it to type number. For my service, making this change no longer throws the error.
//Gnuget Package Manager Install-Package numeral
if (typeof newValue === 'string')
{
newValue = numeral().unformat(newValue);
}
odatajs.oData.request(
{
requestUri: xxx,
method: "PATCH",
data: { PriceNull: newValue }
}

FlexJSON - JSONDeserializer String Cannot be caste as Boolean

I am using FlexJson within my play framework application but at the point I am trying to deseralize the json string it throws a java.lang.ClassCastException: java.lang.String cannot be cast to java.lang.Boolean:
User user = new JSONDeserializer<User>()
.use(null, User.class).deserialize(body);
Body is the json string passed into the controller using standard jquery/ajax and
where User has the following boolean value declared:
public Boolean isCurrentUser;
Any ideas as to what I am doing wrong?
Thanks
In Json, Boolean is a type. Your JSon is:
{"user_id":"18","isCurrentUser":"true","title":"mr","description":"description"}
when it should be:
{"user_id":"18","isCurrentUser":true,"title":"mr","description":"description"}
Note that true is not a String, but a boolean. The parser fails because it finds a String instead of the expected boolean type. Fix the JSon generation to add a boolean, not a String.