Hi I'm facing problem with serialization of map where key is an custom class.
data class KeyClass(val id: Int, val name: String) {
fun toJSON() = "\"KeyClass\": {\"id\":$id,\"name\":\"$name\"}"
}
Invocation:
fun method(): Map<KeyClass, List<Something>> = ...
My jackson Serializer ofc I'm also adding this as module in objectMapper:
class KeyClassSerializer : JsonSerializer<KeyClass>() {
override fun serialize(value: KeyClass, gen: JsonGenerator, serializers: SerializerProvider) {
gen.writeRawValue(value.toJSON())
}
}
class KeyClassSerializerModule : SimpleModule() {
init {
addKeySerializer(KeyClass::class.java, KeyClassSerializer())
}
}
And JSON I'm receiving is:
"\"KeyClass\": {\"id\":1,\"name\":\"Thomas\"}" : [Something:...]
I mean the value of map is serialized correctly but key isn't.
I assume the expected result is :
"KeyClass": {
"id": 1,
"name":"Thomas"
} : [...]
But it's not valid Json. You can still do something like :
{
"key" : {
"id": 1,
"name":"Thomas"
},
"value" : [...]
}
Related
I am trying to serialize a kotlin.collections.Map from Json using Ktor and I continue to get an error, basically telling me it doesn't know how. I assumed this case was basic.
val beans = beans {
bean("oauthClient") {
HttpClient(CIO) {
expectSuccess = true
install(DefaultRequest){
url("${env["oauth.url"]!!}/ms_oauth/oauth2/endpoints/oauthservice/tokens")
}
install(Auth) {
basic {
credentials {
BasicAuthCredentials(username = env["client.id"]!!, password = env["client.secret"]!!)
}
}
}
install(ContentNegotiation) {
json()
}
}
}
bean {
val oauthResults: Map<String,Any> = runBlocking { // blows up here
ref<HttpClient>("oauthClient").submitForm("${env["oauth.url"]!!}/ms_oauth/oauth2/endpoints/oauthservice/tokens",
Parameters.build {
append("grant_type", "password")
append("scope", "bug.rest.public")
append("username", env["ssoUsername"]!!)
append("password", env["ssoPassword"]!!)
}
).body()
}
}
}
error:
Caused by: io.ktor.client.call.NoTransformationFoundException:
No transformation found: class io.ktor.utils.io.ByteBufferChannel -> class kotlin.collections.Map
Do I have to explicitly enable something? Or am I making a different mistake?
Update
I've since tried to marshal it into a data object and I get the same error, but referencing that object:
#Serializable
data class Oauth constructor(
#SerialName("expires_in")
val expiration: Int,
#SerialName("token_type")
val tokenType:String,
#SerialName("access_token")
val accessToken: String
)
error:
io.ktor.client.call.NoTransformationFoundException:
No transformation found: class io.ktor.utils.io.ByteBufferChannel -> class com.company.Oauth
Hope anybody could guide me here. I spend some hours on it and can't understand what's going on.
Mission: Replace a json element by a jsonpath search tag. (sort of $ref feature)
In my code example below i want to replace the value of DataReaderUser by a value found by the json path search $.UsersAndGroups.Users[?(#.Name == 'OMDASAccountUser')].Username . In this case it should result in the value "contoso\SVCSCOM-DO-OMDAS"
The code below works as expected.. the issue is below this code ..
https://dotnetfiddle.net/gEjggK
using System;
using Newtonsoft.Json.Linq;
public class Program
{
public static void Main()
{
string json = #"{
""SQLServer"": {
""SQLReportingServices"": {
""AccountSettings"": {
""DataReaderUser"": {""$JsonPath"": ""$.UsersAndGroups.Users[?(#.Name == 'OMDASAccountUser')].Username""},
}
}
},
""UsersAndGroups"": {
""Users"": [
{
""Name"": ""OMActionAccountUser"",
""Username"": ""contoso\\SVCSCOM-DO-OMDAS"",
},
{
""Name"": ""OMDASAccountUser"",
""Username"": ""contoso\\SVCSCOM-DO-OMDAS"",
}
]
}
}";
JObject jo = JObject.Parse(json);
var JsonPath = jo.SelectToken("..$JsonPath");
JsonPath.Parent.Parent.Replace(jo.SelectToken(JsonPath.ToString()));
Console.WriteLine(jo.ToString());
}
}
The output will be :
{
"SQLServer": {
"SQLReportingServices": {
"AccountSettings": {
"DataReaderUser": "contoso\\SVCSCOM-DO-OMDAS"
}
}
},
"UsersAndGroups": {
"Users": [
{
"Name": "OMActionAccountUser",
"Username": "contoso\\SVCSCOM-DO-OMDAS"
},
{
"Name": "OMDASAccountUser",
"Username": "contoso\\SVCSCOM-DO-OMDAS"
}
]
}
}
Now the issue:
I want to do the same for all possible jsonpaths refers. So i use the SelectTokens and an foreach . But it looks like the behavior is different , the parents are null.
https://dotnetfiddle.net/lZW3XP
using System;
using Newtonsoft.Json.Linq;
public class Program
{
public static void Main()
{
string json = #"{
""SQLServer"": {
""SQLReportingServices"": {
""AccountSettings"": {
""DataReaderUser"": {""$JsonPath"": ""$.UsersAndGroups.Users[?(#.Name == 'OMDASAccountUser')].Username""},
}
}
},
""UsersAndGroups"": {
""Users"": [
{
""Name"": ""OMActionAccountUser"",
""Username"": ""contoso\\SVCSCOM-DO-OMDAS"",
},
{
""Name"": ""OMDASAccountUser"",
""Username"": ""contoso\\SVCSCOM-DO-OMDAS"",
}
]
}
}";
JObject jo = JObject.Parse(json);
var JsonPaths = jo.SelectTokens("..$JsonPath");
foreach (var JsonPath in JsonPaths )
{
JsonPath.Parent.Parent.Replace(jo.SelectToken(JsonPath.ToString()));
}
Console.WriteLine(jo.ToString());
}
}
And the output:
Run-time exception (line 34): Object reference not set to an instance of an object.
Stack Trace:
[System.NullReferenceException: Object reference not set to an instance of an object.]
at Newtonsoft.Json.Linq.JsonPath.PathFilter.GetNextScanValue(JToken originalParent, JToken container, JToken value)
at Newtonsoft.Json.Linq.JsonPath.ScanFilter.<ExecuteFilter>d__4.MoveNext()
at Program.Main() :line 34
would be great to get some directions since i am spinning my head here.
michel
SelectTokens uses lazy evaluation and if you modify the token while enumerating all matches it can break in unexpected ways. A simple fix is to add ToArray() to force eager evaluation:
var JsonPaths = jo.SelectTokens("..$JsonPath").ToArray();
I'm trying to fetch and deserialize some data that is being hosted on github.
{
"Meals": [
{
"id": "1598044e-5259-11e9-8647-d663bd870b02",
"name": "Tomato pasta",
"quantity": [{
"quantity": 1 },
{
"quantity": 2
},
{
"quantity": 3
}],
"availableFromDate": "1605802429",
"expiryDate": "1905802429",
"info": "Vegetarian",
"hot": false,
"locationLat": 57.508865,
"locationLong": -6.292,
"distance": null
},
{
"id": "2be2d854-a067-43ec-a488-2e69f0f2a624",
"name": "Pizza",
"quantity": [{
"quantity": 1 },
{
"quantity": 2
},
{
"quantity": 3
}
],
"availableFromDate": "1605802429",
"expiryDate": "1905902429",
"info": "Meat",
"hot": false,
"locationLat": 51.509465,
"locationLong": -0.135392,
"distance": null
}
]
}
If I spin up a json-server locally then it works perfectly, so I know that my data class is not the problem. However when I try to do the same thing from that github link I get this error:
Error Domain=KotlinException Code=0 "No transformation found: class io.ktor.utils.io.ByteChannelNative -> class kotlin.collections.List
I have a feeling it might be to do with setting a ContentType or something along those lines but I haven't had any success specifying that so far.
Here is my code to make the request:
class MealApi {
private val httpClient = HttpClient {
install(JsonFeature) {
val json = Json { ignoreUnknownKeys = true }
serializer = KotlinxSerializer(json)
}
}
suspend fun getAllMeals(): List<Meal> {
return httpClient.get(endpoint)
}
}
and here is my data class just for completeness:
#Serializable
data class Meal(
#SerialName("id")
val id: String,
#SerialName("name")
val name: String,
#SerialName("quantity")
val quantity: List<Quantity>,
#SerialName("availableFromDate")
var availableFromDate: String,
#SerialName("expiryDate")
var expiryDate: String,
#SerialName("info")
val info: String,
#SerialName("hot")
val hot: Boolean,
#SerialName("locationLat")
val locationLat: Float,
#SerialName("locationLong")
val locationLong: Float,
#SerialName("distance")
var distance: Double? = null
)
#Serializable
data class Quantity(
#SerialName("quantity")
val quantity: Int
)
UPDATE
I've found that this server https://gitcdn.link/ allows you to serve your raw github files with the right Content-Type.
I've searched a lot how to change the server response headers (to change the plain/text one to application/json) but it seems that ktor actually doesn't allow to do that:
https://youtrack.jetbrains.com/issue/KTOR-617
https://youtrack.jetbrains.com/issue/KTOR-580
A nice way should be to allow the ResponseObserver to change the server response headers and pass through the modify response. But you can't actually.
Your problem depends, as you pointed out, from the fact that the raw github page provides an header Content-Type=plain/text instead of ContentType=application/json.
So IRL when you are running your API in a real server this won't occur as you'll take care to put the right content type at server level.
But if you want a workaround to this you could rewrite your api call in this way:
suspend fun getAllMealsWithFallback() {
var meals: Meals? = null
try {
meals = httpClient.get(endpoint)
} catch (e: NoTransformationFoundException) {
val mealsString: String = httpClient.get(endpoint)
val json = kotlinx.serialization.json.Json {
ignoreUnknownKeys = true
}
meals = json.decodeFromString(mealsString)
} finally {
println("Meals: ${meals?.meals}")
}
}
I had to add this class to conform to the json text you have provided in the github link.
#Serializable
data class Meals(
#SerialName("Meals")
val meals: List<Meal>,
)
Try this:
install(JsonFeature) {
serializer = KotlinxSerializer(KotlinJson { ignoreUnknownKeys = true })
acceptContentTypes = acceptContentTypes + ContentType.Any
}
If you'd like to accept all content types. Or Use ContentType.Text.Any, ContentType.Text.Html if you preferred.
In case the issue is the Content-Type:
You can alter the list of response content types, for which the KotlinxSerializer gets active, by extending the JsonFeature block to:
install(JsonFeature) {
val json = Json { ignoreUnknownKeys = true }
serializer = KotlinxSerializer(json)
receiveContentTypeMatchers += object : ContentTypeMatcher {
override fun contains(contentType: ContentType): Boolean =
contentType == ContentType("text", "plain")
}
}
If you're using Ktor 2.0, you would need ContentNegotiation plugin instead of JsonFeature.
For example if you use Gson:
install(io.ktor.client.plugins.contentnegotiation.ContentNegotiation) {
gson()
}
As a workaround for ktor version 2.0.3 you can create you own AppContentNegotiation class and in the scope.responsePipeline.intercept provide needed contentType
/**
* A plugin that serves two primary purposes:
* - Negotiating media types between the client and server. For this, it uses the `Accept` and `Content-Type` headers.
* - Serializing/deserializing the content in a specific format when sending requests and receiving responses.
* Ktor supports the following formats out-of-the-box: `JSON`, `XML`, and `CBOR`.
*
* You can learn more from [Content negotiation and serialization](https://ktor.io/docs/serialization-client.html).
*/
public class AppContentNegotiation internal constructor(
internal val registrations: List<Config.ConverterRegistration>
) {
/**
* A [ContentNegotiation] configuration that is used during installation.
*/
public class Config : Configuration {
internal class ConverterRegistration(
val converter: ContentConverter,
val contentTypeToSend: ContentType,
val contentTypeMatcher: ContentTypeMatcher,
)
internal val registrations = mutableListOf<ConverterRegistration>()
/**
* Registers a [contentType] to a specified [converter] with an optional [configuration] script for a converter.
*/
public override fun <T : ContentConverter> register(
contentType: ContentType,
converter: T,
configuration: T.() -> Unit
) {
val matcher = when (contentType) {
ContentType.Application.Json -> JsonContentTypeMatcher
else -> defaultMatcher(contentType)
}
register(contentType, converter, matcher, configuration)
}
/**
* Registers a [contentTypeToSend] and [contentTypeMatcher] to a specified [converter] with
* an optional [configuration] script for a converter.
*/
public fun <T : ContentConverter> register(
contentTypeToSend: ContentType,
converter: T,
contentTypeMatcher: ContentTypeMatcher,
configuration: T.() -> Unit
) {
val registration = ConverterRegistration(
converter.apply(configuration),
contentTypeToSend,
contentTypeMatcher
)
registrations.add(registration)
}
private fun defaultMatcher(pattern: ContentType): ContentTypeMatcher =
object : ContentTypeMatcher {
override fun contains(contentType: ContentType): Boolean =
contentType.match(pattern)
}
}
/**
* A companion object used to install a plugin.
*/
#KtorDsl
public companion object Plugin : HttpClientPlugin<Config, AppContentNegotiation > {
public override val key: AttributeKey<AppContentNegotiation> =
AttributeKey("ContentNegotiation")
override fun prepare(block: Config.() -> Unit): AppContentNegotiation {
val config = Config().apply(block)
return AppContentNegotiation(config.registrations)
}
override fun install(plugin: AppContentNegotiation, scope: HttpClient) {
scope.requestPipeline.intercept(HttpRequestPipeline.Transform) { payload ->
val registrations = plugin.registrations
registrations.forEach { context.accept(it.contentTypeToSend) }
if (subject is OutgoingContent || DefaultIgnoredTypes.any { it.isInstance(payload) }) {
return#intercept
}
val contentType = context.contentType() ?: return#intercept
if (payload is Unit) {
context.headers.remove(HttpHeaders.ContentType)
proceedWith(EmptyContent)
return#intercept
}
val matchingRegistrations =
registrations.filter { it.contentTypeMatcher.contains(contentType) }
.takeIf { it.isNotEmpty() } ?: return#intercept
if (context.bodyType == null) return#intercept
context.headers.remove(HttpHeaders.ContentType)
// Pick the first one that can convert the subject successfully
val serializedContent = matchingRegistrations.firstNotNullOfOrNull { registration ->
registration.converter.serialize(
contentType,
contentType.charset() ?: Charsets.UTF_8,
context.bodyType!!,
payload
)
} ?: throw ContentConverterException(
"Can't convert $payload with contentType $contentType using converters " +
matchingRegistrations.joinToString { it.converter.toString() }
)
proceedWith(serializedContent)
}
scope.responsePipeline.intercept(HttpResponsePipeline.Transform) { (info, body) ->
if (body !is ByteReadChannel) return#intercept
if (info.type == ByteReadChannel::class) return#intercept
// !!!!!!! Provide desired content type here
val contentType = <HERE> // default implementation is - context.response.contentType() ?: return#intercept
val registrations = plugin.registrations
val matchingRegistrations = registrations
.filter { it.contentTypeMatcher.contains(contentType) }
.takeIf { it.isNotEmpty() } ?: return#intercept
// Pick the first one that can convert the subject successfully
val parsedBody = matchingRegistrations.firstNotNullOfOrNull { registration ->
registration.converter
.deserialize(context.request.headers.suitableCharset(), info, body)
} ?: return#intercept
val response = HttpResponseContainer(info, parsedBody)
proceedWith(response)
}
}
}
}
And then install it for HttpClient
HttpClient {
install(AppContentNegotiation) {
json(json)
addDefaultResponseValidation()
}
}
In my case, this exception was thrown when I was attempting to access the body() potion of the HTTP response as such:
val httpResponse = httpClient.post(urlString) { ... }
val body = httpResponse.body<YourExpectedSerializableResponseType>
In the happy path scenario, the server would return a body that matched YourExpectedSerializableResponseType and everything would work as expected.
However, for a particular edge case that returned a different (still considered successful) status code, the server returned an empty response body. Since the client was expecting a response body and in this case, there wasn't any, this exception was thrown because it could not serialize an empty response body to the expected type YourExpectedSerializableResponseType.
My recommendation: In addition to ensuring your server is returning the type you expect to serialize/consume on your client, confirm your server is actually returning an object.
Internal Dialogue: I wonder if this exception could be more clear in this case as the issue is more so that the client expected a response body to exist and less so that an empty response body ("") couldn't be serialized into an expected type - especially given that an empty response body isn't even valid JSON. Hm. 🤔
So according to various suggestions I updated my web api controller method that accepts multiple complex parameter objects from
//public IHttpActionResult PostCreateCase([FromBody] ARC.Donor.Business.Case.CreateCaseInput CreateCaseInput, [FromBody] ARC.Donor.Business.Case.SaveCaseSearchInput SaveCaseSearchInput)
to
public IHttpActionResult PostCreateCase(JObject jsonObject)
{
}
and then parse them accordingly ...
public IHttpActionResult PostCreateCase(JObject jsonObject)
{
var CreateCaseInput = jsonObject["CreateCaseInput"].ToObject<CreateCaseInput>();
var SaveCaseSearchInput = jsonObject["SaveCaseSearchInput"].ToObject<SaveCaseSearchInput>();
ARC.Donor.Service.Case.CaseServices cs = new ARC.Donor.Service.Case.CaseServices();
var searchResults = cs.createCase(CreateCaseInput, SaveCaseSearchInput);
------
}
but even after that when I test my controller using the Json object as
"CreateCaseInput":[
{
"case_nm":"EFG Test",
"case_desc":"EFG is a test",
"report_dt" : "04/12/2015"
}
],
"SaveCaseSearchInput":[
{
"firstName" : "Chiranjib",
"constType" : "IN"
}
]
I still get CreateCaseInput and SaveCaseSearchInput objects in
public IList<ARC.Donor.Business.Case.CreateCaseOutput> createCase(ARC.Donor.Business.Case.CreateCaseInput CreateCaseInput, ARC.Donor.Business.Case.SaveCaseSearchInput SaveCaseSearchInput)
{
.......
}
to be null.
What am I doing wrong ?
You JSON is wrong because is an array, not a object.
Try this way:
{
"CreateCaseInput":
{
"case_nm":"EFG Test",
"case_desc":"EFG is a test",
"report_dt" : "04/12/2015"
},
"SaveCaseSearchInput":
{
"firstName" : "Chiranjib",
"constType" : "IN"
}
}
Hope it helps :)
I want to be able to parse a string to an object that I can access using the dot notation e.g. myobject.property, instead of the array notation e.g. myobject['property']. The array notation works fine. Here's what I have so far.
I have some XML:
<level1 name="level1name">
<level2 type="level2Type">
<entry>level2entry</entry>
<entry>level2entry</entry>
</level2>
</level1>
Which converts to the JSON:
{
"level1": {
"name": "level1name",
"level2": {
"type": "level2Type",
"entry": [
"level2entry",
"level2entry"
]
}
}
}
I have the following Dart code:
Object jsonObject = JSON.parse("""{
"level1": {
"name": "level1name",
"level2": {
"type": "level2Type",
"entry": [
"level2entry",
"level2entry"
]
}
}
}
""");
print("my test 1 == ${jsonObject}");
print("my test 2 == ${jsonObject['level1']}");
print("my test 3 == ${jsonObject['level1']['name']}");
which produce the (desired) output:
my test 1 == {level1: {name: level1name, level2: {type: level2Type, entry: [level2entry, level2entry]}}}
my test 2 == {name: level1name, level2: {type: level2Type, entry: [level2entry, level2entry]}}
my test 3 == level1name
But when I try:
print("my test 1 == ${jsonObject.level1}");
I get the following:
Exception: NoSuchMethodException : method not found: 'get:level1'
Receiver: {level1: {name: level1name, level2: {type: level2Type, entry: [level2entry, level2entry]}}}
Arguments: []
Stack Trace: 0. Function: 'Object.noSuchMethod' url: 'bootstrap' line:717 col:3
Ideally, I want an object that I can access using the dot notation and without the compiler giving warning about Object not having property. I tried the following:
class MyJSONObject extends Object{
Level1 _level1;
Level1 get level1() => _level1;
set level1(Level1 s) => _level1 = s;
}
class Level1 {
String _name;
String get name() => _name;
set name(String s) => _name = s;
}
...
MyJSONObject jsonObject = JSON.parse("""{
"level1": {
"name": "level1name",
"level2": {
"type": "level2Type",
"entry": [
"level2entry",
"level2entry"
]
}
}
}
""");
...
print("my test 1 == ${jsonObject.level1.name}");
but instead of giving me 'level1name' as hoped, I get:
Exception: type 'LinkedHashMapImplementation<String, Dynamic>' is not a subtype of type 'MyJSONObject' of 'jsonObject'.
What am I doing wrong here? Is there any way to do what I'm trying? Thanks.
At the moment, JSON.parse only returns Lists (array), Maps, String, num, bool, and null
(api ref).
I suspect that until reflection makes it way into the language, it won't be able to re-construct objects based upon the keys found in json.
You could, however, create a constructor in your MyJsonObject which took a string, called JSON.parse internally, and assigned the various values.
Something like this works in the dart editor:
#import("dart:json");
class Level2 {
var type;
var entry;
}
class Level1 {
var name;
var level2;
}
class MyJSONObject {
Level1 level1;
MyJSONObject(jsonString) {
Map map = JSON.parse(jsonString);
this.level1 = new Level1();
level1.name = map['level1']['name'];
level1.level2 = new Level2();
level1.level2.type = map['level1']['level2']['type'];
//etc...
}
}
main() {
var obj = new MyJSONObject(json);
print(obj.level1.level2.type);
}
A non trivial version would needs some loops and possible recursion if you had deeper nested levels.
Update: I've hacked together a non-trivial version (inspired by the post below), it's up on github (also taking Seth's comments re the constructor):
Chris is completely right. I will only add that the JSON parser could be modified to return a little richer object (something like JsonMap instead of pure Map) that could allow jsonObj.property by implementing noSuchMethod. That would obviously perform worse than jsonObj['property'].