How to encode a raw, unescaped JSON string? - json

Question

I have a Kotlin server that acts as a gateway, handling communication between a client and a number of backing services over a REST APIs, using JSON. My server uses Kotlinx Serialization for serialization.
Usually I need to parse and adapt the responses from the backing services, but occasionally I just want to return the raw JSON content as a response.
For example:
import kotlinx.serialization.json.*
fun main() {
// I get some JSON from a backing service
val backingServiceResponse = """
{"some":"json",id:123,content:[]}
""".trimIndent()
// I create a response object, that I will return to the client
val packet = ExampleClientResponse("name", backingServiceResponse)
val encodedPacket = Json.encodeToString(packet)
println(encodedPacket)
// I expect that the JSON is encoded without quotes
require("""{"name":"name","content":{"some":"json",id:123,content:[]}}""" == encodedPacket)
}
#Serializable
data class ExampleClientResponse(
val name: String,
val content: String, // I want this to be encoded as unescaped JSON
)
However, the value of content is surrounded by quotes, and is escaped
{
"name":"name",
"content":"{\"some\":\"json\",id:123,content:[]}"
}
What I want is for the content property to be literally encoded:
{
"name":"name",
"content":{
"some":"json",
"id":123,
"content":[]
}
}
I am using Kotlin 1.8.0 and Kotlinx Serialization 1.4.1.

Answer 1

Encoding raw JSON is possible in Kotlinx Serialization 1.5.0-RC, which was released on 26th Jan 2023, and is experimental. It is not possible in earlier versions.
Create a custom serializer
First, create a custom serializer, RawJsonStringSerializer, that will encode/decode strings.
Encoding needs to use the new JsonUnquotedLiteral function to encode the content, if we're encoding the string as JSON
Since the value being decoded might be a JSON object, array, or primitive, it must use JsonDecoder, which has the decodeJsonElement() function. This will dynamically decode whatever JSON data is present to a JsonElement, which can be simply be converted to a JSON string using toString().
import kotlinx.serialization.*
import kotlinx.serialization.descriptors.*
import kotlinx.serialization.encoding.*
import kotlinx.serialization.json.*
private object RawJsonStringSerializer : KSerializer<String> {
override val descriptor = PrimitiveSerialDescriptor("my.project.RawJsonString", PrimitiveKind.STRING)
/**
* Encodes [value] using [JsonUnquotedLiteral], if [encoder] is a [JsonEncoder],
* or with [Encoder.encodeString] otherwise.
*/
#OptIn(ExperimentalSerializationApi::class)
override fun serialize(encoder: Encoder, value: String) = when (encoder) {
is JsonEncoder -> encoder.encodeJsonElement(JsonUnquotedLiteral(value))
else -> encoder.encodeString(value)
}
/**
* If [decoder] is a [JsonDecoder], decodes a [kotlinx.serialization.json.JsonElement] (which could be an object,
* array, or primitive) as a string.
*
* Otherwise, decode a string using [Decoder.decodeString].
*/
override fun deserialize(decoder: Decoder): String = when (decoder) {
is JsonDecoder -> decoder.decodeJsonElement().toString()
else -> decoder.decodeString()
}
}
Apply the custom serializer
Now in your class, you can annotated content with #Serializable(with = ...) to use the new serializer.
import kotlinx.serialization.*
#Serializable
data class ExampleClientResponse(
val name: String,
#Serializable(with = RawJsonStringSerializer::class)
val content: String,
)
Result
Nothing has changed in the main method - Kotlinx Serialization will automatically encode content literally, so it will now succeed.
import kotlinx.serialization.*
import kotlinx.serialization.descriptors.*
import kotlinx.serialization.encoding.*
import kotlinx.serialization.json.*
fun main() {
val backingServiceResponse = """
{"some":"json",id:123,content:[]}
""".trimIndent()
val packet = ExampleClientResponse("name", backingServiceResponse)
val encodedPacket = Json.encodeToString(packet)
println(encodedPacket)
require("""{"name":"name","content":{"some":"json",id:123,content:[]}}""" == encodedPacket)
}
Reducing duplication
Using #Serializable(with = ...) is simple when it's a on-off usage, but what if you have lots of properties that you want to encode as literal JSON?
Typealias serialization
When a fix is released in Kotlin 1.8.20 this will be possible with a one-liner
// awaiting fix https://github.com/Kotlin/kotlinx.serialization/issues/2083
typealias RawJsonString = #Serializable(with = RawJsonStringSerializer::class) String
#Serializable
data class ExampleClientResponse(
val name: String,
val content: RawJsonString, // will be encoded literally, without escaping
)
Raw encode a value class
Until Kotlinx Serialization can handle encoding typealias-primitives, you can use an inline value class, which we tell Kotlinx Serialization to encode using RawJsonStringSerializer.
#JvmInline
#Serializable(with = RawJsonStringSerializer::class)
value class RawJsonString(val content: String) : CharSequence by content
Now now annotation is needed in the data class:
#Serializable
data class ExampleClientResponse(
val name: String,
val content: RawJsonString, // will be encoded as a literal JSON string
)
RawJsonStringSerializer needs to be updated to wrap/unwrap the value class
#OptIn(ExperimentalSerializationApi::class)
private object RawJsonStringSerializer : KSerializer<RawJsonString> {
override val descriptor = PrimitiveSerialDescriptor("my.project.RawJsonString", PrimitiveKind.STRING)
override fun deserialize(decoder: Decoder): RawJsonString = RawJsonString(decoder.decodeString())
override fun serialize(encoder: Encoder, value: RawJsonString) = when (encoder) {
is JsonEncoder -> encoder.encodeJsonElement(JsonUnquotedLiteral(value.content))
else -> encoder.encodeString(value.content)
}
}
The tradeoff is that it's a little clunky to convert to/from the new value class.
val backingServiceResponse = """
{"some":"json",id:123,content:[]}
""".trimIndent()
// need to wrap backingServiceResponse in the RawJsonString value class
val packet = ExampleClientResponse("name", RawJsonString(backingServiceResponse))
This answer was written using
Kotlin 1.8
Kotlinx Serialization 1.5.0-RC.