How do I convert arbitrary structure into json?
let Prelude = ./include/Prelude.dhall
let JSON = Prelude.JSON
let Foo = { a: Natural, t: Text }
let foo = { a = 10, b = "foo" }
in (DO_MAGIC foo) : JSON.Type
I know there is toMap builtin function, but it expects a homogeneous record.
What I actually trying to do is to write OpenAPI specification in dhall. Most parts of it are simple and nice, but json schema that describe incoming data shape is recursive, which is hard in Dhall. What I want would be expressed in Haskell like following
data Schema
= SInteger { minimum :: Maybe Int, example :: Maybe Int }
| SString { format :: Maybe String, length :: Maybe Int }
| Object [(String, Schema, Bool)] -- (name, schema, required)
deriving (ToJSON)
Since it looked hard in Dhall, I decided that I would go this way:
data SInteger = SInteger { minimum :: Maybe Int, example :: Maybe Int }
data SString = SString { format :: Maybe String, length :: Maybe Int }
data Object = Object [(String, Schema, Bool)] -- (name, schema, required)
integer :: SInteger -> Schema
string :: SString -> Schema
object :: Object -> Schema
type Schema = JSON
but on this road I am stuck too. I am willing to sacrifice some of type rigidity for not patching dhall-json.
The basic idea is outlined in this guide:
How to translate recursive code to Dhall
… and here is how that looks in the context of your example:
let List/map = https://prelude.dhall-lang.org/v17.1.0/List/map.dhall
let JSON = https://prelude.dhall-lang.org/v17.1.0/JSON/Type
let JSON/render = https://prelude.dhall-lang.org/v17.1.0/JSON/render
let SInteger = { minimum : Optional Integer, example : Optional Integer }
let SString = { format : Optional Text, length : Optional Natural }
let SObject =
λ(Schema : Type) → List { name : Text, schema : Schema, required : Bool }
let Constructors =
λ(Schema : Type) →
{ Integer : SInteger → Schema
, String : SString → Schema
, Object : SObject Schema → Schema
}
let Schema
: Type
= ∀(Schema : Type) → ∀(schema : Constructors Schema) → Schema
let integer
: SInteger → Schema
= λ(x : SInteger) →
λ(Schema : Type) →
λ(schema : Constructors Schema) →
schema.Integer x
let string
: SString → Schema
= λ(x : SString) →
λ(Schema : Type) →
λ(schema : Constructors Schema) →
schema.String x
let object
: List { name : Text, schema : Schema, required : Bool } → Schema
= λ(x : SObject Schema) →
λ(Schema : Type) →
λ(schema : Constructors Schema) →
let Input = { name : Text, schema : Schema#1, required : Bool }
let Output = { name : Text, schema : Schema, required : Bool }
let adapt =
λ(y : Input) →
{ schema = y.schema Schema schema } ∧ y.{ name, required }
in schema.Object (List/map Input Output adapt x)
let toJSON
: Schema → JSON
= λ(schema : Schema) →
λ(JSON : Type) →
λ ( json
: { array : List JSON → JSON
, bool : Bool → JSON
, double : Double → JSON
, integer : Integer → JSON
, null : JSON
, object : List { mapKey : Text, mapValue : JSON } → JSON
, string : Text → JSON
}
) →
schema
JSON
{ Integer =
λ(x : SInteger) →
json.object
( toMap
{ minimum =
merge
{ None = json.null, Some = json.integer }
x.minimum
, example =
merge
{ None = json.null, Some = json.integer }
x.example
}
)
, String =
λ(x : SString) →
json.object
( toMap
{ format =
merge
{ None = json.null, Some = json.string }
x.format
, length =
merge
{ None = json.null
, Some =
λ(n : Natural) →
json.integer (Natural/toInteger n)
}
x.length
}
)
, Object =
λ(x : SObject JSON) →
let Input = { name : Text, schema : JSON, required : Bool }
let Output = { mapKey : Text, mapValue : JSON }
let adapt =
λ(y : Input) →
{ mapKey = y.name
, mapValue =
json.object
( toMap
{ schema = y.schema
, required = json.bool y.required
}
)
}
in json.object (List/map Input Output adapt x)
}
let example =
let input =
object
[ { name = "foo"
, required = True
, schema = string { format = None Text, length = Some 10 }
}
, { name = "bar"
, required = False
, schema = integer { minimum = Some +0, example = Some +10 }
}
]
let output =
''
{
"foo": {
"required": true,
"schema": {
"format": null,
"length": 10
}
},
"bar": {
"required": false,
"schema": {
"example": 10,
"minimum": 0
}
}
}
''
in assert : JSON/render (toJSON input) ≡ output
in { Schema, integer, string, object, toJSON }
Related
Question
Is there a way to create a record with a field called "data"?
data MyRecord =
MyRecord {
otherField :: String,
data :: String -- compilation error
}
Why do I need it?
I've been writing a wrapper around a JSON API using Aeson and the remote service decided to call one of the fields data.
{
pagination: {
..
},
data: [
{ .. },
{ .. },
]
}
Yes, you can name the field something else than data, like:
data MyRecord =
MyRecord {
otherField :: String,
recordData :: String
}
And then derive a ToJSON with a key modifier:
labelMapping :: String -> String
labelMapping "recordData" = "data"
labelMapping x = x
instance ToJSON MyRecord where
toJSON = genericToJSON defaultOptions {
fieldLabelModifier = labelMapping
}
instance FromJSON Coord where
parseJSON = genericParseJSON defaultOptions {
fieldLabelModifier = labelMapping
}
I have a simple Kotlin program that access a Mongo database and produce a JSON string as below;
"{
"_id" : { "$oid" : "593440eb7fa580d99d1abe85"} ,
"name" : "Firstname Secondname" ,
"reg_number" : "ATC/DCM/1016/230" ,
"oral" : 11 ,
"oral_percent" : 73 ,
"cat_1" : 57 ,
"cat_2" : 60 ,
"cat_average" : 59 ,
"assignment" : 90
}"
How do I map this in Kotlin Map/MutableMap? Is there an API in Kotlin to read JSON and map it to Map/MutableMap?
No additional library is needed:
val jsonObj = JSONObject(jsonString)
val map = jsonObj.toMap()
where toMap is:
fun JSONObject.toMap(): Map<String, *> = keys().asSequence().associateWith {
when (val value = this[it])
{
is JSONArray ->
{
val map = (0 until value.length()).associate { Pair(it.toString(), value[it]) }
JSONObject(map).toMap().values.toList()
}
is JSONObject -> value.toMap()
JSONObject.NULL -> null
else -> value
}
}
This can be done with Klaxon. With this you can easily read the Json data as JsonObject which is actually a MutableMap.
val json: JsonObject = Parser().parse(jsonData) as JsonObject
Using Jackson's kotlin module, you can create a Map/MutableMap as below:
val jsonString = "{\n" +
" \"_id\": {\n" +
" \"\$oid\": \"593440eb7fa580d99d1abe85\"\n" +
" },\n" +
" \"name\": \"Firstname Secondname\",\n" +
" \"reg_number\": \"ATC/DCM/1016/230\",\n" +
" \"oral\": 11,\n" +
" \"oral_percent\": 73,\n" +
" \"cat_1\": 57,\n" +
" \"cat_2\": 60,\n" +
" \"cat_average\": 59,\n" +
" \"assignment\": 90\n" +
"}"
val map = ObjectMapper().readValue<MutableMap<Any, Any>>(jsonString)
Note: In case you're getting the below compilation error
None of the following functions can be called with the arguments supplied
Please ensure that you have added the the dependency of jackson-module-kotlin (for gradle: compile "com.fasterxml.jackson.module:jackson-module-kotlin:${jackson_version}") and have added the import for the readValue implementation as import com.fasterxml.jackson.module.kotlin.readValue in the place where you're using readValue
This can be done without any third party library:
#Throws(JSONException::class)
fun JSONObject.toMap(): Map<String, Any> {
val map = mutableMapOf<String, Any>()
val keysItr: Iterator<String> = this.keys()
while (keysItr.hasNext()) {
val key = keysItr.next()
var value: Any = this.get(key)
when (value) {
is JSONArray -> value = value.toList()
is JSONObject -> value = value.toMap()
}
map[key] = value
}
return map
}
#Throws(JSONException::class)
fun JSONArray.toList(): List<Any> {
val list = mutableListOf<Any>()
for (i in 0 until this.length()) {
var value: Any = this[i]
when (value) {
is JSONArray -> value = value.toList()
is JSONObject -> value = value.toMap()
}
list.add(value)
}
return list
}
Usage to convert JSONObject to Map:
val jsonObject = JSONObject(jsonObjStr)
val map = jsonObject.toMap()
Usage to convert JSONArray to List:
val jsonArray = JSONArray(jsonArrStr)
val list = jsonArray.toList()
More info is here
This is now also possible with kotlinx.serialization:
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.jsonObject
val input = """{
"_id" : { "some_id" : "593440eb7fa580d99d1abe85"} ,
"name" : "Firstname Secondname" ,
"reg_number" : "ATC/DCM/1016/230" ,
"oral" : 11 ,
"oral_percent" : 73 ,
"cat_1" : 57 ,
"cat_2" : 60 ,
"cat_average" : 59 ,
"assignment" : 90
}"""
val json = Json.parseToJsonElement(input)
val map = json.jsonObject.toMap()
The latest version of the org.json library has JSONObject.toMap() so you can just do
val map = JSONObject(string).toMap()
So just add the org.json:json:20220924 dependency to your project.
If you're writing for Android (and thus have an old version of the org.json library) then the following can be used.
import org.json.JSONArray
import org.json.JSONObject
fun JSONObject.toMap(): Map<String, Any?> =
keys().asSequence().associateWith { key -> toValue(get(key)) }
fun JSONArray.toList(): List<Any?> =
(0 until length()).map { index -> toValue(get(index)) }
private fun toValue(element: Any) = when (element) {
JSONObject.NULL -> null
is JSONObject -> element.toMap()
is JSONArray -> element.toList()
else -> element
}
I have the following JSON data in a documentdb and I would like to parse this into an F# multiway tree discriminated union
"commentTree": {
"commentModel": {
"commentId": "",
"userId": "",
"message": ""
},
"forest": []
}
F# multiway discriminated union
type public CommentMultiTreeDatabaseModel =
| CommentDatabaseModelNode of CommentDatabaseModel * list<CommentMultiTreeDatabaseModel>
where CommentMultiTreeDatabaseModel is defined as
type public CommentDatabaseModel =
{ commentId : string
userId : string
message : string
}
I am referencing Fold / Recursion over Multiway Tree in f# extensively. I am not sure where to begin to parse such a JSON structure into an F# multiway tree. Any suggestions will be much appreciated. Thanks
One way to think about this is by looking at what data you need in order to construct a CommentMultiTreeDatabaseModel. It needs a CommentDatabaseModel and a list of CommentMultiTreeDatabaseModel. So we need to write the following two functions:
let parseComment (input : JSON) : CommentDatabaseModel =
...
let parseTree (input : JSON) : CommentMultiTreeDatabaseModel =
...
But wait, the parseTree function is the one we're trying to write right now! So instead of writing a new function, we just mark our current function with the rec keyword and have it call itself where needed.
Below is a rough example of how it could be done. The key thing to look at is parseTree which builds up the data by recursively calling itself. I've represented the JSON input data with a simple DU. A library like Chiron can produce something like this.
Note that this code parses all of the JSON in one go. Also, it's not tail-recursive, so you'll have to be careful with how deep your tree structure is.
[<RequireQualifiedAccess>]
type JSON =
| String of string
| Object of (string * JSON) list
| Array of JSON list
type public CommentDatabaseModel = {
commentId : string
userId : string
message : string
}
type public CommentMultiTreeDatabaseModel =
| CommentDatabaseModelNode of CommentDatabaseModel * list<CommentMultiTreeDatabaseModel>
let parseComment = function
| JSON.Object [ "commentId", JSON.String commentId; "userId", JSON.String userId; "message", JSON.String message ] ->
{
commentId = commentId
userId = userId
message = message
}
| _ -> failwith "Bad data"
let rec parseTree (input : JSON) : CommentMultiTreeDatabaseModel =
match input with
| JSON.Object [ "commentModel", commentModel; "forest", JSON.Array forest ] ->
CommentDatabaseModelNode (parseComment commentModel, List.map parseTree forest)
| _ -> failwith "Bad data"
let parse (input : JSON) : CommentMultiTreeDatabaseModel =
match input with
| JSON.Object [ "commentTree", commentTree ] ->
parseTree commentTree
| _ -> failwith "Bad data"
let comment text =
JSON.Object [
"commentId", JSON.String ""
"userId", JSON.String ""
"message", JSON.String text
]
let sampleData =
JSON.Object [
"commentTree", JSON.Object [
"commentModel", comment "one"
"forest", JSON.Array [
JSON.Object [
"commentModel", comment "two"
"forest", JSON.Array []
]
JSON.Object [
"commentModel", comment "three"
"forest", JSON.Array []
]
]
]
]
parse sampleData
(*
val it : CommentMultiTreeDatabaseModel =
CommentDatabaseModelNode
({commentId = "";
userId = "";
message = "one";},
[CommentDatabaseModelNode ({commentId = "";
userId = "";
message = "two";},[]);
CommentDatabaseModelNode ({commentId = "";
userId = "";
message = "three";},[])])
*)
//Item.scala
package model
trait Item {val id: String}
class MItem(override val id: String, val name: String) extends Item
class DItem(override val id: String, override val name: String, val name2: String)
extends MItem(valid, name, name2)
object ItemWrites {
implicit val MItemWrites = new Writes[MItem] {
def writes(dItem: MItem) = Json.obj(
"id" -> mItem.id,
"name" -> mItem.name
)
}
implicit val DItemWrites = new Writes[DItem] {
def writes(dItem: DItem) = Json.obj(
"id" -> dItem.id,
"name" -> dItem.name,
"name2" -> dItem.name2
)
}
}
//Response.scala
package model
import ItemWrites.MItemWrites
import ItemWRites.DItemWrites
trait Response {
val title : String,
val items : Seq[Item],
}
case class MResponse(title: String, items: Seq[MItem]) extends Response
case class DResponse(title: String, items: Seq[DItem]) extends Response
object ResponseWrites {
implicit val mResponseWrite = new Writes[MResponse] {
def writes(mResponse: MResponse) = Json.obj(
"title" -> mResponse.title,
"items" -> mResponse.items
)
}
implicit val dResponseWrite = new Writes[DResponse] {
def writes(dResponse: DResponse) = Json.obj(
"title" -> dResponse.title,
"items" -> dResponse.items // **compile time error**
)
}
}
I get compile time error on line dResponse.items.
> [error] Note: implicit value dtcTopicPageResponseWrite is not
> applicable here because it comes after the application point and it
> lacks an explicit result type
> [error] "items" -> dResponse.items
Seems that this is happening as DItem inherits form MItem and the compile is getting confused between subclass and super class. How do i get around this.
I was able to get your code compiling in Scala 2.11, Play 2.5.x by explicitly stating the types of the Writes you've declared in ItemWrites - namely:
object ItemWrites {
implicit val MItemWrites:Writes[MItem] = new Writes[MItem] {
def writes(mItem: MItem) = Json.obj(
"id" -> mItem.id,
"name" -> mItem.name
)
}
implicit val DItemWrites:Writes[DItem] = new Writes[DItem] {
def writes(dItem: DItem) = Json.obj(
"id" -> dItem.id,
"name" -> dItem.name,
"name2" -> dItem.name2
)
}
}
Once I did that, Scala was able to find out which implicit was right and away it went.
You should explicitly set which Writer[T] is going to be used because they use inheritance. I didn't test the code below but it should compile and work;
implicit val dResponseWrite = new Writes[DResponse] {
import ItemWrites.DItemWrites
def writes(dResponse: DResponse) = Json.obj(
"title" -> dResponse.title,
"items" -> Json.toJson(dResponse.items) // **compile time error**
)
}
I think it's better for you to have a Writes for the super type Item that will just do match case to check it's type and do toJson. Don't forget to have it implicitly imported
object ItemWrites {
implicit val MItemWrites = new Writes[MItem] {
def writes(dItem: MItem) = Json.obj(
"id" -> mItem.id,
"name" -> mItem.name
)
}
implicit val DItemWrites = new Writes[DItem] {
def writes(dItem: DItem) = Json.obj(
"id" -> dItem.id,
"name" -> dItem.name,
"name2" -> dItem.name2
)
}
implicit val ItemWrites = new Writes[Item] {
def writes(item: Item) = item match {
case i: DItem => i.toJson //code for converting DItem to Json
case i: MItem => i.toJson //code to convert MItem to Json
case _ => throws new RuntimeException("Unsupported Type.")
}
}
}
I am building a web app using Scala / Play Framework and Reactive Mongo and I want the models to be defined in the database instead of having them hardcoded.
To do so, I am writing a class EntityInstance taking a Sequence of FieldInstance :
case class EntityInstance(fields: Seq[FieldInstance])
I am trying to accept fields from any types and to convert them to Json : example
new FieldInstance("name", "John") | json: { "name": "John" }
new FieldInstance("age", 18) | json: { "age": 18 }
At the moment I am trying to accept Strings, Booleans and Integers and if the type is not supported I write some error :
new FieldInstance("profilePicture", new Picture("john.jpg") | json: { "profilePicture": "Unsupported type
I wrote a FieldInstance class taking a fieldName as a String and a value as any type. As soon as that class is instantiated I cast the value to a known type or to the String describing the error.
class FieldInstance(fieldNamec: String, valuec: Any) {
val fieldName = fieldNamec
val value = valuec match {
case v: Int => v
case v: String => v
case v: Boolean => v
case _ => "Unrecognized type"
}
}
object FieldInstance {
implicit val fieldInstanceWrites = new Writes[FieldInstance] {
def writes(fieldInstance: FieldInstance) = Json.obj(
fieldInstance.fieldName -> fieldInstance.value
)
}
}
I created a companion object with an implicit Write to json so I can call "Json.toJson()" on an instance of FieldInstance and get a json as described on my examples above.
I get an error : found: Any required: play.api.libs.json.Json.JsValueWrapper
I understand that it comes from the fact that my value is of type Any but I thought the cast would change that Any to String || Boolean || Int before hitting the Writer.
PS: Ignore the bad naming of the classes, I could not name EntityInstance and FieldInstance, Entity and Field because these as the classes I use to describe my models.
I found a fix to my problem :
The type matching that I was doing in the class should be done in the implicit Write !
class FieldInstance(fieldNamec: String, valuec: Any) {
val fieldName = fieldNamec
val value = valuec
override def toString(): String = "(" + fieldName + "," + value + ")";
}
object FieldInstance {
implicit val fieldInstanceWrites = new Writes[FieldInstance] {
def writes(fieldInstance: FieldInstance) =
fieldInstance.value match {
case v: Int => Json.obj(fieldInstance.fieldName -> v.asInstanceOf[Int])
case v: String => Json.obj(fieldInstance.fieldName -> v.asInstanceOf[String])
case v: Boolean => Json.obj(fieldInstance.fieldName -> v.asInstanceOf[Boolean])
case _ => Json.obj(fieldInstance.fieldName -> "Unsupported type")
}
}
}
This code now allows a user to create an EntityInstance with Fields of Any type :
val ei = new EntityInstance(Seq[FieldInstance](new FieldInstance("name", "George"), new FieldInstance("age", 25), new FieldInstance("married", true)))
println("-- TEST ENTITY INSTANCE TO JSON --")
println(Json.toJson(ei))
prints : {"entity":[{"name":"George"},{"age":25},{"married":true}]}
Here is my EntityInstance code if you are trying to test it :
case class EntityInstance(fields: Seq[FieldInstance])
object EntityInstance {
implicit val EntityInstanceWrites = new Writes[EntityInstance] {
def writes(entityInstance: EntityInstance) =
Json.obj("entity" -> entityInstance.fields)
}
}
It is returning a String, Int or Boolean but Json.obj is expecting the value parameter of type (String, JsValueWrapper)
def obj(fields: (String, JsValueWrapper)*): JsObject = JsObject(fields.map(f => (f._1, f._2.asInstanceOf[JsValueWrapperImpl].field)))
a quick fix could be to convert the matched value v with toJson provided the implicit Writes[T] for type T is available (which they are for String, Int and Boolean)
class FieldInstance(fieldNamec: String, valuec: Any) {
val fieldName = fieldNamec
val value = valuec match {
case v: Int => Json.toJson(v)
case v: String => Json.toJson(v)
case v: Boolean => Json.toJson(v)
case _ => Json.toJson("Unrecognized type")
}
}
If you'd like to see which DefaultWrites are available you can browse them in the play.api.libs.json package in trait DefaultWrites
for example:
/**
* Serializer for Boolean types.
*/
implicit object BooleanWrites extends Writes[Boolean] {
def writes(o: Boolean) = JsBoolean(o)
}