Klaxon: Expected a name but got LEFT_BRACE - json

How can I read a "complex" json using Klaxon?
I'm trying to use klaxon's stream api as documentation say .
I'm using the beginObject method. If I use a json as given in the example everything is fine
val objectString = """{
"name" : "Joe",
"age" : 23,
"flag" : true,
"array" : [1, 3],
"obj1" : { "a" : 1, "b" : 2 }
}"""
But if I try to parse a json with a nested object as the following example I get the following error: "Expected a name but got LEFT_BRACE"
val objectString = """{
"name" : "Joe",
"age" : 23,
"flag" : true,
"array" : [1, 3],
"obj1" : {
"hello": { "a" : 1, "b" : 2 }
}
}"""
I don't see any reported issues in the github repo, so I wonder if there is a way to get this working.
cheers

Ok so I checked out the source, and it seems that nextObject() assumes that you are dealing with a simple key value object, where values are not objects.
Anyway, there is another way to parse a JSON of the format you specified, like below:
val objectString = """{
"name" : "Joe",
"age" : 23,
"flag" : true,
"array" : [1, 2, 3],
"obj1" : { "hello": {"a" : 1, "b" : 2 } }
}"""
class ABObj(val a: Int, val b: Int)
class HelloObj(val hello: ABObj)
class Obj(val name: String, val age: Int, val flag: Boolean, val array: List<Any>, val obj1: HelloObj)
val r = Klaxon().parse<Obj>(objectString)
// r is your parsed object
print(r!!.name) // prints Joe
print(r!!.array) // prints [1, 2, 3]
The classes I created are like below:
ABObj represents this JSON:
{"a": 1, "b": 2}
HelloObj represents this JSON:
{"hello": {"a": 1, "b": 2}}
And finally, Obj refers to the top level object.
Hope this helps.

With the Streaming API, you are in charge of parsing objects and arrays yourself, so when you encounter the name "obj1", you need to start a new beginObject() { ... } and parse that object there.
The documentation that you linked does exactly that:
while (reader.hasNext()) {
val readName = reader.nextName()
when (readName) {
"name" -> name = reader.nextString()
"age" -> age = reader.nextInt()
"flag" -> flag = reader.nextBoolean()
"array" -> array = reader.nextArray()
"obj1" -> obj1 = reader.nextObject()

Related

F# Define Types for Json with nested list

Follow up question to this question, to parse json to deedle dataframe.
My json includes some list with nested lists:
apiResponse = {"data":
{"Shipments":
{"ErrorMessage":null,
"Success":true,
"ValidationResult":null,
"TotalCount":494,
"Data":[
{
"CompletedDate":"2021-04-12T10:13:13.436Z",
"ItemId":"dd4520bb-aa0a-...",
"Organization":{
"OrganizationId":"cac43a32-1f08-...",
"OrganizationName":"XXX"
},
"ShipmentContent" : [
{
"MaterialId": "cac43a32-1f08-..",
"unit": "kg",
"quantity": 2.0,
"labels" : ["A1", "A2", "A3"]
},
{
"MaterialId": "cac43333-1f08-",
"unit": "t",
"quantity": 0.4,
"labels" : ["B1", "B2", "B3"]
}
]
},
{......,
....}]}}}
My code so far:
type ApiTypes =
{
ItemId : Guid
CompletedDate : Nullable<DateTime>
Organization:
{|
OrganizationId : Guid
OrganizationName : string
|}
ShipmentContent : // ? How to define the List
{|
MaterialId : Guid
Quantity : Nullable<decimal>
Unit : string
Labels: List // ?
|}
}
How can I define these types? The next step would be:
let jsonObject = JObject.Parse(graphResponse)
let frame =
jsonObject.["data"].["Shipments"].["Data"].Children()
|> Seq.map (fun jtok -> jtok.ToObject<ApiTypes>())
|> Frame.ofRecords
Can I define all in one type, or do I have to define a seperate type for ShipmentContents?
I realy apreciate your help!
Fiddle here (doesn't work, but for sharing the code)
This should work if you want to have it all in one type:
type ApiTypes =
{
ItemId : Guid
CompletedDate : Nullable<DateTime>
Organization:
{|
OrganizationId : Guid
OrganizationName : string
|}
ShipmentContent :
{|
MaterialId : Guid
Quantity : Nullable<decimal>
Unit : string
Labels: string[]
|}[]
}
Note that I've defined the lists as arrays, since arrays are the collection type supported by JSON.

Scala Circe JSON Library - Understanding Implicit Encoder in an Example

I am working with the Scala Circe library. It seems very useful and I want to get better at using it.
One example I have is the following.
Consider the following code:
import io.circe.syntax._
import io.circe.{Json, Encoder}
import io.circe.generic.auto._
sealed trait JsonComponent
case class DataSequences(values: Map[String, List[Int]]) extends JsonComponent
case class DataField(data: List[DataSequences], `type`: String) extends JsonComponent
object Example extends App {
implicit val encodeDataSequences: Encoder[DataSequences] = new Encoder[DataSequences] {
final def apply(sequence: DataSequences): Json = sequence.values.asJson
}
val x = new DataSequences(Map("x" -> List(1, 2, 3), "y" -> List(1, 2, 4)))
val l = List(DataField(List(x, x), "abc"), DataField(List(x, x), "cde")).asJson
println(l)
}
This gives the following output:
[
{
"data" : [
{
"x" : [
1,
2,
3
],
"y" : [
1,
2,
4
]
},
{
"x" : [
1,
2,
3
],
"y" : [
1,
2,
4
]
}
],
"type" : "abc"
}
]
However, if I comment out the encodeDataSequences encoder definition, I get the following instead:
[
{
"data" : [
{
"x" : [
1,
2,
3
],
"y" : [
1,
2,
4
]
},
{
"x" : [
1,
2,
3
],
"y" : [
1,
2,
4
]
}
],
"type" : "abc"
}
]
So now this "values" appears. I do not want the "values" field to show up. I am not sure how the implicit is shaping the Json under the hood, and if someone could highlight what's going on that would be appreciated.
In addition, as a gernal thing, am I writing idiomatic Circe code using that implicit encoder, and if so is there a better way to do what I want?

Use circe to preprocess dot-notation style fields

I have some json that includes some fields that are being flattened into a bson-ish format as in {"foo.bar" : "bash"}. I'd like to transform this to the following representation {"foo" : { "bar" : "bash"}} and wondering where in circe I'd do such an operation. Complicating the problem is that there could be multiple such fields that need to be properly merged, e.g. {"foo.bar" : "a", "foo.bash" : "b", "foo.baz" : "c"} -> {"foo" : { "bar" : "a", "bash" : "b", "baz" : "c"}}.
Here's a quick implementation:
import io.circe.Json
val Dotted = "([^\\.]*)\\.(.*)".r
def expandDotted(j: Json): Json = j.arrayOrObject(
j,
js => Json.fromValues(js.map(expandDotted)),
_.toList.map {
case (Dotted(k, rest), v) => Json.obj(k -> expandDotted(Json.obj(rest -> v)))
case (k, v) => Json.obj(k -> expandDotted(v))
}.reduceOption(_.deepMerge(_)).getOrElse(Json.obj())
)
I haven't really used or tested it in detail, but it seems to work:
scala> import io.circe.literal._
import io.circe.literal._
scala> val j1 = json"""{"foo.bar" : "a", "foo.bash" : "b", "foo.baz" : "c"}"""
j1: io.circe.Json =
{
"foo.bar" : "a",
"foo.bash" : "b",
"foo.baz" : "c"
}
scala> expandDotted(j1)
res1: io.circe.Json =
{
"foo" : {
"baz" : "c",
"bash" : "b",
"bar" : "a"
}
}
And with deeper nesting:
scala> expandDotted(json"""{ "x.y.z": true, "a.b": { "c": 1 } }""")
res2: io.circe.Json =
{
"a" : {
"b" : {
"c" : 1
}
},
"x" : {
"y" : {
"z" : true
}
}
}
And just to confirm that it doesn't mess with undotted keys:
scala> expandDotted(json"""{ "a.b": true, "x": 1 }""").noSpaces
res3: String = {"x":1,"a":{"b":true}}
Note that in the case of "collisions" (paths that lead to both JSON objects and non-object JSON values, or to multiple non-object values), the behavior is that of Json#deepMerge:
scala> expandDotted(json"""{ "a.b": true, "a": 1 }""").noSpaces
res4: String = {"a":1}
scala> expandDotted(json"""{ "a": 1, "a.b": true }""").noSpaces
res5: String = {"a":{"b":true}}
…which is probably what you'd want, but you could also have it fail in these cases, or not expand the colliding path, or do pretty much any other thing you can think of.

How to ignore an item when generating the json string if the value is None?

I'm trying to use Argonaut to generate JSON string from a Scala instance.
import argonaut._, Argonaut._
case class Person(name: Option[String], age: Int, things: List[String])
implicit def PersonCodecJson =
casecodec3(Person.apply, Person.unapply)("name", "age", "things")
val person = Person(Some("Freewind"), 2, List("club"))
val json: Json = person.asJson
val prettyprinted: String = json.spaces2
It will generate:
{
"name" : "Freewind",
"age" : 2,
"things" : [
"club"
]
}
And when the name is None:
val person = Person(None, 2, List("club"))
It will generate:
{
"name" : null,
"age" : 2,
"things" : [
"club"
]
}
But actually I want it to be:
{
"age" : 2,
"things" : [
"club"
]
}
How to do it?
Resolved, the key is to define custom EncodeJson rule and use ->?: and field.map:
implicit def PersonCodecJson: EncodeJson[Person] = EncodeJson((p: Person) =>
p.name.map("name" := _) ->?: ("age" := p.age) ->: ("things" := p.things) ->: jEmptyObject)

Convert List to a specific JSON format in Play framework using Scala

I've a List of a type which contains three fields (services, total, improved). When I convert it to JSON using Json.toJson(myList), I get the JSON in the following format:
[
{
"services":"S4",
"total":1,
"improved":1
},
{
"services":"S1",
"total":2,
"improved":1
},
{
"services":"S2",
"total":3,
"improved":2
}
]
Using JSON library in Play 2.x in Scala, how can I convert myList in the following JSON format?
[
{
"key" : "total",
"values" :[
{
"services" : "s1",
"value" : 2
},
"services" : "s2",
"value" : 3
{
"services" : "s4",
"value" : 1
}
]
},
{
"key" : "improved",
"values" :[
{
"services" : "s1",
"value" : 1
},
"services" : "s2",
"value" : 2
{
"services" : "s4",
"value" : 1
}
]
}
]
Thanks in advance.
Since you are already dealing with OO, I think you could wrap your List into a more specific object and convert it to JSON :
case class Foo(services: String, total: Int, improved: Int)
case class B(key: String, value: Int)
case class A(key: String, values: Seq[B] = Seq())
val myOriginalList = Seq(Foo("S4", 1, 1), Foo("S1", 2, 1), Foo("S2", 3, 2))
val transformedList = myOriginalList.foldLeft((A("total"), A("improved")))({ (res, x) =>
(res._1.copy(values = B(x.services, x.total) +: res._1.values),
res._2.copy(values = B(x.services, x.improved) +: res._2.values))
}).map({x => List(x._1, x._2)})
Json.toJson(transformedList)
One of the problem (or maybe not) using this solution, is that you don't resolve the Foo attributes dynamically.
You could also try yourself around Json Transformers : http://www.playframework.com/documentation/2.2.x/ScalaJsonTransformers
Here's a Scala newbie's solution (I'm sure it's not elegant):
I'm using a solution provided here: http://www.playframework.com/documentation/2.1.1/ScalaJson
//some data
case class PatientAggregateValues(total: Int, improved: Int)
val scoreMap = Map.empty[String, PatientAggregateValues]
scoreMap += ("S1" -> PatientAggregateValues(2, 1))
scoreMap += ("S2" -> PatientAggregateValues(3, 2))
scoreMap += ("S4" -> PatientAggregateValues(1, 1))
//main logic
val totalMap = scoreMap map { case (k,v) => Json.toJson(scala.collection.immutable.Map("service" -> Json.toJson(k), "value" -> Json.toJson(v.total))) } toSeq
val improvedMap = scoreMap map { case (k,v) => Json.toJson(scala.collection.immutable.Map("service" -> Json.toJson(k), "value" -> Json.toJson(v.improved))) } toSeq
Json.toJson(scala.collection.immutable.Map("total" -> totalMap, "improved" -> improvedMap))
Thanks.