Suppose I want to decode some values from a JSON array into a case class with circe. The following works just fine:
scala> import io.circe.generic.auto._, io.circe.jawn.decode
import io.circe.generic.auto._
import io.circe.jawn.decode
scala> case class Foo(name: String)
defined class Foo
scala> val goodDoc = """[{ "name": "abc" }, { "name": "xyz" }]"""
goodDoc: String = [{ "name": "abc" }, { "name": "xyz" }]
scala> decode[List[Foo]](goodDoc)
res0: Either[io.circe.Error,List[Foo]] = Right(List(Foo(abc), Foo(xyz)))
It's sometimes the case that the JSON array I'm decoding contains other, non-Foo-shaped stuff, though, which results in a decoding error:
scala> val badDoc =
| """[{ "name": "abc" }, { "id": 1 }, true, "garbage", { "name": "xyz" }]"""
badDoc: String = [{ "name": "abc" }, { "id": 1 }, true, "garbage", { "name": "xyz" }]
scala> decode[List[Foo]](badDoc)
res1: Either[io.circe.Error,List[Foo]] = Left(DecodingFailure(Attempt to decode value on failed cursor, List(DownField(name), MoveRight, DownArray)))
How can I write a decoder that ignores anything in the array that can't be decoded into my case class?
The most straightforward way to solve this problem is to use a decoder that first tries to decode each value as a Foo, and then falls back to the identity decoder if the Foo decoder fails. The new either method in circe 0.9 makes the generic version of this practically a one-liner:
import io.circe.{ Decoder, Json }
def decodeListTolerantly[A: Decoder]: Decoder[List[A]] =
Decoder.decodeList(Decoder[A].either(Decoder[Json])).map(
_.flatMap(_.left.toOption)
)
It works like this:
scala> val myTolerantFooDecoder = decodeListTolerantly[Foo]
myTolerantFooDecoder: io.circe.Decoder[List[Foo]] = io.circe.Decoder$$anon$21#2b48626b
scala> decode(badDoc)(myTolerantFooDecoder)
res2: Either[io.circe.Error,List[Foo]] = Right(List(Foo(abc), Foo(xyz)))
To break down the steps:
Decoder.decodeList says "define a list decoder that tries to use the given decoder to decode each JSON array value".
Decoder[A].either(Decoder[Json] says "first try to decode the value as an A, and if that fails decode it as a Json value (which will always succeed), and return the result (if any) as a Either[A, Json]".
.map(_.flatMap(_.left.toOption)) says "take the resulting list of Either[A, Json] values and remove all the Rights".
…which does what we want in a fairly concise, compositional way. At some point we might want to bundle this up into a utility method in circe itself, but for now writing out this explicit version isn't too bad.
Related
I have a JSON array containing the following details, I would like to extract the text Alignment value of Right and assign it to a val.
"data":[
{
"formatType": "text",
"value": "bgyufcie huis huids hufhsduhfsl hd"
},
{
"formatType": "text size",
"value": 12
},
{
"formatType": "text alignment",
"value" : "right"
}
]
Any thoughts?
Using the Gson library, you can map the json in a Java object.
So, you have to create a class like this:
public class MyObject{
private String formatType;
private String value;
//Constuctors, Getter and Setter...
//.....
//.....
}
After, using the method fromJson you can create an array of MyObject.
Gson gson = new Gson();
MyObject[] array = gson.fromJson(new FileReader("file.json"), MyObject[].class);
You can also use json4s library as shown next:
import org.json4s._
import org.json4s.jackson.JsonMethods._
val json = """{
"data":[
{
"formatType": "text",
"value": "bgyufcie huis huids hufhsduhfsl hd"
},
{
"formatType": "text size",
"value": 12
},
{
"formatType": "text alignment",
"value" : "right"
}
]
}"""
val parsed = parse(json)
val value = (parsed \ "data" \\ classOf[JObject]).filter(m => m("formatType") == "text alignment")(0)("value")
// value: Any = right
The filter (parsed \ "data" \\ classOf[JObject]) extracts all the items into a List of Map i.e:
List(
Map(formatType -> text, value -> bgyufcie huis huids hufhsduhfsl hd),
Map(formatType -> text size, value -> 12), Map(formatType -> text alignment, value -> right)
).
From those we apply the filter filter(m => m("formatType") == "text alignment") to retrieve the record that we really need.
Use Dijon FTW!
Here is a test that demonstrates how easily the "right" value can be found in samples like yours:
import com.github.pathikrit.dijon._
val json = parse(
"""{
|"data":[
| {
| "formatType": "text",
| "value": "bgyufcie huis huids hufhsduhfsl hd"
| },
| {
| "formatType": "text size",
| "value": 12
| },
| {
| "formatType": "text alignment",
| "value" : "right"
| }
|]
|}""".stripMargin)
assert(json.data.toSeq.collect {
case obj if obj.formatType == "text alignment" => obj.value
}.head == "right")
I would use the Jackson library, it is very helpful for parsing JSON. You can read the JSON using an ObjectMapper.
Here is a full tutorial to get you started: https://www.mkyong.com/java/jackson-how-to-parse-json/
create a multiline JSON string, then parse that string directly into a Scala object, use the net.liftweb package to solve this.
import net.liftweb.json._
object SarahEmailPluginConfigTest {
implicit val formats = DefaultFormats
case class Mailserver(url: String, username: String, password: String)
val json = parse(
"""
{
"url": "imap.yahoo.com",
"username": "myusername",
"password": "mypassword"
}
"""
)
def main(args: Array[String]) {
val m = json.extract[Mailserver]
println(m.url)
println(m.username)
println(m.password)
}
}
https://alvinalexander.com/scala/simple-scala-lift-json-example-lift-framework
Reference link
Suppose I have some JSON data like this:
{
"data": {
"title": "example input",
"someBoolean": false,
"innerData": {
"innerString": "input inner string",
"innerBoolean": true,
"innerCollection": [1,2,3,4,5]
},
"collection": [6,7,8,9,0]
}
}
And I want to flatten it a bit and transform or remove some fields, to get the following result:
{
"data": {
"ttl": "example input",
"bool": false,
"collection": [6,7,8,9,0],
"innerCollection": [1,2,3,4,5]
}
}
How can I do this with Circe?
(Note that I'm asking this as a FAQ since similar questions often come up in the Circe Gitter channel. This specific example is from a question asked there yesterday.)
I've sometimes said that Circe is primarily a library for encoding and decoding JSON, not for transforming JSON values, and in general I'd recommend mapping to Scala types and then defining relationships between those (as Andriy Plokhotnyuk suggests here), but for many cases writing transformations with cursors works just fine, and in my view this kind of thing is one of them.
Here's how I'd implement this transformation:
import io.circe.{DecodingFailure, Json, JsonObject}
import io.circe.syntax._
def transform(in: Json): Either[DecodingFailure, Json] = {
val someBoolean = in.hcursor.downField("data").downField("someBoolean")
val innerData = someBoolean.delete.downField("innerData")
for {
boolean <- someBoolean.as[Json]
collection <- innerData.get[Json]("innerCollection")
obj <- innerData.delete.up.as[JsonObject]
} yield Json.fromJsonObject(
obj.add("boolean", boolean).add("collection", collection)
)
}
And then:
val Right(json) = io.circe.jawn.parse(
"""{
"data": {
"title": "example input",
"someBoolean": false,
"innerData": {
"innerString": "input inner string",
"innerBoolean": true,
"innerCollection": [1,2,3]
},
"collection": [6,7,8]
}
}"""
)
And:
scala> transform(json)
res1: Either[io.circe.DecodingFailure,io.circe.Json] =
Right({
"data" : {
"title" : "example input",
"collection" : [
6,
7,
8
]
},
"boolean" : false,
"collection" : [
1,
2,
3
]
})
If you look at it the right way, our transform method kind of resembles a decoder, and we can actually write it as one (although I'd definitely recommend not making it implicit):
import io.circe.{Decoder, Json, JsonObject}
import io.circe.syntax._
val transformData: Decoder[Json] = { c =>
val someBoolean = c.downField("data").downField("someBoolean")
val innerData = someBoolean.delete.downField("innerData")
(
innerData.delete.up.as[JsonObject],
someBoolean.as[Json],
innerData.get[Json]("innerCollection")
).mapN(_.add("boolean", _).add("collection", _)).map(Json.fromJsonObject)
}
This can be convenient in some situations where you want to perform the transformation as part of a pipeline that expects a decoder:
scala> io.circe.jawn.decode(myJsonString)(transformData)
res2: Either[io.circe.Error,io.circe.Json] =
Right({
"data" : {
"title" : "example input",
"collection" : [ ...
This is also potentially confusing, though, and I've thought about adding some kind of Transformation type to Circe that would encapsulate transformations like this without questionably repurposing the Decoder type class.
One nice thing about both the transform method and this decoder is that if the input data doesn't have the expected shape, the resulting error will include a history that points to the problem.
I have the following JSON file to be parsed into a case class:
{
"root": {
"nodes": [{
"id": "1",
"attributes": {
"name": "Node 1",
"size": "3"
}
},
{
"id": "2",
"attributes": {
"value": "4",
"name": "Node 2"
}
}
]
}
}
The problem is that the attributes could have any value inside it: name, size, value, anything ...
At this moment I have defined my case classes:
case class Attributes(
name: String,
size: String,
value: Sting
)
case class Nodes(
id: String,
attributes: Attributes
)
case class Root(
nodes: List[Nodes]
)
case class R00tJsonObject(
root: Root
)
Whats is the best way to deal with this scenario when I can receive any attribute ?
Currently I am using Json4s to handle son files.
Thanks!
Your attributes are arbitrarily many and differently named, but it seems you can store them in a Map[String, String] (at least, if those examples are anything to go by). In this case, using circe-parser (https://circe.github.io/circe/parsing.html), you could simply use code along these lines in order to convert your JSON directly into a simple case-class:
import io.circe._, io.circe.parser._
import io.circe.generic.semiauto._
case class Node(id: String, attributes: Map[String,String])
case class Root(nodes: List[Node])
implicit val nodeDecoder: Decoder[Node] = deriveDecoder[Node]
implicit val nodeEncoder: Encoder[Node] = deriveEncoder[Node]
implicit val rootDecoder: Decoder[Root] = deriveDecoder[Root]
implicit val rootEncoder: Encoder[Root] = deriveEncoder[Root]
def myParse(jsonString: String) = {
val res = parse(jsonString) match {
case Right(json) => {
val cursor = json.hcursor
cursor.get[Root]("root")
}
case _ => Left("Wrong JSON!")
}
println(res)
}
This snippet will print
Right(Root(List(Node(1,Map(name -> Node 1, size -> 3)), Node(2,Map(value -> 4, name -> Node 2)))))
on the console, for the JSON, you've given. (Assuming, the solution doesn't have to be in Json4s.)
I'm try to get the head from the keys of JsValue type in Scala. I googled a lot to know how to get the head key from JsValue type.
Finally, I found that result.keys.head is the way to get the head key, but it throws error value keys is not a member of play.api.libs.json.JsValue.
And my result variable has the below form of data:
{
"intents": [{
"intent": "feeling",
"confidence": 0.1018563217175903
}],
"entities": [],
"input": {
"text": "{reset-encounter}"
},
"output": "Good"
}
Code:
import play.api.libs.json._
val jsonStr = """
{
"intents": [{
"intent": "feeling",
"confidence": 0.1018563217175903
}],
"entities": [],
"input": {
"text": "{reset-encounter}"
},
"output": "Good"
}
"""
val result = Json.parse(jsonStr)
println("key: ", result.keys.head)
At result.keys.head line, throws error.
I'm not sure but I think, may be I'm doing something wrong here.
Json.parse produces a JsValue, which could represent any type of json object (boolean, number, array, etc). If you know you're working with an object, you can use .as[JsObject]:
import play.api.libs.json._
val result = Json.parse(jsonStr).as[JsObject]
println("key: " + result.keys.head)
What are you trying to get? That's not the way to deal with play.api.Json objects.
.keys would result in a Map, not in a JsValue.
Check the documentation: https://www.playframework.com/documentation/2.5.x/ScalaJson
If you want to access a specific key (https://www.playframework.com/documentation/2.5.x/ScalaJson#Traversing-a-JsValue-structure) you should try:
result \ "keyName"
or for a recursive search:
result \\ "keyName"
How do I read multiple json objects from a request and convert them into my own custom object. For example, lets say we are retrieving a list of users with the following json:
{
"users":[
{
"name": "Bob",
"age": 31.0,
"email": "bob#gmail.com"
},
{
"name": "Kiki",
"age": 25.0,
"email": null
}
]
}
and my case class looks like the following
case class User(firstName: String, age: Double, email: String)
In this case the firstName value is differnt from the "name" json value. How do I get a Seq[User] from the given json file with the different names. I can't find any examples where someone is reading from a json file with multiple objects.
Thanks in advance.
Play's type class-based JSON library provides reader and writer instances for Seq[A] for any A with the appropriate instances, so all you have to do is provide a reader instance for your user type and you get a reader for a collection of users for free.
We can start with the following case class (note that I've made the fact that the email address is optional explicit with Option):
case class User(firstName: String, age: Double, email: Option[String])
Next we define our reader. I'm using 2.1's combinators, but there are other (more verbose) options.
import play.api.libs.json._
import play.api.libs.functional.syntax._
implicit val userReads = (
(__ \ 'name).read[String] and
(__ \ 'age).read[Double] and
(__ \ 'email).read[Option[String]]
)(User.apply _)
We can test it:
val userJson = """{
"users": [
{ "name": "Bob", "age": 31.0, "email": "bob#gmail.com" },
{ "name": "Kiki", "age": 25.0, "email": null }
]
}"""
val users = (Json.parse(userJson) \ "users").as[Seq[User]]
And then:
scala> users foreach println
User(Bob,31.0,Some(bob#gmail.com))
User(Kiki,25.0,None)
As expected.