Suppose I have such json
{
"sha": "some sha",
"parents": [{
"url": "some url",
"sha": "some parent sha"
}]
}
and such case class
case class Commit(sha: String, parentShas: List[String])
In play-json I could write the reads like this:
val commitReads: Reads[Commit] = (
(JsPath \ "sha").read[String] and
(JsPath \ "parents" \\ "sha").read[List[String]]
)(Commit.apply _)
I'm looking for a equivalent way of decoding only the "sha" of "parent" in argonaut/circe but I haven't found any. "HCursor/ACursor" has downArray but from there on I don't know what to do. Thank you very much in advance!
Neither circe nor Argonaut keeps track of which fields have been read in JSON objects, so you can just ignore the extra "url" field (just as in Play). The trickier part is finding the equivalent of Play's \\, which circe doesn't have at the moment, although you've convinced me we need to add it.
First of all, this is relatively easy if you have a separate SHA type:
import io.circe.Decoder
val doc = """
{
"sha": "some sha",
"parents": [{
"url": "some url",
"sha": "some parent sha"
}]
}
"""
case class Sha(value: String)
object Sha {
implicit val decodeSha: Decoder[Sha] = Decoder.instance(_.get[String]("sha")).map(Sha(_))
}
case class Commit(sha: Sha, parentShas: List[Sha])
object Commit {
implicit val decodeCommit: Decoder[Commit] = for {
sha <- Decoder[Sha]
parents <- Decoder.instance(_.get[List[Sha]]("parents"))
} yield Commit(sha, parents)
}
Or, using Cats's applicative syntax:
import cats.syntax.cartesian._
implicit val decodeCommit: Decoder[Commit] =
(Decoder[Sha] |#| Decoder.instance(_.get[List[Sha]]("parents"))).map(Commit(_, _))
And then:
scala> import io.circe.jawn._
import io.circe.jawn._
scala> decode[Commit](doc)
res0: cats.data.Xor[io.circe.Error,Commit] = Right(Commit(Sha(some sha),List(Sha(some parent sha))))
But that's not really an answer, since I'm not going to ask you to change your model. :) The actual answer is a bit less fun:
case class Commit(sha: String, parentShas: List[String])
object Commit {
val extractSha: Decoder[String] = Decoder.instance(_.get[String]("sha"))
implicit val decodeCommit: Decoder[Commit] = for {
sha <- extractSha
parents <- Decoder.instance(c =>
c.get("parents")(Decoder.decodeCanBuildFrom[String, List](extractSha, implicitly))
)
} yield Commit(sha, parents)
}
This is bad, and I'm ashamed it's necessary, but it works. I've just filed an issue to make sure this gets better in a future circe release.
Related
I'm trying to parse a json string of search results of the following form:
"""
{
"metadata": [
"blah",
"unimportant"
],
"result": [
{
"type": "one",
"title": "this",
"weird": "attribute unique to this type",
"other": 7
},
{
"type": "two",
"title": "that",
"another_weird": "a different attribute unique to this second type",
"another": "you get the idea"
},
{
"type": "one",
"title": "back to this type, which has the same fields as the other element of this type",
"weird": "yep",
"other": 8
}
...
]
}
"""
There is a known, fixed number of result element types (given by the type field), each of which correspond to a unique schema. For a given search request, there can be any number of each type returned in any order (up to some fixed total).
Writing out the case classes and Reads implicits for each type is easy enough, but my question is around the best way to handle the variability in the result sequence... pattern matching seems the obvious choice, but I'm just not sure where or how with the play framework. Thanks in advance!
EDIT: Per the suggestion in the comments, I gave it a go by attempting to read the result sequence as subtypes of a common base trait, but that didn't quite work. The following compiles, but doesn't parse the example json correctly. Any additional suggestions are welcome.
sealed trait Parent {def title: String}
case class One(`type`: String, title: String, weird: String, other: Int) extends Parent
case class Two(`type`: String, title: String, another_weird: String, another: String) extends Parent
case class SearchResponse(result: Seq[Parent], metadata: Seq[String])
implicit val oneReader = Json.reads[One]
implicit val twoReader = Json.reads[Two]
implicit val parentReader = Json.reads[Parent]
implicit val searchReader = (
(__ \ "result").read[Seq[Parent]] and (__ \ "metadata").read[Seq[String]]
)(SearchResponse.apply _)
val searchResult = Json.fromJson[SearchResponse](json)
print(searchResult)
Define implicit JsonConfiguration
import play.api.libs.json._
sealed trait Parent {
def title: String
}
case class One(title: String, weird: String, other: Int) extends Parent
case class Two(title: String, another_weird: String, another: String) extends Parent
case class SearchResponse(result: Seq[Parent], metadata: Seq[String])
implicit val cfg = JsonConfiguration(
discriminator = "type",
typeNaming = JsonNaming(_.toLowerCase)
)
implicit val oneReader = Json.reads[One]
implicit val twoReader = Json.reads[Two]
implicit val parentReader = Json.reads[Parent]
implicit val searchReader = Json.reads[SearchResponse]
val searchResult = Json.fromJson[SearchResponse](json)
println(searchResult)
// JsSuccess(SearchResponse(List(One(this,attribute unique to this type,7), Two(that,a different attribute unique to this second type,you get the idea), One(back to this type, which has the same fields as the other element of this type,yep,8)),List(blah, unimportant)),)
https://www.playframework.com/documentation/2.8.x/ScalaJsonAutomated#Custom-Naming-Strategies
I am writing a small scala practice code where my input is going to be in the fashion -
{
"code": "",
"unique ID": "",
"count": "",
"names": [
{
"Matt": {
"name": "Matt",
"properties": [
"a",
"b",
"c"
],
"fav-colour": "red"
},
"jack": {
"name": "jack",
"properties": [
"a",
"b"
],
"fav-colour": "blue"
}
}
]
}
I'll be passing this file as an command line argument.
I want to know that how do I accept the input file parse the json and use the json keys in my code?
You may use a json library such as play-json to parse the json content.
You could either operate on the json AST or you could write case classes that have the same structure as your json file and let them be parsed.
You can find the documentation of the library here.
You'll first have to add playjson as depedency to your project. If you're using sbt, just add to your build.sbt file:
libraryDependencies += "com.typesafe.play" %% "play-json" % "2.6.13"
Play json using AST
Let's read the input file:
import play.api.libs.json.Json
object Main extends App {
// first we'll need a inputstream of your json object
// this should be familiar if you know java.
val in = new FileInputStream(args(0))
// now we'll let play-json parse it
val json = Json.parse(in)
}
Let's extract some fields from the AST:
val code = (json \ "code").as[String]
val uniqueID = (json \ "unique ID").as[UUID]
for {
JsObject(nameMap) ← (json \ "names").as[Seq[JsObject]]
(name, userMeta) ← nameMap // nameMap is a Map[String, JsValue]
} println(s"User $name has the favorite color ${(userMeta \ "fav-colour").as[String]}")
Using Deserialization
As I've just described, we may create case classes that represent your structure:
case class InputFile(code: String, `unique ID`: UUID, count: String, names: Seq[Map[String, UserData]])
case class UserData(name: String, properties: Seq[String], `fav-colour`: String)
In addition you'll need to define an implicit Format e.g. in the companion object of each case class. Instead of writing it by hand you can use the Json.format macro that derives it for you:
object UserData {
implicit val format: OFormat[UserData] = Json.format[UserData]
}
object InputFile {
implicit val format: OFormat[InputFile] = Json.format[InputFile]
}
You can now deserialize your json object:
val argumentData = json.as[InputFile]
I generally prefer this approach but in your case the json structure does not fit really well. One improvement could be to add an additional getter to your InputFile class that makes accesing the fields with space and similar in the name easier:
case class InputFile(code: String, `unique ID`: UUID, count: String, names: Seq[Map[String, String]]) {
// this method is nicer to use
def uniqueId = `unique ID`
}
Suppose I have a case class like the following, and I want to decode a JSON object into it, with all of the fields that haven't been used ending up in a special member for the leftovers:
import io.circe.Json
case class Foo(a: Int, b: String, leftovers: Json)
What's the best way to do this in Scala with circe?
(Note: I've seen questions like this a few times, so I'm Q-and-A-ing it for posterity.)
There are a couple of ways you could go about this. One fairly straightforward way would be to filter out the keys you've used after decoding:
import io.circe.{ Decoder, Json, JsonObject }
implicit val decodeFoo: Decoder[Foo] =
Decoder.forProduct2[Int, String, (Int, String)]("a", "b")((_, _)).product(
Decoder[JsonObject]
).map {
case ((a, b), all) =>
Foo(a, b, Json.fromJsonObject(all.remove("a").remove("b")))
}
Which works as you'd expect:
scala> val doc = """{ "something": false, "a": 1, "b": "abc", "0": 0 }"""
doc: String = { "something": false, "a": 1, "b": "abc", "0": 0 }
scala> io.circe.jawn.decode[Foo](doc)
res0: Either[io.circe.Error,Foo] =
Right(Foo(1,abc,{
"something" : false,
"0" : 0
}))
The disadvantage of this approach is that you have to maintain code to remove the keys you've used separately from their use, which can be error-prone. Another approach is to use circe's state-monad-powered decoding tools:
import cats.data.StateT
import cats.instances.either._
import io.circe.{ ACursor, Decoder, Json }
implicit val decodeFoo: Decoder[Foo] = Decoder.fromState(
for {
a <- Decoder.state.decodeField[Int]("a")
b <- Decoder.state.decodeField[String]("b")
rest <- StateT.inspectF((_: ACursor).as[Json])
} yield Foo(a, b, rest)
)
Which works the same way as the previous decoder (apart from some small differences in the errors you'll get if decoding fails):
scala> io.circe.jawn.decode[Foo](doc)
res1: Either[io.circe.Error,Foo] =
Right(Foo(1,abc,{
"something" : false,
"0" : 0
}))
This latter approach doesn't require you to change the used fields in multiple places, and it also has the advantage of looking a little more like any other decoder you'd write manually in circe.
I'm working on consuming an API which exposes an object at multiple layers within it's response. For example for some responses we get back a:
{
"error": {
"code": "123",
"description": "Description"
}
}
But in other situations it response with:
{
"data": [
{
"message_id": "123",
"error": {
"code": "123",
"description": "Description"
}
}
]
}
In both cases the error object is identical and in both case I don't actually care about the rest of the payload. I was hoping to use the \\ recursive JsPath operator, however the implementation below fails:
case class ErrorMessage(code: String, description: String)
implicit val errorMessageFormat = Json.format[ErrorMessage]
case class ErrorResponse(errors: Seq[ErrorMessage])
implicit val errorResponseFormat: Format[ErrorResponse] = Format(
(__ \\ "error").read[Seq[ErrorMessage]].map(ErrorResponse),
(__ \ "errors").write[Seq[ErrorMessage]].contramap((r: ErrorResponse) => r.errors)
)
This gives an error:
JsError(List((//error,List(ValidationError(List(error.expected.jsarray),WrappedArray())))))
I understand why: The (__ \\ "error") returns a Seq[JsValue], where as my read call is expecting a JsArray.
Is there a nice way a round this?
Since the first piece is already a Seq, just map the internal elements as you would map single objects. I'm not very familiar with the framework (I use Json4s myself mostly), but by your description sounds like
ErrorResponse((__ \\ "error").map(_.read[Seq[ErrorMessage]]))
Should be closer to what you want.(__ \\ "error") gives you a Seq of JsValues, map does something for every JsValue, read converts a single JsValue to an ErrorMessage and the resulting Seq[ErrorMessage] you pass to the ErrorResponse constructor.
So for anyone trying to do something similar, the below works.
val errorResponseReads = Reads[ErrorResponse] { value: JsValue =>
val errorsJsArray: JsArray = JsArray(value \\ "error")
errorsJsArray.validate[Seq[ErrorMessage]].map(ErrorResponse)
}
The format then becomes:
implicit val errorResponseFormat: Format[ErrorResponse] = Format(
errorResponseReads,
(__ \ "errors").write[Seq[ErrorMessage]].contramap((r: ErrorResponse) => r.errors)
)
Basically you need to define Reads that you use explicitly. In the reads you can then use the recursive search to return a Seq[JsValue] and then create a JsArray that can be validated as normal.
This works fine, it would be nice if we didn't have to define a separate Reads[T] though.
I think what I'm trying to do should be relatively simple, but I'm not certain. I'm making an API call to get a list of applications and I'd like to read them out. However, the list is nested into the response. The response looks something like:
{
"response": {
"instances": [
{ /* object I'm concerned with reading */ },
...
]
}
}
I have a reader currently defined as:
implicit val appReader : Reads[App] = (
(__ \ "ip_address").read[String] and
(__ \ "hostname").read[String] and
(__ \ "application_version").read[String]
)(App)
And a class of:
case class App(
ip: String,
hostname: String,
version: String
)
However, I'm not sure of how to get at the data since it is buried in the response or if there is a way that I can discard that data.
I assume that the applications lie within the instances array like this:
val js = Json.parse("""
{
"response": {
"instances": [
{ "ip_address": "192.168.1.1", "hostname": "host1", "application_version": "1.0"},
{ "ip_address": "192.168.1.2", "hostname": "host2", "application_version": "1.0"}
]
}
}
""")
Now, to read the list of applications defined as App:
val responseRead =
(__ \ "response").read(
(__ \ "instances").read[List[App]])
val apps = responseRead.reads(js).get
This gives you the list of applications leaving out the rest. If the data structure is somewhat different than assumed, this should point you in the right direction.
Edit: Most of the time I prefer to use the Json macro inception of play. The code for your example could be written as follows:
case class App(
ip_address: String,
hostname: String,
application_version: String)
object App {
implicit val appFormat = Json.format[App]
}
case class Response(
instances: List[App])
object Response {
implicit val responseFormat = Json.format[Response]
}
case class Root(
response: Response)
object Root {
implicit val rootFormat = Json.format[Root]
}
val root = js.as[Root]
val apps = root.response.instances
As you can see, the case class vals must have the same name as their Json counter parts. Additionally, I used format instead of reads which is of course up to you, but most of the time you need both reads and writes which is what format gives you.