Discard wrapper data when using Reads[T] - json

I think what I'm trying to do should be relatively simple, but I'm not certain. I'm making an API call to get a list of applications and I'd like to read them out. However, the list is nested into the response. The response looks something like:
{
"response": {
"instances": [
{ /* object I'm concerned with reading */ },
...
]
}
}
I have a reader currently defined as:
implicit val appReader : Reads[App] = (
(__ \ "ip_address").read[String] and
(__ \ "hostname").read[String] and
(__ \ "application_version").read[String]
)(App)
And a class of:
case class App(
ip: String,
hostname: String,
version: String
)
However, I'm not sure of how to get at the data since it is buried in the response or if there is a way that I can discard that data.

I assume that the applications lie within the instances array like this:
val js = Json.parse("""
{
"response": {
"instances": [
{ "ip_address": "192.168.1.1", "hostname": "host1", "application_version": "1.0"},
{ "ip_address": "192.168.1.2", "hostname": "host2", "application_version": "1.0"}
]
}
}
""")
Now, to read the list of applications defined as App:
val responseRead =
(__ \ "response").read(
(__ \ "instances").read[List[App]])
val apps = responseRead.reads(js).get
This gives you the list of applications leaving out the rest. If the data structure is somewhat different than assumed, this should point you in the right direction.
Edit: Most of the time I prefer to use the Json macro inception of play. The code for your example could be written as follows:
case class App(
ip_address: String,
hostname: String,
application_version: String)
object App {
implicit val appFormat = Json.format[App]
}
case class Response(
instances: List[App])
object Response {
implicit val responseFormat = Json.format[Response]
}
case class Root(
response: Response)
object Root {
implicit val rootFormat = Json.format[Root]
}
val root = js.as[Root]
val apps = root.response.instances
As you can see, the case class vals must have the same name as their Json counter parts. Additionally, I used format instead of reads which is of course up to you, but most of the time you need both reads and writes which is what format gives you.

Related

Scala - how to take json as input arguments and parse it?

I am writing a small scala practice code where my input is going to be in the fashion -
{
"code": "",
"unique ID": "",
"count": "",
"names": [
{
"Matt": {
"name": "Matt",
"properties": [
"a",
"b",
"c"
],
"fav-colour": "red"
},
"jack": {
"name": "jack",
"properties": [
"a",
"b"
],
"fav-colour": "blue"
}
}
]
}
I'll be passing this file as an command line argument.
I want to know that how do I accept the input file parse the json and use the json keys in my code?
You may use a json library such as play-json to parse the json content.
You could either operate on the json AST or you could write case classes that have the same structure as your json file and let them be parsed.
You can find the documentation of the library here.
You'll first have to add playjson as depedency to your project. If you're using sbt, just add to your build.sbt file:
libraryDependencies += "com.typesafe.play" %% "play-json" % "2.6.13"
Play json using AST
Let's read the input file:
import play.api.libs.json.Json
object Main extends App {
// first we'll need a inputstream of your json object
// this should be familiar if you know java.
val in = new FileInputStream(args(0))
// now we'll let play-json parse it
val json = Json.parse(in)
}
Let's extract some fields from the AST:
val code = (json \ "code").as[String]
val uniqueID = (json \ "unique ID").as[UUID]
for {
JsObject(nameMap) ← (json \ "names").as[Seq[JsObject]]
(name, userMeta) ← nameMap // nameMap is a Map[String, JsValue]
} println(s"User $name has the favorite color ${(userMeta \ "fav-colour").as[String]}")
Using Deserialization
As I've just described, we may create case classes that represent your structure:
case class InputFile(code: String, `unique ID`: UUID, count: String, names: Seq[Map[String, UserData]])
case class UserData(name: String, properties: Seq[String], `fav-colour`: String)
In addition you'll need to define an implicit Format e.g. in the companion object of each case class. Instead of writing it by hand you can use the Json.format macro that derives it for you:
object UserData {
implicit val format: OFormat[UserData] = Json.format[UserData]
}
object InputFile {
implicit val format: OFormat[InputFile] = Json.format[InputFile]
}
You can now deserialize your json object:
val argumentData = json.as[InputFile]
I generally prefer this approach but in your case the json structure does not fit really well. One improvement could be to add an additional getter to your InputFile class that makes accesing the fields with space and similar in the name easier:
case class InputFile(code: String, `unique ID`: UUID, count: String, names: Seq[Map[String, String]]) {
// this method is nicer to use
def uniqueId = `unique ID`
}

Play Scala JSON - conditionally add field to JSON object in Writes

In our app, we have pretty complex structure of objects that is getting converted to JSON and back. Until now most of the formatted are symmetrical (except some very specific cases, and even these for security reasons).
Now we are facing a more complex case where conversion of an object into JSON (writes) needs to create an additional field at a time of conversion while the case class does not have that field.
For example, here is one of our existing formatters:
case class ChecklistColumn(kind: ColumnKind.Value, descriptor: Descriptor.Value, data: JsValue) extends Column
implicit val checklistResultChecklistDataFormat: Format[ChecklistColumn] = (
(__ \ "kind").format[ColumnKind.Value] and
(__ \ "descriptor").format[Descriptor.Value] and
(__ \ "data").format[JsValue]
)(ChecklistColumn.apply, unlift(ChecklistColumn.unapply))
This one creates a json that will looks like:
{
"kind": <String>,
"descriptor": <String>,
"data": <JsValue>
}
What we need to achieve is:
{
"kind": <String>,
"descriptor": <String>,
"data": <JsValue>,
"normalized_data": <JsString>
}
But, only in case when the data is type of JsString (in any other case normalized_data can be left empty, byt ideally should not even exist).
I do understand that we have to create separate Reads & Writes for that.
But, I am not sure, how to implement the logic that will react differently to a different type of data.
Of course, there is always an option to create fully custom writes:
override def writes(column: ChecklistColumn): JsValue = {...}
But, this will create a huge complexity in a code that will be hard to maintain.
What is the cleanest way to implement something like that?
Have a look at ScalaJsonTransformers. You can create a transformer that creates the normalised field from a string data value and use it to, erm, transform your original Format to a new Writes. Here's a slightly simplified example that could doubtless be improved (you'll want to check various edge cases):
case class ChecklistColumn(kind: String, descriptor: String, data: JsValue)
// The original format.
val checklistFormat: Format[ChecklistColumn] = (
(__ \ "kind").format[String] and
(__ \ "descriptor").format[String] and
(__ \ "data").format[JsValue]
)(ChecklistColumn.apply, unlift(ChecklistColumn.unapply))
// A transformer that looks for a "data" field with a string
// value and adds the normalized_data field if it finds one.
val checklistTransformer: Reads[JsObject] = JsPath.json.update(
(__ \ "data").read[String].flatMap (
str => (__ \ "normalized_data").json.put(JsString(str + "!!!"))))
// A new derived Writes which writes the transformed value if
// the transformer succeeds (a data string), otherwise the
// original value.
implicit val checklistWrites: Writes[ChecklistColumn] = checklistFormat
.transform (js => js.transform(checklistTransformer).getOrElse(js))
That gives me:
Json.prettyPrint(Json.toJson(ChecklistColumn("a", "b", JsNumber(1))))
// {
// "kind" : "a",
// "descriptor" : "b",
// "data" : 1
// }
Json.prettyPrint(Json.toJson(ChecklistColumn("a", "b", JsString("c"))))
// {
// "kind" : "a",
// "descriptor" : "b",
// "data" : "c",
// "normalized_data" : "c!!!"
// }

Decode single field of object in a json array with argonaut/circe

Suppose I have such json
{
"sha": "some sha",
"parents": [{
"url": "some url",
"sha": "some parent sha"
}]
}
and such case class
case class Commit(sha: String, parentShas: List[String])
In play-json I could write the reads like this:
val commitReads: Reads[Commit] = (
(JsPath \ "sha").read[String] and
(JsPath \ "parents" \\ "sha").read[List[String]]
)(Commit.apply _)
I'm looking for a equivalent way of decoding only the "sha" of "parent" in argonaut/circe but I haven't found any. "HCursor/ACursor" has downArray but from there on I don't know what to do. Thank you very much in advance!
Neither circe nor Argonaut keeps track of which fields have been read in JSON objects, so you can just ignore the extra "url" field (just as in Play). The trickier part is finding the equivalent of Play's \\, which circe doesn't have at the moment, although you've convinced me we need to add it.
First of all, this is relatively easy if you have a separate SHA type:
import io.circe.Decoder
val doc = """
{
"sha": "some sha",
"parents": [{
"url": "some url",
"sha": "some parent sha"
}]
}
"""
case class Sha(value: String)
object Sha {
implicit val decodeSha: Decoder[Sha] = Decoder.instance(_.get[String]("sha")).map(Sha(_))
}
case class Commit(sha: Sha, parentShas: List[Sha])
object Commit {
implicit val decodeCommit: Decoder[Commit] = for {
sha <- Decoder[Sha]
parents <- Decoder.instance(_.get[List[Sha]]("parents"))
} yield Commit(sha, parents)
}
Or, using Cats's applicative syntax:
import cats.syntax.cartesian._
implicit val decodeCommit: Decoder[Commit] =
(Decoder[Sha] |#| Decoder.instance(_.get[List[Sha]]("parents"))).map(Commit(_, _))
And then:
scala> import io.circe.jawn._
import io.circe.jawn._
scala> decode[Commit](doc)
res0: cats.data.Xor[io.circe.Error,Commit] = Right(Commit(Sha(some sha),List(Sha(some parent sha))))
But that's not really an answer, since I'm not going to ask you to change your model. :) The actual answer is a bit less fun:
case class Commit(sha: String, parentShas: List[String])
object Commit {
val extractSha: Decoder[String] = Decoder.instance(_.get[String]("sha"))
implicit val decodeCommit: Decoder[Commit] = for {
sha <- extractSha
parents <- Decoder.instance(c =>
c.get("parents")(Decoder.decodeCanBuildFrom[String, List](extractSha, implicitly))
)
} yield Commit(sha, parents)
}
This is bad, and I'm ashamed it's necessary, but it works. I've just filed an issue to make sure this gets better in a future circe release.

Play Json Recursive Reads

I'm working on consuming an API which exposes an object at multiple layers within it's response. For example for some responses we get back a:
{
"error": {
"code": "123",
"description": "Description"
}
}
But in other situations it response with:
{
"data": [
{
"message_id": "123",
"error": {
"code": "123",
"description": "Description"
}
}
]
}
In both cases the error object is identical and in both case I don't actually care about the rest of the payload. I was hoping to use the \\ recursive JsPath operator, however the implementation below fails:
case class ErrorMessage(code: String, description: String)
implicit val errorMessageFormat = Json.format[ErrorMessage]
case class ErrorResponse(errors: Seq[ErrorMessage])
implicit val errorResponseFormat: Format[ErrorResponse] = Format(
(__ \\ "error").read[Seq[ErrorMessage]].map(ErrorResponse),
(__ \ "errors").write[Seq[ErrorMessage]].contramap((r: ErrorResponse) => r.errors)
)
This gives an error:
JsError(List((//error,List(ValidationError(List(error.expected.jsarray),WrappedArray())))))
I understand why: The (__ \\ "error") returns a Seq[JsValue], where as my read call is expecting a JsArray.
Is there a nice way a round this?
Since the first piece is already a Seq, just map the internal elements as you would map single objects. I'm not very familiar with the framework (I use Json4s myself mostly), but by your description sounds like
ErrorResponse((__ \\ "error").map(_.read[Seq[ErrorMessage]]))
Should be closer to what you want.(__ \\ "error") gives you a Seq of JsValues, map does something for every JsValue, read converts a single JsValue to an ErrorMessage and the resulting Seq[ErrorMessage] you pass to the ErrorResponse constructor.
So for anyone trying to do something similar, the below works.
val errorResponseReads = Reads[ErrorResponse] { value: JsValue =>
val errorsJsArray: JsArray = JsArray(value \\ "error")
errorsJsArray.validate[Seq[ErrorMessage]].map(ErrorResponse)
}
The format then becomes:
implicit val errorResponseFormat: Format[ErrorResponse] = Format(
errorResponseReads,
(__ \ "errors").write[Seq[ErrorMessage]].contramap((r: ErrorResponse) => r.errors)
)
Basically you need to define Reads that you use explicitly. In the reads you can then use the recursive search to return a Seq[JsValue] and then create a JsArray that can be validated as normal.
This works fine, it would be nice if we didn't have to define a separate Reads[T] though.

Parse json to Case Class

I've got the following JSON that I need to map 2 values to a case class
These values are job_id and state
The JSON is the following
{
"screenshots": [
{
"browser": "chrome",
"state": "pending",
"url": "http://localhost/mirror/52ea1b22e4b0e133507b209b",
"browser_version": "26.0",
"os_version": "7",
"id": "92342eed1fd14c354d9365cbbd3e35ea1fc45df2",
"os": "Windows"
}
],
"wait_time": 5,
"callback_url": "http://localhost/screenshot/accept/52ea1b22e4b0e133507b209b",
"quality": "compressed",
"job_id": "ce991c0c3d140b5a78859b28cf391fd99c63ff98",
"win_res": "1024x768",
"orientation": "portrait",
"mac_res": "1024x768"
}
With the case class being
case class JobInfo(job_id: String)
object JobInfo {
implicit val fmt = Json.format[JobInfo]
}
This works fine but i want to add the screenshot 'state' into the case class without having to have the whole scrernshot as I am only persisting job_id and state so something like the following
case class JobInfo(job_id: String, state: String)
object JobInfo {
implicit val fmt = Json.format[JobInfo]
}
I'm reading the response like this below but screenshot is an array so wanted to know how I could extract the JSON key 'state' from it and map it to the case class
.map {
response => {
val jobInfo = Json.parse(response.body).as[JobInfo]
}
}
You'll need to define a custom Reads[JobInfo] object to handle the parsing. Assuming you intend to grab the first element from the screenshots array to find the state, you could do it as follows:
import play.api.libs.json._
import play.api.libs.functional.syntax._
object JobInfo {
implicit val jobReads: Reads[JobInfo] = (
(__ \ "job_id").read[String] and
((__ \ "screenshots")(0) \ "state").read[String]
)(JobInfo.apply _)
}
Your original call to parse the json will now work; however, you should be using the validate method to properly handle the case where your JSON is not formatted as expected, i.e.:
Json.parse(response).validate[JobInfo].fold(
jobInfo => {
//do something with your jobInfo
println(jobInfo)
},
errors => {
//handle JSON parsing errors
println(errors)
}
)
This way you handle parsing errors more gracefully. The Play documentation describes this more in detail here