I am reading a HDFS sequence file and which is of [Long, String] lets call each record as message. message._2 is a json string and i am trying to parse it using play json library but i get the following error when i do it.
Error:
found : Seq[play.api.libs.json.JsValue]
required: String
Code:
val jsonString = message._2.toString();
val json = Json.parse(jsonString);
code = (json(0) \\ "code"); -- > Error is pointing to \\ on this line
The error message says that (json(0) \\ "code") returns Seq[play.api.libs.json.JsValue], and you're trying to assign this value to the variable code of type String.
So, you may want to do this:
code = (json(0) \\ "code").head.as[String]
which will get the first item of a list and convert JsValue to String.
Update
As #cchantep suggested, the use of head is not safe, so for better safety you can do it with headOption, but the result type will be Option[String]:
val code: Option[String] =
(json \\ "code").headOption.map(_.as[String])
and even more safe code will look like this:
val code: Option[String] =
(json \\ "code").headOption.flatMap(_.asOpt[String])
Related
In my code I am trying to get Income type object list from a json file
val fileJson = File(
Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_DOCUMENTS),
"Income.json")
val strFileJson: String? = fileJson.readText()
val PreviousJsonObj: JSONObject = JSONObject(strFileJson)
val array: JSONArray = PreviousJsonObj.getJSONArray("Incomes")
val typeToken = object : TypeToken<List<Income>>() {}.type
IncomeArry = Gson().fromJson<List<Income>>(strFileJson, typeToken)
But I am getting
com.google.gson.JsonSyntaxException: java.lang.IllegalStateException: Expected BEGIN_ARRAY but was BEGIN_OBJECT at line 1 column 2 path $
The value returned in strFileJson is "Incomes":[{"date":"abc","src": "cfgb","amt": 7890},
{"date":"yui", "src":"tyhv","amt" :6789}]
Tried running with multiple imple,mentations of gson and multiple class architectures
The fragment of JSON you show:
"Incomes":[{"date":"abc","src": "cfgb","amt": 7890}, {"date":"yui", "src":"tyhv","amt" :6789}]
is malformed, did you mis-paste? It needs to be enclosed in curly braces:
{"Incomes":[{"date":"abc","src": "cfgb","amt": 7890}, {"date":"yui", "src":"tyhv","amt" :6789}]}
but this will not solve your problem because both will give the error you describe. You are trying to deserialize an array so you need to either
Change the JSON to be
[{"date":"abc","src": "cfgb","amt": 7890}, {"date":"yui", "src":"tyhv","amt" :6789}]
Or create a wrapper object, something like
data class Incomes(
val Incomes : List<Income>
)
and deserialize in to that.
From a separate system I get a String parameter "messageJson" whose content is in the form:
{"agent1":"smith","agent2":"brown","agent3":{"agent3_1":"jones","agent3_2":"johnson"}}
To use it in my program I parse it with JsonSlurper.
def myJson = new JsonSlurper().parseText(messageJson)
But the resulting Json has the form:
[agent1:smith, agent2:brown, agent3:[agent3_1:jones, agent3_2:johnson]]
Note the square brackets and the lack of double quotes. How can I parse messageJson so that the original structure is kept?
Ok, thanks to the hint by cfrick, I was able to find a solution. In case anyone else has a similar problem, all I needed to do was using JsonOutput in the end to convert the map back to a Json
I.E. :
def myJson = new JsonSlurper().parseText(messageJson)
myJson << [agent4:"jane"]
def backToJson = JsonOutput.toJson(myJson)
I have a JSON object of the form:
{"apps":{"app":[{"id":"application_1481567788061_0002","user":"root","name":"wordcount.py","queue":"default","state":"FAILED","finalStatus":"FAILED","progress":0.0,"trackingUI":"History", "diagnostics":"Application application_1481567788061_0002 failed 2 times due to AM Container for appattempt_1481567788061_0002_000002 exited with exitCode: 255\nFor more detailed output, check application tracking page:http://sandbox:8088/proxy/application_1481567788061_0002/Then, click on links to logs of each attempt.\nDiagnostics: Exception from container-launch.\nContainer id: container_1481567788061_0002_02_000001\nExit code: 255\nStack trace: ExitCodeException exitCode=255: \n\tat org.apache.hadoop.util.Shell.runCommand(Shell.java:538)\n\tat org.apache.hadoop.util.Shell.run(Shell.java:455)\n\tat org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715)\n\tat org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:211)\n\tat org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)\n\tat org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)\n\tat java.util.concurrent.FutureTask.run(FutureTask.java:262)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)\n\tat java.lang.Thread.run(Thread.java:744)\n\n\nContainer exited with a non-zero exit code 255\nFailing this attempt. Failing the application.","clusterId":1481567788061,"applicationType":"SPARK","applicationTags":"","startedTime":1481568051052,"finishedTime":1481568079289,"elapsedTime":28237,"amHostHttpAddress":"sandbox:8042","allocatedMB":-1,"allocatedVCores":-1,"runningContainers":-1,"memorySeconds":55598,"vcoreSeconds":27,"preemptedResourceMB":0,"preemptedResourceVCores":0,"numNonAMContainerPreempted":0,"numAMContainerPreempted":0},{"id":"application_1481567788061_0001","user":"root","name":"pi.py","queue":"default","state":"FINISHED","finalStatus":"SUCCEEDED","progress":100.0,"trackingUI":"History","diagnostics":"","clusterId":1481567788061,"applicationType":"SPARK","applicationTags":"","startedTime":1481567853324,"finishedTime":1481567888648,"elapsedTime":35324,"amContainerLogs":"http://sandbox:8042/node/containerlogs/container_1481567788061_0001_01_000001/root","amHostHttpAddress":"sandbox:8042","allocatedMB":-1,"allocatedVCores":-1,"runningContainers":-1,"memorySeconds":138031,"vcoreSeconds":66,"preemptedResourceMB":0,"preemptedResourceVCores":0,"numNonAMContainerPreempted":0,"numAMContainerPreempted":0}]}}
I would like to extract from it a List[Application], where application is:
case class Application(id: String, user: String, name: String)
I imported spray-json.
If message is a string containing the JSON component, I want to do something like:
val json: JsValue = message.parseJson
val jobsJson = json.first.first
val jobs = jobsJson.map(job => Application(job(0), job(1), job(2)))
But this is not correct because I can't use json.first.
So how can I extract fields nested in the JSON object?
Is there another library that makes things easier?
Note: This answer is about play-json and not spray-json library.
You should be able to get data out of the json object with \ or \\
A single slash will look in the next lever down for what ever you are looking for while a double slash will look through the whole object.
Say you had the following json stored in a variable called obj:
{"foo":"bar","num":3, "value":{"num":4}}
using obj\num you would just get 3. But with obj\\num you would get in iterator with both 3 and 4 in it.
try this link for a little more information.
I have a JSON string like this:
{"id":"111","name":"abc","ids":["740"],"data":"abc"}
I want to cut off the field "ids", however I don't know apriori the values like ["740"]. So, it might be e.g. ["888,222"] or whatever. The goal is to get the json string without the field "ids".
How to do it? Should I use JackMapper?
EDIT:
I tried to use JackMapper as JacksMapper.readValue[Map[String, String]](jsonString)to get only fields that I need. But the problem is that"ids":["740"]` throws the parsing error because it's an array. So, I decided to cut off this field before parsing, though it's an ugly solution and ideally I just want to parse the json string into Map.
Not sure what JackMapper is, but if other libraries are allowed, my personal favourites would be:
Play-JSON:
val jsonString = """{"id":"111","name":"abc","ids":["740"],"data":"abc"}"""
val json = Json.parse(jsonString).as[JsObject]
val newJson = json - "ids"
Circe:
import io.circe.parser._
val jsonString = """{"id":"111","name":"abc","ids":["740"],"data":"abc"}"""
val json = parse(jsonString).right.get.asObject.get // not handling errors
val newJson = json.remove("ids")
Note that this is the minimal example to get you going which doesn't handle bad input etc.
My client side executes a server call encompassing data (queryString) in a JSON object like this:
?q={"title":"Hello"} //non encoded for the sample but using JSON.stringify actually
What is an efficient way to retrieve the title and Hello String?
I tried this:
val params = request.queryString.map {case(k,v) => k->v.headOption}
that returns the Tuple: (q,Some({"title":"hello"}))
I could further extract to retrieve the values (although I would need to manually map the JSON object to a Scala object), but I wonder whether there is an easier and shorter way.
Any idea?
First, if you intend to pluck only the q parameter from a request and don't intend to do so via a route you could simply grab it directly:
val q: Option[String] = request.getQueryString("q")
Next you'd have to parse it as a JSON Object:
import play.api.libs.json._
val jsonObject: Option[JsValue] = q.map(raw: String => json.parse(raw))
with that you should be able to check for the components the jsonObject contains:
val title: Option[String] = jsonObject.flatMap { json: JsValue =>
(json \ "title").asOpt[String]
}
In short, omitting the types you could use a for comprehension for the whole thing like so:
val title = for {
q <- request.getQueryString("q")
json <- Try(Json.parse(q)).toOption
titleValue <- (json \ "title").asOpt[String]
} yield titleValue
Try is defined in scala.util and basically catches Exceptions and wraps it in a processable form.
I must admit that the last version simply ignores Exceptions during the parsing of the raw JSON String and treats them equally to "no title query has been set".
That's not the best way to know what actually went wrong.
In our productive code we're using implicit shortcuts that wraps a None and JsError as a Failure:
val title: Try[String] = for {
q <- request.getQueryString("q") orFailWithMessage "Query not defined"
json <- Try(Json.parse(q))
titleValue <- (json \ "title").validate[String].asTry
} yield titleValue
Staying in the Try monad we gain information about where it went wrong and can provide that to the User.
orFailWithMessage is basically an implicit wrapper for an Option that will transform it into Succcess or Failure with the specified message.
JsResult.asTry is also simply a pimped JsResult that will be Success or Failure as well.