I have a json string and I wast to be able to parse it to get the 'key' values.
jsonString = {"id":2279,
"name":"Test",
"description":null,
"tags":[],
"keys":[{
"key":"WI1MX6XAWSY03X8Y",
"flag":true},
{"key":"BK2Q18T8RSN6VODR",
"flag":false}]}
I want to be able to parse this string and get values for both the keys.
Currently I'm doing:
val details = JSON.parseFull(jsonString)
val keys = details.get.asInstanceOf[Map[String, Any]]("keys")
println(keys)
keys here is:
List(Map(key -> 3JP11GJ5OOGOVV5N, flag -> true), Map(key -> F49M347FOHYKBT9, flag -> false))
Please let me know how i can get both the 'key' values.
There is nothing related to JSON actually, you just have to do:
val keysValues = key.map(k => k("key"))
Related
I have a value in a JsObject which I want to assign to a specific key in Map, and I want to ask if there is a better way to extract that value without using a case matcher.
I have access to a request variable which is a case class that has a parameter myData
myData is an Option[JsValue] which contains multiple fields and I want to return the boolean value of one specific field in there called “myField” in a string format. The below works, but I want to find a more succinct way of getting the value of "myField" without case matching.
val newMap =
Map(
“myNewKey” -> request.myData.map(_ match {
case JsObject(fields) => fields.getOrElse(“myField”, "unknown").toString
case _ => “unknown”})
The output would then be
"myField": "true"
or
"myField": "false"
or if it isn't true or false, i.e the field doesn't exist
"myField": "unknown"
Rather:
myMap ++ (request.myData \ "myField").validateOpt[String].asOpt.
toSeq.collect { // to map entries
case Some(str) => "keyInMap" -> str
}
Do not use .toString to convert a JSON value or pretty print it.
I'm writing some code to auto-gen JSON codecs for Elm data-structures. There is a point my code, where a "sub-structure/sub-type", has already been encoded to a Json.Encode.Value, and I need to add another key-value pair to it. Is there any way to "destructure" a Json.Encode.Value in Elm? Or combine two values of type Json.Encode.Value?
Here's some sample code:
type alias Entity record =
{ entityKey: (Key record)
, entityVal: record
}
jsonEncEntity : (record -> Value) -> Entity record -> Value
jsonEncEntity localEncoder_record val =
let
encodedRecord = localEncoder_record val.entityVal
in
-- NOTE: the following line won't compile, but this is essentially
-- what I'm looking for
Json.combine encodedRecord (Json.Encode.object [ ( "id", jsonEncKey val.entityKey ) ] )
You can decode the value into a list of key value pairs using D.keyValuePairs D.value and then append the new field. Here's how you'd do that:
module Main exposing (..)
import Json.Decode as D
import Json.Encode as E exposing (Value)
addKeyValue : String -> Value -> Value -> Value
addKeyValue key value input =
case D.decodeValue (D.keyValuePairs D.value) input of
Ok ok ->
E.object <| ( key, value ) :: ok
Err _ ->
input
> import Main
> import Json.Encode as E
> value = E.object [("a", E.int 1)]
{ a = 1 } : Json.Encode.Value
> value2 = Main.addKeyValue "b" E.null value
{ b = null, a = 1 } : Json.Encode.Value
If the input is not an object, this will return the input unchanged:
> Main.addKeyValue "b" E.null (E.int 1)
1 : Json.Encode.Value
If you want to do this, you need to use a decoder to unwrap the values by one level into a Dict String Value, then combine the dictionaries, and finally re-encode as a JSON value. You can unwrap like so:
unwrapObject : Value -> Result String (Dict String Value)
unwrapObject value =
Json.Decode.decodeValue (Json.Decode.dict Json.Decode.value) value
Notice that you have to work with Results from this point on because there's the possibility, as far as Elm is concerned, that your JSON value wasn't really an object (maybe it was a number or a string instead, for instance) and you have to handle that case. For that reason, it's not really best practice to do too much with JSON Values directly; if you can, keep things as Dicts or some other more informative type until the end of processing and then convert the whole result into a Value as the last step.
I'm trying to convert a string of key value pairs to a JSON string. The only thing I know about the string of KV pairs is the format of the string i.e. space seperated, comma seperated etc.. For e.g. I don't control over the number or type of the keys coming in as input.
Here is what I came up with and wanted to see if this approach looks OK / awesome / awkward. Would appreciate if there is better alternative than this.
INPUT : clientIp="1.1.1.1" identifier="a.b.c" key1=10 key2="v3"
final val KV_PATTERN = "(\"[^\"]*\"|[^,\\\"\\s]*)=(\"[^\"]*\"|[^,\\\"\\s]*)".r
val cMap = KV_PATTERN.findAllMatchIn(inputString).map(m => (m.group(1).trim(), m.group(2).trim())).toMap
val json = cMap.map { case (key, value) => if (!key.startsWith("\"")) s""""$key"""" + ":" + value else s"$key:$value" }.mkString("{", ",", "}")`
OUTPUT: {"clientIp":"1.1.1.1","identifier":"a.b.c","key1":10,"key2":"v3"}
"{"+ inputString.split(" ").map{case i => val t = i.split("="); s""""${t(0).replaceAll("^\"|\"$", "")}": ${t(1)}"""}.mkString(",") + "}"
Maybe this is more cleaner.
I have a json like :
{"name":"Yin", "address":[{"city":"Columbus","state":"Ohio"},{"city":"Columbus","state":"Ohio"}]}
{"name":"Michael", "address":[{"city":null, "state":"California"},{"city":null, "state":"California"}]}
here address is an array and if i use sqlContext.jsonfile i get the data in schema rdd as follows :
[Yin , [(Columbus , Ohio) , (Columbus , Ohio)]
[Micheal , [(null, California) , (null, California)]
I want to explode the array present and want the data in the following format in schema rdd :
[Yin, Columbus, Ohio]
[Yin, Columbus, Ohio]
[Micheal, null, California]
[Micheal, null, California]
I am using spark SQL
The typical suggestion is drop out of sql for this, but if you want to stay in SQL, here is an answer I got from asking this on the mailing list (nabble isn't showing the response for some reason):
From Michael Armbrust
You can do want with lateral view explode (using HiveContext), but what seems to be missing is that jsonRDD converts json objects into structs (fixed keys with a fixed order) and fields in a struct are accessed using a .
val myJson = sqlContext.jsonRDD(sc.parallelize("""{"foo":[{"bar":1},{"baz":2}]}""" :: Nil))
myJson.registerTempTable("JsonTest")
val result = sql("SELECT f.bar FROM JsonTest LATERAL VIEW explode(foo) a AS f").collect()
myJson: org.apache.spark.sql.DataFrame = [foo: array<struct<bar:bigint,baz:bigint>>]
result: Array[org.apache.spark.sql.Row] = Array([1], [null])
In Spark 1.3 you can also hint to jsonRDD that you'd like the json objects converted into Maps (non-uniform keys) instead of structs, by manually specifying the schema of your JSON.
import org.apache.spark.sql.types._
val schema =
StructType(
StructField("foo", ArrayType(MapType(StringType, IntegerType))) :: Nil)
sqlContext.jsonRDD(sc.parallelize("""{"foo":[{"bar":1},{"baz":2}]}""" :: Nil), schema).registerTempTable("jsonTest")
val withSql = sql("SELECT a FROM jsonTest LATERAL VIEW explode(foo) a AS a WHERE a['bar'] IS NOT NULL").collect()
val withSpark = sql("SELECT a FROM jsonTest LATERAL VIEW explode(foo) a AS a").rdd.filter {
case Row(a: Map[String, Int]) if a.contains("bar") => true
case _: Row => false
}.collect()
schema: org.apache.spark.sql.types.StructType = StructType(StructField(foo,ArrayType(MapType(StringType,IntegerType,true),true),true))
withSql: Array[org.apache.spark.sql.Row] = Array([Map(bar -> 1)])
withSpark: Array[org.apache.spark.sql.Row] = Array([Map(bar -> 1)])
I'm new to the Play Framework, and Scala language. I want to save some data to database only by running URL with specified parameters.
For example I want to run url like:
/DeviceData?device_ID=1&insertDate=2013-01-01&windDirection=50&device_ID=1&insertDate=2013-01-02&windDirection=5
and after that in the database two new records would be inserted (with Device_ID, insertDate and windDirection).
Right now I'm trying to save only one record at once (I don't know how to read list of elements and save them) but event that it's not working. There is no error, it's just not inserted.
DeviceData model
case class DeviceData(data_ID: Long, device_ID: Long, insertDate: String, windDirection: Double)
object DeviceData{
var deviceDataList = new HashMap[Long, DeviceData]
var data_ID = 0L
def nextId(): Long = { data_ID += 1; data_ID}
def createDeviceData(device_ID: Long, insertDate: String, windDirection: Double) :Unit = {
DB.withConnection { implicit connection =>
SQL(
"""
INSERT INTO devicedata(device_ID, insertDate, windDirection)
VALUES ({device_ID}, {insertDate}, {windDirection})
"""
).
on("device_ID" -> device_ID, "insertDate" -> insertDate, "windDirection" -> windDirection).
executeInsert()
}
}
def list(): List[DeviceData] = { deviceDataList.values.toList }
}
DeviceDatas controller
object DeviceDatas extends Controller {
val deviceDataForm = Form(
tuple(
"device_ID" -> of[Long],
"insertDate" -> nonEmptyText,
"windDirection" -> of[Double]
)
)
def listDeviceData() = Action {
Ok(views.html.deviceData(DeviceData.list(), deviceDataForm))
}
def createDeviceData(device_ID: Long, insertDate: String, windDirection: Double) = Action { implicit request =>
deviceDataForm.bindFromRequest.fold(
errors => BadRequest(views.html.deviceData(DeviceData.list(), errors)),
{ case (device_ID, insertDate, windDirection) => {
DeviceData.createDeviceData(device_ID, insertDate, windDirection)
Redirect(routes.DeviceDatas.listDeviceData)
}
}
)
}
}
deviceData.scala.html - it's simple one, just to check if there is any new inserted record.
#(deviceDatas: List[DeviceData], deviceDataForm: Form[(Long, String, Double)])
#import helper._
#main("DeviceDatas"){
<h3>#deviceDatas.size DeviceData(s)</h3>
}
routes file for /deviceDatas
GET /deviceDatas controllers.DeviceDatas.listDeviceData
POST /deviceDatas controllers.DeviceDatas.createDeviceData(device_ID: Long, insertDate: String, windDirection: Double)
Could You help me with that how to insert the data into database, and if there is any possibility to put list of elements with few records to insert. Also what's the best way to insert DateTime (yyyy-MM-dd hh:mm:ss) into URL parameters in Play Framework? I'm stuck and I don't know how to do it.
UPDATED
Thanks Zim-Zam O'Pootertoot for the answer. Unfortunately I need to use parameters, because I'm sending the data through the router. But anyway one more thanks to You because I'll use json in the future.
I decided to not use List of parameter as I said before, but for one new record I'm sending one request (for example: to add 6 new records to the database I need to run 6 times URL on the router:
/DeviceData?device_ID=1&insertDate=2013-01-01&windDirection=50
And my problem was solved by changing the route file to:
GET /deviceDatas controllers.DeviceDatas.listDeviceData
GET /deviceDatas controllers.DeviceDatas.createDeviceData(device_ID: Long, insertDate: String, windDirection: Double)
To pass in data for multiple records, and also to pass in DateTime data, send the data in the request's json body instead of as url params
http://www.playframework.com/documentation/2.2.x/ScalaBodyParsers
http://www.playframework.com/documentation/2.2.x/ScalaJson
Action(parse.json) { implicit request =>
(request.body \ "records") match {
case arr: JsArray => arr.value.foreach(json => {
val deviceId = (json \ "device_ID").as[Long]
val date = (json \ "insertDate").as[String]
val windDirection = (json \ "windDirection").as[Double]
// insert data in database
})
case _ => throw new IllegalArgumentException("Invalid Json: records must be a JsArray")
}}
The json for your records might look something like
{"records" : [
{"device_ID" : 123, "insertDate" : "2014-03-01 12:00:00", "windDirection" : 123.45},
{"device_ID" : 456, "insertDate" : "2014-03-02 12:00:00", "windDirection" : 54.321}]}