I have input json like
{"a": "x", "b": "y", "c": "z", .... }
I want to convert this json to a Map like Map[String, String]
so basically a map of key value pairs.
How can I do this using circe?
Note that I don't know what keys "a", "b", "c" will be present in Json. All I know is that they will always be strings and never any other data type.
I looked at Custom Decoders here https://circe.github.io/circe/codecs/custom-codecs.html but they work only when you know the tag names.
I found an example to do this in Jackson. but not in circe
import com.fasterxml.jackson.module.scala.DefaultScalaModule
import com.fasterxml.jackson.databind.ObjectMapper
val data = """
{"a": "x", "b", "y", "c": "z"}
"""
val mapper = new ObjectMapper
mapper.registerModule(DefaultScalaModule)
mapper.readValue(data, classOf[Map[String, String]])
While the solutions in the other answer work, they're much more verbose than necessary. Off-the-shelf circe provides an implicit Decoder[Map[String, String]] instance, so you can just write the following:
scala> val doc = """{"a": "x", "b": "y", "c": "z"}"""
doc: String = {"a": "x", "b": "y", "c": "z"}
scala> io.circe.parser.decode[Map[String, String]](doc)
res0: Either[io.circe.Error,Map[String,String]] = Right(Map(a -> x, b -> y, c -> z))
The Decoder[Map[String, String]] instance is defined in the Decoder companion object, so it's always available—you don't need any imports, other modules, etc. Circe provides instances like this for most standard library types with reasonable instances. If you want to decode a JSON array into a List[String], for example, you don't need to build your own Decoder[List[String]]—you can just use the one in implicit scope that comes from the Decoder companion object.
This isn't just a less verbose way to solve this problem, it's the recommended way to solve it. Manually constructing an explicit decoder instance and converting from Either to Try to compose parsing and decoding operations is both unnecessary and error-prone (if you do need to end up with Try or Option or whatever, it's almost certainly best to do that at the end).
Assuming:
val rawJson: String = """{"a": "x", "b": "y", "c": "z"}"""
This works:
import io.circe.parser._
val result: Try[Map[String, String]] = parse(rawJson).toTry
.flatMap(json => Try(json.asObject.getOrElse(sys.error("Not a JSON Object"))))
.flatMap(jsonObject => Try(jsonObject.toMap.map{case (name, value) => name -> value.asString.getOrElse(sys.error(s"Field '$name' is not a JSON string"))}))
val map: Map[String, String] = result.get
println(map)
Or with using Decoder:
import io.circe.Decoder
val decoder = Decoder.decodeMap(KeyDecoder.decodeKeyString, Decoder.decodeString)
val result = for {
json <- parse(rawJson).toTry
map <- decoder.decodeJson(json).toTry
} yield map
val map = result.get
println(map)
You can test following invalid inputs and see, what exception will be thrown:
val rawJson: String = """xxx{"a": "x", "b": "y", "c": "z"}""" // invalid JSON
val rawJson: String = """[1,2,3]""" // not a JSON object
val rawJson: String = """{"a": 1, "b": "y", "c": "z"}""" // not all values are string
Related
Is there a way to decode arbitrary json (e.g: We don't know the keys at compile time)?
For example, I need to parse the following json:
{
"Foo": [
"Value 1",
"Value 2"
],
"Bar": [
"Bar Value 1"
],
"Baz": []
}
where the names and number of keys are not known at compile time and may change per GET request. The goal is basically to decode this into a Map String (Array String) type
Is there a way to do this using purescript-argonaut?
You can totally write your own by first parsing the string into Json via jsonParser, and then examining the resulting data structure with the various combinators provided by Argonaut.
But the quickest and simplest way, I think, is to parse it into Foreign.Object (Array String) first, and then convert to whatever your need, like Map String (Array String):
import Data.Argonaut (decodeJson, jsonParser)
import Data.Either (Either)
import Data.Map as Map
import Foreign.Object as F
decodeAsMap :: String -> Either _ (Map.Map String (Array String))
decodeAsMap str = do
json <- jsonParser str
obj <- decodeJson json
pure $ Map.fromFoldable $ (F.toUnfoldable obj :: Array _)
The Map instance of EncodeJSON will generate an array of tuple, you can manually construct a Map and see the encoded json.
let v = Map.fromFoldable [ Tuple "Foo" ["Value1", "Value2"] ]
traceM $ encodeJson v
Output should be [ [ 'Foo', [ 'Value1', 'Value2' ] ] ].
To do the reverse, you need to transform you object to an array of tuple, Object.entries can help you.
An example
// Main.js
var obj = {
foo: ["a", "b"],
bar: ["c", "d"]
};
exports.tuples = Object.entries(obj);
exports.jsonString = JSON.stringify(exports.tuples);
-- Main.purs
module Main where
import Prelude
import Data.Argonaut.Core (Json)
import Data.Argonaut.Decode (decodeJson)
import Data.Argonaut.Parser (jsonParser)
import Data.Either (Either)
import Data.Map (Map)
import Debug.Trace (traceM)
import Effect (Effect)
import Effect.Console (log)
foreign import tuples :: Json
foreign import jsonString :: String
main :: Effect Unit
main = do
let
a = (decodeJson tuples) :: Either String (Map String (Array String))
b = (decodeJson =<< jsonParser jsonString) :: Either String (Map String (Array String))
traceM a
traceM b
traceM $ a == b
I have a string in scala which in terms of formatting, it is a json, for example
{"name":"John", "surname":"Doe"}
But when I generate this value it is initally a string. I need to convert this string into a json but I cannot change the output of the source. So how can I do this conversion in Scala? (I cannot use the Play Json library.)
If you have strings as
{"name":"John", "surname":"Doe"}
and if you want to save to elastic as mentioned here then you should use parseRaw instead of parseFull.
parseRaw will return you JSONType and parseFull will return you map
You can do as following
import scala.util.parsing.json._
val jsonString = "{\"name\":\"John\", \"surname\":\"Doe\"}"
val parsed = JSON.parseRaw(jsonString).get.toString()
And then use the jsonToEs api as
sc.makeRDD(Seq(parsed)).saveJsonToEs("spark/json-trips")
Edited
As #Aivean pointed out, when you already have json string from source, you won't be needing to convert to json, you can just do
if jsonString is {"name":"John", "surname":"Doe"}
sc.makeRDD(Seq(jsonString)).saveJsonToEs("spark/json-trips")
You can use scala.util.parsing.json to convert JSON in string format to JSON (which is basically HashMap datastructure),
eg.
scala> import scala.util.parsing.json._
import scala.util.parsing.json._
scala> val json = JSON.parseFull("""{"name":"John", "surname":"Doe"}""")
json: Option[Any] = Some(Map(name -> John, surname -> Doe))
To navigate the json format,
scala> json match { case Some(jsonMap : Map[String, Any]) => println(jsonMap("name")) case _ => println("json is empty") }
John
nested json example,
scala> val userJsonString = """{"name":"John", "address": { "perm" : "abc", "temp" : "zyx" }}"""
userJsonString: String = {"name":"John", "address": { "perm" : "abc", "temp" : "zyx" }}
scala> val json = JSON.parseFull(userJsonString)
json: Option[Any] = Some(Map(name -> John, address -> Map(perm -> abc, temp -> zyx)))
scala> json.map(_.asInstanceOf[Map[String, Any]]("address")).map(_.asInstanceOf[Map[String, String]]("perm"))
res7: Option[String] = Some(abc)
I have a String with some arbitrary JSON in it. I want to construct a JsObject with my JSON string as a JSON object value, not a string value. For example, assuming my arbitrary string is a boring {} I want {"key": {}} and not {"key": "{}"}.
Here's how I'm trying to do it.
val myString = "{}"
Json.obj(
"key" -> Json.parse(myString)
)
The error I get is
type mismatch; found :
scala.collection.mutable.Buffer[scala.collection.immutable.Map[String,java.io.Serializable]]
required: play.api.libs.json.Json.JsValueWrapper
I'm not sure what to do about that.
"{}" is an empty object.
So, to get {"key": {}} :
Json.obj("key" -> Json.obj())
Update:
Perhaps you have an old version of Play. This works under Play 2.3.x:
scala> import play.api.libs.json._
scala> Json.obj("foo" -> Json.parse("{}"))
res2: play.api.libs.json.JsObject = {"foo":{}}
I was wondering if there is a parser or an easy way to iterate through a json object without knowing the keys/schema of the json ahead of time in scala. I took a look at a few libraries like json4s, but it seems to still require knowing the schema ahead of time before extracting the fields. I just want to iterate over each field, extract the fields and print out their values something like:
json.foreachkey(key -> println(key +":" + json.get(key))
In Play Json you'll initially parse your json into a JsValue; you can then pattern-match this to determine if it is a JsObject (note that you can find the fields of this using fields or value), a JsArray (again, note the value), or a primitive such as JsString or JsNull
def parse(jsVal: JsValue) {
jsVal match {
case json: JsObject =>
case json: JsArray =>
case json: JsString =>
...
}
}
If by json you mean any JValue, then json4s seems to have this functionality out of the box:
scala> import org.json4s.JsonDSL._
import org.json4s.JsonDSL._
scala> import org.json4s.native.JsonMethods._
import org.json4s.native.JsonMethods._
scala> val json = parse(""" { "numbers" : [1, 2, 3, 4] } """)
json: org.json4s.JValue = JObject(List((numbers,JArray(List(JInt(1), JInt(2), JInt(3), JInt(4))))))
scala> compact(render(json))
res1: String = {"numbers":[1,2,3,4]}
Use liftweb, it allows you to parse the json first into JValue - then extract native scala objects from it, no matter the schema:
val jsonString = """{"menu": {
| "id": "file",
| "value": "File",
| "popup": {
| "menuitem": [
| {"value": "New", "onclick": "CreateNewDoc()"},
| {"value": "Open", "onclick": "OpenDoc()"},
| {"value": "Close", "onclick": "CloseDoc()"}
| ]
| }
|}}""".stripMargin
val jVal: JValue = parse(jsonString)
jVal.values
>>> Map(menu -> Map(id -> file, value -> File, popup -> Map(menuitem -> List(Map(value -> New, onclick -> CreateNewDoc()), Map(value -> Open, onclick -> OpenDoc()), Map(value -> Close, onclick -> CloseDoc())))))
I'm using the json library of the play framework version 2.2.3. I have the following json object:
{
"myData":
[
{
"A": "some text",
"B": [10, 20, 30]
},
{
"A": "some other text",
"B": [15, 25, 35]
},
...
]
}
I want to deserialize this json object to a Vector[Map[String, Vector[Int]]]. So the result should be:
Vector(Map("some text" -> Vector(10, 20, 30)), Map("some other text" -> Vector(15, 25, 35)))
While I was trying to achieve that, I was able to write a Reads[Map[String, Vector[Int]]] that does it for a single entry.
import play.api.libs.json._
import play.api.libs.functional.syntax._
implicit val singleEntryReads: Reads[Map[String, Vector[Int]]] = {
(__).read(
(__ \ "A").read[String] and
(__ \ "B").read[Vector[Int]] tupled) map { keyAndValue =>
val (a, b) = keyAndValue
Map(a -> b)
}
}
So the conversion works for a single entry:
scala> (myJsonObject \ "myData")(0).validate[Map[String, Vector[Int]]]
res: play.api.libs.json.JsResult[Map[String, Vector[Int]]] = JsSuccess(Map(some text -> Vector(10, 20, 30)))
But how can I write a Reads[Vector[Map[String, Vector[Int]]]]? My best idea was to do it similar to the answer given to this older question:
implicit val allEntriesReads: Reads[Seq[Map[String, Vector[Int]]]] = Reads.seq(singleEntryReads)
I tried to use it like this:
scala> (myJsonObject \ "myData").validate[Seq[Map[String, Vector[Int]]]]"
res2: play.api.libs.json.JsResult[Seq[Map[String, Vector[Int]]]] = JsError(List(((147)/B,List(ValidationError(error.path.missing,WrappedArray()))), ((148)/B,List(ValidationError(error.path.missing,WrappedArray())))))
But that does not work and gives me a JsError(). How do I have to implement the second Reads to get it working that way?
You don't need to. Play! comes with an implicit (defined in the Reads companion object) that already does what you want:
implicit def traversableReads[F[_], A](implicit bf: CanBuildFrom[F[_], A, F[A]], ra: Reads[A]): Reads[F[A]]
If you have an implicit Reads[A] and an appropriate CanBuildFrom for the sequence type (these already exist for the standard libraries collection types, e.g. Vector) then this implicit will act as an implicit Reads[F[A]] where F is the collection type.
Scala lets you define an implicit def that itself takes implicit parameters, and it will act as an implicit value of its return type. When using your implicit def, Scala will search for the implicit parameters at the call site. So:
.validate[Seq[Map[String, Vector[Int]]]]
becomes:
.validate[Seq[Map[String, Vector[Int]]]](traversableReads)
which then becomes:
.validate[Seq[Map[String, Vector[Int]]]](traversableReads(singleEntryReads, Seq.canBuildFrom)