Producing json in a Scala app using json4s - json

I am attempting to produce JSON in a Scala app using json4s. Fairly straight forward, Here's some sample value I put together to test it in my Scalatra app:
import org.json4s._
import org.json4s.JsonDSL._
object JsonStub {
val getPeople =
("people" ->
("person_id" -> 5) ~
("test_count" -> 5))
}
In my controller, I simply have:
import org.json4s._
import org.json4s.JsonDSL._
import org.json4s.{DefaultFormats, Formats}
class FooController(mongoDb: MongoClient)(implicit val swagger: Swagger) extends ApiStack with NativeJsonSupport with SwaggerSupport {
get ("/people", operation(getPeople)) {
JsonStub.getPeople
}
}
The output I'm seeing in the browser however, is the following:
{
"_1": "people",
"_2": {
"person_id": 5,
"test_count": 5
}
}
Any clue where the _1 and _2 keys are coming from? I was expecting this output instead:
{
"people":{
"person_id": 5,
"test_count": 5
}
}

What you're seeing in the output is a reflectively serialized tuple, which has fields _1 and _2. This is because the return type that the compiler has inferred for JsonStub.getPeople is Tuple2[String, JObject].
The json4s DSL uses implicit conversions to turn values like the tuple into a JValue. But, if you don't tell the compiler you wanted a JValue, it won't apply the conversion.
Ideally, this would result in a compile error, because you tried to produce JSON from something that isn't the right type. Unfortunately, because your web framework assumes you want to fall back to reflection-based serialization, it means there is another way to turn the tuple into JSON, which isn't what you wanted.
If you explicitly tell the compiler that you want a JValue and not a Tuple2, the DSL's implicit conversion will be applied in the correct place.
val getPeople: JValue =
("people" ->
("person_id" -> 5) ~
("test_count" -> 5))

Related

Replacing Fields in JSON with New Object using json4s

I have a scala app in which I'm using json4s to do some json manipulation. I have a few fields that I would like to convert into a new object.
For example I have the following:
"start_datetime":"2016-12-11T01:00:05+0000",
"type":"absolute",
"start":"5",
"type":"offset"
That would like to make into:
"time":[
{
"type":"absolute",
"start_datetime":"2016-12-11T01:00:05+0000"
},
{
"type":"offset",
"start":"10"
}
]
Any way I can do this using json4s?
The below snippet uses native json4s DSL
A Json object is formed by tuples chained together by method ~ and the Json Array is created by creating a Sequence object in Scala. Other primitive types like String, Number, Boolean are mapped to the corresponding types in scala
import org.json4s.native.JsonMethods._
import org.json4s.JsonDSL._
val json = "time" -> Seq(
("type" -> "absolute") ~ ("start_datetime" -> "2016-12-11T01:00:05+0000"),
("type" -> "offset") ~ ("start" -> "10")
)
scala> compact(render(json))
res3: String = {"time":[{"type":"absolute","start_datetime":"2016-12-11T01:00:05+0000"},{"type":"offset","start":"10"}]}

How to serialize JSON with json4s with UTF-8 characters?

I have a really simple example:
import org.json4s._
import org.json4s.native.JsonMethods._
import org.json4s.JsonDSL._
val json = ("english" -> JString("serialization")) ~ ("japanese" -> JString("シリアライゼーション"))
println(pretty(render(json)))
What I get out of that is:
{
"english":"serialization",
"japanese":"\u30b7\u30ea\u30a2\u30e9\u30a4\u30bc\u30fc\u30b7\u30e7\u30f3"
}
What I want is this (perfectly valid AFAIK) JSON:
{
"english":"serialization",
"japanese":"シリアライゼーション"
}
I can't find it now, but I think I've read somewhere that JSON only requires two special UTF-8 characters to be escaped.
Looking at the code for render, it appears that Strings always get this extra double-escaping for non-ASCII characters.
Anyone know how I can get valid JSON without double-escaping all the UTF-8 extended characters? This seems like a very similar issue to: Why does the PHP json_encode function convert UTF-8 strings to hexadecimal entities?
Update: It turns out this is an open issue in json4s with a pending PR #327 which was closed in favor of PR #339 which in turn merged into the 3.4 release branch in a commit on Feb 13, 2016.
#0__, it is not clear what answer you want to get with your bounty. The bug mentioned in the original question has already been fixed, so you can customize whether you want Unicode characters to be encoded or not. You just need to build with a current version, e.g. with a build.sbt like this:
name := "SO_ScalaJson4sUnicodeChars"
version := "1.0"
scalaVersion := "2.12.1"
libraryDependencies += "org.json4s" %% "json4s-native" % "3.5.1"
As #kriegaex mentioned in his comment, UTF-8 is the default encoding for JSON according to RFC 7159, so encoding is not strictly necessary. This is why by default json4s does not encode, just as the OP requested:
package so
import org.json4s.JsonDSL._
import org.json4s._
import org.json4s.native.JsonMethods._
object SOTest extends App {
val json = ("english" -> JString("serialization")) ~ ("japanese" -> JString("シリアライゼーション"))
println(pretty(render(json)))
}
Console log:
{
"english":"serialization",
"japanese":"シリアライゼーション"
}
However if for some compatibility reason you need the output to be encdeded, json4s supports that as well. If you add your own customJsonFormats like this, you get encoded output:
package so
import org.json4s.JsonDSL._
import org.json4s._
import org.json4s.native.JsonMethods._
object SOTest extends App {
val json = ("english" -> JString("serialization")) ~ ("japanese" -> JString("シリアライゼーション"))
implicit val customJsonFormats = new DefaultFormats {
override def alwaysEscapeUnicode: Boolean = true
}
println(pretty(render(json)))
}
Console log:
{
"english":"serialization",
"japanese":"\u30b7\u30ea\u30a2\u30e9\u30a4\u30bc\u30fc\u30b7\u30e7\u30f3"
}

How to create a JSON object in Scala?

First, I searched a lot on Google and StackOverflow for questions like that, but I didn't find any useful answers (to my big surprise).
I saw something about Play Framework, how to create JSON array in Java and how to create JSON objects in Java, but I don't want to use Play Framework and I don't know if the creation of JSON objects differ from Scala to Java.
Following is the JSON I want to create. Later I'll convert the object into a string to send it via a POST request (through an API call).
{
"start_relative": {
"value": "5",
"unit": "years"
},
"metrics": [
{
"name": "DP_391366" # S-Temperature - Celsius
},
{
"name": "DP_812682" # Sensor-A4 Luminosity
}
]
}
How can I do something like that in Scala?
You should use a library that handles serialization/deserialization.
I would consider choosing between Spray Json and Play Json.
I will explain to you how the process works with Play first, and it's very similar to that in Spray.
Let's say you have a class, and an object with an instance and a json as string:
case class MyClass(id: Int,
name: String,
description: String)
object Data {
val obj: MyClass = MyClass(1, "me", "awesome")
val str: String =
"""
|{
| "id": 1,
| "name": "me",
| "description": "awesome"
|}
""".stripMargin
}
For MyClass to be serialized/deserialized, you will need an implicit formatter, specific for this, so you will create an object that contains this formatter using Play.
trait MyClassPlayProtocol {
implicit val formatAbility = Json.format[Ability]
}
object MyClassPlayProtocol extends MyClassPlayProtocol
The serialization/deserialization will look something like this:
object PlayData {
import play.api.libs.json.JsValue
import play.api.libs.json.Json
import MyClassPlayProtocol._
import General._
val str2Json: JsValue = Json.parse(str)
val obj2Json: JsValue = Json.toJson(obj)
val json2Str: String = Json.stringify(str2Json)
val json2Obj: MyClass = obj2Json.as[MyClass]
}
In Spray, the protocol will look like this:
trait MyClassSprayProtocol extends DefaultJsonProtocol {
implicit val myClassFormat = jsonFormat3(MyClass)
}
object MyClassSprayProtocol extends MyClassSprayProtocol
and the serialization/deserialization:
object SprayData {
import spray.json._
import MyClassSprayProtocol._
import General._
val str2Json: JsValue = str.parseJson
val obj2Json: JsValue = obj.toJson
val json2Str: String = str2Json.compactPrint
val json2Obj: MyClass = obj2Json.convertTo[MyClass]
}
As you can see, it's mostly a matter of choice between this two. Both are still improved and probably will be in the near future.
Depending on the benchmark, you will find that one is better than the other by a few miliseconds (usually Spray).
I for one am using Spray at work and Play in some personal projects, and I can't say I found something fundamentally different from one to another.
EDIT:
And to finally answer your question, to go from MyClass to String (serialization), you will do something like this:
PLAY: Json.stringify(Json.toJson(myClass))
SPRAY: myClass.toJson.compactPrint
And the deserialization:
PLAY: Json.parse(string).as[MyClass]
SPRAY: myClass.parseJson.convertTo[MyClass]
You need to use a library if you dont want it to do by yourself there are serveral:
Spray Json - https://github.com/spray/spray-json
Lift Json - https://github.com/lift/lift/tree/master/framework/lift-base/lift-json/
Jerkson - https://github.com/codahale/jerkson
Jackson - You can use Jackson with the scala Module https://github.com/FasterXML/jackson-module-scala
Note: The cool Java Gson LIbrary looses a lot of Magic if you want to use it with Scala, because they dont know Collections. So if you want to use this library you have to convert the "Scala List" to java.util.List and after that use Gson.

Playframework Scala Specs2 JSON Matchers

I'm using the Play! framework and trying to work with JSON response messages in Specs2 tests with no success.
What I am trying to do is assert key->value pairs in a JsValue like in the example below ... but I can't get the matchers to correctly pass.
import org.specs2.mutable._
import play.api.libs.json.{Json, JsValue}
class JsonSpec extends Specification {
"Json Matcher" should {
"Correctly match Name->Value pairs" in {
val resultJson:JsValue = Json.parse("""{"name":"Yardies"}""")
resultJson must /("name" -> "Yardies")
}
"Correctly match Name->Value pairs with numbers as doubles" in {
val resultJson:JsValue = Json.parse("""{"id":1}""")
resultJson must /("id" -> 1.0)
}
}
}
Errors I get are
{name : Yardies} doesn't contain '(name,Yardies)'
and
{id : 1.0} doesn't contain '(id,1.0)'
Not very helpful, I imagine that it is something simple I am missing (new to both Scala and Play)
Steve
The JsonMatchers in specs2 should be tightened a little bit. They are Matcher[Any], where Any is supposed to have a toString method which can be parsed by Scala's json parser (and not Play's one).
The following specification works as expected:
class JsonSpec extends Specification {
"Json Matcher" should {
"Correctly match Name->Value pairs" in {
val resultJson = """{"name":"Yardies"}"""
resultJson must /("name" -> "Yardies")
}
"Correctly match Name->Value pairs with numbers as doubles" in {
val resultJson = """{"id":1}"""
resultJson must /("id" -> 1.0)
}
}
}
In your case I suspect that parsing the toString representation of Play's Json values returns something slightly different what the matchers are expecting. This will be fixed in the next specs2 release.

Convert scala list to Json object

I want to convert a scala list of strings, List[String], to an Json object.
For each string in my list I want to add it to my Json object.
So that it would look like something like this:
{
"names":[
{
"Bob",
"Andrea",
"Mike",
"Lisa"
}
]
}
How do I create an json object looking like this, from my list of strings?
To directly answer your question, a very simplistic and hacky way to do it:
val start = """"{"names":[{"""
val end = """}]}"""
val json = mylist.mkString(start, ",", end)
However, what you almost certainly want to do is pick one of the many JSON libraries out there: play-json gets some good comments, as does lift-json. At the worst, you could just grab a simple JSON library for Java and use that.
Since I'm familiar with lift-json, I'll show you how to do it with that library.
import net.liftweb.json.JsonDSL._
import net.liftweb.json.JsonAST._
import net.liftweb.json.Printer._
import net.liftweb.json.JObject
val json: JObject = "names" -> List("Bob", "Andrea", "Mike", "Lisa")
println(json)
println(pretty(render(json)))
The names -> List(...) expression is implicitly converted by the JsonDSL, since I specified that I wanted it to result in a JObject, so now json is the in-memory model of the json data you wanted.
pretty comes from the Printer object, and render comes from the JsonAST object. Combined, they create a String representation of your data, which looks like
{
"names":["Bob","Andrea","Mike","Lisa"]
}
Be sure to check out the lift documentation, where you'll likely find answers to any further questions about lift's json support.