I'm very new to Argonaut (which I'm forced to use since the codebase is older, using an old version of scalaz, and we have no intentions to update it since we're rewriting this outdated code from scratch), and I'm trying to find a way to dump some data into JSON.
I have an case class called Magnitude. For simplicity, define it as:
case class Magnitude(bandname: String)
What I want is a function that takes in a List[Magnitude] and outputs Json that I can chain together with other Json.
Right now, I'm doing:
def magnitudeFields(magnitudes: List[Magnitude]): Json =
magnitudes.foldLeft(jEmptyObject) { case (j, m) =>
("bandName" := m.bandname) ->: j
}
and then say I have an instantiation of List[Magnitude]:
val magList = List(Magnitude("R"), Magnitude("J"), Magnitude("_uc"), Magnitude("K"))
when I call magnitudeFields(magList) in a call like:
... previousJsonData ->: ('magnitudes' := magnitudeFields(magList)) ->: nextJsonData ...
I only receive the bandName: K and the other three preceding entries are not part of the JSON output by the function:
"magnitudes" : {
"bandName" : "K"
},
instead of what I want, which is:
"magnitudes" : {
"bandName" : "R"
"bandName" : "J"
"bandName" : "_uc"
"bandName" : "K"
},
Any help would be greatly appreciated. I suspect I'm doing something wrong by using jEmptyObject as the default to the foldLeft and may need to use something like jArray at some point (with the function returning List[Json] instead, but I'm not sure how I should be doing this.
You should just define an EncodeJson instance for Magnitude and let Argonaut deal with the rest of the logic:
import argonaut._, Argonaut._
case class Magnitude(bandname: String)
object Magnitude {
implicit val encode: EncodeJson[Magnitude] =
EncodeJson { m =>
jSingleObject("bandName", jString(m.bandname))
}
}
object Main {
val magList = List(Magnitude("R"), Magnitude("J"), Magnitude("_uc"), Magnitude("K"))
def main(args: Array[String]): Unit = {
val prefix = "prefix" := "prefix"
val suffix = "suffix" := "suffix"
val json = prefix ->: ("magnitudes" := magList) ->: suffix ->: jEmptyObject
println(json.asJson.spaces2)
//{
// "prefix" : "prefix",
// "magnitudes" : [
// {
// "bandName" : "R"
// },
// {
// "bandName" : "J"
// },
// {
// "bandName" : "_uc"
// },
// {
// "bandName" : "K"
// }
// ],
// "suffix" : "suffix"
//}
}
}
Related
I wrote a function that takes as input a structure (of type Struct) that can contain primitive types as well as Map and Set objects, and converts it into something that can be JSON-serialized. Examples of input:
let a = 'hello'; // -> "hello"
let b = new Map<string, Set<number>>([['a', new Set<number>([1, 3])]]); // -> {"a": [1, 3]}
let c = {a: new Set<number[]>([[1, 2]])}; // -> {"a": [[1, 2]]}
let d = [{e: false}]; // -> [{"e": false}]
However, I find my code particularly verbose, and I am really not sure about its safety:
type Json = string | number | boolean | null | Json[] | { [key: string]: Json };
type Struct = Json | Struct[] | { [key: string]: Struct } | Map<string, Struct> | Set<Struct>;
function isJson(test: any): test is Json {
if (test == null || ['string', 'number', 'boolean'].indexOf(typeof test) != -1)
return true;
if (Array.isArray(test)) {
// if at least one of the values is not JSON serializable, the array is not JSON-serializable
for (let value of test)
if (!isJson(value))
return false;
return true;
}
if (typeof test == 'object') {
// if it is not a plain object, the object is not JSON-serializable
if (Object.getPrototypeOf(test) != null && test.constructor != Object)
return false;
// if there are symbol properties, the object is not JSON-serializable
if (Object.getOwnPropertySymbols(test).length > 0)
return false;
// if at least one of the values is not JSON serializable, the object is not JSON-serializable
for (let [key, value] of Object.entries(test))
if (!isJson(test[key]))
return false;
return true;
}
return false;
}
function toJson(struct: Struct) {
let json: Json = null;
if (isJson(struct))
json = struct;
else if (Array.isArray(struct) || struct instanceof Set) {
json = [];
let structCast = struct instanceof Set ? struct as Set<Struct> : struct as Struct[];
for (let value of structCast)
json.push(toJson(value));
}
else if (Object.getPrototypeOf(struct) == null || struct.constructor == Object || struct instanceof Map) {
json = {};
let structCast = struct instanceof Map ? struct as Map<string, Struct> : Object.entries(struct);
for (let [key, value] of structCast)
json[key] = toJson(value);
}
return json;
}
I am especially annoyed by the isJson function. Is there no way to get rid of it? I know that most of the typechecking is lost after compilation in TypeScript, but is there a better way do what I want?
Thank you for your help.
I'm going to assume that you're writing TypeScript code and the users of toJson() are also writing TypeScript code. If that's not true, and if the toJson() function is called by some pure JavaScript code somewhere, then you lose any compile time guarantees and will need to add as many runtime checks to toJson() as you feel necessary. From here on out I assume all the relevant code will pass through a compile step.
In this case, you can probably stop worrying about some of the strange edge cases you're dealing with inside isJson(). If the TypeScript Struct type definition already prohibits something then you don't need to write runtime code to deal with it. On the other hand, anything that Struct allows but you'd like to disallow will need runtime checks. For example, it's not generally possible in TypeScript to say "reject object types with any symbol-valued keys". You can use generics to make the compiler reject any object types known to have such keys, but this is more complex, and does not work on the general case where an object of type like {a: string} may or may not have additional unknown keys. I would suggest that, if you really need such checks, to do so by throw statement, since you can't catch them before runtime anyway (unless you want toJson() to return Json | undefined or some other failure marker).
So, let's approach this by merging the isJson() checks we need into the toJson() function and eliminating isJson(). Here's one possible implementation of toJson():
function toJson(struct: Struct): Json {
if (struct === null || typeof struct === "string" ||
typeof struct === "number" || typeof struct === "boolean") {
return struct;
}
if (Array.isArray(struct)) {
return struct.map(toJson);
}
if (struct instanceof Set) {
return Array.from(struct).map(toJson);
}
if (struct instanceof Map) {
return Object.fromEntries(
Array.from(struct).map(([k, v]) => [k, toJson(v)])
);
}
return Object.fromEntries(
Object.entries(struct).map(([k, v]) => [k, toJson(v)])
);
}
The four primitive checks at the beginning could be refactored to an array.includes() or similar code, but then the TypeScript compiler won't understand what's going on and you'd need return struct as Json. The way I've got it is more verbose but the compiler is 100% sure that struct is a valid Json inside that block. Either way is fine.
The rest of the checks are just using built in JavaScript functions and methods like Array.from(), Array.prototype.map(), Object.entries() and Object.fromEntries().
As for the typings, if you want, you can give toJson() a call signature that tries to actually map the input type to the output type. For example:
type ToJson<T> =
Struct extends T ? Json :
Json extends T ? T :
T extends Map<infer K, infer S> ? { [P in Extract<K, string>]: ToJson<S> } :
T extends Set<infer S> ? ToJson<S>[] :
T extends object ? { [K in keyof T]: ToJson<T[K]> } :
T;
function toJson<T extends Struct>(struct: T): ToJson<T>;
ToJson<T> is similar to the implementation, but represented as a type operation. If the input type T is just Struct, then output Json. If it's already a subtype of Json, then output T. If it's a Map<K, S>, then produce an object whose keys are K and whose values are ToJson<S>. If it's a Set<S>, then output an array whose elements are ToJson<S>. And if it's an object (arrays are included), then map all the entries via ToJson. Finally, if it's just a primitive type, return the primitive type.
Let's see if this all works:
const foo = {
str: "hello",
num: 123,
boo: true,
nul: null,
jArr: ["one", 2, { x: 3 }],
jObj: { a: "one", b: 2, c: { x: 3 } },
set: new Set([1, 2, 3]),
map: new Map([["a", 1], ["b", 2]])
}
const json = toJson(foo);
The type seen by the compiler is as follows:
/* const json: {
str: string;
num: number;
boo: boolean;
nul: null;
jArr: (string | number | {
x: number;
})[];
jObj: {
a: string;
b: number;
c: {
x: number;
};
};
set: number[];
map: {
[x: string]: number;
};
} */
Note how the set and map properties have been transformed. Let's make sure the implementation worked also:
console.log(json);
/* {
"str": "hello",
"num": 123,
"boo": true,
"nul": null,
"jArr": [
"one",
2,
{
"x": 3
}
],
"jObj": {
"a": "one",
"b": 2,
"c": {
"x": 3
}
},
"set": [
1,
2,
3
],
"map": {
"a": 1,
"b": 2
}
} */
Also good. The fact that the compiler type and the implementation value agree means that you can, if you want, perform other operations on it and get some type checking and hints:
console.log(json.set[0].toFixed(2)); // 1.00
Playground link to code
I am trying to use Circe's custom codecs to decode json to a List of a trait Feature which is broken down into two case classes:
trait Feature {
def name: String
def index: Int
def frequency: Int
}
case class NumericFeature(name, index, frequency) extends Feature
case class SetFeature(name, index, set, frequency) extends Feature
from the json shown here:
[
{
"name" : "ElectionReturns_G16CountyTurnoutAllRegisteredVoters",
"index" : 1770,
"frequency" : 2992
},
{
"name" : "CommercialDataLL_Home_Owner_Or_Renter",
"index" : 1112,
"set" : [
"Likely Homeowner",
"",
"Likely Renter",
],
"frequency" : 2537
},
.
.
.
.
]
This is the code that works for a single instance of a case class aka:
{
"name" : "ElectionReturns_G16CountyTurnoutAllRegisteredVoters",
"index" : 1770,
"frequency" : 2992
}
import io.circe.Json
import io.circe._
import io.circe.parser._
implicit val decoder: Decoder[Feature] = new Decoder[Feature] {
final def apply(h: HCursor): Decoder.Result[Feature] = for {
name <- h.get[String]("name")
index <- h.get[Int]("index")
set <- h.getOrElse[Set[String]]("set")(Set[String]())
frequency <- h.get[Int]("frequency")
feature: Feature = set match {
case s if s.isEmpty => new NumericFeature(name, index, frequency)
case _ => new SetFeature(name, index, set, frequency)
}
} yield feature
}
val decoded = decoder.decodeJson(parse(json).getOrElse(Json.Null))
How can I adapt this to output a list? I have tried just changing the output type of the decoder to List[Feature] because I thought the for loop would yield a list, but this isn't the case.
Here is my attempt at a Decoder returning a Decoder[List[Feature]]:
implicit val decoder: Decoder[List[Feature]] = new Decoder[List[Feature]] {
final def apply(h: HCursor): Decoder.Result[List[Feature]]= {
val feature = for {
name <- h.get[String]("name")
index <- h.get[Int]("index")
set <- h.getOrElse[Set[String]]("set")(Set[String]())
frequency <- h.get[Int]("frequency")
feature: Feature = set match {
case s if s.isEmpty => new NumericFeature(name, index, frequency)
case _ => new SetFeature(name, index, set, frequency)
}
} yield feature
feature.asInstanceOf[Decoder.Result[List[Feature]]]
}
}
val decoded = decoder.decodeJson(parse(json).getOrElse(Json.Null))
Just use io.circe.parser.decode given you have the Decoder[Feature] in scope:
io.circe.parser.decode[Seq[Feature]](json)
You don't have to provide a Decoder[Seq[Feature]].
try to use custom decoder. This approach works for me:
import cats.Show
import cats.implicits._
import io.circe.Decoder
object Feature {
implicit val featureDecoder: Decoder[Feature] = List[Decoder[Feature]](
Decoder[NumericFeature].widen,
Decoder[SetFeature].widen
).reduceLeft(_ or _)
}
Is there a straight forward way to format a JSON string in scala?
I have a JSON String like this:
val json = {"foo": {"bar": {"baz": T}}}
Can I use a function f such that:
f(json) = {
"foo":
{"bar":
{"baz": T}
}
}
I know the formatting I have done in my answer is no perfect, but you get the point. And yes, can it be done without using Play Framework?
In case you are using Play Framework you could use Json.prettyPrint method to format JsValue:
import play.api.libs.json.Json
val str = """{"foo": {"bar": {"baz": "T"}}}"""
val jsValue = Json parse str
// JsValue = {"foo":{"bar":{"baz":"T"}}}
Json prettyPrint jsValue
// String =
// {
// "foo" : {
// "bar" : {
// "baz" : "T"
// }
// }
// }
In case you are using scala.util.parsing.json you have to create such method by yourself. For instance:
def format(t: Any, i: Int = 0): String = t match {
case o: JSONObject =>
o.obj.map{ case (k, v) =>
" "*(i+1) + JSONFormat.defaultFormatter(k) + ": " + format(v, i+1)
}.mkString("{\n", ",\n", "\n" + " "*i + "}")
case a: JSONArray =>
a.list.map{
e => " "*(i+1) + format(e, i+1)
}.mkString("[\n", ",\n", "\n" + " "*i + "]")
case _ => JSONFormat defaultFormatter t
}
val jsn = JSON.parseRaw("""{"foo": {"bar": {"baz": "T"}, "arr": [1, 2, "x"]}, "foo2": "a"}""").get
// JSONType = {"foo" : {"bar" : {"baz" : "T"}, "arr" : [1.0, 2.0, "x"]}, "foo2" : "a"}
format(jsn)
// String =
// {
// "foo": {
// "bar": {
// "baz": "T"
// },
// "arr": [
// 1.0,
// 2.0,
// "x"
// ]
// },
// "foo2": "a"
// }
Note that this is not an efficient implementation.
I thought I read somewhere that Typesafe was considering separating their JSON processing out of Play, so look into that to apply #senia's solution first.
Otherwise, look at Jackson--or more precisely, the Scala wrapper for Jackson:
val mapper = new ObjectMapper()
mapper.registerModule(DefaultScalaModule)
val writer = mapper.writerWithDefaultPrettyPrinter
val json = writer.writeValueAsString(Object value)
I've also heard that the kids are really into Scala Pickling, which apparently has pretty printing as well.
Given the following JSON...
{ "id":"1234",
"name" -> "joe",
"tokens: [{
"id":"1234",
"id":"2345"
}]
}
... I need to replace the value of all the ids by xxxx like this:
{ "id":"xxxx",
"name" -> "joe",
"tokens: [{
"id":"xxxx",
"id":"xxxx"
}]
}
Let's start create the JSON tree:
val json = Json.obj(
"id" -> "1234",
"name" -> "joe",
"tokens" -> Json.arr(
Json.obj("id" -> "1234"),
Json.obj("id" -> "2345")
)
)
json: play.api.libs.json.JsObject = {"id":"1234","name":"joe","tokens":[{"id":"1234"},{"id":"2345"}]}
Then, getting all the ids is very simple:
json \\ "id"
res64: Seq[play.api.libs.json.JsValue] = List("1234", "1234", "2345")
Now, how do I replace the value of all the ids by xxxx?
There doesn't appear to be a nice way to do this with the standard Play JSON library, although I'd be happy to be proved wrong in that regard. You can however do it easily using the play-json-zipper extensions:
import play.api.libs.json._
import play.api.libs.json.extensions._
val json = Json.obj(
"id" -> "1234",
"name" -> "joe",
"tokens" -> Json.arr(
Json.obj("id" -> "1234"),
Json.obj("id" -> "2345")
)
)
// Using `updateAll` we pattern match on a path (ignoring
// the existing value, as long as it's a string) and replace it
val transformed = json.updateAll {
case (__ \ "id", JsString(_)) => JsString("xxxx")
}
// play.api.libs.json.JsValue = {"id":"xxxx","name":"joe","tokens":[{"id":"xxxx"},{"id":"xxxx"}]}
To make that a re-usable function:
def replaceValue(json: JsValue, key: String, replacement: String) = json.updateAll {
case (__ \ path, JsString(_)) if path == key => JsString(replacement)
}
The json-zipper extensions are still "experimental", but if you want to add them to your project add the following to your project/Build.scala appDependencies:
"play-json-zipper" %% "play-json-zipper" % "1.0"
and the following resolver:
"Mandubian repository releases" at "https://github.com/mandubian/mandubian-mvn/raw/master/releases/"
Probably it isn't most efficient way to do it, but you can try to convert your JSON to an object copy it with new fields and then convert it back to json. Unfortunately currently I don't have environment to check the code, but it should be something like this:
case class MyId(id: String)
case class MyObject(id: String, name: String, tokens: List[MyId])
implicit val idFormat = Json.format[MyId]
implicit val objectFormat = Json.format[MyObject]
val json = Json.parse(jsonString)
val jsResult = Json.fromJson[MyObject](json)
val obj = jsResult match {
case JsSuccess(s, _) => s
case _ => throw new IllegalStateException("Unexpected")
}
val newObj = obj.copy(id = "xxxx")
val result = Json.toJson(newObj)
Is there a straight forward way to format a JSON string in scala?
I have a JSON String like this:
val json = {"foo": {"bar": {"baz": T}}}
Can I use a function f such that:
f(json) = {
"foo":
{"bar":
{"baz": T}
}
}
I know the formatting I have done in my answer is no perfect, but you get the point. And yes, can it be done without using Play Framework?
In case you are using Play Framework you could use Json.prettyPrint method to format JsValue:
import play.api.libs.json.Json
val str = """{"foo": {"bar": {"baz": "T"}}}"""
val jsValue = Json parse str
// JsValue = {"foo":{"bar":{"baz":"T"}}}
Json prettyPrint jsValue
// String =
// {
// "foo" : {
// "bar" : {
// "baz" : "T"
// }
// }
// }
In case you are using scala.util.parsing.json you have to create such method by yourself. For instance:
def format(t: Any, i: Int = 0): String = t match {
case o: JSONObject =>
o.obj.map{ case (k, v) =>
" "*(i+1) + JSONFormat.defaultFormatter(k) + ": " + format(v, i+1)
}.mkString("{\n", ",\n", "\n" + " "*i + "}")
case a: JSONArray =>
a.list.map{
e => " "*(i+1) + format(e, i+1)
}.mkString("[\n", ",\n", "\n" + " "*i + "]")
case _ => JSONFormat defaultFormatter t
}
val jsn = JSON.parseRaw("""{"foo": {"bar": {"baz": "T"}, "arr": [1, 2, "x"]}, "foo2": "a"}""").get
// JSONType = {"foo" : {"bar" : {"baz" : "T"}, "arr" : [1.0, 2.0, "x"]}, "foo2" : "a"}
format(jsn)
// String =
// {
// "foo": {
// "bar": {
// "baz": "T"
// },
// "arr": [
// 1.0,
// 2.0,
// "x"
// ]
// },
// "foo2": "a"
// }
Note that this is not an efficient implementation.
I thought I read somewhere that Typesafe was considering separating their JSON processing out of Play, so look into that to apply #senia's solution first.
Otherwise, look at Jackson--or more precisely, the Scala wrapper for Jackson:
val mapper = new ObjectMapper()
mapper.registerModule(DefaultScalaModule)
val writer = mapper.writerWithDefaultPrettyPrinter
val json = writer.writeValueAsString(Object value)
I've also heard that the kids are really into Scala Pickling, which apparently has pretty printing as well.