Play [Scala]: How to flatten a JSON object - json

Question

Given the following JSON...
{
"metadata": {
"id": "1234",
"type": "file",
"length": 395
}
}
... how do I convert it to
{
"metadata.id": "1234",
"metadata.type": "file",
"metadata.length": 395
}
Tx.

Answer 1

You can do this pretty concisely with Play's JSON transformers. The following is off the top of my head, and I'm sure it could be greatly improved on:
import play.api.libs.json._
val flattenMeta = (__ \ 'metadata).read[JsObject].flatMap(
_.fields.foldLeft((__ \ 'metadata).json.prune) {
case (acc, (k, v)) => acc andThen __.json.update(
Reads.of[JsObject].map(_ + (s"metadata.$k" -> v))
)
}
)
And then:
val json = Json.parse("""
{
"metadata": {
"id": "1234",
"type": "file",
"length": 395
}
}
""")
And:
scala> json.transform(flattenMeta).foreach(Json.prettyPrint _ andThen println)
{
"metadata.id" : "1234",
"metadata.type" : "file",
"metadata.length" : 395
}
Just change the path if you want to handle metadata fields somewhere else in the tree.
Note that using a transformer may be overkill here—see e.g. Pascal Voitot's input in this thread, where he proposes the following:
(json \ "metadata").as[JsObject].fields.foldLeft(Json.obj()) {
case (acc, (k, v)) => acc + (s"metadata.$k" -> v)
}
It's not as composable, and you'd probably not want to use as in real code, but it may be all you need.

Answer 2

This is definitely not trivial, but possible by trying to flatten it recursively. I haven't tested this thoroughly, but it works with your example and some other basic one's I've come up with using arrays:
object JsFlattener {
def apply(js: JsValue): JsValue = flatten(js).foldLeft(JsObject(Nil))(_++_.as[JsObject])
def flatten(js: JsValue, prefix: String = ""): Seq[JsValue] = {
js.as[JsObject].fieldSet.toSeq.flatMap{ case (key, values) =>
values match {
case JsBoolean(x) => Seq(Json.obj(concat(prefix, key) -> x))
case JsNumber(x) => Seq(Json.obj(concat(prefix, key) -> x))
case JsString(x) => Seq(Json.obj(concat(prefix, key) -> x))
case JsArray(seq) => seq.zipWithIndex.flatMap{ case (x, i) => flatten(x, concat(prefix, key + s"[$i]")) }
case x: JsObject => flatten(x, concat(prefix, key))
case _ => Seq(Json.obj(concat(prefix, key) -> JsNull))
}
}
}
def concat(prefix: String, key: String): String = if(prefix.nonEmpty) s"$prefix.$key" else key
}
JsObject has the fieldSet method that returns a Set[(String, JsValue)], which I mapped, matched against the JsValue subclass, and continued consuming recursively from there.
You can use this example by passing a JsValue to apply:
val json = Json.parse("""
{
"metadata": {
"id": "1234",
"type": "file",
"length": 395
}
}
"""
JsFlattener(json)
We'll leave it as an exercise to the reader to make the code more beautiful looking.

Answer 3

Here's my take on this problem, based on #Travis Brown's 2nd solution.
It recursively traverses the json and prefixes each key with its parent's key.
def flatten(js: JsValue, prefix: String = ""): JsObject = js.as[JsObject].fields.foldLeft(Json.obj()) {
case (acc, (k, v: JsObject)) => {
if(prefix.isEmpty) acc.deepMerge(flatten(v, k))
else acc.deepMerge(flatten(v, s"$prefix.$k"))
}
case (acc, (k, v)) => {
if(prefix.isEmpty) acc + (k -> v)
else acc + (s"$prefix.$k" -> v)
}
}
which turns this:
{
"metadata": {
"id": "1234",
"type": "file",
"length": 395
},
"foo": "bar",
"person": {
"first": "peter",
"last": "smith",
"address": {
"city": "Ottawa",
"country": "Canada"
}
}
}
into this:
{
"metadata.id": "1234",
"metadata.type": "file",
"metadata.length": 395,
"foo": "bar",
"person.first": "peter",
"person.last": "smith",
"person.address.city": "Ottawa",
"person.address.country": "Canada"
}

Answer 4

#Trev has the best solution here, completely generic and recursive, but it's missing a case for array support. I'd like something that works in this scenario:
turn this:
{
"metadata": {
"id": "1234",
"type": "file",
"length": 395
},
"foo": "bar",
"person": {
"first": "peter",
"last": "smith",
"address": {
"city": "Ottawa",
"country": "Canada"
},
"kids": ["Bob", "Sam"]
}
}
into this:
{
"metadata.id": "1234",
"metadata.type": "file",
"metadata.length": 395,
"foo": "bar",
"person.first": "peter",
"person.last": "smith",
"person.address.city": "Ottawa",
"person.address.country": "Canada",
"person.kids[0]": "Bob",
"person.kids[1]": "Sam"
}
I've arrived at this, which appears to work, but seems overly verbose. Any help in making this pretty would be appreciated.
def flatten(js: JsValue, prefix: String = ""): JsObject = js.as[JsObject].fields.foldLeft(Json.obj()) {
case (acc, (k, v: JsObject)) => {
val nk = if(prefix.isEmpty) k else s"$prefix.$k"
acc.deepMerge(flatten(v, nk))
}
case (acc, (k, v: JsArray)) => {
val nk = if(prefix.isEmpty) k else s"$prefix.$k"
val arr = flattenArray(v, nk).foldLeft(Json.obj())(_++_)
acc.deepMerge(arr)
}
case (acc, (k, v)) => {
val nk = if(prefix.isEmpty) k else s"$prefix.$k"
acc + (nk -> v)
}
}
def flattenArray(a: JsArray, k: String = ""): Seq[JsObject] = {
flattenSeq(a.value.zipWithIndex.map {
case (o: JsObject, i: Int) =>
flatten(o, s"$k[$i]")
case (o: JsArray, i: Int) =>
flattenArray(o, s"$k[$i]")
case a =>
Json.obj(s"$k[${a._2}]" -> a._1)
})
}
def flattenSeq(s: Seq[Any], b: Seq[JsObject] = Seq()): Seq[JsObject] = {
s.foldLeft[Seq[JsObject]](b){
case (acc, v: JsObject) =>
acc:+v
case (acc, v: Seq[Any]) =>
flattenSeq(v, acc)
}
}

Answer 5

Thanks m-z, it is very helpful. (I'm not so familiar with Scala.)
I'd like to add a line for "flatten" working with primitive JSON array like "{metadata: ["aaa", "bob"]}".
def flatten(js: JsValue, prefix: String = ""): Seq[JsValue] = {
// JSON primitive array can't convert to JsObject
if(!js.isInstanceOf[JsObject]) return Seq(Json.obj(prefix -> js))
js.as[JsObject].fieldSet.toSeq.flatMap{ case (key, values) =>
values match {
case JsBoolean(x) => Seq(Json.obj(concat(prefix, key) -> x))
case JsNumber(x) => Seq(Json.obj(concat(prefix, key) -> x))
case JsString(x) => Seq(Json.obj(concat(prefix, key) -> x))
case JsArray(seq) => seq.zipWithIndex.flatMap{ case (x, i) => flatten(x, concat(prefix, key + s"[$i]")) }
case x: JsObject => flatten(x, concat(prefix, key))
case _ => Seq(Json.obj(concat(prefix, key) -> JsNull))
}
}
}

Answer 6

Based on previous solutions, have tried to simplify the code a bit
def getNewKey(oldKey: String, newKey: String): String = {
if (oldKey.nonEmpty) oldKey + "." + newKey else newKey
}
def flatten(js: JsValue, prefix: String = ""): JsObject = {
if (!js.isInstanceOf[JsObject]) return Json.obj(prefix -> js)
js.as[JsObject].fields.foldLeft(Json.obj()) {
case (o, (k, value)) => {
o.deepMerge(value match {
case x: JsArray => x.as[Seq[JsValue]].zipWithIndex.foldLeft(o) {
case (o, (n, i: Int)) => o.deepMerge(
flatten(n.as[JsValue], getNewKey(prefix, k) + s"[$i]")
)
}
case x: JsObject => flatten(x, getNewKey(prefix, k))
case x => Json.obj(getNewKey(prefix, k) -> x.as[JsValue])
})
}
}
}