I've got deeply nested JSON parsers (using json4s.jackson) that I'm trying to simplify using case classes.
My problem is... some of the fields start with numbers, but scala cannot have an arg name that starts with a numeric character.
Example:
import org.json4s.jackson.JsonMethods._
import org.json4s._
implicit val formats = DefaultFormats
val jsonStr = """{"5gLog":{"i":99}}""" // <--- note the field "5gLog"
val jval = parse(jsonStr)
case class Raw5gLog(i: Int)
val raw5gLog = (jval \ "5gLog").extract[Raw5gLog]
This works. But what I need to do, because these fields are nested deep within the JSON... is something like this...
val jsonStr = """{"xgLog":{"i":99}}"""
val jval = parse(jsonStr)
case class RawRecord(xgLog: Raw5gLog)
val rawRecord = jval.extract[RawRecord]
This would work... if the fields were named like xgLog, but the fields are actually named like 5gLog as above, and I can't give an arg name to a class like 5gLog...
case class RawRecord(5gLog: Raw5gLog)
// error: Invalid literal number
I thought about something like
parse(jsonStr.replace("\"5g", "\"fiveg"))
But there's real data, beyond the field names, in the JSON that can be impacted.
The best solution I can figure is to add extra apply methods to the affected case classes...
case class RawRecord(fivegLog: Raw5gLog)
object RawRecord {
def apply(jval: JValue): RawRecord =
RawRecord( (jval \ "5gLog").extract[Raw5gLog] )
}
val rawRecord = RawRecord(jval)
But I feel like every time I make some structurally different workaround like this for an edge case, it's always the beginning of my code turning into a mess. I could give every case class a new apply method and use it for everything, but it seems like a lot of extra code for a small subset of the data.
Is there a better way?
Scala can use any string as a variable name, but you may have to quote it with backticks:
case class RawRecord(`5gLog`: Raw5gLog)
You also need to do this if you have a field called type or any other reserved word. This is how - can be a function name, for example.
Related
I am using jsoniter-scala to map Json files to Scala objects (case classes). To make it simple, assume I have two types of Json files, and thus two case classes, for instance:
case class JsonTypeOne(name: String, id: Long)
case class JsonTypeTwo(name: String, notes: Seq[String])
When I use
val codec: JsonValueCodec[JsonTypeOne] = JsonCodecMaker.make
or
val codec: JsonValueCodec[JsonTypeTwo] = JsonCodecMaker.make
everything works perfectly and as expected. Now, I'd like to create a function that takes the class as an in input parameter and passes it on to jsoniter. In pseudo-code this would be something like this:
def getJsonWithClass(aClass: SomeType, jsonString: String) {
val codec: JsonValueCodec[aClass] = JsonCodecMaker.make
readFromString(jsonString)(codec)
}
and would then be called as follows:
getJsonWithClass(JsonTypeOne, json1String)
getJsonWithClass(JsonTypeTwo, json2String)
Tried a number of variations, including defining a trait for the case classes and using that as "SomeType" or using generics, but thus far without success.
Any suggestions on how to resolve this issue would be highly appreciated.
As it is not possible to parameterize the codec generation as originally intended (thanks for the clarifications Luis and Andriy), I ended up with a workaround. It's not as cleanly decoupled as if parameterized typing would be possible, but at least the coupling is localized in a pattern matching and can easily be extended:
def getJsonWithClass[T](aJsonClass: T, jsonString: String):T = {
val jsonCodec = aJsonClass match {
case JsonTypeOne => JsonCodecMaker.make[JsonTypeOne]
case JsonTypeTwo => JsonCodecMaker.make[JsonTypeTwo]
}
val jsonObj = readFromString(jsonString)(jsonCodec)
jsonObj.asInstanceOf[T]
}
This can then be called as in
getJsonWithClass(JsonTypeOne, aJsonTypeOneString)
I am looking for a solution on how to create and/or append multiple fields to a play.api.libs.json.JsObject. I have a list of (string -> JsValue) in which I don't know how many there are, nor specifically which field names are available.
Json.obj does accepts multiple fields, but looks like you cannot pass in a list. Instead you need to specifically pass all the fields in like this:
Json.obj((k1 -> v1), (k2 -> v2), ...)
Which won't work for my use case. I would want something like this instead:
Json.obj(listOfFields) // listOfFields: List[(String, JsValue)]
Thank you in advance!
By having a look at the documentation, it's visible that JsObject companion extends (Seq[(String, JsValue)]) ⇒ JsObject, so rather than calling Json.obj (with extra conversion):
import play.api.libs.json._
def foo(fields: List[(String, JsValue)]): JsObject = JsObject(fields)
I don't know that specific library, but this should break the list into a set of separate arguments:
Json.obj(listOfFields:_*)
I have used Circe previously for case class serialization / deserialization, and love how it can be used without the boilerplate code required by other Scala JSON libraries, but I'm running into an issue now I'm not sure how to resolve. I have an ADT (a sealed trait with several case class instances) that I would like to treat (from my Akka Http Service, using akka-http-json) generically (ie, return a List[Foo], where Foo is the trait-type), but when I do so using Circe's auto-deriviation (via Shapeless), it serializes the instances using the specific case class name as a 'discriminator' (eg, if my List[Foo] contains instances of Foo1, then each element in the resulting serialized list will have the key Foo1). I would like to eliminate the type name as a discriminator (ie, so that instead of having each element in the sequence prefixed with the type name-- eg, "Foo1": {"id : "1", name : "First",...}, I just want to serialize the case class instances to contain the fields of the case class: eg, {"id":"1,"name:"First",...}...Essentially, I'd like to eliminate the type name keys (I don't want the front-end to have to know what concrete case class each element belongs to on the back-end).All elements in the list to be serialized will be of the same concrete-type, all of which would be subtypes of my ADT (trait) type. I believe this can be done using Circe's semi-auto derivation, though I haven't had a chance to figure out exactly how. Basically, I would like to use as much of Circe's auto-derivation as possible, but eliminate outer-level class names from appearing in the resulting JSON. Any help / suggestions would be very much appreciated! Thanks!
you can do it following the instruction in the doc: https://circe.github.io/circe/codecs/adt.html
import cats.syntax.functor._
import io.circe.{ Decoder, Encoder }, io.circe.generic.auto._
import io.circe.syntax._
object GenericDerivation {
implicit val encodeEvent: Encoder[Event] = Encoder.instance {
case foo # Foo(_) => foo.asJson
case bar # Bar(_) => bar.asJson
case baz # Baz(_) => baz.asJson
case qux # Qux(_) => qux.asJson
}
implicit val decodeEvent: Decoder[Event] =
List[Decoder[Event]](
Decoder[Foo].widen,
Decoder[Bar].widen,
Decoder[Baz].widen,
Decoder[Qux].widen
).reduceLeft(_ or _)
}
import GenericDerivation._
import io.circe.parser.decode
decode[Event]("""{ "i": 1000 }""")
// res0: Either[io.circe.Error,Event] = Right(Foo(1000))
(Foo(100): Event).asJson.noSpaces
// res1: String = {"i":100}
This may not be the best answer, but after some more searching this is what I've been able to find. Instead of having the class name as a key in the Json produced, it can be serialized as a field as following:
implicit val genDevConfig: Configuration = Configuration.default.withDescriminator("type")
(you can use whatever field name here you'd like; Travis Brown's previous example for a similar issue used a field named what_am_i). So my apologies-- I do not yet know if there is a canonical or widely accepted solution to this problem, especially one that will easily work with Akka Http, using libraries such as akka-http-json, where I still seem to be encountering some issues, though I'm sure I'm probably overlooking something obvious! Anyway, my apologies for asking a question that seems to come up repeatedly!
I'm working with JSON4S and it handles missing fields correctly, when the corresponding object's field is an option.
I'm using
implicit val formats =
Serialization.formats(NoTypeHints) +
new org.json4s.ext.EnumNameSerializer(E)
and read[T](json).
It's perfect, except one thing. I'd like to designate between missing and null fields. What I'd like to have for each field of my T is to have something like a Triple instead of Option, where this triple would be either a Some(t), Missing or Nullified in analogue to how Option works. I have no problem in defining such a structure, but unfortunatelly I'm not so familiar with how JSON4S works, or how could I (maybe) "intercept" the parsing of a field to achieve such a value-extraction.
As an alternative, it also would be great if the parser would set the corresponding field of T to null if the field is field: null instead of setting it to None. This would not feel Scalaish I think.
You should probably implement a custom serializer for T. I would use the following format because it allows for more flexibility and order independent input:
import org.json4s.CustomSerializer
import org.json4s.JsonAST._
import org.json4s.JsonDSL._
case class Animal(name: String, nrLegs: Int)
class AnimalSerializer extends CustomSerializer[Animal](format => ( {
case jsonObj: JObject =>
val name = (jsonObj \ "name") //JString
val nrLegs = (jsonObj \ "nrLegs") //JInt
Animal(name.extract[String], nrLegs.extract[Int])
}, {
case animal: Animal =>
("name" -> animal.name) ~
("nrLegs" -> animal.nrLegs)
}
))
To handle null/empty values take a look at the JSValue trait. For null values you should match with JNull and for non present values with JNothing.
Is there any already available library that can convert JSON string (most probably more than 1 rows of data) to CSV file.
I googled a lot for any such libraries in Scala, but I could find none.
What I am required to do is retrieve Data from a DB source, the result-set are in JSON format, and convert them into CSV.
Before what I did was converted the JSON into relevant Seq[case-class] and tried to used libraries like:
Scala-csv (tototoshi)
Might-csv
But these didn't prove much useful when in case of case class containing deep hierarchies.
Any Suggestions??
product-collections will convert a Seq[case class] to csv.
case class Foo(a:Int,b:String)
Seq(Foo(1,"aa"),Foo(2,"bb")).csvIterator.mkString("\n")
res27: String =
1,"aa"
2,"bb"
To deserialize json I'd probably use scala pickling.
"Deep hierarchies" are likely to be problematic.
Not entirely sure what you mean by deep hierarchies. Something like this?
case class Baz(i: Int)
case class Bar(baz: Baz)
case class Foo(bar: Bar)
In which case kantan.csv can answer part of the equation: turning deep case class hierarchies into CSV data. It's fairly trivial, provided you don't mind a shapeless dependency:
import kantan.csv.ops._
import kantan.csv.generic.codecs._
val fs: List[Foo] = ???
fs.foldLeft(new File("output.csv").asCsvWriter[Foo])(_ write _).close
If you have an issue with shapeless, you can provide encoders for your case classes yourself:
import kantan.csv._
implicit val bazCodec = Codec.caseCodec1(Baz.apply, Baz.unapply)(0)
implicit val barCodec = Codec.caseCodec1(Bar.apply, Bar.unapply)(0)
implicit val fooCodec = Codec.caseCodec1(Foo.apply, Foo.unapply)(0)
A bit more boilerplatey, but still acceptable, I think.