We want to save our data models into DynamoDB. We use scanamo with alpakka for nonblocking I/O.
For numerous reasons, we don't want the keys and data to be auto generated to dynamo format. We already have Play-Json formatters for all our case classes and want the data to be saved in Dynamo from JsObjects.
For saving the data as JsObject, each repository has the following
import com.gu.scanamo.Table
val table = Table[JsObject](name)
Always end up receiving this error:
could not find implicit value for evidence parameter of type com.gu.scanamo.DynamoFormat[play.api.libs.json.JsObject]
I can't find a way to make it accept JsObject or create a formatter that will fit.
Will much appreciate any help.
Sidenote: I've looked at PlayDynamo-Repo but they actually create the whole request from scratch and we'd like to use scanamo's API.
I ended up using the following code which works just as expected. I cannot share the sub-functions but it should give the general idea.
implicit val dynamoFormat: DynamoFormat[JsValue] = new DynamoFormat[JsValue] {
override def read(av: AttributeValue): Either[DynamoReadError, JsValue] = {
Option(av.getS).map {
fromStringAttributeValue
} orElse Option(av.getN).map { n =>
Right(JsNumber(BigDecimal.apply(n)))
} orElse Option(av.getBOOL).map { b =>
Right(JsBoolean(b))
} orElse Option(av.isNULL).map { _ =>
Right(JsNull)
} orElse Option(av.getSS).map { ss =>
Right(JsArray(ss.asScala.map(JsString.apply)))
} orElse Option(av.getNS).map { ns =>
Right(JsArray(ns.asScala.map(n => JsNumber(BigDecimal(n)))))
} orElse Option(av.getL).map { l =>
traverse(l.asScala.toList)(read).right.map(JsArray.apply)
} orElse Option(av.getM).map { m =>
traverse(m.asScala) {
case (k, v) => read(v).right.map(j => k -> j)
}.right.map(values => JsObject(values.toMap))
} getOrElse {
Left(YOUR_ERROR_HERE)
}
}
override def write(t: JsValue): AttributeValue = {
val res = new AttributeValue()
t match {
case JsNumber(n) => res.setN(n.toString())
case JsBoolean(b) => res.setBOOL(b)
case JsString(s) => res.setS(stringToAttributeValueString(s))
case a: JsArray => res.setL(a.value.map(write).asJava)
case o: JsObject => res.setM(o.value.mapValues(write).asJava)
case JsNull => res.setNULL(true)
}
res
}
}
Related
I'm trying to decode a JSON array that has the type Js.Json.t (not array(Js.Json.t) apparently). A call to Js.log(jsonList) reveals that it is an array, but I'm not certain how to map over the elements in the array to decode it.
So far, I've got:
let json_to_list = response => {
switch (response |> Js.Json.decodeObject) {
| None => {
Js.log("Decoding JSON failed!!")
None
}
| Some(jsonObject) => {
switch (jsonObject -> Js.Dict.get("list")) {
| None => {
Js.log("JSON didn't have a 'list' key/value.")
None
}
| Some(jsonList) => {
jsonList
|> Js.List.map(
/* compiler is expecting an uncurried function */
record => {
switch (record->Js.Dict.get("session-id") { /* ... */ }
}
)
}
}
}
}
};
The compiler is expecting an uncurried function, which, I don't know how to provide.
EDIT
I'd like to think I'm closer, but I'm getting an This has type: array(unit) Somewhere wanted: unit on the line (below) value |> Array.map(Js.log)
let json_to_list = response => {
Js.log("Decoding JSON")
switch (response |> Js.Json.decodeObject) {
| None => {
Js.log("Decoding JSON failed!!")
None
}
| Some(jsonObject) => {
switch (jsonObject -> Js.Dict.get("list")) {
| None => {
Js.log("JSON didn't have a 'list' key/value.")
None
}
| Some(jsonArray) => switch (Js.Json.decodeArray(jsonArray)) {
| None => {
Js.log("JSON Object wasn't an array.")
None
}
| Some(value) => {
Js.log("Value length: " ++ string_of_int(value|>Js.Array.length))
value |> Array.map(Js.log)
Some(value)
}
}
}
}
}
};
There are several ways of doing this, depending on what you know about the data at compile-time and what you actually need.
If you know exactly what it is, and there's no chance you could receive anything else, you could just cast it into the type you want without doing any checks at runtime:
external toMyType: Js.Json.t => array(something) = "%identity"
let myData = toMyType(json)
If you don't know the shape of the data until runtime, you can use Js.Json.classify:
let decodeArrayItem = ...
let myData : array(something) =
switch Js.Json.classify(json) {
| Js.Json.JSONArray(array) => Array.map(decodeArrayItem, array)
| _ => []
}
}
Or, if you could get anything but arrays are all you care about, you can use Js.Json.,decodeArray as a short-hand, which returns an option you can deal with further:
let decodeArrayItem = ...
let maybeData : option(array(something)) =
Js.Json.decodeArray(json)
|> Option.map(Array.map(decodeArrayItem))
Lastly, my recommended option for most scenarios is to use one of the third-party JSON decoder libraries, which tends to be designed for composition and therefore much more convenient for decoding large data structures. For example, using #glennsll/bs-json (no bias here, obviously):
module Decode = {
let arrayItem = ...
let myData =
Json.Decode.array(arrayItem)
}
let myData =
try Decode.myData(json) catch {
| Js.Json.DecodeError(_) => []
}
Edit: As for the actual error you get, you can turn a curried anonymus function into an uncurried one just by using a slightly different syntax:
let curried = record => ...
let uncurried = (. record) => ...
#scala.throws[scala.Exception]
def processQuery(searchQuery : scala.Predef.String) : scala.concurrent.Future[io.circe.Json] = { /* compiled code */ }
How do I declare the searchResult variable at line 3 so that it can be initailized inside the try block and can be processed if it's successful after and outside the try block. Or, is there any other way to handle the exception? The file containing processQuery function is not editable to me, it's read-only.
def index = Action.async{ implicit request =>
val query = request.body.asText.get
var searchResult : scala.concurrent.Future[io.circe.Json] = Future[io.circe.Json] //line 3
var jsonVal = ""
try {
searchResult = search.processQuery(query)
} catch {
case e :Throwable => jsonVal = e.getMessage
}
searchResult onSuccess ({
case result => jsonVal = result.toString()
})
searchResult.map{ result =>
Ok(Json.parse(jsonVal))
}
}
if declared in the way it has been declared it's showing compilation error
Would using the recover method help you? I also suggest to avoid var and use a more functional approach if possible. In my world (and play Json library), I would hope to get to something like:
def index = Action.async { implicit request =>
processQuery(request.body.asText.get).map { json =>
Ok(Json.obj("success" -> true, "result" -> json))
}.recover {
case e: Throwable => Ok(Json.obj("success" -> false, "message" -> e.getMessage))
}
}
I guess it may be necessary to put the code in another whole try catch:
try {
processQuery....
...
} catch {
...
}
I have here a way to validate on the incoming JSON and fold on the result of the validation:
def returnToNormalPowerPlant(id: Int) = Action.async(parse.tolerantJson) { request =>
request.body.validate[ReturnToNormalCommand].fold(
errors => {
Future.successful{
BadRequest(
Json.obj("status" -> "error", "message" -> JsError.toJson(errors))
)
}
},
returnToNormalCommand => {
actorFor(id) flatMap {
case None =>
Future.successful {
NotFound(s"HTTP 404 :: PowerPlant with ID $id not found")
}
case Some(actorRef) =>
sendCommand(actorRef, id, returnToNormalCommand)
}
}
)
}
I have a sealed trait that is like below:
sealed trait MyMessages
object MyMessages {
case object Init extends MyMessages
case object Destroy extends MyMessages
case class Tick(elem: Long) extends MyMessages
}
I have to now write a formatter for serializing and de-serializing this into to and from a JSON. This is what I came up with:
implicit object MyMessagesWrites extends Writes[MyMessages] {
def writes(myMessages: MyMessages): JsValue = myMessages match {
case Init => Json.toJson("INIT")
case Destroy => Json.toJson("DESTROY")
case tick: Tick => Json.toJson(Tick)
}
def reads(json: JsValue): MyMessages = {
// How do I get from JSValue to a MyMessages type???
}
}
Implementing the writes was easy, but how do I implement the reads?
Assuming you serialize the Tick instance as a bare JSON number, I would do it like so:
implicit object MyMessageReads extends Reads[MyMessages] {
def reads(json: JsValue) = json match {
case JsString("INIT") => JsSuccess(MyMessages.Init)
case JsString("DESTROY") => JsSuccess(MyMessages.Destroy)
case JsNumber(n) => JsSuccess(Tick(n.toLongExact))
case e => JsError(s"Invalid message: $e")
}
}
Note that you can also make the reads/writes a bit more succinct by using the more functional style:
implicit val myMessagesWrites = Writes[MyMessages] {
case Init => JsString("INIT")
case Destroy => JsString("DESTROY")
case Tick(n) => JsNumber(n)
}
implicit val myMessageReads = Reads[MyMessages] {
case JsString("INIT") => JsSuccess(MyMessages.Init)
case JsString("DESTROY") => JsSuccess(MyMessages.Destroy)
case JsNumber(n) => JsSuccess(Tick(n.toLongExact))
case e => JsError(s"Invalid message: $e")
}
I have a couple of Boolean attributes in my API model and would like to accept true/false as well as 1/0 values. My first idea was to implement custom formatter:
object UserJsonProtocol extends DefaultJsonProtocol {
implicit object MyBooleanJsonFormat extends JsonFormat[Boolean] {
def write(value: Boolean): JsString = {
return JsString(value.toString)
}
def read(value: JsValue) = {
value match {
case JsString("1") => true
case JsString("0") => false
case JsString("true") => true
case JsString("false") => false
case _ => throw new DeserializationException("Not a boolean")
}
}
}
implicit val userFormat = jsonFormat15(User.apply)
}
where User is a model with Boolean attributes. Unfortunately above solution has no effect - 1/0 are not accepted as Booleans. Any solution?
After fixing some issues with types and pattern matching it seems to work:
implicit object MyBooleanJsonFormat extends JsonFormat[Boolean] {
def write(value: Boolean): JsBoolean = {
return JsBoolean(value)
}
def read(value: JsValue) = {
value match {
case JsNumber(n) if n == 1 => true
case JsNumber(n) if n == 0 => false
case JsBoolean(true) => true
case JsBoolean(false) => false
case _ => throw new DeserializationException("Not a boolean")
}
}
}
Consider an http service that can return two json as response:
successful
{
"yourField":"value"
}
failure
{
"errorCode": 3
}
To deal with these json's I need to create 2 case classes case class RespSucc(yourField:String) and
case class RespFail(errorCode:Int).
For now I have to to something like that:
//unmarshal is spray.httpx.ResponseTransformation#unmarshal
if (response.entity.asString.contains("errorCode")) {
unmarshal[RespSucc].apply(response)
}
else {
unmarshal[RespFail].apply(response)
}
Is there an api to parse these classes automatically without any if? E.g. can unmarshaller looks into json fields and select approriate case class?
spray-json supports Either which is a very useful data type for this kind of situations.
val data = unmarshal[Either[RespFail, RespSucc]].apply(response)
// You can match it
data match {
case Left(err) => handleError(err)
case Right(suc) => handleSuccess(suc)
}
// Or you can fold it (I prefer folding)
data.fold(err => handleError(err), suc => handleSuccess(suc))
You can try something like this:
trait Resp
case class RespSucc(yourField: String) extends Resp
case class RespFail(errorCode: Int) extends Resp
object MyJsonProtocol extends DefaultJsonProtocol {
implicit object ColorJsonFormat extends RootJsonFormat[Resp] {
def write(r: Resp) = r match {
case s: RespSucc =>
JsObject("yourField" -> JsString(s.yourField))
case f: RespFail =>
JsObject("errorCode" -> JsNumber(f.errorCode))
}
def read(value: JsValue) = value.asJsObject.getFields("yourField", "errorCode") match {
case Seq(JsString(yourField)) => RespSucc(yourField)
case Seq(JsNumber(errorCode)) => RespFail(errorCode.intValue())
case _ => deserializationError("Resp expected")
}
}
}
import MyJsonProtocol._
unmarshal[Resp](entitySucc) //Right(RespSucc(abc))
unmarshal[Resp](entityFail) //Right(RespFail(3))