play framework 2 with scala: pretify json - json

I'm using the play framework 2 native support for json (http://www.playframework.org/documentation/latest/ScalaJson) and I have a JsValue that I'm converting to string to save it to a text file, like this
val json: JsValue = [....]
Json.stringify(json)
Which works fine, but generates something like this:
{"tokens":[{"id":"1000","token":"DON...
I was wondering if there's an easy way to generate a formatted json like this
{
"tokens":
[
{
"id":"1000",
"token":"DON...

I used liftweb JSON package with its Printer object/trait. Works 'pretty' well:
import play.api._
import play.api.mvc._
import play.api.libs._, concurrent._, json._
import com.mongodb.casbah.Imports.{MongoConnection, MongoCursor, WriteConcern}
import com.mongodb.casbah.query.Imports._
import com.novus.salat.json._
import net.liftweb.json.{render => jsonRender, _}
class Application extends Controller {
def getJson(id: String) = Action { implicit request =>
val objPromise = Akka.future(Database.getById(id))
Async {
objPromise.orTimeout("Error", 1000).map { o =>
o.fold(
hit => Ok(hit.map{ o: DBObject => pretty(jsonRender(ToJValue(o)))}.getOrElse("")).as("text/json"),
timeout => InternalServerError(timeout)
)
}
}
}

Related

How to consume HttpResponse in Akka Http

import akka.actor.ActorSystem
import akka.http.scaladsl.Http
import akka.http.scaladsl.model.{HttpMethods, HttpRequest, Uri}
import akka.stream.scaladsl.{Flow, Source}
import akka.stream.{ActorMaterializer, OverflowStrategy}
import spray.json._
import java.util.UUID
import scala.concurrent.duration.DurationInt
import scala.language.postfixOps
import scala.util.{Failure, Success}
object SoftwareRegistry extends App with Formatter {
implicit val system = ActorSystem("NPMRegistry")
implicit val materializer = ActorMaterializer()
case class NPMPackage(name: String)
// reading the packages
val filename = "B:\\Scala\\NPMRegistry\\src\\main\\resources\\packages.txt"
val bufferedSource = scala.io.Source.fromFile(filename)
val listOfPackages: List[NPMPackage] = (for (line <- bufferedSource.getLines) yield {
NPMPackage(line.trim)
}).toList
bufferedSource.close()
// requests
val serverHttpRequests = listOfPackages.map(pkg =>
(HttpRequest(
HttpMethods.GET,
uri = Uri(s"/${pkg.name}")
),
UUID.randomUUID().toString)
)
// source
val sourceList = Source(serverHttpRequests)
val bufferedFlow = Flow[(HttpRequest, String)]
.buffer(10, overflowStrategy = OverflowStrategy.backpressure)
.throttle(1, 3 seconds)
val dd = sourceList
.via(bufferedFlow).async
.via(Http().cachedHostConnectionPoolHttps[String]("registry.npmjs.org"))
.runForeach {
case (Success(response), oId) =>
println(s"$oId $response")
case (Failure(ex), oId) => println(ex)
}
In the above code, I can print the response to the console and I want to know how to consume entity and access the data from the response in a streamed way, not in a future.
Following is the result of the existing code
You basically need to keep you logic within Akka Streams API and not terminating it with runForEach like you did.
A simplified example of this can look like this:
.via(Http().cachedHostConnectionPoolHttps[String]("registry.npmjs.org"))
.flatMapConcat {
case (Success(response), _) => Source.single(response)
case (Failure(_), _) => Source.empty //warning, ignoring errors
}
.map(httpResponse => httpResponse.entity)
.flatMapConcat(e => e.getDataBytes().map(bytes => ???))
.runWith(Sink.ignore)
Instead of runforEach I am flatMapConcatenating to get the HttpRespnse ignoring errors and the context of the request for simplicity of the example. Then mapping to get HttpEntity and then flatMapConcatenating again to get the ByteStrings representing the response body. There could be multiple of those coming form every HttpRequest and that's, what I am guessing you're referring to by "streamed way".

Scala: How to fix type mismatch error trying to decode JSON with Circe Decoder

I need some help with the following situation. I have a JSON API with a registration request. The request takes 4 parameters: Standard phone, password, email, and a person parameter where a customer chooses a type. Based on the chosen type, they have to specify one more parameter: Name or age.
When I get an HTTP request, I need to decode it into my request object. Doing it with the code below results in a compilation error of type:
type mismatch;
found : Unit
required: io.circe.Decoder[RegistrationReq]
Any ideas why that can be the case and how to fix it? Thanks in advance!
import cats.effect.Sync
import cats.syntax.either._
import circe.jsonOf
import io.circe.{Decoder, DecodingFailure, HCursor}
import io.circe.generic.semiauto._
import io.circe.refined._
import io.circe.Decoder.Result
import org.http4s.EntityDecoder
trait RegJsonCodec {
import JsonCodec._ //all common encoders and decoders come from there
implicit val RegistrationReqDecoder: Decoder[RegistrationReq] = {
def apply(c: HCursor): Result[RegistrationReq] =
for {
phone <- getPhone(c)
password <- c.downField("password").as[Password]
email <- c.downField("email").as[Email] //Password and Email are refined Strings
person <- getPerson(c)
} yield RegistrationReq(phone, password, email, person)
}
private def getPhone(c: HCursor): Either[DecodingFailure, Long] =
c.downField("phone").as[Long].orElse {
c.downField("phone").as[String].flatMap {
str =>
Either.catchNonFatal(str.toLong)
.leftMap(e => DecodingFailure(e.getMessage, c.history))
}
}.flatMap(phone => Either.right(phone))
private def getPerson(c: HCursor): Either[DecodingFailure, Person] = {
c.downField("person").downField("type").as[String].flatMap { type =>
c.downField("person").downField("name").as[String]
.orElse(c.downField("person").downField("age").as[String]).flatMap { extra =>
Either.catchNonFatal(Person.fromPair(type, extra))
.leftMap(e => DecodingFailure(e.getMessage, c.history))
.flatMap(person => Either.right(person))
}
}
}
implicit def requestEntityDecoder[F[_] : Sync]: EntityDecoder[F, RegistrationReq] = jsonOf[F, RegistrationReq]
}
object RegJsonCodec extends RegJsonCodec

How do I write a generic JSON parser in Play 2.7 for Scala that validates inbound requests?

I have a Play 2.7 controller in Scala that validates inbound JSON requests against a case class schema, and reports inbound request payload errors (note that I extracted this sample from a larger codebase, attempting to preserve its correct compilability and functionality, though there may be minor mistakes):
import scala.concurrent.{ExecutionContext, Future}
import scala.util.{Failure, Success, Try}
import com.google.inject.Inject
import play.api.libs.json.{JsError, JsPath, JsSuccess, JsValue, Json, JsonValidationError}
import play.api.mvc.{AbstractController, Action, ControllerComponents, Request, Result}
class Controller #Inject() (playEC: ExecutionContext, cc: ControllerComponents) extends AbstractController(cc) {
case class RequestBody(id: String)
implicit val requestBodyFormat = Json.format[RequestBody]
private val tryJsonParser = parse.tolerantText.map(text => Try(Json.parse(text)))(playEC)
private def stringify(path: JsPath, errors: Seq[JsonValidationError]): String = {
s"$path: [${errors.map(x => x.messages.mkString(",") + (if (x.args.size > 0) (": " + x.args.mkString(",")) else "")).mkString(";")}]"
}
private def runWithRequest(rawRequest: Request[Try[JsValue]], method: (RequestBody) => Future[Result]): Future[Result] = {
rawRequest.body match {
case Success(validBody) =>
Json.fromJson[RequestBody](validBody) match {
case JsSuccess(r, _) => method(r)
case JsError(e) => Future.successful(BadRequest(Json.toJson(e.map(x => stringify(x._1, x._2)).head)))
}
case Failure(e) => Future.successful(BadRequest(Json.toJson(e.getMessage.replaceAll("\n", ""))))
}
}
// POST request processor
def handleRequest: Action[Try[JsValue]] = Action(tryJsonParser).async { request: Request[Try[JsValue]] =>
runWithRequest(request, r => {
Future.successful(Ok(r.id))
})
}
}
The validation works like this when sending a POST request to the "handleRequest" endpoint:
with the payload {malformed,,, I get a 400 response back with Unexpected character ('m' (code 109)): was expecting double-quote to start field name at [Source: (String)"{malformed,,"; line: 1, column: 3].
with the payload {} I get a 400 response back with /id: [error.path.missing]
What I would like to do is to make the parsing and validating generic,
moving that logic into a utility class for the cleanest re-use possible in the handleRequest method. For example, something like this:
import scala.concurrent.{ExecutionContext, Future}
import scala.util.{Failure, Success, Try}
import com.google.inject.{Inject, Singleton}
import play.api.{Configuration, Logging}
import play.api.libs.json.{JsError, JsPath, JsSuccess, JsValue, Json, JsonValidationError}
import play.api.mvc.{AbstractController, Action, ActionBuilderImpl, AnyContent, BodyParsers, ControllerComponents, Request, Result}
object ParseAction {
// TODO: how do I work this in?
val tryJsonParser = parse.tolerantText.map(text => Try(Json.parse(text)))(playEC)
}
class ParseAction #Inject()(parser: BodyParsers.Default)(implicit ec: ExecutionContext) extends ActionBuilderImpl(parser) {
private def stringify(path: JsPath, errors: Seq[JsonValidationError]): String = {
s"$path: [${errors.map(x => x.messages.mkString(",") + (if (x.args.size > 0) (": " + x.args.mkString(",")) else "")).mkString(";")}]"
}
// TODO: how do I make this method generic?
override def invokeBlock[A](rawRequest: Request[A], block: (A) => Future[Result]) = {
rawRequest.body match {
case Success(validBody) =>
Json.fromJson[A](validBody) match {
case JsSuccess(r, _) => block(r).getFuture
case JsError(e) => Future.successful(BadRequest(Json.toJson(e.map(x => stringify(x._1, x._2)).head)))
}
case Failure(e) => Future.successful(BadRequest(Json.toJson(e.getMessage.replaceAll("\n", ""))))
}
}
}
class Controller #Inject() (cc: ControllerComponents) extends AbstractController(cc) {
case class RequestBody(id: String)
implicit val requestBodyFormat = Json.format[RequestBody]
// route processor
def handleRequest = ParseAction.async { request: RequestBody =>
Future.successful(Ok(r.id))
}
}
I'm aware that this code doesn't compile as-is due to blatant Scala and Play API misuse rather than just small coding mistakes. I tried pulling from Play's own documentation about Action composition, but I have not had success in getting things right, so I left all of the pieces around hoping someone can help me to work them together into something that works.
How can I change this second code sample around to compile and behave functionally identically to the first code sample?
I archived similar goal by using implicit class for ActionBuilder:
trait ActionBuilderImplicits {
implicit class ExActionBuilder[P](actionBuilder: ActionBuilder[Request, P])(implicit cc: ControllerComponents) {
def validateJson[A](implicit executionContext: ExecutionContext, reads: Reads[A]): ActionBuilder[Request, A] = {
actionBuilder(cc.parsers.tolerantJson.validate(jsValue => {
jsValue.validate.asEither.left
.map(errors => BadRequest(JsError.toJson(errors)))
}))
}
}
}
object ActionBuilderImplicits extends ActionBuilderImplicits
Then in controller you can import ActionBuilderImplicits and use it as
Action.validateJson[A].async { request =>
processingService.process(request.body)
}
Here is request.body already type of A

Using PlayJson in akka-http client

I want to write a polling client in akka-http that converts all response bodies into a Play JsObject. What I have so far is to code below that uses this library wich should make things simple (i think?). However when I try to run the code below I get the following error:
Error:(26, 56) type mismatch;
found : akka.http.scaladsl.unmarshalling.FromEntityUnmarshaller[play.api.libs.json.JsObject]
(which expands to) akka.http.scaladsl.unmarshalling.Unmarshaller[akka.http.scaladsl.model.HttpEntity,play.api.libs.json.JsObject]
required: akka.http.scaladsl.unmarshalling.Unmarshaller[akka.http.scaladsl.model.HttpResponse,play.api.libs.json.JsObject]
Unmarshaller.byteStringUnmarshaller.mapWithCharset { (data, charset) =>
What do I need to change to get thing working as expected?
import java.util.UUID
import akka.actor.ActorSystem
import akka.http.scaladsl.Http
import akka.http.scaladsl.model.{HttpEntity, HttpRequest, HttpResponse}
import akka.stream.ActorMaterializer
import akka.stream.scaladsl.Source
import akka.http.scaladsl.unmarshalling.{Unmarshal, Unmarshaller}
import de.heikoseeberger.akkahttpplayjson.PlayJsonSupport._
import play.api.libs.json.{JsObject, Json}
import scala.concurrent.duration._
import scala.util.{Success, Try}
object Main extends App {
implicit val system = ActorSystem("TestSys")
implicit val ec = system.dispatcher
implicit val materializer = ActorMaterializer()
implicit val um:Unmarshaller[HttpResponse, JsObject] = {
Unmarshaller.byteStringUnmarshaller.mapWithCharset { (data, charset) =>
Json.parse(data.toArray).as[JsObject]
}
}
val request = HttpRequest(uri="https://www.google.com/finance/info?q=INDEXDB%3ADAX") -> UUID.randomUUID()
val pool = Http().superPool[UUID]()
val googleFinanceFlow =
Source.tick(0 milliseconds,10000 milliseconds,request)
.via(pool)
.runForeach {
case (Success(response),_) =>
val json = Unmarshal(response).to[JsObject]
println(json.onSuccess{case r => println(r.toString())})
case _ => Json.obj()
}
}
Just delete the explicit implicit (wow, that sounds nice, eh?) definition of the Unmarshaller[HttpResponse, JsObject]. That's not needed, because a suitable unmarshaller is provided by PlayJsonSupport.

Need help, accessing more levels of a json/twitter search

I'm using example from #diegolparra of doing a twitter search and/or a streaming.
package controllers
import play.api.mvc.{WebSocket, Action, Controller}
import play.api.libs.functional.syntax._
import play.api.libs.json._
import play.api.libs.concurrent.Execution.Implicits.defaultContext
import play.api.libs.ws.WS
import play.api.libs.iteratee.{Iteratee, Concurrent, Enumerator}
import play.api.libs.oauth.{OAuthCalculator, RequestToken, ConsumerKey}
case class Tweet(from: String, text: String)
object Tweets extends Controller {
implicit val tweetReads = (
(__ \ "from_user_name").read[String] and
(__ \ "text").read[String]
)(Tweet)
def tweetList(query: String) = Action {
Async {
val results = 50
val responsePromise =
WS.url("http://search.twitter.com/search.json")
.withQueryString("q" -> query, "rpp" -> results.toString).get
responsePromise.map {
response =>
val tweets = Json.parse(response.body).\("results").as[Seq[Tweet]]
Ok(views.html.tweetlist(tweets))
}
}
}
val consumerKey = ConsumerKey("EBcP4MM9VnI64L8RZLO7g","SVi7XyyNpWzidR2Zx2HVNZ7kZTwFGxpqGKqhOeA")
val accessToken = RequestToken("1228081488-sLSztNAm0ST2kssCkBNRyhSsmk8SP5dtcbX1ZE2", "GxHUymXdyTYRZxw4bbTbgN8Xh53jKxC1KwvgsVwUU")
def stream(keywords: String) = WebSocket.using[String] { request =>
val out: Enumerator[String] = Concurrent.unicast[String](onStart = pushee => {
def twitterIteratee = Iteratee.foreach[Array[Byte]]{ ba =>
val msg = new String(ba, "UTF-8")
pushee.push(msg)
println(msg)
}
WS.url("https://stream.twitter.com/1.1/statuses/filter.json?track=" + keywords)
.sign(OAuthCalculator(consumerKey, accessToken))
.get(headers=> twitterIteratee)
})
val in = Iteratee.ignore[String]
(in, out)
}
}
This example get from result of twitter search the fields, from_user_name and text. Now i need to access created_at date and one more level (node user) to get from user: profile_image_url and lang.
Please, any help?
Thanks.
At Devoxx France, Guillaume Bort showed an example of tweets-fetching getting some "deep" informations. You can find the code here: https://github.com/adericbourg/devoxxfr2013/blob/master/app/controllers/Application.scala
It does not show the properties you need but using that sample and the Twitter API documentation, you may manage to get what you need.
Hope this helps, so.