Okay so.... I have this:
def convertPost = extract {
_.request.entity.asString.parseJson.convertTo[CustomClass]
}
private def myRoute: Route =
(post & terminalPath("routeness")) {
convertPost { req =>
detach() {
ThingHandler.getMyResults( req )
}
}
}
but I want to template it, like this:
def convertPost[T] = extract {
_.request.entity.asString.parseJson.convertTo[T]
}
private def myRoute: Route =
(post & terminalPath("routeness")) {
convertPost[CustomClass] { req =>
detach() {
ThingHandler.getMyResults( req )
}
}
}
But that doesn't work. I am using spray-json-shapeless. My error is
Error:(28, 50) Cannot find JsonReader or JsonFormat type class for T
_.request.entity.asString.parseJson.convertTo[T]
^
when I try:
def getStuff[T] = extract {
_.request.entity.asInstanceOf[T] // .convertTo[T]
}
it gives:
spray.http.HttpEntity$NonEmpty cannot be cast to com.stuff.CustomClass
Related
I'm having trouble using a custom JSON marshaller/unmarshaller. This much works fine:
trait EWorksJsonSupport extends SprayJsonSupport with DefaultJsonProtocol {
implicit object IndividualJsonFormat extends RootJsonFormat[Individual] {
def write(individual: Individual) = JsObject(
// blah blah blah
)
def read(value: JsValue): Individual = {
// blah blah blah
}
}
The problem is that Unsupported Content-Type, supported: application/json is returned as shown below:
import akka.http.scaladsl.model.ContentTypes._
import akka.http.scaladsl.model.HttpEntity
import akka.http.scaladsl.testkit.ScalatestRouteTest
import akka.http.scaladsl.unmarshalling._
import eworks.model.immutableModel.SpeciesAll
import eworks.model.mutableModel.{Individual, Individuals, VirtualWorld}
import eworks.model.{Fixtures, LoadableModel, SpeciesDefaultLike}
import org.junit.runner.RunWith
import org.scalatest.Matchers._
import org.scalatest._
import org.scalatest.junit.JUnitRunner
import spray.json._
#RunWith(classOf[JUnitRunner])
class TestRest extends WordSpec with SpeciesDefaultLike with LoadableModel with ScalatestRouteTest with Fixtures with EWorksJsonSupport {
"EWorksJsonSupport" should {
"work for Individuals" in {
val jsObject: JsValue = harry.toJson
val entity = HttpEntity(`application/json`, jsObject.toString)
Post("/addIndividual", entity) ~> new RestHttp()(speciesDefaults).route ~> check {
handled === true
contentType === `application/json`
status.intValue === 200
val individual1 = Unmarshal(response.entity).to[Individual]
// ErrorFuture(akka.http.scaladsl.unmarshalling.Unmarshaller$UnsupportedContentTypeException: Unsupported Content-Type, supported: application/json)
val individual2 = responseAs[Individual]
responseAs[Individual] shouldBe harry
}
}
}
}
If you can't change the content type you could:
val stringR : String = Await.result(Unmarshal(r).to[String],Duration.Inf)
val ind : Individual = Unmarshal(stringR).to[Individual]
The HttpResponse response you get from the new RestHttp()(speciesDefaults).route router by posting your entity to /addIndividual (as logged, see below) has text/plain as content-type, you should fix that. Also its content does not look like valid JSON (see below).
Response was:
HttpResponse(
200 OK,
List(),
HttpEntity.Strict(
text/plain; charset=UTF-8,
Individual added: harry is a human; (unborn); lifeStage 'adult'
), HttpProtocol(HTTP/1.1)
)
The key to the solution is to call complete with the desired ContentType. Here is a method I wrote that provides an HttpResponse with Content-Type application/json along with the desired content, computed when block is evaluated:
#inline def wrap(block: => JsValue): StandardRoute =
complete(
try {
HttpResponse(entity = HttpEntity(ContentTypes.`application/json`, success(block)))
} catch {
case e: Exception =>
HttpResponse(entity = HttpEntity(ContentTypes.`application/json`, error(e.getMessage)))
}
)
I made a trait to encapsulate this handy utility method:
import akka.http.scaladsl.model.{ContentTypes, HttpEntity, HttpHeader, HttpResponse}
import akka.http.scaladsl.server.{Directives, MediaTypeNegotiator, Route, StandardRoute, UnsupportedRequestContentTypeRejection}
import akka.http.scaladsl.unmarshalling._
import spray.json._
import scala.collection.immutable.Seq
trait RestHttpSupport extends Directives {
#inline def error (msg: String): String = JsObject("error" -> JsString(msg)).prettyPrint
#inline def success(msg: String): String = JsObject("success" -> JsString(msg)).prettyPrint
#inline def error (msg: JsValue): String = JsObject("error" -> msg).prettyPrint
#inline def success(msg: JsValue): String = JsObject("success" -> msg).prettyPrint
#inline def wrap(block: => JsValue): StandardRoute =
complete(
try {
HttpResponse(entity = HttpEntity(ContentTypes.`application/json`, success(block)))
} catch {
case e: Exception =>
HttpResponse(entity = HttpEntity(ContentTypes.`application/json`, error(e.getMessage)))
}
)
#inline def completeAsJson[T](requestHeaders: Seq[HttpHeader])
(body: T => StandardRoute)
(implicit um: FromRequestUnmarshaller[T]): Route = {
import akka.http.scaladsl.model.MediaTypes.`application/json`
if (new MediaTypeNegotiator(requestHeaders).isAccepted(`application/json`)) {
entity(as[T]) { body }
} else {
reject(UnsupportedRequestContentTypeRejection(Set(`application/json`)))
}
}
#inline def postAsJson[T](body: T => StandardRoute)
(implicit um: FromRequestUnmarshaller[T]): Route = {
(post & extract(_.request.headers)) { requestHeaders =>
completeAsJson[T](requestHeaders) { body }
}
}
}
One the trait is mixed in, and assuming that implicit serializers built from SprayJsonSupport with DefaultJsonProtocol are in scope, an Akka HTTP path can be defined using the wrap method. All of this code is taken from EmpathyWorks™ (which is not open source):
path("definedEvents") {
get { wrap(allDefinedEvents.toJson) }
} ~
path("listIndividuals") {
get { wrap(individuals.toJson) }
} ~
path("listSpecies") {
get { wrap(speciesAll.toJson) }
} ~
path("listSpeciesNames") {
get { wrap(speciesAll.collection.map(_.name).toJson) }
}
I'm new to the play framework generally and how to use it with Scala. I want to build a proxy for big Json objects. I achieved so far that the json is stored in a cache and if it is not there, requested from a webservice.
However when two requests are coming in, targeting the same end point (webservice and path are identicall) only one call should be performed and the other request should wait for the result of the first call. At the moment it is performing a call to the service with every request.
This is my controller:
#Singleton
class CmsProxyController #Inject()(val cmsService: CmsProxyService) extends Controller {
implicit def ec : ExecutionContext = play.api.libs.concurrent.Execution.defaultContext
def header(path: String) = Action.async { context =>
cmsService.head(path) map { title =>
Ok(Json.obj("title" -> title))
}
}
def teaser(path: String) = Action.async { context =>
cmsService.teaser(path) map { res =>
Ok(res).as(ContentTypes.JSON)
}
}
}
This is the service:
trait CmsProxyService {
def head(path: String): Future[String]
def teaser(path: String): Future[String]
}
#Singleton
class DefaultCmsProxyService #Inject()(cache: CacheApi, cmsCaller: CmsCaller) extends CmsProxyService {
private val BASE = "http://foo.com"
private val CMS = "bar/rest/"
private val log = Logger("application")
override def head(path: String) = {
query(url(path), "$.payload[0].title")
}
override def teaser(path: String) = {
query(url(path), "$.payload[0].content.teaserText")
}
private def url(path: String) = s"${BASE}/${CMS}/${path}"
private def query(url: String, jsonPath: String): Future[String] = {
val key = s"${url}?${jsonPath}"
val payload = findInCache(key)
if (payload.isDefined) {
log.debug("found payload in cache")
Future.successful(payload.get)
} else {
val queried = parse(fetch(url)) map { json =>
JSONPath.query(jsonPath, json).as[String]
}
queried.onComplete(value => saveInCache(key, value.get))
queried
}
}
private def parse(fetched: Future[String]): Future[JsValue] = {
fetched map { jsonString =>
Json.parse(jsonString)
}
}
//retrieve the requested value from the cache or from ws
private def fetch(url: String): Future[String] = {
val body = findInCache(url)
if (body.isDefined) {
log.debug("found body in cache")
Future.successful(body.get)
} else {
cmsCaller.call(url)
}
}
private def findInCache(key: String): Option[String] = cache.get(key)
private def saveInCache(key: String, value: String, duration: FiniteDuration = 5.minutes) = cache.set(key, value, 5.minutes)
}
And finally the call to the webservice:
trait CmsCaller {
def call(url: String): Future[String]
}
#Singleton
class DefaultCmsCaller #Inject()(wsClient: WSClient) extends CmsCaller {
import scala.concurrent.ExecutionContext.Implicits.global
//keep those futures which are currently requested
private val calls: Map[String, Future[String]] = TrieMap()
private val log = Logger("application")
override def call(url: String): Future[String] = {
if(calls.contains(url)) {
Future.successful("ok")
}else {
val f = doCall(url)
calls put(url, f)
f
}
}
//do the final call
private def doCall(url: String): Future[String] = {
val request = ws(url)
val response = request.get()
val mapped = mapResponse(response)
mapped.onComplete(_ => cmsCalls.remove(url))
mapped
}
private def ws(url: String): WSRequest = wsClient.url(url)
//currently executed with every request
private def mapResponse(f: Future[WSResponse]): Future[String] = {
f.onComplete(_ => log.debug("call completed"))
f map {res =>
val status = res.status
log.debug(s"ws called, response status: ${status}")
if (status == 200) {
res.body
} else {
""
}
}
}
}
My question is: How can only one call to the webservice beeing executed? Even if there are several requests to the same target. I don't want to block it, the other request (not sure if I use the right word here) shall just be informed that there is already a webservice call on the way.
The request to head and teaser, see controller, shall perform only one call to the webservice.
Simple answer using Scala lazy keyword
def requestPayload(): String = ??? //do something
#Singleton
class SimpleCache #Inject() () {
lazy val result: Future[String] = requestPayload()
}
//Usage
#Singleton
class SomeController #Inject() (simpleCache: SimpleCache) {
def action = Action { req =>
simpleCache.result.map { result =>
Ok("success")
}
}
}
First request will trigger the rest call and all the other requests will use the cached result. Use map and flatMap to chain the requests.
Complicated answer using Actors
Use Actor to queue requests and Cache the result of the first successful request json result. All the other requests will read the result of the first request.
case class Request(value: String)
class RequestManager extends Actor {
var mayBeResult: Option[String] = None
var reqs = List.empty[(ActorRef, Request)]
def receive = {
case req: Request =>
context become firstReq
self ! req
}
def firstReq = {
case req: Request =>
process(req).onSuccess { value =>
mayBeResult = Some(value)
context become done
self ! "clear_pending_reqs"
}
context become processing
}
def processing = {
case req: Request =>
//queue requests
reqs = reqs ++ List(sender -> req)
}
def done = {
case "clear_pending_reqs" =>
reqs.foreach { case (sender, _) =>
//send value to the sender
sender ! value.
}
}
}
handle the case where the first request fails. In the above code block if the first request fails then actor will never go to the done state.
I solved my problem with a synchronization of the cache in the service. I'm not sure if this an elegant solution, but it works for me.
trait SyncCmsProxyService {
def head(path: String): String
def teaser(path: String): String
}
#Singleton
class DefaultSyncCmsProxyService #Inject()(implicit cache: CacheApi, wsClient: WSClient) extends SyncCmsProxyService with UrlBuilder with CacheAccessor{
private val log = Logger("application")
override def head(path: String) = {
log.debug("looking for head ...")
query(url(path), "$.payload[0].title")
}
override def teaser(path: String) = {
log.debug("looking for teaser ...")
query(url(path), "$.payload[0].content.teaserText")
}
private def query(url: String, jsonPath: String) = {
val key = s"${url}?${jsonPath}"
val payload = findInCache(key)
if (payload.isDefined) {
payload.get
}else{
val json = Json.parse(body(url))
val queried = JSONPath.query(jsonPath, json).as[String]
saveInCache(key, queried)
}
}
private def body(url: String) = {
cache.synchronized {
val body = findInCache(url)
if (body.isDefined) {
log.debug("found body in cache")
body.get
} else {
saveInCache(url, doCall(url))
}
}
}
private def doCall(url : String): String = {
import scala.concurrent.ExecutionContext.Implicits.global
log.debug("calling...")
val req = wsClient.url(url).get()
val f = req map { res =>
val status = res.status
log.debug(s"endpoint called! response status: ${status}")
if (status == 200) {
res.body
} else {
""
}
}
Await.result(f, 15.seconds)
}
}
Note that I omitted the traits UrlBuilder and CacheAccessor here because they are trivial.
In my project, I am trying to return selected elements as JSON from requested parameter.
Domain Class:
class Component{
String name
String level
.
.
.
}
I have http request like
http://localhost:8080/myapp/component/showJson?name=name
so I should return only
{
name:xyz
}
If my request is like
http://localhost:8080/myapp/component/showJson?name=name&level=level
Then I should return
{
name:xyz
level:1
}
Any suggestions is appreciated.
Updated JSON (Multilevel)
[
{"name":"one","level":0,"
componentTypes":[
{"name":"one one","level":1,
"componentTypes":[
{"name":"one one one","level":2,"componentTypes":[]}
]
},
{"name":"one two","level":1,"componentTypes":[]}
]
},
{"name":"two","level":0,"componentTypes"[]},
{"name":"three","level":0,"componentTypes":[]}
]
class ComponentType {
String name
Integer level
static hasMany = [componentTypes:ComponentType]
ComponentType parent
static constraints = {
parent nullable:true
}
static mapWith = "mongo"
}
Controller action
componentTypeList = ComponentType.createCriteria().list(){
eq("level", 0)
}
You can intersect the params map with object properties map and return the result. I haven't tried this, but i can't think of a reason why it will not work.
def properties = component.properties;
def result = properties.subMap(params.keySet())
render result as JSON
Update:
class ComponentType {
.
.
.
def toJSON(def params) {
def properties = this.properties
def result = properties.subMap(params.keySet())
if(this.componentTypes) {
result.componentTypes = componentTypes*.toJSON(params)
}
result
}
}
def componentTypeList = ComponentType.createCriteria().list(){ eq("level", 0) }
render componentTypeList*.toJSON(params) as JSON
I'm trying to write some integration tests for a RestfulController in Grails 2.4.0 responding in JSON format. The index()-Method is implemented like this:
class PersonController extends RestfulController<Person> {
...
def index(final Integer max) {
params.max = Math.min(max ?: 10, 100)
respond listAllResources(params), [includes: includeFields]
}
...
}
The integration test looks like this:
void testListAllPersons() {
def controller = new PersonController()
new Person(name: "Person", age: 22).save(flush:true)
new Person(name: "AnotherPerson", age: 31).save(flush:true)
controller.response.format = 'json'
controller.request.method = 'GET'
controller.index()
assertEquals '{{"name":"Person", "age": "22"},{"name":"AnotherPerson", "age": "31"}}', controller.response.json
}
What i don't understand is controller.response.json only contains the "AnotherPerson" instead of both entries.
When i start the server with run-app und test it with a Rest-Client i get both entries.
Any Ideas?
You haven't included enough information to say for sure what the problem is but the following test passes with 2.4.0.
The domain class:
// grails-app/domain/com/demo/Person.groovy
package com.demo
class Person {
String name
Integer age
}
The controller:
// grails-app/controllers/com/demo/PersonController.groovy
package com.demo
class PersonController extends grails.rest.RestfulController<Person> {
PersonController() {
super(Person)
}
def index(final Integer max) {
params.max = Math.min(max ?: 10, 100)
respond listAllResources(params)
}
}
The test:
// test/unit/com/demo/PersonControllerSpec.groovy
package com.demo
import grails.test.mixin.TestFor
import spock.lang.Specification
#TestFor(PersonController)
#Mock(Person)
class PersonControllerSpec extends Specification {
void "test index includes all people"() {
given:
new Person(name: "Person", age: 22).save(flush:true)
new Person(name: "AnotherPerson", age: 31).save(flush:true)
when:
request.method = 'GET'
response.format = 'json'
controller.index()
then:
response.status == 200
response.contentAsString == '[{"class":"com.demo.Person","id":1,"age":22,"name":"Person"},{"class":"com.demo.Person","id":2,"age":31,"name":"AnotherPerson"}]'
}
}
I simplified the example a little too much. I used a named object marshaller which i created (incorrect) in bootstrap.groovy like this:
JSON.createNamedConfig('simplePerson') { converterConfig ->
converterConfig.registerObjectMarshaller(Person) {
JSON.registerObjectMarshaller(Person) {
def map = [:]
map['name'] = it.name
map['age'] = it.age
return map
}
}
}
And used it in the controller:
...
JSON.use("simplePerson")
...
The problem is solved by creating the object marshaller like this:
JSON.createNamedConfig('simplePerson') { converterConfig ->
converterConfig.registerObjectMarshaller(Person) {
def map = [:]
map['name'] = it.name
map['age'] = it.age
return map
}
}
I have an iterator (actually a Source.getLines) that's reading an infinite stream of data from a URL. Occasionally the iterator throws a java.io.IOException when there is a connection problem. In such situations, I need to re-connect and re-start the iterator. I want this to be seamless so that the iterator just looks like a normal iterator to the consumer, but underneath is restarting itself as necessary.
For example, I'd like to see the following behavior:
scala> val iter = restartingIterator(() => new Iterator[Int]{
var i = -1
def hasNext = {
if (this.i < 3) {
true
} else {
throw new IOException
}
}
def next = {
this.i += 1
i
}
})
res0: ...
scala> iter.take(6).toList
res1: List[Int] = List(0, 1, 2, 3, 0, 1)
I have a partial solution to this problem, but it will fail on some corner cases (e.g. an IOException on the first item after a restart) and it's pretty ugly:
def restartingIterator[T](getIter: () => Iterator[T]) = new Iterator[T] {
var iter = getIter()
def hasNext = {
try {
iter.hasNext
} catch {
case e: IOException => {
this.iter = getIter()
iter.hasNext
}
}
}
def next = {
try {
iter.next
} catch {
case e: IOException => {
this.iter = getIter()
iter.next
}
}
}
}
I keep feeling like there's a better solution to this, maybe some combination of Iterator.continually and util.control.Exception or something like that, but I couldn't figure one out. Any ideas?
This is fairly close to your version and using scala.util.control.Exception:
def restartingIterator[T](getIter: () => Iterator[T]) = new Iterator[T] {
import util.control.Exception.allCatch
private[this] var i = getIter()
private[this] def replace() = i = getIter()
def hasNext: Boolean = allCatch.opt(i.hasNext).getOrElse{replace(); hasNext}
def next(): T = allCatch.opt(i.next).getOrElse{replace(); next}
}
For some reason this is not tail recursive but it that can be fixed by using a slightly more verbose version:
def restartingIterator2[T](getIter: () => Iterator[T]) = new Iterator[T] {
import util.control.Exception.allCatch
private[this] var i = getIter()
private[this] def replace() = i = getIter()
#annotation.tailrec def hasNext: Boolean = {
val v = allCatch.opt(i.hasNext)
if (v.isDefined) v.get else {replace(); hasNext}
}
#annotation.tailrec def next(): T = {
val v = allCatch.opt(i.next)
if (v.isDefined) v.get else {replace(); next}
}
}
Edit: There is a solution with util.control.Exception and Iterator.continually:
def restartingIterator[T](getIter: () => Iterator[T]) = {
import util.control.Exception.allCatch
var iter = getIter()
def f: T = allCatch.opt(iter.next).getOrElse{iter = getIter(); f}
Iterator.continually { f }
}
There is a better solution, the Iteratee:
http://apocalisp.wordpress.com/2010/10/17/scalaz-tutorial-enumeration-based-io-with-iteratees/
Here is for example an enumerator that restarts on encountering an exception.
def enumReader[A](r: => BufferedReader, it: IterV[String, A]): IO[IterV[String, A]] = {
val tmpReader = r
def loop: IterV[String, A] => IO[IterV[String, A]] = {
case i#Done(_, _) => IO { i }
case Cont(k) => for {
s <- IO { try { val x = tmpReader.readLine; IO(x) }
catch { case e => enumReader(r, it) }}.join
a <- if (s == null) k(EOF) else loop(k(El(s)))
} yield a
}
loop(it)
}
The inner loop advances the Iteratee, but the outer function still holds on to the original. Since Iteratee is a persistent data structure, to restart you just have to call the function again.
I'm passing the Reader by name here so that r is essentially a function that gives you a fresh (restarted) reader. In practise you will want to bracket this more effectively (close the existing reader on exception).
Here's an answer that doesn't work, but feels like it should:
def restartingIterator[T](getIter: () => Iterator[T]): Iterator[T] = {
new Traversable[T] {
def foreach[U](f: T => U): Unit = {
try {
for (item <- getIter()) {
f(item)
}
} catch {
case e: IOException => this.foreach(f)
}
}
}.toIterator
}
I think this very clearly describes the control flow, which is great.
This code will throw a StackOverflowError in Scala 2.8.0 because of a bug in Traversable.toStream, but even after the fix for that bug, this code still won't work for my use case because toIterator calls toStream, which means that it will store all items in memory.
I'd love to be able to define an Iterator by just writing a foreach method, but there doesn't seem to be any easy way to do that.