Test RestfulController with Grails - json

I'm trying to write some integration tests for a RestfulController in Grails 2.4.0 responding in JSON format. The index()-Method is implemented like this:
class PersonController extends RestfulController<Person> {
...
def index(final Integer max) {
params.max = Math.min(max ?: 10, 100)
respond listAllResources(params), [includes: includeFields]
}
...
}
The integration test looks like this:
void testListAllPersons() {
def controller = new PersonController()
new Person(name: "Person", age: 22).save(flush:true)
new Person(name: "AnotherPerson", age: 31).save(flush:true)
controller.response.format = 'json'
controller.request.method = 'GET'
controller.index()
assertEquals '{{"name":"Person", "age": "22"},{"name":"AnotherPerson", "age": "31"}}', controller.response.json
}
What i don't understand is controller.response.json only contains the "AnotherPerson" instead of both entries.
When i start the server with run-app und test it with a Rest-Client i get both entries.
Any Ideas?

You haven't included enough information to say for sure what the problem is but the following test passes with 2.4.0.
The domain class:
// grails-app/domain/com/demo/Person.groovy
package com.demo
class Person {
String name
Integer age
}
The controller:
// grails-app/controllers/com/demo/PersonController.groovy
package com.demo
class PersonController extends grails.rest.RestfulController<Person> {
PersonController() {
super(Person)
}
def index(final Integer max) {
params.max = Math.min(max ?: 10, 100)
respond listAllResources(params)
}
}
The test:
// test/unit/com/demo/PersonControllerSpec.groovy
package com.demo
import grails.test.mixin.TestFor
import spock.lang.Specification
#TestFor(PersonController)
#Mock(Person)
class PersonControllerSpec extends Specification {
void "test index includes all people"() {
given:
new Person(name: "Person", age: 22).save(flush:true)
new Person(name: "AnotherPerson", age: 31).save(flush:true)
when:
request.method = 'GET'
response.format = 'json'
controller.index()
then:
response.status == 200
response.contentAsString == '[{"class":"com.demo.Person","id":1,"age":22,"name":"Person"},{"class":"com.demo.Person","id":2,"age":31,"name":"AnotherPerson"}]'
}
}

I simplified the example a little too much. I used a named object marshaller which i created (incorrect) in bootstrap.groovy like this:
JSON.createNamedConfig('simplePerson') { converterConfig ->
converterConfig.registerObjectMarshaller(Person) {
JSON.registerObjectMarshaller(Person) {
def map = [:]
map['name'] = it.name
map['age'] = it.age
return map
}
}
}
And used it in the controller:
...
JSON.use("simplePerson")
...
The problem is solved by creating the object marshaller like this:
JSON.createNamedConfig('simplePerson') { converterConfig ->
converterConfig.registerObjectMarshaller(Person) {
def map = [:]
map['name'] = it.name
map['age'] = it.age
return map
}
}

Related

"No transformation found: class io.ktor.utils.io.ByteChannelNative" error using Ktor

I'm trying to fetch and deserialize some data that is being hosted on github.
{
"Meals": [
{
"id": "1598044e-5259-11e9-8647-d663bd870b02",
"name": "Tomato pasta",
"quantity": [{
"quantity": 1 },
{
"quantity": 2
},
{
"quantity": 3
}],
"availableFromDate": "1605802429",
"expiryDate": "1905802429",
"info": "Vegetarian",
"hot": false,
"locationLat": 57.508865,
"locationLong": -6.292,
"distance": null
},
{
"id": "2be2d854-a067-43ec-a488-2e69f0f2a624",
"name": "Pizza",
"quantity": [{
"quantity": 1 },
{
"quantity": 2
},
{
"quantity": 3
}
],
"availableFromDate": "1605802429",
"expiryDate": "1905902429",
"info": "Meat",
"hot": false,
"locationLat": 51.509465,
"locationLong": -0.135392,
"distance": null
}
]
}
If I spin up a json-server locally then it works perfectly, so I know that my data class is not the problem. However when I try to do the same thing from that github link I get this error:
Error Domain=KotlinException Code=0 "No transformation found: class io.ktor.utils.io.ByteChannelNative -> class kotlin.collections.List
I have a feeling it might be to do with setting a ContentType or something along those lines but I haven't had any success specifying that so far.
Here is my code to make the request:
class MealApi {
private val httpClient = HttpClient {
install(JsonFeature) {
val json = Json { ignoreUnknownKeys = true }
serializer = KotlinxSerializer(json)
}
}
suspend fun getAllMeals(): List<Meal> {
return httpClient.get(endpoint)
}
}
and here is my data class just for completeness:
#Serializable
data class Meal(
#SerialName("id")
val id: String,
#SerialName("name")
val name: String,
#SerialName("quantity")
val quantity: List<Quantity>,
#SerialName("availableFromDate")
var availableFromDate: String,
#SerialName("expiryDate")
var expiryDate: String,
#SerialName("info")
val info: String,
#SerialName("hot")
val hot: Boolean,
#SerialName("locationLat")
val locationLat: Float,
#SerialName("locationLong")
val locationLong: Float,
#SerialName("distance")
var distance: Double? = null
)
#Serializable
data class Quantity(
#SerialName("quantity")
val quantity: Int
)
UPDATE
I've found that this server https://gitcdn.link/ allows you to serve your raw github files with the right Content-Type.
I've searched a lot how to change the server response headers (to change the plain/text one to application/json) but it seems that ktor actually doesn't allow to do that:
https://youtrack.jetbrains.com/issue/KTOR-617
https://youtrack.jetbrains.com/issue/KTOR-580
A nice way should be to allow the ResponseObserver to change the server response headers and pass through the modify response. But you can't actually.
Your problem depends, as you pointed out, from the fact that the raw github page provides an header Content-Type=plain/text instead of ContentType=application/json.
So IRL when you are running your API in a real server this won't occur as you'll take care to put the right content type at server level.
But if you want a workaround to this you could rewrite your api call in this way:
suspend fun getAllMealsWithFallback() {
var meals: Meals? = null
try {
meals = httpClient.get(endpoint)
} catch (e: NoTransformationFoundException) {
val mealsString: String = httpClient.get(endpoint)
val json = kotlinx.serialization.json.Json {
ignoreUnknownKeys = true
}
meals = json.decodeFromString(mealsString)
} finally {
println("Meals: ${meals?.meals}")
}
}
I had to add this class to conform to the json text you have provided in the github link.
#Serializable
data class Meals(
#SerialName("Meals")
val meals: List<Meal>,
)
Try this:
install(JsonFeature) {
serializer = KotlinxSerializer(KotlinJson { ignoreUnknownKeys = true })
acceptContentTypes = acceptContentTypes + ContentType.Any
}
If you'd like to accept all content types. Or Use ContentType.Text.Any, ContentType.Text.Html if you preferred.
In case the issue is the Content-Type:
You can alter the list of response content types, for which the KotlinxSerializer gets active, by extending the JsonFeature block to:
install(JsonFeature) {
val json = Json { ignoreUnknownKeys = true }
serializer = KotlinxSerializer(json)
receiveContentTypeMatchers += object : ContentTypeMatcher {
override fun contains(contentType: ContentType): Boolean =
contentType == ContentType("text", "plain")
}
}
If you're using Ktor 2.0, you would need ContentNegotiation plugin instead of JsonFeature.
For example if you use Gson:
install(io.ktor.client.plugins.contentnegotiation.ContentNegotiation) {
gson()
}
As a workaround for ktor version 2.0.3 you can create you own AppContentNegotiation class and in the scope.responsePipeline.intercept provide needed contentType
/**
* A plugin that serves two primary purposes:
* - Negotiating media types between the client and server. For this, it uses the `Accept` and `Content-Type` headers.
* - Serializing/deserializing the content in a specific format when sending requests and receiving responses.
* Ktor supports the following formats out-of-the-box: `JSON`, `XML`, and `CBOR`.
*
* You can learn more from [Content negotiation and serialization](https://ktor.io/docs/serialization-client.html).
*/
public class AppContentNegotiation internal constructor(
internal val registrations: List<Config.ConverterRegistration>
) {
/**
* A [ContentNegotiation] configuration that is used during installation.
*/
public class Config : Configuration {
internal class ConverterRegistration(
val converter: ContentConverter,
val contentTypeToSend: ContentType,
val contentTypeMatcher: ContentTypeMatcher,
)
internal val registrations = mutableListOf<ConverterRegistration>()
/**
* Registers a [contentType] to a specified [converter] with an optional [configuration] script for a converter.
*/
public override fun <T : ContentConverter> register(
contentType: ContentType,
converter: T,
configuration: T.() -> Unit
) {
val matcher = when (contentType) {
ContentType.Application.Json -> JsonContentTypeMatcher
else -> defaultMatcher(contentType)
}
register(contentType, converter, matcher, configuration)
}
/**
* Registers a [contentTypeToSend] and [contentTypeMatcher] to a specified [converter] with
* an optional [configuration] script for a converter.
*/
public fun <T : ContentConverter> register(
contentTypeToSend: ContentType,
converter: T,
contentTypeMatcher: ContentTypeMatcher,
configuration: T.() -> Unit
) {
val registration = ConverterRegistration(
converter.apply(configuration),
contentTypeToSend,
contentTypeMatcher
)
registrations.add(registration)
}
private fun defaultMatcher(pattern: ContentType): ContentTypeMatcher =
object : ContentTypeMatcher {
override fun contains(contentType: ContentType): Boolean =
contentType.match(pattern)
}
}
/**
* A companion object used to install a plugin.
*/
#KtorDsl
public companion object Plugin : HttpClientPlugin<Config, AppContentNegotiation > {
public override val key: AttributeKey<AppContentNegotiation> =
AttributeKey("ContentNegotiation")
override fun prepare(block: Config.() -> Unit): AppContentNegotiation {
val config = Config().apply(block)
return AppContentNegotiation(config.registrations)
}
override fun install(plugin: AppContentNegotiation, scope: HttpClient) {
scope.requestPipeline.intercept(HttpRequestPipeline.Transform) { payload ->
val registrations = plugin.registrations
registrations.forEach { context.accept(it.contentTypeToSend) }
if (subject is OutgoingContent || DefaultIgnoredTypes.any { it.isInstance(payload) }) {
return#intercept
}
val contentType = context.contentType() ?: return#intercept
if (payload is Unit) {
context.headers.remove(HttpHeaders.ContentType)
proceedWith(EmptyContent)
return#intercept
}
val matchingRegistrations =
registrations.filter { it.contentTypeMatcher.contains(contentType) }
.takeIf { it.isNotEmpty() } ?: return#intercept
if (context.bodyType == null) return#intercept
context.headers.remove(HttpHeaders.ContentType)
// Pick the first one that can convert the subject successfully
val serializedContent = matchingRegistrations.firstNotNullOfOrNull { registration ->
registration.converter.serialize(
contentType,
contentType.charset() ?: Charsets.UTF_8,
context.bodyType!!,
payload
)
} ?: throw ContentConverterException(
"Can't convert $payload with contentType $contentType using converters " +
matchingRegistrations.joinToString { it.converter.toString() }
)
proceedWith(serializedContent)
}
scope.responsePipeline.intercept(HttpResponsePipeline.Transform) { (info, body) ->
if (body !is ByteReadChannel) return#intercept
if (info.type == ByteReadChannel::class) return#intercept
// !!!!!!! Provide desired content type here
val contentType = <HERE> // default implementation is - context.response.contentType() ?: return#intercept
val registrations = plugin.registrations
val matchingRegistrations = registrations
.filter { it.contentTypeMatcher.contains(contentType) }
.takeIf { it.isNotEmpty() } ?: return#intercept
// Pick the first one that can convert the subject successfully
val parsedBody = matchingRegistrations.firstNotNullOfOrNull { registration ->
registration.converter
.deserialize(context.request.headers.suitableCharset(), info, body)
} ?: return#intercept
val response = HttpResponseContainer(info, parsedBody)
proceedWith(response)
}
}
}
}
And then install it for HttpClient
HttpClient {
install(AppContentNegotiation) {
json(json)
addDefaultResponseValidation()
}
}
In my case, this exception was thrown when I was attempting to access the body() potion of the HTTP response as such:
val httpResponse = httpClient.post(urlString) { ... }
val body = httpResponse.body<YourExpectedSerializableResponseType>
In the happy path scenario, the server would return a body that matched YourExpectedSerializableResponseType and everything would work as expected.
However, for a particular edge case that returned a different (still considered successful) status code, the server returned an empty response body. Since the client was expecting a response body and in this case, there wasn't any, this exception was thrown because it could not serialize an empty response body to the expected type YourExpectedSerializableResponseType.
My recommendation: In addition to ensuring your server is returning the type you expect to serialize/consume on your client, confirm your server is actually returning an object.
Internal Dialogue: I wonder if this exception could be more clear in this case as the issue is more so that the client expected a response body to exist and less so that an empty response body ("") couldn't be serialized into an expected type - especially given that an empty response body isn't even valid JSON. Hm. 🤔

Insert record into Db using Slick (Scala), Best practices for an Entity

First to say, I'm newcomer in Scala and really need a little help. I need to build a web api, and I'll try to insert one record into database, but have some problems with mapping the entity (db table) into a model (class). I worked with .Net Core Web API (there I used Entity Framework Core, here in Scala use Slick) and try to keep same arhitecture in Scala, but need some more informations, because on the internet I find a lot of versions, and can not choose the best.
As database, MySQL is used.
User.scala
case class User(
id: Int = 0,
userName: String,
firstName: String,
lastName: String
) {
override def equals(that: Any): Boolean = true
}
object User {
implicit object UserFormat extends Format[User] {
def writes(user: User): JsValue = {
val userSeq = Seq(
"id" -> JsNumber(user.id),
"userName" -> JsString(user.userName),
"firstName" -> JsString(user.firstName),
"lastName" -> JsString(user.lastName)
)
JsObject(userSeq)
}
def reads(json: JsValue): JsResult[User] = {
JsSuccess(User(
(json \ "id").as[Int].value,
(json \ "userName").as[String].value,
(json \ "firstName").as[String].value,
(json \ "lastName").as[String].value)
)
}
}
def tupled = (this.apply _).tupled
}
class UserMap #Inject()(protected val dbConfigProvider: DatabaseConfigProvider)(implicit ex: ExecutionContext) {
val dbConfig: DatabaseConfig[JdbcProfile] = dbConfigProvider.get[JdbcProfile]
val db: JdbcBackend#DatabaseDef = dbConfig.db
val dbUsers = TableQuery[UserDef]
def getAll(): Unit = {
val action = sql"SELECT Id, UserName, FirstName, LastName FROM Users".as[(Int, String, String, String)]
return db.run(action)
}
def add(user: User): Future[Seq[User]] = {
dbUsers += user
db.run(dbUsers.result)
}
}
UserDef.scala (which is a mapper of db table / entity)
class UserDef(tag: Tag) extends Table[User](tag, "Users") {
def id = column[Int]("Id", O.PrimaryKey, O.AutoInc)
def userName = column[String]("UserName")
def firstName = column[String]("FirstName")
def lastName = column[String]("LastName")
override def * = (id, userName, firstName, lastName) <> (create, extract)
def create(user: (Int, String, String, String)): User = User(user._1, user._2, user._3, user._4)
def extract(user: User): Option[(Int, String, String, String)] = Some((user.id, user.userName,user.firstName,user.lastName))
}
UsersController.scala
def createUser = Action(parse.json) { implicit request => {
val userJson = request.body
var user = new User(
-1,
(userJson \ "userName").as[String].value,
(userJson \ "firstName").as[String].value,
(userJson \ "lastName").as[String].value
)
var users = TableQuery[UserDef]
Await.result(db.run(DBIO.seq(
users += user,
users.result.map(println))), Duration.Inf
)
Ok(Json.toJson(user))
}
}
How I see the problem:
UserDef is an Entity and must remain clean, only table columns definitions
UserMap is the bridge between User class and UserDef (entity), can be used as a repository with crud methods (getAll(), getById(id), create(user), update(user), delete(id)). This is in same file as User class, but probably must be moved in another.
User class is the model and need to contain only their parameters and writes/reads (Scala specifics)
and now in the controller:
If I try to insert a record into database, with current method, first I need to get all rows from table, and then to add the new record in the list. What happening if I have 3 4mil records in this table? Will get all these rows useless to insert only a new row.
Then, after inserting this new row, I need to return it into client, but how I can get it updated (Id is every time -1, but if I get entire list to see what it contain, I can see the correct id for the newest entity)
thx
Finally, I found a good solution and post it here, maybe somebody need this:
UserMap, for me at least will become UserRepository. There I have CRUD operations and maybe some extra :
def getAll(): Future[Seq[User]] = {
db.run(dbUsers.result)
}
def getById(id: Int): Future[Option[User]] ={
val action = dbUsers.filter(_.id === id).result.headOption
db.run(action)
}
def create(user: User): Future[User] = {
val insertQuery = dbUsers returning dbUsers.map(_.id) into ((x, id) => x.copy(id = id))
val action = insertQuery += user
db.run(action)
}
def update(user: User) {
Try( dbUsers.filter(_.id === user.id).update(user)) match {
case Success(response) => db.run(response)
case Failure(_) => println("An error occurred!")
}
}
def delete(id: Int) {
Try( dbUsers.filter(_.id === id).delete) match {
case Success(response) => db.run(response)
case Failure(_) => println("An error occurred!")
}
}
and UsersController:
def getAll() = Action {
var users = Await.result(usersRepository.getAll(), Duration.Inf)
Ok(Json.toJson(users))
}
def getById(id: Int) = Action { implicit request => {
val user = Await.result(usersRepository.getById(id), Duration.Inf)
Ok(Json.toJson(user))
}
}
def create = Action(parse.json) { implicit request => {
val userJson = request.body
var user = new User(
-1,
(userJson \ "userName").as[String].value,
(userJson \ "firstName").as[String].value,
(userJson \ "lastName").as[String].value
)
var createdUser = Await.result(usersRepository.create((user)), Duration.Inf)
Ok(Json.toJson(createdUser))
}
}
def update(id: Int) = Action(parse.json) { implicit request => {
val userJson = request.body
var user = new User(
(userJson \ "id").as[Int].value,
(userJson \ "userName").as[String].value,
(userJson \ "firstName").as[String].value,
(userJson \ "lastName").as[String].value
)
var updatedUser = usersRepository.update(user)
Ok(Json.toJson(user))
}
}
def delete(id: Int) = Action {
usersRepository.delete(id)
Ok("true")
}
Anyway, I know I have some bad blocks of code there...especially in create & update methods, where convert json to User.
I wanted to give it a try, and here is a full working example of a Play 2.7/Scala 2.13/Slick 4.0.2 REST-API controller bound to a MySQL database.
Since you are starting with Scala, maybe it is a bit overwhelming at first to get eased with Play, Slick, etc...
So here is an humble skeleton (derived from Play-Slick GitHub)
So first, since we want to write an API, here is the conf/routes file:
GET /users controllers.UserController.list()
GET /users/:uuid controllers.UserController.get(uuid: String)
POST /users controllers.UserController.create()
PUT /users controllers.UserController.update()
DELETE /users/:uuid controllers.UserController.delete(uuid: String)
Nothing to fancy here, we just bind routes to functions in the upcoming controller.
Just notice that the 2nd GET and the DELETE expect an UUID as query param, while Json bodies with be used for the POST and PUT.
It would be nice to see the model right now, in app/models/User.scala:
package models
import java.util.UUID
import play.api.libs.json.{Json, OFormat}
case class User(
uuid: UUID,
username: String,
firstName: String,
lastName: String
) {
}
object User {
// this is because defining a companion object shadows the case class function tupled
// see: https://stackoverflow.com/questions/22367092/using-tupled-method-when-companion-object-is-in-class
def tupled = (User.apply _).tupled
// provides implicit json mapping
implicit val format: OFormat[User] = Json.format[User]
}
I used an uuid instead using a numerical id, but basically, it is the same.
Notice that a Json serializer/deserializer can be written in just one line (you don't need to detail it with case classes). I think it is also a good practice to not override it to produce Seq as found on your code, since this serializer will be very usefull when converting objects to Json on the controller.
Now the tupled definition is most likelly a hack (see comment) that will be required later on the DAO...
Next, we need a controller in app/controllers/UserController.scala:
package controllers
import java.util.UUID
import forms.UserForm
import javax.inject.Inject
import play.api.Logger
import play.api.data.Form
import play.api.i18n.I18nSupport
import play.api.libs.json.Json
import play.api.mvc._
import services.UserService
import scala.concurrent.{ExecutionContext, Future}
import scala.util.{Failure, Success, Try}
class UserController #Inject()(userService: UserService)
(implicit ec: ExecutionContext) extends InjectedController with I18nSupport {
lazy val logger: Logger = Logger(getClass)
def create: Action[AnyContent] = Action.async { implicit request =>
withFormErrorHandling(UserForm.create, "create failed") { user =>
userService
.create(user)
.map(user => Created(Json.toJson(user)))
}
}
def update: Action[AnyContent] = Action.async { implicit request =>
withFormErrorHandling(UserForm.create, "update failed") { user =>
userService
.update(user)
.map(user => Ok(Json.toJson(user)))
}
}
def list: Action[AnyContent] = Action.async { implicit request =>
userService
.getAll()
.map(users => Ok(Json.toJson(users)))
}
def get(uuid: String): Action[AnyContent] = Action.async { implicit request =>
Try(UUID.fromString(uuid)) match {
case Success(uuid) =>
userService
.get(uuid)
.map(maybeUser => Ok(Json.toJson(maybeUser)))
case Failure(_) => Future.successful(BadRequest(""))
}
}
def delete(uuid: String): Action[AnyContent] = Action.async {
Try(UUID.fromString(uuid)) match {
case Success(uuid) =>
userService
.delete(uuid)
.map(_ => Ok(""))
case Failure(_) => Future.successful(BadRequest(""))
}
}
private def withFormErrorHandling[A](form: Form[A], onFailureMessage: String)
(block: A => Future[Result])
(implicit request: Request[AnyContent]): Future[Result] = {
form.bindFromRequest.fold(
errors => {
Future.successful(BadRequest(errors.errorsAsJson))
}, {
model =>
Try(block(model)) match {
case Failure(e) => {
logger.error(onFailureMessage, e)
Future.successful(InternalServerError)
}
case Success(eventualResult) => eventualResult.recover {
case e =>
logger.error(onFailureMessage, e)
InternalServerError
}
}
})
}
}
So here:
basically, each of our 5 functions referenced from the routes file check input, and then delegate the work to an injected UserService (more on that later)
for the create and update functions, you can see that we use Play Forms that I think is also a good practice. Their role is to validate the incoming Json, and that Marshall it into a User type.
Also, you can see that we use Action.async: Scala offers a very powerfull leverage with Futures so lets use it! Basically by doing so, you ensure that your code is not-blocking, thus easing the IOPS on your hardware.
Finally for the case of GET (one), GET (all), POST and PUT, since we return users, and have a deseralizer, a simple Json.toJson(user) do the work.
Before jumping to service and dao, lets see the form, in app/forms/UserForm.scala:
package forms
import java.util.UUID
import models.User
import play.api.data.Form
import play.api.data.Forms.{mapping, nonEmptyText, _}
object UserForm {
def create: Form[User] = Form(
mapping(
"uuid" -> default(uuid, UUID.randomUUID()),
"username" -> nonEmptyText,
"firstName" -> nonEmptyText,
"lastName" -> nonEmptyText,
)(User.apply)(User.unapply)
)
}
Nothing too fancy here, just as the doc says, although there is just a trick : when no uuid is defined (in the POST case, then we generate one).
Now, the service... not so much required in this very case, but in practice it might be a good thing to have an extra layer (dealing with acls for example), in app/services/UserService.scala:
package services
import java.util.UUID
import dao.UserDAO
import javax.inject.Inject
import models.User
import scala.concurrent.{ExecutionContext, Future}
class UserService #Inject()(dao: UserDAO)(implicit ex: ExecutionContext) {
def get(uuid: UUID): Future[Option[User]] = {
dao.get(uuid)
}
def getAll(): Future[Seq[User]] = {
dao.all()
}
def create(user: User): Future[User] = {
dao.insert(user)
}
def update(user: User): Future[User] = {
dao.update(user)
}
def delete(uuid: UUID): Future[Unit] = {
dao.delete(uuid)
}
}
As you can see, here, it is just a wrapper around the dao, and finnally the dao in app/dao/UserDao.scala:
package dao
import java.util.UUID
import javax.inject.Inject
import models.User
import play.api.db.slick.{DatabaseConfigProvider, HasDatabaseConfigProvider}
import play.db.NamedDatabase
import slick.jdbc.JdbcProfile
import scala.concurrent.{ExecutionContext, Future}
class UserDAO #Inject()(#NamedDatabase("mydb") protected val dbConfigProvider: DatabaseConfigProvider)(implicit executionContext: ExecutionContext) extends HasDatabaseConfigProvider[JdbcProfile] {
import profile.api._
private val users = TableQuery[UserTable]
def all(): Future[Seq[User]] = db.run(users.result)
def get(uuid: UUID): Future[Option[User]] = {
db.run(users.filter(_.uuid === uuid).result.headOption)
}
def insert(user: User): Future[User] = {
db.run(users += user).map(_ => user)
}
def update(user: User): Future[User] = {
db.run(users.filter(_.uuid === user.uuid).update(user)).map(_ => user)
}
def delete(uuid: UUID): Future[Unit] = {
db.run(users.filter(_.uuid === uuid).delete).map(_ => ())
}
private class UserTable(tag: Tag) extends Table[User](tag, "users") {
def uuid = column[UUID]("uuid", O.PrimaryKey)
def username = column[String]("username")
def firstName = column[String]("firstName")
def lastName = column[String]("lastName")
def * = (uuid, username, firstName, lastName) <> (User.tupled, User.unapply)
}
}
So, here I have just adapted the code from the official play-slick example, so I guess, I do not have better comment than theirs...
Hope, the whole things helps to get a better picture :)
If something is unclear, feel free to ask!

Do one webservice request only from play framework

I'm new to the play framework generally and how to use it with Scala. I want to build a proxy for big Json objects. I achieved so far that the json is stored in a cache and if it is not there, requested from a webservice.
However when two requests are coming in, targeting the same end point (webservice and path are identicall) only one call should be performed and the other request should wait for the result of the first call. At the moment it is performing a call to the service with every request.
This is my controller:
#Singleton
class CmsProxyController #Inject()(val cmsService: CmsProxyService) extends Controller {
implicit def ec : ExecutionContext = play.api.libs.concurrent.Execution.defaultContext
def header(path: String) = Action.async { context =>
cmsService.head(path) map { title =>
Ok(Json.obj("title" -> title))
}
}
def teaser(path: String) = Action.async { context =>
cmsService.teaser(path) map { res =>
Ok(res).as(ContentTypes.JSON)
}
}
}
This is the service:
trait CmsProxyService {
def head(path: String): Future[String]
def teaser(path: String): Future[String]
}
#Singleton
class DefaultCmsProxyService #Inject()(cache: CacheApi, cmsCaller: CmsCaller) extends CmsProxyService {
private val BASE = "http://foo.com"
private val CMS = "bar/rest/"
private val log = Logger("application")
override def head(path: String) = {
query(url(path), "$.payload[0].title")
}
override def teaser(path: String) = {
query(url(path), "$.payload[0].content.teaserText")
}
private def url(path: String) = s"${BASE}/${CMS}/${path}"
private def query(url: String, jsonPath: String): Future[String] = {
val key = s"${url}?${jsonPath}"
val payload = findInCache(key)
if (payload.isDefined) {
log.debug("found payload in cache")
Future.successful(payload.get)
} else {
val queried = parse(fetch(url)) map { json =>
JSONPath.query(jsonPath, json).as[String]
}
queried.onComplete(value => saveInCache(key, value.get))
queried
}
}
private def parse(fetched: Future[String]): Future[JsValue] = {
fetched map { jsonString =>
Json.parse(jsonString)
}
}
//retrieve the requested value from the cache or from ws
private def fetch(url: String): Future[String] = {
val body = findInCache(url)
if (body.isDefined) {
log.debug("found body in cache")
Future.successful(body.get)
} else {
cmsCaller.call(url)
}
}
private def findInCache(key: String): Option[String] = cache.get(key)
private def saveInCache(key: String, value: String, duration: FiniteDuration = 5.minutes) = cache.set(key, value, 5.minutes)
}
And finally the call to the webservice:
trait CmsCaller {
def call(url: String): Future[String]
}
#Singleton
class DefaultCmsCaller #Inject()(wsClient: WSClient) extends CmsCaller {
import scala.concurrent.ExecutionContext.Implicits.global
//keep those futures which are currently requested
private val calls: Map[String, Future[String]] = TrieMap()
private val log = Logger("application")
override def call(url: String): Future[String] = {
if(calls.contains(url)) {
Future.successful("ok")
}else {
val f = doCall(url)
calls put(url, f)
f
}
}
//do the final call
private def doCall(url: String): Future[String] = {
val request = ws(url)
val response = request.get()
val mapped = mapResponse(response)
mapped.onComplete(_ => cmsCalls.remove(url))
mapped
}
private def ws(url: String): WSRequest = wsClient.url(url)
//currently executed with every request
private def mapResponse(f: Future[WSResponse]): Future[String] = {
f.onComplete(_ => log.debug("call completed"))
f map {res =>
val status = res.status
log.debug(s"ws called, response status: ${status}")
if (status == 200) {
res.body
} else {
""
}
}
}
}
My question is: How can only one call to the webservice beeing executed? Even if there are several requests to the same target. I don't want to block it, the other request (not sure if I use the right word here) shall just be informed that there is already a webservice call on the way.
The request to head and teaser, see controller, shall perform only one call to the webservice.
Simple answer using Scala lazy keyword
def requestPayload(): String = ??? //do something
#Singleton
class SimpleCache #Inject() () {
lazy val result: Future[String] = requestPayload()
}
//Usage
#Singleton
class SomeController #Inject() (simpleCache: SimpleCache) {
def action = Action { req =>
simpleCache.result.map { result =>
Ok("success")
}
}
}
First request will trigger the rest call and all the other requests will use the cached result. Use map and flatMap to chain the requests.
Complicated answer using Actors
Use Actor to queue requests and Cache the result of the first successful request json result. All the other requests will read the result of the first request.
case class Request(value: String)
class RequestManager extends Actor {
var mayBeResult: Option[String] = None
var reqs = List.empty[(ActorRef, Request)]
def receive = {
case req: Request =>
context become firstReq
self ! req
}
def firstReq = {
case req: Request =>
process(req).onSuccess { value =>
mayBeResult = Some(value)
context become done
self ! "clear_pending_reqs"
}
context become processing
}
def processing = {
case req: Request =>
//queue requests
reqs = reqs ++ List(sender -> req)
}
def done = {
case "clear_pending_reqs" =>
reqs.foreach { case (sender, _) =>
//send value to the sender
sender ! value.
}
}
}
handle the case where the first request fails. In the above code block if the first request fails then actor will never go to the done state.
I solved my problem with a synchronization of the cache in the service. I'm not sure if this an elegant solution, but it works for me.
trait SyncCmsProxyService {
def head(path: String): String
def teaser(path: String): String
}
#Singleton
class DefaultSyncCmsProxyService #Inject()(implicit cache: CacheApi, wsClient: WSClient) extends SyncCmsProxyService with UrlBuilder with CacheAccessor{
private val log = Logger("application")
override def head(path: String) = {
log.debug("looking for head ...")
query(url(path), "$.payload[0].title")
}
override def teaser(path: String) = {
log.debug("looking for teaser ...")
query(url(path), "$.payload[0].content.teaserText")
}
private def query(url: String, jsonPath: String) = {
val key = s"${url}?${jsonPath}"
val payload = findInCache(key)
if (payload.isDefined) {
payload.get
}else{
val json = Json.parse(body(url))
val queried = JSONPath.query(jsonPath, json).as[String]
saveInCache(key, queried)
}
}
private def body(url: String) = {
cache.synchronized {
val body = findInCache(url)
if (body.isDefined) {
log.debug("found body in cache")
body.get
} else {
saveInCache(url, doCall(url))
}
}
}
private def doCall(url : String): String = {
import scala.concurrent.ExecutionContext.Implicits.global
log.debug("calling...")
val req = wsClient.url(url).get()
val f = req map { res =>
val status = res.status
log.debug(s"endpoint called! response status: ${status}")
if (status == 200) {
res.body
} else {
""
}
}
Await.result(f, 15.seconds)
}
}
Note that I omitted the traits UrlBuilder and CacheAccessor here because they are trivial.

How to serialize object type to JSON in Scalatra?

I am a newbie in Scalatra. I have a servlet with JacksonJsonSupport which serves REST endpoint with list of objects.
class OperationsController extends MyappStack with JacksonJsonSupport {
before() {
contentType = formats("json")
}
get("/") {
Data.operations
}
}
The Operation is implemented by either Adding or Removing case classes.
How do I add to the GET / response the specific class to the value? I would like to get as a response:
[
{
"operation": "Adding",
"value": 100
}
]
Instead of
[
{
"value": 100
}
]
Where Adding is a class that extends Operation.
For polymorphic values json4s can add the concrete type as an additional field. This is called a "type hint":
[{
"jsonClass": "Adding",
"value": 10
}, {
"jsonClass": "Adding",
"value": 20
}, {
"jsonClass": "Removing",
"value": 20
}]
This is for example using the ShortTypeHints:
import org.json4s.{ShortTypeHints, DefaultFormats}
import org.scalatra.ScalatraServlet
import org.scalatra.json.JacksonJsonSupport
import org.scalatra.test.specs2.MutableScalatraSpec
sealed trait Operation
case class Adding(value: Int) extends Operation
case class Removing(value: Int) extends Operation
class json1 extends MutableScalatraSpec {
mount(new ScalatraServlet with JacksonJsonSupport {
def typeHints = new ShortTypeHints(List(
classOf[Adding], classOf[Removing]
))
implicit lazy val jsonFormats = DefaultFormats + typeHints
before() {
contentType = formats("json")
}
get("/") {
List(
Adding(10),
Adding(20),
Removing(20)
)
}
}, "/*")
"Should return a list of operations" in {
get("/", headers = Seq("Content-type" -> "application/json")) {
println(body)
status should beEqualTo(200)
}
}
}
I think, that the easiest way is to update your case classes like
case class Adding(value: Int, operation: String = "Adding")
case class Removing (value: Int, operation: String = "Removing")
Another way is to update your jsonFormats with custom serializer, I found example of jsons custom serialization here
json_conversion.scala file we created the trait SimpleMongoDbJsonConversion and we use this in MyScalatraServlet.scala file, see the example below.
json_conversion.scala
package com.example.app
import org.scalatra._
import com.mongodb.casbah.Imports._
trait SimpleMongoDbJsonConversion extends ScalatraBase with ApiFormats {
def renderMongo = {
case dbo: DBObject =>
contentType = formats("json")
dbo.toString
case xs: TraversableOnce[_] =>
contentType = formats("json")
val l = xs map (x => x.toString) mkString(",")
"[" + l + "]"
}: RenderPipeline
override protected def renderPipeline = renderMongo orElse super.renderPipeline
}
MyScalatraServlet.scala
package com.example.app
import org.scalatra._
import com.mongodb.casbah.Imports._
class MyScalatraMongoServlet(mongoColl: MongoCollection) extends MyScalatraWebAppStack with SimpleMongoDbJsonConversion {
get("/") {
<html>
<body>
<h1>Hello, world!</h1>
Say hello to Scalate.
</body>
</html>
}
post("/insert") {
val key = params("key")
val value = params("value")
val newObj = MongoDBObject(key->value)
mongoColl += newObj
}
get("/users") {
mongoColl.find()
for { x <- mongoColl } yield x
}
}

JSON Marshaller for each action in controller (Grails)

In grails how to have JSON.registerObjectMarshaller for each action in controller.
Here is an example
My User domain object:
String username
String empId
String attendanceID
String password
String firstName
in my controller:
def myaction1() {
def user=User.getAll()
// XXX here i want to return just username and empId
render user as JSON
}
def myaction2() {
def user=User.getAll()
// XXX here i want to return just username and firstName
render user as JSON
}
While it may be a bit overkill for such as simple domain, and you could likely get away with just returning a Map of your data, the question is still valid.
How do you register custom named marshallers?
Typically you will do this inside your grails-app/conf/BootStrap.groovy (or a new file grails-app/conf/CustomMarshallersBootStrap.groovy if you want to keep things clean). An example of this might look like this:
// Bootstrap.groovy
import grails.converters.JSON
import com.example.User
class BootStrap {
def init = { servletContext ->
JSON.createNamedConfig("userEmployeeView", {
JSON.registerObjectMarshaller(User) { User o ->
return [
username: o.username,
empId: o.empId
]
}
})
JSON.createNamedConfig("userOtherView", {
JSON.registerObjectMarshaller(User) { User o ->
return [
username: o.username,
firstName: o.firstName
]
}
})
}
def destroy = { }
}
This will register two named marshallers which you can use in your controller(s) like this:
// UserController.groovy
package com.example
import grails.converters.JSON
class UserController {
def action1() {
def users = User.getAll()
JSON.use("userEmployeeView") {
render users as JSON
}
}
def action2() {
def users = User.getAll()
JSON.use("userOtherView") {
render users as JSON
}
}
}
The above uses named marshllers which allows you to control which JSON representation (actually just a Map) will be used when creating the final JSON output.
Hope this helps, and forgive any typos as I wrote this off the top of my head.