I am new to Scala. I want to parse JSON data in scala store to database table.
My GET method looks like this (Please ignore the permissions):
def Classes = withAuth { username =>
implicit request =>
User.access(username, User.ReadXData).map { user =>
implicit val writer = new Writes[Class] {
def writes(entry: Class): JsValue = Json.obj(
"id" -> entry.id,
"name" -> entry.name
)
}
val classes = (Class.allAccessible(user))
Ok(Json.obj("success" -> true, "classes" -> classes))
}.getOrElse(Forbidden(Application.apiMessage("Not authorised"))) }
This GET method returns the json below:
"success":true,"schools":[{"id":93,"name":"Happy unniversity",}]}
I'm currently rendering the JSOn in a datatables js (editor) grid - with success
HOWEVER, I'm unable to parse and POST the JSON and store it to the database (mysql) table.
Thank you for your guidance!
Looks you are using play-json.
For class User
import play.api.libs.json.Json
final case class User(id: String, name: String)
object User {
implicit val userFormat = Json.format[User]
}
object UserJson {
def main(args: Array[String]): Unit = {
val user = User("11", "Peter")
val json = Json.toJson(user).toString()
println("json ===> " + json)
val user2 = Json.parse(json).as[User]
println("name ===> " + user2.name)
}
}
I definitely recommend this lib: "de.heikoseeberger" %% "akka-http-jackson" % "1.27.0" for akka-http.
Related
I want to convert variable message which is of type scala.Seq[Scala.Document] to JSON format in following code:
path("getMessages"){
get {
parameters('roomname.as[String]) {
(roomname) =>
try {
val messagesByGroupName = MongoDatabase.collectionForChat.find(equal("groupChatName",roomname)).toFuture()
val messages = Await.result(messagesByGroupName,60.seconds)
println("Messages:"+messages)
complete(messages)
}
catch {
case e:TimeoutException =>
complete("Reading file timeout.")
}
}
}
But it is giving me error on complete(messages) line. It is not accepting message of that type.
I tried to convert it into JSON by using following :
import play.api.libs.json._
object MyJsonProtocol{
implicit object ChatFormat extends Format[Chat] {
def writes(c: Chat) : JsValue = {
val chatSeq = Seq (
"sender" -> JsString(c.sender),
"receiver" -> JsString(c.receiver),
"message" -> JsString(c.message),
"groupChatName" -> JsString(c.groupChatName),
)
JsObject(chatSeq)
}
def reads(value: JsValue) = {
JsSuccess(Chat("","","",""))
}
}
}
But it is not working.
My Chat.scala class is as follows:
import play.api.libs.json.{Json, Reads, Writes}
class Chat(var sender:String,var receiver:String,var message:String, var groupChatName:String){
def setSenderName(senderName:String) = {
sender = senderName
}
def setReceiverName(receiverName:String) = {
receiver = receiverName
}
def setMessage(getMessage:String) = {
message = getMessage
}
def setGroupChatName(chatName:String) = {
groupChatName = chatName
}
}
object Chat {
def apply(sender: String, receiver: String, message: String, groupname: String): Chat
= new Chat(sender, receiver, message,groupname)
def unapply(arg: Chat): Option[(String, String, String,String)] = ???
implicit val requestReads: Reads[Chat] = Json.reads[Chat]
implicit val requestWrites: Writes[Chat] = Json.writes[Chat]
}
I am also not able to figure out what to write in unapply method.
I am new to scala and akka.
EDIT:
My MongoDatabase.scala which has collection is as follows:
object MongoDatabase {
val chatCodecProvider = Macros.createCodecProvider[Chat]()
val codecRegistry = CodecRegistries.fromRegistries(
CodecRegistries.fromProviders(chatCodecProvider),
DEFAULT_CODEC_REGISTRY
)
implicit val system = ActorSystem("Scala_jwt-App")
implicit val executor: ExecutionContext = system.dispatcher
val mongoClient: MongoClient = MongoClient()
val databaseName = sys.env("database_name")
// Getting mongodb database
val database: MongoDatabase = mongoClient.getDatabase(databaseName).withCodecRegistry(codecRegistry)
val registrationCollection = sys.env("register_collection_name")
val chatCollection = sys.env("chat_collection")
// Getting mongodb collection
val collectionForUserRegistration: MongoCollection[Document] = database.getCollection(registrationCollection)
collectionForUserRegistration.drop()
val collectionForChat: MongoCollection[Document] = database.getCollection(chatCollection)
collectionForChat.drop()
}
And if try to change val collectionForChat: MongoCollection[Document] = database.getCollection(chatCollection)
to
val collectionForChat: MongoCollection[Chat] = database.getCollection[Chat](chatCollection)
then I get error on in saveChatMessage() method below:
def saveChatMessage(sendMessageRequest: Chat) : String = {
val senderToReceiverMessage : Document = Document(
"sender" -> sendMessageRequest.sender,
"receiver" -> sendMessageRequest.receiver,
"message" -> sendMessageRequest.message,
"groupChatName" -> sendMessageRequest.groupChatName)
val chatAddedFuture = MongoDatabase.collectionForChat.insertOne(senderToReceiverMessage).toFuture()
Await.result(chatAddedFuture,60.seconds)
"Message sent"
}
on val chatAddedFuture = MongoDatabase.collectionForChat.insertOne(senderToReceiverMessage).toFuture() this line since it accepts data of type Seq[Document] and I am trying to add data of type Seq[Chat]
I am going to assume that MongoDatabase.collectionForChat.find(equal("groupChatName",roomname)) returns either Seq[Chat], or Chat. Both of them are the same for play.
You have 2 options:
Adding the default format on the companion object:
object Chat {
implicit val format: Format[Chat] = Json.format[Chat]
}
In this case you can delete the object MyJsonProtocol which is not used.
In case you want to keep your own serializers(i.e. MyJsonProtocol), you need to rename MyJsonProtocol into Chat. This way the complete route will be able to find the implicit Format.
create case class for the message object you want to send
for example:
case class MyMessage(sender: String, receiver: String, message: String, groupChatName: String)
You should create Format for the type of case class
implicit val MessageTypeFormat = Json.format[MyMessage]
if complete should get JSON type - then call complete myMessage when myMessage is an instance of MyMessage.
complete(Json.toJson(myMessage))
I am using the Play Framework and trying to build JSON validator for a class with abstract members. Shown below, the DataSource class is the base class which I am trying to validate the format against.
// SourceTypeConfig Trait.
trait SourceTypeConfig
final case class RDBMSConfig(...) extends SourceTypeConfig
object RDBMSConfig { implicit val fmt = Json.format[RDBMSConfig] }
final case class DirectoryConfig(
path: String,
pathType: String // Local, gcloud, azure, aws, etc.
) extends SourceTypeConfig
object DirectoryConfig { implicit val fmt = Json.format[DirectoryConfig] }
// FormatConfig trait.
trait FormatConfig
final case class SQLConfig(...) extends FormatConfig
object SQLConfig { implicit val fmt = Json.format[SQLConfig]}
final case class CSVConfig(
header: String,
inferSchema: String,
delimiter: String
) extends FormatConfig
object CSVConfig { implicit val fmt = Json.format[CSVConfig]}
// DataSource base class.
case class DataSource(
name: String,
sourceType: String,
sourceTypeConfig: SourceTypeConfig,
format: String,
formatConfig: FormatConfig
)
What I am hoping to accomplish:
val input: JsValue = Json.parse(
"""
{
"name" : "test1",
"sourceType" : "directory",
"sourceTypeConfig" : {"path" : "gs://test/path", "pathType" "google"},
"format" : "csv",
"formatConfig" : {"header" : "yes", "inferSchema" : "yes", "delimiter" : "|"}
}
"""
)
val inputResult = input.validate[DataSource]
What I am struggling with is building the DataSource object and defining its reads/writes/format. I would like it to contain a match based on the sourceType and format values that direct it to point towards the associated sourceTypeConfig and formatConfig's formats so it can parse out the JSON.
Instead of building a parser at the DataSource level, I defined parsers at the SourceConfig and FormatConfig levels, similar to what is shown below.
sealed trait SourceConfig{val sourceType: String}
object SourceConfig{
implicit val fmt = new Format[SourceConfig] {
def reads(json: JsValue): JsResult[SourceConfig] = {
def from(sourceType: String, data: JsObject): JsResult[SourceConfig] = sourceType match {
case "RDBMS" => Json.fromJson[RDBMSConfig](data)(RDBMSConfig.fmt)
case "directory" => Json.fromJson[DirectoryConfig](data)(DirectoryConfig.fmt)
case _ => JsError(s"Unknown source type: '$sourceType'")
}
for {
sourceType <- (json \ "sourceType").validate[String]
data <- json.validate[JsObject]
result <- from(sourceType, data)
} yield result
}
def writes(source: SourceConfig): JsValue =
source match {
case b: RDBMSConfig => Json.toJson(b)(RDBMSConfig.fmt)
case b: DirectoryConfig => Json.toJson(b)(DirectoryConfig.fmt)
}
}
}
Then, DataSource could be simply defined as:
object DataSource { implicit val fmt = Json.format[DataSource] }
Another option is to use play-json-derived-codecs library:
libraryDependencies += "org.julienrf" %% "play-json-derived-codecs" % "4.0.0"
import julienrf.json.derived.flat
implicit val format1: OFormat[RDBMSConfig] = Json.format[RDBMSConfig]
implicit val format2: OFormat[DirectoryConfig] = Json.format[DirectoryConfig]
implicit val format3: OFormat[SourceTypeConfig] = flat.oformat((__ \ "sourceType").format[String])
I am new with play framework and i want to ask periodically to amazon about some products in order to insert them into a kafka topic, an error happens when i try to compile the code.
This is the code of the KafkaProducer:
file example.model.AmazonProducerExample
//UPDATED method with the suggestions from the users, thank you guys!
package example.utils
import jodd.lagarto.dom.{NodeSelector, LagartoDOMBuilder}
import example.model.AmazonProduct
import scala.collection.JavaConversions._
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.Future
import play.api.libs.json._
import example.utils._
import example.producer._
object AmazonPageParser {
private val topicName = "amazonRatingsTopic"
private val producer = Producer[String](topicName)
def parse(productId: String): Future[AmazonProduct] = {
val url = s"http://www.amazon.com/dp/$productId"
HttpClient.fetchUrl(url) map {
httpResponse =>
if (httpResponse.getStatusCode == 200) {
val body = httpResponse.getResponseBody
val domBuilder = new LagartoDOMBuilder()
val doc = domBuilder.parse(body)
val responseUrl = httpResponse.getUri.toString
val nodeSelector = new NodeSelector(doc)
val title = nodeSelector.select("span#productTitle").head.getTextContent
val img = nodeSelector.select("div#main-image-container img").head.getAttribute("src")
val description = nodeSelector.select("div#feature-bullets").headOption.map(_.getHtml).mkString
val amazonProduct = AmazonProduct(productId, title, responseUrl, img, description)
println("amazonProduct is " + amazonProduct.toString)
amazonProduct
} else {
println("An error happened!")
throw new RuntimeException(s"Invalid url $url")
}
}//map
}//parse method
def main(args: Array[String]): Unit = {
//Scala Puzzlers...
AmazonPageParser.parse("0981531679").onSuccess { case amazonProduct =>
implicit val amazonFormat = Json.format[AmazonProduct]
producer.send(Json.toJson(amazonProduct).toString)
println("amazon product sent to kafka cluster..." + amazonProduct.toString)
}
}
}
file example.model.Models
package example.model
import play.api.libs.json.Json
import reactivemongo.bson.Macros
case class AmazonProduct(itemId: String, title: String, url: String, img: String, description: String)
case class AmazonRating(userId: String, productId: String, rating: Double)
case class AmazonProductAndRating(product: AmazonProduct, rating: AmazonRating)
// For MongoDB
object AmazonRating {
implicit val amazonRatingHandler = Macros.handler[AmazonRating]
implicit val amazonRatingFormat = Json.format[AmazonRating]
}
file example.utils.AmazonPageParser
The compiler returns me this error:
[error] /Users/aironman/my-recommendation-spark-engine/src/main/scala/example/producer/AmazonProducerExample.scala:25: No Json serializer found for type scala.concurrent.Future[example.model.AmazonProduct]. Try to implement an implicit Writes or Format for this type.
[error] producer.send(Json.toJson(amazonProduct).toString)
[error] ^
I have readed this post with most votes but it does not work for me.
Can anybody help me?
Writes[T] produces Json. You can't produce it directly from Future without blocking.
However you can add "callback" to this future, like this:
amazonPageParser.parse(productId).onSuccess { case amazonProduct =>
producer.send(Json.toJson(amazonProduct).toString)
}
Or with other Future methods, like map or foreach.
I know how to do json parsing using play json library for play application. For example I have following code:
class PersonController extends Controller {
case class Person(age: Int, name: String)
implicit val personReads = Json.reads[Person]
implicit val personWrites = Json.writes[Person]
def create = Action(parse.json) { implicit request =>
rs.body.validate[Person] match {
case s: JsSuccess => ...
case e: JsError => ...
}
}
}
How should I write the code like Reads and Writes using Argonaut instead of Play json?
The docs for argonaut are fairly comprehensive in this regard. You need to generate a Codec with
implicit def PersonCodecJson: CodecJson[Person] =
casecodec2(Person.apply, Person.unapply)("age", "name")
And then use decodeValidation to get a Validation object back.
val result2: Validation[String, Person] =
Parse.decodeValidation[Person](json)
May be this is what you are looking for?
class PersonController extends Controller {
case class Person(age: Int, name: String)
implicit val PersonCodecJson: CodecJson[Person] =
casecodec2(Person.apply, Person.unapply)("age", "name")
val argonautParser = new BodyParser[Person] {
override def apply(v1: RequestHeader): Iteratee[Array[Byte], Either[Result, Person]] =
}
def argonautParser[T]()(implicit decodeJson: DecodeJson[T]) = parse.text map { body =>
Parse.decodeOption[T](body)
}
def create = Action(argonautParser) { implicit request =>
Ok(request.body.name) //request.body is the decoded value of type Person
}
}
Actually, you need a body parser that parses the request body using argonaut instead of play json. Hope this helps.
Im having some problems returning an UUID in a JSON with my application in Scala and Spray.
When the entity User(id: UUID, name: String) is parsed to JSON I received:
{
"id": {
"mostSigBits": 1310448748437770800,
"leastSigBits": -7019414172579620000
},
"name": "Sharekhan"
}
I would like to receive the uuid in a String format. Something like:
{
"id": "122fa631-92fd-11e2-9e96-0800200c9a63",
"name": "Sharekhan"
}
I defined the UUID format and the Read is executed when I parse from JSON to User but the Write isn't used in the inverse order (User -> Json)
implicit object UuidJsonFormat extends RootJsonFormat[UUID] {
def write(x: UUID) = JsString(x.toString) //Never execute this line
def read(value: JsValue) = value match {
case JsString(x) => UUID.fromString(x)
case x => deserializationError("Expected UUID as JsString, but got " + x)
}
}
Is any way to do this o should I convert the UUID into a String in the User entity?
Any help will be appreciated,
Thanks.
Make sure to have the implicit format for the Uuid before the user format that uses it and it should work:
object UserJsonProtocol extends DefaultJsonProtocol {
implicit object UuidJsonFormat extends RootJsonFormat[UUID] {
def write(x: UUID) = JsString(x.toString) //Never execute this line
def read(value: JsValue) = value match {
case JsString(x) => UUID.fromString(x)
case x => deserializationError("Expected UUID as JsString, but got " + x)
}
}
implicit val UserFormat = jsonFormat2(User)
}