Is it possible to invoke jsonStringify() to get a properly formatted JSON string outside a Gatling EL?
I need to convert a Map into its JSON String to calculate a signature.
val scn = scenario("My Scenario")
.exec(buildPayload)
.exec(http("Post")
.post("/api/postSomething")
.asJson
.body(StringBody("${payload.jsonStringify()}"))
def buildPayload: Expression[Session] = session => {
val header = Map(...)
val data = Map(...)
val signature = calculateSignature(JsonStringify(data)) // << is it possible??
val payload = Map(
"header" -> header,
"data" -> data,
"signature" -> signature
)
session.set("payload", payload)
}
def calculateSignature(payload: String): String = {
...
}
Do you see any other approach?
Related
I want to convert variable message which is of type scala.Seq[Scala.Document] to JSON format in following code:
path("getMessages"){
get {
parameters('roomname.as[String]) {
(roomname) =>
try {
val messagesByGroupName = MongoDatabase.collectionForChat.find(equal("groupChatName",roomname)).toFuture()
val messages = Await.result(messagesByGroupName,60.seconds)
println("Messages:"+messages)
complete(messages)
}
catch {
case e:TimeoutException =>
complete("Reading file timeout.")
}
}
}
But it is giving me error on complete(messages) line. It is not accepting message of that type.
I tried to convert it into JSON by using following :
import play.api.libs.json._
object MyJsonProtocol{
implicit object ChatFormat extends Format[Chat] {
def writes(c: Chat) : JsValue = {
val chatSeq = Seq (
"sender" -> JsString(c.sender),
"receiver" -> JsString(c.receiver),
"message" -> JsString(c.message),
"groupChatName" -> JsString(c.groupChatName),
)
JsObject(chatSeq)
}
def reads(value: JsValue) = {
JsSuccess(Chat("","","",""))
}
}
}
But it is not working.
My Chat.scala class is as follows:
import play.api.libs.json.{Json, Reads, Writes}
class Chat(var sender:String,var receiver:String,var message:String, var groupChatName:String){
def setSenderName(senderName:String) = {
sender = senderName
}
def setReceiverName(receiverName:String) = {
receiver = receiverName
}
def setMessage(getMessage:String) = {
message = getMessage
}
def setGroupChatName(chatName:String) = {
groupChatName = chatName
}
}
object Chat {
def apply(sender: String, receiver: String, message: String, groupname: String): Chat
= new Chat(sender, receiver, message,groupname)
def unapply(arg: Chat): Option[(String, String, String,String)] = ???
implicit val requestReads: Reads[Chat] = Json.reads[Chat]
implicit val requestWrites: Writes[Chat] = Json.writes[Chat]
}
I am also not able to figure out what to write in unapply method.
I am new to scala and akka.
EDIT:
My MongoDatabase.scala which has collection is as follows:
object MongoDatabase {
val chatCodecProvider = Macros.createCodecProvider[Chat]()
val codecRegistry = CodecRegistries.fromRegistries(
CodecRegistries.fromProviders(chatCodecProvider),
DEFAULT_CODEC_REGISTRY
)
implicit val system = ActorSystem("Scala_jwt-App")
implicit val executor: ExecutionContext = system.dispatcher
val mongoClient: MongoClient = MongoClient()
val databaseName = sys.env("database_name")
// Getting mongodb database
val database: MongoDatabase = mongoClient.getDatabase(databaseName).withCodecRegistry(codecRegistry)
val registrationCollection = sys.env("register_collection_name")
val chatCollection = sys.env("chat_collection")
// Getting mongodb collection
val collectionForUserRegistration: MongoCollection[Document] = database.getCollection(registrationCollection)
collectionForUserRegistration.drop()
val collectionForChat: MongoCollection[Document] = database.getCollection(chatCollection)
collectionForChat.drop()
}
And if try to change val collectionForChat: MongoCollection[Document] = database.getCollection(chatCollection)
to
val collectionForChat: MongoCollection[Chat] = database.getCollection[Chat](chatCollection)
then I get error on in saveChatMessage() method below:
def saveChatMessage(sendMessageRequest: Chat) : String = {
val senderToReceiverMessage : Document = Document(
"sender" -> sendMessageRequest.sender,
"receiver" -> sendMessageRequest.receiver,
"message" -> sendMessageRequest.message,
"groupChatName" -> sendMessageRequest.groupChatName)
val chatAddedFuture = MongoDatabase.collectionForChat.insertOne(senderToReceiverMessage).toFuture()
Await.result(chatAddedFuture,60.seconds)
"Message sent"
}
on val chatAddedFuture = MongoDatabase.collectionForChat.insertOne(senderToReceiverMessage).toFuture() this line since it accepts data of type Seq[Document] and I am trying to add data of type Seq[Chat]
I am going to assume that MongoDatabase.collectionForChat.find(equal("groupChatName",roomname)) returns either Seq[Chat], or Chat. Both of them are the same for play.
You have 2 options:
Adding the default format on the companion object:
object Chat {
implicit val format: Format[Chat] = Json.format[Chat]
}
In this case you can delete the object MyJsonProtocol which is not used.
In case you want to keep your own serializers(i.e. MyJsonProtocol), you need to rename MyJsonProtocol into Chat. This way the complete route will be able to find the implicit Format.
create case class for the message object you want to send
for example:
case class MyMessage(sender: String, receiver: String, message: String, groupChatName: String)
You should create Format for the type of case class
implicit val MessageTypeFormat = Json.format[MyMessage]
if complete should get JSON type - then call complete myMessage when myMessage is an instance of MyMessage.
complete(Json.toJson(myMessage))
I am new to Scala. I want to parse JSON data in scala store to database table.
My GET method looks like this (Please ignore the permissions):
def Classes = withAuth { username =>
implicit request =>
User.access(username, User.ReadXData).map { user =>
implicit val writer = new Writes[Class] {
def writes(entry: Class): JsValue = Json.obj(
"id" -> entry.id,
"name" -> entry.name
)
}
val classes = (Class.allAccessible(user))
Ok(Json.obj("success" -> true, "classes" -> classes))
}.getOrElse(Forbidden(Application.apiMessage("Not authorised"))) }
This GET method returns the json below:
"success":true,"schools":[{"id":93,"name":"Happy unniversity",}]}
I'm currently rendering the JSOn in a datatables js (editor) grid - with success
HOWEVER, I'm unable to parse and POST the JSON and store it to the database (mysql) table.
Thank you for your guidance!
Looks you are using play-json.
For class User
import play.api.libs.json.Json
final case class User(id: String, name: String)
object User {
implicit val userFormat = Json.format[User]
}
object UserJson {
def main(args: Array[String]): Unit = {
val user = User("11", "Peter")
val json = Json.toJson(user).toString()
println("json ===> " + json)
val user2 = Json.parse(json).as[User]
println("name ===> " + user2.name)
}
}
I definitely recommend this lib: "de.heikoseeberger" %% "akka-http-jackson" % "1.27.0" for akka-http.
I'm consuming a Kafka topic inside a spark streaming program like this:
import ...
object KafkaStreaming {
def main(args: Array[String]) {
val conf = new SparkConf().setAppName("KafkaStreaming").setMaster("local[*]")
val sc = new SparkContext(conf)
val ssc = new StreamingContext(sc, Seconds(10))
val kafkaConf = Map(
...
)
val messages = KafkaUtils.createDirectStream[String, String](
ssc,
LocationStrategies.PreferConsistent,
ConsumerStrategies.Subscribe[String, String](Seq("topic"), kafkaConf)
)
val lines: DStream[String] = messages.map(_.value)
val line: DStream[String] = lines.flatMap(_.split("\n"))
process(line)
ssc.start()
ssc.awaitTermination()
}
def process(line: DStream[String]): Unit =
{
// here is where I want to convert the DStream to JSON
var json: Option[Any] = JSON.parseFull(line) // <--
println(json.getOrElse("json is NULL"))
if(json.isEmpty == false) {
println("NOT FALSE")
var map = json.get.asInstanceOf[Map[String, Any]]
// use every member of JSON document to access the value
map.get("any json element").toString
// do some other manipulation
}
}
}
Inside the process function I want to manipulate each line of string to extract a JSON object out of it and perform further processing and persisting. How can I do it?
Instead of taking a DStream[String], use can use DStream.map and then foreachRDD:
def process(line: String): Unit = ???
And then:
messages
.map(_.value)
.flatMap(_.split("\n"))
.map(process)
.foreachRDD { rdd =>
rdd.foreachPartition { itr =>
// Do stuff with `Iterator[String]` after JSON transformation
}
}
Reading headers from JSON file and setting as tuples for Header in API Call using Play in SCALA
I have Json file which have headers and body like this:
File name: test.json
{
"request": {
"headers" : {
"Provider":"1122321",
"Authorization":"44444444",
"Agent":"Web",
"Content-Type":"application/json"
}
"body":{
}
}
}
Now what I am trying to do is, reading Body and Headers from Json file and using later in making a call to API, I am using play WS and Play JSON (2.5.x) for this, but I am unable to get and set headers:
Below is my code,currently I have hard-coded headers, but I need to get and set from JSON, one way I tried is to add headers in List of tuple and use, but I am unable to figure our how I can get headers from JSON file and convert them to list of tuple
val file = new File("E:\\test.json")
val fileIn = new FileInputStream(file)
val json: JsValue = Json.parse(fileIn)
val jsonbody = (json \ "data" \ "request" \"body").get
val jsonheader = (json \ "data" \ "request" \"headers").get
println(jsonheader)
// {"Provider":"1122321","Authorization":"44444444","Agent":"Web","Content-Type":"application/json"}
var url: String = "http://test.com/api/test"
// wsClient.url(url2).withHeaders("Provider"->"1122321", "Authorization" ->` "44444444", "Agent" -> "Web", "Content-Type" -> "application/json", "ClientVersion" -> "3").post(jsonbody).map { response =>
val resbody1: String = response.body
val resstatus1: String = response.statusText
}
You need to cast your JSON to JsObject instead of JsValue
As cchantep said, it is easier to do with a case class :
case class Request(headers:JsObject, body:JsObject)
object Request {
implicit lazy val format = Json.format[Request]
}
And inside your code you can map your Json like this :
val file = new File("E:\\test.json")
val fileIn = new FileInputStream(file)
val json: JsValue = Json.parse(fileIn)
val req = (json \ "request").as[Request]
The list of tuples is now accessible with the 'fields' attribute :
val tuples:Seq[(String, JsValue)] = req.fields
val headers:Map[String, String] = tuples.map{ header =>
header._1 -> header._2.as[String]
}.toMap
I was able to achieve what I was trying for, but I know it's not an efficient way, if anyone can help me improve this will be much appreciated:
val file = new File("E:\\test.json")
val fileIn = new FileInputStream(file)
val json: JsValue = Json.parse(fileIn)
val jsonbody = (json \ "data" \ "request" \"body").get
val jsonheader = (json \ "data" \ "request" \"headers").get
println(jsonheader)
// {"Provider":"1122321","Authorization":"44444444","Agent":"Web","Content-Type":"application/json"}
val jsonheaderlist = jsonheader.toString().substring(1, jsonfile.headersval.toString().length - 1)
.split(",")
.map(_.split(":"))
.map { case Array(k, v) => (k.substring(1, k.length-1), v.substring(1, v.length-1))}
.toMap.toList
var url: String = "http://test.com/api/test"
var req = wsClient.url(url)
jsonheaderlist.foreach{case(value)=>
req = req.withHeaders(value)
}
req = req.post(jsonbody).map { response =>
val resbody1: String = response.body
val resstatus1: String = response.statusText
}
var jsonElements = List[String]()
val params = Map("host"->host)
for((key,value)<-mapSql){
val map = Map("metric"->key,"timestamp"->new Date().getTime,"value"->value,"tags"->params)
jsonElements=JsonUtility.toJSONString(map) :: jsonElements
}
val entity = new StringEntity(JsonUtility.toJSONString(jsonElements))
println("json elements final list is "+jsonElements)
println("json elements final JSON Obj is "+JsonUtility.toJSONString(jsonElements))
entity.setContentType(new BasicHeader("Content-Type", "application/json"))
val postRequest: HttpPost = new HttpPost(putURL)
postRequest.setEntity(entity)
val postResponse: CloseableHttpResponse = httpclient.execute(postRequest)
I basically need to add values to a list and then send them together in a JSON Array.
However this is introducing unnecessary escape characters "/" in the output which is rendering the post request useless and I am getting an error to the API hit. the following is the response :
json elements final list is List({"metric":"replicationLag","timestamp":1410179907871,"value":0.0,"tags":{"host":"tg-em-db01.nm.xxxx.com"}}, {"metric":"status","timestamp":1410179907824,"value":1,"tags":{"host":"tg-em-db01.nm.xxxxx.com"}})
json elements final JSON Obj is ["{\"metric\":\"replicationLag\",\"timestamp\":1410179907871,\"value\":0.0,\"tags\":{\"host\":\"tg-em-db01.nm.xxxx.com\"}}","{\"metric\":\"status\",\"timestamp\":1410179907824,\"value\":1,\"tags\":{\"host\":\"tg-em-db01.nm.xxxxx.com\"}}"]
I can replace and remove all the escape characters by the replaceAll function but I do not want to do that. is there a better way to append objects to an already existing JSON object and then change it to an array ( which i can easily do by new JsonArray(List(JsonObj)) ) so that i dont get any escape characters anywhere.
Something like this :
val params = Map("host"->host)
var map = Map[String,Any]()
for((key,value)<-mapSql){
map ++= Map("metric"->key,"timestamp"->new Date().getTime,"value"->value,"tags"->params)
}
val entity = new StringEntity(JsonUtility.toJSONString(List(map)))
println("json elements final list is "+map)
println("json elements final JSON Obj is "+JsonUtility.toJSONString(List(map)))
is giving me this as an ouput :
json elements final list is Map(metric -> replicationLag, timestamp -> 1410180939983, value -> 0.0, tags -> Map(host -> tg-em-db01.nm.xxxx.com))
json elements final JSON Obj is [{"metric":"replicationLag","timestamp":1410180939983,"value":0.0,"tags":{"host":"tg-em-db01.nm.xxxxx.com"}}]
But I need something like this :
[ {"metric":blah blah} , {"metric":blah blah} ]
Is there a way to append to maps such that the same key values are not clubbed ?
Thanks in advancE!
var jsonElements = List[Map[String, Any]]()
val params = Map("host" -> host)
for ((key, value) <- mapSql) {
val map = Map("metric" -> key, "timestamp" -> new Date().getTime, "value" -> value, "tags" -> params)
jsonElements = map :: jsonElements
}
val entity = new StringEntity(JsonUtility.toJSONString(jsonElements))
entity.setContentType(new BasicHeader("Content-Type", "application/json"))
val postRequest: HttpPost = new HttpPost(putURL)
postRequest.setEntity(entity)
val postResponse: CloseableHttpResponse = httpclient.execute(postRequest)
object JsonUtility {
def toJSONString(obj:Any):String = {
compact(JsonAST.render(decompose(obj)))
}
}