i am new to MongoDB and Scala language
i am using scala language to connect mongodb locally
i am using below dependency
//
https://mvnrepository.com/artifact/org.mongodb.scala/mongo-scala-driver
libraryDependencies += "org.mongodb.scala" %% "mongo-scala-driver" %
"4.2.3"
what I tried
object Demo extends App {
val mongoClient: MongoClient = MongoClient("mongodb://127.0.0.1:27017/")
val database: MongoDatabase = mongoClient.getDatabase("DemoDB")
println(database)
val collection: MongoCollection[Document] =database.getCollection("demodata");
val observable = collection.find();
}
the above code returning the data in below format
FindObservable(com.mongodb.reactivestreams.client.internal.FindPublisherImpl#6253c26)
I also tried with
observable.subscribe ( new Observer[Document] {
override def onNext(result: Document): Unit = println(result.toJson())
override def onError(e: Throwable): Unit = println("Failed" + e.getMessage)
override def onComplete(): Unit = println("Completed")
})
i also tried printResult() and printHeadResult() method also but none of the way is working
please help
thanks in advance
Mongo Scala driver works in a non-blocking manner by returning Observables which need to be Subsribed on to consume the published data.
When you are subscribing to the observable like following,
object Demo extends App {
val mongoClient: MongoClient = MongoClient("mongodb://127.0.0.1:27017/")
val database: MongoDatabase = mongoClient.getDatabase("DemoDB")
println(database)
val collection: MongoCollection[Document] = database.getCollection("demodata")
val observable = collection.find()
observable.subscribe ( new Observer[Document] {
override def onNext(result: Document): Unit = println(result.toJson())
override def onError(e: Throwable): Unit = println("Failed" + e.getMessage)
override def onComplete(): Unit = println("Completed")
})
}
Your code does not wait for observable to actually publish anything, it just finishes right after subscribing. Hence you don't get anything.
You can either add a Something like a Thread.sleep(5000) at the end to block and give the obeservable some time to (hopefully finish and) publish the data.
Or, you can add val resultSeq = observable.collect to block and collect all of published data in a single Sequence.
I found this link
it works for printResult() and printHeadResult() method
Printing query results from Mongodb in Scala using mongo-scala-driver
Related
I am trying to build a minimal example, of how to generate an AXI4Stream interface using Chisel and diplomacy. I am using the diplomatic interface already available in rocket-chip (freechips.rocketchip.amba.axis). I have some experience with Chisel, but I am still trying to learn diplomacy.
Anyway, I've managed to create a small APB example using the answer provided here: IP block generation/testing when using diplomacy. Possible to give dummy node?
Following that, I tried to create a similar, simple AXI Stream example, but I keep getting errors. Concretely, I get the following error:
[error] (Compile / run) java.lang.Exception: Unable to use BundleMap.cloneType on class freechips.rocketchip.amba.axis.AXISBundleBits, probably because class freechips.rocketchip.amba.axis.AXISBundleBits does not have a constructor accepting BundleFields. Consider overriding cloneType() on class freechips.rocketchip.amba.axis.AXISBundleBits
The code:
package chipyard.example
import chisel3._
import chisel3.internal.sourceinfo.SourceInfo
import chisel3.stage.ChiselStage
import freechips.rocketchip.config.{Config, Parameters}
import freechips.rocketchip.amba.axis._
import freechips.rocketchip.diplomacy.{SimpleNodeImp, ValName, SourceNode, NexusNode,
SinkNode, LazyModule, LazyModuleImp, TransferSizes,
SimpleDevice, AddressSet}
class MyAxisController(implicit p: Parameters) extends LazyModule {
val device = new SimpleDevice("my-device", Seq("tutorial,my-device0"))
val axisParams = AXISSlaveParameters.v1(name = "axisSlave", supportsSizes = TransferSizes(8,8))
val axisPortParams = AXISSlavePortParameters.v1(slaves = Seq(axisParams))
val node = AXISSlaveNode(portParams = Seq(axisPortParams))
lazy val module = new LazyModuleImp(this) {
val ins = node.in.unzip._1
val register = RegInit(UInt(8.W), 0.U)
register := register + ins(0).bits.data
}
}
class AXISMaster()(implicit p: Parameters) extends LazyModule {
val axisMasterParams = AXISMasterParameters.v1(
name = "axisMaster", emitsSizes = TransferSizes(8, 8)
)
val axisMasterPortParams = AXISMasterPortParameters.v1(
masters = Seq(axisMasterParams),
beatBytes = Option(8)
)
val node = AXISMasterNode(
portParams = Seq(axisMasterPortParams)
)
lazy val module = new LazyModuleImp(this) {
//The dontTouch here preserves the interface so logic is generated
dontTouch(node.out.head._1)
}
}
class MyAxisWrapper()(implicit p: Parameters) extends LazyModule {
val master = LazyModule(new AXISMaster)
val slave = LazyModule(new MyAxisController()(Parameters.empty))
slave.node := master.node
lazy val module = new LazyModuleImp(this) {
//nothing???
}
}
and Main.scala:
package chipyard.example
import chisel3._
import freechips.rocketchip.config.Parameters
import freechips.rocketchip.diplomacy._
import java.io.File
import java.io.FileWriter
/**
* An object extending App to generate the Verilog code.
*/
object Main {
def main(args: Array[String]): Unit = {
//(new chisel3.stage.ChiselStage).execute(args, Seq(ChiselGeneratorAnnotation(() => LazyModule(new MyWrapper()(Parameters.empty)).module)))
val verilog = (new chisel3.stage.ChiselStage).emitVerilog(
LazyModule(new MyAxisWrapper()(Parameters.empty)).module
)
//println(s"```verilog\n$verilog```")
val fileWriter = new FileWriter(new File("./gen/gen.v"))
fileWriter.write(verilog)
fileWriter.close()
}
}
The code is also available at https://github.com/jurevreca12/temp_dspblock_example/tree/axistream2/scala/main.
My question is. Why do I get this error? Or am I doing something wrong in the first place, and is there an easier way to create an AXIStream module?
I appreciate any feedback.
This looks to be an issue with Rocket-Chip's changes to bump to Chisel 3.5. During those changes, AXISBundleBits had its cloneType removed even though it extends off BundleMap (and therefore requires cloneType due to extending off Record).
I don't have all the details of cloneType at this time, but essentially:
Records require cloneType
Bundles used to require cloneType, but since the compiler plugin was implemented, as of 3.5 they no longer require cloneType.
BundleMap is a confusing case because it is a custom Bundle type extending directly off Record and isn't of type Bundle. Therefore, it shouldn't have had its cloneType method removed during the 3.5 Chisel bump and that will need to be added back for AXIS in RC's main branch to start working again.
Edit: the cloneType exception issue is now fixed for 3.5 on the main branch :)
I'm using Jackson in my Scala/Spark program and I've distilled my issue to the simple example
below. My problem is that when my case class has the Option[Int] field (age) set to None
I see reasonable deserialization output (that is: a struct with empty=true). However, when
age is defined, i.e., set to some Int like Some(99), I never see the integer value in the
deserialization output .
Given :
import com.fasterxml.jackson.databind.ObjectMapper
import java.io.ByteArrayOutputStream
import scala.beans.BeanProperty
case class Dog(#BeanProperty name: String, #BeanProperty age: Option[Integer])
object OtherTest extends App {
jsonOut(Dog("rex", None))
jsonOut(Dog("mex", Some(99)))
private def jsonOut(dog: Dog) = {
val mapper = new ObjectMapper();
val stream = new ByteArrayOutputStream();
mapper.writeValue(stream, dog);
System.out.println("result:" + stream.toString());
}
}
My output is as shown below. Any hints/help greatly appreciated !
result:{"name":"rex","age":{"empty":true,"defined":false}}
result:{"name":"mex","age":{"empty":false,"defined":true}}
Update after the Helpful Answer
Here are the dependencies that worked for me:
implementation 'org.scala-lang:scala-library:2.12.2'
implementation "org.apache.spark:spark-sql_2.12:3.1.2"
implementation "org.apache.spark:spark-sql-kafka-0-10_2.12:3.1.2"
implementation "org.apache.spark:spark-avro_2.12:3.1.2"
implementation 'com.fasterxml.jackson.module:jackson-module-scala_2.12:2.10.0'
Here is the updated code (with frequent flyer bonus - round trip example):
private def jsonOut(dog: Dog) = {
val mapper = new ObjectMapper()
mapper.registerModule(DefaultScalaModule)
val stream = new ByteArrayOutputStream();
mapper.writeValue(stream, dog);
val serialized = stream.toString()
System.out.println("result:" + serialized);
// verify we can read the serialized thing back to case class:
val recovered = mapper.readValue(serialized, classOf[Dog])
System.out.println("here is what we read back:" + recovered);
}
Here is the resultant output (as expected now ;^) ->
> Task :OtherTest.main()
result:{"name":"rex","age":null}
here is what we read back:Dog(rex,None)
result:{"name":"mex","age":99}
here is what we read back:Dog(mex,Some(99))
You need to add the Jackson module for Scala to make it work with standard Scala data types.
Add this module as your dependency: https://github.com/FasterXML/jackson-module-scala
Follow the readme on how to initialize your ObjectMapper with this module.
I'm a newbie when it comes to Scala and right now trying to learn Slick. I have a database set up much like in the Slick tutorial and querying it with
val users:TableQuery[user] = TableQuery[user]
How do I output users -value in to JSON from Play framework? I have tried defining case class for user and making reader+writer for userFormat = Json.format[user] but I have no clue where I'm going wrong or anything. I've done similar kind of stuff with Spray.IO and it seemed simple and straightforward there, but I want to learn how to do it with Play and Slick and I am completely lost :(
Here is my Tables.scala:
package assets
import org.joda.time.DateTime
import play.api.libs.json._
import scala.slick.driver.MySQLDriver.simple._
import com.github.tototoshi.slick.MySQLJodaSupport._
import scala.slick.lifted.{ProvenShape}
case class user
(
id:Int,
textId:String,
sessionId:String,
stamp:DateTime
)
class users(tag: Tag) extends Table[(Int, String, String, DateTime)](tag, "USER") {
implicit object userFormat extends Format[user] {
def writes(a: user): JsValue = {
Json.obj(
"id" -> JsNumber(a.id),
"textId" -> JsString(a.textId),
"sessionId" -> JsString(a.sessionId),
"stamp" -> JsString(a.stamp.toString())
)
}
def reads(json: JsValue): user = user(
(json \ "id").as[Int],
(json \ "textId").as[String],
(json \ "sessionId").as[String],
(json \ "stamp").as[DateTime]
)
}
def id: Column[Int] = column[Int]("id", O.PrimaryKey)
def textId: Column[String] = column[String]("textId")
def sessionId: Column[String] = column[String]("sessionId")
def stamp: Column[DateTime] = column[DateTime]("stamp")
def * : ProvenShape[(Int, String, String, DateTime)] = (id,textId,sessionId,stamp)
}
First of all, please add some context to your question as it's very hard to see what is not working (e.g. do you get compilation errors? Do you get exceptions?).
With respect to the question content, you should think of getting data from slick and translating it into JSON as two totally independent steps; in fact, there is nothing in slick that will help you in making JSON out of your case classes.
Given that, you will need to run your slick queries and get case class instances for your types out of them, e.g. you should get a Seq[user]. To convert that to JSON you should invoke Json.toJson(myUserSeq). There is one thing that will probably get in your way for what I can see in your code: to have the Format correctly in scope you should put it in the companion object of your case class, e.g.:
case class User(...)
object User {
implicit object format extends Format[User] { ... }
}
As a sideline, please be aware of the language's coding standards if you want people to help you and your code to be readable (e.g. having lower-case class names in your code makes it very hard to follow).
I'm creating an application in Scala using Play 2.2. I'm using play-slick 0.5.0.8 as my MySQL DB connector. I have the following application controller:
package controllers
import models._
import models.database._
import play.api._
import play.api.mvc._
import play.api.Play.current
import play.api.db.slick._
object Application extends Controller {
// WORKS:
def test = DBAction {
implicit session => Ok(views.html.test(Cameras.findById(1)))
}
// DOES NOT WORK:
def photo = Action {
val p = PhotoFetcher.fetchRandomDisplayPhoto(someParametersBlah))
Ok(views.html.photo(p))
}
}
As you can see, the test DBAction works, and it's able to fetch a photo from the DB just fine. Unfortunately, the photo Action does not work.
My PhotoFetcher.fetchRandomDisplayPhoto(blah) does a bunch of different things. Buried inside of it is a call to Cameras.findById(blah), which should return a Camera object (which works in the test DBAction). However, with this configuration I get the following error:
could not find implicit value for parameter s: slick.driver.MySQLDriver.simple.Session
I have tried making the photo Action into a DBAction, like so:
def photo = DBAction {
implicit session => {
val p = PhotoFetcher.fetchRandomDisplayPhoto(someParametersBlah))
Ok(views.html.photo(p))
}
}
But that just results in the same missing session error. It's like PhotoFetcher doesn't know about the implicit session.
The other thing I've tried is importing slick.session.Database.threadLocalSession in my PhotoFetcher, but that only results in the following error:
SQLException: No implicit session available; threadLocalSession can only be used within a withSession block
If it's any help, this is a simplified version of my Cameras object:
package models.database
import models.Format.Format
import scala.slick.driver.MySQLDriver.simple._
case class Camera(id: Long,
otherStuff: String)
trait CamerasComponent {
val Cameras: Cameras
class Cameras extends Table[Camera]("cameras") {
def id = column[Long]("id", O.PrimaryKey, O.AutoInc)
def otherStuff = column[String]("otherStuff", O.NotNull)
def * = id ~ otherStuff <> (Camera.apply _, Camera.unapply _)
val byId = createFinderBy(_.id)
val byOtherStuff = createFinderBy(_.otherStuff)
}
}
object Cameras extends DAO {
def insert(camera: Camera)(implicit s: Session) { Cameras.insert(camera) }
def findById(id: Long)(implicit s: Session): Option[Camera] = Cameras.byId(id).firstOption
def findByOtherStuff(otherStuff: String)(implicit s: Session): Option[Camera] = Cameras.byOtherStuff(model).firstOption
}
So, it seems as if I've gotten crossed-up somewhere. Right now it's only possible for me to access my DAO objects directly from a Controller DBAction, and not from inside of some different class. Any help would be appreciated. Thanks!
Does your definition of PhotoFetcher.fetchRandomDisplayPhoto.fetchRandomDisplayPhoto take an implicit session?
// PhotoFetcher
def fetchRandomDisplayPhoto(args: Blah*)(implicit s: Session) = {
// ...
val maybeCam = Cameras.findById(blah) // <- sees the implicit session
// ...
}
Or are you relying on a threadLocalsession in PhotoFetcher? (no implicit session argument for fetchRandomDisplayPhoto)?
While Slick's threadLocalSession is handy for quickly trying out stuff, it can lead to confusion and loss of clarity later on. It's best to just use explicit (implicit s: Session) parameter lists for all methods that call your Slick models. This also plays
well with DBAction, letting the framework manage sessions.
The downside is you have to have (implicit s: Session) on all your methods - there
are workarounds like this:
https://github.com/freekh/play-slick/issues/20
Scala isn't verbose and is very amenable to refactoring - so I'd recommend thinking
about crossing that bridge when you come to it, and use DBAction for all actions
that do database stuff; give all methods that call your database models an
implicit session, and see how much that mileage that gives you.
I'm trying to render some response like this
def doAjax = Action { request =>
object MyResult {
val resultCode = 0
val resultTextMessage = "sss"
}
Ok(Json(MyResult)) // It's not working anymore - not compiling in v2.0!
}
but how to map my object (MyResult) to JSON with Play 2.0?
In Play 1.0 with scala module I did successfully the following:
def dosomeaj = {
object MyResult{
val resultCode = 0
val resultTextMessage = "sss"
}
Json(MyResult) // It's working in 1.0
}
EDIT2
New Wiki link for v2.1. The old link below is not working anymore.
EDIT
We'll all be happy to read the new Wiki entry for this point. Check this out
PREVIOUS
Here is the comment from the community about the state of Json support in play 2.0.
link to Post
They are moving from Jackson to a philosophy inspired by SJSON that offers more control on the un/marshalling, that brings facilities to manage them, w/o the overhead of Reflection (which I agree with them is a pain for performance and is fragile against Class changes...)
So here is what you can read on the post:
case class Blah(blah: String)
// if you want to directly serialize/deserialize, you need to write yourself a formatter right now
implicit object BlahFormat extends Format[Blah] {
def reads(json: JsValue): Blah = Blah((json \ "blah").as[String])
def writes(p: Blah): JsValue = JsObject(List("blah" -> JsString(p.blah)))
}
def act = Action { implicit request =>
// to get a Blah object from request content
val blah = Json.parse(request.body.asText.get).as[Blah]
// to return Blah as application/json, you just have to convert your Blah to a JsValue and give it to Ok()
Ok(toJson(blah))
}
In the second link (SJSON), I propose you to pay special attention to the generic formatting possible by using case class and their deconstruction method (unapply).
Play 2 comes with Jerkson
case class Blah(blah: String)
import com.codahale.jerksHon.Json._
def act = Action { implicit request =>
Ok(generate(parse[Blah](request.body.asText.get))).as("application/json")
}
This code will deserialize and reserialize the json.
For more information https://github.com/codahale/jerkson
I've found this solution in Play integration tests right now.
It's require to define in app/models/MyResult2.scala with this content:
case class MyResult2(resultCode: Int, resultTextMessage: String)
object Protocol {
implicit object MyResult2Format extends Format[MyResult2] {
def writes(o: MyResult2): JsValue = JsObject(
List("resultCode" -> JsNumber(o.resultCode),
"resultTextMessage" -> JsString(o.resultTextMessage)
)
)
def reads(json: JsValue): MyResult2 = MyResult2(
(json \ "resultCode").as[Int],
(json \ "resultTextMessage").as[String]
)
}
}
And after this you can use it in your controller class like this:
import play.api._
import play.api.mvc._
import play.api.libs.json._
import models._
import models.Protocol._
object Application extends Controller {
def doAjax = Action { request =>
Ok(toJson(MyResult2(0, "Ney")))
}
}
It's now required some manual static marshalling code.
You can use the 'play.api.mvc.as'
def demo = Action {
....
Ok(jsonString).as("text/json")
}