There is a MongoDB instance on my computer with a database. A couple of documents are present in one of the collections, I inserted them manually. There is a Scala application to manipulate the database. There is a case class called Location.
case class Location(_id: Option[ObjectId] = None, name: String) {
var visible: Boolean = false
}
This is the MongoDB configuration in the application.
private val customCodecs = fromProviders(
classOf[Location]
)
private val javaCodecs =
fromCodecs(new LocalDateTimeDateCodec(), new LocalDateDateCodec())
private val codecRegistry =
fromRegistries(customCodecs, javaCodecs,
DEFAULT_CODEC_REGISTRY)
val dbConnection = MongoClient(dbURI)
val database: MongoDatabase = dbConnection.getDatabase(dbName).withCodecRegistry(codecRegistry)
There are more classOf definitions in the customCodecs, just removed them. The dbURI string is retrieved from a config file.
There is a controller endpoint, which returns all Locations from the database. The result is this:
[{"_id":{},"name":"Hungary","visible":false},{"_id":{},"name":"Germany","visible":false},{"_id":{},"name":"France","visible":false},{"_id":{},"name":"Switzerland","visible":false},{"_id":{},"name":"Poland","visible":false}]
The documents in the database have ObjectId, since I entered them manually, and some documents should have the visibility property true. I suspect there is something wrong with the JSON serialization, but cannot figure it out what.
This is the code which queries the collection.
val query = collection.find().toFuture()
Await.result(query, 10.seconds).toList
The service method calls this code and passes the result to the controller.
import org.json4s.native.Serialization.write
val languages = enrollmentService.getAllLanguages
logger.info("GET all languages")
Ok(Json.parse(write[List[Language]](languages)))
I use json4s for JSON serialization / deserialization.
What could be the issue here?
Perhaps you need to include org.json4s.mongo.ObjectIdSerializer?
Related
I have a data structure and I am serializing this data structure to a JSON-File using Jackson Databind. As time progresses, the data model changes, but my application still needs to be able to read an old version of the JSON. My intention is that when an old version of the JSON is encountered, it is implicitly converted to the new version of the data structure in memory and when serialized for the next time, it is stored as the new format version.
For the case of newly added properties, this is simple: I simply specify a default value in Kotlin and Jackson uses that default value if the property is missing in the JSON. However, this case is more complicated: Previously, I had the following data structure:
data class Options(
var applyClahePerColorChannel: Boolean = false
)
Now, I want to make this more general and change the data structure to the following:
data class Options(
var multichannelMode: MultichannelMode = MultichannelMode.ApplyToLuminance
)
enum class MultichannelMode {
ApplyToLuminance, ApplyToAllColorsSeparately
}
Now, when reading an old version of the JSON, applyClahePerColorChannel == false should implicitly be translated to multichannelMode == ApplyToLuminance and applyClahePerColorChannel == true to multichannelMode == ApplyToAllColorsSeparately.
How can I achieve that in Jackson in a concise way?
Here's a solution I found, but I'm still open to better suggestions.
A simple renamed property can be handled with #JsonAlias like so:
If this data model...
data class Options(
var applyClahePerColorChannel: Boolean = false
)
...changes to...
data class Options(
var applyPerColorChannel: Boolean = false
)
...the old name can be added as an alias like so:
data class Options(
#get:JsonAlias("applyClahePerColorChannel")
var applyPerColorChannel: Boolean = false
)
Jackson will then treat both the same way, but upon serialization, Jackson will use the new name.
However, in my case, I also changed the type of the variable, requiring a custom converter like so:
data class Options(
#get:JsonAlias("applyClahePerColorChannel")
#get:JsonDeserialize(converter = BooleanToMultichannelModeConverter::class)
var multichannelMode: MultichannelMode = MultichannelMode.ApplyToLuminance
)
class BooleanToMultichannelModeConverter : Converter<String, MultichannelMode> {
override fun convert(value: String): MultichannelMode {
return when (value) {
in listOf("true", "True", "TRUE") -> ApplyToAllColorsSeparately
in listOf("false", "False", "FALSE") -> ApplyToLuminance
else -> MultichannelMode.valueOf(value)
}
}
#OptIn(ExperimentalStdlibApi::class)
override fun getInputType(typeFactory: TypeFactory): JavaType = typeFactory
.constructType(String::class.starProjectedType.javaType)
#OptIn(ExperimentalStdlibApi::class)
override fun getOutputType(typeFactory: TypeFactory): JavaType = typeFactory
.constructType(MultichannelMode::class.starProjectedType.javaType)
}
The converter basically tells Jackson to not do any parsing on its own and instead simply hand the unparsed string to the converter class, which will first attempt to parse the string as a boolean and if that fails as an enum value.
I'm calling different APIs, that use the same key name in the JSON file. Depending on the response, there's one field that may be different types.
To be clear:
The key "results" when calling the API nº1 is a JSON object
The key "results" when calling the API nº2 is a JSON array
My code looks like this when using the second API:
data class Result(
#SerializedName("results") var persons:ArrayList<Person> =ArrayList()
)
The question is if there's any way to use the same class, without taking care if it's a JSON array or a JSON object.
I believe you can define results as an instance of com.fasterxml.jackson.databind.JsonNode.
data class Result(
val results: JsonNode
)
Then you can process results based on it's type—whether it is an ArrayNode or an ObjectNode (as both extend JsonNode):
fun processResults(results: JsonNode) = when{
results.isArray -> processArrayNode(results)
else -> processObjectNode(results)
}
private fun processArrayNode(list: JsonNode): *return whatever you need*{
val elements = list
.elements()
.asSequence()
.toList()
val mappedElements = elements.map{
processObjectNode(it)
}
// do whatever you need with the array
}
private fun processObjectNode(person: JsonNode): *return whatever you need*{
//** this will transform the json node into a linkedHashMap where the keys are the json keys and the values are the values (here interpreted as jsonNodes) **/
val fieldsMap = person
.fields()
.asSequence()
.associateBy( {it.key}, {it.value} )
// process whatever you need
}
This is one way to use the same DTO for both API calls. In my opinion, it is not worth the extra work. I would create two DTOs containing the results field, where in one it is an instance of Person, and in the other it is an instance of List<Person>.
Edit: One little upgrade to the above snippet would be to add extension methods to JsonNode:
fun JsonNode.elementsToList(): List<JsonNode> = this
.elements()
.asSequence()
.toList()
fun JsonNode.fieldsToMap(): Map<String, JsonNode> = this
.fields()
.asSequence()
.associateBy({it.key}, {it.value})
You can use ObjectMapper.typeFactory.constructParametricType to handle generic types:
data class Result<T>(
var x:T
)
val om = ObjectMapper()
om.registerModule(KotlinModule())
val parsedList = om.readValue<Result<List<String>>>(
"""{"x":["x1", "x2"]}""",
om.typeFactory.constructParametricType(Result::class.java, List::class.java)
)
println(parsedList)
val parsedMap = om.readValue<Result<Map<String, String>>>(
"""{"x":{"k1": "v1", "k2": "v2"}}""",
om.typeFactory.constructParametricType(Result::class.java, Map::class.java)
)
println(parsedMap)
Gives output:
Result(x=[x1, x2])
Result(x={k1=v1, k2=v2})
i am new to MongoDB and Scala language
i am using scala language to connect mongodb locally
i am using below dependency
//
https://mvnrepository.com/artifact/org.mongodb.scala/mongo-scala-driver
libraryDependencies += "org.mongodb.scala" %% "mongo-scala-driver" %
"4.2.3"
what I tried
object Demo extends App {
val mongoClient: MongoClient = MongoClient("mongodb://127.0.0.1:27017/")
val database: MongoDatabase = mongoClient.getDatabase("DemoDB")
println(database)
val collection: MongoCollection[Document] =database.getCollection("demodata");
val observable = collection.find();
}
the above code returning the data in below format
FindObservable(com.mongodb.reactivestreams.client.internal.FindPublisherImpl#6253c26)
I also tried with
observable.subscribe ( new Observer[Document] {
override def onNext(result: Document): Unit = println(result.toJson())
override def onError(e: Throwable): Unit = println("Failed" + e.getMessage)
override def onComplete(): Unit = println("Completed")
})
i also tried printResult() and printHeadResult() method also but none of the way is working
please help
thanks in advance
Mongo Scala driver works in a non-blocking manner by returning Observables which need to be Subsribed on to consume the published data.
When you are subscribing to the observable like following,
object Demo extends App {
val mongoClient: MongoClient = MongoClient("mongodb://127.0.0.1:27017/")
val database: MongoDatabase = mongoClient.getDatabase("DemoDB")
println(database)
val collection: MongoCollection[Document] = database.getCollection("demodata")
val observable = collection.find()
observable.subscribe ( new Observer[Document] {
override def onNext(result: Document): Unit = println(result.toJson())
override def onError(e: Throwable): Unit = println("Failed" + e.getMessage)
override def onComplete(): Unit = println("Completed")
})
}
Your code does not wait for observable to actually publish anything, it just finishes right after subscribing. Hence you don't get anything.
You can either add a Something like a Thread.sleep(5000) at the end to block and give the obeservable some time to (hopefully finish and) publish the data.
Or, you can add val resultSeq = observable.collect to block and collect all of published data in a single Sequence.
I found this link
it works for printResult() and printHeadResult() method
Printing query results from Mongodb in Scala using mongo-scala-driver
I just saw that there are some libraries for running a Dart web server, like Start.
So I was thinking something like this..
If both client and server code is written in Dart, is it possible to send "Dart objects" via websockets (or normal REST for that matter) so that the type information remains on the other end? Or do I need to serialize/deserialize via JSON or something similar on the way? Or am I over thinking things here?
regards Oskar
You will need to serialize the Dart object somehow. You can try JSON, or you can try the heavy-duty serialization package.
There is no fully automatic JSON serialization for custom Dart classes. You will need to add a custom toJson serializer and create some sort of fromJson constructor.
e.g. if you had a Person class, you could do something like this:
import 'dart:json' as json;
class Person {
String name;
int age;
Person(this.name, this.age);
Person.fromJson(String json) {
Map data = json.parse(json);
name = data['name'];
age = data['age'];
}
Map toJson() {
return {'name': name, 'age': age};
}
}
Note: the fromJson is just a convention. You will need to call it somehow, there is no built-in mechanism to take an arbitrary JSON string and call the right constructors on your custom object.
As mentioned above, the serialization package is more heavy weight, but much more full featured. Here is an example from its docs:
// uses the serialization package
var address = new Address();
address.street = 'N 34th';
address.city = 'Seattle';
var serialization = new Serialization()
..addRuleFor(address);
Map output = serialization.write(address);
and
// uses serialization
var serialization = new Serialization()
..addRuleFor(address,
constructor: "create",
constructorFields: ["number", "street"],
fields: ["city"]);
You can use the 'exportable' package to render your class to JSON or a map in a more declarative fashion.
import 'package:exportable/exportable.dart';
class Product extends Object with Exportable
{
#export String ProductName;
#export num UnitPrice;
#export bool Discontinued;
#export num UnitsInStock;
Product(this.ProductName, this.UnitPrice, this.Discontinued, this.UnitsInStock);
}
Product prod = new Product("First", 1.0, false, 3 );
var json = prod.toJson(); // {"ProductName":"First","UnitPrice":1.0,"Discontinued":false,"UnitsInStock":3}
I'm new to Scala and I'm trying to do my first Play 2.0 application. I'm using:
Scala 2.9.3
Play 2.0.4
SalatWithPlay2 1.1
I have this in my conf/routes file:
PUT /newVenue controllers.Application.createNewVenue
I have this in my Application.scala file:
def createNewVenue = Action(parse.json) { request =>
val newVenue = Venue.fromJSON(request.body.toString)
Venue.insert(newVenue)
Ok("New Venue Created")
}
And this is the code for Venue.scala:
import play.api.Play.current
import com.novus.salat._
import com.novus.salat.global._
import com.novus.salat.annotations._
import com.novus.salat.dao._
import com.mongodb.casbah.Imports._
import se.radley.plugin.salat._
object Venue extends ModelCompanion[Venue, ObjectId] {
val collection = mongoCollection("venues")
val dao = new SalatDAO[Venue, ObjectId](collection = collection) {}
}
case class Venue(
#Key("_id") val venueId:ObjectId,
var playlist:Playlist,
var isPlaying:Boolean = false)
To test if this is working I send a PUT request to localhost:9000/newVenue with this JSON as a body:
{"venueId": 3,"playlist":{"playlistId":2,"currentSongPosition":5},"isPlaying":false}
And then I get this error:
[error] application -
! #6d7oco1hf - Internal server error, for request [PUT /newVenue] ->
play.core.ActionInvoker$$anonfun$receive$1$$anon$1: Execution exception [[NoSuchMethodException: model.Venue$.apply$default$1()]]
at play.core.ActionInvoker$$anonfun$receive$1.apply(Invoker.scala:134) [play_2.9.1.jar:2.0.4]
at play.core.ActionInvoker$$anonfun$receive$1.apply(Invoker.scala:115) [play_2.9.1.jar:2.0.4]
at akka.actor.Actor$class.apply(Actor.scala:318) [akka-actor.jar:2.0.2]
at play.core.ActionInvoker.apply(Invoker.scala:113) [play_2.9.1.jar:2.0.4]
at akka.actor.ActorCell.invoke(ActorCell.scala:626) [akka-actor.jar:2.0.2]
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:197) [akka-actor.jar:2.0.2]
Caused by: java.lang.NoSuchMethodException: model.Venue$.apply$default$1()
at java.lang.Class.getMethod(Class.java:1605) ~[na:1.6.0_37]
at com.novus.salat.ConcreteGrater.defaultArg(Grater.scala:350) ~[salat-core_2.9.1-1.9.1.jar:1.9.1]
at com.novus.salat.ConcreteGrater.safeDefault(Grater.scala:360) ~[salat-core_2.9.1-1.9.1.jar:1.9.1]
at com.novus.salat.ConcreteGrater$$anonfun$6$$anonfun$apply$7.apply(Grater.scala:319) ~[salat-core_2.9.1-1.9.1.jar:1.9.1]
at com.novus.salat.ConcreteGrater$$anonfun$6$$anonfun$apply$7.apply(Grater.scala:319) ~[salat-core_2.9.1-1.9.1.jar:1.9.1]
at scala.Option.orElse(Option.scala:218) ~[scala-library.jar:0.11.3]
I think the line triggering the error is val newVenue = Venue.fromJSON(request.body.toString).
Does anyone have a clue of what's wrong?
I followed the tutorial in SalatWithPlay2 page and also followed some advices in similar problems), but I had no luck so far.
UPDATE:
I found a workaround, although it's not exactly a solution, but maybe it's helpful for somebody else and for understanding what is the actual solution.
If I remove the annotation #Key, the code of the case class Venue looks like this:
case class Venue(
val venueId:ObjectId,
var playlist:Playlist,
var isPlaying:Boolean = false)
And then I get this other error:
[RuntimeException: in: unexpected OID input class='net.liftweb.json.JsonAST$JInt', value='3']
And if instead of using ObjectId I use Long for example, the code looks like this:
case class Venue(
val venueId:Long,
var playlist:Playlist,
var isPlaying:Boolean = false)
And I get no error!
So the NoSuchMethodException apparently is related to the #Key annotation related to the _id. I also tried to rename venueId to _id and the same NoSuchMethodException error appeared. So the question is: why can't I use the #Key annotation to say which of my attributes maps to the object id of the document in the mongo database?
Besides that, regular Integers cannot be automatically converted to ObjectId instances for some other reason.
Thanks!
Just a shot in the dark, but can you try naming your object class to something else? Maybe to VenueCompanion? It seems it is possible the compiler is getting confused on which object to apply to. IE it is not applying to the case class Venue, that you want. I am curious if the exception is still the same afterwards.
So I found what was the problem.
The attributes of the Json sent in the body of the request must match the names of the attributes of the mongodb's document, not the attributes of the Scala object.
So if my class Venue is defined like this:
case class Venue(
#Key("_id") val venueId:ObjectId,
var playlist:Playlist,
var isPlaying:Boolean = false)
The Json I have to send in the PUT request is:
{"_id": 3,"playlist":{"playlistId":2,"currentSongPosition":5},"isPlaying":false}
And NOT:
{"venueId": 3,"playlist":{"playlistId":2,"currentSongPosition":5},"isPlaying":false}