I am a complete newbie in scala, all I want to do here is have a JSON file simply parsed using scala and print the same. I am facing an error on compiling which I am not able to solve. Thanks in advance for any help on this. PFB the scala code, SBT file, JSON file and error:
json_example.scala
import scala.io.Source
import play.api.libs.json._
import play.api.libs.json._
object Test extends App {
val line :String = "Foo";
val filename = "users.json"
for (line <- Source.fromFile(filename).getLines().mkString) {
println(line);
val json: JsValue = Json.parse(line);
}
}
JSON File (users.json)
{"users":[
{"ID":"1","firstName":"John", "lastName":"Doe"},
{"ID":"2","firstName":"Anna", "lastName":"Smith"},
{"ID":"3","firstName":"Peter", "lastName":"Jones"}
{"ID":"1","firstName":"Stewie", "lastName":"Doe"},
{"ID":"2","firstName":"Chris", "lastName":"Smith"},
{"ID":"3","firstName":"Louis", "lastName":"Jones"}
{"ID":"2","firstName":"Brian", "lastName":"Smith"},
{"ID":"3","firstName":"Meg", "lastName":"Jones"}
]}
SBT file (simple.sbt)
lazy val root = (project in file(".")).
settings(
name := "JSON_GRAPHX",
version := "1.0",
scalaVersion := "2.10.1",
libraryDependencies ++= Seq("com.github.scala-incubator.io" %% "scala-io-file" % "0.4.2",
"com.typesafe.play" %% "play-json" % "2.3.4"),
resolvers += "Typesafe Repo" at "http://repo.typesafe.com/typesafe/releases/"
)
Error
[info] Set current project to JSON_GRAPHX (in build file:/F:/Graphx_app/JSON_GRAPHX/)
[info] Compiling 1 Scala source to F:\Graphx_app\JSON_GRAPHX\target\scala-2.10\classes...
[error] F:\Graphx_app\JSON_GRAPHX\json_example.scala:15: overloaded method value parse with alternatives:
[error] (input: Array[Byte])play.api.libs.json.JsValue <and>
[error] (input: String)play.api.libs.json.JsValue
[error] cannot be applied to (Char)
[error] val json: JsValue = Json.parse(line);
[error] ^
[error] one error found
[error] (compile:compileIncremental) Compilation failed
[error] Total time: 4 s, completed Aug 18, 2016 8:51:22 PM
To build the JSON object, you can call the parse method directly in the String returned by mkString. Something like:
val json = Json.parse(Source.fromFile(filename).getLines().mkString)
By doing:
for (line <- Source.fromFile(filename).getLines().mkString)
You are actually iterating over all characters of the Json String - that's why you get the error that the parse method cannot be applied to a Char.
Once you have the JSON object, you can print it minified:
println(Json.stringify(json))
Or you can print it in a readable format:
println(Json.prettyPrint(son))
Related
I want to send a Http error response with a message in JSON format in the body. I am having trouble using the PredefinedToResponseMarshallers.
I saw an implementation in the Akka docs but I tried a similar thing and it throws a compilation error.
import argonaut._, Argonaut._
import akka.http.scaladsl.marshalling.Marshal
import akka.http.scaladsl.server.Directives._
import akka.http.scaladsl.model.HttpResponse
import akka.http.scaladsl.model.headers._
import akka.http.scaladsl.model.StatusCodes._
import akka.http.scaladsl.marshalling.{ Marshaller, ToResponseMarshaller }
trait Sample extends Marshallers with Directives {
def login(user: Login): CRIX[HttpResponse] = {
for {
verification ← verify(user)
resp = if (verification) {
HttpResponse(NoContent, headers = Seq(
.........
))
}//below is my http Error response
else Marshal(401 → "It is an Unauthorized Request".asJson).to[HttpResponse]
} yield resp
}
}
It gives this compilation error :
Sample.scala:164: type mismatch;
[error] found : Object
[error] required: akka.http.scaladsl.model.HttpResponse
[error] } yield resp
[error] ^
[error] one error found
[error] (http/compile:compileIncremental) Compilation failed
I have just started Akka Http so forgive me if it is trivially easy.
TL;DR: I want(examples) to learn how to use ToResponseMarshallers in Akka Http.
Method to[HttpResponse] of negative condition takes on Future[HttpResponse]. At the same time positive condition returns HttpResponse.
Try something like (I assume verify takes on Future[T]):
for {
verification <- verify(user)
resp <- if (verification)
Future.successful(HttpResponse(NoContent, headers = Seq(.........)) )
else
Marshal(401 → "It is an Unauthorized Request".asJson).to[HttpResponse]
} yield resp
I am trying to create a JSON String from a Scala Object as described here.
I have the following code:
import scala.collection.mutable._
import net.liftweb.json._
import net.liftweb.json.Serialization.write
case class Person(name: String, address: Address)
case class Address(city: String, state: String)
object LiftJsonTest extends App {
val p = Person("Alvin Alexander", Address("Talkeetna", "AK"))
// create a JSON string from the Person, then print it
implicit val formats = DefaultFormats
val jsonString = write(p)
println(jsonString)
}
My build.sbt file contains the following:
libraryDependencies += "net.liftweb" %% "lift-json" % "2.5+"
When I build with sbt package, it is a success.
However, when I try to run it as a Spark job, like this:
spark-submit \
--packages com.amazonaws:aws-java-sdk-pom:1.10.34,org.apache.hadoop:hadoop-aws:2.6.0,net.liftweb:lift-json:2.5+ \
--class "com.foo.MyClass" \
--master local[4] \
target/scala-2.10/my-app_2.10-0.0.1.jar
I get this error:
Exception in thread "main" java.lang.RuntimeException: [unresolved dependency: net.liftweb#lift-json;2.5+: not found]
at org.apache.spark.deploy.SparkSubmitUtils$.resolveMavenCoordinates(SparkSubmit.scala:1068)
at org.apache.spark.deploy.SparkSubmit$.prepareSubmitEnvironment(SparkSubmit.scala:287)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:154)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
What am I doing wrong here? Is net.liftweb:lift-json:2.5+ in my packages argument incorrect? Do I need to add a resolver in build.sbt?
Users may also include any other dependencies by supplying a comma-delimited list of maven coordinates with --packages.
2.5+ in your build.sbt is Ivy version matcher syntax, not actual artifact version needed for Maven coordinates. spark-submit apparently doesn't use Ivy for resolution (and I think it would be surprising if it did; your application could suddenly stop working because a new dependency version was published). So you need to find what version 2.5+ resolves to in your case e.g. using https://github.com/jrudolph/sbt-dependency-graph (or trying to find it in show dependencyClasspath).
I am using play framework 2.5.3 with reactive mongoDB.
import javax.inject._
import model._
import play.api.Logger
import play.api.libs.json._
import play.api.mvc._
import play.modules.reactivemongo._
import reactivemongo.api.ReadPreference
import reactivemongo.play.json._
import reactivemongo.play.json.collection._
import scala.concurrent.{ExecutionContext, Future}
class InsertController #Inject()(val reactiveMongoApi: ReactiveMongoApi)(implicit exec: ExecutionContext) extends Controller with MongoController with ReactiveMongoComponents {
def dataFuture: Future[JSONCollection] = database.map(_.collection[JSONCollection]("data"))
def createFromJson = Action.async(parse.json) { request =>
Json.fromJson[jsonWrapper](request.body) match {
case JsSuccess(data, _) =>
for {
data <- dataFuture
lastError <- data.insert(data)
} yield {
Logger.debug(s"Successfully inserted with LastError: $lastError")
Ok("Inserted into db")
}
case JsError(errors) =>
Future.successful(BadRequest("Something went wrong"))
}
}
Here is my controller and when compiling i get the following exception:
[info] Compiling 6 Scala sources and 1 Java source to /home/***/target/scala-2.11/classes...
[error] /home/***/app/controllers/InsertController.scala:38: No Json serializer as JsObject found for type reactivemongo.play.json.JSONSerializationPack.type. Try to implement an implicit OWrites or OFormat for this type.
[error] lastError <- data.insert(data.pack)
[error] ^
[error] one error found
[error] (compile:compile) Compilation failed
[info] Compiling 6 Scala sources and 1 Java source to /home/***/target/scala-2.11/classes...
[error] /home/***/app/controllers/InsertController.scala:38: No Json serializer as JsObject found for type reactivemongo.play.json.JSONSerializationPack.type. Try to implement an implicit OWrites or OFormat for this type.
[error] lastError <- data.insert(data.pack)
[error] ^
[error] one error found
[error] (compile:compile) Compilation failed
[error] application -
! #705di1397 - Internal server error, for (GET) [/] ->
play.sbt.PlayExceptions$CompilationException: Compilation error[No Json serializer as JsObject found for type reactivemongo.play.json.JSONSerializationPack.type. Try to implement an implicit OWrites or OFormat for this type.]
at play.sbt.PlayExceptions$CompilationException$.apply(PlayExceptions.scala:27)
at play.sbt.PlayExceptions$CompilationException$.apply(PlayExceptions.scala:27)
at scala.Option.map(Option.scala:145)
at play.sbt.run.PlayReload$$anonfun$taskFailureHandler$1.apply(PlayReload.scala:49)
at play.sbt.run.PlayReload$$anonfun$taskFailureHandler$1.apply(PlayReload.scala:44)
at scala.Option.map(Option.scala:145)
at play.sbt.run.PlayReload$.taskFailureHandler(PlayReload.scala:44)
at play.sbt.run.PlayReload$.compileFailure(PlayReload.scala:40)
at play.sbt.run.PlayReload$$anonfun$compile$1.apply(PlayReload.scala:17)
at play.sbt.run.PlayReload$$anonfun$compile$1.apply(PlayReload.scala:17)
It recommends writing a Owrites or OFormat for JSONCollection which is part of the reactivemongo.play.json._ package and should already have these defined to my understanding.
Here is my jsonWrapper class:
case class jsonWrapper(tables : tables, userId : String)
object jsonWrapper{
implicit val jsonRead: Reads[jsonWrapper] = (
(JsPath \ "tables").read[tables] and
(JsPath \ "userID").read[String]
)(jsonWrapper.apply _)
implicit val jsonWrites: Writes[jsonWrapper] = (
(JsPath \ "tables").write[tables] and
(JsPath \ "userID").write[String]
)(json => (json.tables, json.userId))
implicit val jsonWrapperFormat : Format[jsonWrapper] = Json.format[jsonWrapper]
}
The tables class also has implictly defined format, Reads and writes.
I orignally used this example to get started:
https://github.com/jonasanso/play-reactive-mongo-db#master, which works but when i try to adapt it to my needs (i.e. with my jsonWrapper class) i get this error and do not understand why it is not working.
Many Thanks,
Peter M.
I Found my error.
def createFromJson = Action.async(parse.json) { request =>
Json.fromJson[jsonWrapper](request.body) match {
case JsSuccess(data, _) =>
for {
data <- dataFuture
lastError <- data.insert(data)
} yield {
Logger.debug(s"Successfully inserted with LastError: $lastError")
Ok("Inserted into db")
}
case JsError(errors) =>
Future.successful(BadRequest("Something went wrong"))
}
My case entry initiates an object named "data" which i follow up by overriding with my dataFuture object. Thus, causing the error. I simply had to change the variable names...
I fell kind of silly.
I am trying to write a Map in key as int to json string but I am not able to do so:
import org.json4s._
import org.json4s.jackson.JsonMethods._
import org.json4s.JsonDSL._
object MyObject {
def main(args: Array[String]) {
// Works fine
//val myMap = Map("a" -> List(3,4), "b" -> List(7,8))
// Does not work
val myMap = Map(4 -> Map("a" -> 5))
val jsonString = pretty(render(myMap))
println(jsonString)
}
I am receiving the following error:
[error] /my_stuff/my_file.scala:14: overloaded method value render with alternatives:
[error] (value: org.json4s.JValue)org.json4s.JValue <and>
[error] (value: org.json4s.JValue)(implicit formats: org.json4s.Formats)org.json4s.JValue
[error] cannot be applied to (scala.collection.immutable.Map[Int,scala.collection.immutable.Map[String,Int]])
[error] val jsonString = pretty(render(myMap))
[error] ^
[error] one error found
[error] (compile:compileIncremental) Compilation failed
I vaguely understand the error message, it looks like render expects JValue as an input, and I am not providing it, but I don't the first case either, and the code works as I expect.
How do I write such map to json string?
Edit: My source of confusion
I am mostly a python programmer, and in python
In [1]: import json
In [2]: wrong = {2: 5}
In [3]: with open("wrong.json") as f:
...: json.dump(wrong, f)
works perfectly fine, of course python stringifies the 2.
I think it is an expected result. If you check the json specification you will see that you need to use strings for the names of the elements.
So I am afraid you will need something like:
val myMap = Map("4" -> Map("a" -> 5))
The methods in ADSRegistrationMap are used to save and retrieve the document from MongoDB. ObjectId is created during initialization. I have to do the same to load Registration from Json that is part of POST body, so I thought I could just add ADSRegistrationProtocol object to do that. It fails with compilation error. Any idea on how to fix it or do this better?
package model
import spray.json._
import DefaultJsonProtocol._
import com.mongodb.casbah.Imports._
import org.bson.types.ObjectId
import com.mongodb.DBObject
import com.mongodb.casbah.commons.{MongoDBList, MongoDBObject}
case class Registration(
system: String,
identity: String,
id: ObjectId = new ObjectId())
object RegistrationProtocol extends DefaultJsonProtocol {
implicit val registrationFormat = jsonFormat2(Registration)
}
object RegistrationMap {
def toBson(registration: Registration): DBObject = {
MongoDBObject(
"system" -> registration.system,
"identity" -> registration.identity,
"_id" -> registration.id
)
}
def fromBson(o: DBObject): Registration = {
Registration(
system = o.as[String]("system"),
identity = o.as[String]("identity"),
id = o.as[ObjectId]("_id")
)
}
}
Compilation Error:
[error] /model/Registration.scala:20: type mismatch;
[error] found : model.Registration.type
[error] required: (?, ?) => ?
[error] Note: implicit value registrationFormat is not applicable here because it comes after the application point and it lacks an explicit result type
[error] implicit val registrationFormat = jsonFormat2(Registration)
[error] ^
[error] one error found
[error] (compile:compile) Compilation failed
Updated ObjectId to String and jsonFormat2 to jsonFormat3 to fix the compilation error.
case class Registration(
system: String,
identity: String,
id: String = (new ObjectId()).toString())
object RegistrationProtocol extends DefaultJsonProtocol {
implicit val registrationFormat = jsonFormat3(Registration)
}
Getting runtime error now when converting body of POST request to the Registration object. Any idea?
val route: Route = {
pathPrefix("registrations") {
pathEnd {
post {
entity(as[Registration]) { registration =>
Here is what is in build.sbt
scalaVersion := "2.10.4"
scalacOptions ++= Seq("-feature")
val akkaVersion = "2.3.8"
val sprayVersion = "1.3.1"
resolvers += "spray" at "http://repo.spray.io/"
resolvers += "Sonatype releases" at "https://oss.sonatype.org/content/repositories/releases"
// Main dependencies
libraryDependencies ++= Seq(
"com.typesafe.akka" %% "akka-actor" % akkaVersion,
"com.typesafe.akka" %% "akka-slf4j" % akkaVersion,
"com.typesafe.akka" %% "akka-camel" % akkaVersion,
"io.spray" % "spray-can" % sprayVersion,
"io.spray" % "spray-routing" % sprayVersion,
"io.spray" % "spray-client" % sprayVersion,
"io.spray" %% "spray-json" % sprayVersion,
"com.typesafe" % "config" % "1.2.1",
"org.apache.activemq" % "activemq-camel" % "5.8.0",
"ch.qos.logback" % "logback-classic" % "1.1.2",
"org.mongodb" %% "casbah" % "2.7.4"
)
Error:
12:33:03.477 [admcore-microservice-system-akka.actor.default-dispatcher-3] DEBUG s.can.server.HttpServerConnection - Dispatching POST request to http://localhost:8878/api/v1/adsregistrations to handler Actor[akka://admcore-microservice-system/system/IO-TCP/selectors/$a/1#-1156351415]
Uncaught error from thread [admcore-microservice-system-akka.actor.default-dispatcher-3] shutting down JVM since 'akka.jvm-exit-on-fatal-error' is enabled for ActorSystem[admcore-microservice-system]
java.lang.NoSuchMethodError: spray.json.JsonParser$.apply(Ljava/lang/String;)Lspray/json/JsValue;
at spray.httpx.SprayJsonSupport$$anonfun$sprayJsonUnmarshaller$1.applyOrElse(SprayJsonSupport.scala:36)
at spray.httpx.SprayJsonSupport$$anonfun$sprayJsonUnmarshaller$1.applyOrElse(SprayJsonSupport.scala:34)
To avoid any issue, I would define the Register class (which seems a data model) as follows
case class Register(system: String, identity: String, id: String)
That's because it makes more sense to me having an id field as String rather than as BSON ObjectId (I'm used to datamodel which don't depend on 3rd party libraries).
Therefore, the right SprayJson protocol would make use of jsonFormat3 rather than jsonFormat2:
object RegistrationProtocol extends DefaultJsonProtocol {
implicit val registrationFormat = jsonFormat3(Registration)
}
And that would solve any kind of JSON serialization issue.
Finally, your toBson and fromBson converters would be:
def toBson(r: Registration): DBObject = {
MongoDBObject(
"system" -> r.system,
"identity" -> r.identity,
"_id" -> new ObjectId(r.id)
)
}
and
def fromBson(o: DBObject): Registration = {
Registration(
system = o.as[String]("system"),
identity = o.as[String]("identity"),
id = o.as[ObjectId]("_id").toString
)
}
A that's where the BSON ObjectId is being used: much closer to the MongoDB dependant logic.