What do i need to do to extract the value for friends_count. i noticed that screen_name are already define in the Status object and case class. Do still require to extends Js or JsObject different
object TweetDetails extends Js { val friends_count = 'friends_count ? num }
and then pattern match it against each json object in the list of JsObjects as represented below. The symbols are confusing:
scala> val friends_count = 'friends_count ! num // I wish SO understood Scala's symbols
val twtJsonList = http(Status("username").timeline)
twtJsonList foreach {
js =>
val Status.user.screen_name(screen_name) = js
val Status.text(text) = js
val friends_counts(friends_count) = js //i cannot figure out how to extract this
println(friends_count)
println(screen_name)
println(text)
}
Normally, Scala symbols can be thought of as a unique identifier which will always be the same. Every symbol that is lexi-graphically identical refers to the exact same memory space. There's nothing else that's special about them from Scala's point of view.
However, Dispatch-Json pimps out symbols making them JSON property extractors. To see the code which is responsible for the pimping, check out the SymOp class and the rest of the JsonExtractor.scala code.
Let's write some code which solves the problem you are looking at and then analyze what's going on:
trait ExtUserProps extends UserProps with Js {
val friends_count = 'friends_count ! num
}
object ExtUser extends ExtUserProps with Js
val good_stuff = for {
item <- http(Status("username").timeline)
msg = Status.text(item)
user = Status.user(item)
screen_name = ExtUser.screen_name(user)
friend_count = ExtUser.friends_count(user)
} yield (screen_name, msg, friend_count)
The first thing that we're doing is extending the UserProps trait in the Dispatch-Twitter module to give it a friends_count extractor and then defining a ExtUser object which we can use to get access to that extractor. Because the ExtUserProps extends UserProps, which also extends Js, we get the method sym_add_operators in scope which turns our symbol 'friends_count into a SymOp case class. We then call the ! method on that SymOp which we then pass the Extractor num to, which creates an Extractor that looks for a property "friends_count" on a JSON object and then parses it as a number before returning. Quite a bit going on there for such a small bit of code.
The next part of the program is just a for-comprehension that calls out to the Twitter timeline for a user and parses it into JsObjects which represent each status item, them we apply the Status.text extractor to pull out the status message. Then we do the same to pull out the user. We then pull the screen_name and friend_count out of the user JsObject and finally we yield a Tuple3 back with all of the properties we were looking for. We're then left with a List[Tuple3[String,String,BigDecimal]] which you could then iterate on to print out or do whatever with.
I hope that clears some things up. The Dispatch library is very expressive but can be a little tough to wrap your head around as it uses a lot of Scala tricks which someone just learning Scala won't get right away. But keep plugging around and playing with, as well as looking at the tests and source code, and you'll see how to create powerful DSL's using Scala.
Related
Actually, the main problem is still that there are no reified typeargs for classes in Kotlin. But here is why this bothers me in this specific case:
Suppose you have a wrapper class Wrapper that takes in a string content and a class* type and can output an object of class type retrieved by parsing content as JSON by demand by calling the function getObj():
class Wrapper<T>(private val content: String, /*private val type: KClass<*>*/) {
fun getObj(): T {
// ?
}
}
And I want to use kotlinx.serialization. Now, you might have noticed how I put an asterisk after "class" before. Here's the reason: Yes, Wrapper has to take the target class in some way, but how? Should it be just the typearg (won't work because type erausre) or a KClass reference (won't work because I need a reified typearg)?
The thing is that as far as I know, the only way to decode a generic JSON to a serializable target class is to use Json.decodeFromString<T>(content), where T is the target type and content is the JSON string. Now, T is defined to be reified (so that the type can be processed at runtime) and can only be filled with another reified typearg or an actual class reference. I can't use another reified typearg because I am in the context of a class and a class cannot have reified typeargs. I can also not use an actual class reference because the user of the class should be able to construct it with different targets, e.g. they decide what the target is, not me.
So, how do I do this with kotlinx.serialization? Is it even possible?
Ok so no one answered the question yet, but I also posted this question in the r/Kotlin subreddit. Here it is.
I actually got an answer there (credits to u/JakeWharton), and since you might get across this StackOverflow question because you googled the same question, you might be happy to find an answer here. So here's my try to paraphrase the answer:
So, basically, kotlinx-serialization does indeed not work with KClasses. But when you think about it, you only need the KClass to determine how to serialize it. And since that is determined at compile-time when you work with KXS, you actually just need to pass the serializer (the actual strategy defining how to serialize / deserialize your class). You can obtain a serializer for every class annotated with #Serializable by invoking .serializer() on it; the result will be of the type KSerializer<T>. So, instead of having
class Wrapper<T>(private val content: String, private val type: KClass<T>)
and constructing it via
val wrapper = Wrapper("{}", Foo::class)
You can do it like this:
class Wrapper<T>(private val content: String, private val serializer: KSerializer<T>)
and then construct it like this:
val wrapper = Wrapper("{}", Foo.serializer())
(supposing Foo is annotated with #Serializable)
you can then serialize and deserialize by using the KSerializer instead of a typearg, like this:
val obj: T = Json.decodeFromString(serializer, "[Your JSON String]")
val str: String = Json.encodeToString(serializer, obj)
And that's it! Just swap out your regular (K)Class approach by KSerializer and it'll work with KXS.
I have a dstream and trying to parse each string into an object using play json library.
It works on the outside but it doesn't work on the inside. It says no implicits found.
val textStream: ReceiverInputDStream[String] = streamingContext.socketTextStream("localhost",12345)
textStream.map{record=>
Json.parse(record).as[Car] //this doesn't works, it shows as: Json.parse(record).as[Car](...)
}
This is defined in the companion object of Car:
object Car {
implicit val carFormat = Json.format[Car]
}
It works when implicits are given explicitly inside and it seems a bit odd.
textStream.map{record=>
implicit val carFormat = Json.format[Car] //How do I avoid this
Json.parse(record).as[Car]
}
How do I avoid defining inside and make it work by defining the implicits outside ?
I also tried importing explicit inside the companion object but it doesn't make any difference.
No instance of play.api.libs.json.Format is available for models.AccountStatus in the implicit scope.
This is the code taken from a github page, and only class names and variable names are changed.
package models
import slick.jdbc.H2Profile._
import play.api.libs.json._
case class Account(id: Long, name: String, category: Int, status:AccountStatus)
object Account {
implicit val accountFormat = Json.format[Account]
}
sealed abstract class AccountStatus(val as:Int)
object AccountStatus{
final case object Draft extends AccountStatus(0)
final case object Active extends AccountStatus(1)
final case object Blocked extends AccountStatus(2)
final case object Defaulter extends AccountStatus(3)
implicit val columnType: BaseColumnType[AccountStatus] = MappedColumnType.base[AccountStatus,Int](AccountStatus.toInt, AccountStatus.fromInt)
private def toInt(as:AccountStatus):Int = as match {
case Draft => 0
case Active => 1
case Blocked => 2
case Defaulter => 3
}
private def fromInt(as: Int): AccountStatus = as match {
case 0 => Draft
case 1 => Active
case 2 => Blocked
case 3 => Defaulter
_ => sys.error("Out of bound AccountStatus Value.")
}
}
https://github.com/playframework/play-scala-slick-example/blob/2.6.x/app/models/Person.scala
So, this code needs to be added inside of the object AccountStatus code block since we need to use fromInt to transform an Int to an AccountStatus. This is a Reads defined for AccountStatus:
implicit object AccountStatusReads extends Reads[AccountStatus] {
def reads(jsValue: JsValue): JsResult[AccountStatus] = {
(jsValue \ "as").validate[Int].map(fromInt)
}
}
What's a Reads? It's just a trait that defines how a JsValue (the play class encapsulating JSON values) should be deserialized from JSON to some type. The trait only requires one method to be implemented, a reads method which takes in some json and returns a JsResult of some type. So you can see in the above code that we have a Reads that will look for a field in JSON called as and try to read it as an integer. From there, it will then transform it into an AccountStatus using the already defined fromInt method. So for example in the scala console you could do this:
import play.api.libs.json._
// import wherever account status is and the above reader
scala> Json.parse("""{"as":1}""").as[AccountStatus]
res0: AccountStatus = Active
This reader isn't perfect though, mainly because it's not handling the error your code will give you on out of bound numbers:
scala> Json.parse("""{"as":20}""").as[AccountStatus]
java.lang.RuntimeException: Out of bound AccountStatus Value.
at scala.sys.package$.error(package.scala:27)
at AccountStatus$.fromInt(<console>:42)
at AccountStatusReads$$anonfun$reads$1.apply(<console>:27)
at AccountStatusReads$$anonfun$reads$1.apply(<console>:27)
at play.api.libs.json.JsResult$class.map(JsResult.scala:81)
at play.api.libs.json.JsSuccess.map(JsResult.scala:9)
at AccountStatusReads$.reads(<console>:27)
at play.api.libs.json.JsValue$class.as(JsValue.scala:65)
at play.api.libs.json.JsObject.as(JsValue.scala:166)
... 42 elided
You could handle this by making the Reads handle the error. I can show you how if you want, but first the other part of a Format is a Writes. This trait, unsurprisingly is similar to reads except it does the reverse. You're taking your class AccountStatus and creating a JsValue (JSON). So, you just have to implement the writes method.
implicit object AccountStatusWrites extends Writes[AccountStatus] {
def writes(as: AccountStatus): JsValue = {
JsObject(Seq("as" -> JsNumber(as.as)))
}
}
Then this can be used to serialize that class to JSON like so:
scala> Json.toJson(Draft)
res4: play.api.libs.json.JsValue = {"as":0}
Now, this is actually enough to get your error to go away. Why? Because Json.format[Account] is doing all the work we just did for you! But for Account. It can do this because it's a case class and has less than 22 fields. Also every field for Account has a way to be converted to and from JSON (via a Reads and Writes). Your error message was showing that Account could not have a format automatically created for it because part of it (status field) had no formatter.
Now, why do you have to do this? Because AccountStatus is not a case class, so you can't call Json.format[AccountStatus] on it. And because the subclasses of it are each objects, which have no unapply method defined for them since they're not case classes. So you have to explain to the library how to serialize and deserialize.
Since you said you're new to scala, I imagine that the concept of an implicit is still somewhat foreign. I recommend you play around with it / do some reading to get a grasp of what to do when you see that the compiler is complaining about not being able to find an implicit it needs.
Bonus round
So, you might really not want to do that work yourself, and there is a way to avoid having to do it so you can do Json.format[AccountStatus]. You see Json.format uses the apply and unapply methods to do its dirty work. In scala, these two methods are defined automatically for case classes. But there's no reason you can't define them yourself and get everything they give you for free!
So, what do apply and unapply look like type signature wise? It changes per class, but in this case apply should match Int => AccountStatus (a function that goes from an int to an AccountStatus). So it's defined like so:
def apply(i: Int): AccountStatus = fromInt(i)
and unapply is similar to the reverse of this, but it needs to return an Option[Int], so it looks like
def unapply(as: AccountStatus): Option[Int] = Option(as.as)
with both of these defined you don't need to define the reads and writes yourself and instead can just call
// this is still inside the AccountStatus object { ... }
implicit val asFormat = Json.format[AccountStatus]
and it will work in a similar fashion.
.P.S. I'm traveling today, but feel free to leave any comments if some of this doesn't make sense and I'll try to get back to you later on
I'm very new to play! and scala and I'm trying to parse an array composed of json objects. I need to go through the array, count the number of specific occurrences in every object, add them up and pass them on to the html index. Here's what my controller would roughy look like:
object Application extends Controller {
def stringArray=<array of strings, each a JSValue>
var counter=0
for(i<-0 to stringArray.length){
counter+=(((Json.parse(stringArray(i))\"some_element").toString()).count(y=>y=="some_keyword"))
}
def index = Action {
Ok(views.html.index(counter))
}
}
But there's virtually no way to implement a for loop in the application controller. I've tried to pass on the array to index but other scala functions such as Json.parse and count seem to not be recognized the html template. What would be a possible workaround?
What about this?
object Application extends Controller {
val stringArray=<array of strings, each a JSValue>
def index = Action {
var counter = 0
for(s<-stringArray){
counter+=(((Json.parse(s)\"some_element").toString()).count(y=>y=="some_keyword"))
}
Ok(views.html.index(counter))
}
}
I haven't checked the inside part of the loop, but you seem to be confused about where to put the loop. Maybe this can be also rewritten as fold to have nicer code (for loops are generally considered not nice in Scala, as far as I understand ;) ). To use the functions in the templates, you may need to import them first. After the first line of template, where you declare the template function header, you can easily import stuff like for example: #import java.util.Date. Just make sure you import the correct class and you should be able to use the functions in templates as well.
And final note: 1 to 3 gives {1, 2, 3}, so you usually want 1 until array.length, as 1 until 3 gives {1, 2}. Usually you can use the for (element <- array) notation, which is easier to look at.
I need to parse some messages. The first 4 bytes of a message identify the type of message, so, using that, I can instantiate an object of the proper type. To make this an efficient operation, I thought I would create a hash map where they key is the first 4 bytes, and the value is the object constructor. I can just look up the constructor and invoke it.
After all, constructors are just functions, and there shouldn't be any problem putting functions in a map. It turns out that I am having some difficulty with this because I don't know how to express the reference to the constructor properly.
To get concrete with a simplified example, suppose we have a message base class, MsgBase, and a couple subclasses, MsgA and MsgB. If I create a companion object for each of the messages and put a factory function into it, I can make the array without any problem using those functions.
Here is a simplified sample which takes the message as a string.
class MsgBase(message: String) { }
class MsgA(message: String) extends MsgBase(message) { }
object MsgA { def makeIt(message: String): MsgA = new MsgA(message) }
and where MsgB is similar. Then I can make the map:
val cm = Map[String, (String) => MsgBase]("a" -> MsgA.makeIt, "b" -> MsgB.makeIt)
val myMsg = cm("a")("a.This is the message")
It seems like I should be able to refer to the message object constructor directly in the expression building the map, rather than using the trivial function in the companion object, but I haven't figured out any way to express that. Is there a way?
Try
"a" -> (new MsgA(_))
(all parentheses are needed).
Even if this didn't work, you could of course always define the function explicitly:
"a" -> ( (s: String) => new MsgA(s) )
For this case it would be better to use case classes, which automatically provide you functions for creating new objects.
scala> case class MsgA(message: String) extends MsgBase(message)
scala> case class MsgB(message: String) extends MsgBase(message)
So you can refer them just by name, without any syntactical overhead
scala> val m = Map("a"->MsgA, "b"->MsgB)
m: scala.collection.immutable.Map[java.lang.String,scala.runtime.AbstractFunction1[java.lang.String,Product with MsgBase]] = Map((a,<function1>), (b,<function1>))
scala> m("a")("qqq")
res1: Product with MsgBase = MsgA(qqq)
As an alternative approach you can create companion object with overrided apply method by hand. For details see Programming scala, chapter 6
val cm = Map[String, (String) => MsgBase]("a" -> (new MsgA(_)), "b" -> (new MsgB(_)))