I am starting up on Scala, doing a project with circe to handle JSON.
I am coming accross a lot of Either returns from functions, and I don't seem to find a elegant way to handle all of them.
For instance, for a single either, I do as in this snippet:
if (responseJson.isRight) {
//do something
} else {
//do something else
}
But what should I do when I have a lot of them in sequence, such as this example in which I just go straight for the right side and I feel I should be doing some extra validation:
ClassA(
someValue,
someValue,
someJson.hcursor.get[Double]("jsonKey1").right.get,
someJson.hcursor.get[Double]("jsonKey2").right.get,
someJson.hcursor.get[Double]("jsonKey3").right.get
)
How should/can I handle multiple Either objects (without ending up with a bunch of if-elses, or similar) when I want to get their contents if they are a Right, but not I am not sure they are always a Right ?
Lets say you have a case class,
case class Demo(i: Int, s: String)
and two eithers,
val intEither: Either[Throwable, Int] = ???
val stringEither: Either[Throwable, Int] = ???
So... lets start with the most basic and obvious one,
val demoEither: Either[Throwable, Demo] =
intEither.flatMap(i =>
stringEither.map(s => Demo(i, s))
)
Another way is to do the same as above is to use for-comprehensions,
val demoEither: Either[Throwable, Demo] =
for {
i <- intEither
s <- stringEither
} yield Demo(i, s)
But, monads are sequential, which means that if the first Either is a Left then you will not even look at the second Either and just get a Left. This is mostly undesirable for validations because you don't want to loose the validation information of all components, so what you actually want is an Applicative.
And Either is not an Applicative, you will have to use cats or scalaz or implement your own applicative for this.
cats provides the Validated applicative for this express purpose which lets you validate and keep all error information of the validated components.
import cats.data._
import cats.implicits._
val intValidated: ValidatedNec[Throwable, Int] =
intEither.toValidatedNec
val stringValidated: ValidatedNec[Throwable, String] =
stringEither.toValidatedNec
val demoValidated: ValidatedNec[Throwable, Demo] =
(intValidated, stringValidated).mapN(Demo)
val demoEither: Either[List[Throwable], Demo] =
demoValidated.leftMap(errorNec => errorNec.toList)
Or, if you are doing this just once and don't want to depend on cats, you can just use pattern-matching which is very versatile
val demoEither: Either[List[Throwable], Demo] =
(intEither, stringEither) match {
case (Right(i), Right(s)) => Right(Demo(i, s))
case (Left(ti), Left(ts)) => Left(List(ti, ts))
case (Left(ti), _) => Left(List(ti))
case (_, Left(ts)) => Left(List(ts))
}
How should/can I handle multiple Either objects (without ending up with a bunch of if-elses, or similar) when I want to get their contents if they are a Right, but not I am not sure they are always a Right?
So you have some Either instances, all with the same type signature.
val ea :Either[Throwable,String] = Right("good")
val eb :Either[Throwable,String] = Left(new Error("bad"))
val ec :Either[Throwable,String] = Right("enough")
And you want all the Right values, ignoring any Left values.
List(ea, eb, ec).collect{case Right(x) => x}
//res0: List[String] = List(good, enough)
You don't know which Either contains which String but I think that's what you asked for.
Related
I have a module like so...
class ApbSplitter (clients : List[ApbRange]) extends MultiIOModule {
val nApb = clients.length
val apb = IO(Vec(nApb, new ApbChannel()))
val apb_m = IO(Flipped(new ApbChannel))
...
What I'd like to do is suggestName to each element of the Vec so that instead of prefixed as apb_0_ apb_1_ etc... it's whatever I provide for each element.
I can apb.suggestName but that only affects the leading prefix and the array indices remain. Doing apb(idx).suggestName("blah") compiles but has no effect.
Any way to make this happen?
Got this to work by eliminating the Vec and creating a list of IO
case class ApbRange (name: String, loAddr : Int, hiAddr : Int)
class ApbSplitter (clients : List[ApbRange]) extends MultiIOModule {
val apb = clients.map({x => IO(new ApbChannel).suggestName(x.name)})
val apb_m = IO(Flipped(new ApbChannel))
...
Not sure if this is canonical but seems to do the trick just fine.
Answering this with Brian's other post and comment on his own answer on this post in mind. This is going to be a long answer because it touches on a couple of warts in the Chisel API that are being improved but are certainly relevant in the current version (v3.4.3 as of 12 Aug 2021).
Brian's answer is correct that if you want to name the individual fields you need to use a Seq and not a Vec. The reason for this is that, from Chisel's perspective, an IO of type Vec is a single port with an aggregate type, whereas the Seq is just a sequence of unrelated ports. The Seq is a Scala construct (whereas Vec comes from Chisel), so Chisel itself doesn't know anything about the relationship between the ports in the Seq.
The problem then, is that you need a Vec to do dynamic indexing. You can use VecInit to create a dynamically indexable Wire from your Seq whenever you need to do dynamic indexing:
For example:
class MyModule(names: Seq[String]) extends RawModule {
val enq = names.map(n => IO(Flipped(Decoupled(UInt(8.W)))).suggestName(n))
val idx = IO(Input(UInt(log2Ceil(names.size).W)))
val deq = IO(Decoupled(UInt(8.W)))
// enqWire connects all fields of enq
val enqWire = VecInit(enq)
// Need to make sure backpressure is always driven
enqWire.foreach(_.ready := false.B)
deq <> enqWire(idx)
}
This will work so long as deq is itself a port. It will not work if deq were a Wire because <> is a commutative operator and is thus ambiguous when connecting 2 bidirectional wires. For a longer explanation, see this PR comment.
If deq needs to be a Wire for some reason, you could use a helper module that does have Vecs as ports:
For example:
class InnerHelper(n: Int) extends RawModule {
val enq = IO(Flipped(Vec(n, Decoupled(UInt(8.W)))))
val idx = IO(Input(UInt(log2Ceil(n).W)))
val jdx = IO(Input(UInt(log2Ceil(n).W)))
val deq = IO(Vec(n, Decoupled(UInt(8.W))))
// backpressure defaults
enq.foreach(_.ready := false.B)
deq.foreach { x =>
x.valid := false.B
x.bits := DontCare
}
deq(jdx) <> enq(idx)
}
class MyModule(names: Seq[String]) extends RawModule {
val enq = names.map(n => IO(Flipped(Decoupled(UInt(8.W)))).suggestName(n))
val idx = IO(Input(UInt(log2Ceil(names.size).W)))
val jdx = IO(Input(UInt(log2Ceil(names.size).W)))
val deq = names.map(n => IO(Decoupled(UInt(8.W))).suggestName(s"${n}_out"))
val helper = Module(new InnerHelper(names.size))
helper.enq <> enq
helper.idx := idx
helper.jdx := jdx
helper.deq <> deq
}
It's a bit of a pain, but it at least resolves the ambiguity. There are other utilities we could build--for example, instead of a custom InnerHelper for each case, we could make a utility method that creates a module so that the returned value of dynamically indexing a Seq is a port of a new submodule, but it's a bit tricky.
The good news is that a better way is coming--DataView in Chisel 3.5 should make it possible to view a Seq as a Vec (rather than having to use VecInit which creates a Wire) which makes it easier to avoid this Wire <> connect ambiguity issue. I also hope to either "fix" <> for Wires or perhaps provide a new operator that is not commutative :<>, but that is not yet being worked on.
I am guessing your new apbChannel has a bunch of Input Output signals or wires. So instead of apb(idx).suggestName if your apbChannel has a (say) val ip = Input(Bool()) you can do apb(idx).ip.suggestName("blah")
I'm trying to get the index of the Max element in a UInt vector.
My code looks like this
val pwr = Vec.tabulate(N) {i => energyMeters(i).io.pwr}
val maxPwr = pwr.indexOf(pwr.max)
However this code generate compilation error:
No implicit Ordering Defined for Chisel.UInt.
val maxPwr = pwr.indexOf(pwr.max)
^
I understand that I probably need to implement the max function , can someone give an example how this should be done ?
Edit:
I also tried this:
val pwr = Vec.tabulate(N) {i => energyMeters(i).io.pwr}
val maxPwr = pwr reduceLeft {(x,y) => Mux(x > y,x,y)}
val maxPwridx = pwr.indexOf(maxPwr)
But it fails on elaboration , when I tried to cast maxPwridx to UInt.
I've ended up with this workaround:
val pwr = Vec.tabulate(N) {i => energyMeters(i).io.pwr}
val maxPwr = pwr reduceLeft {(x,y) => Mux(x > y,x,y)}
val maxPwridx = pwr.indexWhere((x : UInt => x === maxPwr))
Chisel's Vec extends Scala's Seq. This means that a Vec has both dynamic access hardware methods that will allow you to generate hardware to search for something in a Vec (e.g., indexWhere, onlyIndexWhere, lastIndexWhere) as well as all the methods available to normal Scala sequences (e.g., indexOf).
For the purposes of doing hardware operations, you want to use the former (as you found in your last edit---which looks great!) as opposed to the latter.
To get some handle on this, the screenshot below shows the Chisel 3.3.0-RC1 API documentation for VecLike, filtered to excluded inherited methods. Notable here are indexWhere, onlyIndexWhere, lastIndexWhere, exists, forall, and contains:
And the documentation for Vec. The only interesting method here would be reduceTree:
what's the best way to turn a Right[List] into a List
I will parse a Json String like so
val parsed_states = io.circe.parser.decode[List[List[String]]](source)
And that will create an value equivalent to this
val example_data = Right(List(List("NAME", "state"), List("Alabama", "01"), List("Alaska", "02"), List("Arizona", "04")))
I'm trying to grok Right, Left, Either and implement the best way to get a list of StateName, StateValue pairs out of that list above.
I see that any of these ways will give me what I need (while dropping the header):
val parsed_states = example_data.toSeq(0).tail
val parsed_states = example_data.getOrElse(<ProbUseNoneHere>).iterator.to(Seq).tail
val parsed_states = example_data.getOrElse(<ProbUseNoneHere>).asInstanceOf[Seq[List[String]]].tail
I guess I'm wondering if I should do it one way or another based on the possible behavior upstream coming out of io.circe.parser.decode or am I overthinking this. I'm new to the Right, Left, Either paradigm and not finding terribly many helpful examples.
in reply to #slouc
trying to connect the dots from your answer as they apply to this use case. so something like this?
def blackBox: String => Either[Exception, List[List[String]]] = (url:String) => {
if (url == "passalong") {
Right(List(List("NAME", "state"), List("Alabama", "01"), List("Alaska", "02"), List("Arizona", "04")))
}
else Left(new Exception(s"This didn't work bc blackbox didn't parse ${url}"))
}
//val seed = "passalong"
val seed = "notgonnawork"
val xx: Either[Exception, List[List[String]]] = blackBox(seed)
def ff(i: List[List[String]]) = i.tail
val yy = xx.map(ff)
val zz = xx.fold(
_ => throw new Exception("<need info here>"),
i => i.tail)
The trick is in not getting state name / state value pairs out of the Either. They should be kept inside. If you want to, you can transform the Either type into something else (e.g. an Option by discarding whatever you possibly had on the left side), but don't destroy the effect. Something should be there to show that decoding could have failed; it can be an Either, Option, Try, etc. Eventually you will process left and right case accordingly, but this should happen as late as possible.
Let's take the following trivial example:
val x: Either[String, Int] = Right(42)
def f(i: Int) = i + 1
You might argue that you need to get the 42 out of the Right so that you can pass it to f. But that's not correct. Let's rewrite the example:
val x: Either[String, Int] = someFunction()
Now what? We have no idea whether we have a Left or a Right in value x, so we can't "get it out". Which integer would you obtain in case it's a Left? (if you really do have an integer value to use in that case, that's fair enough, and I will address that use case a bit later)
What you need to do instead is keep the effect (in this case Either), and you need to continue working in the context of that effect. It's there to show that there was a point in your program (in this case someFunction(), or decoding in your original question) that might have gone wrong.
So if you want to apply f to your potential integer, you need to map the effect with it (we can do that because Either is a functor, but that's a detail which probably exceeds the scope of this answer):
val x: Either[String, Int] = Right(42)
def f(i: Int) = i + 1
val y = x.map(value => f(value)) // Right(43)
val y = x.map(f) // shorter, point-free notation
and
val x: Either[String, Int] = someFunction()
def f(i: Int) = i + 1
// either a Left with some String, or a Right with some integer increased by 1
val y = x.map(f)
Then, at the very end of the chain of computations, you can handle the Left and Right cases; for example, if you were processing an HTTP request, then in case of Left you might return a 500, and in case of Right return a 200.
To address the use case with default value mentioned earlier - if you really want to do that, get rid of the Left and in that case resolve into some value (e.g. 0), then you can use fold:
def f(i: Int) = i + 1
// if x = Left, then z = 0
// if x = Right, then z = x + 1
val z = x.fold(_ => 0, i => i + 1)
simple question:
How can one use play-json (2.3.x) to sort all JsArrays in some JsValue (recursively)?
my usecase:
consider an app that uses Set[String] internally, and when data is requested,
the output JSON serialize the set as a JSON array. the order is not important.
now, if one wants to write some tests to cover this functionality, since the order of items is not important (it is a set after all. internally, and conceptually), and all I want to check is that everything returned as it should, I may want to compare the response JSON with an "expected" JSON object I create explicitly.
for that exact reason, I want to sort the JSON arrays, and compare the JsValue's.
how would one write such transformer?
EDIT:
I have managed to write a transformer that answers my needs, but it won't sort every JsArry in some JsValue. I'll post it here, since it might be useful for others, but it is not what I was asking for.
val jsonSortTransformer = (__ \ 'fields).json.update(
Reads.JsObjectReads.map{
case JsObject(xs) => JsObject(
xs.map{
case (n,jv) => {
n -> (jv match {
case JsArray(arr) if arr.forall(_.isInstanceOf[JsString]) => JsArray(arr.sortBy(_.as[String]))
case _ => jv
})
}
}
)
}
)
You can use the value property on JsArray to get a Seq[JsValue], then sort arbitrarily, and then recreate a JsArray. For example:
scala> myJsArray
play.api.libs.json.JsArray = ["11","4","5","1","22","2"]
scala> JsArray(myJsArray.value.sortBy(_.as[JsString].value.toInt))
play.api.libs.json.JsArray = ["1","2","4","5","11","22"]
If all you're doing is trying to compare actual and expected values of what you know is a set, you can also just use value on both properties, build a Set and check for equality:
Set(actual.value: _*) == Set(expected.value: _*)
Or sort them both:
val sortedSeq: JsArray => Seq[String] = array => array.value.map(_.toString).sorted
sortedSeq(actual) == sortedSeq(expected)
To recursively sort all the JsArrays in an arbitrary JsValue, it might look something like:
def sortArrays(json: JsValue): JsValue = json match {
case JsObject(obj) => JsObject(obj.toMap.mapValues(sortArrays(_)).toList)
case JsArray(arr) => JsArray(arr.map(sortArrays).sortBy(_.toString))
case other => other
}
scala> myObj
play.api.libs.json.JsValue = {"a":[2,1],"b":[{"c":[3,2]},{"d":[4,3]}],"e":{"f":[5,4]}}
scala> sortArrays(myObj)
play.api.libs.json.JsValue = {"a":[1,2],"b":[{"c":[2,3]},{"d":[3,4]}],"e":{"f":[4,5]}}
I'm afraid that #Ben's answer is quite incorrect.
I would approach this problem by defining an Ordering class for JsValues and then use its comparison method to verify equality (meaning this should actually be an object - not an anonymous class, as shown in the example).
One doesn't have to use Ordering, I just find it a bit more convenient than a simple compareTo method. Of course, one can also define this class/object as implicit.
val jsonOrdering: Ordering[JsValue] = new Ordering[JsValue]() {
override def compare(x: JsValue, y: JsValue): Int = {
x.getClass.getName.compareTo(y.getClass.getName) match {
case 0 =>
(x, y) match {
case (JsNull, JsNull) => 0
case (JsString(valueX), JsString(valueY)) =>
valueX.compareTo(valueY)
case (JsNumber(valueX), JsNumber(valueY)) =>
valueX.compare(valueY)
case (JsBoolean(boolX), JsBoolean(boolY)) =>
boolX.compareTo(boolY)
case (JsArray(elementsX), JsArray(elementsY)) =>
elementsX.size.compareTo(elementsY.size) match {
case 0 =>
elementsX
// .sorted(this) // uncomment if array order DOES NOT matter
.zip(elementsY
// .sorted(this) // uncomment if array order DOES NOT matter
)
.view
.map {
case (elementX, elementY) => compare(elementX, elementY)
}
.find(_ != 0)
.getOrElse(0)
case nonZero => nonZero
}
case (JsObject(fieldsX), JsObject(fieldsY)) =>
fieldsX.size.compareTo(fieldsY.size) match {
case 0 =>
fieldsX.toSeq
.sortBy(_._1)
.zip(fieldsY.toSeq.sortBy(_._1))
.view
.flatMap {
case ((keyX, valueX), (keyY, valueY)) =>
Seq(keyX.compareTo(keyY), compare(valueX, valueY))
}
.find(_ != 0)
.getOrElse(0)
case nonZero => nonZero
}
}
case nonZero => nonZero
}
}
I would perhaps split some parts into private/nested functions (I got lazy this time). Anyway, let's go over this:
Compare the two values' class names, and if they aren't the same then return the comparison between their names.
If the values are of any primitive JSON type, simply return the comparison between them.
If the values are arrays, then:
Compare their sizes, and if they aren't the same then return the comparison between the sizes.
Only if the order of arrays doesn't matter - sort each of the arrays (with the same ordering class; i.e, this is recursive).
Zip the elements of both arrays (so that you get an array of pairs of elements).
Find the first pair that its two elements are not the same, and return their comparison.
If no such pair exists, this means that the arrays are the same (return 0).
If the values are maps (objects):
Compare their sizes, and if they aren't the same then return the comparison between the sizes.
Turn the maps into a sequence of tuples, and sort these sequences by their key (first element of a tuple).
Zip the tuples of both sequences (so that you get an array of pairs of tuples).
Find the first pair that its tuples are not the same, and return their comparison. Compare these tuples in the following manner:
Compare their keys (strings), and return their comparison if they're not the same.
Compare their values (JsValue, thus using the same method recursively), and return their comparison if they're not the same.
Otherwise, they are the same.
If no such pair exists, this means that the maps (objects) are the same (return 0).
Note that although this ordering is consistent and deterministic, it is quite arbitrary and doesn't convey much logical meaning.
I get the following as a result from using the scala json parse.
import scala.util.parsing.json.JSON._
val j: String = """["this",["a","b",["c","d"]]]"""
val parse_test=parseFull(j)
now from this I get a result of Option[Any]
I can use get to obtain the results (in this case I am not concerned about invalid json format, so this should be safe, right?)
parse_test.get
res26: Any = List(this, List(a, b, List(c, d)))
Now, how should I go about going from this Any to the List that I had expected? I assume I should use pattern matching, but I can't figure it out. Any help would be much appreciated
Here is my solution:
scala> val Some(xs # List(_*)) = parse_test
xs: List[Any] = List(this, List(a, b, List(c, d)))
What you could do is a fold with a pattern match and a cast:
test_result.fold[List[String]](Nil){
case _ :: list :: _ => list.asInstanceOf[List[String]]
case _ => Nil
}
Assuming you're trying to throw out of the first element and that the 2nd element is the list you wanted.
Edit:
Be aware that if the 2nd element isn't a list this cast would cause an exception. It's really horrible dealing with a List[Any] and trying to decode what's in there...