How can I convert the following Map structure which is a Map[String,Any] to a Json in scala? I am using Play.
val result = s
.groupBy(_.dashboardId)
.map(
each => Map(
"dashboardId" -> each._1,
"cubeId" -> each._2.head.cubeid,
"dashboardName" -> each._2.head.dashboardName,
"reports" -> each._2.groupBy(_.reportId).map(
reportEach => Map(
"reportId" -> reportEach._1,
"reportName" -> reportEach._2.head.reportName,
"selectedColumns" -> reportEach._2.groupBy(_.selectedColumnid).map(
selectedColumnsEach => Map(
"selectedColumnId" -> selectedColumnsEach._1,
"columnName" ->
selectedColumnsEach._2.head.selectColumnName.orNull,
"seq" ->selectedColumnsEach._2.head.selectedColumnSeq,
"formatting" -> selectedColumnsEach._2
)
)
)
)
)
)
You cannot convert a Map[String, Any] to Json but you can convert a Map[String, String] or Map[String, JsValue].
In your case, you can do by converting each map value to a JsValue before hand with:
Map(
"dashboardId" -> Json.toJson(each._1),
"cubeId" -> Json.toJson(each._2.head.cubeid),
"dashboardName" -> Json.toJson(each._2.head.dashboardName),
"reports" -> Json.toJson(each._2.groupBy(_.reportId).map(
reportEach => Map(
"reportId" -> Json.toJson(reportEach._1),
"reportName" -> (reportEach._2.find(_.reportName != null) match {
case Some(reportNameData) => Json.toJson(reportNameData.reportName)
case None => JsNull
})),
...
)
I read the results into a Seq[Map[String,Any]] by using .toSeq and then used toJson to convert it into a Json this worked.
val s = new SaveTemplate getReportsWithDashboardId(dashboardId)
val result : Seq[Map[String,Any]] = s.groupBy(_.dashboardId)
.map(
each => Map(
"dashboardId" -> each._1,
"cubeId" -> each._2.head.cubeid,
"dashboardName" -> each._2.head.dashboardName,
"reports" -> each._2.groupBy(_.reportId).map(
reportEach => Map(
"reportId" -> reportEach._1,
"reportName" -> (reportEach._2.find(_.reportName != null) match {
case Some(reportNameData) => reportNameData.reportName
case None => null
}),
"selectedColumns" -> reportEach._2.groupBy(_.selectedColumnid).map(
selectedColumnsEach => Map(
"selectedColumnId" -> selectedColumnsEach._1,
"columnName" -> selectedColumnsEach._2.head.selectColumnName.orNull,
"seq" ->selectedColumnsEach._2.head.selectedColumnSeq,
"formatting" -> Map(
"formatId" -> (selectedColumnsEach._2.find(_.formatId != null) match {
case Some(reportNameData) => reportNameData.formatId
case None => null
}),
"formattingId" -> (selectedColumnsEach._2.find(_.formattingid != null)
match {
case Some(reportNameData) => reportNameData.formattingid
case None => null
}),
"type" -> (selectedColumnsEach._2.find(_.formattingType != null) match
{
case Some(reportNameData) => reportNameData.formattingType
case None => null
})
)
)
)
)
)
)
).toSeq
val k = toJson(result)
Ok(k)
Related
I have a collection structure as shown below. I need to build the statement based on k,v pairs in the map. The level of nesting is not known prior and code needs to parse it dynamically.
Structure:
case class Step(id: Option[String], stepId: Option[String], parameters: Option[Map[String, Any]])
val sample = Step(id = Some("a1"), Some("case"), Some(
Map(
"CONDITIONAL" -> List(
Map("comparison" -> List(
Map("comparision" -> List(
Map("comparision" -> List(
Map("comparator" -> List(
Map(
"EVAL" -> "c11",
"OPERATION" -> "equals to",
"WHEN" -> List("A")),
Map(
"eval" -> "c12",
"operation" -> "in",
"when" -> List("B")),
Map(
"eval" -> "c13",
"operation" -> "starts with",
"when" -> List("C")
)),
"operator" -> "OR"
),
Map("comparator" -> List(
Map(
"EVAL" -> "c21",
"OPERATION" -> "equals to",
"WHEN" -> List("A")),
Map(
"eval" -> "c22",
"operation" -> "in",
"when" -> List("B")
)),
"operator" -> "OR"
)),
"operator" -> "AND"
),
Map("comparator" -> List(
Map(
"EVAL" -> "c31",
"OPERATION" -> "equals to",
"WHEN" -> List("A")),
Map(
"EVAL" -> "c32",
"OPERATION" -> "in",
"WHEN" -> List("B")),
Map(
"EVAL" -> "c33",
"OPERATION" -> "starts with",
"WHEN" -> List("C")
)),
"operator" -> "OR"
)),
"operator" -> "OR"
)),
"THEN" -> "result"
)),
"ELSE" -> "default"
))
)
val stepsLower = sample.parameters.get.map(p => p._1.toLowerCase -> p._2)
val comparisons = stepsLower("conditional").asInstanceOf[List[Map[String, Any]]]
Now I need to parse data under the conditional (in comparisons) by evaluating the structure.
The result should be ((c11 OR c12 OR c13 ) AND (c21 OR c22)) OR (c31 OR c32 OR c33 OR c34)
Here is the code snippet that should work for all your use cases.
case class Step(id: Option[String], stepId: Option[String], parameters: Option[Map[String, Any]])
val sample1 = Step(id = Some("a1"), Some("case"), Some(Map(
"CONDITIONAL" -> List(
Map("comparison" -> List(
Map("comparison" -> List(
Map("comparison" -> List(
Map("comparator" -> List(
Map(
"eval" -> "c11",
"operation" -> "equals to",
"when" -> List("A")),
Map(
"eval" -> "c12",
"operation" -> "in",
"when" -> List("B")),
Map(
"eval" -> "c13",
"operation" -> "starts with",
"when" -> List("C")
)),
"operator" -> "OR"
),
Map("comparator" -> List(
Map(
"eval" -> "c21",
"operation" -> "equals to",
"when" -> List("A")),
Map(
"eval" -> "c22",
"operation" -> "in",
"when" -> List("B")
)),
"operator" -> "OR"
)),
"operator" -> "AND"
),
Map("comparator" -> List(
Map(
"eval" -> "c31",
"operation" -> "equals to",
"when" -> List("A")),
Map(
"eval" -> "c32",
"operation" -> "in",
"when" -> List("B")),
Map(
"eval" -> "c33",
"operation" -> "starts with",
"when" -> List("C")
)),
"operator" -> "OR"
)),
"operator" -> "OR"
)),
"then" -> "result"
)),
"else" -> "default"
))
)
val sample2 = Step(id = Some("a1"), Some("case"), Some(
Map(
"conditional" -> List(
Map("comparison" -> List(
Map("comparison" -> List(
Map("comparator" -> List(
Map(
"eval" -> "c11",
"operation" -> "equals to",
"when" -> List("A")),
Map(
"eval" -> "c12",
"operation" -> "in",
"when" -> List("B", "b"))),
"operator" -> "AND"),
Map("comparator" -> List(
Map("eval" -> "c21",
"operation" -> "equals to",
"when" -> List("A")),
Map("eval" -> "c31",
"operation" -> "equals to",
"when" -> List("A"))),
"operator" -> "OR")),
"operator" -> "OR")),
"then" -> "something"
),
Map("comparison" -> List(
Map("comparison" -> List(
Map("comparator" -> List(
Map(
"eval" -> "c111",
"operation" -> "equals to",
"when" -> List("A")),
Map(
"eval" -> "c121",
"operation" -> "in",
"when" -> List("B", "b"))
),
"operator" -> "AND"),
Map("comparator" -> List(
Map(
"eval" -> "c212",
"operation" -> "equals to",
"when" -> List("A")),
Map(
"eval" -> "c313",
"operation" -> "equals to",
"when" -> List("A"))
),
"operator" -> "OR")),
"operator" -> "AND")),
"then" -> "somethingelse")
),
"else" -> "default"
)
))
def constructCase(c: List[Map[String, Any]], parentOp: Option[String] = None, parentThen: Option[String] = None): String = {
val result = c.map { x =>
val currentOp: Option[String] = x.get("operator").asInstanceOf[Option[String]]
val currentThen: Option[String] = x.get("then").asInstanceOf[Option[String]]
if (x contains "comparator") {
"(" + x("comparator").asInstanceOf[List[Map[String, Any]]].map(x => s"${x("eval")} ${x("operation")} ${x("when").asInstanceOf[List[String]].mkString(",")}").mkString(" " + currentOp.getOrElse("") + " ") + ")"
} else {
val tmp = constructCase(x("comparison").asInstanceOf[List[Map[String, Any]]], currentOp, currentThen)
if (tmp contains "WHEN") tmp else "(" + tmp + ")"
}
}.mkString(if (parentOp.isDefined) s" ${parentOp.get} " else "")
if (parentThen.isDefined) s"WHEN $result THEN ${parentThen.get}\n" else result.trim
}
val samples = Set(sample1, sample2)
samples.foreach { sample =>
val stepsLower = sample.parameters.get.map(p => p._1.toLowerCase -> p._2)
val comparisons = stepsLower("conditional").asInstanceOf[List[Map[String, Any]]]
println("CASE \n" + constructCase(comparisons) + s"\nELSE ${stepsLower("else")}")
println("")
}
Here is the result
CASE
WHEN (((c11 equals to A OR c12 in B OR c13 starts with C) AND (c21 equals to A OR c22 in B)) OR (c31 equals to A OR c32 in B OR c33 starts with C)) THEN result
ELSE default
CASE
WHEN ((c11 equals to A AND c12 in B,b) OR (c21 equals to A OR c31 equals to A)) THEN something
WHEN ((c111 equals to A AND c121 in B,b) AND (c212 equals to A OR c313 equals to A)) THEN somethingelse
ELSE default
Let me know if it doesn't work for any of your use case.
// Reads Json file
val input_file = ("\\path\\to\\MyNew.json");
val json_content = scala.io.Source.fromFile(input_file).mkString
// parsing the json file
val details = JSON.parseFull(json_content)
// checking the matched result
details match {
case mayBeList: Some[Map[String, Any]] =>
val z = mayBeList.get.tails.toSet.flatten
z.foreach(println)
case None => println("Parsing failed")
case other => println("Unknown data structure: " + other)
}
getting following Output:
Map(Name -> Harish, Company -> In Equity, Sal -> 50000)
Map(Name -> Veer, Company -> InOut, Sal -> 20000)
Map(Name -> Zara, Company -> InWhich, Sal -> 90000)
Map(Name -> Singh, Company -> InWay, Sal -> 30000)
Map(Name -> Chandra, Company -> InSome, Sal -> 60000)
Expected Output
Harish, In Quality, 50000- (only values of Map)
Use .values for the values and .keys for the keys.
val m: Map[String, Int] = Map("a" -> 1, "b" -> 2)
m.values // res0: Iterable[Int] = MapLike(1, 2)
m.keys // res1: Iterable[String] = Set(a, b)
All you need is to iterate though the elements of your list i.e. z and extract values from each map like this,
List(Map("Name" -> "Harish", "Company" -> "In Equity", "Sal" -> 50000),
Map("Name" -> "Veer", "Company" -> "InOut", "Sal" -> 20000),
Map("Name" -> "Zara", "Company" -> "InWhich", "Sal" -> 90000),
Map("Name" -> "Singh", "Company" -> "InWay", "Sal" -> 30000),
Map("Name" -> "Chandra", "Company" -> "InSome", "Sal" -> 60000)
)
.map(_.values.toList).foreach(println)
//List[List[Any]] = List(List(Harish, In Equity, 50000), List(Veer, InOut, 20000), List(Zara, InWhich, 90000), List(Singh, InWay, 30000), List(Chandra, InSome, 60000))
Hope this helps you.
Update
In response to your comment, use this code
import scala.util.parsing.json._
val input_file = ("C:\\Users\\VishalK\\IdeaProjects\\ScalaCassan\\src\\main\\scala\\MyNew.json");
val json_content = scala.io.Source.fromFile(input_file)
// parsing the json file
val details: Option[Any] = JSON.parseFull(json_content.mkString)
details match {
case mayBeList: Some[Any] =>
mayBeList.getOrElse(Seq.empty[Map[String, Any]]).asInstanceOf[List[Map[String, Any]]].map(_.values.toList).toSet
case None => println("Parsing failed")
}
in your match block :
In first case I don't get why you are using .tails.toSet.flatten on Any data type.
You can remove the third case as Some and None are the only possible outcomes of Option data-type.
scala> val l = List(Map("Name" -> "Harish", "Company" -> "In Equity", "Sal" -> 50000),
| Map("Name" -> "Veer", "Company" -> "InOut", "Sal" -> 20000),
| Map("Name" -> "Zara", "Company" -> "InWhich", "Sal" -> 90000),
| Map("Name" -> "Singh", "Company" -> "InWay", "Sal" -> 30000),
| Map("Name" -> "Chandra", "Company" -> "InSome", "Sal" -> 60000)
| )
l: List[scala.collection.immutable.Map[String,Any]] = List(Map(Name -> Harish, Company -> In Equity, Sal -> 50000), Map(Name -> Veer, Company -> InOut, Sal -> 20000), Map(Name -> Zara, Company -> InWhich, Sal -> 90000), Map(Name -> Singh, Company -> InWay, Sal -> 30000), Map(Name -> Chandra, Company -> InSome, Sal -> 60000))
scala> l.map(_.values).foreach(x => println(x.toList.mkString(", ")))
Harish, In Equity, 50000
Veer, InOut, 20000
Zara, InWhich, 90000
Singh, InWay, 30000
Chandra, InSome, 60000
I'm new to elixir and I want to parse a json file. One of the parts is a question answer array of objects.
[
{
"questionId":1,
"question":"Information: Personal Information: First Name",
"answer":"Joe"
},
{
"questionId":3,
"question":"Information: Personal Information: Last Name",
"answer":"Smith"
},
...
]
I know what questionId's I want and I'm going to make a map for 1 = First Name, 2 = Last Name.
But currently I'm doing the following to put the data into the struct.
defmodule Student do
defstruct first_name: nil, last_name: nil, student_number: nil
defguard is_first_name(id) when id == 1
defguard is_last_name(id) when id == 3
defguard is_student_number(id) when id == 7
end
defmodule AFMC do
import Student
#moduledoc """
Documentation for AFMC.
"""
#doc """
Hello world.
## Examples
iex> AFMC.hello
:world
"""
def main do
get_json()
|> get_outgoing_applications
end
def get_json do
with {:ok, body} <- File.read("./lib/afmc_import.txt"),
{:ok,body} <- Poison.Parser.parse(body), do: {:ok,body}
end
def get_outgoing_applications(map) do
{:ok,body} = map
out_application = get_in(body,["outgoingApplications"])
Enum.at(out_application,0)
|> get_in(["answers"])
|> get_person
end
def get_person(answers) do
student = Enum.reduce(answers,%Student{},fn(answer,acc) ->
if Student.is_first_name(answer["questionId"]) do
acc = %{acc | first_name: answer["answer"]}
end
if Student.is_last_name(answer["questionId"]) do
acc = %{acc | last_name: answer["answer"]}
end
if Student.is_student_number(answer["questionId"]) do
acc = %{acc | student_number: answer["answer"]}
end
acc
end)
IO.inspect "test"
s
end
end
I'm wondering what is a better way to do get_person with out having to do if statements. If I know I will be mapping 1 to questionId 1 in the array of objects.
The data will then be saved into a DB.
Thanks
I'd store a mapping of id to field name. With that you don't need any if inside the reduce. Some pattern matching will also make it unnecessary to do answer["questionId"] etc.
defmodule Student do
defstruct first_name: nil, last_name: nil, student_number: nil
#fields %{
1 => :first_name,
3 => :last_name,
7 => :student_number
}
def parse(answers) do
Enum.reduce(answers, %Student{}, fn %{"questionId" => id, "answer" => answer}, acc ->
%{acc | #fields[id] => answer}
end)
end
end
IO.inspect(
Student.parse([
%{"questionId" => 1, "question" => "", "answer" => "Joe"},
%{"questionId" => 3, "question" => "", "answer" => "Smith"},
%{"questionId" => 7, "question" => "", "answer" => "123"}
])
)
Output:
%Student{first_name: "Joe", last_name: "Smith", student_number: "123"}
Edit: to skip ids not present in the map, change:
%{acc | #fields[id] => answer}
to:
if field = #fields[id], do: %{acc | field => answer}, else: acc
I have a map in scala like this.
val someData = Some(Map(genderKey -> gender,agekey -> age))
How to get the output as:
val key= genderkey
val value= gender
val key2 = agekey (Dynamic variable name)
val value2= age (Dynamic variable name)
Like this
someData.map(_.map {
case (k,v) => s"$k = $v"
}
.mkString(" and \n"))
.foreach(result => println(result))
I have RDD[Row] :
|---itemId----|----Country-------|---Type----------|
| 11 | US | Movie |
| 11 | US | TV |
| 101 | France | Movie |
How to do GroupBy itemId so that I can save the result as List of json where each row is separate json object(each row in RDD) :
{"itemId" : 11,
"Country": {"US" :2 },"Type": {"Movie" :1 , "TV" : 1} },
{"itemId" : 101,
"Country": {"France" :1 },"Type": {"Movie" :1} }
RDD :
I tried :
import com.mapping.data.model.MappingUtils
import com.mapping.data.model.CountryInfo
val mappingPath = "s3://.../"
val input = sc.textFile(mappingPath)
The input is list of jsons where each line is json which I am mapping to the POJO class CountryInfo using MappingUtils which takes care of JSON parsing and conversion:
val MappingsList = input.map(x=> {
val countryInfo = MappingUtils.getCountryInfoString(x);
(countryInfo.getItemId(), countryInfo)
}).collectAsMap
MappingsList: scala.collection.Map[String,com.mapping.data.model.CountryInfo]
def showCountryInfo(x: Option[CountryInfo]) = x match {
case Some(s) => s
}
val events = sqlContext.sql( "select itemId EventList")
val itemList = events.map(row => {
val itemId = row.getAs[String](1);
val çountryInfo = showTitleInfo(MappingsList.get(itemId));
val country = if (countryInfo.getCountry() == 'unknown)' "US" else countryInfo.getCountry()
val type = countryInfo.getType()
Row(itemId, country, type)
})
Can some one let me know how can I achieve this ?
Thank You!
I can't afford the extra time to complete this, but can give you a start.
The idea is that you aggregate the RDD[Row] down into a single Map that represents your JSON structure. Aggregation is a fold that requires two function parameters:
seqOp How to fold a collection of elements into the target type
combOp How to merge two of the target types.
The tricky part comes in combOp while merging, as you need to accumulate the counts of values seen in the seqOp. I have left this as an exercise, as I have a plane to catch! Hopefully someone else can fill in the gaps if you have trouble.
case class Row(id: Int, country: String, tpe: String)
def foo: Unit = {
val rows: RDD[Row] = ???
def seqOp(acc: Map[Int, (Map[String, Int], Map[String, Int])], r: Row) = {
acc.get(r.id) match {
case None => acc.updated(r.id, (Map(r.country, 1), Map(r.tpe, 1)))
case Some((countries, types)) =>
val countries_ = countries.updated(r.country, countries.getOrElse(r.country, 0) + 1)
val types_ = types.updated(r.tpe, types.getOrElse(r.tpe, 0) + 1)
acc.updated(r.id, (countries_, types_))
}
}
val z = Map.empty[Int, (Map[String, Int], Map[String, Int])]
def combOp(l: Map[Int, (Map[String, Int], Map[String, Int])], r: Map[Int, (Map[String, Int], Map[String, Int])]) = {
l.foldLeft(z) { case (acc, (id, (countries, types))) =>
r.get(id) match {
case None => acc.updated(id, (countries, types))
case Some(otherCountries, otherTypes) =>
// todo - continue by merging countries with otherCountries
// and types with otherTypes, then update acc
}
}
}
val summaryMap = rows.aggregate(z) { seqOp, combOp }