Can anyone help to convert a list to list of Maps in Groovy ?
I have a list as below
[war, jar]
and a variable with name "value" = 2.0.0
I want to convert this into a list with maps [["pack": "war", "ver": "2.0.0"],["pack": jar, "ver": "2.0.0"]]
and create a json.
{
"ProjectId": "Projects-16",
"ChannelId": "Channels-41",
"Version": "2.0.1.0-10",
"selectedPackages": [{"pack": "war", "ver": "2.0.0"}, {"pack": jar, "ver": "2.0.0"]}]
}
Here is the solution:
import groovy.json.JsonOutput
def packages = ["war", "jar"]
def value = "2.0.0"
def list = packages.collect { [pack : it, ver : value] }
def object = [
ProjectId: "Projects-16",
ChannelId: "Channels-41",
Version: "2.0.1.0-10",
selectedPackages: list
]
def json = JsonOutput.toJson(object)
println JsonOutput.prettyPrint(json)
Each {} entity in JSON is a map in Groovy therefore the expected [{}, {}] is a list of maps. You need just to construct the right objects using collect with the right transformation.
Edit
I updated the solution to return the exact json output as desired using a list and the variable value.
The output is this:
{
"ProjectId": "Projects-16",
"ChannelId": "Channels-41",
"Version": "2.0.1.0-10",
"selectedPackages": [
{
"pack": "war",
"ver": "2.0.0"
},
{
"pack": "jar",
"ver": "2.0.0"
}
]
}
Related
Im using Spark 2.4.3 and Scala 2.11
Below is my current JSON string in a DataFrame column.
Im trying to store the schema of this JSON string in another column using schema_of_json function.
But its throwing below the error. How could I resolve this?
{
"company": {
"companyId": "123",
"companyName": "ABC"
},
"customer": {
"customerDetails": {
"customerId": "CUST-100",
"customerName": "CUST-AAA",
"status": "ACTIVE",
"phone": {
"phoneDetails": {
"home": {
"phoneno": "666-777-9999"
},
"mobile": {
"phoneno": "333-444-5555"
}
}
}
},
"address": {
"loc": "NORTH",
"adressDetails": [
{
"street": "BBB",
"city": "YYYYY",
"province": "AB",
"country": "US"
},
{
"street": "UUU",
"city": "GGGGG",
"province": "NB",
"country": "US"
}
]
}
}
}
Code:
val df = spark.read.textFile("./src/main/resources/json/company.txt")
df.printSchema()
df.show()
root
|-- value: string (nullable = true)
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|value |
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|{"company":{"companyId":"123","companyName":"ABC"},"customer":{"customerDetails":{"customerId":"CUST-100","customerName":"CUST-AAA","status":"ACTIVE","phone":{"phoneDetails":{"home":{"phoneno":"666-777-9999"},"mobile":{"phoneno":"333-444-5555"}}}},"address":{"loc":"NORTH","adressDetails":[{"street":"BBB","city":"YYYYY","province":"AB","country":"US"},{"street":"UUU","city":"GGGGG","province":"NB","country":"US"}]}}}|
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
df.withColumn("jsonSchema",schema_of_json(col("value")))
Error:
Exception in thread "main" org.apache.spark.sql.AnalysisException: cannot resolve 'schemaofjson(`value`)' due to data type mismatch: The input json should be a string literal and not null; however, got `value`.;;
'Project [value#0, schemaofjson(value#0) AS jsonSchema#10]
+- Project [value#0]
+- Relation[value#0] text
The workaround solution that I found was to pass the column value as below to the schema_of_json function.
df.withColumn("jsonSchema",schema_of_json(df.select(col("value")).first.getString(0)))
Courtesy:
Implicit schema discovery on a JSON-formatted Spark DataFrame column
Since SPARK-24709 was introduced schema_of_json accepts just literal strings. You can extract schema of String in DDL format by calling
spark.read
.json(df.select("value").as[String])
.schema
.toDDL
If one is looking for a pyspark answer :
import pyspark.sql.functions as F
import pyspark.sql.types as T
import json
def process(json_content):
if json_content is None :
return []
try:
# Parse the content of the json, extract the keys only
keys = json.loads(json_content).keys()
return list(keys)
except Exception as e:
return [e]
udf_function = F.udf(process_file, T.ArrayType(T.StringType()))
my_df = my_df.withColumn("schema", udf_function(F.col("json_raw"))
I have a JSON that I want to parse in Ruby. Ruby is completely new to me, but I have to work with it :-)
Here is my litte snippet, that should do the parsing:
response = File.read("app/helpers/example_announcement.json")
JSON.parse(response)
this works pretty fine. The only downside is, I do not know the properties at the point where I use it, it is not typesafe. So I created the objects for it
class Announcements
##announcements = Hash # a map key => value where key is string and value is type of Announcement
end
class Announcement
##name = ""
##status = ""
##rewards = Array
end
And this is how the json looks like
{
"announcements": {
"id1" : {
"name": "The Diamond Announcement",
"status": "published",
"reward": [
{
"id": "hardCurrency",
"amount": 100
}
]
},
"id2": {
"name": "The Normal Announcement",
"players": [],
"status": "published",
"reward": []
}
}
}
So I tried JSON parsing like this
response = File.read("app/helpers/example_announcement.json")
JSON.parse(response, Announcements)
But this is not how it works^^can anybody help me with this?
im trying to extract my data from json into a case class without success.
the Json file:
[
{
"name": "bb",
"loc": "sss",
"elements": [
{
"name": "name1",
"loc": "firstHere",
"elements": []
}
]
},
{
"name": "ca",
"loc": "sss",
"elements": []
}
]
my code :
case class ElementContainer(name : String, location : String,elements : Seq[ElementContainer])
object elementsFormatter {
implicit val elementFormatter = Json.format[ElementContainer]
}
object Applicationss extends App {
val el = new ElementContainer("name1", "firstHere", Seq.empty)
val el1Cont = new ElementContainer("bb","sss", Seq(el))
val source:String=Source.fromFile("src/bin/elementsTree.json").getLines.mkString
val jsonFormat = Json.parse(source)
val r1= Json.fromJson[ElementContainer](jsonFormat)
}
after running this im getting inside r1:
JsError(List((/elements,List(ValidationError(List(error.path.missing),WrappedArray()))), (/name,List(ValidationError(List(error.path.missing),WrappedArray()))), (/location,List(ValidationError(List(error.path.missing),WrappedArray())))))
been trying to extract this data forever, please advise
You have location instead loc and, you'll need to parse file into a Seq[ElementContainer], since it's an array, not a single ElementContainer:
Json.fromJson[Seq[ElementContainer]](jsonFormat)
Also, you have the validate method that will return you either errors or parsed json object..
I'm using Django REST framework to make an API.
For my serialized models I get a response looks like this:
[GET] http://12.0.0.1/0.5/barrios/?format=json
[
{
"name": "La Julia",
"zone": 1
},
{
"name": "La Floresta",
"zone": 2
}
]
But, what I want looks like this:
{
"barrios": {
"barrio": [
{
"name": "La Julia",
"zone": 1
},
{
"name": "La Floresta",
"zone": 2
}
]
}
}
Any thoughts?
You can override the list() method in your view to get the desired response.
class MyView(..):
def list(self, request, *args, **kwargs):
queryset = self.filter_queryset(self.get_queryset())
serializer = self.get_serializer(queryset, many=True)
serializer_data = serializer.data # get the default serialized representation
custom_data = {'barrios': {'barrio': serializer_data}} # custom representation
return Response(custom_data)
I'm using the next Json
{
"ID": 8,
"MenuItems": [
{
"ID": 38,
"Name": "Home",
"URL": "{\"PageLayout\":\"Home\",\"Icon\":\"home\"}",
"Culture": "en",
"Children": null
},
{
"ID": 534,
"Name": "GUIDE ",
"URL": "{''PageLayout'':''Page A'', ''Icon'':''A''}",
"MenuType": 1,
"PageID": 0,
"Culture": "en",
"Children": [
{
"ID": 6,
"Name": "Form A",
"URL": "[''Type'':''Form A'',''Icon'':''Form'',''ItemID'':\"358\"]",
"Culture": "he",
"RuleID": 0
},
{
"ID": 60,
"Name": "Drama",
"URL": "[''Type'':''Form B'',''Icon'':''Form'',''ItemID'':\"3759\"]",
"Culture": "en",
"RuleID": 0
}
]
}
]
}
i'm using Groovy script in soapUI and i need to:
Assert the exitance of node that has the name GUIDE
Extract a list of all Itemsid
You can parse the JSON content using JsonSlurper and then work with the results like so:
import groovy.json.JsonSlurper
// Assuming your JSON is stored in "jsonString"
def jsonContent = new JsonSlurper().parseText(jsonString)
// Assert node exists with name GUIDE
assert(jsonContent.MenuItems.Name.contains("GUIDE"))
// Get list of ItemIDs
def itemList = jsonContent.MenuItems.Children.URL.ItemID[0].toList()
// List the items
itemList.each {log.info it}
Note that the above will fail given your current example, because of a few issues. Firstly, Name contains "GUIDE " (trailing space) rather than "GUIDE" (so you will need to adapt accordingly). Secondly, it is invalid JSON; the URL nodes have various erroneous characters.
On a side note, if you first need to retrieve your JSON from the response associated with a previous TestStep (say one called "SendMessage") within the existing testCase, this can be done as follows:
def jsonString = context.testCase.getTestStepByName("SendMessage").testRequest.response.getContentAsString()