I have a json like this:
[
{
"ser": 345,
"City": "New York",
"Gen": 1
},
{
"ser": 55,
"City": "New York",
"Gen": 2
},
{
"ser": 19,
"City": "New York",
"Gen": 3
}
]
I want to make some query to this json ( example: SELECT SUM(ser) AS Serlab, City, Gen FROM libri GROUP BY City, Gen ORDER BY City)...i need spark rdd to make this.
After this query i should modify the structure of my input json in this new:
[ { "label": [1,2,3] //that is the Gen value,
"values": [
{ "label":"New York"
"values":["200","10","66"] //is the ser about all Gen (for eexample 200 is ser is for label/gen 1 .. 10 is for label/gen 2 etc.
},
{ "label":"London"
"values":["500","150","46"]
},
{ "label":"London"
"values":["500","150","46"]
},
.
.
.
.
.
]
}
]
And then i send this new JSON to a Http request
How can i make this in scala,play,spark? Please help me :)
Related
I'm new in neo4j and i have this json file:
{
"locations_connections": {
"locations": [
{
"id": "aws.us-east-1",
"longitude": 72.8777,
"latitude": 19.0760
},
{
"id": "aws.us-east-2",
"longitude": 126.9780,
"latitude": 37.5665
},
{
"id": "aws.us-west-1",
"longitude": 103.8517837,
"latitude": 1.287950
}
],
"connections": [
{
"aws.us-west-1": [
{
"id": "aws.us-west-1",
"latency": 3.16,
"cost": 0.02
},
{
"id": "aws.us-east-1",
"latency": 53.47,
"cost": 0.02
},
{
"id": "aws.us-east-2",
"latency": 53.47,
"cost": 0.02
}
]
},
{
"aws.us-east-1": [
{
"id": "aws.us-east-1",
"latency": 3.16,
"cost": 0.02
},
{
"id": "aws.us-east-2",
"latency": 53.47,
"cost": 0.02
}
]
},
{
"aws.us-east-2": [
{
"id": "aws.us-east-2",
"latency": 53.47,
"cost": 0.02
}
]
}
]
}
}
After reading the json using the apoc.load.json(URL) procedure , what query do I write to represent this as a graph?
where the Node will contain the information name like for example aws.us-east-1, value of longitude and value of latitude and the edges will have the latency and the cost
I have this code:
call apoc.load.json("/file.json") yield value
UNWIND value.locations_connections.locations as loc
UNWIND value.locations_connections.connections as con
MERGE (e:Element {id:loc.id}) ON CREATE
SET e.longitude = loc.longitude, e.latitude = loc.latitude
WITH con
FOREACH (region_source IN KEYS(con)|
FOREACH (data in con[region_source]|
MERGE (e1:Element {id: region_source})
MERGE (e1)<-[:CONNECT]-(e2:Element {id:data.id, latency:data.latency, cost:data.cost})
))
and the execution result is incorrect:
Added 9 labels, created 9 nodes, set 27 properties, created 6 relationships, completed after 60 ms.and I have seen this one,But this is not what I expected
You cannot use match inside a FOREACH so when you put MERGE and :CONNECT inside the for loop, it is creating multiple nodes. This is what I did and tell us if it works for you or not.
call apoc.load.json("/file.json") yield value
// read the json file
WITH value, value.locations_connections.locations as locs
// for loop to create the locations (or regions)
FOREACH (loc in locs | MERGE (e:Element {id:loc.id}) ON CREATE
SET e.longitude = loc.longitude, e.latitude = loc.latitude
)
// get the data for the connections
WITH value.locations_connections.connections as cons
UNWIND cons as con
// the keys and value are assigned to variables region and data
WITH KEYS(con)[0] as region_source, con[KEYS(con)[0]] as dat
// unwind is similar to a for loop
UNWIND dat as data
// look for the nodes that we want
MATCH (e1:Element {id: region_source}), (e2:Element {id: data.id})
// create the connection between regions
MERGE (e1)<-[:CONNECT {latency:data.latency, cost:data.cost}]-(e2)
RETURN e1, e2
See result below:
i'm trying to convert the inspection_date field from string to date for every object inside my db.
Every object is built like this one.
"name": "$1 STORE",
"address": "5573 ROSEMEAD BLVD",
"city": "TEMPLE CITY",
"zipcode": "91780",
"state": "California",
"violations": [{
"inspection_date": "2015-09-29",
"description": " points ... violation_status\n62754 1 ... OUT OF COMPLIANCE\n62755 1 ... OUT OF COMPLIANCE\n62756 2 ... OUT OF COMPLIANCE\n\n[3 rows x 5 columns]",
"risk": "Risk 3 (Low)"
}, {
"inspection_date": "2016-08-18",
"description": " points ... violation_status\n338879 2 ... OUT OF COMPLIANCE\n\n[1 rows x 5 columns]",
"risk": "Risk 3 (Low)"
} //could be more than 2 or less then 2 object inside violations array//]}
How can i convert all of the inspection_date field avoiding doing it by myself one by one?
As suggested by #turivishal, you have to have to make use of $map and $dateFromString operators.
db.collection.aggregate([
{
"$addFields": {
"violations": {
"$map": {
"input": "$violations",
"in": {
"$mergeObjects": [
"$$this",
{
"inspection_date": {
"$dateFromString": {
"dateString": "$$this.inspection_date",
"format": "%Y-%m-%d",
"onError": null,
"onNull": null
}
}
}
],
},
}
}
}
},
])
Mongo Playground Sample Execution
The question is to write a JSONiq FLWOR expression that can display the name of the products which their price are at least 3.
I have tried the answers provided on How to run JSONiq from JSON with try.zorba.io but that's not the answers that i expect. Beside that, I also tried a lot of JSON FLWOR expression but still getting errors in try.zobia.io. This is my JSON file.
{
"supermarket": {
"visit": [ {
"_type": "bought",
"date": "March 8th, 2019",
"product": [ {
"name": "Kit Kat",
"amount": 3,
"cost": 3.5
},
{
"name": "Coca Cola",
"amount": 2,
"cost": 3
},
{
"name": "Apple",
"amount": "Some",
"cost": 5.9
}
]
},
{
"_type": "planning",
"product": [{
"name": "Pen",
"amount": 2
},
{
"name": "Paper",
"amount": "One ream"
}
]
}
]
}
}
This is my current JSONiq expression.
jsoniq version "1.0";
let $a := { (: my JSON file :) }
for $x in $a.supermarket.visit
let $y = $x.product()
where $y.price >= "3.0"
return $y.name
The final output should be Kit Kat, Coca Cola and Apple. I would appreciate some helps for my JSON file or JSONiq.
visit also is an array, so you need the parenthesis to get to the individual visits in the for. This
jsoniq version "1.0";
let $data := { (: my JSON file :) }
for $visit in $data.supermarket.visit()
for $product in $visit.product()
where $product.cost ge 3
return $product.name
would return
Kit Kat Coca Cola Apple
Since the above produces a sequence, you can use it anywhere where sequences are allowed.
let $data := { (: my JSON file :) }
return string-join(
for $visit in $data.supermarket.visit()
for $product in $visit.product()
where $product.cost ge 3
return $product.name
, ", ")
Result:
Kit Kat, Coca Cola, Apple
Of course this would work as well:
for $product in $data.supermarket.visit().product()
where $product.cost ge 3
return $product.name
I am trying to parse a JSON response that has repeating objects with JsonSlurper to compare to a JDBC query. However, I only want to compare objects where a certain values exist within that object.
If I had a response that looks like this, how would I only parse the objects where the country equals USA or Canada, therefore ignoring anything else?
{
"info": [{
"name": "John Smith",
"phone": "2125557878",
"country": {
"value": "USA"
}
},
{
"name": "Jane Smith",
"phone": "2125551212",
"country": {
"value": "USA"
}
},
{
"name": "Bob Jones",
"phone": "4165558714",
"country": {
"value": "Canada"
}
},
{
"name": "George Tucker",
"phone": "4454547171",
"country": {
"value": "UK"
}
},
{
"name": "Jean Normand",
"phone": "4454547171",
"country": {
"value": "France"
}
}]
}
This is what I have in groovy:
def jsonResponse = context.expand('${RESTRequest#Response}')
def parsedJson = new JsonSlurper().parseText(jsonResponse)
def info = parsedJson.info
def jsonDataObjects = []
info.each { json ->
jsonDataObjects.add(Model.buildJSONData(json))
}
I am building a collection of the elements that I need to compare to a database. How do I only add to that collection where the info.country.value = USA or Canada?
I tried using .findAll like this just to test if I could get it to filter by just one of the countries:
def info = parsedJson.info.country.findAll{it.value == "USA"}
But, when I do that, only the value field is kept. I lose the name and phone from the parse.
Thanks in advance for any assistance.
Did you try
def info = parsedJson.info.findAll{it.country.value == "USA"}
?
This is the json object I am working with
{
"name": "John Smith",
"age": 32,
"employed": true,
"address": {
"street": "701 First Ave.",
"city": "Sunnyvale, CA 95125",
"country": "United States"
},
"children": [
{
"name": "Richard",
"age": 7
},
{
"name": "Susan",
"age": 4
},
{
"name": "James",
"age": 3
}
]
}
I want this as another key-value pair :
"collegeId": {
"eventno": "6062",
"eventdesc": "abc"
};
I tried concat but that gave me the result with || symbol and I cdnt iterate. I used spilt but that removes only commas.
concattedjson = JSON.stringify(JSON.parse(json1).concat(JSON.parse(json2)));
How do I add a key pair value to an existing json object ?
I am working in javascript.
This is the easiest way and it's working to me.
var testJson = {
"name": "John Smith",
"age": 32,
"employed": true,
"address": {
"street": "701 First Ave.",
"city": "Sunnyvale, CA 95125",
"country": "United States"
},
"children": [
{
"name": "Richard",
"age": 7
},
{
"name": "Susan",
"age": 4
},
{
"name": "James",
"age": 3
}
]
};
testJson.collegeId = {"eventno": "6062","eventdesc": "abc"};
Just convert the JSON string to an object using JSON.parse() and then add the property. If you need it back into a string, do JSON.stringify().
BTW, there's no such thing as a JSON object. There are objects, and there are JSON strings that represent those objects.
You need to make an object at reference "collegeId", and then for that object, make two more key value pairs there like this:
var concattedjson = JSON.parse(json1);
concattedjson["collegeId"] = {};
concattedjson["collegeId"]["eventno"] = "6062";
concattedjson["collegeId"]["eventdesc"] = "abc";
Assuming that concattedjson is your json object. If you only have a string representation you will need to parse it first before you extend it.
Edit
demo for those who think this will not work.
const newTestJson = JSON.parse(JSON.stringify(testJson));
newTestJson.collegeId = {"eventno": "6062","eventdesc": "abc"};
testJson = newTestJson;