Creating nodes from nested JSON using neo4J query - json

I'm new in neo4j and i have this json file:
{
"locations_connections": {
"locations": [
{
"id": "aws.us-east-1",
"longitude": 72.8777,
"latitude": 19.0760
},
{
"id": "aws.us-east-2",
"longitude": 126.9780,
"latitude": 37.5665
},
{
"id": "aws.us-west-1",
"longitude": 103.8517837,
"latitude": 1.287950
}
],
"connections": [
{
"aws.us-west-1": [
{
"id": "aws.us-west-1",
"latency": 3.16,
"cost": 0.02
},
{
"id": "aws.us-east-1",
"latency": 53.47,
"cost": 0.02
},
{
"id": "aws.us-east-2",
"latency": 53.47,
"cost": 0.02
}
]
},
{
"aws.us-east-1": [
{
"id": "aws.us-east-1",
"latency": 3.16,
"cost": 0.02
},
{
"id": "aws.us-east-2",
"latency": 53.47,
"cost": 0.02
}
]
},
{
"aws.us-east-2": [
{
"id": "aws.us-east-2",
"latency": 53.47,
"cost": 0.02
}
]
}
]
}
}
After reading the json using the apoc.load.json(URL) procedure , what query do I write to represent this as a graph?
where the Node will contain the information name like for example aws.us-east-1, value of longitude and value of latitude and the edges will have the latency and the cost
I have this code:
call apoc.load.json("/file.json") yield value
UNWIND value.locations_connections.locations as loc
UNWIND value.locations_connections.connections as con
MERGE (e:Element {id:loc.id}) ON CREATE
SET e.longitude = loc.longitude, e.latitude = loc.latitude
WITH con
FOREACH (region_source IN KEYS(con)|
FOREACH (data in con[region_source]|
MERGE (e1:Element {id: region_source})
MERGE (e1)<-[:CONNECT]-(e2:Element {id:data.id, latency:data.latency, cost:data.cost})
))
and the execution result is incorrect:
Added 9 labels, created 9 nodes, set 27 properties, created 6 relationships, completed after 60 ms.and I have seen this one,But this is not what I expected

You cannot use match inside a FOREACH so when you put MERGE and :CONNECT inside the for loop, it is creating multiple nodes. This is what I did and tell us if it works for you or not.
call apoc.load.json("/file.json") yield value
// read the json file
WITH value, value.locations_connections.locations as locs
// for loop to create the locations (or regions)
FOREACH (loc in locs | MERGE (e:Element {id:loc.id}) ON CREATE
SET e.longitude = loc.longitude, e.latitude = loc.latitude
)
// get the data for the connections
WITH value.locations_connections.connections as cons
UNWIND cons as con
// the keys and value are assigned to variables region and data
WITH KEYS(con)[0] as region_source, con[KEYS(con)[0]] as dat
// unwind is similar to a for loop
UNWIND dat as data
// look for the nodes that we want
MATCH (e1:Element {id: region_source}), (e2:Element {id: data.id})
// create the connection between regions
MERGE (e1)<-[:CONNECT {latency:data.latency, cost:data.cost}]-(e2)
RETURN e1, e2
See result below:

Related

Couchbase - SELECT a subset of fields from array of objects

I am using the travel-sample data set, and am running the following query:
SELECT id, schedule FROM `travel-sample`WHERE type = "route" LIMIT 1;
It is returning with the following results:
[
{
"id": 10000,
"schedule": [
{
"day": 0,
"flight": "AF198",
"utc": "10:13:00"
},
{
"day": 0,
"flight": "AF547",
"utc": "19:14:00"
},
...
]
}
]
However, I don't want to return the schedule.$.day field; i.e. I want my results to be:
[
{
"id": 10000,
"schedule": [
{
"flight": "AF198",
"utc": "10:13:00"
},
{
"flight": "AF547",
"utc": "19:14:00"
},
...
]
}
]
How can I SELECT only a subset of object fields from an array of objects?
I have tried UNNEST but I don't want to have a separate record for each schedule element - I want the schedule elements to remain nested inside the document.
I have also tried using OBJECT_REMOVE
SELECT id, ARRAY OBJECT_REMOVE(x, 'day') FOR x in schedule END AS schedule FROM `travel-sample` WHERE type = "route" LIMIT 1;
But I want to whitelist rather than blacklist fields.
Your last attempt was close. Instead of using OBJECT_REMOVE, you can simply construct the object you want returned.
SELECT id, ARRAY {"flight": x.flight, "utc": x.utc} FOR x in schedule END AS schedule FROM `travel-sample` WHERE type = "route" LIMIT 1;
You will get the following results:
[
{
"id": 10000,
"schedule": [
{
"flight": "AF198",
"utc": "10:13:00"
},
{
"flight": "AF547",
"utc": "19:14:00"
},
...
]
}
]

DocumentDB get min of subarray

my json data:
[
{
"Code": "GB-00001",
"BasicInformation": {
"WGS84Longitude": -4.670000,
"WGS84Latitude": 50.340000
},
"Availability": [{
"ArrivalDate": "2017-04-21",
"Price": 689
},
{
"ArrivalDate": "2017-04-28",
"Price": 1341
}
]},
{
"Code": "GB-00002",
"BasicInformation": {
"WGS84Longitude": -4.680000,
"WGS84Latitude": 50.350000
},
"Availability": [{
"ArrivalDate": "2017-04-21",
"Price": 659
},
{
"ArrivalDate": "2017-04-28",
"Price": 1440
}
]}
}]
I'd like the result to be like:
[
{
"HouseCode": "GB-00001",
"Country": "GB",
"location": {
"type": "Point",
"coordinates": [
50.340000,
-4.670000
]
}, "lowestPrice": 689
},
{
"HouseCode": "GB-00002",
"Country": "GB",
"location": {
"type": "Point",
"coordinates": [
50.350000,
-4.680000
]
}, "lowestPrice" : 659
}
My problem is: how to use the min(c.Availability.Price)
This is my current query with the lat lng convert to point, but no idea how to get the minimum/lowest price.
SELECT c.Code, c.BasicInformation.Country ,
{"type":"Point","coordinates": [c.BasicInformation.Latitude, c.BasicInformation.Longitude]} as location
FROM c
already tried with Join c.Availability a and , min(a.Price)
edit perhaps I am too early? https://feedback.azure.com/forums/263030-documentdb/suggestions/18561901-add-group-by-support-for-aggregate-functions
found that url in https://stackoverflow.com/a/42697673/169714
This is a pretty close to ideal situation for a user defined function (UDF).
Here is one that should do the trick:
function lowestPrice(availability) {
var i, len, lowest, row;
lowest = 2e308;
for (i = 0, len = availability.length; i < len; i++) {
row = availability[i];
lowest = Math.min(lowest, row.Price);
}
return lowest;
};
You call it like this:
SELECT
c.Code,
c.BasicInformation.Country,
{"type":"Point","coordinates": [
c.BasicInformation.Latitude, c.BasicInformation.Longitude
]} as location,
udf.lowestPrice(c.Availability) as lowestPrice
FROM c
AFAIK, you could only use UDF to achieve your requirement for now. Also, I have checked the code provided by Larry Maccherone, and it could both work on Azure DocumentDB service and my DocumentDB Emulator (version 1.11.136.2) as follows:
DocumentDB.GatewayService.exe has stopped working
For DocumentDB.GatewayService crash, I assumed that you need to collect the dump files and attach them with an email to askdocdb#microsoft.com. For more details, you could refer to DocumentDB Emulator troubleshooting.

Insert into existing map a map structure in DynamoDB using Nodejs

Structure of an item in database is as shown below:
{
"cars": {
"x": [
{
"time": 1485700907669,
"value": 23
}
]
},
"date": 1483214400000,
"id":"1"
}
I have to add a new item "z" of type list to cars like
{
"cars": {
"x": [
{
"time": 1485700907669,
"value": 23
}
],
"z": [
{
"time": 1485700907669,
"value": 23
}
]
},
"date": 1483214400000,
"id": "1"
}
What would the update expression in Node.js look like if I want to achieve somethings like this?
So far this is what I came up with:
set #car.#model= list_append(if_not_exists(#car.#model, :empty_list), :value)
However, if the item does not exist at the time of creation it throws error. Any idea how to do this?
This is the updated parameter I am using, still doesn't work
var params = {
TableName:table,
Key:{
"id": id,
"date": time.getTime()
},
ReturnValues: 'ALL_NEW',
UpdateExpression: 'SET #car.#model = if_not_exists(#car.#model,
:empty_list)',
ExpressionAttributeNames: {
'#car': 'cars',
'#model':"z"
},
ExpressionAttributeValues: {
':empty_list': [],
}
};
The solution is to update operation in two steps, first create a empty map for the parent since it does not exist in the first place.
So, in my case
SET #car= :empty_map
where :empty_map = {}
after doing this run the other update expression
SET #car.#model = list_append(if_not_exists(#car.#model, :empty_list), :value)
where :empty_list=[] and :value= {
"time": 1485700907669,
"value": 23
}
Break your update expression apart into two separate expressions:
SET #car.#model = if_not_exists(#car.#model, :empty_list) SET #car.#model = list_append(#car.#model, :value)

How import CSV using JSON in OrientDB using ID from one table and the values of another?

I have 3 CSV files: one for node A, one for node B and one for edge A_to_B.
I can import the nodes into OrientDB just fine. It's the edges that I'm having problems with.
node a's CSV file contains (where id is integer index):
id, value
0, a
1, b
2, c
3, d
...
node b's CSV file contains (where id is a integer index):
id, Category
10, cat_x
11, cat_y
12, cat_z
edge_a_b CSV contains:
a_id, Category
0, cat_x
1, cat_z
2, cat_z
...
I can get the two "a" and "b" properly into the database. However, when I run this ETL json...
{
"source": { "file": { "path": "/mypath/edge_a_b.csv" } },
"extractor": { "csv": {} },
"transformers": [
{ "vertex": { "class": "b", "skipDuplicates": true } },
{ "edge": { "class": "Involves", "joinFieldName": "a_id", "lookup": "a.id", "direction": "in" } }
],
"loader": {
"orientdb": {
"dbURL": "plocal:../databases/mydb",
"dbType": "graph",
"classes": [
{"name": "a", "extends": "V"},
{"name": "b", "extends": "V"},
{"name": "Involves", "extends": "E"}
], "indexes": [
{"class":"a", "fields":["id:integer"], "type":"UNIQUE" },
{"class":"b", "fields":["id:integer"], "type":"UNIQUE" }
]
}
}
}
I only get 1 of the 215 vertices I'm expecting matched.
| => ./oetl.sh /mypath/edge_a_b.json
OrientDB etl v.2.2.11 (build 2.2.x#r8b3a478e3ca7321a48e7cf0f5991569bbe06ed89; 2016-10-03 09:39:41+0000) www.orientdb.com
BEGIN ETL PROCESSOR
[file] INFO Reading from file /mypath/edge_a_b.csv with encoding UTF-8
Started execution with 1 worker threads
[orientdb] INFO committing
END ETL PROCESSOR
+ extracted 215 rows (0 rows/sec) - 215 rows -> loaded 1 vertices (0 vertices/sec) Total time: 172ms [0 warnings, 0 errors]
I have the nodes created. It's the edges that I'm finding difficult to create. I've tried various approaches.
You could use
"extractor": {"row": {}},
"transformers": [{
"csv": {
"separator": ","
}
},
{
"command" : {
"command" : "create edge Involves from (select from a where id= ${input.a_id}) to (select from b where Category= '${input.Category}')",
"output" : "edge"
}
}
],
Hope it helps.

How to search nested JSON in MySQL

I am using MySQL 5.7+ with the native JSON data type. Sample data:
[
{
"code": 2,
"stores": [
{
"code": 100,
"quantity": 2
},
{
"code": 200,
"quantity": 3
}
]
},
{
"code": 4,
"stores": [
{
"code": 300,
"quantity": 4
},
{
"code": 400,
"quantity": 5
}
]
}
]
Question: how do I extract an array where code = 4?
The following (working) query has the position of the data I want to extract and the search criterion hardcoded:
SELECT JSON_EXTRACT(data_column, '$[0]')
FROM json_data_table
WHERE data_column->'$[1].code' = 4
I tried using a wildcard (data_column->'$[*].code' = 4) but I get no results in return.
SELECT row FROM
(
SELECT data_column->"[*]" as row
FROM json_data_table
WHERE 4 IN JSON_EXTRACT(data_column, '$[*].code')
)
WHERE row->".code" = 4
... though this would be much easier to work with if this wasn't an unindexed array of objects at the top level. You may want to consider some adjustments to the schema.
Note:
If you have multiple rows in your data, specifying "$[i]" will pick that row, not the aggregate of it. With your dataset, "$[1].code" will always evaluate to the value of code in that single row.
Essentially, you were saying:
$ json collection
[1] second object in the collection.
.code attribute labeled "code".
... since there will only ever be one match for that query, it will always eval to 4...
WHERE 4 = 4
Alternate data structure if possible
Since the entire purpose of "code" is as a key, make it the key.
[
"code2":{
"stores": [
{
"code": 100,
"quantity": 2
},
{
"code": 200,
"quantity": 3
}
]
},
"code4": {
"stores": [
{
"code": 300,
"quantity": 4
},
{
"code": 400,
"quantity": 5
}
]
}
]
Then, all it would require would be:
SELECT datacolumn->"[code4]" as code4
FROM json_data_table
This is what you are looking for.
SELECT data_column->'$[*]' FROM json_data_table where data_column->'$[*].code' like '%4%'.
The selected data will have [] around it when selecting from an array thus data_column->'$[*].code' = 4 is not possible.