There's a certain web request which has the following response:
{
"data": {
"articles": [
{
"id": "1355",
"slug": "smart-device-connectivity's-impact-on-homes-workplaces",
"title": "Smart device connectivity's impact on homes, workplaces",
"published_at": "2022-01-28T21:30:00.000Z",
"avg_rating": 0,
"click_count": 60,
},
{
"id": "1363",
"slug": "you-need-to-nurture-and-amplify-human-capabilities",
"title": "You need to nurture and amplify human capabilities",
"published_at": "2022-01-28T19:00:00.000Z",
"avg_rating": 0,
"click_count": 22,
}]}}
There are a total of 702 records which may increase or decrease over the coming months. Now I have been successfully able to extract ID & slug into separate variables. My aim is to pass these two variables into another request in the following format so that I can eventually run that 702 times or number of times = ID array or slug array size:
testurl.com/insight/${id}/${slug}
Example:
testurl.com/insight/1355/smart-device-connectivity's-impact-on-homes-workplaces
testurl.com/insight/1363/you-need-to-nurture-and-amplify-human-capabilities
I made use of Foreach controller & was able to pass slug but ID does not work. Does anyone know the solution?
If you're using ForEach Controller for iterating slug variable the id one needs to be handed a little bit differently:
use __jm__ForEach Controller__idx pre-defined variable to get current iteration of the ForEach Controller
use __intSum() function to increment it by 1 as the above variable is zero-based
use __V() function to calculate the value of id_x variable
putting everything together:
testurl.com/insight/${__V(id_${__intSum(${__jm__ForEach Controller__idx},1,)},)}/${slug}
What error do you get?
I was able to emulate the same
I saved your json in a variable
Foreach controller
another JSON inside foreach
Using the extracted values
Overall JMX structure
Final output
Related
I've got a couple hundred JSONs in a structure like the following example:
{
"JsonExport": [
{
"entities": [
{
"identity": "ENTITY_001",
"surname": "SMIT",
"entityLocationRelation": [
{
"parentIdentification": "PARENT_ENTITY_001",
"typeRelation": "SEEN_AT",
"locationIdentity": "LOCATION_001"
},
{
"parentIdentification": "PARENT_ENTITY_001",
"typeRelation": "SEEN_AT",
"locationIdentity": "LOCATION_002"
}
],
"entityEntityRelation": [
{
"parentIdentification": "PARENT_ENTITY_001",
"typeRelation": "FRIENDS_WITH",
"childIdentification": "ENTITY_002"
}
]
},
{
"identity": "ENTITY_002",
"surname": "JACKSON",
"entityLocationRelation": [
{
"parentIdentification": "PARENT_ENTITY_002",
"typeRelation": "SEEN_AT",
"locationIdentity": "LOCATION_001"
}
]
},
{
"identity": "ENTITY_003",
"surname": "JOHNSON"
}
],
"identification": "REGISTRATION_001",
"locations": [
{
"city": "LONDON",
"identity": "LOCATION_001"
},
{
"city": "PARIS",
"identity": "LOCATION_002"
}
]
}
]
}
With these JSON's, I want to make a graph consisting of the following nodes: Registration, Entity and Location. This part I've figured out and made the following:
WITH "file:///example.json" AS json_file
CALL apoc.load.json(json_file,"$.JsonExport.*" ) YIELD value AS data
MERGE(r:Registration {id:data.identification})
WITH json_file
CALL apoc.load.json(json_file,"$.JsonExport..locations.*" ) YIELD value AS locations
MERGE(l:Locations{identity:locations.identity, name:locations.city})
WITH json_file
CALL apoc.load.json(json_file,"$.JsonExport..entities.*" ) YIELD value AS entities
MERGE(e:Entities {name:entities.surname, identity:entities.identity})
All the entities and locations should have a relation with the registration. I thought I could do this by using the following code:
MERGE (e)-[:REGISTERED_ON]->(r)
MERGE (l)-[:REGISTERED_ON]->(r)
However this code doesn’t give the desired output. It creates extra "empty" nodes and doesn't connect to the registration node. So the first question is: How do I connect the location and entities nodes to the registration node. And in light of the other JSON's, the entities and locations should only be linked to the specific registration.
Furthermore, I would like to make the entity -> location relation and the entity - entity relation and use the given type of relation (SEEN_AT or FRIENDS_WITH) as label for the given relation. How can this be done? I'm kind of lost at this point and don’t see how to solve this. If someone could guide me into the right direction I would be much obliged.
Variable names (like e and r) are not stored in the DB, and are bound to values only within individual queries. MERGE on a pattern with an unbound variable will just create the entire pattern (including creating an empty node for unbound node variables).
When you MERGE a node, you should only specify the unique identifying property for that node, to avoid duplicates. Any other properties you want to set at the time of creation should be set using ON CREATE SET.
It is inefficient to parse through the JSON data 3 times to get different areas of the data. And it is especially inefficient the way your query was doing it, since each subsequent CALL/MERGE group of clauses would be done multiple times (since every previous CALL produces multiple rows, and the number of rows increases multiplicative). You can use aggregation to get around that, but it is unnecessary in your case, since you can just do the entire query in a single pass through the JSON data.
This may work for you:
CALL apoc.load.json(json_file,"$.JsonExport.*" ) YIELD value AS data
MERGE(r:Registration {id:data.identification})
FOREACH(ent IN data.entities |
MERGE (e:Entities {identity: ent.identity})
ON CREATE SET e.name = ent.surname
MERGE (e)-[:REGISTERED_ON]->(r)
FOREACH(loc1 IN ent.entityLocationRelation |
MERGE (l1:Locations {identity: loc1.locationIdentity})
MERGE (e)-[:SEEN_AT]->(l1))
FOREACH(ent2 IN ent.entityEntityRelation |
MERGE (e2:Entities {identity: ent2.childIdentification})
MERGE (e)-[:FRIENDS_WITH]->(e2))
)
FOREACH(loc IN data.locations |
MERGE (l:Locations{identity:loc.identity})
ON CREATE SET l.name = loc.city
MERGE (l)-[:REGISTERED_ON]->(r)
)
For simplicity, it hard-codes the FRIENDS_WITH and REGISTERED_ON relationship types, as MERGE only supports hard-coded relationship types.
So playing with neo4j/cyper I've learned some new stuff and came to another solution for the problem. Based on the given example data, the following can create the nodes and edges dynamically.
WITH "file:///example.json" AS json_file
CALL apoc.load.json(json_file,"$.JsonExport.*" ) YIELD value AS data
CALL apoc.merge.node(['Registration'], {id:data.identification}, {},{}) YIELD node AS vReg
UNWIND data.entities AS ent
CALL apoc.merge.node(['Person'], {id:ent.identity}, {}, {id:ent.identity, surname:ent.surname}) YIELD node AS vPer1
UNWIND ent.entityEntityRelation AS entRel
CALL apoc.merge.node(['Person'],{id:entRel.childIdentification},{id:entRel.childIdentification},{}) YIELD node AS vPer2
CALL apoc.merge.relationship(vPer1, entRel.typeRelation, {},{},vPer2) YIELD rel AS ePer
UNWIND data.locations AS loc
CALL apoc.merge.node(['Location'], {id:loc.identity}, {name:loc.city}) YIELD node AS vLoc
UNWIND ent.entityLocationRelation AS locRel
CALL apoc.merge.relationship(vPer1, locRel.typeRelation, {},{},vLoc) YIELD rel AS eLoc
CALL apoc.merge.relationship(vLoc, "REGISTERED_ON", {},{},vReg) YIELD rel AS eReg1
CALL apoc.merge.relationship(vPer1, "REGISTERED_ON", {},{},vReg) YIELD rel AS eReg2
CALL apoc.merge.relationship(vPer2, "REGISTERED_ON", {},{},vReg) YIELD rel AS eReg3
RETURN vPer1,vPer2, vReg, vLoc, eLoc, eReg1, eReg2, eReg3
I am using Laravel 5.1 and I am using Illuminate\Http\Request instance for accessing sent file requests, but I can't seem to find the correct way to access the media element. I can access all other values in the object.
Here is the JSON data format I am using:
{"_method":"PUT",
"topic":"1 test",
"description":"1 test",
"media_description":"1 test",
"old_parts":
{"part-1":
{"sub_header":"test 2",
"text_field":"test 2 ",
"article_id":"18",
"media":{}},
"part-2":
{"sub_header":"test 3 ",
"text_field":"test 3",
"article_id":"18",
"media":{}}
},
"published":"1",
"media":{}
}
I access media in controller using two foreach loops, one loop is for database collection and other is for form content. I can use following method for accessing media elements.
if ($file = $request->file('media-' . $running_number)) {...}
But it decreases readability by huge amount as I should name the fields f.e. as media-1, media-2, ... and not access them using foreach loop like I do with everything else.
So is there a way to access file with foreach loop variable?
f.e. something like this
foreach($input['old_parts'] as $old_part) {
if ($old_part->hasFile('media')) { ... }
}
If I use the lines I typed the function returns a FatalThrowableError : Call to a member function hasFile() on array.
Or should I just use something like
$request->file('old_parts[part-'.$x.'][media]');
To access the file input and use custom variable to count the index? The problem is just that I can't figure out how to go inside multiple levels in JSON using file, as either old_parts['..']['..'] and old_parts['part-1']->media don't seem to return anything (return null).
I am writing a short Ruby program that is going to take a zipcode and return the names of cities within 2 miles of that zipcode. I successfully called an API and was able to parse the JSON data, but I'm unsure how to access the 'city' key.
url = API call (not going to replicate here since it requires a key)
uri = URI(url)
response = Net::HTTP.get(uri)
JSON.parse(response)
Here's what my JSON looks like.
{
"results": [
{
"zip": "08225",
"city": "Northfield",
"county": "Atlantic",
"state": "NJ",
"distance": "0.0"
},
{
"zip": "08221",
"city": "Linwood",
"county": "Atlantic",
"state": "NJ",
"distance": "1.8"
}
]
}
I've been trying to access 'city' like this:
response['result'][0]['city']
This appears to be incorrect. Also tried
response[0][0]['city']
And a couple of other permutations of the same code.
How can I get the value 'Northfield' out of the JSON data?
You're almost there, just use results instead of result on the result of JSON.parse(response) instead of on response:
JSON.parse(response)["results"][0]["city"]
#=> "Northfield"
JSON parse will create a hash then you can target the results which is an array of hashes, like so:
hash = JSON.parse(response)
hash['results'].select{|h| h['city'] == 'Northfield'}
Or if you only care about the results:
array = JSON.parse(response)['results']
array.select{|a| a['city' == 'Northfield'} #
To get just a single data point from the data, you might select one item in the array and then the key of the value you want:
array[0]['city']
For all the cities
cities = array.map{|k,v| k['city']}
You have a typo error, instead of response['result'] you can use it like response[:results].
And if you want to get the value of city key from all the hash, then response['result'][0]['city'] will not work.
After parsing response you will get an array of hashes, i.e
[{:zip=>"08225", :city=>"Northfield", :county=>"Atlantic", :state=>"NJ", :distance=>"0.0"}, {:zip=>"08221", :city=>"Linwood", :county=>"Atlantic", :state=>"NJ", :distance=>"1.8"}]
And if you want to fetch the values of key city from all the hash then you can try this steps
response[:results].map{|x| x[:city]}
which will give the result
["Atlantic", "Atlantic"]
I'm fairly new to REST. All of our legacy webservices were SOAP based with enterprise (ORACLE or DB2) databases. We are now moving to REST/couchbase.
Our team is looking into implementing a complex query method. We already have implemented simple query methods using GET, for example GET returns all entries and a GET/067e6162-3b6f-4ae2-a171-2470b63dff00 would return the entry for 067e6162-3b6f-4ae2-a171-2470b63dff00.
We want to support a query method that would support receiving several query parameters such a list of Ids and date ranges. The number of Ids can number into a few thousand and because of this, we realize we cannot pass these query parameters in a GET HTTP header since there is a limit on header size.
We are starting to look into passing our query parameters into the JSON body of a POST request. For example, we could have client pass in a few thousand Ids as an array and also pass in a date range, so we'd have each query param/filter be an object. The JSON body would then be an array of objects. For example:
{
"action" : "search",
"queryParameters" : {
[
{
“operation”: “in”,
"key" : "name.of.attribute.Id",
"value" : "[{ "id: "067e6162-3b6f-4ae2-a171-2470b63dff00"}, {"id": "next id"....}],
},
{
“operation”: “greater”,
"key" : "name.of.attribute “,
"value" : "8/20/2016"
},
{
“operation”: “less”,
"key" : "name.of.attribute “,
"value" : "8/31/2016"
}
]
}
The back end code would then receive POST and read the body. It would see action is a search and then look for any entries in the list that are in the list of Ids that are in the date range of > 8/20/2016 and < 8/31/2016.
I've been trying to look online for tips/best practices on how best to structure the JSON body for complex queries but have not found much. So any tips, guidance or advice would be greatly appreciated.
thanks.
I'm using wget to fetch several dozen JSON files on a daily basis that go like this:
{
"results": [
{
"id": "ABC789",
"title": "Apple",
},
{
"id": "XYZ123",
"title": "Orange",
}]
}
My goal is to find row's position on each JSON file given a value or set of values (i.e. "In which row XYZ123 is located?"). In previous example ABC789 is in row 1, XYZ123 in row 2 and so on.
As for now I use Google Regine to "quickly" visualize (using the Text Filter option) where the XYZ123 is standing (row 2).
But since it takes a while to do this manually for each file I was wondering if there is a quick and efficient way in one go.
What can I do and how can I fetch and do the request? Thanks in advance! FoF0
In python:
import json
#assume json_string = your loaded data
data = json.loads(json_string)
mapped_vals = []
for ent in data:
mapped_vals.append(ent['id'])
The order of items in the list will be indexed according to the json data, since the list is a sequenced collection.
In PHP:
$data = json_decode($json_string);
$output = array();
foreach($data as $values){
$output[] = $values->id;
}
Again, the ordered nature of PHP arrays ensure that the output will be ordered as-is with regard to indexes.
Either example could be modified to use a mapped dictionary (python) or an associative array (php) if needs demand.
You could adapt these to functions that take the id value as an argument, track how far they are into the array, and when found, break out and return the current index.
Wow. I posted the original question 10 months ago when I knew nothing about Python nor computer programming whatsoever!
Answer
But I learned basic Python last December and came up with a solution for not only get the rank order but to insert the results into a MySQL database:
import urllib.request
import json
# Make connection and get the content
response = urllib.request.urlopen(http://whatever.com/search?=ids=1212,125,54,454)
content = response.read()
# Decode Json search results to type dict
json_search = json.loads(content.decode("utf8"))
# Get 'results' key-value pairs to a list
search_data_all = []
for i in json_search['results']:
search_data_all.append(i)
# Prepare MySQL list with ranking order for each id item
ranks_list_to_mysql = []
for i in range(len(search_data_all)):
d = {}
d['id'] = search_data_all[i]['id']
d['rank'] = i + 1
ranks_list_to_mysql.append(d)