I have a json file like this and I want to save both "id" and "tenantId" in some map as key/value pair to process further. Is there a way I can parse my JSON to do that?
{
"type": "nodesRepresentation",
"paging": {
"cursorAfter": "NjA3YmUyMjA4ZjdkMTMwMDAxMDc1MDlk",
"limit": 50,
"pageSize": 2,
"links": [
{
"href": "/api/node/nodes%3Fafter=NjA3YmUyMjA4ZjdkMTMwMDAxMDc1MDlk&limit=50",
"rel": "next"
}
]
},
"pageSize": 2,
"values": [
{
"id": "a423a565-8ae3-4d99-9c8a-92b8af843228",
"tenantId": "4d010b5a-a9ab-4e7d-b989-6a05245105ed"
},
{
"id": "e350ffaa-d62e-4cfc-9a3b-ae4624d35dc0",
"tenantId": "8725905d-90c4-4728-a94a-47c0d57f8251"
}
]
}
You could use jq. Since you haven’t specified precisely the result you want, I’d suggest starting with
jq ‘.values[]' myfile.json
Related
I have a requirement to roll a collection of nodes that uses the current node name (within the collection) and for the value take each child nodes value (single node) into a string array, then use the parents key as the key.
Given.
{
"client": {
"addresses": [
{
"id": "27ef465ef60d2705",
"type": "RegisteredOfficeAddress"
},
{
"id": "b7affb035be3f984",
"type": "PlaceOfBusiness"
},
{
"id": "a8a3bef166141206",
"type": "EmailAddress"
}
],
"links": [
{
"id": "29a9de859e70799e",
"type": "Director",
"name": "Bob the Builder"
},
{
"id": "22493ad4c4fd8ac5",
"type": "Secretary",
"name": "Jennifer"
}
],
"Names": [
{
"id": "53977967eadfffcd",
"type": "EntityName",
"name": "Banjo"
}
]
}
}
from this the output needs to be
{
"client": {
"addresses": [
"RegisteredOfficeAddress",
"PlaceOfBusiness",
"EmailAddress"
],
"links": [
"Director",
"Secretary"
],
"Names": [
"EntityName"
]
}
}
What is the best way to achieve this? Any pointers to what/how to do this would be greatly appreciated.
Ron.
You can iterate over entries of your client object first with the help of the $each function, then get types for each of them, and combine via $merge:
{
"client": client
~> $each(function($list, $key) {{ $key: $list.type }})
~> $merge
}
Live playground: https://stedi.link/OpuRdE9
Using an AWS step function, I'm attempting to select only certain data from the results of a task. For some unfathomable reason, AWS have chosen not to allow Query in step functions, so I'm using ResultsSelector. However, I'm struggling with the JSONPath that is required.
How can I use the ResultsSelector to construct my desired JSON object?
Take this result -
{
"IsTruncated": false,
"KeyMarker": "",
"MaxKeys": 1000,
"Name": "some-bucket-name",
"Prefix": "some/prefix/",
"VersionIdMarker": "",
"Versions": [
{
"ETag": "\"02e9c20b7cd36fcf6e47926c26f0b39e\"",
"IsLatest": true,
"Key": "some/prefix/my.file",
"LastModified": "2021-09-30T15:34:59Z",
"Owner": {
"Id": "1fd170056d1480a7c1c9b43f5bf0603d91cbabc4ec77eefdcaa10218c3a920f6"
},
"Size": 69606,
"StorageClass": "STANDARD",
"VersionId": "y6XzRsCUZcXMPHqwwnhAGLwTlmPoj9dj"
},
{
"ETag": "\"01bc5b65afe6b0cc0722fc5da32a8a44\"",
"IsLatest": false,
"Key": "some/prefix/my.file",
"LastModified": "2021-09-30T15:34:21Z",
"Owner": {
"Id": "1fd170056d1480a7c1c9b43f5bf0603d91cbabc4ec77eefdcaa10218c3a920f6"
},
"Size": 69407,
"StorageClass": "STANDARD",
"VersionId": "jPdKeUqnYlf0_eNXzHaYCvDfdHLOvRX7"
}
]
}
What I'd like is to use the ResultsSelector to construct this JSON object -
{
"Objects": [
{
"Key": "some/prefix/my.file",
"VersionId": "y6XzRsCUZcXMPHqwwnhAGLwTlmPoj9dj"
},
{
"Key": "some/prefix/my.file",
"VersionId": "jPdKeUqnYlf0_eNXzHaYCvDfdHLOvRX7"
}
]
}
However, the closest I've been able to get so far is by using this -
{
"Key.$": "$.Versions[*].Key",
"VersionId.$": "$.Versions[*].VersionId"
}
Which gets me this -
{
"VersionId": [
"y6XzRsCUZcXMPHqwwnhAGLwTlmPoj9dj",
"jPdKeUqnYlf0_eNXzHaYCvDfdHLOvRX7",
],
"Key": [
"some/prefix/my.file",
"some/prefix/my.file"
]
}
The Jayway JsonPath implementation gives the output with the Key. You need to verify if AWS Step function supports the below jsonpath.
Tool : https://jsonpath.herokuapp.com/
$.Versions[*].['Key','VersionId']
If it works then you can do something like
"Objects.$": "$.Versions[*].['Key','VersionId']",
In the Json string given below, I want to find all elements in which category = m AND the "middle" array contains elements which match this condition - the element's "middle" array has objects whose itemType = Executable.
I would like to use jsonpath to get the desired objects. I prefer to not use jmespath because it can be too complex for my purpose. But, I am new to jsonpath and I am not able to figure out the json query from online tutorials which are too trivial or basic. I wonder if its better to use a programming language instead to get the data I need. Please advise.
So far, I was able to only extract elements in which category = m by using this jsonpath query $.[?(#.category=="m")]. How do I do the remaining part ?
Json :
Overview - Every object has a "content" object. Each content object generally has a start, middle and end array besides other fields. Middle arrays can have multiple content objects inside them and so on. Some of the content objects have only a middle array. I am interested in locating items in such content objects as mentioned above.
Note that this is not the actual json which I have to process. It is an imitation which has been sanitized for SO.
{
"id": "123",
"contents": {
"title": "B1",
"start": [],
"middle": [
{
"level": "1",
"contents": {
"title": "C1",
"category": "c",
"start": [],
"middle": [
{
"level": "2",
"contents": {
"title": "M1",
"category": "m",
"start": [],
"middle": [
{
"level": "3",
"contents": {
"title": "MAT1",
"middle": [
{
"itemType": "Data"
}
]
}
},
{
"level": "3",
"contents": {
"title": "MAT2",
"middle": [
{
"itemType": "Executable",
"id": "exec1"
}
]
}
},
{
"level": "3",
"contents": {
"title": "MAT3",
"middle": [
{
"itemType": "Data"
}
]
}
}
],
"end": []
}
},
{
"level": "2",
"contents": {
"title": "M2",
"category": "m",
"start": [],
"middle": [
{
"level": "3",
"contents": {
"title": "MAT1",
"middle": [
{
"itemType": "Data"
}
]
}
},
{
"level": "3",
"contents": {
"title": "MAT2",
"middle": [
{
"itemType": "Executable",
"id": "exec2"
}
]
}
}
],
"end": []
}
}
],
"end": []
}
},
{
"level": "1",
"contents": {
"title": "C2",
"category": "c",
"start": [],
"middle": [
{
"level": "2",
"contents": {
"title": "M1",
"category": "m",
"start": [],
"middle": [
{
"level": "3",
"contents": {
"title": "MAT1",
"middle": [
{
"itemType": "Data"
}
]
}
},
{
"level": "3",
"contents": {
"title": "MAT2",
"middle": [
{
"itemType": "Executable",
"id": "exec3"
}
]
}
},
{
"level": "3",
"contents": {
"title": "MAT3",
"middle": [
{
"itemType": "Data"
}
]
}
}
],
"end": []
}
},
{
"level": "2",
"contents": {
"title": "M2",
"category": "m",
"start": [],
"middle": [
{
"level": "3",
"contents": {
"title": "MAT1",
"middle": [
{
"itemType": "Data"
}
]
}
},
{
"level": "3",
"contents": {
"title": "MAT2",
"middle": [
{
"itemType": "Executable",
"id": "exec4"
}
]
}
},
{
"level": "3",
"contents": {
"title": "MAT3",
"middle": [
{
"itemType": "Data"
}
]
}
}
],
"end": []
}
}
],
"end": []
}
}
],
"end": []
}
}
Context
json with nested objects1
jsonpath expression language
choosing between jsonpath and jmespath (or other JSON expression engine)
Problem
DeveMasterJoe2 wants to extract some values from nested JSON
Discussion
There are lots of implementations of jsonpath out there, and they do not all support the same features
The structure and normalization of the source JSON is going to influence how easily this can be done with pure jsonpath
In choosing a JSON expression engine, one has to weigh multiple factors
how consistent are the implementations across languages?
how many choices are there within a given language?
how clear is the specification?
how many examples, unit-tests or tutorials are available?
who is supporting it?
Example solution using Python and jsonpath-ng
Here is an example solution using python 3.7 and jsonpath-ng
This example uses a mix of jsonpath and python instead of just pure jsonpath, because of the heavily-nested JSON
I will leave it for someone else to provide an answer that relies on pure jsonpath
Note that the source JSON arguably could stand to be cleaned up a bit
(for example, why is there no id field attached to itemType==Data elements?)
(for example, why is category not found on all contents elements?)
(for example, if you expressly specify level why complicate things with heavily nested objects when you can determine depth by level ?)
This example:
## import libraries
import codecs
import json
import jsonpath_ng
from jsonpath_ng.ext import parse
##;;
## init vars
href="path/to/my/jsonfile/nested_dict.json"
json_string = codecs.open(href, 'rb', encoding='utf8').read()
json_dataroot = json.loads(json_string)
final_result = []
##;;
## init jsonpath outer-query
match = parse('$..contents.middle[*]').find(json_dataroot)
##;;
## iterate through outer-query and gather subelements
for ijj,item in enumerate(match):
## restrict to desired category == 'm'
if(match[ijj].value.get('contents',{}).get('category','') == 'm'):
## extract out desired subelements
json_datafrag001 = [item.get('contents',{}).get('middle',{})[0]
for item in match[ijj].value.get('contents',{}).get('middle',{})
]
match001 = parse("$[?(#.itemType=='Executable')]").find(json_datafrag001)
final_result.extend(list(match001[ikk].value for ikk,item in enumerate(match001)))
pass
##;;
## show final result
vout = json.dumps(final_result, sort_keys=True,indent=4, separators=(',', ': '))
print(vout)
##;;
... produces this result ...
[
{
"id": "exec1",
"itemType": "Executable"
},
{
"id": "exec2",
"itemType": "Executable"
},
{
"id": "exec3",
"itemType": "Executable"
},
{
"id": "exec4",
"itemType": "Executable"
}
]
1 (aka dictionary, associative-array, hash)
When I try to save my changes from the Google Apps Script file I get the following error:
Missing ] after element list.
This is my code:
var request = {
"name": "Name",
"id": 3,
"rules":[
{
"name": "Nested",
"tags": [
{
"tagId": 1,
"variables":[
[
"variable": "Var1"
]
],
"condition": false,
},
{
"tagId": 1,
"condition": false,
}
],
"ruleSetId": 3,
}
]
}
The error indicates that the problem is on the line that contains "variable": allScopes[i].variable, but I can't find where the problem is...
This is an example with the JSON object that I need to build:
Note that you are trying to use the array literal to construct an object:
"variables":[
[
"variable": "Var1"
]
],
As it looks like you need an object, not an array here, replace the inner [] with {}:
"variables":[
{
"id": null,
...,
"value": ".*"
}
],
I'm trying to save this to a Cloudant database:
[
{
"_id": "document_one",
"heynow": [
{
"Name": "one",
"Duration": 2,
"DurationUnit": "Hours"
},
{
"Name": "two",
"Duration": 40,
"DurationUnit": "Minutes"
}
]
},
{
"_id": "document_two",
"heynow": [
{
"Name": "three",
"Duration": 2,
"DurationUnit": "Hours"
},
{
"Name": "four",
"Duration": 40,
"DurationUnit": "Minutes"
}
]
}
]
But apparently it doesn't like the outer brackets because it's telling me:
"error":"bad_request","reason":"Document must be a JSON object"
JSONLint says it's valid, so I guess I'm asking if anyone knows how to format this so it can be entered into a Couchdb because the outer brackets seem to be causing problems.
CouchDB has a bulk documents API;
https://wiki.apache.org/couchdb/HTTP_Bulk_Document_API