Given this
{
"some": "property",
"nested": {
"hello": "world"
}
}
I'd like to get this result with jq
{
"some": "property",
"nested": {
"hello": "world",
"freshly": "added"
}
}
So how can I add the freshly added field ? I don't know how many properties are at root level (and I want to keep them all), I only know the name of the nested object (here "nested"), the name of the property I'd like to add (here "freshly") and its value.
Just assign the new value to the nested object.
.nested.freshly = "added"
Well I found out myself how to do it. If you have a better solution, you're more than welcome to give it here.
jq '.nested=(.nested + {"freshly": "added"})'
You can also do simply
.nested += {freshly: "added"}
Then you can add multiple nested keys at once
Related
I have an object that looks like this
{
"my_list": [
{
"name": "an_image",
"image": "an_image.com"
},
{
"name": "another_image",
"image": "another_image.com"
},
...<more objects with image property>
],
"foo": {
"image": "foobar.io"
},
"bar": {
"image": "bar_image.io"
},
...<more objects with image property>
}
I'd like to get all of the image properties from each of the objects, and from each object in my_list and other lists that have objects that include an image property. So in this example I'd like to get
"an_image.com"
"another_image.com"
"foobar.io"
"bar_image.io"
We don't know the keys of any of these objects at runtime, so we can't reference my_list, foo, or bar in this example.
Previously we didn't have my_list in the object and jq '.[].image' worked, but now that results in jq: error (at bar.json:18): Cannot index array with string "image".
The problem is that we don't know the name of the objects that contain image properties so we can't reference them explicitly, and now that we've added another element that's a list we're running into type errors, that I'm not sure how to solve.
I've tried various combinations of .[].image, but they all seem to run into issues with typing.
If you don't mind the terseness, you could perhaps go with:
jq '..|select(.image?).image'
You could select by objects and items of arrays:
jq '.[] | ., arrays[] | objects.image'
"an_image.com"
"another_image.com"
"foobar.io"
"bar_image.io"
Demo
Using recursive descent .. is more elegant:
jq '.. | .image? // empty'
If the input is large, you might want to consider streaming the data in:
$ jq --stream -r 'select(.[0][-1] == "image")[1] // empty' input.json
an_image.com
another_image.com
foobar.io
bar_image.io
When streamed, your input will be processed as path/value pairs for the most part. Filter the paths you want, then return the value.
I am trying to apply a JDT transform to a JSON document in order to modify a property in a N-th array element. Is that possible without having to replace the entire element or even the entire array?
{
"array": [
{
name: "A",
value: 0
},
{
name: "B",
value: 3.14
}
]
}
Is there a transform that gets me to the following? I want to alter the 2nd array element and only its "value" property. I don't want to search for it by "name" but rather access by index.
{
"array": [
{
name: "A",
value: 0
},
{
name: "B",
value: 12345678
}
]
}
The challenge
It is easy to do your transform with some libraries in JSON. If your object is called foo, you mainly want to something like foo.array[1].value = "12345678" without any kind of looping.
The JDT way
I found How to use SlowCheetah to transform an array elements in Json config file? which asks
For example, if my base config file has this setting:
{
"Settings" : [1, 2, 3]
}
and I want to transfer it to:
{
"Settings" : [4, 5, 6]
}
The solution by Collin K was
{
"#jdt.replace": {
"#jdt.path": "$.Settings",
"#jdt.value": [4,5,6]
}
}
This seems like you need to actually replace the whole array.
Digging further let me to an open issue of JDT which seems to confirm this assumption.
Disclaimer
I have not used JDT myself, but I have been struggling with nested JSONs of various kinds e.g. with Elasticsearch.
Further references
https://github.com/microsoft/json-document-transforms/wiki/Replace-Transformation
Using jq to update objects within a JSON document if the value corresponding to a given key starts with a specified string - the JQ way I would use
Here is the desired schema and json for illustration purpose. Please see the link below.
JSON Schema and JSON
{
"id": "123" ,
"ts": "1234567890",
"complex_rules":
[
{
"type":"admin",
"rule":{
"rights":"all",
"remarks": "some admin remarks"
}
},
{
"type":"guest",
"rights": "limited"
},
{
"type":"anonymous",
"rights": "blocked"
}
]
}
The 'complex_rules' is an array of json object:
With type either be a : "admin", "guest", "anonymous" and the 'type' attribute is MANDATORY.
Each object in array can have its own structure, but the type can be either of: "admin", "guest", "anonymous" only. No other type attribute is acceptable.
The conditions to evaluate:
The type of object in the array cannot re-occur in the array. (I know this seems to be not possible, so we can ignore this)
If attribute "rights" in the {type=admin object} with any value, then we cannot have "rights": "limited" or any value in {type=guest object}. The JSON Schema validation must complain about this.
Another twist, either object {type":"guest"}or {type":"anonymous"} can exist. Both types cannot coexist along with other types.
----Update
The above link is the solution this question.
In regards to 1 and 2:
You need to use a combination of if, then, and not keywords to construct the logic you require with the correct level of applicability.
In regards to 3:
The type of object in the array cannot re-occur in the array. (I know
this seems to be not possible, so we can ignore this)
Right, that's correct, it's not possible as of draft-7 JSON Schema.
I'm setting up a devops pipeline so that certain data profiles stored in JSON format can be shifted across different servers. While downloading it from the current server I need to clean up all the protected keys and unique identifiers. I'm looking for the cleanest way to do the following in JQ
Input:
{
"TopKey1":{
"some_key":"some_value"
},
"TopKey2":{
"some_key2":"some_value2"
},
"KeytoSearch":[
{
"_id":"sdf",
"non_relevant_key1":"val"
},
{
"_id":"sdfdsdf",
"non_relevant_key2":"val"
},
{
"_id":"sgf",
"non_relevant_key3":"val"
}
]
}
Output:
{
"TopKey1":{
"some_key":"some_value"
},
"TopKey2":{
"some_key2":"some_value2"
},
"KeytoSearch":[
{
"non_relevant_key1":"val"
},
{
"non_relevant_key2":"val"
},
{
"non_relevant_key3":"val"
}
]
}
In python terms if this were a dictionary
for json_object in dictionary["KeytoSearch"]:
json_object.pop("_id")
I've tried combinations of map and del but can't seem to figure out the nested indexing with this. The error messages I get are along the lines of jq: error (at <stdin>:277): Cannot index string with string "_id" which sort of tells me I haven't fundamentally understood how jq works or is to be used, but this is the route I need to go because using a Python script to clean up JSON objects is something I'd rather avoid
Going with your input JSON and assuming there are other properties in your KeytoSearch object along with the _id fields, you could just do below.
jq 'del(.KeytoSearch[]._id)'
See this jqplay.org snippet for a demo. The quotes around the property key containing _ are not needed as confirmed in one of the comments below. Some meta-characters (e.g. . in the property key values needs be accessed with quotes as ".id") needs to be quoted properly, but _ is clearly not one of them.
I've tried combinations of map and del
Good! You were probably just missing the '|=' magic ingredient:
.Keytosearch |= map( del(._id) )
alternatively, you could use a walk-path unix tool for JSON: jtc and apply changes right into the sourse json file (-f):
bash $ jtc -fpw'[KeytoSearch]<_id>l:' file.json
bash $
bash $
bash $ jtc file.json
{
"KeytoSearch": [
{
"non_relevant_key1": "val"
},
{
"non_relevant_key2": "val"
},
{
"non_relevant_key3": "val"
}
],
"TopKey1": {
"some_key": "some_value"
},
"TopKey2": {
"some_key2": "some_value2"
}
}
bash $
if given json snippet is a part of a larger JSON (and [KeytoSearch] is not addressable from the root), then replace it with the search lexeme: <KeytoSearch>l.
PS> Disclosure: I'm the creator of the jtc tool
I have json that looks like this:
{
"Workflow": [
{
"Parameters": {
"Project": "/Path/To/File",
"OtherParam": "True"
}
}
],
"Overrides": [
{
"Special": {
"Parameters": {
"NewParam": "NewStuffGoesHere",
"OtherParam": "False"
}
}
}
]
}
... where I want to use the Overrides.Special section to add or update fields in the workflow object. In other words, given the json above, I want to do something like this:
$config = Get-Content workflow.json | out-string | ConvertFrom-Json
$configWithOverrides = Merge-Object $config.Workflow $config.Overrides.Special
And end up with something like this:
$configWithOverrides
Parameters
----------
#{Project=/Path/To/File; NewParam=NewStuffGoesHere; OtherParam=False}
I can certainly write the Merge-Object function above to add or update values as needed based on what's in the override section, but it seems there should (could?) be a built-in or one-liner way to handle this.
I tried this:
$test = $config.Workflow + $config.Overrides.Special
...but that doesn't quite work.
$test
Parameters
----------
#{Project=/Path/To/File; OtherParam=True}
#{NewParam=NewStuffGoesHere; OtherParam=False}
This enables adding parameters:
>$test.Parameters.NewParam
NewStuffGoesHere
...but it's not so great for updating them
>$test.Parameters.OtherParam
True
False
Note - in this example, I'm choosing to handle the merge after converting the json to a psobject, but that's not a requirement.
I have a one-liner to do what you're asking for. Notice that, as far as I know, PowerShell does not deal directly with json strings. But, once converted to PowerShell objects, it's like any other object.
So, firstly, define your json file, and read it as a single string:
# Requires -Version 4
$jsonFile='c:\temp\jsonfile.json'
$jsonObj=#(gc $jsonFile -raw)|ConvertFrom-Json
Define the property upon which you want to merge the json's objects, and the 1st and 2nd objects:
$property='Parameters'
$1=$jsonObj.Workflow.$property
$2=$jsonObj.Overrides.Special.$property
Now, see the one-liner (which I've splitted in 3, for the sake of clarity):
$MergedJson=[pscustomobject]#{
$property=$2.psobject.properties|%{$11=$1}{$11|add-member $_.name $_.value -ea Ignore}{$11}
}|ConvertTo-Json
You see? $MergedJson holds the following string (using your json string):
{
"Parameters": {
"Project": "/Path/To/File",
"OtherParam": "True",
"NewParam": "NewStuffGoesHere"
}
}
Is that what you're looking for?
P.S.: if you swap the roles of $1 and $2, the common parameters' (like OtherParam) values that prevail, change.