How would I parse this object, removing all entries where "field"="status", but at the same time retaining the validity of the JSON object (i.e. also removing the extraneous closing braces or brackets at the end of the object?
{"logic":"and","filters":[{"filters":[{"field":"name","operator":"contains","value":"JOHNSON"},{"field":"city","operator":"contains","value":"MILWAUKEE"}],"logic":"and"},{"logic":"or","filters":[{"field":"status","operator":"eq","value":"A"},{"field":"status","operator":"eq","value":"G"},{"field":"status","operator":"eq","value":"O"},{"field":"status","operator":"eq","value":"P"},{"field":"status","operator":"eq","value":"S"}]}]
Here is a solution using jq based on my answer to a similar question
def clean(condition):
if type == "object" then
if condition
then empty
else
with_entries(
if (.value|type) == "object" and (.value|condition)
then empty
else .value |= clean(condition)
end
)
end
elif type == "array" then
map(
if type == "object" and condition
then empty
else clean(condition)
end
)
else .
end
;
clean(
has("field") and (.field == "status")
)
If filter.jq contains this filter and data.json contains the sample data then the command
$ jq -M -f filter.jq data.json
produces
{
"logic": "and",
"filters": [
{
"filters": [
{
"field": "name",
"operator": "contains",
"value": "JOHNSON"
},
{
"field": "city",
"operator": "contains",
"value": "MILWAUKEE"
}
],
"logic": "and"
},
{
"logic": "or",
"filters": []
}
]
}
Related
I have some json that I want to add to based on a walk. A simple example of the json I have is below:
{
"valueSource": "memory",
"dataType": "Boolean",
"alarms": [
{
"setpointA": 1.0,
"name": "Alarm",
"priority": "Diagnostic",
"ackMode": "Auto",
}
],
"name": "Test Alarm",
"value": false,
"tagType": "AtomicTag"
}
I want to add to each object in the "alarms" key's array the following key:
{
"bindType": "Tag",
"value": "[.]<parent.name>.Name"
}
where <parent.name> is "Test Alarm" in this example which is the parent-of-the-alarm-array-item's "name" key.
I've gotten this jq filter so far that adds the object, but the value key value is wrong (it's getting the alarm array item's name instead of its parent's name):
walk( if type == "object" and .setpointA then .label = {"bindType":"Tag", "value": "[.]\(.name).Name"} else . end)
Essentially I want this:
{
"valueSource": "memory",
"dataType": "Boolean",
"alarms": [
{
"setpointA": 1.0,
"name": "Alarm",
"priority": "Diagnostic",
"ackMode": "Auto",
"label": {
"bindType": "Tag",
"value": "[.]Test Alarm.Name"
}
}
],
"name": "Test Alarm",
"value": false,
"tagType": "AtomicTag"
}
Here is my jqplay below. It has the final result in the JSON section, where the Result should match this but doesn't at the moment.
https://jqplay.org/s/-qHFIWolrD
Follow up question:
How would I add this displayPath key as well?
You cannot reference to parent, you have to save the reference in a variable beforehand, and descend with having access to that variable.
.tags[] |= (
.name as $name
| # rest of your code using $name
walk(
if type == "object" and .setpointA
then .label = {"bindType":"Tag", "value": "[.]\($name).Name"}
else . end
)
)
Demo
As you happen to know that the objects are located in the .alarms array, you could also just iterate over the items, select only those matching the condition and then assign to their .label whatever you want (including $name)
.tags[] |= (
.name as $name
| (.alarms[] | select(has("setpointA"))).label = {
bindType: "Tag", value: "[.]\($name).Name"
}
)
Demo
Edit responding to OP's follow-up question
I actually don't care about the setpointA key, I just want to add the label key (as well as another displayPath key) to each item in all alarms arrays.
.tags[] |= (.name as $name | .alarms[] += (
{bindType: "Tag", value: "[.]\($name)."} | {
label: (.value += "Name"),
displayPath: (.value += "Documentation")
}
))
Demo
As jq does not allow you to refer to parent object, you need to work on parent level :
jq 'walk(if type == "object" and .alarms and ( .alarms | arrays )
then (.alarms[] | select(.setpointA)).label =
{ bindType: "Tag", value: "[.]\(.name).Name"}
else . end
)' data.json
The intent for the JSON data below is to update the value of the field dst with the value of src within all elements of type t, regardless of depth within the tree, while at the same time preserving the whole structure of the data.
Is this possible with jq? My several attempts have boiled down to the following command that is not working to achieve the intended purpose:
$ jq -r 'map_values(select(.. | .type? == "t" |= (.dst = .src)))'
{
"a": "b",
"c": [
{
"type": "t",
"src": "xx",
"dst": "zz"
},
{
"type": "t",
"src": "xx",
"dst": "zz"
}
],
"d": [
{
"e": [
{
"type": "t",
"src": "xx",
"dst": "zz"
}
]
},
{
"type": "t2",
"src": "xx",
"dst": "zz"
}
]
}
Is this possible with jq?
jq is Turing-complete :-)
Here's a simple solution:
walk( if type == "object" and .type == "t" then .dst = .src else . end)
If your jq does not have walk/1, then it might be a good time to upgrade (to jq 1.6); otherwise, you can snarf its def from the web, e.g. by googling: jq "def walk"
Alternatively ...
reduce paths as $x (.;
if (getpath($x)|.type? // false) == "t"
then setpath( $x + ["dst"]; getpath( $x + ["src"] ))
else . end)
Given the input in this form
[
{
"DIR" : "/foo/bar/a/b/c",
"OUT" : "/foo/bar/x/y/z",
"ARG" : [ "aaa", "bbb", "/foo/bar/a", "BASE=/foo/bar" ]
},
{
"DIR" : "/foo/baz/d/e/f",
"OUT" : "/foo/baz/x/y/z",
"ARG" : [ "ccc", "ddd", "/foo/baz/b", "BASE=/foo/baz" ]
},
{
"foo" : "bar"
}
]
I'm trying to find out how to make jq transform that into this:
[
{
"DIR" : "BASE/a/b/c",
"OUT" : "BASE/x/y/z",
"ARG" : [ "aaa", "bbb", "BASE/a", "BASE=/foo/bar" ]
},
{
"DIR" : "BASE/d/e/f",
"OUT" : "BASE/x/y/z",
"ARG" : [ "ccc", "ddd", "BASE/b", "BASE=/foo/baz" ]
},
{
"foo" : "bar"
}
]
In other words, objects having an "ARG" array, containing a string that starts with "BASE=" should use the string after "BASE=", e.g. "/foo" to substitute other string values that start with "/foo" (except the "BASE=/foo" which should remain unchanged")
I'm not even close to finding a solution myself, and at this point I'm unsure that jq alone will do the job.
With jq:
#!/usr/bin/jq -f
# fix-base.jq
def fix_base:
(.ARG[] | select(startswith("BASE=")) | split("=")[1]) as $base
| .DIR?|="BASE"+ltrimstr($base)
| .OUT?|="BASE"+ltrimstr($base)
| .ARG|=map(if startswith($base) then "BASE"+ltrimstr($base) else . end)
;
map(if .ARG? then fix_base else . end)
You can run it like this:
jq -f fix-base.jq input.json
or make it an executable like this:
chmod +x fix-base.jq
./fix-base.jq input.json
Don't worry, jq alone will do the job:
jq 'def sub_base($base): if (startswith("BASE") | not) then sub($base; "BASE") else . end;
map(if .["ARG"] then ((.ARG[] | select(startswith("BASE=")) | split("=")[1]) as $base
| to_entries
| map(if (.value | type == "string") then .value |= sub_base($base)
else .value |= map(sub_base($base)) end)
| from_entries)
else . end)' input.json
The output:
[
{
"DIR": "BASE/a/b/c",
"OUT": "BASE/x/y/z",
"ARG": [
"aaa",
"bbb",
"BASE/a",
"BASE=/foo/bar"
]
},
{
"DIR": "BASE/d/e/f",
"OUT": "BASE/x/y/z",
"ARG": [
"ccc",
"ddd",
"BASE/b",
"BASE=/foo/baz"
]
},
{
"foo": "bar"
}
]
Some helper functions make the going much easier. The first is generic and worthy perhaps of your standard library:
# Returns the integer index, $i, corresponding to the first element
# at which f is truthy, else null
def indexof(f):
label $out
| foreach .[] as $x (null; .+1;
if ($x|f) then (.-1, break $out) else empty end) // null;
# Change the string $base to BASE using gsub
def munge($base):
if type == "string" and (test("^BASE=")|not) then gsub($base; "BASE")
elif type=="array" then map(munge($base))
elif type=="object" then map_values(munge($base))
else .
end;
And now the easy part:
map(if has("ARG")
then (.ARG|indexof(test("^BASE="))) as $ix
| if $ix
then (.ARG[$ix]|sub("^BASE=";"")) as $base | munge($base)
else . end
else . end )
Some points to note:
You may wish to use sub rather than gsub in munge;
The above solution assumes you want to make the change in all keys, not just "DIR", "OUT", and "ARG"
The above solution allows specifications of BASE that include one or more occurrences of "=".
I have the following dataset:
[
{
"py/object": "bit.ast.Node",
"_children": [
{
"py/object": "bit.ast.Node",
"_children": [
"main",
{
"py/object": "bit.ast.Node",
"_children": [
"args",
{
"py/object": "bit.ast.Node",
"_children": [
{
"py/object": "bit.ast.Node",
"_children": [
"str"
],
"source_column": 2,
"source_filename": "tests/fixture/hello.b",
"source_line": 1,
"tag": "type-named"
}
],
"base": {
"py/id": 10
},
"source_column": 2,
"source_filename": "tests/fixture/hello.b",
"source_line": 1,
"tag": "type",
"type": "array"
}
],
(and so on...)
How do I get jq to exclude the _children property from all objects that have it? What about all properties that start with _?
None of the following seem to work:
jq 'map(del (._children))'
jq 'map(if has("_children") then del (._children) end)'
jq 'del(._children)'
jq 'del(.[]._children)'
jq 'del(.[]|._children)'
I keep getting an error similar to:
jq: error (at <stdin>:1): Cannot index string with string "_children"
exclude the _children property from all objects that have it
If your jq has walk/1 then you could:
walk( if type == "object" then del(._children) else . end )
If not, first include its jq definition (readily googleable) e.g. in ~/.jq
What about all properties that start with _ ?
For this, you could also use walk/1. For clarity and maintainability, it would make sense to define a helper function:
def deleteall(f): with_entries(select(.key | f | not ));
Which you'd invoke as: deleteall( startswith("_") )
Here's some example JSON:
{
"Tags": [
{
"Key": "Name",
"Value": "foo"
},
{
"Key": "Type",
"Value": "C"
}
]
}
I want to print the value of "Value" only when "Key" is "Type". So it should print out "C". This is what I have so far.
echo $MY_TAGS | jq 'if .Tags[].Key == "Type" then .Tags[].Value else empty end'
But it prints out:
"foo"
"C"
Is there a way to do this?
Try this:
.Tags[] | select(.Key == "Type") | .Value