I am validating a json file, I need to validate in a field that if it is null, assign a default value, I can't find how to perform a validation with if conditional
file.json
{
"timeout": 100
}
jq -r .timeout fiile.json
here it prints the value correctly, I need to validate if this field is null to assign it a default value with jq
Thanks in advance
Use the update operator |= to update the field in question. For the conditional just compare to null.
jq '.timeout |= if . == null then 200 else . end'
If your input file is
{
"timeout": 100
}
it will stay the same, as .timeout is not null. But if it is
{
"timeout": null
}
then it will be changed to the default value given, here:
{
"timeout": 200
}
Note that this only triggers for the content of null. There are other means to test for false, the number zero 0, the empty string "", etc., even the complete absence of that field.
You can try alternative operator //
jq '.timeout //= 200' file.json
Related
I am having a complex nested json
{
...
"key1": {
"key2" : [
{ ...
"base_score" :4.5
}
]
"key3": {
"key4": [
{ ...
"base_score" : 0.5
...
}
]
}
...
}
}
There maybe multiple "base_score" in the json("base_score" path is unknown) and the corresponding value will be a number, I have to check if at least one such value is greater than some known value 7.0, and if there is, I have to do "exit 1". I have to write this query in shell script.
Assuming the input is valid JSON in a file named input.json, then based on my understanding of the requirements, you could go with:
jq --argjson limit 7.0 '
any(.. | select(type=="object" and (.base_score|type=="number")) | .base_score; . > $limit)
| halt_error(if . then 1 else 0 end)
' input.json
You can modify the argument to halt_error to set the exit code as you wish.
Note that halt_error redirects its input to stderr, so you might want to append 2> /dev/null (or the equivalent expression appropriate for your shell) to the above invocation.
You can easily get a stream of base_score values at any level and use that with any:
any(..|.base_score?; . > 7)
The stream will contain null values for objects without the property, but null is not greater than any number, so that shouldn't be a stopper.
You could then compare the output or specify -e/--exit-status to be used with a condition directly:
jq -e 'any(..|.base_score?; . > 7)' complexnestedfile.json >/dev/null && exit 1
Goal is to search for null values of part_description and insert in part field value.
[
{
"part": "brake-01982",
"part_description": null,
}
]
Expected Output
[
{
"part": "brake-01982",
"part_description": "brake-01982",
}
]
Command:
jq '(.[] | select(.part_description==null).part_description) |= .part'
Results in no change
What does work is if I try to insert a string value. The double quoted string results in what I would expect. Demo below. How do I pass another fields value into this command vs a quoted string?
Command: jq '(.[] | select(.part_description==null).part_description) |= "test"'
Demo
Output
[
{
"part": "brake-01982",
"part_description": "test"
}
]
The alternative operator // retains the first value, unless it is null, false, or empty (i.e. missing), in which case it takes on the second. Combined with the update operator |=, this can be contracted to //=.
jq '.[] |= (.part_description //= .part)'
[
{
"part": "brake-01982",
"part_description": "brake-01982"
}
]
Demo
Edit: To strictly update only in the case of null, i.e. to keep a value of false, and to not create the field if it's missing, the filter needs to check explicity for null:
jq '.[] |= ((select(has("part_description")).part_description | select(. == null)) = .part)'
[
{
"part": "brake-01982",
"part_description": "brake-01982"
}
]
Demo
Goal is to search for null values of part_description ....
If this is indeed the goal, then you would need to test for null rather than relying on // or //=. So you could write:
map( if has("part_description") and .part_description == null
then .part_description = .part else . end )
I am trying to convert belowJSON to CSV using jq command but the final CSV is unable to place deviceName field properly as it's missing in some JSON lines.
{
"id": "ABC",
"deviceName": "",
"total": 100,
"master": 20
}
{
"id": "ABC",
"total": 100,
"master": 20
}
How can i make sure empty value gets when Key is missing ?.
I Tried below command to generate CSV
./jq -r '[.[]] | #csv' > final.csv
But it gives CSV like below as you can see when deviceName key is missing in JSON it's cell shifting left side.
"ABC","",100,20
"ABC",100,20
I want output something like below which adds empty value if deviceName is missing.
"ABC","",100,20
"ABC","",100,20
In jq you can use the alternate operator // that can be used to return default values. E.g. .foo // 1 will evaluate to 1 if there's no .foo element in the input
Using that and appending an empty string "" if the key is not present, you can do
jq -r '[.id // "", .deviceName // "", .total // "", .master // ""] | #csv'
Note: The alternate operator .foo // 1 causes the evaluation to 1 for case when the value of foo is null or false. You may wish to modify the above program if your data contains null or false values.
You can take the first object as a reference for a full record, like
keys_unsorted as $k | (., inputs) | [.[$k[]]] | #csv
For your sample this produces following
"ABC","",100,20
"ABC",,100,20
I've got two different JSON structures to retrieve a specific object value from, basically something like this
{
"one": {
"foo": {
"bar": "baz"
}
}
}
and another like that
{
"two": {
"foo": {
"bar": "qux"
}
}
}
I'd like to return the bar value in both cases plus an additional return variant error in case neither case 1 - baz - nor case 2 - qux - matches anything (i.e. matches null).
Is there a simple way to do that with just jq 1.6?
Update:
Here are snippets of actual JSON files:
/* manifest.json, variant A */
{
"browser_specific_settings": {
"gecko": {
"id": "{95ad7b39-5d3e-1029-7285-9455bcf665c0}",
"strict_min_version": "68.0"
}
}
}
/* manifest.json, variant B */
{
"applications": {
"gecko": {
"id": "j30D-3YFPUvj9u9izFoPSjlNYZfF22xS#foobar",
"strict_min_version": "53.0"
}
}
}
I need the id values (*gecko.id so to speak) or error if there is none:
{95ad7b39-5d3e-1029-7285-9455bcf665c0}
j30D-3YFPUvj9u9izFoPSjlNYZfF22xS#foobar
error
You can use a filter as below that could work with both your sample JSON content provided
jq '.. | if type == "object" and has("id") then .id else empty end'
See them live jqplay - VariantA and jqplay - VariantB
Note: This only gets the value of .id when it is present, see others answers (oguz ismail's) for displaying error when the object does not contain the required field.
(.. | objects | select(has("id"))).id // "error"
This will work with multiple files and files containing multiple separate entities.
jqplay demo
You can use a combination of the ? "error suppression" and // "alternative` operators :
jq -n --slurpfile variantA yourFirstFile --slurpfile variantB yourSecondFile \
'(
($variantA[0].browser_specific_settings.gecko.id)?,
($variantB[0].applications.gecko.id)?
) // "error"'
This will output the id from the first file and/or the id from the second file if any of the two exist, avoiding to raise errors when they don't, and output error instead if none of them can be found.
The command can be shortened as follows if it makes sense in your context :
jq -n --slurpfile variantA yourFirstFile --slurpfile variantB yourSecondFile \
'(($variantA[0].browser_specific_settings, $variantB[0].applications) | .gecko.id)? // "error"'
I think you are looking for hasOwnProperty()
for example:
var value;
if(applications.gecko.hasOwnProperty('id'))
value = applications.gecko.id;
else
value = 'error';
I need the id values (*gecko.id so to speak) or error if there is none:
In accordance with your notation "*gecko.id", here are two solutions, the first interpreting the "*" as a single unknown key (or index), the second interpreting it (more or less) as any number of keys or indices:
.[] | .gecko.id // "error"
.. | objects | select(has("gecko")) | (.gecko.id // "error")
If you don't really care about whether there's a "gecko" key, you might wish to consider:
first(.. | objects | select(has("id")).id ) // "error"
Using jq I need to delete a specific key:value pair if value contains string "null". The string could be " null" or "null ". Hence need to use contains and not exact string match. If not a string it will be a number
My sample json is as below: (The null value is expected only in the 'th' and 'sf' keys)
'dets':{
'S1':{
'type':'class',
'input': [12,7,6,19],
'config':{
'file':'sfile1',
'th': -10,
'sf': 'null'
}
},
'S2':{
'type':'class',
'input': [12,7,6,19],
'config':{
'file':'sfile2',
'th': -5,
'sf': 3
}
},
'S3':{
'type':'bottom',
'input': [12,7,16],
'config':{
'file':'sfile3',
'th': ' null',
'sf': 'null '
}
}
}
The required output should be like:
'dets':{
'S1':{
'type':'class',
'input': [12,7,6,19],
'config':{
'file':'sfile1',
'th': -10
}
},
'S2':{
'type':'class',
'input': [12,7,6,19],
'config':{
'file':'sfile2',
'th': -5,
'sf': 3
}
},
'S3':{
'type':'bottom',
'input': [12,7,16],
'config':{
'file':'sfile3'
}
}
}
I believe something on the lines of del(.[][].config.smoothing_span | select(contains("null"))) but i am running into a problem since type is different.
The given data is not valid JSON but it is valid HJSON, so the first step in the following jq-oriented solution is to use hjson to convert the data to JSON:
hjson -j input.hjson
The concept of what values should be regarded as "null" might change over time, so in the following let's define a filter that can be used to capture whichever definition is appropriate, e.g.
def isnull: . == null or (type=="string" and test("null"));
(Perhaps a better definition would use test("^ *null *$").)
If you want to delete all keys whose value isnull, you could use walk/1:
walk(if type=="object"
then with_entries(select(.value|isnull|not))
else . end)
(If your jq does not have walk, you could simply copy-and-paste its definition from https://github.com/stedolan/jq/blob/master/src/builtin.jq or elsewhere on the web.)
Assuming your jq has walk, we could therefore write:
hjson -j input.hjson |
jq 'def isnull: . == null or (type=="string" and test("null"));
walk(if type=="object"
then with_entries(select(.value|isnull|not))
else . end)'
If you want to "walk" the input but restrict attention to specific keys, you can easily modify the selection criterion in (3).
delkey
If you want the scope of the changes to be as narrow as possible, you could use map_values, e.g. in conjunction with a helper function for checking and possibly deleting a specific key:
# key is assumed to be a filter, e.g. .foo
def delkey(key): if key | isnull then del(key) else . end;
.dets |= map_values(.config |= (delkey(.th) | delkey(.sf)))
delkeys
If there are several specific keys to be checked, it might be more convenient to define a function for checking a list of keys:
# keys is assumed to be an array of strings
def delkeys(keys):
with_entries(select(.key as $k
| (keys|index($k)) and (.value|isnull) | not));
.dets |= map_values(.config |= delkeys(["th", "sf"]))