This is my first post so apologies if I make mistakes. Consider I have the following json output
{
"records": [
{
"title": "root logon",
"login": "61819009",
"uid": "ajsd879asdjksasda123asd1asd1",
"password": "OizfD19jC$ySaV$MKpSF",
"login_url": "http://192.168.0.1/",
"notes": ""
},
{
"title": "important admin account",
"login": "admin",
"uid": "asdjhkasdh89eoajdiuas98ue9aoi",
"password": "0z5gDUC#Rb354TlLq$KJ",
"login_url": "",
"notes": "",
"folders": [
{
"shared_folder": "Department",
"folder": "Important",
"option1": false,
"option2": false
}
]
}
{
"title": "another important admin",
"login": "admin#domain.com",
"uid": "asjhe98asiajsijeouiaueiaiu",
"password": "3pUs#uXEqsxCv7PRkDlJ",
"login_url": "http://192.168.0.2/",
"notes": ""
},
{
"title": "switch admin",
"login": "admin",
"uid": "asjhe89ausiodjakljskea90ik",
"password": "hMB!eMsAE8q4aDQuM4LY",
"login_url": "",
"notes": "",
"folders": [
{
"shared_folder": "Department2",
"folder": "network\\switches",
"option1": false,
"option2": false
}
]
}
]
}
from the list above, I would like to only export title, login, uid, shared folder, and folder fields to csv file. When I run the following command
jq --raw-output '.records[] | [.title, .login, .uid, .shared_folder, .folder] | # tsv > file
what I am expecting as a CSV output is
title
login
uid
shared_folder
folder
root logon
61819009
ajsd879asdjksasda123asd1asd1
important admin account
admin
asdjhkasdh89eoajdiuas98ue9aoi
Department
Important
another important admin
admin#domain.com
asjhe98asiajsijeouiaueiaiu
switch admin
admin
asjhe89ausiodjakljskea90ik
Department2
network\switches
instead I'm getting the following CSV output:
title
login
uid
root logon
61819009
ajsd879asdjksasda123asd1asd1
important admin account
admin
asdjhkasdh89eoajdiuas98ue9aoi
another important admin
admin#domain.com
asjhe98asiajsijeouiaueiaiu
switch admin
admin
asjhe89ausiodjakljskea90ik
my goal is to eliminate capturing following fields
password
login_url
notes
option1
option2
You an error in your JSON, you must add a ,.
Once done, the command is this
jq --raw-output '.records[] | [.title, .login, .uid, .folders[0].shared_folder,.folders[0].folder] |#tsv'
Here's one way:
.records[]
| [.title, .login, .uid] + ((.folders[]? // null) | [.shared_folder, .folder])
| #tsv
Related
I'm trying to extract values from nested arrays in JSON below and output as CSV.
Fields to extract:
templates.name
items.name
triggers.name
Output as:
templates.name; items.name; triggers.name
Anticipated output something like:
"Template App Agent"; "Host name of zabbix_agentd running"; "Host name of zabbix_agentd was changed on {HOST.NAME}"
"Template App Agent"; "Agent ping"; "Zabbix agent on {HOST.NAME} is unreachable for 5 minutes"
"Template App Agent"; "Version of zabbix_agent(d) running"; ""
Note:
Not every item has a trigger.
Several triggers may exist for an item.
I'm new to JQ. So far only success is extracting the template name.
jq '.[] | {templates: [.templates[].name]}'
Data:
{
"zabbix_export": {
"version": "5.4",
"date": "2022-05-17T06:25:59Z",
"groups": [
{
"uuid": "7df96b18c230490a9a0a9e2307226338",
"name": "Templates"
}
],
"templates": [
{
"uuid": "e60e6598cf19448089a5f5a6c5d796a2",
"template": "Template App Agent",
"name": "Template App Agent",
"groups": [
{
"name": "Templates"
}
],
"items": [
{
"uuid": "24c03ed734d54dc8868a282a83a02200",
"name": "Host name of zabbix_agentd running",
"key": "agent.hostname",
"delay": "1h",
"history": "1w",
"trends": "0",
"value_type": "CHAR",
"request_method": "POST",
"tags": [
{
"tag": "Application",
"value": "Zabbix agent"
}
],
"triggers": [
{
"uuid": "d2d12d9e7dfe4fedb252f19b85e5e6aa",
"expression": "(last(/Template App Agent/agent.hostname,#1)<>last(/Template App Agent/agent.hostname,#2))>0",
"name": "Host name of zabbix_agentd was changed on {HOST.NAME}",
"priority": "INFO"
}
]
},
{
"uuid": "abacad4ca5eb46d29864d8a4998f1cbb",
"name": "Agent ping",
"key": "agent.ping",
"history": "1w",
"description": "The agent always returns 1 for this item. It could be used in combination with nodata() for availability check.",
"valuemap": {
"name": "Zabbix agent ping status"
},
"request_method": "POST",
"tags": [
{
"tag": "Application",
"value": "Zabbix agent"
}
],
"triggers": [
{
"uuid": "6d2a73199f3b4288bf36331a142c1725",
"expression": "nodata(/Template App Agent/agent.ping,5m)=1",
"name": "Zabbix agent on {HOST.NAME} is unreachable for 5 minutes",
"priority": "AVERAGE"
}
]
},
{
"uuid": "2cc337555efd43d181c28c792f8cbbdb",
"name": "Version of zabbix_agent(d) running",
"key": "agent.version",
"delay": "1h",
"history": "1w",
"trends": "0",
"value_type": "CHAR",
"request_method": "POST",
"tags": [
{
"tag": "Application",
"value": "Zabbix agent"
}
]
}
],
"valuemaps": [
{
"uuid": "3d66c59a28c04b0ca8227c87902ddb4d",
"name": "Zabbix agent ping status",
"mappings": [
{
"value": "1",
"newvalue": "Up"
}
]
}
]
}
]
}
}
.zabbix_export.templates[] | .name as $tn | .items[] | [ $tn, .name, .triggers[]?.name? ] | join("; ")
Loop over the templates
.zabbix_export.templates[]
Save the template name in a var
.name as $tn
Loop over the items
.items[]
Create an array with fields you like (including the name from step 1
[ $tn, .name, .triggers[]?.name? ]
Join the array to a string
join("; ")
Will output:
"Template App Agent; Host name of zabbix_agentd running; Host name of zabbix_agentd was changed on {HOST.NAME}"
"Template App Agent; Agent ping; Zabbix agent on {HOST.NAME} is unreachable for 5 minutes"
"Template App Agent; Version of zabbix_agent(d) running"
Online demo
This is a nested structure, you need to iterate level by level and add up the items you want to be in one output line. Store values from previous levels in variables.
To account for an inexistent .triggers array, you may use the Error Suppression Operator ? in combination with Alternative Operator //.
Finally, wrap the items in quotes (here using map), join them using join, and output them as raw text using the -r option
jq -r '
.[].templates[] | .name as $t
| .items[] | .name as $i
| [$t, $i, (.triggers[].name)? // ""]
| map("\"\(.)\"") | join("; ")
'
"Template App Agent"; "Host name of zabbix_agentd running"; "Host name of zabbix_agentd was changed on {HOST.NAME}"
"Template App Agent"; "Agent ping"; "Zabbix agent on {HOST.NAME} is unreachable for 5 minutes"
"Template App Agent"; "Version of zabbix_agent(d) running"; ""
Demo
Also consider using the #csv builtin, which gives you valid CSV right away (properly encoded (not just quoted) items, but separated with commas, not semicolons):
jq -r '
.[].templates[] | .name as $t
| .items[] | .name as $i
| [$t, $i, (.triggers[].name)? // ""]
| #csv
'
"Template App Agent","Host name of zabbix_agentd running","Host name of zabbix_agentd was changed on {HOST.NAME}"
"Template App Agent","Agent ping","Zabbix agent on {HOST.NAME} is unreachable for 5 minutes"
"Template App Agent","Version of zabbix_agent(d) running",""
Demo
I'm able to retrieve each user's data into m4-$u.json file with the below shell script
#!/bin/bash
USERID=ricardo.sanchez
PASSWORD=password
PORT=2728
for u in `cat user-list.txt`;
do echo $u;
curl --user $USERID:$PASSWORD http://198.98.99.12:46567/$PORT/protects/$u | jq '.' > m4-$u.json
done
One for the user's output of m4-daniel.json file few lines as follows.
[
{
"depotFile": "//ABND/JJEB/...",
"host": "*",
"isgroup": "",
"line": "16",
"perm": "open",
"user": "5G_USER_GROUP"
},
{
"depotFile": "//LIB/...",
"host": "*",
"isgroup": "",
"line": "19",
"perm": "write",
"user": "6G_USER_GROUP"
},
{
"depotFile": "//AND/RIO/...",
"host": "*",
"isgroup": "",
"line": "20",
"perm": "write",
"user": "AND_USER_GROUP"
},
During shell script run time additionally $PORT & $u variable values need to be added in output of each json file.
Expected json output:-
{
"depotFile": "//ABND/JJEB/...",
"host": "*",
"isgroup": "",
"line": "16",
"perm": "open",
"user": "5G_USER_GROUP",
"port": "2728",
"userid": "daniel"
},
To achieve this any help will be appreciated.
By using --arg with jq you can pass paremeter and use as a variables
( more info https://stedolan.github.io/jq/manual/v1.5/#Invokingjq )
by using map and + you can iterate over the array and add a new property for each associative array
in you case :
curl --user $USERID:$PASSWORD http://198.98.99.12:46567/$PORT/protects/$u \
| jq --arg a_port $PORT --arg $u $USER 'map(.+{"userid":$a_userid}+{"port":$a_port|tonumber})' > m4-$u.json
see addition and map in
https://stedolan.github.io/jq/manual/v1.5/#Builtinoperatorsandfunctions
I have a json output, from which I need to get id value and IPv4_address value where IPv4_address exists (this shouldn't be null). Have to use this ID value for another request along with random generated string.
Here is the breakdown of the requirement :
STEP 1 :
In the following example, for the ipv4_address:1.1.1.1 & ipv4_address:1.1.1.2, i need to get the id output which is "4e-0365-4e29-95ca-329165eecf8a" and "c9061b6674a8546cea" along with IP address.
Example of my output should look like (something similar):
1.1.1.1 4e-0365-4e29-95ca-329165eecf8a
1.1.1.2 c9061b6674a8546cea
I was trying to use jq but with this I'm not able to get the both values :
ID="$(echo "$test" \n | jq -r '.USER[] | select(.ipv4_address) | .ipv4_address')"
ID1="$(echo "$test" \n | jq -r '.USER[] | select(.ipv4_address) | .id')"
Sample output which is getting displayed with the above 2 commands :
ID value is : 1.1.1.1 1.1.1.2
ID1 value is : 4e-0365-4e29-95ca-329165eecf8a c9061b6674a8546cea
STEP 2: Profile creation: I need to use each $ID1 value in another request along with random generated string. Random string is generated as per the count of $ID1's (so here I will generate 2 random string)
And thus 2 profiles are created.
Ques: How can I get each ID from the $ID1 variable ? I tried something like ID1[0] but that seems to be wrong
STEP 3 :
Will use each ID and random string for another request, Once its done or if that step is failed, i need to provide the output to a file & output should look like :
My requirement for the final output is :
1.1.1.1 4e-0365-4e29-95ca-329165eecf8a <randomvalue-1> <profile-1> DONE
1.1.1.2 c9061b6674a8546cea <randomvalue-2> <profile-2> FAILED
where random value will be generated randomly and shall be used against the ID.
JSON output which needs to be parsed:
{
"errorcode": 0,
"message": "Done",
"operation": "get",
"resourceType": "USER",
"username": "root",
"tenant_name": "Owner",
"tenant_id": "05db6674ad458546cd2",
"resourceName": "",
"USER": [
{
"is_default": "false",
"session_timeout": "0",
"permission": "root",
"name": "ee",
"session_timeout_unit": "",
"tenant_id": "55bcb6674ad45854",
"id": "4e-0365-4e29-95ca-329165eecf8a",
"ipv4_address": "1.1.1.1",
"state": "Up",
"tenant_name": "Owner",
"encrypted": "false",
"groups": [
"owner"
],
"root_user": ""
},
{
"is_default": "false",
"session_timeout": "0",
"permission": "read",
"name": "test",
"session_timeout_unit": "",
"tenant_id": "bc906674ad458546cd2",
"id": "12cd0-fb7f-4abf-b060-48e98b794b06",
"tenant_name": "Owner",
"encrypted": "false",
"groups": [
"read"
],
"root_user": ""
},
{
"is_default": "true",
"session_timeout": "0",
"permission": "root",
"name": "root",
"session_timeout_unit": "",
"tenant_id": "c905db6d458546cd2",
"id": "c9061b6674a8546cea",
"ipv4_address": "1.1.1.2",
"state": "Not Reachable",
"tenant_name": "Owner",
"encrypted": "false",
"groups": [
"owner"
],
"root_user": ""
},
{
"is_default": "false",
"session_timeout": "0",
"permission": "readonly",
"name": "a",
"session_timeout_unit": "",
"tenant_id": "c905674ad458546cd2",
"id": "bc8a-4fd6-bc09-8c39c131b54e",
"tenant_name": "Owner",
"encrypted": "false",
"groups": [
"read"
],
"root_user": ""
}
]
}
Not quite clear with the logic of marking it DONE and FAILED. But to answer your first question where you want to select the multiple fields, you can do something like this:
$ cat input.js | jq -r '.USER[] | select(.ipv4_address) | "\(.ipv4_address) \(.id)"' > result.js
This will output the result in a file named result.js. You can apply your custom logic of marking DONE and Failed on this file.
In the above command when you do select(.ipv4_address) It basically drops all the records for which ipv4_address value is null or it is not present.
if you want to select the records which have ipv4_address as null, then your select statement would become something like this
select(.ipv4_address == null)
I've a json file from a az cli command, I want to deploy with a powershell script from a tool like jenkins or rundeck, I need to take the field "name" from json but not entirely only the word after forward slash
[
{
"Group": "KC-EMEA-RSGP-BPTRAINING-01",
"id": "/subscriptions/KC-EMEA-RSGP-BPTRAINING-01/providers/Microsoft.Sql/servers/",
"name": "kc-emea-sqsrv-bptraining-dev-01/master"
},
{
"Group": "KC-EMEA-RSGP-NAVISIONKM-DEV-01",
"id": "/subscriptions/KC-EMEA-RSGP-NAVISIONKM-DEV-01/providers/Microsoft.Sql/servers/",
"name": "km-emea-sqsrv-navision-tst-01/km-emea-sqdb-navision-tst-01"
},
{
"Group": "KC-EMEA-RSGP-NAVISIONKM-DEV-01",
"id": "/subscriptions/KC-EMEA-RSGP-NAVISIONKM-DEV-01/providers/Microsoft.Sql/servers/",
"name": "km-emea-sqsrv-navision-tst-01/master"
},
{
"Group": "KC-EMEA-RSGP-PROJECTS-DEV-01",
"id": "/subscriptions/KC-EMEA-RSGP-PROJECTS-DEV-01/providers/Microsoft.Sql/servers/",
"name": "kc-emea-sqsrv-projects-dev-01/KC-EMEA-SQDB-BPTRAINING-TRAINEE-01"
}
]
$file = "pathtojsonfile.json"
$jsonContent = Get-Content $file | ConvertFrom-Json;
$namedb = $jsonContent.name;
$value= $namedb.Substring($namedb.IndexOf('/')+1) --> this doesn't work.
Try
$namedb[0].Substring($namedb[0].IndexOf('/')+1)
And for all of them:
foreach($name in $namedb) {$name.Substring($name.IndexOf('/')+1)}
Using jq, I would like to output multiple values on different levels of a JSON file based on whether they exist in an array.
My data looks like the following. It displays a number of hosts I examine regarding the people who have access to it:
[
{
"server": "example_1",
"version": "Debian8",
"keys": [
{
"fingerprint": "SHA256:fingerprint1",
"for_user": "root",
"name": "user1"
},
{
"fingerprint": "SHA256:fingerprint2",
"for_user": "git",
"name": "user2"
}
]
},
{
"server": "example_2",
"version": "Debian9",
"keys": [
{
"fingerprint": "SHA256:fingerprint2",
"for_user": "root",
"name": "user2"
},
{
"fingerprint": "SHA256:fingerprint2",
"for_user": "www",
"name": "user2"
}
]
},
{
"server": "example_3",
"version": "CentOS",
"keys": [
null
]
}
]
I want to extract the value for server and the value of for_user any occurence where user2 is found as a name in .keys[]. Basically, the output could look like this:
example1, git
example2, root
example2, www
What I can already do is displaying the first column, so the .server value:
cat test.json | jq -r '.[] | select(.keys[].name | index("user2")) | .server'`
How could I also print a value in the selected array element?
You can use the following jq command:
jq -r '.[]|"\(.server), \(.keys[]|select(.name=="user2").for_user)"'