My question is very similar to one found here -
I have the following JSON
{
"FOO": {
"id": "23432423",
"Result": {
"Status": "SUCCESS",
"Reason": ""
}
},
"BAR": {
"id": "45345535",
"Result": {
"Status": "FAILURE",
"Reason": ""
}
},
"BAZ": {
"id": "123432423",
"Result": {
"Status": "SUCCESS",
"Reason": ""
}
}
}
Using jq I wanted the original object format back filtering on status FAILED
Result:
"BAR": {
"id": "45345535",
"Result": {
"Status": "FAILURE",
"Reason": ""
}
}
I tried both solutions suggested from above post
to_entries | map(select(.value.Status=="FAILURE")) | from_entries and 'with_entries(select(.value.Status =="FAILURE"))' both are giving empty results. Going round and round. Any help apprecaited
Your Status property is nested inside a Result object, but you are selecting Status directly. You must select on .value.Result.Status:
with_entries(select(.value.Result.Status == "FAIL"))
map_values is a bit shorter even:
map_values(select(.Result.Status == "FAIL"))
Output:
{
"BAR": {
"id": "x....4",
"Result": {
"Status": "FAIL",
"Reason": ""
}
}
}
Related
I have json file with multiple domains which is formated as is showed below. How can I delete whole blocks with domains? For example if I will want to delete whole block in json for domain domain.tld?
I tryed this, but output is error:
jq '."http-01"."domain"[]."main"="domain.tld"' acme.json
jq: error (at acme.json:11483): Cannot iterate over null (null)
formating example file:
{
"http-01": {
"Account": {
"Email": "mail#placeholder.tld",
"Registration": {
"body": {
"status": "valid",
"contact": [
"mailto:mail#placeholder.tld"
]
},
"uri": "https://acme-v02.api.letsencrypt.org/acme/acct/110801506"
},
"PrivateKey": "main_priv_key_string",
"KeyType": "4096"
},
"Certificates": [
{
"domain": {
"main": "www.some_domain.tld"
},
"certificate": "cert_string",
"key": "key_string",
"Store": "default"
},
{
"domain": {
"main": "some_domain.tld"
},
"certificate": "cert_string",
"key": "key_string",
"Store": "default"
},
{
"domain": {
"main": "www.some_domain2.tld"
},
"certificate": "cert_string",
"key": "key_string",
"Store": "default"
},
{
"domain": {
"main": "some_domain2.tld"
},
"certificate": "cert_string",
"key": "key_string",
"Store": "default"
}
]
}
}
To delete domain block "www.some_domain.tld" :
jq '."http-01".Certificates |= map(select(.domain.main != "www.some_domain.tld"))' input.json
Your question is quite broad. What is a "block"?
Let's assume you want to delete from within the object under http-01 each field that is of type array and has at index 0 an object satisfying .domain.main == "domain.tld". Then first navigate to where you want to delete from, and update it (|=) using del and select which performs the filtered deletion.
jq '
."http-01" |= del(
.[] | select(arrays[0] | objects.domain.main == "domain.tld")
)
' acme.json
{
"http-01": {
"Account": {
"Email": "email#domain.tld",
"Registration": {
"body": {
"status": "valid",
"contact": [
"mailto:email#domain.tld"
]
},
"uri": "https://acme-v02.api.letsencrypt.org/acme/acct/110801506"
},
"PrivateKey": "long_key_string",
"KeyType": "4096"
}
}
}
Demo
If your "block" is deeper, go deeper before updating. If it is higher, the whole document for instance, there's no need to update, just start with del.
I am new to jq and I'm trying to use it to search for a value in a json file based on a key that is located deep in the json structure. Here is a sample of my json file:
{
"data": {
"inventory": {
"location": "remote",
"list": {
"content": [
{
"item": {
"name": "minivan"
},
"owner": {
"id": "12345",
"state": "CA"
}
},
{
"item": {
"name": "sedan"
},
"owner": {
"id": "67890",
"state": "AZ"
}
}
]
}
}
}
}
An example of search that I'm trying to do is:
select item.name where owner.id = "67890"
and the expected output would be:
item.name = "sedan"
I'm trying to run the following:
jq '.[] | select .owner.id = "67890" | .item.name' json
and it generates an error:
jq: error: select/0 is not defined at <top-level>, line 1:
.[] | select .owner.id = "67890" | .item.name
jq: 1 compile error
Any pointers on how to do this in jq would be much appreciated!
Thanks!
First, you have to "navigate" to where you want to make the query. This seems to be an array.
.data.inventory.list.content
[
{
"item": {
"name": "minivan"
},
"owner": {
"id": "12345",
"state": "CA"
}
},
{
"item": {
"name": "sedan"
},
"owner": {
"id": "67890",
"state": "AZ"
}
}
]
Demo
Next, let's iterate over that array's items, which gives us a stream of objects.
.[]
{
"item": {
"name": "minivan"
},
"owner": {
"id": "12345",
"state": "CA"
}
}
{
"item": {
"name": "sedan"
},
"owner": {
"id": "67890",
"state": "AZ"
}
}
Demo
From these objects we select those that match your criteria.
select(.owner.id == "67890")
{
"item": {
"name": "sedan"
},
"owner": {
"id": "67890",
"state": "AZ"
}
}
Demo
Finally, we extract the value you're interested in.
.item.name
"sedan"
Demo
Everything combined in a jq call would be:
jq '.data.inventory.list.content[] | select(.owner.id == "67890").item.name'
"sedan"
Demo
This output is still valid JSON document (containing nothing but a JSON string). If you want to process the output as raw text, use the --raw-output (or -r) option:
jq -r '.data.inventory.list.content[] | select(.owner.id == "67890").item.name'
sedan
Demo
Here's a solution that avoids having to "navigate" to the right place, and which is also quite close to your SQL-like query:
..
| objects
| select(.owner and
(.owner|type=="object" and .id == "67890"))
.item.name
or more succinctly:
..|objects|select(.owner.id? == "67890").item.name
I have a JSON file that has following contents:
{
"status": "UP",
"details": {
"graphDBCheck": {
"status": "UP"
},
"ds": {
"status": "UP",
"details": {
"total": 100,
"free": 50,
"threshold": 30
}
},
"db": {
"status": "UP",
"details": {
"ADS": {
"status": "UP",
"details": {
"database": "Influx",
"hello": "Hello"
}
},
"EARDS": {
"status": "UP",
"details": {
"database": "Oracle",
"hello": "Hello"
}
},
"EFRDS": {
"status": "UP",
"details": {
"database": "Sybase",
"hello": "Hello"
}
}
}
}
}
}
I need to be able to transform this into a CSV file that has each element's name as header and it's status or value as the next row. First "status" would have column name "API_Status"
For instance:
API_Status,graphDBCheck,ds,db,ADS,EARDS,EFRDS
UP,UP,UP,UP,UP,UP,UP
Challenge here is to make this dynamic so the output will always include any other element added that has "status" in it.
I tried this and it works but I need a dynamic way do to this:
$x = Invoke-RestMethod $url -Verbose:$VerbosePreference
[pscustomobject][ordered]#{
'API_Status' = $x.status
'db' = $x.details.db.status
'ds' = $x.details.diskspace.status
'ds_Total' = $x.details.ds.details.total
'ds_Free' = $x.details.ds.details.free
'graphDBCheck' = $x.details.graphDBCheck.status
'ADS' = $x.details.db.details.auroraDataSource.status
'EARDS' = $x.details.db.details.EARDS.status
'EFRDS' = $x.details.db.details.edsFirstRowsDataSource.status
}
In an ideal world, the json would be structured like this, as an expandable array with uniform properties.
[
{
"name": "API_Status",
"status": "UP"
},
{
"name": "graphDBCheck",
"status": "UP"
},
{
"name": "ds",
"status": "UP"
},
{
"name": "db",
"status": "UP"
},
{
"name": "ADS",
"status": "UP"
},
{
"name": "EARDS",
"status": "UP"
},
{
"name": "EFRDS",
"status": "UP"
}
]
Or as a csv:
name,status
API_Status,UP
graphDBCheck,UP
ds,UP
db,UP
ADS,UP
EARDS,UP
EFRDS,UP
There's plenty of other posts about looping through powershell properties Iterate over PSObject properties in PowerShell or looping through json properties: Iterating through a JSON file PowerShell
I'm getting the following error when using the Google API Explorer to insert into GCP's Datastore.
I've tried using another name and ID, but still the error. How do I rectify this?
Below are the request body and error.
{
"mode": "Transactional",
"mutations": [
{
"insert": {
"key": {
"path": [
{
"id": "56294995342131231",
"name": "CL-001",
"kind": "Log"
}
],
"partitionId": {
"namespaceId": "",
"projectId": "triplog-169706"
}
},
"properties": {
"Title": {
"stringValue": "Space Needle"
},
"Longitude": {
"doubleValue": 0.00
},
"Latitude": {
"doubleValue": 0.00
},
"Date": {
"timestampValue": "2015-07-03T10:51:50.649Z"
},
"Rating": {
"integerValue": "5"
},
"Notes": {
"stringValue": "Wonderful site to see"
}
}
}
}
]
}
{
"error": {
"code": 400,
"message": "Invalid value at 'mutations[0].insert.key.path[0]' (oneof), oneof field 'id_type' is already set. Cannot set 'name'",
"status": "INVALID_ARGUMENT",
"details": [
{
"#type": "type.googleapis.com/google.rpc.BadRequest",
"fieldViolations": [
{
"field": "mutations[0].insert.key.path[0]",
"description": "Invalid value at 'mutations[0].insert.key.path[0]' (oneof), oneof field 'id_type' is already set. Cannot set 'name'"
}
]
}
]
}
}
Based on the docs, it looks like you should only be setting either a "name" or "id" in the Path Element, not both.
https://cloud.google.com/datastore/docs/reference/rest/v1/Key#PathElement
I've finally got it.
To get the "transaction", just run the beginTransaction API.
POST https://datastore.googleapis.com/v1/projects/{projectId}:beginTransaction
This will response with an identifier, which is a long string. Use that as the value of the "transaction".
I have the following JSON message. This is a single complete message. There are so many messages like this in a single file. This json message was generated from a unformated json message using jq.
{
"header": {
"user": "baskar"
},
"requests": [
{
"first_name": "mike",
"last_name": "mat"
},
{
"first_name": "mike",
"last_name": "mat"
}
],
"check": [
"Y"
]
}
{
"header": {
"user": "baskar"
},
"message": {
"header": {
"user": "baskar"
},
"response": {
"resultsList": {
"result": [
{
"first_name": "mike1",
"last_name": "mat"
}
]
},
"errorMsg": null
}
}
}
I would like to do some filtering on this. For example, when i search for first_name, mike1, I should get the header and the matching request inside request. Also the matching result inside the Response message. So, the output is expected as follows for the search string mike1.
{
"header": {
"user": "baskar"
},
"requests": [
{
"first_name": "mike1",
"last_name": "mat"
}
],
"check": [
"Y"
]
}
{
"header": {
"user": "baskar"
},
"message": {
"header": {
"user": "baskar"
},
"response": {
"resultsList": {
"result": [
{
"first_name": "mike1",
"last_name": "mat"
}
]
},
"errorMsg": null
}
}
}
Basically, i want to filter out unmatched request inside the requests array and unmatched result inside the result array.
Currently, I use the following script to get the formatted json message from the unformatted json message log file.
sed -n "/<SEARCH_STRING>/ s/.*Service - //p" $1/test.log* | jq . > ~/result.log
Thanks,
Baskar.S
jq --arg key first_name \
--arg value mike1 \
'select(.message.response.resultsList.result[]?[$key]==$value) | .message' \
<in.json
...returns only message content where the result list contains at least one first_name of mike1.