Basically, I have a retrieved values from a DynamoDB using Powershell and have gotten a row in JSON format like the following
function gettagsfromdynamo() {
$table_name = "table_name"
$dbkey = '{"ReleaseId":{"AttributeValueList":[ {"N":"1"} ],"ComparisonOperator": "EQ"}}' | ConvertTo-Json -Compress
$dbvalue = aws dynamodb query --table-name $table_name --key-conditions $dbkey --region $region
$latest_tags = $dbvalue
$latest_tags
}
$db_tags = gettagsfromdynamo
This is what db_tags looks like
{
"Items": [
{
"Comment": {
"S": "The first PC release."
},
"Product": {
"S": "PC"
},
"ReleaseId": {
"N": "1"
},
"CreatedOn": {
"S": "12/14/2020 15:23:32"
},
"Tags": {
"S": "{\n \"software1\": \"software1.zip\",\n \"software2\": \"software2.zip\",\n \"software3\":\n [\n \"software3.zip\",\n \"software4.zip\",\n \"software5.zip\"\n ],\n \" data1 \": \"2020_NA\",\n \" 2020_EU \": \"20201_EU\",\n \" 2020_WW \": \"2021_WW\",\n \" dataversions\":\n [\n \"2020\",\n \"2019\",\n \"2018\",\n \"2017"\n ],\n \" products \": \" \"\n}"
}
}
],
"Count": 1,
"ScannedCount": 1,
"ConsumedCapacity": null
}
The task I want to achieve is to be able to get the "dataversions" value which is --> [Items][Tags][Dataversions] and write that value to a JSON file available locally. I have tried various things including using the Convert-ToJson and ConvertFrom-Json.
The tags json value looks like this without the escaped spaces (/n)
{
"software1": "software1.zip",
"software2": "software2.zip",
"software3":
[
"software3.zip",
"software4.zip",
"software5.zip"
],
" data1": "2020_NA",
" 2020_eu ": "2020_EU",
" 2020_ww": "2021_WW",
" dataversions":
[
"2020",
"2019",
"2018",
"2017"
],
" products ": " "
}
How do I retrieve the value of 'dataversions', which is a list of strings. Right now, I can only get it like this after using various tries:
{"S":"{\n \"software1\": \"software1.zip\",\n \"software2\": \"software2.zip\",\n \"software3\":\n [\n \"software3.zip\",\n \"software4.zip\",\n \"software5.zip\"\n ],\n \" data1\": \"2020_NA\",\n \" 2020_EU\": \"20201_EU\",\n \" 2020_WW\": \"2021_WW\",\n \" dataversions\":\n [\n \"2020\",\n \"2019\",\n \"2018\",\n \"2017\"\n ],\n \" products\": \" \"\n}"}
I want to be able to get the value of dataversions inorder to overwrite another 'dataversions' which is inside example.json file. How do I get to the dataversions value and also clean up the \n?
In your JSON file, the property name " dataversions" contains a leading space. When the JSON string is converted to a custom object (via ConvertFrom-Json), the space will be included in the property name. Therefore it must be considered when using member access (object.property) syntax:
($db_tags | ConvertFrom-Json).' dataversions'
Related
I have the following JSON
{
"jps": {
"onInstall": [{
"cmd [sqldb]": [
"command"
]
}]
}
}
Without [sqldb], I was doing the query as follows:
jq '.jps.onInstall.cmd=["long line", "another line"]' my-app.json
And it worked, but with [sqldb], how can I translate it to jq query format?
I tried
jps.onInstall.cmd\[sqldb\] and jps.onInstall.cmd.sqldb but no luck
The problem is not with the special character "cmd [sqldb]" but with your onInstall type, which is a list type, but you are missing the .[] indicator
.jps.onInstall[]."cmd [sqldb]" = ["long line", "another line"]
or
.jps.onInstall[]["cmd [sqldb]"] = ["long line", "another line"]
I have json format Splunk search results like below :
"{
"Name": "RUNQDATA",
"RunId": "2021021701",
"Details": <{
"RunQID": "796562",
"TQID": "796562",
"Ent": {
"NAME": "Inv",
"Store": {
"NAME": "FSW",
"TYPE": "QUEUE",
"USERNAME": "abc"
}
},
"ADD_COUNT": "5740",
"UPDATE_COUNT": "0",
"DELETE_COUNT": "0"
}>,
"status": "success",
}"
How can I extract the fields like ADD_COUNT or UPDATE_COUNT from this ? I tried spath & other options , however not able to get the required results. Probably because the json contains <>.
Any help here is appreciated.
Confirmed. If the angle brackets are removed then the spath command will parse the whole thing. The spath command doesn't handle malformed JSON.
If you can't change the format of the event then you'll have to use the rex command to extract the fields as in this run-anywhere example
| makeresults
| eval _raw="{
\"Name\": \"RUNQDATA\",
\"RunId\": \"2021021701\",
\"Details\": <{
\"RunQID\": \"796562\",
\"TQID\": \"796562\",
\"Ent\": {
\"NAME\": \"Inv\",
\"Store\": {
\"NAME\": \"FSW\",
\"TYPE\": \"QUEUE\",
\"USERNAME\": \"abc\"
}
},
\"ADD_COUNT\": \"5740\",
\"UPDATE_COUNT\": \"0\",
\"DELETE_COUNT\": \"0\"
}>,
\"status\": \"success\",
}"
| rex "UPDATE_COUNT\": \"(?<UPDATE_COUNT>\d+)"
| rex "DELETE_COUNT\": \"(?<DELETE_COUNT>\d+)"
I have the following json layout:
test.json
{
"end": 9,
"previous_page_uri": null,
"messages": [
{
"error_message": null,
"num_media": "1",
"status": "received"
},
{
"error_message": null,
"num_media": "2",
"status": "received"
}
],
"end1": "end page 1",
"end2": "end page 2"
}
I want to output the .messages object as csv, followed by the "end1" value.
Is there a way to do that in jq?
To produce the csv:
jq '.messages[] | [.error_message, .num_media, .status]|#csv' test.json
which produces this:
",\"1\",\"received\""
",\"2\",\"received\""
How can I add .end1?
You can use , Comma to concatenate the output of two filters and ( ) Parenthesis to specify them separately. For example with the sample input you provided the filter
( .messages[] | [.error_message, .num_media, .status] | #csv ), .end1
generates
",\"1\",\"received\""
",\"2\",\"received\""
"end page 1"
Try it online!
Hey I am using conduit curl method to create tasks from post. It work fine when I run from terminal with hardcoded values. But when I try to execute it with variables it throws an error:
Script:
#!/bin/bash
echo "$1"
echo "$2"
echo "$3"
echo "$4"
echo "$5"
echo '{
"transactions": [
{
"type": "title",
"value": "$1"
},
{
"type": "description",
"value": "$2"
},
{
"type": "status",
"value": "$3"
},
{
"type": "priority",
"value": "$4"
},
{
"type": "owner",
"value": "$5"
}
]
}' | arc call-conduit --conduit-uri https://mydomain.phacility.com/ --conduit-token mytoken maniphest.edit
execution:
./test.sh "test003 ticket from api post" "for testing" "open" "high" "ahsan"
Output:
test003 ticket from api post
for testing
open
high
ahsan
{"error":"ERR-CONDUIT-CORE","errorMessage":"ERR-CONDUIT-CORE: Validation errors:\n - User \"$5\" is not a valid user.\n - Task priority \"$4\" is not a valid task priority. Use a priority keyword to choose a task priority: unbreak, very, high, kinda, triage, normal, low, wish.","response":null}
As you can see in error its reading $4 and $5 as values not variables. And I am failing to understand how to use $variables as input in these arguments.
You're using single quotes around the last echo to so that you can use double-quotes inside the JSON, but that causes echo to print the string without expanding anything. You need to use double quotes for the string, so you'll have to escape the double quotes inside of it.
Replace the last echo with this:
echo "{
\"transactions\": [
{
\"type\": \"title\",
\"value\": \"$1\"
},
{
\"type\": \"description\",
\"value\": \"$2\"
},
{
\"type\": \"status\",
\"value\": \"$3\"
},
{
\"type\": \"priority\",
\"value\": \"$4\"
},
{
\"type\": \"owner\",
\"value\": \"$5\"
}
]
}"
and it'll work. To avoid issues like this you can check http://wiki.bash-hackers.org and http://mywiki.wooledge.org/BashGuide for some general tips for bash newbies. Also, you can use shellcheck with a lot of text editors, which will spot errors like this automatically.
JSON in question:
{
"search_id": "",
"type": "Search.filter",
"query": "bar,club",
"params": {
"search_id": "",
"user_id": "",
"client": "ios",
"lat": 40.73199375351,
"lon": -74.00080404533901,
"radius": 20
}
}
Code to Retrieve the Data:
val json = Json.parse(new String(body))
println((json \ "search_id") + " | " + (json \ "query"))
println(json)
printing just the json JsValue prints out the entire JSON as expected. printing out the first item produces: "" | "bar,club"
Why is it maintaining the quotes from JSON formatting? That's not part of the string, it's basically saying that the content inside the quotes is a string. How do I fix this?
According to the doc, you should call .as[sometype] (unsafe conversion) or asOpt[sometype] (safe).
println((json \ "search_id").as[String] + " | " + (json \ "query").as[String])