String Manipulation within a JSON array using jq - json

I am writing a bash script and I am looking to replace a character within a JSON field in a JSON array. In this case, I am trying to change the "." (period) character to a "-" (hyphen) in the name field. I am using jq to parse my JSON. Any tips on how I can achieve this will greatly help. Thank you!
Bash Script so far:
RAW=$(curl ${URL})
function manip() {
# Function for string manipulation.
}
echo "${RAW}" | jq '.data | .[].name = $manip' # Unable to make a function call in there.
Sample JSON:
[
{"id":"1","name":"a.a"},
{"id":"2","name":"b.b"},
{"id":"3","name":"c.c"}
]
Expected Output:
[
{"id":"1","name":"a-a"},
{"id":"2","name":"b-b"},
{"id":"3","name":"c-c"}
]


To replace a dot with a dash, use the sub function:
jq '.[].name |= sub("\\."; "-")' file.json

Related

How to prettify json with jq given a string with escaped double quotes

I would like to pretty print a json string I copied from an API call which contains escaped double quotes.
Similar to this:
"{\"name\":\"Hans\", \"Hobbies\":[\"Car\",\"Swimming\"]}"
However when I execute pbpaste | jq "." the result is not changed and is still in one line.
I guess the problem are the escaped double quotes, but I don't know how to tell jq to remove them if possible, or any other workaround.

What you have is a JSON string which happens to contain a JSON object. You can decode the contents of the string with the fromjson function.
$ pbpaste | jq "."
"{\"name\":\"Hans\", \"Hobbies\":[\"Car\",\"Swimming\"]}"
$ pbpaste | jq "fromjson"
{
"name": "Hans",
"Hobbies": [
"Car",
"Swimming"
]
}

Extract part of the string elements in an array using jq

I'm trying to extract part of the string in elements of an array, and create a new array with these extractions.
[
"local/binaries/app-2.21.0.tar.gz",
"local/binaries/app-2.20.0.tar.gz",
"local/binaries/app-2.19.1.tar.gz",
"local/binaries/app-2.19.0.tar.gz",
"local/binaries/app-2.18.0.tar.gz"
]
Desired output
[
"app-2.21.0",
"app-2.20.0",
"app-2.19.1",
"app-2.19.0",
"app-2.18.0"
]

You can use jq's capture function with regular expressions.
jq '[.[] | capture("(?<captured>app-[0-9]+\\.[0-9]+\\.[0-9]+)") | .[]]'
Try it out on jq playground.
Documentation: https://stedolan.github.io/jq/manual/#RegularexpressionsPCRE

Create//append JSON array from text file in linux with loop

I have a below file in txt format. I want to arrange the data in json array format in linux and append more such data with for/while loop in the same json array based on condition. Please help me with the best way to achieve this.
File
Name:Rock
Name:Clock
{“Array" :[
{
"Name": "Rock",
},
{
"Name”: "Clock”,
}
]

Suppose your initial file is object.json and that it contains an empty object, {};
and that at the beginning of each iteration, the key:value pairs are defined in another file, kv.txt.
Then at each iteration, you can update array.json using the invocation:
< kv.txt jq -Rn --argfile object object.json -f program.jq | sponge object.json
where program.jq contains the jq program:
$object | .Array +=
reduce inputs as $in ([]; . + [$in | capture("(?<k>^[^:]*): *(?<v>.*)") | {(.k):.v} ])
(sponge is part of the moreutils package. If it cannot be used, then you will have to use another method of updating object.json.)

replace comma in json file's field with jq-win

I have a problem in working JSON file. I launch curl in AutoIt sciript to download a json file from web and then convert it to csv format by jq-win
jq-win32 -r ".[]" -c class.json>class.txt
and the json is in the following format:
[
{
"id":"1083",
"name":"AAAAA",
"channelNumber":8,
"channelImage":""},
{
"id":"1084",
"name":"bbbbb",
"channelNumber":7,
"channelImage":""},
{
"id":"1088",
"name":"CCCCCC",
"channelNumber":131,
"channelImage":""},
{
"id":"1089",
"name":"DDD,DDD",
"channelNumber":132,
"channelImage":""},
]
after jq-win, the file should become:
{"id":"1083","name":"AAAAA","channelNumber":8,"channelImage":""}
{"id":"1084","name":"bbbbb","channelNumber":7,"channelImage":""}
{"id":"1088","name":"CCCCCC","channelNumber":131,"channelImage":""}
{"id":"1089","name":"DDD,DDD","channelNumber":132,"channelImage":""}
and then the csv file will be further process by the AutoIt script and become:
AAAAA,1083
bbbbb,1084
CCCCCC,1088
DDD,DDD,1089
The json has around 300 records and among them, 5~6 record has comma in it eg DDD,DDD
so when I tried read in the csv file by _FileReadToArray, the comma in DDD,DDD cause trouble.
My question is: can I replace comma in the field using jq-win ?
(I tried use fart.exe but it will replace all comma in json file which is not suitable for me.)
Thanks a lot.
Regds
LAM Chi-fung

can I replace comma in the field using jq-win ?
Yes. For example, use gsub, pretty much as you’d use awk’s gsub, e.g.
gsub(","; "|")
If you want more details, please provide more details as per [mcve].
Example
With the given JSON input, the jq program:
.[]
| .name |= gsub(",";";")
| [.[]]
| map(tostring)
| join(",")
yields:
1083,AAAAA,8,
1084,bbbbb,7,
1088,CCCCCC,131,
1089,DDD;DDD,132,

Parsing errors of JSON using jq

I have a curl command which results in the following example json:
json={"id":"12345","key":"ABC-DEF","url":"https://google.com"}
Now, I want to parse this, and get the key out of it and store it in a variable. What I did was the following:
json={"id":"12345","key":"ABC-DEF","url":"https://google.com"}
ID=$(echo $json | jq '.key' )
But the above gives me a error as: parse error: Invalid numeric literal at line 1, column 4. Can someone help me with this? Thanks!

You will need to quote the input string so the shell doesn't do anything with your string
json='{"id":"12345","key":"ABC-DEF","url":"https://google.com"}'
ID=$(echo "$json" | jq '.key' )