I have a json file named output.json. It has a simple key:value format, e.g.:
{
"key":"value",
"key":"value",
"key":"value",
"key":"value",
}
I want to extract "value part".
If anyone can write me a command that will be really helpful.
With jq (which is much better suited for parsing and filtering JSON than grep/sed/awk/etc) you can extract all values with values function:
$ echo '{"a":1, "b":2, "c":3}' | jq '.[]|values'
1
2
3
Alternatively (since you mention you already use Python in your pipeline), you can do it like:
#!/usr/bin/env python
import json
my_values = json.load('output.json').values()
Related
I am writing script (bash script for Azure pipeline) and I need to combine JSON from different variables. For example, I have:
TYPE='car'
COLOR='blue'
ADDITIONAL_PARAMS='{"something": "big", "etc":"small"}'
So, as you can see, I have several string variables and one which consist JSON.
I need to combine these variables with this format (and I cant :( ):
some_script --extr-vars --extra_vars '{"var_type": "'$TYPE'", "var_color": "'$COLOR'", "var_additional_data": "'$ADDITIONAL_PARAMS'"}'
But this combination is not working, I have a string something like:
some_script --extr-vars --extra_vars '{"var_type": "car", "var_color": "blue", "var_additional_data": " {"something": "big", "etc":"small"} "}'
which is not correct and valid JSON.
How I can combine existing JSON (already formatted with double quotes ") with other variables? I am using bash / console / yq utilite (to convert yaml to json)
Use jq to generate the JSON. (You can probably do this in one step with yq, but I'm not as familiar with that tool.)
ev=$(jq --arg t "$TYPE" \
--arg c "$COLOR" \
--argjson ap "$ADDITIONAL_PARAMS" \
-n '{var_type: $t, var_color: $c var_additional_data: $ap}')
some_script --extr-vars --extra_vars "$ev"
I want to filter JSON on the command line.
Task: print the "name" of each dictionary in the json list.
Example json:
[
{
"id":"d963984c-1075-4d25-8cd0-eae9a7e2d130",
"extra":{
"foo":false,
"bar":null
},
"created_at":"2020-05-06T15:31:59Z",
"name":"NAME1"
},
{
"id":"ee63984c-1075-4d25-8cd0-eae9a7e2d1xx",
"name":"NAME2"
}
]
Desired output:
NAME1
NAME2
This script would work:
#!/usr/bin/env python
import json
import sys
for item in json.loads(sys.stdin.read()):
print(item['name'])
But since I am very lazy, I am looking for a solution where I need to type less. For example in on the command line in a pipe:
curl https://example.com/get-json | MAGIC FILTER
I asked at code golf but they told me that it would make more sense to ask here.
You can use jq https://stedolan.github.io/jq/manual/
% curl https://example.com/get-json | jq -r '.[].name'
NAME1
NAME2
-r if the filter's result is a string then it will be written directly to standard output rather than being formatted as a JSON string with quotes
I have a JSON file that looks like this:
{
"key1": "value1",
"key2": "value2",
...
"keyn": "valuen"
}
No arrays or nested objects, just a simple key->value map. I want to use jq to convert this file to a plain text configuration file of a following format:
KEY1=value1
KEY2=value2
...
KEYn=valuen
(the keys should be uppercased in the result file). I have searched various jq tutorials available on the Net expecting that such an obvious (in my opinion) example would be covered there (at least without uppercasing), but it is not. All tutorials I have found use more complicated examples, like extracting specific values from JSON file, handling arrays or nested structures etc. However, no simple JSON-to-text conversion. man jq doesn't help either, it's not too clear. Could you help me in obtaining the result I want?
Note: it must be done with jq, not any other tool, because it will be used in a script that has access to jq, but no other tool for handling JSON is guaranteed to be present on the system.
You can iterate over the entries in the object with to_entries, then format each to a string and use the -r (--raw-output) flag, like so:
$ cat example.json
{
"key1": "value1",
"key2": "value2",
"keyn": "valuen"
}
$ jq -r 'to_entries[] | (.key | ascii_upcase) + "=" + .value' < example.json
KEY1=value1
KEY2=value2
KEYN=valuen
I have a problem in working JSON file. I launch curl in AutoIt sciript to download a json file from web and then convert it to csv format by jq-win
jq-win32 -r ".[]" -c class.json>class.txt
and the json is in the following format:
[
{
"id":"1083",
"name":"AAAAA",
"channelNumber":8,
"channelImage":""},
{
"id":"1084",
"name":"bbbbb",
"channelNumber":7,
"channelImage":""},
{
"id":"1088",
"name":"CCCCCC",
"channelNumber":131,
"channelImage":""},
{
"id":"1089",
"name":"DDD,DDD",
"channelNumber":132,
"channelImage":""},
]
after jq-win, the file should become:
{"id":"1083","name":"AAAAA","channelNumber":8,"channelImage":""}
{"id":"1084","name":"bbbbb","channelNumber":7,"channelImage":""}
{"id":"1088","name":"CCCCCC","channelNumber":131,"channelImage":""}
{"id":"1089","name":"DDD,DDD","channelNumber":132,"channelImage":""}
and then the csv file will be further process by the AutoIt script and become:
AAAAA,1083
bbbbb,1084
CCCCCC,1088
DDD,DDD,1089
The json has around 300 records and among them, 5~6 record has comma in it eg DDD,DDD
so when I tried read in the csv file by _FileReadToArray, the comma in DDD,DDD cause trouble.
My question is: can I replace comma in the field using jq-win ?
(I tried use fart.exe but it will replace all comma in json file which is not suitable for me.)
Thanks a lot.
Regds
LAM Chi-fung
can I replace comma in the field using jq-win ?
Yes. For example, use gsub, pretty much as you’d use awk’s gsub, e.g.
gsub(","; "|")
If you want more details, please provide more details as per [mcve].
Example
With the given JSON input, the jq program:
.[]
| .name |= gsub(",";";")
| [.[]]
| map(tostring)
| join(",")
yields:
1083,AAAAA,8,
1084,bbbbb,7,
1088,CCCCCC,131,
1089,DDD;DDD,132,
i am wring the shell script as i am new. my query is
i have json object like
{
"logo": {"name":"logo.png","type":"image\/jpeg","tmp_name":"C:\\xampp\\tmp\\php8B97.tmp","error":0,"size":110290},
"template":"template1",
"firstname":"a",
"lastname":"a",
"username":"a",
"password":"aa",
"email":"a",
"categoriesListArr":"{\"Women\":[\"All footwear\",\"All footwear\",\"All Clothing\",\"All Clothing\",\"All Watches\",\"All Watches\",\"All Sunglasses\",\"All Sunglasses\"],\"Men\":[\"All Mens Accessories\",\"All Bags,Belts And wallets\",\"All Fragrances\",\"All Grooming and wellness\"]}",
"aboutUs":"aa",
"contactUs":"78787878878787",
"deliveryInfo":"aa",
"privacyPolicy":"aa",
"t&d":"aa"
}
i extracted the categoriesListArr using jq filter as follows:
categories=`cat detail.json| jq '.categoriesListArr'`
detail.json is the name of the file.
now categories is stringify object...i need to parse it and convert it into json object.
"{\"Women\":[\"All footwear\",\"All footwear\",\"All Clothing\",\"All Clothing\",\"All Watches\",\"All Watches\",\"All Sunglasses\",\"All Sunglasses\"],\"Men\":[\"All Mens Accessories\",\"All Bags,Belts And wallets\",\"All Fragrances\",\"All Grooming and wellness\"]}"
how i can convert it using jq filter
fromjson is your friend:
$ jq '.categoriesListArr | fromjson' detail.json
Or, if you want to retain the original structure:
$ jq '.categoriesListArr |= fromjson' detail.json