I found that using jq I can compose a dynamic JSON file with arguments.
For example profile.jq:
{
"name": $name,
"age": $age,
"secretIdentity": $id,
"powers":{
"sid": $something
}
}
jq -n --arg name "FL" \
--arg age 60 \
--arg id 1234 \
--something "ss" \
-f profile.jq > out.json
so I can get a json file out.json
{
"name": "FL",
"age": 60,
"secretIdentity": 1234,
"powers":{
"sid": "ss"
}
}
However, there are many args, is there any easier way than passing each arg using --arg? Like, is that possible to give all args in a JSON file, something like:
somecommand --argfile arg-file.json --inputfile template-json.json
PS: I noticed that jq has an option --args, but I couldn't find any example of how to use it.
Related
Given the following JSON file (sample.json)
{
"api": "3.0.0",
"data": {
"description": "something",
"title": "hello",
"version": "1.0",
"app": {
"name": "abc",
"id": "xyz"
}
}
}
I wish to add the following JSON object at root level to the file above:
{
"heading": {
"user": ["$username"]
}
}
Where $username is a Bash variable.
Is there a better way to achieve this than the following?
blob=$(jq -n --arg foo API_NAME '{"heading": {"user": [env.username]}}')
jq --argjson obj "$(echo $blob)" '. + $obj' < sample.json
Just move what you create as blob directly into the other filter, ending up with just one jq call:
jq --arg username "$username" '. + {heading: {user: [$username]}}' sample.json
I want to create JSON output with jq that looks like this:
{
"records": [
{
"id": "1234",
"song": "Yesterday",
"artist": "The Beatles"
}
]
}
I assumed I have to twiddle with the "filter" of jq whose concept I don't fully get after reading the doc.
This is what I got so far:
$ jq --arg id 1234 \
--arg song Yesterday \
--arg artist "The Beatles" \
'.' \
<<<'{ "records" : [{ "id":"$id", "song":"$song", "artist":"$artist" }] }'
which prints
{
"records": [
{
"id" : "$id",
"song" : "$song",
"artist" : "$artist"
}
]
}
Do I modify the filter? Do I change the input?
An alternate way to your original attempt, on jq-1.6 you can use the $ARGS.positional attribute to construct your JSON from scratch
jq -n '
$ARGS.positional | {
records: [
{
id: .[0],
song: .[1],
artist: .[2]
}
]
}' --args 1234 Yesterday "The Beatles"
As for why your original attempt didn't work, looks you are not modifying your json at all, with your filter '.' you are basically just reading in and printing out "untouched". The arguments set using --arg need to be set to the object inside the filter.
You are looking for something like this:
jq --null-input \
--arg id 1234 \
--arg song Yesterday \
--arg artist "The Beatles" \
'.records[0] = {$id, $song, $artist}'
Each variable reference between curly brackets is converted to a key-value pair where its name is the key, and its value is the value. And assigning the resulting object to .records[0] forces the creation of the surrounding structure.
jq --null-input\
--argjson id 1234\
--arg song Yesterday\
--arg artist "The Beatles"\
'{ "records" : [{ $id, $song, $artist }] }'
gives
{
"records": [
{
"id": 1234,
"song": "Yesterday",
"artist": "The Beatles"
}
]
}
I think you got the JSON/JQ the wrong way round:
This should be your JQ script:
rec.jq
{
records: [
{
id: $id,
song: $song,
artist: $artist
}
]
}
And this should be your JSON (empty):
rec.json
{}
Then:
jq --arg id 123 --arg song "Yesterday" --arg artist "The Beatles" -f rec.jq rec.json
Which produces:
{
"records": [
{
"id": "123",
"song": "Yesterday",
"artist": "The Beatles"
}
]
}
Start with an empty JSON and add the missing bits:
$ jq --arg id 1234 \
--arg song Yesterday \
--arg artist "The Beatles" \
'. | .records[0].id=$id | .records[0].song=$song | .records[0].artist=$artist' \
<<<'{}'
Outputs
{
"records": [
{
"id": "1234",
"song": "Yesterday",
"artist": "The Beatles"
}
]
}
Another, cleaner, approach based on the answer of #Inian could be
jq -n \
--arg id 1234
--arg song Yesterday
--arg artist "The Beatles"
'{records: [{id:$id, song:$song, artist:$artist}]}'
Using jq how can I convert an array into object indexed by filename, or read multiple files into one object indexed by their filename?
e.g.
jq -s 'map(select(.roles[]? | contains ("mysql")))' -C dir/file1.json dir/file2.json
This gives me the data I want, but I need to know which file they came from.
So instead of
[
{ "roles": ["mysql"] },
{ "roles": ["mysql", "php"] }
]
for output, I want:
{
"file1": { "roles": ["mysql"] },
"file2": { "roles": ["mysql", "php"] }
}
I do want the ".json" file extension stripped too if possible, and just the basename (dir excluded).
Example
file1.json
{ "roles": ["mysql"] }
file2.json
{ "roles": ["mysql", "php"] }
file3.json
{ }
My real files obviously have other stuff in them too, but that should be enough for this example. file3 is simply to demonstrate "roles" is sometimes missing.
In other words: I'm trying to find files that contain "mysql" in their list of "roles". I need the filename and contents combined into one JSON object.
To simplify the problem further:
jq 'input_filename' f1 f2
Gives me all the filenames like I want, but I don't know how to combine them into one object or array.
Whereas,
jq -s 'map(input_filename)' f1 f2
Gives me the same filename repeated once for each file. e.g. [ "f1", "f1" ] instead of [ "f1", "f2" ]
If your jq has inputs (as does jq 1.5) then the task can be accomplished with just one invocation of jq.
Also, it might be more efficient to use any than iterating over all the elements of .roles.
The trick is to invoke jq with the -n option, e.g.
jq -n '
[inputs
| select(.roles and any(.roles[]; contains("mysql")))
| {(input_filename | gsub(".*/|\\.json$";"")): .}]
| add' file*.json
jq approach:
jq 'if (.roles[] | contains("mysql")) then {(input_filename | gsub(".*/|\\.json$";"")): .}
else empty end' ./file1.json ./file2.json | jq -s 'add'
The expected output:
{
"file1": {
"roles": [
"mysql"
]
},
"file2": {
"roles": [
"mysql",
"php"
]
}
}
i create a json object by a script with jq.
jq -n --arg date "$DATE" --arg script "$SCRIPT" --arg log_level "$LOG_LEVEL" --arg ppid "$PPID" --arg message "$MESSAGE" '{"t":$date, "service":$script, "level":$log_level, "pid":$ppid, "message":$message}'
The Variable log_level and message are provided by another script and in message there can be stored a plain string or a json string.
Example Json String:
{"text":"value", "text2":"value2"}
Output if a a plain string is provided:
{
"t": "2017-10-12 16:52:26",
"service": "import.sh",
"level": "INFO",
"pid": "23425",
"message": "START"
}
Output if a json object is provided:
{
"t": "2017-10-12 17:01:16",
"service": "import.sh",
"level": "INFO",
"pid": "13069",
"message": "{\"text\":\"value\", \"text2\":\"value2\"}"
}
What i expected if i provide the json object:
{
"t": "2017-10-12 17:01:16",
"service": "cis_import.sh",
"level": "INFO",
"pid": "13069",
"message": {
"text": "value",
"text2": "value2"
}
}
Am i right that jq add the \ for each " because it gets added to the json object like it would be a simple string?
How can i get my expected json?
Got it finally. Thanks for the hints chepner
and blusky
To validate if $MESSAGEis a json i use jq -n $MESSAGE see jq manual instead of jq . /path/to/file.json
To use the mentioned --argjson i will add " around $MESSAGEif the $MESSAGE is a plain string MESSAGE="\"$MESSAGE\"". I do this with DATE, SCRIPT, LOG_LEVEL, PPID too.
Command to create the json object looks like this:
jq -cn --argjson date "$DATE" --argjson script "$SCRIPT" --argjson log_level "$LOG_LEVEL" --argjson ppid "$PPID" --argjson message "$MESSAGE" '{"t":$date, "service":$script, "level":$log_level, "pid":$ppid, "message":$message}'
I'm trying to create a JSON file by executing the following command:
jq --arg greeting world '{"hello":"$greeting"}' > file.json
This command stuck without any input. While
jq -n --arg greeting world '{"hello":"$greeting"}' > file.json
doesn't parse correctly. I'm just wondering is really possible to create a JSON file.
So your code doesn't work because included the variable inside double quotes which gets treated as string. That is why it is not working
As #Jeff Mercado, pointed out the solution is
jq -n --arg greeting world '{"hello":$greeting}' > file.json
About the - in a name. This is actually possible. But as of now this is not available in released version of jq. If you compile the master branch of jq on your system. There is a new variable called $ARGS.named which can be used to access the information.
I just compiled and check the below command and it works like a charm
./jq -n --arg name-is tarun '{"name": $ARGS.named["name-is"]}'
{
"name": "tarun"
}
$ARGS provides access to named (--arg name value) and positional (--args one two three) arguments from the jq command line, and allows you to build up objects easily & safely.
Named arguments:
$ jq -n '{christmas: $ARGS.named}' \
--arg one 'partridge in a "pear" tree' \
--arg two 'turtle doves'
{
"christmas": {
"one": "partridge in a \"pear\" tree",
"two": "turtle doves"
}
}
Positional arguments:
$ jq -n '{numbers: $ARGS.positional}' --args 1 2 3
{
"numbers": [
"1",
"2",
"3"
]
}
Note you can access individual items of the positional array, and that the named arguments are directly available as variables:
jq -n '{first: {name: $one, count: $ARGS.positional[0]}, all: $ARGS}' \
--arg one 'partridge in a "pear" tree' \
--arg two 'turtle doves' \
--args 1 2 3
{
"first": {
"name": "partridge in a \"pear\" tree",
"count": "1"
},
"all": {
"positional": [
"1",
"2",
"3"
],
"named": {
"one": "partridge in a \"pear\" tree",
"two": "turtle doves"
}
}
}
To add to what Jeff and Tarun have already said, you might want to use the \() string interpolation syntax in your command. eg.
jq -n --arg greeting world '{"hello":"\($greeting)"}'
for me this produces
{
"hello": "world"
}
Regarding your reply to Jeff's comment, the argument name you choose has to be a valid jq variable name so an arg like greeting-for-you won't work but you could use underscores so greeting_for_you would be ok. Or you could use the version Tarun described.