I have a json file temp.json like this -
{
"data": {
"stuff": [
.....
]
},
"time": {
"metrics": 83
}
}
I want to remove this particular block of code from the above json file -
,
"time": {
"metrics": 83
}
After removal I want to rewrite new json in the same file so that new content in the same file will be -
{
"data": {
"stuff": [
.....
]
}
}
Is this possible to do by any chance?
Note: number 83 can be any number in general.
Here's an excellent tutorial: Baeldung: Guide to Linux jq Command for JSON Processing.
Maybe you can try something like this: jq 'del(.time)' temp.json > temp2.json.
Note that jq works at the semantic level; it's not just "text substitution". So things like the "comma" separators between objects will be deleted from the JSON text when you use jq to delete the object.
Experiment, and see what works best for your particular scenario.
Related
I already tried a lot, without success.
I'm facing a problem with tranforming the following JSON structure:
JSON:
{
"?xml":{
"#version":"1.0",
"#encoding":"UTF-8"
},
"methodResponse":{
"params":{
"param":{
"value":{
"struct":{
"member":[
{
"name":"severity",
"value":{
"string":"Important"
}
},
{
"name":"product",
"value":{
"string":"this is a product name"
}
},
{
"name":"notes",
"value":{
"string":"Here are some notes"
}
},
{
"name":"references",
"value":{
"string":"This is a reference"
}
},
{
"name":"release",
"value":{
"i4":"1"
}
},
{
"name":"last_modified_date",
"value":{
"string":"2020-03-10 15:21:50.147896"
}
}
]
}
}
}
}
}
}
I'd like to filter it by key names like in this transform statement (which already works with one key name):
Transform statement:
map(.methodResponse.params.param.value.struct.member[] | select(.name == "product" ).value.string as $product| {$product})
But I'm not sure how the syntax must look like when I want to returning several key names in one statement.
I would expect a result like this one:
[
{
"product": "This is a product name",
"release": "1",
"last_modified_date": "2020-03-09 16:39:39.0"
}
]
I am not quite as familiar with this topic and trying it alone takes forever...
Can anyone help please? Any help is much appreciated!
Thanks!
First, if you want to use jq, you need well formated JSON data. The human JSON tool hjson might help on your input data.
hjson -j file
Extracting the wanted data could be done in the following way:
hjson -j file | \
jq '.methodResponse.params.param.value.struct.member |
[map({(.name):(.value.string//.value.i4)})|add]'
It simply creates ne object base on the name and value parameters.
Note that if .value.string doesn't exist, it uses .value.i4 with the alternate operator //.
This is a question about the command line json processor jq. It is not about javascript or jQuery or anything else with js and qs in its name :-)
I have input data like:
{ "id": "person1", "name": "wilma", "age": "quite old"}
{ "id": "person2", "name": "fred"}
{ "id": "person1", "name": "betty", "x": "extra"}
I want output like this:
{
"person1": { "name": "betty", "age": "quite old", "x": "extra" },
"person2": { "name": "fred" }
}
I have tried various things!
e.g.
jq -s '.[] | { (.id) : . }' <data
gives
{ "person1": { "id": "person1", "name": "wilma", "age": "quite old" }}
{ "person2": { "id": "person2", "name": "fred" }}
{ "person1": { "id": "person1", "name": "betty", "x": "extra" }}
Which is sort of there, except it's outputting a stream of objects instead of just one. I need to merge all those objects together.
jqplay.org example
I've also tried using group_by(.id)[]|add which merges each item but still results in a stream. https://jqplay.org/s/lh6QUQ0DO4
You could tweak your attempt as follows:
jq -s 'map({ (.id) : . }) | add' <data
However, it would be more efficient to use inputs and reduce with the -n command-line option instead of -s.
Of course, using this approach runs the risk of collisions.
You might also want to add del(.id)
Ah! I've got it! Or I've got one solution - please post if there's a better way.
jq -s '[group_by(.id)[]| add | { (.id) : . } ]|add' <data
https://jqplay.org/s/BfAdRBZUMW
group_by groups the inputs by their .id value and produces an array of arrays - the inner arrays are the values that match on id.
for each group the inner arrays are passed to add which, because the things in the inner arrays are objects, merges them.
That leaves a 2 item array. We feed that to an object constructor which plucks the id as the key and the whole item as the value. This still leaves an array of items.
the outer [] (starts at start of pattern) says take all those and feed it to add (again), which merges the final objects created in (3).
It works, but there may be a cleaner way.
EDIT
This is uglier but produces the same result and is ~24% faster on a 9MB dataset.
jq -s 'reduce [.[]|{ (.id) : . }][] as $item ({}; . * $item )' <data
This uses reduce <list> as <$var> (<initiation>; <iteration>) starting with an empty object {} and using the merge operator * starting from the incoming item . to create the output. I'm surprised it's faster, but I understand that group_by does a sort, so I guess that's an additional time cost.
Using jq, how can I take a json object from a file (input_02.json), and append it to output.json, while retaining everything already in output.json (e.g. an object originating from file input_01.json).
The object to be appended in both cases is literally the entire contents of the file, with the file's "id" field as the object's key.
I'm taking a large list of input files (all with the same syntax) and essentially combining them like that.
The command i'm using to create the object to be appended is as follows:
jq '{(.id):(.)} ' input_01.json
which gives me:
{
"input1_id": {
}
}
input_1.json:
{
"id": "input1_id",
"val": "testVal1"
}
input2.json:
{
"id": "input2_id",
"val": "testVal2"
}
desired output:
{
"input1_id": {
"id": "input1_id",
"val": "testVal1"
},
"input2_id": {
"id": "input2_id",
"val": "testVal2"
}
}
You’re on the right track with {(.id):(.)}. The following should handle the case you mentioned, and might give you some ideas about similar cases:
program.jq: map({(.id):(.)}) | add
Invocation:
jq -s -f program.jq input_01.json input_02.json
You could use "jf" for this from https://pypi.python.org/pypi/jf
$ pip install jf
$ jf 'chain(), {y["id"]: y for y in x}' input1.json input2.json
{
"input2_id": {
"id": "input2_id",
"val": "testVal2"
},
"input1_id": {
"id": "input1_id",
"val": "testVal1"
}
}
I would like to convert 500+ json files to csv format using the solution provided in How to convert arbirtrary simple JSON to CSV using jq? but my json files are not in the same format as per the proposed solution.
Following represents a sample json file:
[
{
"jou_entry": {
"id": 655002886,
"units": 2
}
},
{
"jou_entry": {
"id": 655002823,
"units": 4
}
},
{
"jou_entry": {
"id": 657553949,
"units": 6
}
}
]
Where as the proposed solution requires the json in the following format:
[
{
"id": 655002886,
"units": 2
},
{
"id": 655002823,
"units": 4
},
{
"id": 657553949,
"units": 6
}
]
I am able to convert the json from source format to required format using the following jq filter
jq -r '[.[] | ."jou_entry"]'
But I don't like the hard-coding of key "jou_entry" in the filter. As this will require individual key definition for so many files. I would like to have the conversion without the hard-coded value.
How can I do this? Please help
This gets the desired output
jq '[.[] | .[]]'
Explanation from the manual. When .[] is used on an array, it returns all of the elements of an array. When it is used on an object it will return all the values of the object.
Using jq I tried to update this json document:
{
"git_defaults": {
"branch": "master",
"email": "jenkins#host",
"user": "Jenkins"
},
"git_namespaces": [
{
"name": "NamespaceX",
"modules": [
"moduleA",
"moduleB",
"moduleC",
"moduleD"
]
},
{
"name": "NamespaceY",
"modules": [
"moduleE"
]
}
]
}
with adding moduleF to NamespaceY. I need to write the file back again to the original source file.
I came close (but no cigar) with:
jq '. | .git_namespaces[] | select(.name=="namespaceY").modules |= (.+ ["moduleF"])' config.json
and
jq '. | select(.git_namespaces[].name=="namespaceY").modules |= (.+ ["moduleF"])' config.json
The following filter should perform the update you want:
(.git_namespaces[] | select(.name=="NamespaceY").modules) += ["moduleF"]
Note that the initial '.|' in your attempt is not needed; that "NamespaceY" is capitalized in config.json; that the parens as shown are the keys to success; and that += can be used here.
One way to write back to the original file would perhaps be to use 'sponge'; other possibilities are discussed on the jq FAQ https://github.com/stedolan/jq/wiki/FAQ