Changing the key of JSON file whilst retaining data within that bracket section - json

My first thought was a simple "search and replace of the underlying JSON code, inspecting it line by line. That will not work as the same "keyword" could be used in various places, but only one specific line needs to be changed.
Then I thought, read the existing data in, create a new entry, push the data into that block, and then delete the original key. The former works ... but leaves the original and all its data intact.
Here is a simplified version. (DOn't think 500+ lines would be appreciated)
{
project 1:{
ad_campaign:{
ad_1{
data1,
data2
},
ad_2{
data1,
data2
}
}
}
project 2:{
ad_campaign:{
ad_1{
data1,
data2
}
ad_201{
data1,
data2
}
}
}
}
The "data" entries are 6 lines per ad, and there could be dozens of ads per project, and dozens of projects !!
They want to change Projec1 "ad_1" to "ad_101"
As you can see, both projects have ads named "ad_1", so a simple search and replace of the underlying text code file is a no-go.
If I use $project1->{"ad_campaign"}, I can get all the data. If I then use: $project1->{"ad_campaign"}=$new_ad_id ... it creates a new section (which I =could= then copy the data line - by - line) ... but I still cannot get rid of the original "ad_1" entry! ($project1->{"ad_campaign"}->{"ad_1"} =null (no bareword allowed) / "null" / "" have no effect - even when trying to remove the internal data first. That could result in data1:'',data2:'' etc)
What I really need is a way to do $customer1->{"ad_campaign"}->{"new_ad_id"} to simply change that third key. (I've Googled "change" / "replace" JSON key, but no results ... especially as I am using PERL)
One more "spanner in the works"; the file gets saved as a single line, (Using JSON::PP The "pretty" seems to add too many spaces - tab indents would seem better) so would be difficult to break apart, and do a line by line scan anyway. (Another idea was to reiterate through file, set a flag once hit "customer1", another loop, and another flag for "ad_campaign", and then final loop seeking "ad_1". Seems a bit inefficient ... plus all the decoded data is in a hash anyway!)
Please bear in mind, I have simplified this code. Between "Project1" and "ad_campaign" could be another 50 key:value pairs PER PROJECT
There is possibly some simple solution finding / changing the value via a hash "arguement" ... but I've yet to find it!
Somewhere else I read: delete $JSON->{"project1"}->{"ad_campaign"}->{"ad_id"} but that didn't delete the original either!
Just had a closing thought: Maybe I could do something like "indexOf" to locate project1/ad_campaign etc, and then do split/splice, pushing one half of data, then split again etc until I find the word to replace. But again, that does seem overkill for what could be a pretty basic problem

Try code below.
use strict ;
use warnings;
use feature 'say';
use JSON;
my $data = <<EOM;
{
"project 1": {
"ad_campaign": {
"ad_1": [
"p1: ad:1 data1",
"p1: ad:1 data2"
],
"ad_2": [
"p1: ad:2 data1",
"p2: ad:2 data2"
]
}
},
"project 2": {
"ad_campaign": {
"ad_1": [
"p2: ad:1 data1",
"p2: ad:1 data2"
],
"ad_201": [
"p2: ad:201 data1",
"p2: ad:201 data2"
]
}
}
}
EOM
my $in = from_json($data);
# create "ad_101" to reference to the existing "ad_1" data tree
$in->{"project 1"}{"ad_campaign"}{"ad_101"} = $in->{"project 1"}{"ad_campaign"}{"ad_1"};
# now delete the reference to "ad_1"
delete $in->{"project 1"}{"ad_campaign"}{"ad_1"};
# using pretty & canonical here to make the results easier to read
# don't use pretty if you want all JSON output as a single line
say to_json($in, {pretty => 1, canonical => 1});
output is
{
"project 1" : {
"ad_campaign" : {
"ad_101" : [
"p1: ad:1 data1",
"p1: ad:1 data2"
],
"ad_2" : [
"p1: ad:2 data1",
"p2: ad:2 data2"
]
}
},
"project 2" : {
"ad_campaign" : {
"ad_1" : [
"p2: ad:1 data1",
"p2: ad:1 data2"
],
"ad_201" : [
"p2: ad:201 data1",
"p2: ad:201 data2"
]
}
}
}
Key point to note is that nested hashes in perl use references, which are akin to pointers in C/C++ land. Means that after running this line
$in->{"project 1"}{"ad_campaign"}{"ad_101"} = $in->{"project 1"}{"ad_campaign"}{"ad_1"};
both the ad_1 and ad_101 keys reference the same data, namely
[
"p1: ad:1 data1",
"p1: ad:1 data2"
],
[EDIT - answering some of the comments]
I changed the data values to highlight that the reference in ad_1 and ad_101 are pointing to unique data in the JSON document. You didn't supply any data, so I made some up.
Use of from_json and to_json rather than decode_json & encode_json is purely to allow the pretty parameter to be controlled. Use the variant that suits your use-case. Check out the docs here

Another option is to use jq, rather than Perl
Assuming data.json contains this
{
"project 1": {
"ad_campaign": {
"ad_1": [
"p1: ad:1 data1",
"p1: ad:1 data2"
],
"ad_2": [
"p1: ad:2 data1",
"p2: ad:2 data2"
]
}
},
"project 2": {
"ad_campaign": {
"ad_1": [
"p2: ad:1 data1",
"p2: ad:1 data2"
],
"ad_201": [
"p2: ad:201 data1",
"p2: ad:201 data2"
]
}
}
}
this one-liner to do the rename
$ jq '."project 1".ad_campaign |= ( .ad_101 = .ad_1 | del(.ad_1) ) ' data.json
output is
{
"project 1": {
"ad_campaign": {
"ad_2": [
"p1: ad:2 data1",
"p2: ad:2 data2"
],
"ad_101": [
"p1: ad:1 data1",
"p1: ad:1 data2"
]
}
},
"project 2": {
"ad_campaign": {
"ad_1": [
"p2: ad:1 data1",
"p2: ad:1 data2"
],
"ad_201": [
"p2: ad:201 data1",
"p2: ad:201 data2"
]
}
}
}
to get jq to output as a single line, add the -c option to the commandline
$ jq -c '."project 1".ad_campaign |= ( .ad_101 = .ad_1 | del(.ad_1) ) ' data.json
{"project 1":{"ad_campaign":{"ad_2":["p1: ad:2 data1","p2: ad:2 data2"],"ad_101":["p1: ad:1 data1","p1: ad:1 data2"]}},"project 2":{"ad_campaign":{"ad_1":["p2: ad:1 data1","p2: ad:1 data2"],"ad_201":["p2: ad:201 data1","p2: ad:201 data2"]}}}

The others have answered this question adequately, and I'd also choose jq (see Using jq how can I replace the name of a key with something else and the jq FAQ).
But, I want to emphasis what #pmqs is doing but not yelling from the rooftops.
It's just Perl
If you want to do something in Perl, you basically have scalars, arrays, and hashes. If you are already in Perl land, that's what you are manipulating.
You don't change a JSON key in Perl; you change a hash key. It doesn't matter that you got that hash from JSON or will output it as JSON. That's how you get to #pmqs's delete:
$hash->{$new_key} = delete $hash->{$old_key};
Everything beyond that is just input and output.
The right tool for the right job
You have a few other questions about the output, and those questions are unrelated to changing a hash key. When I deal with this, I simply ignore it. If it outputs it all on one line, I don't really care. I can get pretty output with jq:
% jq .
If I want to minify it, I can use testing:
% jq -r tostring data.json
You mentioned that you didn't like the indention. That's easy to change. jq uses 2 space indents, but I can make that smaller (or larger, or tabs) (see also How to restrict indentation while using jq filters on json file):
% jq --indent 1 . data.json
You say that you can't use jq because you are using Perl, but remember, Perl is the glue language of the internet. You can easily call other programs to do your work and you are expected to do so when something else can do the job better. Why wrestle with Perl and its modules when an existing tool already does a better job? If you aren't able to install it for whatever reason, that's fine. But saying that you can only choose one tool unnecessarily limits you.
Setting aside that, JSON::PP has settings for space_before, space_after, indent, and indent_length. You can adjust those how you like. If you have problems with those, you can ask a separate question.

Related

in place edit, search for nested value and then replace another value

I have an input JSON document with roughly the following form (actual data has additional keys, which should be passed through unmodified; the whitespace is adjusted for human readability, and there's no expectation that it be maintained):
{
"Rules": [
{"Filter": { "Prefix": "to_me/" }, "Status": "Enabled" },
{"Filter": { "Prefix": "from_me/" }, "Status": "Enabled" },
{"Filter": { "Prefix": "__bg/" }, "Status": "Enabled" }
]
}
I need to match .Rules[].Filter.Prefix=="to_me/" and then change the associated "Status": "Enabled" to "Disabled". Since only the first rule above has a prefix of to_me/, status of that rule would be changed to Disabled, making correct output look like the following:
{
"Rules": [
{"Filter": { "Prefix": "to_me/" }, "Status": "Disabled" },
{"Filter": { "Prefix": "from_me/" }, "Status": "Enabled" },
{"Filter": { "Prefix": "__bg/" }, "Status": "Enabled" }
]
}
I've tried several different combinations but can't seem to get it right.
Anyone have ideas?
I prefer the idiom ARRAY |= map(...) over ARRAY[] |= ..., mainly because the former can be used reliably whether or not any of the substitutions evaluate to empty:
jq '.Rules |= map(if .Filter.Prefix == "to_me/"
then .Status="Disabled" else . end)'
To overwrite the input file, you might like to consider sponge from moremutils.
Doing in-place updates can be done with |=, and deciding whether to modify content in-place can be done with if/then/else. Thus:
jq '.Rules[] |= (if .Filter.Prefix == "to_me/" then .Status="Disabled" else . end)'

Regex words list to array for a JSON in Atom

I want to achieve this using the Atom search/replace tool (with Regex on), otherwise it's not funny ;)
INPUT:
{
"data" : [
{
"name": "Random data",
"words": "super, long, list, of, elements, who, never, ever, end, I, mean, at, some, point, yes, but, not, now"
},
{
"name": "Random data",
"words": "another, super, long, list, of, elements, who, never, ever, end, I, mean, at, some, point, yes, but, not, now"
},
]
}
OUTPUT expected :
{
"data" : [
{
"name": "Random data",
"words": ["super", "long", "list", "of", "elements", "who", "never", "ever", "end", "I", "mean", "at", "some", "point", "yes", "but", "not", "now"]
},
...
]
}
Right now I have this SEARCH regex (but I don't know how to include the repeat in ATOM):
"words": "(.*),.*"
And the REPLACE:
"words": ["$1"]
Search for ", " and replace with "\", \"". Would not require regex I recon or would be fine with that too.

What is the most efficient way to patch a JSON patch?

Say, I have this JSON patch, which adds something into the groups array
[
{"op":"add","path":"/food/groups/-",
"value":{
"select" : 55,
"pool" : [
[ 0.111, "biscuit" ],
[ 0.111, "cookie" ],
[ 0.111, "pretzel" ]
]
}
}
]
I need to make a patch to this patch, that removes [ 0.111, "pretzel" ]. Normally I can just utilize the operation remove.
Problem is, I cannot be so sure which number that biscuit/cookie/pretzel patch will come up. I don't know if it'll be the 2nd in the array, or the 5th, because users might apply another patch that adds something into the array.
What would be the most efficient way to solve this problem, without making an exact copy of the patch but without the pretzel?

Shell script to get key value pair from JSON object

I have a JSON object like
{
"Men": [
"All Clothing",
"All Clothing",
"All footwear",
"All footwear",
"All Watches",
"All Watches",
"All Sunglasses",
"All Sunglasses"
],
"Electronics": [
"Mobiles",
"Tablets",
"Wearable Smart Devices",
"Mobile Accessories",
"Headphones and headsets",
"Tablet Accessories",
"Computer Accessories",
"Televisions",
"Large Appliances",
"Small Appliances",
"Kitchen Appliances",
"Personal Care",
"Audio and video",
"Laptop"
],
"Women": [
"Ethnic wear",
"Western wear",
"Lingerie & Sleep Wear",
"All Bags, Belts & Wallets",
"All jewellery",
"All Perfumes",
"Spectacle Frames",
"Beauty & Personal Care",
"The International Beauty Shop"
]
}
I want to get key value pair from this object.m using jq filter but it doesnot work.
keys=`jq 'keys' $categories`
$categories is the name of variable of json object. suggestions are welcome.
It's not really clear what you are asking. If $categories contains your JSON data then you need to pipe it to jq somehow. With Bash, you could use a here string:
jq keys <<<"$categories"
or more traditionally (and portably), a pipe:
printf '%s\n' "$categories" | jq keys
To capture the value of the keys into a variable, use a command substitution:
keys=$(jq 'keys' <<<"$categories")
(or `backticks` like in your attempt; but the modern notation is much preferable);
or better yet, obtain this value in the same way you assigned categories in the first place.

JSON format with gzip compression

My current project sends a lot of data to the browser in JSON via ajax requests.
I've been trying to decide which format I should use. The two I have in mind are
[
"colname1" : "content",
"colname2" : "content",
],
[
"colname1" : "content",
"colname2" : "content",
],
...
and
{
"columns": [
"column name 1",
"column name 2",
],
"rows": [
[
"content",
"content"
],
[
"content",
"content"
]
...
]
}
The first method is better because it is easier to work with. I just have to convert to an object once received. The second will need some post processing to convert it into a format more like the first so it is easier to work with in JavaScript.
The second is better because it is less verbose and therefore takes up less bandwidth and downloads more quickly. Before compression it is usually between 0.75% and 0.85% of the size of the first format.
GZip compression complicates things further. Making the difference in file size nearer 0.85% to 0.95%
Which format should I go with and why?
I'd suggest using RJSON:
RJSON (Recursive JSON) converts any JSON data collection into more compact recursive form. Compressed data is still JSON and can be parsed with JSON.parse. RJSON can compress not only homogeneous collections, but any data sets with free structure.
Example:
JSON:
{
"id": 7,
"tags": ["programming", "javascript"],
"users": [
{"first": "Homer", "last": "Simpson"},
{"first": "Hank", "last": "Hill"},
{"first": "Peter", "last": "Griffin"}
],
"books": [
{"title": "JavaScript", "author": "Flanagan", "year": 2006},
{"title": "Cascading Style Sheets", "author": "Meyer", "year": 2004}
]
}
RJSON:
{
"id": 7,
"tags": ["programming", "javascript"],
"users": [
{"first": "Homer", "last": "Simpson"},
[2, "Hank", "Hill", "Peter", "Griffin"]
],
"books": [
{"title": "JavaScript", "author": "Flanagan", "year": 2006},
[3, "Cascading Style Sheets", "Meyer", 2004]
]
}
Shouldn't the second bit of example 1 be "rowname1"..etc.? I don't really get example 2 so I guess I would aim you towards 1. There is much to be said for having data immediately workable without pre-processing it first. Justification: I once spend too long optimizing array system that turned out to work perfectly but its hell to update it now.