How to add an element to json file in perl [closed] - json

Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 1 year ago.
Improve this question
To change the md5 checksum of a json file, I am using the following filter in my httpd.conf:
ExtFilterDefine jsonfilter mode=output intype=application/json cmd="/usr/bin/perl -pe 'END { unless (-f q{/tmp/md5_filter.tmp}) { print qq(\\n\,\"STRING\"\: \") . time() . qq(\x0D\"\\n) }'"
But after the filter run, I receive an error, probally because the new string / timestamp get added after the last bracket } and leads to an unvalid json format:
SyntaxError: JSON.parse: unexpected non-whitespace character after JSON data at line 224 column 2 of the JSON data
Does someone know how to get this run correctly? Just to be sure: I just want to change the output of the file, before it gets response to the client browser. I don't want to change the original file itself. This should be untouched.

probally because the new string / timestamp get added after the last bracket } and leads to an unvalid json format
I don't know any details of the environment that you're working in, but I suspect that your suspicion is correct here.
You could try to fix your string-manipulating solution (perhaps by working out when you're processing the final line of the input and then printing your addition before printing the closing brace. But the best solution is to gather the input into a single string and then process it using a JSON parser.
Something like this (untested):
perl -MJSON -ne '$json .= $_; END { my $data = decode_json($json); $data->{STRING} = time; print encode_json($data);'
Update: To explain the various parts of my code.
-MJSON - loads the JSON module
-ne - n iterates over the input (putting each line, in turn, into $_); e executes code from the command line
$json .= $_ - this is run for every line of the input. It just takes each line and gathers them all together in a variable called $json
END { ... } - this block of code is only run once - after the input has all been read
my $data = decode_json($json) - takes the JSON text string (now stored in $json) and decodes it into a Perl data structure. That data structure is stored in $data
$data->{STRING} = time - adds another key/value pair to the data structure
print encode_json($data) - encodes the altered data structure back into a JSON string and prints that string to STDOUT

Related

What is the correct way of editing a specific value in a json file? [duplicate]

This question already has answers here:
change json file by bash script
(6 answers)
Closed 6 months ago.
I have a file called usr.json, and this file contains a JSON object. The object looks as following:
{
"users": {
"accessRights":"FFFFF",
"name": "admin"
}
}
Now I want to replace the string for accessRights with a new value which is stored in a variable called newUsrCfg.
Since I'm working with a remote device I need to ssh to it, and for that I have a script.
However, I call all this in Python code:
def test_tc():
newUsrCfg = "F7FFF"
cmd = f'sed -ie \'s/\"accessRights\":.*/\"accessRights\": \"{newUsrCfg}\"/g\' "/etc/hgp/runtime/user/user1.json"'
subprocess.call(["bash", pathToSshCommand, cmd])
The commands need to be either called with single quotes or double ones. And when opening the file the output of this is:
{
"users":{
"accessRights": "f7fff"
I tried many other sed commands with different formats, but nothing worked.
If jq is available to you, you can
jq '.users.accessRights = "f7fff"' usr.json
As also commented below, manipulating json with text replacement is likely to break if your input does not exactly meet your expectations (for instance though differences in whitespace). If you have a tool like jq which understands json, that is always the preferable option.
However, in case you still want to do it with sed you could adjust your regex to be less inclusive. Since you want to replace 5 hex values, something along the following lines should work (I hope I escaped everything correctly, can't test since I'm on the way).
cmd = 'sed -ie \'s/\"accessRights\":\s*\"[0-9,a-f,A-F]\{5\}\"/\"accessRights\": \"{newUsrCfg}\"/g\' "/etc/hgp/runtime/user/user1.json"'
See regex101.
A third option would to both load the file via ssh and manipulate the json in Python, or do it directly in Python on the target system, if Python is available there as well.

How to convert a JSON array to a CSV row? [closed]

Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 3 years ago.
Improve this question
I'm trying to convert a large JSON file to a CSV format. (I've just learned how to use jq so I'm still a beginner).
I've successfully managed to convert most of the data, however, I'm stuck at an array. Every JSON object in the file is supposed to be converted to a single CSV row, and I can't get that to work.
I've been trying to help myself with an existing answer:
Convert/Export JSON to CSV
But the problem is that this method writes a row for each item in the array, needlessly repeating information.
I am using the same type of command as the above answer, the only difference being the name of the columns, but the array blocks...
For instance, I could have a JSON file resembling:
{resources:[
{"id":"001","name"="Robert","items":[
{"label":"00A","name":"Pen"},
{"label":"00B","name":"Paper"}],
{"id":"002","name"="Bruce","items":[
{"label":"00A","name":"Pen"},
{"label":"00B","name":"Computer"},
{"label":"00C","name":"Headphones"}]
]
}
That I would like to become:
001,Robert,Pen,Paper,
002,Bruce,Pen,Computer,Headphones
I only need the name columns of the array
For the moment, the result is:
001,Robert,Pen
001,Robert,Paper
002,Bruce,Pen,
002,Bruce,Computer
002,Bruce,Headphones
The problem is the actual array is about 30 items long for each JSON object, making it impossible to use this way.
$ jq -r '.resources[] | [.id,.name,.items[].name] | #csv' < /tmp/b.json
"001","Robert","Pen","Paper"
"002","Bruce","Pen","Computer","Headphones"
You can use awk to fix up your current output file.
awk -F, '{a[$1","$2]=a[$1","$2]","$3}; END{for (v in a) print v a[v]}' in.txt | sort >out.txt
Input:
001,Robert,Pen
001,Robert,Paper
002,Bruce,Pen,
002,Bruce,Computer
002,Bruce,Headphones
Output:
001,Robert,Pen,Paper
002,Bruce,Pen,Computer,Headphones

decoding JSON issue in Perl code: , or ] expected while parsing array, at character offset

I have written Perl code that was working until recently, when I tried to run it again. The problem seems to originate from the JSON::XS "decode_json" method.
Code Snippet:
use warnings;
use strict;
use MooseX::Singleton;
use Array::Utils qw(:all);
use Data::Dumper;
use JSON::XS qw(encode_json decode_json);
use Storable;
use Tie::IxHash;
open (my $observations_fh, '<', 'observations.json') or die "Could not open observations.json\n";
my $observations_json = <$obserations_fh>;
my #decoded_observations = #{decode_json($observations_json)};
Usually, after this code I was able to go through each JSON component in a for loop and take specific information, but now I get the error:
, or ] expected while parsing array, at character offset 5144816
(before "(end of string)")
I saw a similar question here, but it didn't resolve my problem.
I also have similar json decoding going on that doesn't utilize #{decode_json($variable)}, but when I tried that with this observations.json file, the same error was output.
I also tried just using the JSON module, but same error occurred.
Any insight would be greatly appreciated!
-cookersjs
That probably indicates you have incomplete JSON in $observations_json. Your assumption that the entire file consists of just one line is probably incorrect. Use
my $observations;
{
open (my $observations_fh, '<', 'observations.json')
or die("Can't open observations.json: $!\n");
local $/;
my $observations_json = <$obserations_fh>;
$observations = decode_json($observations_json);
}
If that doesn't help, observations.json doesn't contain valid JSON.

Get Data from Perl format and update sql table

I'm relatively new to Perl and trying to self teach. However i have read all of the related threads on this page and others and none of them seem to work for me.
Below is my code - trying to get a lot of data from a webpage in Perl format and export it to update values in an SQL table.
Currently i can't even data dumper the results of the url out.
Any help would be great.
#!/usr/bin/perl
#
use LWP::Simple;
use warnings;
use strict;
use JSON qw( decode_json from_json );
use LWP::Simple;
use Data::Dumper;
use utf8;
my $url = "http://.sensitivedata.txt";
my #json= from_json(get ( $url ));
die "Couldn't get $url" if not defined #json;
##my $decoded_json = decode_json( #json);
print Dumper #json;
exit 0;
This is the error message it is giving me:
defined(#array) is deprecated at alarms.pl line 14.
(Maybe you should just omit the defined()?)
malformed JSON string, neither array, object, number, string or atom, at character offset 0 (before "(end of string)") at /opt/csw/share/perl/csw/JSON.pm line 168
The error message is pretty clear about a) what the problem is and b) how to get rid of it.
defined(#array) is deprecated at alarms.pl line 14. (Maybe you should
just omit the defined()?)
Calling defined() on #json is pointless. You're really just checking to see if there is any data in the array so replace if not defined #json with if not #json.
That will get rid of the error message. But you'll still have a problem as your program will almost certainly now die on the same line with the error message "Couldn't get http://.sensitivedata.txt". And that's probably not an accurate error message.
The problem is that this error can be caused by two problems. Either you can't get the data or you can't parse the data. Your error message only mentions one of these possibilities. Better to split the error checking into two.
# Step 1: Get the data
my $raw_json = get($url);
die "Can't get data from $url" unless $raw_json;
# Step 2: Parse the data
my #json = from_json($raw_json);
if (!#json) {
warn $raw_json;
die "Can't parse data from $url";
}
With code more like this, you'll be able to see what the problem is.
There's another little problem here, so to pre-empt your next question...
from_json always returns a scalar. It will either be a hash reference or an array reference (depending on the JSON you get). Looks like you're expecting an array. You'll need to store the reference in a scalar and dereference it.
my $json_array_ref = from_json($raw_json);
if (!#$json_array_ref) {
warn $raw_json;
die "Can't parse data from $url";
}
my #json = #$json_array_ref;

Iterating through an array of Perl Hashes

I have written a perl script which accessed JIRA REST API to GET a list of issues that match a specific JQL query. Sometimes thee results are only one issue and other times I get many back.
$client->GET(
$apiPath.$jql.$fieldRes,
$headers);
#a perl hash of results
my $response = from_json($client->responseContent());
while $response is a perl hash, if I try to drill down into the hash I hit an issue.
There is an array of "issues" within the hash.
I am trying to pull data with "foreach" for each specific issue but I keep getting errors:
foreach my $issues ($response->{'issues'})
{
print STDERR Dumper($issues->{'key'});
}
Error...
Pseudo-hashes are deprecated at script.pl line #.
Argument "JIRA-10011" isn't numeric in hash element at script.pl line #.
Bad index while coercing array into hash at script.pl line #.
Any help is appreciated
There is an array of "issues" within the hash.
You cannot put arrays into hashes in Perl, this is only possible for array references. So you need to dereference it when iterating the hash(ref) with your foreach.
foreach my $issues ( #{ $response->{'issues'} } )
{
print STDERR Dumper($issues->{'key'});
}
Since you will get one issue per iteration, you should rename $issues to $issue so you won't get confused later.