This question already has answers here:
change json file by bash script
(6 answers)
Closed 6 months ago.
I have a file called usr.json, and this file contains a JSON object. The object looks as following:
{
"users": {
"accessRights":"FFFFF",
"name": "admin"
}
}
Now I want to replace the string for accessRights with a new value which is stored in a variable called newUsrCfg.
Since I'm working with a remote device I need to ssh to it, and for that I have a script.
However, I call all this in Python code:
def test_tc():
newUsrCfg = "F7FFF"
cmd = f'sed -ie \'s/\"accessRights\":.*/\"accessRights\": \"{newUsrCfg}\"/g\' "/etc/hgp/runtime/user/user1.json"'
subprocess.call(["bash", pathToSshCommand, cmd])
The commands need to be either called with single quotes or double ones. And when opening the file the output of this is:
{
"users":{
"accessRights": "f7fff"
I tried many other sed commands with different formats, but nothing worked.
If jq is available to you, you can
jq '.users.accessRights = "f7fff"' usr.json
As also commented below, manipulating json with text replacement is likely to break if your input does not exactly meet your expectations (for instance though differences in whitespace). If you have a tool like jq which understands json, that is always the preferable option.
However, in case you still want to do it with sed you could adjust your regex to be less inclusive. Since you want to replace 5 hex values, something along the following lines should work (I hope I escaped everything correctly, can't test since I'm on the way).
cmd = 'sed -ie \'s/\"accessRights\":\s*\"[0-9,a-f,A-F]\{5\}\"/\"accessRights\": \"{newUsrCfg}\"/g\' "/etc/hgp/runtime/user/user1.json"'
See regex101.
A third option would to both load the file via ssh and manipulate the json in Python, or do it directly in Python on the target system, if Python is available there as well.
Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 3 years ago.
Improve this question
I'm trying to convert a large JSON file to a CSV format. (I've just learned how to use jq so I'm still a beginner).
I've successfully managed to convert most of the data, however, I'm stuck at an array. Every JSON object in the file is supposed to be converted to a single CSV row, and I can't get that to work.
I've been trying to help myself with an existing answer:
Convert/Export JSON to CSV
But the problem is that this method writes a row for each item in the array, needlessly repeating information.
I am using the same type of command as the above answer, the only difference being the name of the columns, but the array blocks...
For instance, I could have a JSON file resembling:
{resources:[
{"id":"001","name"="Robert","items":[
{"label":"00A","name":"Pen"},
{"label":"00B","name":"Paper"}],
{"id":"002","name"="Bruce","items":[
{"label":"00A","name":"Pen"},
{"label":"00B","name":"Computer"},
{"label":"00C","name":"Headphones"}]
]
}
That I would like to become:
001,Robert,Pen,Paper,
002,Bruce,Pen,Computer,Headphones
I only need the name columns of the array
For the moment, the result is:
001,Robert,Pen
001,Robert,Paper
002,Bruce,Pen,
002,Bruce,Computer
002,Bruce,Headphones
The problem is the actual array is about 30 items long for each JSON object, making it impossible to use this way.
$ jq -r '.resources[] | [.id,.name,.items[].name] | #csv' < /tmp/b.json
"001","Robert","Pen","Paper"
"002","Bruce","Pen","Computer","Headphones"
You can use awk to fix up your current output file.
awk -F, '{a[$1","$2]=a[$1","$2]","$3}; END{for (v in a) print v a[v]}' in.txt | sort >out.txt
Input:
001,Robert,Pen
001,Robert,Paper
002,Bruce,Pen,
002,Bruce,Computer
002,Bruce,Headphones
Output:
001,Robert,Pen,Paper
002,Bruce,Pen,Computer,Headphones
I have written Perl code that was working until recently, when I tried to run it again. The problem seems to originate from the JSON::XS "decode_json" method.
Code Snippet:
use warnings;
use strict;
use MooseX::Singleton;
use Array::Utils qw(:all);
use Data::Dumper;
use JSON::XS qw(encode_json decode_json);
use Storable;
use Tie::IxHash;
open (my $observations_fh, '<', 'observations.json') or die "Could not open observations.json\n";
my $observations_json = <$obserations_fh>;
my #decoded_observations = #{decode_json($observations_json)};
Usually, after this code I was able to go through each JSON component in a for loop and take specific information, but now I get the error:
, or ] expected while parsing array, at character offset 5144816
(before "(end of string)")
I saw a similar question here, but it didn't resolve my problem.
I also have similar json decoding going on that doesn't utilize #{decode_json($variable)}, but when I tried that with this observations.json file, the same error was output.
I also tried just using the JSON module, but same error occurred.
Any insight would be greatly appreciated!
-cookersjs
That probably indicates you have incomplete JSON in $observations_json. Your assumption that the entire file consists of just one line is probably incorrect. Use
my $observations;
{
open (my $observations_fh, '<', 'observations.json')
or die("Can't open observations.json: $!\n");
local $/;
my $observations_json = <$obserations_fh>;
$observations = decode_json($observations_json);
}
If that doesn't help, observations.json doesn't contain valid JSON.
I'm relatively new to Perl and trying to self teach. However i have read all of the related threads on this page and others and none of them seem to work for me.
Below is my code - trying to get a lot of data from a webpage in Perl format and export it to update values in an SQL table.
Currently i can't even data dumper the results of the url out.
Any help would be great.
#!/usr/bin/perl
#
use LWP::Simple;
use warnings;
use strict;
use JSON qw( decode_json from_json );
use LWP::Simple;
use Data::Dumper;
use utf8;
my $url = "http://.sensitivedata.txt";
my #json= from_json(get ( $url ));
die "Couldn't get $url" if not defined #json;
##my $decoded_json = decode_json( #json);
print Dumper #json;
exit 0;
This is the error message it is giving me:
defined(#array) is deprecated at alarms.pl line 14.
(Maybe you should just omit the defined()?)
malformed JSON string, neither array, object, number, string or atom, at character offset 0 (before "(end of string)") at /opt/csw/share/perl/csw/JSON.pm line 168
The error message is pretty clear about a) what the problem is and b) how to get rid of it.
defined(#array) is deprecated at alarms.pl line 14. (Maybe you should
just omit the defined()?)
Calling defined() on #json is pointless. You're really just checking to see if there is any data in the array so replace if not defined #json with if not #json.
That will get rid of the error message. But you'll still have a problem as your program will almost certainly now die on the same line with the error message "Couldn't get http://.sensitivedata.txt". And that's probably not an accurate error message.
The problem is that this error can be caused by two problems. Either you can't get the data or you can't parse the data. Your error message only mentions one of these possibilities. Better to split the error checking into two.
# Step 1: Get the data
my $raw_json = get($url);
die "Can't get data from $url" unless $raw_json;
# Step 2: Parse the data
my #json = from_json($raw_json);
if (!#json) {
warn $raw_json;
die "Can't parse data from $url";
}
With code more like this, you'll be able to see what the problem is.
There's another little problem here, so to pre-empt your next question...
from_json always returns a scalar. It will either be a hash reference or an array reference (depending on the JSON you get). Looks like you're expecting an array. You'll need to store the reference in a scalar and dereference it.
my $json_array_ref = from_json($raw_json);
if (!#$json_array_ref) {
warn $raw_json;
die "Can't parse data from $url";
}
my #json = #$json_array_ref;
I have written a perl script which accessed JIRA REST API to GET a list of issues that match a specific JQL query. Sometimes thee results are only one issue and other times I get many back.
$client->GET(
$apiPath.$jql.$fieldRes,
$headers);
#a perl hash of results
my $response = from_json($client->responseContent());
while $response is a perl hash, if I try to drill down into the hash I hit an issue.
There is an array of "issues" within the hash.
I am trying to pull data with "foreach" for each specific issue but I keep getting errors:
foreach my $issues ($response->{'issues'})
{
print STDERR Dumper($issues->{'key'});
}
Error...
Pseudo-hashes are deprecated at script.pl line #.
Argument "JIRA-10011" isn't numeric in hash element at script.pl line #.
Bad index while coercing array into hash at script.pl line #.
Any help is appreciated
There is an array of "issues" within the hash.
You cannot put arrays into hashes in Perl, this is only possible for array references. So you need to dereference it when iterating the hash(ref) with your foreach.
foreach my $issues ( #{ $response->{'issues'} } )
{
print STDERR Dumper($issues->{'key'});
}
Since you will get one issue per iteration, you should rename $issues to $issue so you won't get confused later.