Storing "JSON to HASH" output in a variable makes Data::Dumper not work in perl - json

I´m getting an issue when trying to store a json comming from a String into a Hash. Take a look at this example:
use strict;
use warnings;
use utf8;
use JSON;
use Data::Dumper;
my %hash1 = %{get_hash_from_json()};
print "Final Dump:\n";
print Dumper \%hash1 . "\n";
print "Keys:\n";
for (keys %hash1) {printf "key is $_\n";}
sub get_hash_from_json (){
my $json = '{"hello": "world", "hello2": "world"}';
print "Dumper without variable:\n";
print Dumper (from_json($json)) . "\n";
print "Dumper with variable:\n";
my $hash_ref = from_json($json);
my %hash = %{$hash_ref};
print Dumper \%hash . "\n";
return from_json($json);
}
And the output is:
main::get_hash_from_json() called too early to check prototype at example.pl line 10.
Dumper without variable:
$VAR1 = {
'hello' => 'world',
'hello2' => 'world'
};
Dumper with variable:
$VAR1 = 'HASH(0x29c88e0)
';
Final Dump:
$VAR1 = 'HASH(0x2512438)
';
Keys:
key is hello2
key is hello
Does anyone understand why is this happening? Somehow the hash is there but Data::Dumper won´t take it?

Precedence issue.
print Dumper \%hash . "\n";
means
print(Dumper(\%hash . "\n"));
but you want
print(Dumper(\%hash) . "\n");
The thing is, the value returned by Dumper will already end with a new line, so you really don't need another one. The following would do:
print(Dumper(\%hash));
If you want to omit the parens, there's no harm here.
print Dumper \%hash;
As for the prototype-related warning, you are getting it because a call to a sub with a prototype is encountered by the compiler before a declaration for that sub is encountered. In that situation, the call did not use the prototype, so Perl is letting you know this happened.
The simple solution is to just remove the useless prototype (the () in sub get_hash_from_json ()).

You are falling victim to precedence.
print Dumper \%hash1 . "\n";
The . concatenates \%hash1 and the newline, and that's what Dumper outputs. Put parentheses around it to make it work.
print Dumper(\%hash1) . "\n";
Or use say.

Related

Parsing JSON In Perl

I'm trying to parse JSON data in Perl. it is request to Cisco Prime Service. My script works, but parsing doesn't work. And I have a warning,
malformed JSON string, neither array, object, number, string or atom, at character offset 0 (before "HTTP::Response=HASH(...") at InternetBSP.pl line 39.
It is here:
my $json_text = $json->allow_nonref->utf8->relaxed->escape_slash->loose->allow_singlequote->allow_barekey->decode($res);
have no Idee how should I fix it...
use strict;
use warnings;
use JSON -support_by_pp;
use LWP 5.64;
use LWP::UserAgent;
use MIME::Base64;
use REST::Client;
use IO::Socket::SSL;
#So dass es auch ohne SSL Sertifizierung funktioniert
BEGIN { $ENV{PERL_LWP_SSL_VERIFY_HOSTNAME} = 0 }
#Create a user agent object
my $ua = LWP::UserAgent->new(
ssl_opts => {
SSL_verify_mode => SSL_VERIFY_NONE(),
verify_hostname => 0,
}
);
#Create a request
my $req = HTTP::Request->new( GET => 'https://10.10.10.10/webacs/api/v1/data/AccessPoints.json?.full=true' );
$req->content_type('application/json');
$req->authorization_basic( "Username", "Password" );
#Pass request to the user agent and get a response back
my $res = $ua->request($req);
#Check the outcome of the Response
if ( $res->is_success ) {
print $res->content;
} else {
print $res->status_line, "n";
}
my $json = new JSON;
my $json_text = $json->allow_nonref->utf8->relaxed->escape_slash->loose->allow_singlequote->allow_barekey->decode($res);
#my try to pasre the data
foreach my $ap ( #{ $json_text->{queryResponse}->{'entity'} } ) {
print "------------------------\nAccess Point " . $ap->{'accessPointsDTO'}->{'#id'} . "\n";
print "Model:" . $ap->{'accessPointsDTO'}->{'model'} . "\n";
print "MAC Address:" . $ap->{'accessPointsDTO'}->{'macAddress'} . "\n";
print "Serial Number:" . $ap->{'accessPointsDTO'}->{'serialNumber'} . "\n";
print "Software Version:" . $ap->{'accessPointsDTO'}->{'softwareVersion'} . "\n";
print "Status:" . $ap->{'accessPointsDTO'}->{'status'} . "\n";
print "Location:" . $ap->{'accessPointsDTO'}->{'location'} . "\n";
}
I have this like outcome:
{"queryResponse":{"#last":"7","#first":"0","#count":"8","#type":"AccessPoints","#responseType":"listEntityInstances","#requestUrl":"https:\/\/10.66.1.23\/webacs\/api\/v1\/ data\/AccessPoints?.full=true","#rootUrl":"https:\/\/10.66.1.23\/webacs\/api\/v1\/data","entity":[{"#dtoType":"accessPointsDTO","#type":"AccessPoints","#url":"https:\/\/10 .66.1.23\/webacs\/api\/v1\/data\/AccessPoints\/205320"
But it shoud be smth like:
{"queryResponse":
{"#type":"AccessPoints",
"#rootUrl":"https://172.18.138.90/webacs/api/v1/data",
"#requestUrl":"https://172.18.138.90/webacs/api/v1/data/AccessPoints?.full=true",
"#responseType":"listEntityInstances",
"entity":[
{"#url":"https://172.18.138.90/webacs/api/v1/data/AccessPoints/13544533",
"#type":"AccessPoints",
"#dtoType":"accessPointsDTO",
"accessPointsDTO":
{"#id":"13544533",
"#displayName":"13544533",
"adminStatus":"ENABLE",
"bootVersion":"12.4.23.0",
"clientCount":0,
After update :)
------------------------
Access Point 205320
Model:AIR-LAP1142N-E-K9
MAC Address:6c:9c:ed:b5:45:60
Serial Number:FCZ1544W51B
Software Version:7.6.130.0
Status:CLEARED
Location:de.bw.stu.
------------------------
Access Point 205322
Model:AIR-CAP3502I-E-K9
MAC Address:0c:f5:a4:ee:70:10
Serial Number:FCZ184680VB
Software Version:7.6.130.0
Status:CLEARED
Location:de.bw.stu.
------------------------
Access Point 205324
Model:AIR-LAP1142N-E-K9
MAC Address:6c:9c:ed:86:9d:20
Serial Number:FCZ1544W50Y
Software Version:7.6.130.0
Status:CLEARED
Location:de.bw.stu.
malformed JSON string, neither array, object, number, string or atom, at character offset 0 (before "HTTP::Response=HASH(...")
This error message means that the data you are giving to decode is not JSON.
You are passing $res to decode, which is an HTTP::Response object (see above, emphasis mine). You need to use $res->content, which you use for debugging output a few lines above.
if ($res->is_success) {
print $res->content;
} else {print $res->status_line, "n";
}
I would rewrite that whole block of code to this.
die $res->status_line unless $res->is_success;
my $json = JSON->new->allow_nonref
->utf8->relaxed
->escape_slash->loose
->allow_singlequote->allow_barekey;
my $json_text = $json->decode( $res->content );
Instead of printing some debug output and then going on anyway if things went wrong you can just die if the request was not successful.
After that, create your JSON object and configure it. This is way more readable than this long line of code, and we're using a method call to new instead of indirect object notation.
Finally, we are decodeing $res->content.

How do I use encode_json with string in Perl?

Here is my code that I try to open the file to get data and change it to UTF-8, then read each line and store it in variable my $abstract_text and send it back in JSON structure.
my $fh;
if (!open($fh, '<:encoding(UTF-8)',$path))
{
returnApplicationError("Cannot read abstract file: $path ($!)\nERRORCODE|111|\n");
}
printJsonHeader;
my #lines = <$fh>;
my $abstract_text = '';
foreach my $line (#lines)
{
$abstract_text .= $line;
}
my $json = encode_json($abstract_text);
close $fh;
print $json;
By using that code, I get this error;
hash- or arrayref expected (not a simple scalar, use allow_nonref to allow this)
error message also point out that the problem is in this line;
my $json = encode_json($abstract_text);
I want to send the data back as a string (which is in UTF-8). Please help.
I assume you're using either JSON or JSON::XS.
Both allow for non-reference data, but not via the procedural encode_json routine.
You'll need to use the object-oriented approach:
use strict; # obligatory
use warnings; # obligatory
use JSON::XS;
my $encoder = JSON::XS->new();
$encoder->allow_nonref();
print $encoder->encode('Hello, world.');
# => "Hello, world."

Perl variable becoming undefined in loop

Edit: modified code and output to make it more clear
Edit 2: Added example input for reproduction
I have a JSON file and a CSV file and I am running comparisons on the two. The problem is that $asset_ip is correctly defined in the outer foreach loop, but when in the nested loop $asset_ip becomes undefined.
Why is $asset_ip becoming undefined?
#!/usr/bin/perl
# perl -e'use CPAN; install "Text::CSV"'
use strict;
use warnings;
use JSON::XS;
use File::Slurp;
use Text::CSV;
my $csv = Text::CSV->new( { sep_char => ',' } );
my $csv_source = "servers.csv";
my $json_source = "assets.json";
my $dest = "servers_for_upload.csv";
# defined these here as I need to use them in foreach loop and if statement:
my $csv_ip;
my #fields;
open( my $csv_fh, '<', $csv_source ) or die "$! error trying to read";
open( my $dest_fh, '>', $dest ) or die "$! error trying to read";
my $json = read_file($json_source);
my $json_array = decode_json $json;
foreach my $item (#$json_array) {
my $id = $item->{id};
my $asset_ip = $item->{interfaces}->[0]->{ip_addresses}->[0]->{value};
# test the data is there:
if ( defined $asset_ip ) {
print "id: " . $id . "\nip: " . $asset_ip . "\n";
}
while (my $line = <$csv_fh>) {
chomp $line;
if ( $csv->parse($line) ) {
#fields = $csv->fields();
$csv_ip = $fields[0];
}
else {
warn "Line could not be parsed: $line\n";
}
if ( $csv_ip eq $asset_ip ) {
# preppend id to csv array and write these lines to new file
unshift( #fields, $id );
print $dest_fh join( ", ", #fields );
}
}
}
close $csv_fh;
Output:
Use of uninitialized value $asset_ip in string eq at script.pl line 43, <$csv_fh> line 1.
Use of uninitialized value $asset_ip in string eq at script.pl line 43, <$csv_fh> line 2.
Use of uninitialized value $asset_ip in string eq at script.pl line 43, <$csv_fh> line 3.
id: 1003
ip: 192.168.0.2
id: 1004
ip: 192.168.0.3
id: 1005
ip: 192.168.0.4
assets.json:
[{"id":1001,"interfaces":[]},{"id":1003,"interfaces":[{"ip_addresses":[{"value":"192.168.0.2"}]}]},{"id":1004,"interfaces":[{"ip_addresses":[{"value":"192.168.0.3"}]}]},{"id":1005,"interfaces":[{"ip_addresses":[{"value":"192.168.0.4"}]}]}]
Note, that for the first iteration, $asset_ip will be undefined. I will therefore alter the code to only run the eq comparison if $asset_ip is defined. However, for this example I am not doing the check because all iterations are undefined.
servers.csv:
192.168.0.3,Brian,Germany
192.168.0.4,Billy,UK
192.168.0.5,Ben,UK
I think your problem will be this:
foreach my $line (<$csv_fh>) {
You execute this within our outer loop. But when you do this, your $csv_fh ends up at the end of file.
Once you have done this, subsequent iterations of your outer loop will not execute this inner loop, because there's nothing left for it to read from $csv_fh.
An easy test if this is your problem is to add a seek e.g. seek ( $csv_fh, 0, 0 );.
But this isn't an efficient thing to do, because then you'll be looping through the file multiple times - you should instead read it into a data structure and use that.
Edit: Here is your problem:
[{"id":1001,"interfaces":[]},{"id":1003,"interfaces":[{"ip_addresses":[{"value":"192.168.0.2"}]}]},{"id":1004,"interfaces":[{"ip_addresses":[{"value":"192.168.0.3"}]}]},{"id":1005,"interfaces":[{"ip_addresses":[{"value":"192.168.0.4"}]}]}]
And specifically:
[{"id":1001,"interfaces":[]}
Your first element in that array doesn't have a $asset_ip defined.
This means - on your first pass - $asset_ip is undefined and generates the errors. (no line is printed because of your if defined test).
But then - the code proceeds to traverse $csv_fh - reading to the end of file - looking for matches (and fails 3 times, generating 3 error messages.
Second iteration - for id 1002 - the IP isn't in the file anyway, but $csv_fh has already been read to end-of-file (EOF) - so that foreach loop doesn't execute at all.
This can be made workable by:
adding else next; after that if defined.
adding seek to after the while loop.
But really - a rewrite would be in order so you're not re-reading a file over and over anyway.
Very crudely:
#!/usr/bin/perl
# perl -e'use CPAN; install "Text::CSV"'
use strict;
use warnings;
use JSON::XS;
use File::Slurp;
use Text::CSV;
my $csv = Text::CSV->new( { sep_char => ',' } );
my $csv_source = "servers.csv";
my $json_source = "assets.json";
my $dest = "servers_for_upload.csv";
# defined these here as I need to use them in foreach loop and if statement:
my $csv_ip;
my #fields;
open( my $csv_fh, '<', $csv_source ) or die "$! error trying to read";
open( my $dest_fh, '>', $dest ) or die "$! error trying to read";
my $json = read_file($json_source);
my $json_array = decode_json $json;
foreach my $item (#$json_array) {
my $id = $item->{id};
my $asset_ip = $item->{interfaces}->[0]->{ip_addresses}->[0]->{value};
# test the data is there:
if ( defined $asset_ip ) {
print "id: " . $id . "\nip: " . $asset_ip . "\n";
}
else {
print "asset_ip undefined for id $id\n";
next;
}
while ( my $line = <$csv_fh> ) {
chomp $line;
if ( $csv->parse($line) ) {
#fields = $csv->fields();
$csv_ip = $fields[0];
}
else {
warn "Line could not be parsed: $line\n";
}
if ( $csv_ip eq $asset_ip ) {
# preppend id to csv array and write these lines to new file
unshift( #fields, $id );
print {$dest_fh} join( ", ", #fields ),"\n";
}
}
seek( $csv_fh, 0, 0 );
}
close $csv_fh;
I would suggest this also needs:
change of while so you're not re-reading the file each time
You're using Text::CSV so using a print join ( ","... doesn't seem a consistent choice. If your data warrants Text::CSV it's worth keeping it for output too.

Delete one character at End of File in PERL

So I have encountered a problem while programming with PERL. I use a foreach loop to get some data out of the hash, so it has to loop through it.
The Code:
foreach $title (keys %FilterSPRINTHASH) {
$openSP = $FilterSPRINTHASH{$title}{openSP};
$estSP = $FilterSPRINTHASH{$title}{estSP};
$line = "'$title':{'openSP' : $openSP, 'estSP' : $estSP}\n";
print $outfile "$line\n";
}
The thing is, that I am creating a seperate File with the PERL's writting to a file expression, which will be a JSONP text (later used for HTML).
Back to the problem:
As JSONP requires comma's "," after every line that is not the last one, i had to put a comma at the end of line, however when the last line comes in, I have to remove the comma.
I have tried with CHOP function, but not sure where to put it, since if I put it at the end of foreach, it will just chop the comma in $line, but this wont chop it in the new file I created.
I have also tried with while (<>) statement, with no success.
Any ideas appreaciated.
BR
Using JSON module is far less error prone; no need to reinvent the wheel
use JSON;
print $outfile encode_json(\%FilterSPRINTHASH), "\n";
You can check if it is the last iteration of the loop, then remove the comma from line.
So something like
my $count = keys %FilterSPRINTHASH; #Get number of keys (scalar context)
my $loop_count = 1; #Use a variable to count number of iteration
foreach $title (keys %FilterSPRINTHASH){
$openSP = $FilterSPRINTHASH{$title}{openSP};
$estSP = $FilterSPRINTHASH{$title}{estSP};
$line = "'$title':{'openSP' : $openSP, 'estSP' : $estSP}\n";
if($loop_count == $count){
#this is the last iteration, so remove the comma from line
$line =~ s/,+$//;
}
print $outfile "$line\n";
$loop_count++;
}
i would approach this by storing your output in an array and then joining that with the line separators you wish:
my #output; # storage for output
foreach $title (keys %FilterSPRINTHASH) {
# create each line
my $line = sprintf "'%s':{'openSP' : %s, 'estSP' : %s}", $title, $FilterSPRINTHASH{$title}{openSP}, $FilterSPRINTHASH{$title}{estSP};
# and put it in the output container
push #output, $line;
}
# join all outputlines with comma and newline and then output
print $outfile (join ",\n", #output);

Error use of uninitialized value

So I made this small script today in perl.
For some reason it doesn't seem to be downloading anything, and an error keeps popping up saying
Use of uninitialized value $id in concatenation (.) or string at room.pl line 18.
Use of uninitialized value $id in concatenation (.) or string at room.pl line 18.
Could someone help me fix this code?
Also is using File::Path okay? and this is the json http://media1.clubpenguin.com/play/en/web_service/game_configs/rooms.json
use strict;
use warnings;
use File::Path;
mkpath "rooms/";
use JSON;
use LWP::Simple qw(mirror);
open FILE, 'rooms.json' or die "Could not open file inputfile: $!";
sysread(FILE, my $result, -s FILE);
close FILE or die "Could not close file: $!";
my $json = decode_json($result);
foreach $item ($json) {
my $id = $item->{room_key};
mirror "http://media1.clubpenguin.com/play/v2/content/global/rooms/$id.swf" => "rooms/$id.swf";
}
foreach my $item ... at line 16 should do the trick!
As its a hash-ref you have to loop over it this way:
...
foreach my $item (sort keys %$json) {
my $id = $json->{$item}->{room_key};
print $id . "\n";
#mirror "http://media1.clubpenguin.com/play/v2/content/global/rooms/$id.swf" => "rooms/$id.swf";
}