How to access data on Perl Object structures - json

I have the following perl code in where I have a perl structure as follows:
`
use Data::Dumper;
my %data = (
'status' => 200,
'message' => '',
'response' => {
'name' => 'John Smith',
'id' => '1abc579',
'ibge' => '3304557',
'uf' => 'XY',
'status' => bless( do{\(my $o = 1)}, 'JSON::PP::Boolean' )
}
);
my $resp = $data{'status'};
print "Response is $resp \n";
print Dumper(%data->{'response'});
Getting the status field works, however If I try something like this:
my $resp = $data{'response'}
I get Response is HASH(0x8b6640)
So I'm wondering if there's a way I can extract all the data of the 'response' field on the same way I can do it for 'status' without getting that HASH...
I've tried all sort of combinations when accessing the data, however I'm still getting the HASH back when I try to get the content of 'response'

$data{'response'} is the correct way to access that field on a hash called %data. It's returning a hash reference, which prints out by default in the (relatively unhelpful) HASH(0x8b6640) syntax you've seen. But if you pass that reference to Dumper, it'll show you everything.
print Dumper($data{'response'});
to actually access those subfields, you need to dereference, which is done with an indirection -> operation.
print $data{'response'}->{'name'}
The first access doesn't need the -> because you're accessing a field on a hash variable (i.e. a variable with the % sigil). The second one does because you're dereferencing a reference, which, at least in spirit, has the $ sigil like other scalars.

Thanks for your posts. I fixed the code as follows:
use Data::Dumper;
my %data = (
'status' => 200,
'message' => '',
'response' => {
'name' => 'John Smith',
'id' => '1abc579',
'ibge' => '3304557',
'uf' => 'XY',
'status' => bless( do{\(my $o = 1)}, 'JSON::PP::Boolean' )
}
);
my $resp = $data{'response'};
print Dumper($resp);
Now it works like a charm, and I'm able to get the data I want.

Related

WP_remote_post how to add filters to JSON API call

I am trying to integrate an API into my Wordpress plugin. The following PHP code successfully connects to the API and retrieves a list of Estates (the API is from a real-estate software):
$url = 'https://api.whise.eu/v1/estates/list';
$body = array(
'Filter' => array( 'languageId' => 'nl-BE'),
);
$args = array(
'headers' => array( 'Authorization' => 'Bearer ' . $token),
'body' => json_decode($body)
);
$response = wp_remote_post($url,$args);
As per the documentation (http://api.whise.eu/WebsiteDesigner.html#operation/Estates_GetEstates) it is possible to filter the results, but I haven't been able to get this to work. I don't have much experience with APIs and JSON, so I might be missing something here.
The code above still retrieves the data in English, even though I added the language filter as explained in the docs. When I replace 'body' => json_decode($body) with 'body' => $body, I get the following response:
{"Message":"The request entity's media type 'application/x-www-form-urlencoded' is not supported for this resource."}
Thanks!
Just so that this question doesn't go unanswered:
You need to add the Content-Type header and set it to application/json. This is so the endpoint can interpret your data as a JSON string.
You also need to change 'body' => json_decode($body) to 'body' => json_encode($body) as you want to convert your $body array into a JSON string (see json_decode() vs json_encode()).
This is how your code should look now:
$url = 'https://api.whise.eu/v1/estates/list';
$body = array(
'Filter' => array( 'languageId' => 'nl-BE'),
);
$args = array(
'headers' => array(
'Authorization' => 'Bearer ' . $token,
'Content-Type' => 'application/json'
),
'body' => json_encode($body)
);
$response = wp_remote_post($url,$args);

Read CSV to parse data and store it in Hash

I have a CSV file, which contains data like below:
I want parse data from above csv file and store it in a hash initially. So my hash dumper %hash would look like this:
$VAR1 = {
'1' => {
'Name' => 'Name1',
'Time' => '7/2/2020 11:00'
'Cell' => 'NCell1',
'PMR' => '1001',
'ISD' => 'ISDVAL1',
'PCO' => 'PCOVAL1'
},
'2' => {
'Name' => 'Name2',
'Time' => '7/3/2020 13:10',
'Cell' => 'NCell2',
'PMR' => '1002',
'PCO' => 'PCOVAL2',
'MKR' => 'MKRVAL2',
'STD' => 'STDVAL2'
},
'3' => {
'Name' => 'Name3',
'Time' => '7/4/2020 20:15',
'Cell' => 'NCell3',
'PMR' => '1003',
'ISD' => 'ISDVAL3',
'MKR' => 'MKRVAL3'
},
};
Script is below:
#!/usr/bin/perl
use strict;
use warnings;
use Text::CSV;
use Data::Dumper;
my %hash;
my $csv = Text::CSV->new ({ binary => 1, auto_diag => 1 });
open my $fh, "<:encoding(utf8)", "input_file.csv" or die "input_file.csv: $!";
while (my $row = $csv->getline ($fh)) {
my #fields = #$row;
$hash{$fields[0]}{"Time"} = $fields[1];
$hash{$fields[0]}{"Name"} = $fields[2];
$hash{$fields[0]}{"Cell"} = $fields[3];
}
close $fh;
print Dumper(\%hash);
Here id is an key element in each line and based on the data value each data should be stored in respective names of an id.
Problem here is, till column D (Cell) I am able to parse data in above script and there after column D there won't be a header line and it will be like column E will act as header and column F is the value for the particular header's particular id. Similar condition goes to rest of the data values until end. And in middle we can see some values also will be missing. For example there is No MKR value for id 1.
How can I parse these data and store it in hash, so that my hash would look like above. TIA.
Changes made to the script posted was to remove the header line so that it does not form part of the result and added a for loop to set the reset of the data.
Test Data Used:
id,Time,Name,Cell,,,,,
1,7/2/2020 11:00,Name1,NCell1,PMR,1001,ISD,ISDVAL1
2,7/3/2020 13:10,Name2,NCell3,PMR,1002,PCO,PCOVAL2,MKR,MKRVAL2
Updated Script: (This was the first version suggest using the improved version in the edit)
#!/usr/bin/perl
use strict;
use warnings;
use Text::CSV;
use Data::Dumper;
my %hash;
my $csv = Text::CSV->new ({ binary => 1, auto_diag => 1 });
open my $fh, "<:encoding(utf8)", "input_file.csv" or die "input_file.csv: $!";
my $headers = $csv->getline ($fh);
while (my $row = $csv->getline ($fh)) {
$hash{$row->[0]}{Time} = $row->[1];
$hash{$row->[0]}{Name} = $row->[2];
$hash{$row->[0]}{Cell} = $row->[3];
for (my $i = 4; $i < scalar (#{$row}); $i += 2) {
$hash{$row->[0]}{$row->[$i]} = $row->[$i + 1];
}
}
close $fh;
print Dumper(\%hash);
Output:
$VAR1 = {
'2' => {
'MKR' => 'MKRVAL2',
'Name' => 'Name2',
'PCO' => 'PCOVAL2',
'Cell' => 'NCell3',
'Time' => '7/3/2020 13:10',
'PMR' => '1002'
},
'1' => {
'Name' => 'Name1',
'ISD' => 'ISDVAL1',
'Cell' => 'NCell1',
'Time' => '7/2/2020 11:00',
'PMR' => '1001'
}
};
Edit:
Thanks to comment from #choroba here is an improved version of the script setting the hash with all the additional row values first and then adding the first values Time Name Cell using the header line read from the file.
#!/usr/bin/perl
use strict;
use warnings;
use Text::CSV;
use Data::Dumper;
my %hash;
my $csv = Text::CSV->new ({ binary => 1, auto_diag => 1 });
open my $fh, "<:encoding(utf8)", "input_file.csv" or die "input_file.csv: $!";
my $headers = $csv->getline ($fh);
while (my $row = $csv->getline ($fh)) {
$hash{$row->[0]} = { #$row[4 .. $#$row] };
#{$hash{$row->[0]}}{#$headers[1, 2, 3]} = #$row[1, 2, 3];
}
close $fh;
print Dumper(\%hash);
There are some Text::CSV features that you can use to make this a bit simpler. There's a lot of readability to gain by removing density in the loop.
First, you can set the column names for missing header values. I don't know what those columns represent so I've called them K1, V1, and so on. You can substitute better names for them. How I do that isn't as important is that I do that. I'm using v5.26 because I'm using postfix dereferencing:
use v5.26;
my $headers = $csv->getline($fh);
my #kv_range = 1 .. 4;
$headers->#[4..11] = map { ("K$_", "V$_") } #kv_range;
$csv->column_names( $headers );
If I knew the names, I could use those instead of numbers. I merely change the stuff in #kv_range:
my #kv_range = qw(machine test regression ice_cream);
And, when the data file changes, I handle all of that here. When it's outside the loop, there's much less to miss.
Now that I have all columns named, I use getline_hr to get back a hash reference of the line. The keys are the column names I just set. This does a lot of the work for you already. You have to handle the pairs at the end, but that's going to be easy too:
my %Grand;
while( my $row = $csv->getline_hr($fh) ) {
foreach ( #kv_range ) {
no warnings 'uninitialized';
$row->{ delete $row->{"K$_"} } = delete $row->{"V$_"};
}
$Grand{ $row->{id} } = $row;
delete $row->#{ 'id', '' };
}
Now to handle the pairs at the end: I want to take the value in the column K1 and make it a key, then take the value in V1 and make that the value. At the same time, I need to remove those K1 and V1 columns. delete has the nice behavior in that it returns the value for the key you deleted. This way doesn't require any sort of pointer math or knowledge about positions. Those things might change and I've handled all of that before I got this far:
$row->{ delete $row->{"K$_"} } = delete $row->{"V$_"};
You could also do this in a couple steps if that statement is too much for you:
my( $key, $value ) = delete $row->#{ "K$_", "V$_" };
$row->{$key} = $value;
I'd leave the id column in there, but if you don't want it, get rid of it. Also, that step with the deletes might have made some empty string keys for the cells that had no values. Instead of guarding against that and making the foreach more complicated, I let it happen and get rid of it at the end:
delete $row->#{ 'id', '' };
Altogether, it looks like this. It's doing the same thing as Piet Bosch's answer, but I've pushed a lot of the complexity back into the module as well as doing a little pre-loop work:
use v5.26;
use strict;
use warnings;
use Data::Dumper;
use Text::CSV;
my $csv = Text::CSV->new({
binary => 1,
auto_diag => 1
});
open my $fh, "<:encoding(utf8)", "input_file.csv"
or die "input_file.csv: $!";
my $headers = $csv->getline($fh);
my #kv_range = 1 .. 4;
$headers->#[4..11] = map { ("K$_", "V$_") } #kv_range;
$csv->column_names( $headers );
my %Grand;
while( my $row = $csv->getline_hr($fh) ) {
foreach ( #kv_range ) {
no warnings 'uninitialized';
$row->{ delete $row->{"K$_"} } = delete $row->{"V$_"};
}
$Grand{ $row->{id} } = $row;
delete $row->#{ 'id', '' };
}
say Dumper( \%Grand );
And the output looks like this:
$VAR1 = {
'2' => {
'PMR' => '1002',
'PCO' => 'PCOVAL2',
'MKR' => 'MKRVAL2',
'Name' => 'Name2',
'Time' => '7/3/2020 13:10',
'Cell' => 'NCell3'
},
'1' => {
'Cell' => 'NCell1',
'Time' => '7/2/2020 11:00',
'ISD' => 'ISDVAL1',
'PMR' => '1001',
'Name' => 'Name1'
}
};

Creating hash of hashes from list of network interface config files in perl

I'm trying to load the list of network interface configuration files on Linux into the hash of hashes and further encode them into JSON. This is the code that I'm using:
#!/usr/bin/env perl
use strict;
use diagnostics;
use JSON;
use Data::Dumper qw(Dumper);
opendir (DIR, "/etc/sysconfig/network-scripts/");
my #configs =grep(/^ifcfg-*/, readdir(DIR));
my $output = "metadata/json_no_comment";
my %configuration;
my $key;
my $value;
my %temp_hash;
foreach my $input ( #configs) {
$input= "/var/tmp/rhel6.8/" . $input;
open (my $JH, '<', $input) or die "Cannot open the input file $!\n";
while (<$JH>) {
s/#.*$//g;
next if /^\s*#/;
next if /^$/;
for my $field (split ) {
($key, $value) = split /\s*=\s*/, $field;
$temp_hash{$key} = $value;
}
$configuration{$input} = \%temp_hash;
}
close $JH;
}
print "-----------------------\n";
print Dumper \%configuration;
print "-----------------------\n";
my $json = encode_json \%configuration;
open (my $JNH, '>', $output) or die "Cannot open the output file $!\n";
print $JNH $json;
close $JNH;
The data structure, that I'm getting is following:
$VAR1 = {
'/etc/sysconfig/network-scripts/ifcfg-lo' => {
'BOOTPROTO' => 'dhcp',
'NAME' => 'loopback',
'TYPE' => 'Ethernet',
'IPV6INIT' => 'yes',
'HWADDR' => '"52:54:00:65:e7:8c"',
'DEVICE' => 'lo',
'NETBOOT' => 'yes',
'NETMASK' => '255.0.0.0',
'BROADCAST' => '127.255.255.255',
'IPADDR' => '127.0.0.1',
'NETWORK' => '127.0.0.0',
'ONBOOT' => 'yes'
},
'/etc/sysconfig/network-scripts/ifcfg-eth0' => $VAR1->{'/etc/sysconfig/network-scripts/ifcfg-lo'}
};
The data structure, I'm looking for is the following:
$VAR1 = {
'/etc/sysconfig/network-scripts/ifcfg-lo' => {
'BOOTPROTO' => 'dhcp',
'NAME' => 'loopback',
'TYPE' => 'Ethernet',
'IPV6INIT' => 'yes',
'HWADDR' => '"52:54:00:65:e7:8c"',
'DEVICE' => 'lo',
'NETBOOT' => 'yes',
'NETMASK' => '255.0.0.0',
'BROADCAST' => '127.255.255.255',
'IPADDR' => '127.0.0.1',
'NETWORK' => '127.0.0.0',
'ONBOOT' => 'yes'
},
'/etc/sysconfig/network-scripts/ifcfg-eth0' => {
'BOOTPROTO' => 'dhcp',
'NAME' => '"eth0"',
'TYPE' => 'Ethernet',
'IPV6INIT' => 'yes',
'HWADDR' => '"52:54:00:65:e7:8c"',
'NETBOOT' => 'yes',
'ONBOOT' => 'yes'
}
};
Any idea what am I doing wrong? Why the first nested hash is created correctly and the second one is not? I suspect, that it has something to do with reading the files line by line, but I have to do it, because I need to filter out the commented lines before JSON conversion.
Thanks for any help.
Edit: I have modified the script as suggested by Borodin and it works. Thanks!
The problem is that $configuration{$input} always refers to the same hash %temp_hash because you have declared it at file level. You need to created a new hash for each config file by declaring %temp_hash inside the for loop
Also note that next if /^\s*#/ can have no effect because you just deleted any hashes in the line. Your sanitisation should look like
s/#.*//;
next unless /\S/;

from_json and decode_json in perl not working as expected

I am having a json like below.
$response = {"entries":[{"content":{"eStatus":0,"id":"0","enabled":false,"isOK":false,}}]}
$responseContent = from_json($response);
when i am using from_json or decode_json within my perl code I get following as the converted json.
'content' => {
'id' => '0',
'esrsVeAddress' => '',
'isOK' => bless( do{\(my $o = 0)}, 'JSON::PP::Boolean' ),
'enabled' => $VAR1->{'entries'}[0]{'content'}{'isOK'},
'eStatus' => 0
}
When value of element is not boolean ( true/false ) conversion happens properly but when its boolean then output is corrupted.
If you will see value of element 'enabled' and 'isOK' both are wrong.
Am i using wrong function 'from_json' or 'decode_json' here.
Any suggestions or guidance is appreciated.
This is the way i am trying to use value of element isOK and enabled.
if ( $isOK = $responseContent->{'entries'}[0]->{'content'}-> {'isOK'} eq "1" ) {
c4lx_log "value is found to be true and so do some business logic";
}
else {
c4lx_log "value is found to be false and so dont do anything here";
}
NOTE:
Input to fucntion 'from_json' or 'decode_json' is coming from the REST response in format as shown above. I have verified that input is passed correctly and as expected. Its just the conversion that is issue here.
If you will see value of element 'enabled' and 'isOK' both are wrong.
No, they are correct. Both ->{enabled} and ->{isOK} are false as in the JSON.
How can i avoid conversion like above
You are actually asking why the module doesn't convert the value (to a string or number). That would cause information to be lost. For example, if the conversion you desire would be performed, encode_json(decode_json($json)) would change the data.
What can be done here to make the value as I am expecting [which is] 'content' => { 'id' => '0', 'esrsVeAddress' => '', 'isOK' => false, 'enabled' => false, 'eStatus' => 0 }
That's impossible using Data::Dumper. Data::Dumper produces valid Perl code, and that's not valid Perl code. However, you can get close to that by using
print(Data::Dumper->Dump(
[ JSON::false, JSON::true, $responseContent ],
[qw( $false $true $responseContent )]));
It produces the following output:
$false = bless( do{\(my $o = 0)}, 'JSON::PP::Boolean' );
$true = bless( do{\(my $o = 1)}, 'JSON::PP::Boolean' );
$responseContent = {
'entries' => [
{
'content' => {
'id' => '0',
'eStatus' => 0,
'enabled' => $false,
'isOK' => $false
}
}
]
};
This is the way i am trying to use value of element isOK and enabled.
It's wrong to expect a boolean to have a specific value. Change
my $isOK;
if ($isOK = $responseContent->{'entries'}[0]->{'content'}->{'isOK'} eq "1")
to
if ($responseContent->{'entries'}[0]->{'content'}->{'isOK'})
That can be simplified to
if ($responseContent->{entries}[0]{content}{isOK})
If you want to store the result for later, use
my $isOK = $responseContent->{entries}[0]{content}{isOK};
if ($isOK)

Perl hash to json conversion?

i have coded a crawler in perl to to get the url, title of the site and put it into a hash as following
$VAR1 = {
'address1' => {
'url' => 'dthree',
'title' => 'done'
},
'address2' => {
'url' => 'dthree',
'title' => 'done'
}
};
then how can i convert it into json format .. im using MOJO::Json
The first four lines in Mojo::JSON's SYNOPSIS will tell you.
use Mojo::JSON;
my $json = Mojo::JSON->new->encode( $VAR1 );