Accessing values of json structure in perl - json

I have a json structure that I'm decoding that looks like this:
person => {
city => "Chicago",
id => 123,
name => "Joe Smith",
pets => {
cats => [
{ age => 6, name => "cat1", type => "siamese", weight => "10 kilos" },
{ age => 10, name => "cat2", type => "siamese", weight => "13 kilos" },
],
dogs => [
{ age => 7, name => "dog1", type => "siamese", weight => "20 kilos" },
{ age => 5, name => "dog2", type => "siamese", weight => "15 kilos" },
],
},
},
}
I'm able to print the city, id, name by doing:
foreach my $listing ($decoded->{person})
{
my $city = $listing->{city};
my $name = $listing->{name};
name - $city - \n";
}
however, I'm unsure of how to print the pets->cats or pets->dogs. I'm able to do a dump of them by:
my #pets = $listing->{pets}->{cats};
dump #pets;
but I'm not sure how to access them through the hash structure.

Digging into a big structure is pretty simple, once you know the rules:
Wrap hash keys in {}
Wrap array indexes in []
If your top level variable is a reference, use -> before the first identifier.
After the first set of braces or brackets, additional arrows (->) are optional.
So:
* $data->{person}{name} returns 'Joe Smith'
* $data->{person}->{name} also returns 'Joe Smith'
* $data->{pets}{cats}[0]{age} returns 6.
For way more detail on this topic, see the Perl Data Structures Cookbook (perldoc perldsc)
When you work with big structures like this there are some important things to be aware of. The biggest of these is autovivification. Autoviv means that Perl will automatically make data structure elements pop into existence for you to make your life easier. Unfortunately it can also make things difficult.
For example, autoviv is great when I do this:
my $data;
$data->{horse}[0]{color} = 'brown';
Autoviv magically turns $data into a hashref that contains the key horse with an array ref as its value. The array ref gets populated by a hash ref. The final hash ref then gets the key value pair of color => brown.
The problem comes in when you are walking a structure and do deep tests for existence:
# Code from above continues:
if( exists $data->{cat}[5]{color} ) {
print "Cat 5 has a color\n";
}
use Data::Dumper;
print Dumper $data;
Here, autovivification burns you by creating a bunch of junk in data, here's the program output:
$VAR1 = {
'cat' => [
undef,
undef,
undef,
undef,
undef,
{}
],
'horse' => [
{
'color' => 'brown'
}
]
};
Now you can guard against this kind of thing by carefully testing each layer of your structure for existence, but it's a huge pain in the butt. Instead, I prefer to use Data::Diver.
use Data::Diver qw( Dive );
my $dog_20_color = Dive( $data, 'dog', 20, 'color' );
print "Dog 20 is $dog_20_color\n" if defined $dog_20_color;
$data is unchanged here.
Also, you may have noticed that since Dive takes a list of keys or indexes, that means its easy to programatically build up a list of keys/indexes and descend an arbitrary path in your code.
Data::Diver can be a real life saver when you have to do a lot of manipulation of big, wonky data structures.

Assuming your $listing is a person you have to dereference array and hash refs.
# as long as we are assuming $listing is a person
# this goes inside the foreach you posted in your
# question.
# this will print all cats' names
foreach my $cat ( #{ $listing->{pets}->{cats} } )
{
# here $cat is a hash reference
say $cat->{name}; # cat's name
}
and so on for other stuff.
To access them from the structure you can do:
say $listing->{pets}->{cats}->[0]->{name}; # this will print 'cat1'

my #pets = $listing->{pets}->{cats};
This isn't doing what you think it is. $listing->{pets}->{cats} contains a reference to an array. Your new #pets array ends up contains just one element - the array reference.
What you actually need is
my #pets = #{ $listing->{pets}{cats} };
This deferences the array reference and gets you the actual array. Notice that I've also dropped the optional second arrow in the expression.
Once you've got the array, each element of it is a hash reference.
foreach (#pets) {
say $_->{name};
# etc ...
}
Of course, you don't need the intermediate array at all.
foreach (#{ $listing->{pets}{cats} }) {
say $_->{name};
}

Related

Blessed objects conversion to JSON

How do I convert a blessed object into JSON in Perl? Following is the array I have:
#x = ({
'notificationtype' => 'TRAP',
'receivedfrom' => 'UDP: [10.127.34.212]:48909->[10.127.34.182]:162',
'version' => 1,
},
[
[
bless( {
'oidptr' => bless( do{\(my $o = '140059234062224)}, ''netsnmp_oidPtr' )
}, 'NetSNMP::OID' ),
'600',
67
],
[
bless( {
'oidptr' => bless( do{\(my $o = '140059236784112)}, ''netsnmp_oidPtr' )
}, 'NetSNMP::OID' ),
'OID: .iso.org.dod.internet.private.enterprises.14296.1.100.0.0.1',
6
]
]);
I am able to convert $VAR1 alone using encode_json \#x, but when I the use the blessed object, it's not working. I am getting the error:
encountered object 'NetSNMP::OID=HASH(0x136b278)', but neither allow_blessed nor convert_blessed settings are enabled at u.pl line 256.
I expected a serialized JSON object so that I can send it over to server.
How can I do it?
It depends, you want to preserve the visuals ? or the data and hierarchy ? same positions could not be rendered in other terminal sizes. I would preserve just the data and hierarchy. Probably I would diregard focus and events. I would start umarating the properties like the following and then creaginf a factory These are the ones I found for Box
container.options
container.position,
container.getLines()
container.getText()
container.getContent
container.children
container.parent
container.style
container.type
container.visible
container.height
container.hidden
container.index
subclasses have rows, data, lines, and more... I dont think there is a mechanic way of doing it...
if you want to recreate the hierarchy, you also need to iterate on children and parent, probaly the entire tree.
Thinking about it perhaps just with this.options is enough and children and parent.. good luck will try the same...

return json structure using perl

i have the following sql statement inside a function..
my $sth = $dbh->prepare(qq[SELECT device_uuid,device_name FROM ].DB_SCHEMA().qq[.user_device WHERE user_id = ?]);
$sth->execute($user_id) || die $dbh->errstr;
the results are being fetched using the following statement
while(my $data = $sth->fetchrow_arrayref()) {
}
my question is how can i create and return a json structure containing objects for every row being fetched?something like this
{
object1:{
"device_uuid1":"id1",
"device_name1":"name1"
},
object2:{
"device_uuid2":"id2",
"device_name2":"name2"
},
object3:{
"device_uuid3":"id3",
"device_name3":"name3"
}
}
the total number of json objects will be equal to the number of rows returned by the sql statement.
i have managed to build the structure like this
$VAR1 = [{"device_name":"device1","device_id":"device_id1"},{"device_name":"device2","device_id":"device_id2"}]
how can i iterate through the array refs and get "device_name" and "device_id" values?
For your needs, this library should work well. What you need to do is have a scalar variable defined as below and push the element for each iteration in while loop
my $json = JSON->new->utf8->space_after->encode({})
while(my $data = $sth->fetchrow_arrayref()) {
#Push new element here in $json using incr_parse method
#or using $json_text = $json->encode($perl_scalar)
}
Hope this helps you.
finally what i did was to create an array ref and push the fetched rows which are being returned as hash refs
my #device = ();
while(my $data = $sth->fetchrow_hashref()) {
push(#device, $data);
}
last i convert the #device array ref to json and return the outcome
return encode_json(\#device);
The statement handle method fetchall_arrayref() can return an array reference where each element in the referenced array is a hash reference containing details of one row in the resultset. This seems to me to be exactly the data structure that you want. So you can just call that method and pass the returned data structure to a JSON encoding function.
# Passing a hash ref to fetchall_arrayref() tells it to
# return each row as a hash reference.
my $json = encode_json($sth->fetchall_arrayref({});
Your sample JSON is incorrect - JSON is actually quite nicely represented by perl data structures - [] denotes array, {} denotes key-value (very similar to hash).
I would rather strongly suggest though, that what you've asked for is probably not what you want - you've seemingly gone for globally unique keys, which ... isn't good style when they're nested.
Why? well, so you can do things like this:
print $my_data{$_}->{'name'} for keys %my_data;
Far better to go for something like:
#!/usr/bin/env perl
use strict;
use warnings;
use Data::Dumper;
use JSON;
my %my_data = (
object1 => {
uuid => "id1",
name => "name1"
},
object2 => {
uuid => "id2",
name => "name2"
},
object3 => {
uuid => "id3",
name => "name3"
},
);
print Dumper \%my_data;
print to_json ( \%my_data, { 'pretty' => 1 } )."\n";
Now, that does assume your 'object1' is a unique key - if it isn't, you can instead do something like this - an array of anonymous hashes (for bonus points, it preserves ordering)
my #my_data = (
{ object1 => {
uuid => "id1",
name => "name1"
}
},
{ object2 => {
uuid => "id2",
name => "name2"
}
},
{ object3 => {
uuid => "id3",
name => "name3"
}
},
);
Now, how to take your example and extend it? Easy peasy really - assemble what you want to add to your structure in your loop, and insert it into the structure:
while(my $data = $sth->fetchrow_arrayref()) {
my $objectname = $data -> [0]; #assuming it's this element!
my $uuid = $data -> [1];
my $name = $data -> [2];
my $new_hash = { uuid => $uuid, name => $name };
$mydata{$objectname} = $new_hash;
}

Building ordered JSON in perl

I'm having a bit of a fiddle with JSON and D3, trying to represent some disk usage information in a 'bubble' style.
Based on this initially: http://bl.ocks.org/mbostock/4063269
The JSON is pretty simple - hierarchical data looking a bit like this:
http://bl.ocks.org/mbostock/raw/4063530/flare.json
{
"name": "flare",
"children": [
{
"name": "analytics",
"children": [
{
"name": "cluster",
"children": [
{"name": "AgglomerativeCluster", "size": 3938},
{"name": "CommunityStructure", "size": 3812},
{"name": "HierarchicalCluster", "size": 6714},
{"name": "MergeEdge", "size": 743}
]
}
]
}
]
}
Now, what I'm trying to do is take a quotas report from a NetApp - in XML form.
I have multiple 'per server' XML files looking approximately like:
<quotas>
<quota>
<quota-target>/vol/vol1/qtree1</quota-target>
<volume>vol1</volume>
<disk-used>554444</disk-used>
<disk-limit>2000000</disk-limit>
</quota>
<quota>
<quota-target>/vol/vol1/qtree2</quota-target>
<volume>vol1</volume>
<disk-used>1235655</disk-used>
<disk-limit>2000000</disk-limit>
</quota>
<quota>
<quota-target>/vol/vol2/qtree1</quota-target>
<volume>vol2</volume>
<disk-used>987664</disk-used>
<disk-limit>2000000</disk-limit>
</quota>
</quotas>
What I'm trying to do is assemble some JSON for use with D3 that's hierarchical:
site
server
volume
quota-target
disk-used
I'm doing ok with a foreach loop:
#!/usr/bin/env perl
use strict;
use warnings;
use XML::Twig;
use JSON;
my %sites = (
'site1' => [qw ( servera serverb )],
'site2' => [qw ( s2serverc s2serverd)],
);
my $data;
$data->{'name'} = "quotas";
foreach my $sitename ( keys %sites ) {
my $site = { 'name' => $sitename };
push( #{ $data->{'children'} }, $site );
foreach my $server ( #{ $sites{$sitename} } ) {
my $server = { 'name' => $server };
push( #{ $site->{'children'} }, $server );
$twig->parsefile("$server.quotas.xml");
foreach my $quota ( $twig->get_xpath('//quota') ) {
push(
#{ $server->{'children'} },
{ 'name' => $quota->first_child_text('quota-target'),
'size' => $quota->first_child_text('disk-used')
}
)
}
}
}
open( my $output, ">", "quotas.json" ) or die $!;
print {$output} to_json( $data, { 'pretty' => 1 } );
close($output);
This is broadly working, and producing me pretty pictures.
However I'm having two problems:
Ordering of the JSON changes each run, because I'm using a hash. Whilst not a show stopper - is there a way I can enforce an order in the JSON output? (Not necessarily just 'sorted' alphabetically)
Similarly - I'm looking at how to insert a 'volume' level node which isn't currently present, as I'm creating new anonymous hashes to insert into children at each layer of the foreach loop. This feels a bit clunky, but what I'm thinking is:
Extract a list of volumes with get_xpath('//volume') and uniqueify it.
Either iterate per volume finding subnodes that match (Is there an xpath expression to specify a child value?)
Or create a 'staging' hash of hashes that I then 'merge' into children in the JSON.
Does anyone have any better suggestions?
I can quite easily create a hash of the desired structure e.g.
$stuff{$site}{$server}{$volume}{$qtree} = $size;
But then would have to turn that into the appropriate JSON (which I suppose might be a better approach overall).
is there a way I can enforce an order in the JSON output?
Yeah, use an array instead of an object. JSON objects are unordered.
An object is an unordered set of name/value pairs.
But it seems you're already using arrays for your lists.
Maybe you're want to be able to perform diffs, in which case JSON.pm provides a coarse means of specifying keys in the form of sort_by. Useful if you want to perform diffs.
If you just want to "enforce" some sorting by the hash key, you can use the "canonical" feature, as also said here.

What is the best means to get the unique hash data into a JSON object without using an array when cycling through a DB Dataset?

I need to get the data into a JSON object but because I'm using the %data hash and it has the same address I'm getting the same data repeatedly in my JSON object.
This is the code that produces the JSON.
while (my ($orderID, $possessorName, $itemDescription, $...) = $sth->fetchrow_array)
{
%data = (orderID => $orderID, possessorName => $possessorName, itemDescription => $itemDescription,...);
$query_results{"job$index"} = {"data" => \%data};
$index++;
}
return $json_obj->pretty->encode(\%query_results, {ascii => 1, pretty => 1});
The problem is that the last item in my data set is masking all the previous items so I end up with one large JSON of the same exact data. I could use an array of hashes I suppose but this seems really messy and sloppy. How do I write the cleanest code to get my data? If an array of hashes is the best way to go please let me know and I'll do it. I all ready know how or can figure it out on my own.
What happens when you try:
my $index = 0;
my %query_results;
while (my ($orderID, $possessorName, $itemDescription, $...) = $sth->fetchrow_array) {
my %data = (orderID => $orderID, possessorName => $possessorName, itemDescription => $itemDescription,...);
$query_results{"job$index"}{'data'} = \%data;
$index++;
}
Previously, you used a %data hash declared in an outside scope; or worse, you didn't use strict; use warnings so %data was in fact an implicit global. Now, we declare the %data inside the loop which makes all the hashes distinct.
You could also copy the hash into a new hashref by {%data}.
That said, you don't even need that variable:
$query_results{"job$index"}{data} = {
# anonymous hashref here
orderID => $orderId,
possessorName => $possessorName,
itemDescription => ...
};

How do I create a simple HTML table with hyperlinks from a hash in Perl?

I have a Perl hash of "people" like this:
my $data = {
124535 => {
NAME => "abe",
AGE => 100,
SEX => "m",
HOMEPAGE => qw (http://abe.knaan.old)
},
54478 => {
NAME => "joe",
AGE => 18,
SEX => "m",
HOMEPAGE => qw (http://slappy.joe.com)
},
54478 => {
NAME => "jane",
AGE => 20,
SEX => "f",
HOMEPAGE => qw (http://i.am.jane/jane.html)
},
};
I would like to print an HTML page with a table of all people, one row per person, with all its data including the hash key (i.e. 5 columns), including a hyperlinks to it's homepage.
I can write a long piece of ugly code that prints all the HTML headers etc., but is there a nicer, cleaner way to that? Perhaps using some modules for this, I guess, quite popular task?
I found HTML::QuickTable but I'm not sure how to convert my structure to an appropriate one.
This should do it:
my #names = qw(NAME AGE SEX HOMEPAGE);
my #data = [#names, 'KEY'];
for my $k (keys %$data) {
my #t = #{$data->{$k}}{#names};
$t[-1] = qq{$t[-1]};
push #data, [#t, $k]
}
use HTML::QuickTable;
my $qt = HTML::QuickTable->new(... labels => 1);
print $qt->render(\#data);