Perl - De- Referencing a Hash - json

I am just wondering If I can get some help with dereferencing in Perl?
I have a while loop where I am querying a DB and iterating over what I get back. I then write the data I need into a hash and push the hash into an array. This is all forming part of a JSON string.
However, I can only push the reference to the hash and not the hash itself (I've tried all sorts of things), meaning if the loop goes (e.g.) 3 times, I get the same thing appearing 3 times in the JSON I am trying to PUT.
Here is the code:
my $json = new JSON::XS;
my $json_text = JSON::XS->new->decode (shift->content);
my $sig_num = 0;
my %sig_hash;
<MySQL Stuff -removed for readability>
while($query_handle->fetch())
{
$sig_num++;
$sig_hash{position} = 'below';
$sig_hash{signature_text} = $sig;
$sig_hash{signature_name} = 'Signature '.$sig_num;
$sig_hash{signature_default} = JSON::XS::true;
push (#{$json_text->{data}->{mail}->{signatures}}, \%sig_hash);
}
return $json_text;
Thanks for any help!

The hash ref that you're pushing on to the array is scoped at the outer level (outside the while loop). This means there is only one hash being referenced: you are pushing references to the same hash on to the array multiple times. I assume you want a fresh hash for each iteration of the loop. If so, declare my %sig_hash inside the loop rather than outside.
You can experiment with this script to see the difference. First run it as it is; then move the my %h outside the loop and run it again.
my #data;
for (1..3){
my %h; # New hash for each iteration of the loop.
$h{a} = 10 * $_;
$h{b} = 20 * $_;
push #data, \%h;
}
use Data::Dumper;
print Dumper(\#data);

I suggest you use an autovivified anonymous hash delared, as FMc explains, within the while loop. The code becomes simpler that way, and becomes
my $json = new JSON::XS;
my $json_text = JSON::XS->new->decode(shift->content);
my $sig_num = 0;
while ($query_handle->fetch) {
my $sig_hash;
$sig_hash->{position} = 'below';
$sig_hash->{signature_text} = $sig;
$sig_hash->{signature_name} = "Signature ".++$sig_num;
$sig_hash->{signature_default} = JSON::XS::true;
push #{$json_text->{data}{mail}{signatures}}, $sig_hash;
}
return $json_text;
or if you prefer you can build and push an anonymous hash directly onto the stack without assigning it to a variable
my $json = new JSON::XS;
my $json_text = JSON::XS->new->decode(shift->content);
my $sig_num = 0;
while ($query_handle->fetch) {
push #{$json_text->{data}{mail}{signatures}}, {
position => 'below',
signature_text => $sig,
signature_name => "Signature ".++$sig_num,
signature_default => JSON::XS::true,
};
}
return $json_text;

Related

Reason my subroutine won't recurse

#!/usr/bin/perl
use strict;
use warnings;
use List::MoreUtils 'uniq';
my %functiontable =();
$functiontable{foo} = \&foo;
sub iterate {
my ($function, $iterations, $argument) = #_;
return $argument unless 0 < $iterations;
return $argument unless $function = $functiontable{$function};
my #functioned = $function->($argument);
my #refunctioned = ();
for my $i (0 .. #functioned - 1) {
push #refunctioned, iterate ($function, ($iterations - 1), $functioned[$i]);
}
return uniq #refunctioned;
}
sub foo {
my ($argument) = #_;
my #list = ($argument, $argument.'.', $argument.',');
return #list;
}
my #results = iterate 'foo', 2, 'the';
print "#results";
This prints the the. the,, i.e. it doesn't iterate (recurse). I would expect it to print the the. the, the.. the., the,. the,,.
(I used Smart::Comments to check whether it enters iterate a second time, and it does, but it doesn't seem to do everything in the function.)
I can't figure out why. Can someone please help me figure out why, or propose a fix?
This line:
return $argument unless $function = $functiontable{$function};
doesn't make sense. In your subroutine iterate, $function is a string and $functiontable{$function} is a reference to a subroutine. I am not sure what the purpose of this is: is it to compare against the stored function? is it to use the function referenced by the name $function?
Assuming the latter it would make more sense to simply pass in a reference to a function when you call iterate:
sub iterate {
my ($function, $iterations, $argument) = #_;
return $argument unless 0 < $iterations;
my #functioned = $function->($argument);
my #refunctioned = ();
for my $i (0 .. #functioned - 1) {
push #refunctioned, iterate ($function, ($iterations - 1), $functioned[$i]);
}
return uniq #refunctioned;
}
my #results = iterate($functiontable{foo}, 2, 'the');
print "#results";
output:
the the. the, the.. the., the,. the,,
The problem is this line.
return $argument unless $function = $functiontable{$function};
The variable $function is being repurposed and overwritten from a string (the function name) to a code reference (the function to be executed). Later, it's passed into iterate which faithfully ignores it.
Two things would improve this code and avoid that sort of problem. First is to not repurpose variables, use two variables.
return $argument unless $function_ref = $functiontable{$function_name};
Now the mistake cannot happen. One strong indicator that you're repurposing a variable is that it changes type, like from a string to a code reference.
Note that I threw out $function entirely because it's too generic in this context. Is that the function's name or the function's reference? Neither one is obvious, so make it obvious.
Finally, iterate can be made more flexible by eliminating the function table entirely. Pass in the code reference directly. If you want a function table, write a wrapper.
sub select_iteration {
my($iteration_type, $iterations, $argument) = #_;
my $iteration_code = $iteration_types{$iteration_type};
return iterate($iteration_code, $iterations, $argument);
}
The first time your subroutine iterate is called it translates the subroutine name in $function from a name to a subroutine reference
So the first time iterate calls itself it is passing the subroutine reference, and the line
return $argument unless $function = $functiontable{$function};
will stringify the reference and attempt to find an element of the hash using a key something like CODE(0x23e0838)
Clearly that element doesn't exist, so your unless fails and $argument is returned immediately without continuing the recursion
Update
I would write something like this
#!/usr/bin/perl
use strict;
use warnings;
use 5.10.0;
my %functions = ( foo => \&foo );
sub iterate {
my ($func, $arg, $depth) = #_;
return $arg unless $depth;
map {iterate($func, $_, $depth - 1); } $functions{$func}->($arg);
}
sub foo {
my ($arg) = #_;
map "$arg$_", '', '.', ',';
}
my #results = iterate('foo', 'the', 2);
say "#results";
output
the the. the, the. the.. the., the, the,. the,,

'Not an ARRAY reference' error thrown

I'm writing a Perl script that is meant to deal with an API which returns metrics about a set of URLs that I pull from MySQL then post these metrics back into a different table. Currently this piece of code:
my $content = $response->content;
my $jsontext = json_to_perl($content);
my $adsql = 'INSERT INTO moz (url_id,page_authority,domain_authority,links,MozRank_URL,MozRank_Subdomain,external_equity_links) VALUES (?,?,?,?,?,?,?)';
my $adrs = $db->prepare( $adsql );
my $adsql2 = 'UPDATE url
SET moz_crawl_date = NOW()
where url_id = ?;';
my $adrs2 = $db->prepare( $adsql2 );
my $currentUrlId = 0;
foreach my $row (#$jsontext){
$adrs->execute($url_ids[$currentUrlId], $row->{'fmrp'}, $row->{'upa'}, $row->{'pda'}, $row->{'uid'}, $row->{'umrp'}, $row->{'ueid'});# || &die_clean("Couldn't execute\n$adsql\n".$db->errstr."\n" );
$adrs2->execute($url_ids[$currentUrlId]);
$currentUrlId++;
}
is throwing this error:
Not an ARRAY reference at ./moz2.pl line 124.
this is line 124:
foreach my $row (#$jsontext){
this whole chunk of code is in a while loop. I am actually able to iterate a couple times and fill my MySQL table before the script fails (technically the program works, but I don't want to just leave an error in it).
Anybody have any suggestions?
Perl gave you the correct answer
Not an ARRAY reference: #$jsontext
You are dereferencing $jsontext, which is the result of json_to_perl(string), to an array.
But json_to_perl() didn't return an arrayref.
json_to_perl seems to be from this API: http://search.cpan.org/~bkb/JSON-Parse-0.31/lib/JSON/Parse.pod#json_to_perl
which returns according to the doc either an arrayref or a hashref.
Apparently it did return a hashref in your case, so you have to add the logic to deal with the HASH case. Which seems to be a single row.
if (ref $jsontext eq 'HASH') {
# seems to be a single row
$adrs->execute($url_ids[$currentUrlId], $jsontext->{'fmrp'}, $jsontext->'upa'}, $jsontext->'pda'}, $jsontext->'uid'}, $jsontext->'umrp'}, $jsontext->'ueid'});# || &die_clean("Couldn't execute\n$adsql\n".$db->errstr."\n" );
$adrs2->execute($url_ids[$currentUrlId]);
$currentUrlId++;
} elsif (ref $jsontext eq 'ARRAY') {
foreach my $row (#$jsontext){
$adrs->execute($url_ids[$currentUrlId], $row->{'fmrp'}, $row->{'upa'}, $row->{'pda'}, $row->{'uid'}, $row->{'umrp'}, $row->{'ueid'});# || &die_clean("Couldn't execute\n$adsql\n".$db->errstr."\n" );
$adrs2->execute($url_ids[$currentUrlId]);
$currentUrlId++;
}
}

how to specify the order of attributes of json object in perl?

I'm parsing a table in html file to make it a json file.
I set up list of attribute names. Each time designated element was found i store it in a perl hash with next attibute name in the list(see code below). Then us JSON module to encode the hash. But the order of each attribute in a object was not the order they were inserted.
sub scan_line
{
my($elem) = #_; # HTML::Element
my %result = ();
my #tds = $elem->find("td");
my $index = 0; # of attrnames
foreach my $td (#tds){
$result{$attrnames[$index]} = $td->as_text();
$index++;
}
my $text = $json->encode(\%result);
print TARGET $text;
}
Are there methods in perl that can specify the order of attribute or add attribute manually like "$jobj->add_attr($attr, $value)"?
You can use:
my $text = $json->sort_by(sub { $JSON::PP::a cmp $JSON::PP::b })->encode(\%result);
Of course you can use <=> instead of cmp or any other sorting function. Note that if your JSON version is below 2.0, you have to explicitly create $json as JSON::PP->new (not JSON->new).
Also be aware that this will decrease performance (and not only of the encode call, but of all $json usages, AFAIU).

Problems parsing Reddit's JSON

I'm working on a perl script that parses reddit's JSON using the JSON module.
However I do have the problem of being very new to both perl and json.
I managed to parse the front page and subreddits successfully, but the comments have a different structure and I can't figure out how to access the data I need.
Here's the code that successfully finds the "data" hash for the front page and subreddits:
foreach my $children(#{$json_text->{"data"}->{"children"}}) #For values of children.
{
my $data = $children->{"data"}; #accessing each data hash.
my %phsh = (); #my hash to collect and print.
$phsh{author} = $data->{"author"};#Here I get the "author" value from "data"
*Etc....
This successfully gets what I need from http://www.reddit.com/.json
But when I go to the json of a comment, this one for example, it has a different format and I can't figure out how to parse it. If I try the same thing as before my parser crashes, saying it is not a HASH reference.
So my question is: How do access the "children" in the second JSON? I need to get both the data for the Post and the data for the comments. Can anybody help?
Thanks in advance!
(I know it may be obvious, but I'm running on very little sleep XD)
You need to either look at the JSON data or dump the decoded data to see what form it takes. The comment data, for example is an array at the top level.
Here is some code that prints the body field of all top-level comments. Note that a comment may have an array of replies in its replies field, and each reply may also have replies in turn.
Depending on what you want to do you may need to check whether a reference is to an array or a hash by checking the value returned by the ref operator.
use strict;
use warnings;
binmode STDOUT, ':utf8';
use JSON;
use LWP;
use Data::Dump;
my $ua = LWP::UserAgent->new;
my $resp = $ua->get('http://www.reddit.com/r/funny/comments/wx3n5/caption_win.json');
die $resp->status_line unless $resp->is_success;
my $json = $resp->decoded_content;
my $data = decode_json($json);
die "Error: $data->{error}" if ref $data eq 'HASH' and exists $data->{error};
dd $data->[1]{data}{children}[0];
print "\n\n";
my $children = $data->[1]{data}{children};
print scalar #$children, " comments:\n\n";
for my $child (#$children) {
print $child->{data}{body}, "\n";
}

Is there a neat way to serialize a Perl hash into an HTML querystring

I have a perl script using CGI.
The browser calls it with some parameters.
I want to take those parameters, modify some of them and then send back a redirect with a new querystring representing the modified parameters.
I know that I could do this, like this:
my $cgi = CGI->new()
my %vars = $cgi->Vars
$vars{'modify_me'} .=' more stuff';
my $serialized = join '&', map {$_.'='.$cgi->escapeHTML($vars{$_})} keys %vars;
However, this just feels like it might be missing something. In addition, it doesn't do anything to handle multivalued parameters. Who knows what else it fails to do.
So, is there a module out there that just deals with this problem? I'm not interested in reinventing a wheel that a more talented wright wrought. Right?
The URI module is your friend. It has a query_form method that takes a hash, hashref or arrayref of parameters and generates a query string from it.
It will URL Encode your data for you (and note that you do want it URL Encoded and not HTML Encoded).
So you might have something like:
#!/usr/bin/perl
use strict;
use warnings;
use CGI;
use URI;
my $q = CGI->new;
my #data = map {
my $name = $_;
my #values = $q->param($name);
my $value;
if (scalar #values == 1) {
($value) = #values;
} else {
$value = \#values;
}
if ($name eq "foo") {
$value = "replaced";
}
($name, $value);
} $q->param;
my $uri = URI->new('http://example.com/myAlternative.cgi');
$uri->query_form(\#data);
print $q->redirect(
-uri=> $uri,
-status => 301
);
Have you looked at Data::URIEncode or URI::QueryParam?
Turns out, there's a way to achieve my specific need using just the CGI module. However, the other answers cover a wider need, to serialize an arbitrary hash.
If you want to modify incoming parameters and then create a link to the same script with modified parameters you can do this:
my $params = $cgi->Vars;
$ Modify the values in hash that $params references
my $new_url = $cgi->self_url(); # URL with modified parameters