I'm looking for help understanding why json_decode returns a scalar instead of a hash. I'm still learning perl and a description or some reference lit. would be great.
So the questions:
Why does json_decode return a scalar? (or is it not a scalar)
Is there a better way for me to work with the data?
Here is my code:
use strict;
use warnings;
use JSON qw(decode_json);
use LWP::UserAgent;
my $url = "http://api.bf4stats.com/api/playerInfo?plat=xbox&name=Ttylz00&output=json";
my $ua = LWP::UserAgent->new;
my $data = $ua->get($url);
my $json;
if($data->is_success){
$json = decode_json($data->decoded_content);
}
&sData($json);
sub sData {
my $data = $_[0];
my $kdr = int($data->{stats}->{extra}->{kdr}*100)/100;
printf "\nName: %s\nRank: %s, %s\nKDR: %s\n", $data->{player}->{name},
$data->{player}->{rank}->{nr}, $data->{player}->{rank}->{name},
$kdr;
}
In Perl, a function can only really return a scalar or a list.
Since hashes can be initialized or assigned from lists (e.g. %foo = (a => 1, b => 2)), I guess you're asking why json_decode returns something like { a => 1, b => 2 } (a reference to an anonymous hash) rather than (a => 1, b => 2) (a list that can be copied into a hash).
I can think of a few good reasons for this:
in Perl, an array or hash always contains scalars. So in something like { "a": { "b": 3 } }, the { "b": 3 } part has to be a scalar; and for consistency, it makes sense for the whole thing to be a scalar in the same way.
if the hash is quite large (many keys at top-level), it's pointless and expensive to iterate over all the elements to convert it into a list, and then build a new hash from that list.
in JSON, the top-level element can be either an object (= Perl hash) or an array (= Perl array). If json_decode returned a list in the former case, it's not clear what it would return in the latter case. After decoding the JSON string, how could you examine the result to know what to do with it? (And it wouldn't be safe to write %foo = json_decode(...) unless you already knew that you had a hash.) So json_decode's behavior works better for any general-purpose library code that has to use it without already knowing very much about the data it's working with.
Related
I'm trying to create a loop to read, for example, 4200 users from 1000 to 1000 but I can't get it to cut when it reaches the end. I tried it with if, for and I couldn't do it.
I have programmed in JAVA but with Groovy I see that the structure is different.
urlUsers = urlUsers.concat("/1/1000");
List<UserConnectorObject> usersList = null;
while({
gesdenResponse = GesdenUtils.sendHttpRequest(urlUsers, "LOOKUP", null,
request.getMetaData()?.getLogin(), request.getMetaData()?.getPassword());
log.info("Users data in JSON: "+gesdenResponse.getOutput())
usersList = GesdenUtils.fromJSON(gesdenResponse.getOutput(), GesdenConstants.USER_IDENTITY_KEY);
usersList.size() == 10;
log.info("List size in JSON "+usersList.size());
}()) continue
Groovy has lots of loop structures, but it is crucial to separate the regular ones (lang built-ins) and the api functions which take closure as an argument
take closure - no plain way to escape
If you want to iterate from A to B users, you can use, for instance,
(10..20).each { userNo -> // Here you will have all 10 iterations
if ( userNo == 5) {
return
}
}
If something outrageous happens in the loop body and you cannot use return to escape, as loop boddy is a closure (separate function) and this resurn just exits this closure. Next iteration will happen just after.
use regular lang built-in loop structures - make use of break/continue
for (int userNo in 1..10) { // Here you will have only 5 iterations
if (userNo == 5) {
break
}
}
It looks like your closure always return falsy because there is no explicit return, and the last statement evaluated is the call to log.info(String) which returns void.
Use an explicit return or move/delete the log statement.
I can't get this Perl code to return true integer values for integers in the table. The MySQL table columns are correctly specified as integers, yet the JSON output here wraps all query values in quotes. How can I correctly preserve data-types (esp. integers and boolean values) as specified?
use strict;
use warnings;
use DBI;
use JSON;
my $sth = "SELECT id, name, age FROM table";
my $data = $dbh->selectall_arrayref($sth, {Slice => {}});
my $response = encode_json($data);
print $response;
## outputs: {"id":"1","name":"Joe Blodge","age":"42"}
What am I doing wrong here? How can I get this to output the correctly formatted JSON:
{"id":1,"name":"Joe Blodge","age":42}
DBD::mysql returns all results as strings (see https://github.com/perl5-dbi/DBD-mysql/issues/253). Normally Perl doesn't care, encoding to JSON is one of the few times when it matters. You can either use Cpanel::JSON::XS::Type to provide type declarations for your JSON structure:
use Cpanel::JSON::XS;
use Cpanel::JSON::XS::Type;
my $response = encode_json($data, {id => JSON_TYPE_INT, name => JSON_TYPE_STRING, age => JSON_TYPE_INT});
or you can go through and numify the appropriate elements before JSON encoding.
$data->{$_} += 0 for qw(id age);
It is possible to check the type (as indicated by MySQL) of each returned column, if you construct and execute your query using a statement handle then the type will be available as an array in $sth->{TYPE}, but this is pretty complex and may not be reliable.
I'm trying to write a custom validateParameter function with Perl.
I have the following code which also works:
sub validateParameter {
my ($args, $list) = #_;
if ( ref($list) eq "ARRAY" ) {
foreach my $key (#$list) {
if ( not defined $args->{$key} ) {
die "no $key given!";
}
}
}
#elsif ( check if string ) {
#}
}
I want to call my function the following way:
validateParameter({ hallo => "Welt", test => "Blup"}, ["hallo", "test"]);
But I also want to call my function like this:
validateParameter({ hallo => "Welt", test => "Blup"}, "hallo");
I know that Perl only has the following three data-types (scalars, hashes, arrays). But maybe there is a smart way to check if a variable is a string.
How can I check if the given arg is a string?
Update: I somehow missed the end of the question. Just testing ref($list) eq 'ARRAY' will work most of the time, but to properly allow even overloaded objects, you should just try dereferencing the parameter:
if ( eval { \#$list } ) {
# it was an array
}
else {
# assume it is a string
}
Original answer:
You can check a number of things about a parameter:
if ( ! defined $param ) {
# undefined
}
elsif ( defined Scalar::Util::blessed($param) ) {
# object
}
elsif ( ref $param ) {
# reference (return of ref will give the type)
}
elsif ( length do { no warnings "numeric"; $param & '' } ) {
# number
}
else {
# string
}
But all of that (except perhaps the defined check) kind of defeats the purpose of Perl's automatically converting to your desired type and will limit what can be passed (for instance, a string or dualvar where a number is wanted, or an overloaded object where a string, number, or reference is wanted, or tied variables or magic variables such as $!).
You may want to also just look at what Params::Validate can do.
Don't base behaviour on the "type" of arguments because there really isn't such a thing. You will run into problems if you use type-base polymorphism because Perl values often have more than one type.
For example,
The scalar produced by "123" is stored as as string, but Perl doesn't distinguish it from the scalar produced by 123 which isn't stored as a string.
Scalars can contain both a number and a cached stringification of that number. (e.g. my $i = 123; "".$i;)
Scalars can contain both a number and a string (that isn't a stringification of the number). Common examples of these are $! and !1.
A reference to an object that overloads stringification is also a useful string. (e.g. DateTime->now)
A reference to an array may overload %{} to usable as a hash reference. A reference to an hash may overload #{} to usable as an array reference.
And more.
No, there is no way to check if a scalar is a string, as Perl does implicit type conversions depending on context. If you give a number as the second argument to your function, and you use it in a context that requires a string, it will be automatically converted to a string. So, just check if ref($list) is empty - in such case, $list is not a reference, and therefore it is a string or a number, and you don't need to distinguish between those two.
I have an array
my #cols = ("accountid", "balance");
and a dataset
my $rowsref=$dbh->selectall_arrayref($_[0]);
foreach my $row (#$rowsref) {
print join(", ", map {defined $_ ? $_ : "(null)"} #$row), "\n";
}
which prints "1, 150".
I would like to get a JSON output like [{"accountid": 1, "balance": 150},{..}].
I have the JSON module loaded, but unsure how to merge #cols with each $row.
edit: added explanation of column name transaction in 2nd example
edit: Fixed for your requirement of a single JSON encoding of the whole resultset.
edit: forgot keys in cols mapping code in 2nd example.
edit: typo'd hashref everywhere to arrayref. :-
Firstly, use selectall_hashref instead, so that it already contains the key names.
Then use one of the JSON encoding modules to encode each row.
(making the same assumptions as your code....)
Using the list-of-hashrefs from selectall_hashref() as-is:
use JSON::XS;
use 5.10.0;
my $rowsref = $dbh->selectall_hashref($_[0]);
print JSON::XS::encode_json($rowsref),"\n";
Performing translation on colnames from selectall_hashref():
If the column names from the database aren't the same as your column names, then you'll need a mapping:
use JSON::XS;
use 5.10.0;
my $trans = { account => 'accountid', amount => 'balance' };
my $rowsref = $dbh->selectall_hashref($_[0]);
my $output = [];
for my $row (#$rowsref) {
push #$output, {
map {
my $colname = exists($trans->{$_}) ? $trans->{$_} : $_;
my $value = $row->{$_};
$colname => $value;
} keys %$row
});
}
print JSON::XS::encode_json($output),"\n";
For each $row above of the resultset, keys %$row gives back the column names in the row as returned from the database.
The map operation takes each of those column names and producues 2 scalar values; (1) $colname is either the original database column name or (if it's found in the $trans hashref) a 'translation' of the column name; (2) $value is the value returned by the database for this column name in this particular $row. $colname => $value returns both the $colname and $value from the map as a 'flattened' pair of scalar values. That means that the map operation returns a list of scalar values twice as long as the original list of column names returned by keys %$row.
Finally, the push #$output, { ... } creates an anonymous hash reference from that list of scalar values in key,value,key,value,... order and adds it to the end of the $output array reference.
Blind translation from selectall_arrayref()
If (for some reason) you have a pathological aversion to querying hashrefs from your database, I guess you could do:
use 5.10.0;
my #cols = ("accountid", "balance");
my $rowsref = $dbh->selectall_arrayref($_[0]);
my $output = [];
for my $row (#$rowsref) {
my %row = map { $cols[$_] => $row->[$_] } (0 .. $#cols);
push #$output, \%row;
}
print JSON::XS::encode_json($output),"\n";
That assumes there are only two columns coming back from the query, though.
"Defined or" operator:
By the way... assuming a late enough perl, you can replace this sort of thing:
defined $_ ? $_ : "(null)"
with this:
$_ // "(null)"
Your code editor (e.g: Vim) might not syntax highlight it correctly if it's not up to date with perl. (e.g: it might treat it as an m// construct).
Note that PostgreSQL can also generate JSON. If it is an option for you then Perl JSON module is redundant.
I'm working on a perl script that parses reddit's JSON using the JSON module.
However I do have the problem of being very new to both perl and json.
I managed to parse the front page and subreddits successfully, but the comments have a different structure and I can't figure out how to access the data I need.
Here's the code that successfully finds the "data" hash for the front page and subreddits:
foreach my $children(#{$json_text->{"data"}->{"children"}}) #For values of children.
{
my $data = $children->{"data"}; #accessing each data hash.
my %phsh = (); #my hash to collect and print.
$phsh{author} = $data->{"author"};#Here I get the "author" value from "data"
*Etc....
This successfully gets what I need from http://www.reddit.com/.json
But when I go to the json of a comment, this one for example, it has a different format and I can't figure out how to parse it. If I try the same thing as before my parser crashes, saying it is not a HASH reference.
So my question is: How do access the "children" in the second JSON? I need to get both the data for the Post and the data for the comments. Can anybody help?
Thanks in advance!
(I know it may be obvious, but I'm running on very little sleep XD)
You need to either look at the JSON data or dump the decoded data to see what form it takes. The comment data, for example is an array at the top level.
Here is some code that prints the body field of all top-level comments. Note that a comment may have an array of replies in its replies field, and each reply may also have replies in turn.
Depending on what you want to do you may need to check whether a reference is to an array or a hash by checking the value returned by the ref operator.
use strict;
use warnings;
binmode STDOUT, ':utf8';
use JSON;
use LWP;
use Data::Dump;
my $ua = LWP::UserAgent->new;
my $resp = $ua->get('http://www.reddit.com/r/funny/comments/wx3n5/caption_win.json');
die $resp->status_line unless $resp->is_success;
my $json = $resp->decoded_content;
my $data = decode_json($json);
die "Error: $data->{error}" if ref $data eq 'HASH' and exists $data->{error};
dd $data->[1]{data}{children}[0];
print "\n\n";
my $children = $data->[1]{data}{children};
print scalar #$children, " comments:\n\n";
for my $child (#$children) {
print $child->{data}{body}, "\n";
}