I am trying to pull back all the data from my sql statement but I am only able to pull back the first row data in the first put statement. So what I am trying to do is do a for loop through the $data variable. The for loop won't run because it says that the variable is an array.
orasql crsr $sql
orafetch $crsr -datarray data -indexbyname
puts $data(customer)
foreach value $data {
puts $data(customer)
}
The problem is when you do foreach value $data; since the data variable is an array, you can't read it as a simple value. To print the keys and values in the array, do this:
foreach {key value} [array get data] {
puts "$key => $value"
}
The array get command takes the name of an array variable and gets a list of keys and values from it, which can be iterated over with a two-variable foreach. (The parray command does a somewhat more sophisticated version of this; you could use parray data in place of that loop above, but then you'd have no real other route into processing the row.)
You usually ought to know the names of the columns you're getting back from an SQL query so that you don't need to loop over it like that. Also, the order of columns doesn't really matter, but neither does the order of entries in the array. (The order of rows in the result set of the query matters, but orafetch only gets one at a time.)
Related
I have difficulty understanding this while loop:
The Condition
The condition:
while {1 == [string equal $result Completed]} {
could be written more shortly as:
while {$result eq "Completed"} {
That is, it means "do the body while $result is equal to the literal string Completed.
The Body
The body of the loop calls $mode run (what exactly that does isn't described here). Then it gets the result by calling $mode getRunResult and extracts the first word of the list, and assigns it to the variable result. The final step of the loop is to use switch to print a message whenever $result is either Error or SolverFailure (it also has clauses for Completed and StopCritera, but that's empty so nothing happens).
The Overall Effect
The loop calls $mode run until the first word of the result of $mode getRunResult after that run is not Completed.
$mode is a handle returned by pw::Application begin ExtrusionSolver $block, and $mode end is called after the loop terminates, presumably to clean things up.
I can do this all as one function, but in trying to port it over to my packages of functions (library) I am missing something.
Here's what I want to do from my main Perl script
my #rows;
$result = Funx::dbcdata($myConnection,
"SELECT * FROM Inv where name like \"%DOG%\";", \#rows);
Then in my library package I am attempting this
sub dbcdata
{
my ($connection, $command, $array) = #_;
my $query = $connection->prepare($command);
my $result = $query->execute();
my $i =0;
while(my $row = $query->fetchrow_arrayref() )
{
#{$array}[$i] = $row;
$i++;
}
$query->finish;
return $result;
}
I was hoping to get back pointers or references to each row (which was 4in this case) but am not. Every element in #rows is the same:
ARRAY(0x5577a0f77ec0) ARRAY(0x5577a0f77ec0) ARRAY(0x5577a0f77ec0)
ARRAY(0x5577a0f77ec0)
Nor do I know how to turn each one into the original separate row. Any help would be appreciated, thanks.
From the documentation for fetchrow_arrayref:
Note that the same array reference is returned for each fetch, so don't store the reference and then use it after a later fetch. Also, the elements of the array are also reused for each row, so take care if you want to take a reference to an element.
Sounds like you want fetchall_arrayref:
The fetchall_arrayref method can be used to fetch all the data to be returned from a prepared and executed statement handle. It returns a reference to an array that contains one reference per row.
After executing the statement, you can do something like
#{$array} = $query->fetchall_arrayref->#*;
instead of that ugly loop.
But selectall_array might be even better. Your whole function can be replaced by a call to it:
my #rows =
$myConnection->selectall_array(q/SELECT * FROM Inv WHERE name LIKE '%DOG%'/);
I have an array
my #cols = ("accountid", "balance");
and a dataset
my $rowsref=$dbh->selectall_arrayref($_[0]);
foreach my $row (#$rowsref) {
print join(", ", map {defined $_ ? $_ : "(null)"} #$row), "\n";
}
which prints "1, 150".
I would like to get a JSON output like [{"accountid": 1, "balance": 150},{..}].
I have the JSON module loaded, but unsure how to merge #cols with each $row.
edit: added explanation of column name transaction in 2nd example
edit: Fixed for your requirement of a single JSON encoding of the whole resultset.
edit: forgot keys in cols mapping code in 2nd example.
edit: typo'd hashref everywhere to arrayref. :-
Firstly, use selectall_hashref instead, so that it already contains the key names.
Then use one of the JSON encoding modules to encode each row.
(making the same assumptions as your code....)
Using the list-of-hashrefs from selectall_hashref() as-is:
use JSON::XS;
use 5.10.0;
my $rowsref = $dbh->selectall_hashref($_[0]);
print JSON::XS::encode_json($rowsref),"\n";
Performing translation on colnames from selectall_hashref():
If the column names from the database aren't the same as your column names, then you'll need a mapping:
use JSON::XS;
use 5.10.0;
my $trans = { account => 'accountid', amount => 'balance' };
my $rowsref = $dbh->selectall_hashref($_[0]);
my $output = [];
for my $row (#$rowsref) {
push #$output, {
map {
my $colname = exists($trans->{$_}) ? $trans->{$_} : $_;
my $value = $row->{$_};
$colname => $value;
} keys %$row
});
}
print JSON::XS::encode_json($output),"\n";
For each $row above of the resultset, keys %$row gives back the column names in the row as returned from the database.
The map operation takes each of those column names and producues 2 scalar values; (1) $colname is either the original database column name or (if it's found in the $trans hashref) a 'translation' of the column name; (2) $value is the value returned by the database for this column name in this particular $row. $colname => $value returns both the $colname and $value from the map as a 'flattened' pair of scalar values. That means that the map operation returns a list of scalar values twice as long as the original list of column names returned by keys %$row.
Finally, the push #$output, { ... } creates an anonymous hash reference from that list of scalar values in key,value,key,value,... order and adds it to the end of the $output array reference.
Blind translation from selectall_arrayref()
If (for some reason) you have a pathological aversion to querying hashrefs from your database, I guess you could do:
use 5.10.0;
my #cols = ("accountid", "balance");
my $rowsref = $dbh->selectall_arrayref($_[0]);
my $output = [];
for my $row (#$rowsref) {
my %row = map { $cols[$_] => $row->[$_] } (0 .. $#cols);
push #$output, \%row;
}
print JSON::XS::encode_json($output),"\n";
That assumes there are only two columns coming back from the query, though.
"Defined or" operator:
By the way... assuming a late enough perl, you can replace this sort of thing:
defined $_ ? $_ : "(null)"
with this:
$_ // "(null)"
Your code editor (e.g: Vim) might not syntax highlight it correctly if it's not up to date with perl. (e.g: it might treat it as an m// construct).
Note that PostgreSQL can also generate JSON. If it is an option for you then Perl JSON module is redundant.
I need to generate an XML file from database records, and I get the error "out of memory". Here's the script I am using, it's found on Google, but it's not suitable for me, and it's also killing the server's allocated memory. It's a start though.
#!/usr/bin/perl
use warnings;
use strict;
use XML::Simple;
use DBI;
my $dbh = DBI->connect('DBI:mysql:db_name;host=host_address','db_user','db_pass')
or die DBI->errstr;
# Get an array of hashes
my $recs = $dbh->selectall_arrayref('SELECT * FROM my_table',{ Columns => {} });
# Convert to XML where each hash element becomes an XML element
my $xml = XMLout( {record => $recs}, NoAttr => 1 );
print $xml;
$dbh->disconnect;
This script only prints the records, because I tested with a where clause for a single row id.
First of all, I couldn't manage to make it to save the output to a file.xml.
Second, I need somehow to split the "job" in multiple jobs and then put together the XML file all in one piece.
I have no idea how to achieve both.
Constraint: No access to change server settings.
These are problem lines:
my $recs = $dbh->selectall_arrayref('SELECT * FROM my_table',{ Columns => {} });
This reads the whole table into memory, representing every single row as an array of values.
my $xml = XMLout( {record => $recs}, NoAttr => 1 );
This is probably even larger structure, it is a the whole XML string in one go.
The lowest memory-use solution needs to involve loading the table one item at a time, and printing that item out immediately. In DBI, it is possible to make a query so that you fetch one row at a time in a loop.
You will need to play with this before the result looks like your intended output (I haven't tried to match your XML::Simple output - I'm leaving that to you:
print "<records>\n";
my $sth = $dbh->prepare('SELECT * FROM my_table');
$sth->execute;
while ( my $row = $sth->fetchrow_arrayref ) {
# Convert db row to XML row
print XMLout( {row => $row}, NoAttr => 1 ),"\n";
}
print "</records>\n";
Perl can use e.g. open( FILEHANDLE, mode, filename ) to start access to a file and print FILEHANDLE $string to print to it, or you could just call your script and pipe it to a file e.g. perl myscript.pl > table.xml
It's the select * with no contraints that will be killing your memory. Add some constraint to your query ie date or id and use a loop to execute the query and do your output in chunks. That way you won't need to load the whole table in mem before your get started on the output.
I'm working on a perl script that parses reddit's JSON using the JSON module.
However I do have the problem of being very new to both perl and json.
I managed to parse the front page and subreddits successfully, but the comments have a different structure and I can't figure out how to access the data I need.
Here's the code that successfully finds the "data" hash for the front page and subreddits:
foreach my $children(#{$json_text->{"data"}->{"children"}}) #For values of children.
{
my $data = $children->{"data"}; #accessing each data hash.
my %phsh = (); #my hash to collect and print.
$phsh{author} = $data->{"author"};#Here I get the "author" value from "data"
*Etc....
This successfully gets what I need from http://www.reddit.com/.json
But when I go to the json of a comment, this one for example, it has a different format and I can't figure out how to parse it. If I try the same thing as before my parser crashes, saying it is not a HASH reference.
So my question is: How do access the "children" in the second JSON? I need to get both the data for the Post and the data for the comments. Can anybody help?
Thanks in advance!
(I know it may be obvious, but I'm running on very little sleep XD)
You need to either look at the JSON data or dump the decoded data to see what form it takes. The comment data, for example is an array at the top level.
Here is some code that prints the body field of all top-level comments. Note that a comment may have an array of replies in its replies field, and each reply may also have replies in turn.
Depending on what you want to do you may need to check whether a reference is to an array or a hash by checking the value returned by the ref operator.
use strict;
use warnings;
binmode STDOUT, ':utf8';
use JSON;
use LWP;
use Data::Dump;
my $ua = LWP::UserAgent->new;
my $resp = $ua->get('http://www.reddit.com/r/funny/comments/wx3n5/caption_win.json');
die $resp->status_line unless $resp->is_success;
my $json = $resp->decoded_content;
my $data = decode_json($json);
die "Error: $data->{error}" if ref $data eq 'HASH' and exists $data->{error};
dd $data->[1]{data}{children}[0];
print "\n\n";
my $children = $data->[1]{data}{children};
print scalar #$children, " comments:\n\n";
for my $child (#$children) {
print $child->{data}{body}, "\n";
}