Extracting values from hash created by perl JSON::Syck::Load - json

I've got a very simple Perl issue that I can't for the life of me figure out.
I'm consuming JSON formatted data from a REST endpoint in a perl script. The data is shaped like this:
{
"ScriptRunning": true
}
There's some other stuff, but really all I care about is the ScriptRunning tag. I'm consuming this data using JSON::Syck::Load like so:
my $running_scripts = JSON::Syck::Load($output_from_rest_call)
(in my current environment it is not possible to get other libraries for CPAN, so I'm stuck with that). All that is working correctly as far as I can tell, I used Data::Dumper to confirm the data looks good:
$VAR1 = {
'ScriptRunning' => 1 # or '' if false
}
However, I can't figure out how to actually get the value of 'ScriptRunning'. I've done print ref $running_scripts and confirmed that it is a HASH, however when I try to index into the hash I'm not getting anything. I've tried the following:
my $script_is_running = $running_scripts{'ScriptRunning'};
my $script_is_running = $running_scripts{ScriptRunning};
my $keys_in_running_scripts = keys $running_scripts; # keys_in_running_scripts is empty
my $keys_in_running_scripts = keys %running_scripts; # keys_in_running_scripts is empty
Any ideas?

You need to use strict; (and use warnings; while you are at it, maybe use diagnostics; too, when you are really stuck). As a general rule, ALWAYS use strict; and use warnings; because they prevent problematic code from running and give you some much more helpful output.
You should also read perldoc perlreftut, which helps explain what you are dealing with.
Your variable $running_scripts is not a hash, but a "hash reference", which is an important distinction. When you call ref on a real hash, it returns a false value, since it is not a reference.
What you need to do is "dereference" using the arrow operator to get the value.
To get the keys call to work, there's a separate syntax for dereferencing.
my $script_is_running = $running_scripts->{ScriptRunning};
my #keys_in_running_scripts = keys %{$running_scripts};

Related

Json in Perl error in opensips

What is the difference between json and json::PP in Perl?
I meet this error when use Json and Json:PP when writing perl script in opensips
ERROR:core:XS_OpenSIPS__Message_log:
perl warning: Prototype mismatch: sub main::decode_json ($) vs none.
I have problem with these codes:
my %postObject = ("callId" => $callID);
$postObject{'endTime'} = time() . "";
$postObject{'key'} = "12345#qwerty";
my $post_data = encode_json \%postObject;
The "Prototype mismatch" warning typically means that you've defined a sub twice in some way, and the two definitions' prototypes don't match.
Do you have a sub decode_json ($) in your main code somewhere? If you do, I'd suggest removing or renaming it, because it is conflicting with decode_json from one of the JSON modules. If you don't, then you may be getting a second decode_json from another module you are loading, in which case you'd have to track that down, or provide us with a Minimal, Complete, and Verifiable example.
I'd strongly recommend turning on warnings, because then you will additionally get "Subroutine redefined" warnings to help you track the issue down.

Perl: JSON fails if a thread is started

Can someone please tell my why JSON is not working if some thread is started?
use strict;
use warnings;
use JSON;
use threads;
use threads::shared;
sub th { }
threads->create(\&th)->join() if $ARGV[0];
my $json = to_json({ val => "123"}); # WTF?!?
print "$json\n";
Works fine and prints the JSON-string. But pass 1 as an argument to the script to create the thread and to_json will fail with
hash- or arrayref expected (not a simple scalar, use allow_nonref to allow this)
Same effect if I use encode_json insead.
On the manpage of JSON the word thread is not present and I see no reason why a thread should harm an outside string-conversion.
???
JSON(.pm) is just a front end for JSON::PP, JSON::XS or Cpanel::JSON::XS.
You have found a bug in JSON::XS. About this, JSON::XS's documentation says:
(I-)THREADS
This module is not guaranteed to be ithread (or MULTIPLICITY-) safe and there are no plans to change this. Note that perl's builtin so-called theeads/ithreads are officially deprecated and should not be used.
[Note that the last part is incorrect. The official position is actually: Threads are hard, so you should use something else instead. It's highly questionable since the alternatives are arguably just as hard.]
Workaround: Use one of the other backends (directly or via JSON(.pm)).
$ PERL_JSON_BACKEND=JSON::XS 46793885 0
{"val":"123"}
$ PERL_JSON_BACKEND=JSON::XS 46793885 1
hash- or arrayref expected (not a simple scalar, use allow_nonref to allow this) at /home/ikegami/usr/perlbrew/perls/5.26.0t/lib/site_perl/5.26.0/JSON.pm line 170.
$ PERL_JSON_BACKEND=Cpanel::JSON::XS 46793885 1
{"val":"123"}
$ PERL_JSON_BACKEND=JSON::PP 46793885 1
{"val":"123"}
You can control this within the script by adding the following before loading JSON:
BEGIN { $ENV{PERL_JSON_BACKEND} = 'Cpanel::JSON::XS' }
I ran into this as well (trying to use JSON with multi-threaded perl). Without launching a background thread, my code worked fine, but got the same error you are getting when there was a thread launched.
Like you, I didn't find any help online specific to threading with regards to this error text. However, following the allow_nonref error text, I found the following in JSON::XS's documentation:
"OLD" VS. "NEW" JSON (RFC 4627 VS. RFC 7159)
TL;DR: Due to security concerns, JSON::XS will not allow scalar data in JSON >texts by default - you need to create your own JSON::XS object and enable
allow_nonref:
my $json = JSON::XS->new->allow_nonref;
$text = $json->encode ($data);
$data = $json->decode ($text);
In your case, you are trying to call to_json, which internally creates a JSON object and calls encode on it. Unfortunately it doesn't specify allow_nonref in its constructor. So to make your code work, you can do something like this:
use JSON::XS;
my $json_obj = JSON::XS->new->allow_nonref;
my $json = $json_obj->encode({ val => "123"});
print "$json\n";
I came up with this solution before reading the other responses here, so those may be better solutions, but this should at least get you past the issue with minimal changes.
This is definitely to do with JSON and global state.
If you require and import JSON, after the thread invocation, it 'works'.
The warning in the module for JSON::XS includes:
(I-)THREADS ^
This module is not guaranteed to be ithread (or MULTIPLICITY-) safe and there are no plans to change this
The 'workaround' for a not-thread-safe module is to not load it via use (which happens at 'compile' time) and instead require and import (at runtime) after the parallel instances of the program (threads) have been started.
E.g.:
use strict;
use warnings;
use threads;
use threads::shared;
sub th { }
my $th = threads->create( \&th )->join;
## Just inside main thread
##can do this within individual threads instead if desired
require JSON;
JSON->import;
my $json = to_json({ val => "123" }); # WTF?!?
print "\n$json\n";

Nokogiri returns "no method error"

I keep getting the same error in my program. I've written a method that takes some messy HTML and turns it into neater strings. This works fine on its own, however when I run the whole program I get the following error:
kamer.rb:9:in `normalise_instrumentation': undefined method `split' for #<Nokogiri::XML::NodeSet:0x007f92cb93bfb0> (NoMethodError)
I'd be really grateful for any info or advice on why this happens and how to stop it.
The code is here:
require 'nokogiri'
require 'open-uri'
def normalise_instrumentation(instrumentation)
messy_array = instrumentation.split('.')
normal_array = []
messy_array.each do |section|
if section =~ /\A\d+\z/
normal_array << section
end
end
return normal_array
end
doc = Nokogiri::HTML(open('http://www.cs.vu.nl/~rutger/vuko/nl/lijst_van_ooit/complete-solo.html'))
table = doc.css('table[summary=works] tr')
work_value = []
work_hash = {}
table.each do |row|
piece = [row.css('td[1]'), row.css('td[2]'), row.css('td[3]')].map { |r|
r.text.strip!
}
work_value = work_value.push(piece)
work_key = normalise_instrumentation(row.css('td[3]'))
work_hash[work_key] = work_value
end
puts work_hash
The problem is here:
row.css('td[3]')
Here's why:
row.css('td[3]').class
# => Nokogiri::XML::NodeSet < Object
You're creating your piece array which then becomes an array of NodeSets, which is probably not what you want, because text against a NodeSet often returns a weird String of concatenated text from multiple nodes. You're not seeing that happen here because you're searching inside a row (<tr>) but if you were to look one level up, in the <table>, you'd have a cocked gun pointed at your foot.
Passing a NodeSet to your normalise_instrumentation method is a problem because NodeSet doesn't have a split method, which is the error you're seeing.
But, it gets worse before it gets better. css, like search and xpath returns a NodeSet, which is akin to an Array. Passing an array-like critter to the method will still result in confusion, because you really want just the Node found, not a set of Nodes. So I'd probably use:
row.at('td[3]')
which will return only the node.
At this point you probably want the text of that node, something like
row.at('td[3]').text
would make more sense because then the method would receive a String, which does have a split method.
However, it appears there are additional problems, because some of the cells you want don't exist, so you'll get nil values also.
This isn't one of my better answers, because I'm still trying to grok what you're doing. Providing us with a minimal example of the HTML you need to parse, and the output you want to capture, will help us fine-tune your code to get what you want.
I had a similar error (undefined method) for a different reason, in my case it was due to an extra dot (put by mistake) like this:
status = data.css.("status font-large").text
where it was fixed by removing the extra dot after the css as shown below
status = data.css("status font-large").text
I hope this helps someone else

passing variables to perl script from html

I am trying to call a perl script from my HTML page. The way am trying to do is to call the url of the perl script located on the server.
Here is the piece of code:
HTML:
var fname = "Bob";
var url='http://xxx.com:30000/cgi-bin/abc.pl?title=fname';
window.open(url,"_self");
The way am trying to retrieve it in perl as:
Perl:
print "$ARGV[0]\n";
Now, I have 3 questions:
I think this is the correct way to pass the variables but am not able to print the argument in perl.
If i want to pass another variable lname, how do i append it to the url?
My window.open should open the output in the same window, since it uses the parameter _self. Still it doesn't.
Could anybody point out the problems?
Thanks,
Buzz
No #ARGV contains command line arguments and will be empty.
You need the CGI module
use warnings;
use strict;
use CGI;
my $query = CGI->new;
print $query->param( 'title' );
Edit:
Take a look at dan1111's answer on how to generate HTML and display it in the browser.
In addition to what Matteo said, a simple print statement is not enough to send some output to the browser.
Please see a recent answer I wrote giving a sample CGI script with output.
In regard to your other issues:
Variables are appended to a url separated with &:
var url='http://xxx.com:30000/cgi-bin/abc.pl?title=fname&description=blah';
Based on this question, perhaps you should try window.location.href = url; instead (though that doesn't explain why your code isn't working).
There are two different environments that each pass variables two different ways. The command line can pass variables through the #ARGV and the browser can pass variables through #ENV. It doesn't matter what language you use, those are the arrays that you will have to employ.

Perl::Mechanize: running a simple crawler with a loop [multiple queries]

currently ironing out a way to parse the data of a page: http://www.foundationfinder.ch/
i love to do it in Perl: Well - i am just musing which is the best way to do the job.
Guess that i am in front of a nice learning curve. ;) This task will give me some nice Perl lessions. At the moment it goes abit over my head...;-)
So here is a sample-page:
... and as i thought i can find all 790 resultpages within a certain range between Id= 0 and Id= 100000 i thought, that i can go the way with a loop:
http://www.foundationfinder.ch/ShowDetails.php?Id=11233&InterfaceLanguage=&Type=Html
http://www.foundationfinder.ch/ShowDetails.php?Id=927&InterfaceLanguage=1&Type=Html
http://www.foundationfinder.ch/ShowDetails.php?Id=949&InterfaceLanguage=1&Type=Html
http://www.foundationfinder.ch/ShowDetails.php?Id=20011&InterfaceLanguage=1&Type=Html
http://www.foundationfinder.ch/ShowDetails.php?Id=10579&InterfaceLanguage=1&Type=Html
i thought i can go the Perl-Way but i am not very very sure: I was trying to use LWP::UserAgent on the same URLs [see below] with different query arguments, and i am wondering if LWP::UserAgent provides a way for us to loop through the query arguments? I am not sure that LWP::UserAgent has a method for us to do that. Well - i sometimes heard that it is easier to use Mechanize. But is it really easier!?
BTW; But if i am going the PHP way i could do it with Curl - couldnt i!?
Here is my approach: I tried to figure it out. And i digged deeper in the Manpages and Howtos. We can have a loop constructing the URLs and use Curl - repeatedly
As noted above: here we have some resultpages;
http://www.foundationfinder.ch/ShowDetails.php?Id=11233&InterfaceLanguage=&Type=Html
http://www.foundationfinder.ch/ShowDetails.php?Id=927&InterfaceLanguage=1&Type=Html
Alternatively we can add a request_prepare handler that computes and add the query
arguments before we send out the request.
Again: What is aimed: i want to parse the data and afterwards i want to store it in a local MySQL-database
should i define a extern_uid !?
and go like this:
for my $i (0..10000) {
$ua->get('http://www.foundationfinder.ch/ShowDetails.php?Id=', id => 21, extern_uid => $i);
# process reply
}
Well but now i get stuck- i need help - can i do the job like this!?
regards
zero
Dont do it like this. Use HTTP live headers (Firefox Plugin) or eqv. to see what the javasript does behind the scenes while you select what you need from here to get to that page (with the table).
To get the data from the table, use HTML::TableExtract or HTML::TreeBuilder::XPath if you want to use XPath
If you do want to iterate over the queries, just create another var:
my $a = 'http://www.foundationfinder.ch/ShowDetails.php?Id=' . $q . '&InterfaceLanguage=&Type=Html';
and increment $q as you go, make sure the page is valid before trying to load it with get