How can i make the HTML::Clean Module work in this perl script.
#!/usr/bin/env perl
use strict;
use warnings;
require LWP::UserAgent;
open FH, "<", "text.txt";
my $ua = LWP::UserAgent->new;
$ua->timeout(10);
foreach my $line (<FH>) {
my $response = $ua->get($line);
my $h = new HTML::Clean(\$response);
if ($response->is_success) {
print $response->decoded_content;
}
else {
die $response->status_line;
}
}
close FH;
The HTML::Clean constructor method takes an argument which is either a filename or a reference to a string containing HTML. You seem to be passing it an HTTP::Response object. Try passing it the HTML instead.
my $response = $ua->get($line);
my $html = $response->decoded_content;
my $h = HTML::Clean->new(\$html);
Related
I have Perl script which contains variable $env->{'arguments'}, this variable should contain a JSON object and I want to pass that JSON object as argument to my other external script and run it using backticks.
Value of $env->{'arguments'} before escaping:
$VAR1 = '{"text":"This is from module and backslash \\ should work too"}';
Value of $env->{'arguments'} after escaping:
$VAR1 = '"{\\"text\\":\\"This is from module and backslash \\ should work too\\"}"';
Code:
print Dumper($env->{'arguments'});
escapeCharacters(\$env->{'arguments'});
print Dumper($env->{'arguments'});
my $command = './script.pl '.$env->{'arguments'}.'';
my $output = `$command`;
Escape characters function:
sub escapeCharacters
{
#$env->{'arguments'} =~ s/\\/\\\\"/g;
$env->{'arguments'} =~ s/"/\\"/g;
$env->{'arguments'} = '"'.$env->{'arguments'}.'"';
}
I would like to ask you what is correct way and how to parse that JSON string into valid JSON string which I can use as argument for my script.
You're reinventing a wheel.
use String::ShellQuote qw( shell_quote );
my $cmd = shell_quote('./script.pl', $env->{arguments});
my $output = `$cmd`;
Alternatively, there's a number of IPC:: modules you could use instead of qx. For example,
use IPC::System::Simple qw( capturex );
my $output = capturex('./script.pl', $env->{arguments});
Because you have at least one argument, you could also use the following:
my $output = '';
open(my $pipe, '-|', './script.pl', $env->{arguments});
while (<$pipe>) {
$output .= $_;
}
close($pipe);
Note that current directory isn't necessarily the directory that contains the script that executing. If you want to executing script.pl that's in the same directory as the currently executing script, you want the following changes:
Add
use FindBin qw( $RealBin );
and replace
'./script.pl'
with
"$RealBin/script.pl"
Piping it to your second program rather than passing it as an argument seems like it would make more sense (and be a lot safer).
test1.pl
#!/usr/bin/perl
use strict;
use JSON;
use Data::Dumper;
undef $/;
my $data = decode_json(<>);
print Dumper($data);
test2.pl
#!/usr/bin/perl
use strict;
use IPC::Open2;
use JSON;
my %data = ('text' => "this has a \\backslash", 'nums' => [0,1,2]);
my $json = JSON->new->encode(\%data);
my ($chld_out, $chld_in);
print("Executing script\n");
my $pid = open2($chld_out, $chld_in, "./test1.pl");
print $chld_in "$json\n";
close($chld_in);
my $out = do {local $/; <$chld_out>};
waitpid $pid, 0;
print(qq~test1.pl output =($out)~);
I am trying to create a script to parse JSON content but it is not working:
#!/usr/local/bin/perl-w
use DBI;
use Parallel::ForkManager;
use LWP::Simple;
use XML::Simple;
use JSON qw( decode_json );
use Data::Dumper;
use DateTime;
use WWW::Mechanize;
use strict;
use warnings;
use JSON;
my $ua = LWP::UserAgent->new();
my $req = new HTTP::Request GET => 'https://google.com/pub/';
my $res = $ua->request($req);
my $contents = $res->content;
##json response:
#{"success":true,"data":"{\"campaign\":\"21490|\",\"format\":\"md5 \",\"delta_timestamp\":\"1528992718\",\"result\":\"success\",\"download_link\":\"https:\\\/\\\/gmail.net\\\/accesskey\\\/getfile\\\/m-spqn-e61-2aef2575a0b5250354f2b0fda033e703?token=HUSYjdC5jyJskXUHiKn13l1A1BaAjH2R&dcma=e8fae90c472ae146\"}","message":null}
print $contents;
#Check the outcome of the Response
if ( $res->is_success ) {
print $res->content;
}
# Decode the main json object
my $jsn = decode_json($res);
# Since 'data' is another serialized object, you need to decode that as well:
my $data = decode_json($jsn);
# Now you can access the contents of 'data'
#want to extract download_link object
print $data->{'download_url'};
I am looking at the content of download_link.
use JSON qw(decode_json);
# from $res->content
my $content = '{"success":true,"data":"{\"campaign\":\"21490|\",\"format\":\"md5 \",\"delta_timestamp\":\"1528992718\",\"result\":\"success\",\"download_link\":\"https:\\\/\\\/gmail.net\\\/accesskey\\\/getfile\\\/m-spqn-e61-2aef2575a0b5250354f2b0fda033e703?token=HUSYjdC5jyJskXUHiKn13l1A1BaAjH2R&dcma=e8fae90c472ae146\"}","message":null}';
print decode_json(decode_json($content)->{data})->{download_link};
I need to print all "time" values from below JSON in Perl. How do it?
#!/usr/bin/perl -w
use strict;
use JSON;
use Data::Dumper;
$a = q({"Response":"Success","Type":100,"Aggregated":true,"Data":[{"time":1521064980,"close":8130.26,"high":8153.57,"low":8121.42,"open":8152.29,"volumefrom":711.3,"volumeto":5815443.51},{"time":1521065160,"close":8161.97,"high":8167.7,"low":8130.51,"open":8130.51,"volumefrom":527.3699999999999,"volumeto":4348419.180000001}{"time":1521075780,"close":8129.27,"high":8129.71,"low":8126.7,"open":8127.4,"volumefrom":50.35,"volumeto":412670.56000000006}],"TimeTo":1521075900,"TimeFrom":1521064980,"FirstValueInArray":true,"ConversionType":{"type":"direct","conversionSymbol":""}});
$b = decode_json $a;
You can try below code to print values of time in your json:
#!/usr/bin/perl -w
use strict;
use JSON;
use Data::Dumper;
my $json = q({"Response":"Success","Type":100,"Aggregated":true,"Data":[{"time":1521064980,"close":8130.26,"high":8153.57,"low":8121.42,"open":8152.29,"volumefrom":711.3,"volumeto":5815443.51},{"time":1521065160,"close":8161.97,"high":8167.7,"low":8130.51,"open":8130.51,"volumefrom":527.3699999999999,"volumeto":4348419.180000001},{"time":1521075780,"close":8129.27,"high":8129.71,"low":8126.7,"open":8127.4,"volumefrom":50.35,"volumeto":412670.56000000006}],"TimeTo":1521075900,"TimeFrom":1521064980,"FirstValueInArray":true,"ConversionType":{"type":"direct","conversionSymbol":""}});
my $ref = decode_json $json;
# access 'Data' in hashref $ref
my $data = $ref->{'Data'};
# loop through item in array ref $data
for my $item (#{$data}) {
# print value of "time" with a newline
print $item->{'time'} . "\n";
}
# print 'TimeTo' value
print "TimeTo: " . $ref->{'TimeTo'};
My custom code (on Perl) give next wrong JSON, missing comma between blocks:
{
"data": [{
"{#LOGFILEPATH}": "/tmp/QRZ2007.tcserverlogs",
"{#LOGFILE}": "QRZ2007"
} **missing comma** {
"{#LOGFILE}": "ARZ2007",
"{#LOGFILEPATH}": "/tmp/ARZ2007.tcserverlogs"
}]
}
My terrible code:
#!/usr/bin/perl
use strict;
use warnings;
use File::Basename;
use utf8;
use JSON;
binmode STDOUT, ":utf8";
my $dir = $ARGV[0];
my $json = JSON->new->utf8->space_after;
opendir(DIR, $dir) or die $!;
print '{"data": [';
while (my $file = readdir(DIR)) {
next unless (-f "$dir/$file");
next unless ($file =~ m/\.tcserverlogs$/);
my $fullPath = "$dir/$file";
my $filenameshort = basename($file, ".tcserverlogs");
my $data_to_json = {"{#LOGFILEPATH}"=>$fullPath,"{#LOGFILE}"=>$filenameshort};
my $data_to_json = {"{#LOGFILEPATH}"=>$fullPath,"{#LOGFILE}"=>$filenameshort};
print $json->encode($data_to_json);
}
print ']}'."\n";
closedir(DIR);
exit 0;
Dear Team i am not a programmer, please any idea how fix it, thank you!
If you do not print a comma, you will not get a comma.
You are trying to build your own JSON string from pre-encoded pieces of smaller data structures. That will not work unless you tell Perl when to put commas. You could do that, but it's easier to just collect all the data into a Perl data structure that is equivalent to the JSON string you want to produce, and encode the whole thing in one go when you're done.
my $dir = $ARGV[0];
my $json = JSON->new->utf8->space_after;
my #data;
opendir( DIR, $dir ) or die $!;
while ( my $file = readdir(DIR) ) {
next unless ( -f "$dir/$file" );
next unless ( $file =~ m/\.tcserverlogs$/ );
my $fullPath = "$dir/$file";
my $filenameshort = basename( $file, ".tcserverlogs" );
my $data_to_json = { "{#LOGFILEPATH}" => $fullPath, "{#LOGFILE}" => $filenameshort };
push #data, $data_to_json;
}
closedir(DIR);
print $json->encode( { data => \#data } );
Here is my code that I try to open the file to get data and change it to UTF-8, then read each line and store it in variable my $abstract_text and send it back in JSON structure.
my $fh;
if (!open($fh, '<:encoding(UTF-8)',$path))
{
returnApplicationError("Cannot read abstract file: $path ($!)\nERRORCODE|111|\n");
}
printJsonHeader;
my #lines = <$fh>;
my $abstract_text = '';
foreach my $line (#lines)
{
$abstract_text .= $line;
}
my $json = encode_json($abstract_text);
close $fh;
print $json;
By using that code, I get this error;
hash- or arrayref expected (not a simple scalar, use allow_nonref to allow this)
error message also point out that the problem is in this line;
my $json = encode_json($abstract_text);
I want to send the data back as a string (which is in UTF-8). Please help.
I assume you're using either JSON or JSON::XS.
Both allow for non-reference data, but not via the procedural encode_json routine.
You'll need to use the object-oriented approach:
use strict; # obligatory
use warnings; # obligatory
use JSON::XS;
my $encoder = JSON::XS->new();
$encoder->allow_nonref();
print $encoder->encode('Hello, world.');
# => "Hello, world."