Extract link from page HTML with Perl - html

Question

To extract links from HTML pages use the code:
#!/usr/bin/perl
use warnings;
use strict;
use XML::LibXML;
open( SITE, "< index.html" );
my $html = <SITE>; # load the HTML file
my $content = 'XML::LibXML'->load_html(string => "index.html", recover => 1);
my #del = qw( Contact Tables );
my $condition = join ' or ', map "text()='$_'", #del;
for my $anch ($content->findnodes("//a[$condition]/..")) {
$anch->parentNode->removeChild($anch);
}
open (NOTEPAD, ">> index.html");
print NOTEPAD "$content";
close(NOTEPAD);
My problem is use the $file variable to read the contents of HTML page, it not work.
If I use the HTML tags inside the $file variable it works. But not is solution.

Answer 1

You can use WWW::Mechanize; to accomplish your task.
my $mech = WWW::Mechanize->new();
$mech->get( $url ); #url to extract links
my #links = $mech->links();
foreach my $link (#links) {
my $curr_url = $link->url_abs;
}
For complete documentation of this module refer WWW::Mechaniize

Answer 2

You can use Mojo::DOM (part of Mojolicious) which allows you to use CSS selectors, which I feel is a much better approach;
use Mojo::DOM;
use Mojo::File qw( path );
my $dom = Mojo::DOM->new( path('index.html')->slurp );
foreach ( $dom->find('a')->each ) {
# Do something with $_
}
my $html = $dom->to_string;

Answer 3

Your program opens file index.html, reads one line of it into $html, and then tries to parse the string index.html as if it were HTML data
Forget about opening and reading the file; you can get XML::LibXML to do all of that for you, with
my $content = XML::LibXML->load_html( location => 'index.html', recover => 1 );