Perl - Print div by class - html

Question

I need to print a specific div with class productSpecs from a webpage. Here is my code.
use strict;
use LWP::Simple;
use HTML::TreeBuilder::XPath qw();
my $url="http://www.flipkart.com/samsung-b310e-guru-music-2/p/itmdz9am8xehucbx";
my $content = get($url);
my $t = HTML::TreeBuilder::XPath->new;
$t->parse($content);
my $rank = $t->findvalue('//*[#class="productSpecs"]');
print $rank;
But I am not getting the content I want. What is wrong with my code?

Answer 1

Inspecting the HTML code you are trying to parse, the required div node has this declaration:
<div class="productSpecs specSection">
so your code should be:
my $rank = $t->findnodes('//div[#class="productSpecs specSection"]');

Answer 2

Just for comparison I tried this with Mojolicious using the ojo tool (great for oneliners) and it seems Mojo::DOM returns the HTML by default unless you ask for the text with a ->text() method. e.g. this seems to do what you want:
perl -Mojo -E 'g("http://www.flipkart.com/samsung-b310e-guru-music-2/p/itmdz9am8xehucbx")
->dom->find("div.productSpecs")->each(sub{say $_})'
cheers,

Answer 3

Hi user2186465 and welcome to Stack Exchange :-)
When you assign and print the output fromHTML::TreeBuilder::XPath's findnodes->() method it seems to default to parsing/rendering the <div> node and returning the content as text. Along with that it returns an XML::XPathEngine::NodeSet object (which HTML::TreeBuilder::XPath uses) and an array with a reference to an HTML::Tree object that has what you want. You need to assign that array element reference to your $rank variable or else you'll just get the text:
my $rank = $t->findnodes('//div[#class="productSpecs specSection"]')->[0];
(NB: this appears somewhere in the documentation as an example, but it is not prominent). Once you have the HTML::Element object you can use one of its methods with your print statement to get at the contents.
Without the ->[0] you get the rendered text and print $rank just shows that; but with ->[0] you get access to the object and its methods so print $rank->as_HTML can show the raw HTML content from the node (->as_XML works as well). HTML::TreeBuilder::XPath also has a as_XML_indented convenience method to make the output easier to read. So:
use strict;
use LWP::Simple;
use HTML::TreeBuilder::XPath qw();
my $url="http://www.flipkart.com/samsung-b310e-guru-music-2/p/itmdz9am8xehucbx";
my $content = get($url);
my $t = HTML::TreeBuilder::XPath->new;
$t->parse($content);
my $rank = $t->findnodes('//div[#class="productSpecs specSection"]')->[0];
print $rank->as_XML_indented ;
should do what you want.
HTH