HTML sorting with Perl regex - html

Question

I have an HTML file consisting of an HTML table with links to Scientific Papers and Authors and with their year of publishing. The html is sorted from oldest to newest. I need to resort the table by parsing the file and getting a new file with the sorted source code from newest to oldest.
Here is a small perl script that should be doing the job but it produces semi-sorted results
local $/=undef;
open(FILE, "pubTable.html") or die "Couldn't open file: $!";
binmode FILE;
my $html = <FILE>;
open (OUTFILE, ">>sorted.html") || die "Can't oupen output file.\n";
map{print OUTFILE "<tr>$_->[0]</tr>"}
sort{$b->[1] <=> $a->[1]}
map{[$_, m|, +(\d{4}).*</a>|]}
$html =~ m|<tr>(.*?)</tr>|gs;
close (FILE);
close (OUTFILE);
And here is my input file:
link
and what I get as an output:
link
From the output you can see the order is going well but then I get the year 1993 after the year 1992 and not in the beginning of the list.

Answer 1

There was a problem with the regex in the map because of the following lines in the html.
,{UCLA}-Report 982051,Los Angeles,,1989,</td> </tr>
and
Phys.Rev.Lett., <b> 60</b>, 1514, 1988</td> </tr>
Phys. Rev. B, <b> 45</b>, 7115, 1992</td> </tr>
J.Chem.Phys., <b> 96</b>, 2269, 1992</td> </tr>
In the 1989 line the year includes a comma at the end and there's no whitespace in front. Because of that, the script threw a lot of warnings and always put that line in the bottom.
The other three lines have a four-digit number (\d{4}) with something behind it .* (the year). So the sorting used the other numbers (7115, 2269, 1514) to sort and those were mixed up with the years.
You need to adjust the regex accordingly to fix those issues.
Before:
map{[$_, m|, +(\d{4}).*</a>|]}
After:
map{[$_, m|, *(\d{4}),?</a>|]}

Answer 2

And a solution with XML::Twig, which can also be used to process HTML. It's fairly robust: it won't process other tables in the file, it will accommodate typos like the one in the year in the UCLA report...
#!/usr/bin/perl
use strict;
use warnings;
use XML::Twig;
my $IN = 'sort_input.html';
my $OUT = 'sort_output.html';
my $t= XML::Twig->new( twig_handlers => { 'table[#class="pubtable"]' => \&sort_table,
},
pretty_print => 'indented',
)
->parsefile_html( $IN)
->print_to_file( $OUT);
sub sort_table
{ my( $t, $table)= #_;
$table->sort_children( sub { if($_[0]->last_child( 'td')->text=~ m{(\d+)\D*$}s) { $1; } },
type => 'numeric', order => 'reverse'
);
}

Answer 3

Solution with a robust HTML parsing/manipulation library:
use strictures;
use autodie qw(:all);
use Web::Query qw();
my $w = Web::Query->new_from_file('pubTable.html');
$w->find('table')->html(
join q(),
map { $_->[0]->html }
sort { $a->[1] <=> $b->[1] }
#{
$w->find('tr')->map(sub {
my (undef, $row) = #_;
my ($year) = $row->find('.pubname')->text =~ /(\d\d\d\d) ,? \s* \z/msx;
return [$row => $year];
})
}
);
open my $out, '>:encoding(UTF-8)', 'sorted.html';
print {$out} $w->html;
close $out;