Parse an HTML table with Nokogiri in Ruby - html

Question

I have an HTML table that looks like the following:
<table id="TTdata" border="0" cellspacing="0" cellpadding="3" align="center">
<tbody>
<tr class="TTdata_ltblue">
<td class="ctr"><b>#</b></td>
<td class="ctr"><b>YEAR<img src="/images/up.gif"></b></td>
<td class="ctr" title="Player's name."><b>NAME</b></td>
<td class="ctr" title="how many pitches a catcher had a chance/need to frame"><b>FR_CHANCES</b></td>
<td class="ctr" title="the number of strikes the catcher is expected to have received according to RPM"><b>PREDICTED_STRIKES</b></td>
<td class="ctr" title="the number of strikes the catcher actually received"><b>ACTUAL_STRIKES</b></td>
<td class="ctr" title="the difference between actual and predicted strikes received by the catcher"><b>EXTRA_STRIKES</b></td>
<td class="ctr" title="runs RPM credits to the catcher, using the ball-strike context to calculated run value"><b>FR_RUNS_ADDED_BY_COUNT<img src="/images/down.gif"></b></td>
<td class="ctr" title="how many runs RPM would assign using a generic .14 runs available per frame"><b>FR_RUNS_ADDED_BY_CALL</b></td>
<td class="ctr" title="pitches the catcher received that could have resulted in a wild pitch or passed ball; this is when runners are on base or a dropped third strike is possible"><b>BL_CHANCES</b></td>
<td class="ctr"><b>PREDICTED_PBWP</b></td>
<td class="ctr" title="the run value accumulated from preventing wild pitches and passed balls (.28 per PB/WP saved)"><b>BL_RUNS_ADDED</b></td>
<td class="ctr" title="the number of passed balls and wild pitches allowed by the catcher"><b>ACTUAL_PBWP</b></td>
<td class="ctr" title="the difference between actual and predicted passed balls and wild pitches allowed by the catcher
"><b>PBWP_SAVED</b></td>
</tr>
<tr class="TTdata">
<td>1.</td>
<td class="right">2015</td>
<td>Yasmani Grandal</td>
<td class="right">2295</td>
<td class="right">871.5</td>
<td class="right">925</td>
<td class="right">53.5</td>
<td class="right">8.0</td>
<td class="right">8.0</td>
<td class="right">1097</td>
<td class="right">18.0</td>
<td class="right">0.0</td>
<td class="right">18</td>
<td class="right">0.0</td>
</tr>
<tr class="TTdata_ltgrey">
<td>2.</td>
<td class="right">2015</td>
<td>Buster Posey</td>
<td class="right">2601</td>
<td class="right">1,011.4</td>
<td class="right">1,056</td>
<td class="right">44.6</td>
<td class="right">6.6</td>
<td class="right">6.6</td>
<td class="right">1232</td>
<td class="right">10.0</td>
<td class="right">0.0</td>
<td class="right">10</td>
<td class="right">0.0</td>
</tr>
<tr class="TTdata">
<td>3.</td>
<td class="right">2015</td>
<td>Francisco Cervelli</td>
<td class="right">2629</td>
<td class="right">989.0</td>
<td class="right">1,033</td>
<td class="right">44.0</td>
<td class="right">6.5</td>
<td class="right">6.5</td>
<td class="right">1357</td>
<td class="right">14.0</td>
<td class="right">0.0</td>
<td class="right">14</td>
<td class="right">0.0</td>
</tr>
<tr class="TTdata_ltgrey">
<td>4.</td>
<td class="right">2015</td>
<td>Mike Zunino</td>
<td class="right">2828</td>
<td class="right">1,128.8</td>
<td class="right">1,169</td>
<td class="right">40.2</td>
<td class="right">6.0</td>
<td class="right">6.0</td>
<td class="right">1325</td>
<td class="right">19.0</td>
<td class="right">0.0</td>
<td class="right">19</td>
<td class="right">0.0</td>
</tr>
<tr class="TTdata">
<td>5.</td>
<td class="right">2015</td>
<td>Caleb Joseph</td>
<td class="right">2713</td>
<td class="right">993.9</td>
<td class="right">1,031</td>
<td class="right">37.1</td>
<td class="right">5.5</td>
<td class="right">5.5</td>
<td class="right">1315</td>
<td class="right">9.0</td>
<td class="right">0.0</td>
<td class="right">9</td>
<td class="right">0.0</td>
</tr>
<tr class="TTdata_ltgrey">
<td>6.</td>
<td class="right">2015</td>
<td>Chris Iannetta</td>
<td class="right">2158</td>
<td class="right">847.5</td>
<td class="right">884</td>
<td class="right">36.5</td>
<td class="right">5.4</td>
<td class="right">5.4</td>
<td class="right">1078</td>
<td class="right">15.0</td>
<td class="right">0.0</td>
<td class="right">15</td>
<td class="right">0.0</td>
</tr>
<tr class="TTdata">
<td>7.</td>
<td class="right">2015</td>
<td>Jason Castro</td>
<td class="right">2679</td>
<td class="right">1,068.9</td>
<td class="right">1,105</td>
<td class="right">36.1</td>
<td class="right">5.4</td>
<td class="right">5.4</td>
<td class="right">1378</td>
<td class="right">18.0</td>
<td class="right">0.0</td>
<td class="right">18</td>
<td class="right">0.0</td>
</tr>
<tr class="TTdata_ltgrey">
<td>8.</td>
<td class="right">2015</td>
<td>Miguel Montero</td>
<td class="right">1977</td>
<td class="right">785.8</td>
<td class="right">820</td>
<td class="right">34.2</td>
<td class="right">5.1</td>
<td class="right">5.1</td>
<td class="right">972</td>
<td class="right">11.0</td>
<td class="right">0.0</td>
<td class="right">11</td>
<td class="right">0.0</td>
</tr>
<tr class="TTdata">
<td>9.</td>
<td class="right">2015</td>
<td>Martin Maldonado</td>
<td class="right">2343</td>
<td class="right">906.0</td>
<td class="right">940</td>
<td class="right">34.0</td>
<td class="right">5.1</td>
<td class="right">5.1</td>
<td class="right">1193</td>
<td class="right">17.0</td>
<td class="right">0.0</td>
<td class="right">17</td>
<td class="right">0.0</td>
</tr>
<tr class="TTdata_ltgrey">
<td>10.</td>
<td class="right">2015</td>
<td>Tyler Flowers</td>
<td class="right">2191</td>
<td class="right">833.4</td>
<td class="right">865</td>
<td class="right">31.6</td>
<td class="right">4.7</td>
<td class="right">4.7</td>
<td class="right">1305</td>
<td class="right">13.0</td>
<td class="right">0.0</td>
<td class="right">13</td>
<td class="right">0.0</td>
</tr>
<tr class="TTdata">
<td>11.</td>
<td class="right">2015</td>
<td>Rene Rivera</td>
<td class="right">2632</td>
<td class="right">1,043.1</td>
<td class="right">1,070</td>
<td class="right">26.9</td>
<td class="right">4.0</td>
<td class="right">4.0</td>
<td class="right">1331</td>
<td class="right">18.0</td>
<td class="right">0.0</td>
<td class="right">18</td>
<td class="right">0.0</td>
</tr>
<tr class="TTdata_ltgrey">
<td>12.</td>
<td class="right">2015</td>
<td>Russell Martin</td>
<td class="right">2919</td>
<td class="right">1,121.3</td>
<td class="right">1,148</td>
<td class="right">26.7</td>
<td class="right">4.0</td>
<td class="right">4.0</td>
<td class="right">1470</td>
<td class="right">27.0</td>
<td class="right">0.0</td>
<td class="right">27</td>
<td class="right">0.0</td>
</tr>
<tr class="TTdata">
<td>13.</td>
<td class="right">2015</td>
<td>Kevin Plawecki</td>
<td class="right">1826</td>
<td class="right">744.0</td>
<td class="right">770</td>
<td class="right">26.0</td>
<td class="right">3.9</td>
<td class="right">3.9</td>
<td class="right">886</td>
<td class="right">9.0</td>
<td class="right">0.0</td>
<td class="right">9</td>
<td class="right">0.0</td>
</tr>
<tr class="TTdata_ltgrey">
<td>14.</td>
<td class="right">2015</td>
<td>David Ross</td>
<td class="right">941</td>
<td class="right">339.6</td>
<td class="right">361</td>
<td class="right">21.4</td>
<td class="right">3.2</td>
<td class="right">3.2</td>
<td class="right">519</td>
<td class="right">5.0</td>
<td class="right">0.0</td>
<td class="right">5</td>
<td class="right">0.0</td>
</tr>
<tr class="TTdata">
<td>15.</td>
<td class="right">2015</td>
<td>Roberto Perez</td>
<td class="right">1969</td>
<td class="right">776.5</td>
<td class="right">789</td>
<td class="right">12.5</td>
<td class="right">1.9</td>
<td class="right">1.9</td>
<td class="right">1090</td>
<td class="right">12.0</td>
<td class="right">0.0</td>
<td class="right">12</td>
<td class="right">0.0</td>
</tr>
<tr class="TTdata_ltgrey">
<td>16.</td>
<td class="right">2015</td>
<td>Welington Castillo</td>
<td class="right">1047</td>
<td class="right">410.6</td>
<td class="right">420</td>
<td class="right">9.4</td>
<td class="right">1.4</td>
<td class="right">1.4</td>
<td class="right">499</td>
<td class="right">4.0</td>
<td class="right">0.0</td>
<td class="right">4</td>
<td class="right">0.0</td>
</tr>
<tr class="TTdata">
<td>17.</td>
<td class="right">2015</td>
<td>Hank Conger</td>
<td class="right">1000</td>
<td class="right">405.2</td>
<td class="right">414</td>
<td class="right">8.8</td>
<td class="right">1.3</td>
<td class="right">1.3</td>
<td class="right">511</td>
<td class="right">4.0</td>
<td class="right">0.0</td>
<td class="right">4</td>
<td class="right">0.0</td>
</tr>
<tr class="TTdata_ltgrey">
<td>18.</td>
<td class="right">2015</td>
<td>Josh Thole</td>
<td class="right">476</td>
<td class="right">168.8</td>
<td class="right">177</td>
<td class="right">8.2</td>
<td class="right">1.2</td>
<td class="right">1.2</td>
<td class="right">275</td>
<td class="right">4.0</td>
<td class="right">0.0</td>
<td class="right">4</td>
<td class="right">0.0</td>
</tr>
<tr class="TTdata">
<td>19.</td>
<td class="right">2015</td>
<td>Tucker Barnhart</td>
<td class="right">934</td>
<td class="right">351.4</td>
<td class="right">357</td>
<td class="right">5.6</td>
<td class="right">0.8</td>
<td class="right">0.8</td>
<td class="right">410</td>
<td class="right">4.0</td>
<td class="right">0.0</td>
<td class="right">4</td>
<td class="right">0.0</td>
</tr>
</tbody>
</table>
In this case, I'm interested in retrieving every "player" that is in a table row with either the class of TTdata or TTdata_ltgrey. This can be achieved using the following:
html = open(url)
doc = Nokogiri::HTML(html)
doc.css('.TTdata, .TTdata_lgrey').each do |catcher|
# parse here
end
My problem is, none of the td entries have classes associated with them. I just know that TD 1 is a position, TD 2 is a year, TD 3 is a name.
What's the right way to access each td using the iteration above so I can create a model/hash of name/val pairs for each row?

Answer 1

Here is one approach I tried. But yes, you can take it further from here to meet the need you have :
require 'nokogiri'
require 'pp'
doc = Nokogiri::HTML.parse(File.read("#{__dir__}/out1.html"))
data = doc.css('.TTdata, .TTdata_lgrey').map do |tr|
%i(position year name).zip(tr.css("td:nth-child(-n+3)").map(&:text)).to_h
end
pp data
output
[{:position=>"1.", :year=>"2015", :name=>"Yasmani Grandal"},
{:position=>"3.", :year=>"2015", :name=>"Francisco Cervelli"},
{:position=>"5.", :year=>"2015", :name=>"Caleb Joseph"},
{:position=>"7.", :year=>"2015", :name=>"Jason Castro"},
{:position=>"9.", :year=>"2015", :name=>"Martin Maldonado"},
{:position=>"11.", :year=>"2015", :name=>"Rene Rivera"},
{:position=>"13.", :year=>"2015", :name=>"Kevin Plawecki"},
{:position=>"15.", :year=>"2015", :name=>"Roberto Perez"},
{:position=>"17.", :year=>"2015", :name=>"Hank Conger"},
{:position=>"19.", :year=>"2015", :name=>"Tucker Barnhart"}]