Web-Scraping for Table Formate - html

Question

I am trying to copy the data from TD and TR format. I am new in Data scraping so I did not find any results in google search.
I want to copy All Unit Size (example 5*5) in 'A' column, Regular
Price in 'B', Cash Price 'D', First 3 months 50% Off(offers) in 'F' and Reserve in 'G' column.
http://westgateselfstorage.com/index.php?page=estimator
Frankly saying, I'm not able to frame the code so it would be really helpful to me if someone helped me to build the code.

Answer 1

Clipboard
Do you care about layout? You can copy direct via clipboard and have same layout as page
Option Explicit
Public Sub GetTable()
Dim html As HTMLDocument, clipboard As Object, ws As Worksheet
Set ws = ThisWorkbook.Worksheets("Sheet1")
Set clipboard = GetObject("New:{1C3B4210-F441-11CE-B9EA-00AA006B1A69}")
Set html = New HTMLDocument
With CreateObject("MSXML2.XMLHTTP")
.Open "GET", "http://westgateselfstorage.com/index.php?page=estimator", False
.send
html.body.innerHTML = .responseText
End With
clipboard.SetText html.querySelector("#hiderow").outerHTML
clipboard.PutInClipboard
ws.Range("A1").PasteSpecial
End Sub
QuerySelector and surrogate:
If you do care more then we can use our usual method of finding the rows and looping, using a surrogate HTMLDocument variable to house html so we can leverage querySelector at a more granular level given we can't chain. The only difference here being that if we chuck html from tr level into HTMLDocument we need to add the table tags onto the html to ensure querySelectorAll can pick up the tds within the row i.e. the columns.
Now, I haven't split out old price from reduced price in the Cash Price column. If you need that just let me know. For now, I have left both.
Option Explicit
Public Sub GetTable()
Dim html As HTMLDocument, html2 As HTMLDocument, ws As Worksheet, i As Long
Set ws = ThisWorkbook.Worksheets("Sheet1")
Set html = New HTMLDocument
Set html2 = New HTMLDocument
With CreateObject("MSXML2.XMLHTTP")
.Open "GET", "http://westgateselfstorage.com/index.php?page=estimator", False
.send
html.body.innerHTML = .responseText
End With
html.body.innerHTML = html.querySelector("#hiderow").outerHTML
Dim headers(), rows As Object, results(), columns As Object
headers = Array("Size", "Reg price", vbNullString, "Cash price", vbNullString, "Offers", "Reserve")
'grab the rows
Set rows = html.querySelectorAll("tr")
ReDim results(1 To rows.Length, 1 To UBound(headers) + 1)
For i = 1 To rows.Length - 1 'skip headers row
html2.body.innerHTML = "<table>" & rows.item(i).outerHTML & "</table>"
Set columns = html2.querySelectorAll("td")
results(i, 1) = columns.item(0).innerText
results(i, 2) = columns.item(3).innerText
results(i, 4) = columns.item(4).innerText
results(i, 6) = columns.item(5).innerText
results(i, 7) = "Reserve this unit"
Next
ws.Cells(1, 1).Resize(1, UBound(headers) + 1) = headers
ws.Cells(2, 1).Resize(UBound(results, 1), UBound(results, 2)) = results
End Sub
getElement(s)Bymethod and chaining:
We could also be more traditional and loop tr/td within a table as with getElement(s)Bymethod allows us chaining to some extent (though I store in variables to make it easy to compare with the above)
Note:
rows(i).getElementsByTagName("td")
is basically chaining onto an individual element (row) within
Set rows = hTable.getElementsByTagName("tr")
e.g.
hTable.getElementsByTagName("tr")(0).getElementsByTagName("td")
All columns in first row via chaining.
VBA:
Option Explicit
Public Sub GetTable()
Dim html As HTMLDocument, hTable As HTMLTable, ws As Worksheet, i As Long
Set ws = ThisWorkbook.Worksheets("Sheet1")
Set html = New HTMLDocument
With CreateObject("MSXML2.XMLHTTP")
.Open "GET", "http://westgateselfstorage.com/index.php?page=estimator", False
.send
html.body.innerHTML = .responseText
End With
Set hTable = html.getElementById("hiderow")
Dim headers(), rows As Object, results(), columns As Object
headers = Array("Size", "Reg price", vbNullString, "Cash price", vbNullString, "Offers", "Reserve")
'grab the rows
Set rows = hTable.getElementsByTagName("tr")
ReDim results(1 To rows.Length, 1 To UBound(headers) + 1)
For i = 1 To rows.Length - 1 'skip headers row
Set columns = rows(i).getElementsByTagName("td")
results(i, 1) = columns(0).innerText
results(i, 2) = columns(3).innerText
results(i, 4) = columns(4).innerText
results(i, 6) = columns(5).innerText
results(i, 7) = "Reserve this unit"
Next
ws.Cells(1, 1).Resize(1, UBound(headers) + 1) = headers
ws.Cells(2, 1).Resize(UBound(results, 1), UBound(results, 2)) = results
End Sub

Answer 2

You can use the python library pandas for this:
import pandas as pd
tabs = pd.read_html('http://westgateselfstorage.com/index.php?page=estimator', header =0)
tabs[0].drop(tabs[0].columns[2], axis=1)
You may also want to drop column G since all rows there are identical.