Extracting information from web through Excel or VBA - html

Question

I am a beginner of coding. I am now learning how to get information from web through Excel or VBA.
My question can be ask through below example:
In below link (https://www.schooland.hk/ss/tsuen-wan),
when you click those individual blue wordings in the red circle which I shown, it will leads to individual pages.
In all those individual pages, for example (https://www.schooland.hk/ss/twgss), they also have a part like this showing same kind of information, like phone number etc.(As circled below).
My work is to use a spreadsheet, like excel, show a table which listed all those individual wordings' informations in the red circled without using copy and paste.
How could I do it?

Answer 1

The following using XHR to achieve your task.
Notes:
GetSchoolInfo is the main sub.
It sends the initial request to "https://www.schooland.hk/ss/tsuen-wan" .It uses the function GetHTMLDoc to return an HTML document for the webpage URL passed in.
The links to individual schools you show in the first red circled area can be retrieved using a CSS selector of
.school-table a
"." means className, so .school-table means all elements with className school-table.
" a" means all a tags contained within that.
Sample results returned by this CSS query:
The actual HTML returned for each link is like the following:
<A title="Tsuen Wan Government Secondary School" href="about:twgss">??????</A>
We can make use of the fact that each linked to page uses the short string after about: i.e. twggs concatentated with a general base string of "https://www.schooland.hk/ss/", to give each school specific URL i.e.
"https://www.schooland.hk/ss/twggs".
Function GethRefSubString obtains this short string which is concantenated with the general base string BASEURL. This school specific link is the added to the schoolLinks collection.
The schoolLinks collection is looped using GetHTMLDoc to process the links into school specific new HTML documents.
The contact info on each school page, the second red circled area in your question, resides with an HTMLDivElement with className contact. The appropriate index is 0 i.e. the first matching className in the collection (also the only!).
All the required contact info can be accessed via the .innerText property of the HTMLDivElement.
Example webpage content:
Example code output:
VBA Code:
Option Explicit
Public Sub GetSchoolInfo()
Application.ScreenUpdating = False
Dim xmlHttp As Object, html As HTMLDocument, links As Object
Const BASEURL As String = "https://www.schooland.hk/ss/"
Set xmlHttp = CreateObject("MSXML2.XMLHTTP.6.0")
Set html = GetHTMLDoc("https://www.schooland.hk/ss/tsuen-wan", xmlHttp)
Set links = html.querySelectorAll(".school-table a[href]") 'get all
Dim link As Long, schoolLinks As Collection
Set schoolLinks = New Collection
For link = 0 To links.Length - 1
schoolLinks.Add BASEURL & GethRefSubString(links(link).outerHTML)
Next link
Dim currentLink, counter As Long
With ActiveSheet
For Each currentLink In schoolLinks
counter = counter + 1
Set html = GetHTMLDoc(currentLink, xmlHttp)
Dim contactInfo As Object '<HTMLDivElement
Set contactInfo = html.getElementsByClassName("contact")(0)
.Cells(counter, 1) = contactInfo.innerText
Next currentLink
End With
Application.ScreenUpdating = True
End Sub
Public Function GetHTMLDoc(ByVal url As String, ByRef xmlHttp As Object) As HTMLDocument
With xmlHttp
.Open "GET", url, False
.setRequestHeader "Content-Type", "text/xml"
.send
Dim html As HTMLDocument
Set html = New HTMLDocument
html.body.innerHTML = .responseText
End With
Set GetHTMLDoc = html
End Function
Public Function GethRefSubString(ByVal aString As String) As String
GethRefSubString = Split(Split(aString, "href=""about:")(1), Chr$(34))(0)
End Function
References required:
VBE > Tools > References > HTML Object library