Visual Basic - Extracting text from a webpage - html

Question

My program is designed to search Amazon and extract bits of data from the results - such as product name and price.
So far, my program pastes what the user has typed into txt_Search and presses 'Go' on the page. Here is the code so far:
Private Sub btn_Search_Click(sender As Object, e As EventArgs) Handles btn_Search.Click
Dim SearchTerm As String
'Set SearchTerm as text input
SearchTerm = txt_Search.Text
'Identify the search text box
Dim AmazonElementCollection As HtmlElementCollection = web_Browser.Document.GetElementsByTagName("Input")
For Each curElement As HtmlElement In AmazonElementCollection
Dim searchtextbox As String = curElement.GetAttribute("id").ToString
If searchtextbox = "twotabsearchtextbox" Then
'Populate the search text box
curElement.SetAttribute("Value", searchterm)
End If
Next
'BUTTON CLICK: Search button
'<input type="submit" value="Go" class="nav-submit-input" title="Go">
For Each curElement As HtmlElement In AmazonElementCollection
'Find the Go button
If curElement.GetAttribute("value").Equals("Go") Then
'Function to click the button
curElement.InvokeMember("click")
End If
Next
So this all works fine as it does all this on a web browser object called web_Browser. But how do I take the text off the web page for the first result for example and put it into a textbox. I would like the title of the first result to be put into a textbox called txt_Title. Thank you.

Answer 1

You need to add an event handler for the DocumentCompleted event of your web_browser control.
In that event handler, you can use the same method to find your results element as you have done to find the search text box and the search button, that is: loop through the elements of the document and use the properties of the elements to identify the one you want.
This might look something like this:
Add event handler:
If curElement.GetAttribute("value").Equals("Go") Then
'Add the handler
AddHandler web_Browser.DocumentCompleted, AddressOf myDocumentCompleted
'Function to click the button
curElement.InvokeMember("click")
End If
Event handler implementation:
Private Sub myDocumentCompleted(ByVal sender As Object, _
ByVal e As WebBrowserDocumentCompletedEventArgs)
Dim AmazonElementCollection As HtmlElementCollection = web_Browser.Document.GetElementsByTagName("something you need to figure out")
For Each curElement As HtmlElement In AmazonElementCollection
Dim resultText As String = curElement.GetAttribute("id").ToString
If resultText = "something you need to figure out" Then
'Populate the text box
txt_Title.Text = curElement.innerHTML;
End If
Next
End Sub
Note: I haven't run this through a compiler or tested it, so be prepared for some problems, but I hope you get the idea.