VBA Excel Run time error 438 / getElementbyClassName - html

I'm a newbie, attempting to web scrape aspect ratio details from the imdb.com website.
I've plundered some code on You Tube and adapted it using inspect element.
The code opens imdb and runs a search by title but returns a Run Time error 438.
Ideally I'd like it to return the html of the top result so I could perform a further click the top result to follow through to the page with tech details from where I could get the aspect ratio information and paste it into a cell.
Unfortunately I get a fail from my Click instruction - haven't even got to the point of extracting the aspect ratio info.
Can anyone see where I've gone wrong?
Many thanks,
Nick
Private Sub Worksheet_Change(ByVal Target As Range)
If Target.Row = Range("Title").Row And Target.Column = Range("Title").Column Then
Dim ie As New InternetExplorer
ie.Visible = True
ie.navigate "https://www.imdb.com/find?ref_=nv_sr_fn&q=" & Range("Title").Value
Do
DoEvents
Loop Until ie.readyState = READYSTATE_COMPLETE
Dim doc As HTMLDocument
Set doc = ie.document
Dim sDD As String
doc.getElementsByTagName("a").Click
End If
End Sub

So, addressing your code
You can use a shorter version of Target.Address = Range("Title").Address
You don't want the first a tag element. You want the first search result a tag element.
You can use a CSS selector combination to get the first search result a tag element as shown below.
I use a CSS selector combination of .result_text a to target elements within parent class result_text with tag a. The . is a class selector.
This combination is known as a descendant selector.
Using search term in sheet of Red October this is what the CSS query first result is:
It is a relative link with base string https://www.imdb.com.
Applying via querySelector method means only first matched result is returned i.e. the top result.
VBA:
Option Explicit
Private Sub Worksheet_Change(ByVal Target As Range)
Application.EnableEvents = False
If Target.Address = Range("Title").Address Then
Dim ie As New InternetExplorer
ie.Visible = True
ie.navigate "https://www.imdb.com/find?ref_=nv_sr_fn&q=" & Range("Title").value
Do
DoEvents
Loop Until ie.readyState = READYSTATE_COMPLETE
Dim doc As HTMLDocument
Set doc = ie.document
doc.querySelector(".result_text a").Click
'other code
End If
Application.EnableEvents = True
End Sub

This line of code:
doc.getElementsByTagName("a")
gives you the Collection of Hyperlinks in your HTML Document. That is, it gives you ALL the elements that match your given criteria, if any are available.
However, some issues may arrive:
There may not be any hyperlinks available - So there are no elements to click on.
You are not referencing any element to click. If you want the first one in the collection of found items, you could go with the index, as suggested. Else, you might look for another clicking criteria (such as what is its text or another given attribute).
Even still, a found element might not be clickable by your browser, if, for example, it is shadowed by another element.

Related

Select HTML Menu Item via VBA

I'm fairly new to HTML, so please bear with me on this. I am using Excel VBA to interact with an website, with the intent to automate this interaction.
Problem Statement
I have a web page with (what looks like) a button that I need to click on. In the HTML it is listed as a Menu Item. Im able to successfully click other buttons on the page, but those have < button > tags.
I have tried to use the click method by selecting it by ID but I get an 'Object Variable or With block variable not set' error.
Sub WD_auto()
Dim IE As New SHDocVw.InternetExplorer
Dim HTMLDoc As MSHTML.HTMLDocument
Dim HTMLSel As MSHTML.IHTMLElement
IE.Visible = True
IE.navigate "https://wd3.myworkday.com/redacted/d/home.htmld"
Do While IE.Busy = True Or IE.readyState <> 4: DoEvents: Loop
Set HTMLDoc = IE.document
Set HTMLSel = HTMLDoc.getElementById("88831e18a0894109a83c10bc9a9be6c7")
HTMLSel.Click
End Sub
The block of HTML that i think i need to interact with is shown below.
<div class="GNMRENADFGC GNMRENADBHC GNMRENADHHC" tabindex="-2"
id="88831e18a0894109a83c10bc9a9be6c7" role="menuitem"
aria-posinset="1" aria-setsize="3">
Any pointers or literature would be appreciated.
Ok after some more research i have a solution. I am just looping all the div tags and checking the Inner Text property of each until I find the one I want to click.
It works, but if anyone has a more elegant solution I'm all ears. The loop seems a bit wasteful, I would prefer to just refer directly to, and then click the element.
Set ElementsA = IE.document.getElementsByTagName("div")
For Each ElementA In ElementsA
On Error Resume Next
If ElementA.innerText = "User Name and Password login" Then
ElementA.Click
Exit For
End If
Next ElementA

How to add elements revealed by click action to elements list in Excel VBA for an html web form

I'm trying to set up an Excel form that auto-fills an HTML web form. I've figured out how to use VBA to get the elements and cycle through them to add values. My issue is with a text field that is revealed with the click of a check box. I can't have Excel check the box until after I've obtained the elements for the form.
I've looked at the html, and it looks like the field is hidden to some degree when the form is first loaded, as the id and everything can only be found in the tree once the box is checked. The problem is, this field won't show up in the VBA elements list no matter what I try. I've tried re-doing the Set command, and gotten an error when I do that. I'm not sure how to refresh the elements list in VBA to include the new input box.
I used the Set command to get all the elements
Set frm = ie.document.getElementByID("form1")
This is fine, but I can't use that same command to try to re-Set the element list. I get the run-time error 438 (Object doesn't support this property or method)
I tried making a Variant titled frm2, but I get the same error
Sub formFill()
Dim ie As Object
Dim frm As Variant
Dim element As Variant
Set ie = CreateObject("InternetExplorer.Application")
ie.navigate "THIS IS THE URL"
While ie.readyState <> 4: DoEvents: Wend
'Get form by ID
Set frm = ie.document.getElementByID("form1")
ie.Visible = True
For Each element In frm.elements
Select Case element.Name
Case "fv_RRFC$chkOtherModel"
element.Checked = True
element.FireEvent ("OnClick")
frm.getElementByID("fv_RRFC_txtOtherModel")(0).Value = "test model" 'I tried using the command here, but it didn't work
Case "fv_RRFC$txtRRFC_PROGRAM"
element.Value = "test"
Case "fv_RRFC$txtOtherModel"
element.Value = "test model"
'My attempt to add it to the Select Case. Not surprised this didn't work, as the for Each loop uses the list it had before
End Select
Next
End Sub
I expected to be able to re-load the elements list to interact and fill the newly revealed box, but I've had no luck finding a way to do that.

How to click on a <a> element of an <li> of a <ul> using vba?

Here is a piece of code of the HTML page I am working on.
I am trying to programmatically click on the element highlighted in blue in the picture above.
Here is the code I wrote:
Set objIE = New SHDocVw.InternetExplorer
Do While objIE.Busy: DoEvents: Loop
Do Until objIE.readyState = READYSTATE_COMPLETE: DoEvents: Loop
Set IeDoc2 = objIE.Document
Set the_input_elements2 = IeDoc2.getElementsByClassName("parent-item")
For Each input_element2 In the_input_elements2
If input_element2.href = "javascript:directToSearch()" Then
input_element2.Click
Exit For
End If
Next input_element2
How do I click on an element of a ul in HTML using VBA?
When I run this code, nothing happens. No errors but no result.
Here's a couple of ways to refer to the element you're interested in:
'Targeted (if you know exactly at which order the elements appear in the list)
Dim li As HTMLListElement
Set li = IeDoc2.getElementById("accordion-leftmenu") 'get the list item you're interested in...
Debug.Print li.Children(1).innerText '...and access its sub-element in a targeted way
Debug.Print li.getElementsByTagName("a")(1).innerText '...or find all <a></a> elements contained in it and access the one you're interested in, in a targeted way
'Non targeted (you have to search for it)
Dim element As HTMLObjectElement
For Each element In IeDoc2.getElementsByTagName("a") 'loop through all <a></a> elements...
If element.innerText = "Advanced Search" Then '...and find the one you're interested in
Debug.Print element.innerText
End If
Next element
For demonstration purposes the above code just prints the inner text of the element. You can modify it accordingly.
References used: Microsoft HTML Object Library
Also on a side note, please post the actual HTML next time rather than a screenshot of it. It would make it easier for people to help you.
Loop all "a" tags inside the ID of the left menu
IeDoc2.getElementById("accordion-leftmenu").getElementsByTagName("a")
in the loop check if the .innerText is "Advanced Search". click if it is.
Dim IeDoc2 As MSHTML.HTMLDocument
Dim the_input_elements2 As MSHTML.IHTMLElementCollection
Dim input_element2 As MSHTML.IHTMLElement
...
Set the_input_elements2 = IeDoc2.getElementById("accordion-leftmenu").getElementsByTagName("a")
For Each input_element2 In the_input_elements2
If input_element2.innerText = "Advanced Search" Then
input_element2.Click
Exit For
End If
Next input_element2
Edit: Alright, I've tested the code and it works. I did however have to make sure that Two references were checked: "Microsoft HTML Object Library" and "Microsoft Internet Controls".

Web scraping by link text

I have some experience and knowledge how to scrape by tagName or ClassName. However in this particular case className is not unique also link is changing all the time after accessing the page so it is not possible to get a direct link. The only unique combination is class and link text. What would be the code to access for example Budget and Forecast updating with a_1_610 and Budget and Forecast updating with a_1_611?
My code (edited according to QHarr answer):
Sub GoToLiinosBot()
'This will load a webpage in IE
Dim ie As InternetExplorer
Dim HWNDSrc As Long
Dim elements As Object
Set ie = Nothing
Set ie = New InternetExplorerMedium
ie.Visible = True
ie.Navigate "http://link.com"
With ie
Do
DoEvents
Loop Until ie.ReadyState = READYSTATE_COMPLETE
End With
Application.Wait (Now + TimeValue("0:00:04"))
ie.Document.querySelector(".data .a_1_611").innerText
'Unload IE
Set ie = Nothing
End Sub
Here is source code:
They are class names not ids. A loop is perhaps required, with test of innerText value of node, if ordering changes but otherwise you want the first match for the example shown in image
.data .a_1_611
Which is
ie.document.querySelector(".data .a_1_611").click
nth-of-type is useful for fixed position selection but more expensive than class selectors.

using MS Excel VBA to extracting data from complex HTML/JS

Short introduction, i consider myself as a intermediate VBA coder without any significant HTML experience. I would like to extract data from a HTML/JS webpage using MS Excel VBA. I have spent couple of hours testing my code on various pages as well as looking for training materials and various forums and Q&A pages.
I am desperately asking for you help. (Office 2013, IE 11.0.96)
The goal is to get the FX rate of a certain bloomberg webpage. The long term goal is to run a macro on various exchange rates and get the daily rate out of the system to an excel table per working day, but i will be handle that part.
I would be happy either with
(1)the current rate (span class="priceText__1853e8a5") or
(2) previous closing (section class="dataBox opreviousclosingpriceonetradingdayago numeric") or
(3) opening rate (section class="dataBox openprice numeric").
My issue is that I cannot fetch the part of the html code where the rate is.
Dim IE As Object
Dim div As Object, holdingsClass As Object, botoes As Object
Dim html As HTMLDocument
Set IE = CreateObject("InternetExplorer.Application")
With IE
.Visible = False
.Navigate "https://www.bloomberg.com/quote/EURHKD:CUR"
Do Until .ReadyState = 4: DoEvents: Loop
End With
Set html = IE.document
Set div = IE.document.getElementById("leaderboard") 'works just fine, populates the objects
Set holdingsClass = IE.document.getElementsByclass("dataBox opreviousclosingpriceonetradingdayago numeric") 'i am not sure is it a class element at all
Set botoes = IE.document.getElementsByTagName("dataBox openprice numeric") 'i am not sure is it a tag name at all
Range("a1").Value = div.textContent 'example how i would place it by using .textContent
Range("A2").Value = holdingsClass.textContent
Range("A3").Value = botoes.textContent
Much appreciate your help!
Instead of digging through html why not use Bloomberg API to request the specific rate?
Likely would be faster and would save you a lot of time in the future doing the same kind of thing.
Please see my similiar project where I create a macro to pull historical FX rates from the European central bank.
https://github.com/dmegaffi/VBA-GET-Requests/blob/master/FX%20-%20GET.xlsm
If you right-click the webpage element you want in chrome and select inspect, it'll bring up the details of that element. You can also press f12 to bring up the HTML of any page. This also works in other browsers.
Is this the element you're looking for?
screen shot of mentioned webpage
Based on your code above, you could reference this element with IE.document.getElementsByclass("priceText__1853e8a5"). Elements in HTML can share classes but can't share ID's, so if there is another element with the class priceText__1853e8a5 it won't work since it won't select a single element. Then, of course, you have to select the text within the element since at this point you'd just have the and would need the text inside of it.
Hope this helps.
To address your questions generally, see below.
(1)the current rate (span class="priceText__1853e8a5")
That can be written as a CSS query selector of:
span.priceText__1853e8a5
(2) previous closing (section class="dataBox
opreviousclosingpriceonetradingdayago numeric")
That can be written as a CSS query selector of:
.dataBox.opreviousclosingpriceonetradingdayago.numeric
(3) opening rate (section class="dataBox openprice numeric")
That can be written as a CSS query selector of:
.dataBox.openprice.numeric
They are applied with querySelector or querySelectorAll (if more than one match and a later match than the first is required) of HTMLDocument.
E.g.
Debug.Print IE.document.querySelector("span.priceText__1853e8a5").innerText
If more using querySelectorAll
IE.document.querySelectorAll("span.priceText__1853e8a5")(0).innerText
In the above you replace 0 with the appropriate index where your target element is found.
Observing the page the actual selectors appear to be as follows but I think this website is probably using ecmascript syntax that is not supported on legacy browsers i.e. Internet Explorer or is attempting blocked cross domain requests.
Option Explicit
Public Sub GetInfo()
Dim IE As New InternetExplorer
With IE
.Visible = True
.navigate "https://www.bloomberg.com/quote/EURHKD:CUR"
While .Busy Or .readyState < 4: DoEvents: Wend
With .document
Debug.Print "Current: " & .querySelector(".priceText__1853e8a5").innerText
Debug.Print "Prev close: " & .querySelector(".value__b93f12ea").innerText
Debug.Print "Open: " & .querySelector(".value__b93f12ea").innerText
End With
.Quit
End With
End Sub
Using Selenium Basic and Chrome the page renders fine:
Option Explicit
Public Sub GetInfo()
Dim d As WebDriver
Set d = New ChromeDriver
Const URL = "https://www.bloomberg.com/quote/EURHKD:CUR"
With d
.Start "Chrome"
.get URL
Debug.Print "Current: " & .FindElementByCss(".priceText__1853e8a5").Text
Debug.Print "Prev close: " & .FindElementByCss(".value__b93f12ea").Text
Debug.Print "Open: " & .FindElementByCss(".value__b93f12ea").Text
.Quit
End With
End Sub