Pull value from website (HTML div class) using Excel VBA - html

I'm trying to automate going to a website and pulling the ratings from several apps.
I've figured out how to navigate and login to the page.
How do I pull the element - the number "3.3" in this case - from this specific section into Excel.
Being unfamiliar with HTML in VBA, I got this far following tutorials/other questions.
Rating on website and the code behind it
Sub PullRating()
Dim HTMLDoc As HTMLDocument
Dim ie As InternetExplorer
Dim oHTML_Element As IHTMLElement
Dim sURL As String
On Error GoTo Err_Clear
sURL = "https://www.appannie.com/account/login/xxxxxxxxxx"
Set ie = New InternetExplorer
ie.Silent = True
ie.navigate sURL
ie.Visible = True
Do
'Wait until the Browser is loaded
Loop Until ie.readyState = READYSTATE_COMPLETE
Set HTMLDoc = ie.Document
HTMLDoc.all.Email.Value = "xxxxxxxxx#xxx.com"
HTMLDoc.all.Password.Value = "xxxxx"
For Each oHTML_Element In HTMLDoc.getElementById("login-form")
If oHTML_Element.Type = "submit" Then oHTML_Element.Click: Exit For
Next
Dim rating As Variant
Set rating = HTMLDoc.getElementsByClassName("rating-number ng-binding")
Range("A1").Value = rating
'ie.Refresh 'Refresh if required
Err_Clear:
If Err <> 0 Then
Err.Clear
Resume Next
End If
End Sub

The code below will let you extract text from first element with class name "rating-number ng-binding" in HTML document. By the way GetElementsByClassName is supported since IE 9.0. I use coding compatible also with older versions in my example.
Dim htmlEle1 as IHTMLElement
For Each htmlEle1 in HTMLDoc.getElementsByTagName("div")
If htmlEle1.className = "rating-number ng-binding" then
Range("A1").Value = htmlEle1.InnerText
Exit For
End if
Next htmlEle1

While Ryszards code should do the trick if you want to use the code you have already written then here is the alterations I believe you need to make.
For Each oHTML_Element In HTMLDoc.getElementById("login-form")
If oHTML_Element.Type = "submit" Then oHTML_Element.Click: Exit For
Next
'Need to wait for page to load before collecting the value
Loop Until ie.readyState = READYSTATE_COMPLETE
Dim rating As IHTMLElement
Set rating = HTMLDoc.getElementsByClassName("rating-number ng-binding")
'Need to get the innerhtml of the element
Range("A1").Value = rating.innerhtml

Related

Use VBA to find submenu element and click

I am fairly new to vba and am self taught but have gotten my vba to log in to my website and get to the homepage. Next I need to click on a submenu item but I am struggling to find it. When manually doing this, the submenu works when I hover over the icon and the click on a button called "Comed Reports" below which I believe is the element ID "Report1017".
Below is the html code from the website:
And below is where my code is at this stage:
Sub Login()
Const Url$ = "examplewebsite.com"
Dim HTMLDoc As HTMLDocument
Dim oHTML_Element As IHTMLElement
Dim ie As Object
Set ie = CreateObject("InternetExplorer.Application")
With ie
.Navigate Url
.Visible = True
Do While ie.Busy Or ie.ReadyState < 4
DoEvents
Loop
Set HTMLDoc = ie.Document
Dim Login As Object
Dim Password As Object
Dim LoginButton As Object
For Each oHTML_Element In HTMLDoc.getElementsByTagName("input")
If oHTML_Element.Name Like "txt_1*" Then Set Login = oHTML_Element
If oHTML_Element.Name Like "txt_2*" Then Set Password = oHTML_Element
If oHTML_Element.Name Like "btnLogin*" Then Set LoginButton = oHTML_Element
Next
Login.Value = ""
Password.Value = ""
LoginButton.Click
Do While ie.Busy Or ie.ReadyState < 4
DoEvents
Loop
' Find Submenu Report1017 here
End With
End Sub
If it is always Report1017 you probably just need to run the javascript:
HTMLDoc.parentWindow.execScript "LoadContent('Report1017','ReportList.aspx?ReportParam=1017')", "JavaScript"
Give it a try.

Can we fetch the specific data via using urls in vba

I have 15 different URLs, and I need to fetch price from the particular website in Excel a particular column, can you please help me out. It's my first VBA program and I try but it show my syntax error.
Sub myfile()
Dim IE As New InternetExplorer Dim url As String Dim item As
HTMLHtmlElement Dim Doc As HTMLDocument Dim tagElements As Object
Dim element As Object Dim lastRow Application.ScreenUpdating =
False Application.DisplayAlerts = False Application.EnableEvents =
False Application.Calculation = xlCalculationManual url =
"https://wtb.app.channeliq.com/buyonline/D_nhoFMJcUal_LOXlInI_g/TOA-60?html=true"
IE.navigate url IE.Visible = True Do DoEvents Loop Until
IE.readyState = READYSTATE_COMPLETE
Set Doc = IE.document
lastRow = Sheet1.UsedRange.Rows.Count + 1 Set tagElements =
Doc.all.tags("tr") For Each element In tagElements
If InStr(element.innerText, "ciq-price")> 0 And
InStr(element.className, "ciq-product-name") > 0 Then
Sheet1.Cells(lastRow, 1).Value = element.innerText
' Exit the for loop once you get the temperature to avoid unnecessary processing
Exit For End If Next
IE.Quit Set IE = Nothing Application.ScreenUpdating = True
Application.DisplayAlerts = True Application.EnableEvents = True
Application.Calculation = xlCalculationAutomatic
End Sub
You can't copy any web scraping macro for your purposes. Every page has it's own HTML code structure. So you must write for every page an own web scraping macro.
I can't explain all about web scraping with VBA here. Please start your recherche for information with "excel vba web scraping" and "document object model". Further you need knowlege about HTML and CSS. In best case also about JavaScript:
The error message user-defined type not defined ocours because you use early binding without a reference to the libraries Microsoft HTML Object Library and Microsoft Internet Controls. You can read here how to set a reference via Tools -> References... and about the differences between early and late binding Early Binding v/s Late Binding and here deeper information from Microsoft Using early binding and late binding in Automation
To get the prices from the shown url you can use the following macro. I use late binding:
Option Explicit
Sub myfile()
Dim IE As Object
Dim url As String
Dim tagElements As Object
Dim element As Object
Dim item As Object
Dim lastRow As Long
lastRow = ActiveSheet.UsedRange.Rows.Count + 1
url = "https://wtb.app.channeliq.com/buyonline/D_nhoFMJcUal_LOXlInI_g/TOA-60?html=true"
Set IE = CreateObject("internetexplorer.application")
IE.navigate url
IE.Visible = True
Do: DoEvents: Loop Until IE.readyState = 4
Set tagElements = IE.document.getElementsByClassName("ciq-online-offer-item ")
For Each element In tagElements
Set item = element.getElementsByTagName("td")(1)
ActiveSheet.Cells(lastRow, 1).Value = Trim(item.innerText)
lastRow = lastRow + 1
Next
IE.Quit
Set IE = Nothing
End Sub
Edit for a second Example:
The new link leads to an offer. I assume the price of the product is to be fetched. No loop is needed for this. You just have to find out in which HTML segment the price is and then you can decide how to get it. In the end there are only two lines of VBA that write the price into the Excel spreadsheet.
I'm in Germany and Excel has automatically set the currency sign from Dollar to Euro. This is of course wrong. Depending on where you are, this may have to be intercepted.
Sub myfile2()
Dim IE As Object
Dim url As String
Dim tagElements As Object
Dim lastRow As Long
lastRow = ActiveSheet.UsedRange.Rows.Count + 1
url = "https://www.wayfair.com/kitchen-tabletop/pdx/cuisinart-air-fryer-toaster-oven-cui3490.html"
Set IE = CreateObject("internetexplorer.application")
IE.navigate url
IE.Visible = True
Do: DoEvents: Loop Until IE.readyState = 4
'Break for 3 seconds
Application.Wait (Now + TimeSerial(0, 0, 3))
Set tagElements = IE.document.getElementsByClassName("BasePriceBlock BasePriceBlock--highlight")(0)
ActiveSheet.Cells(lastRow, 1).Value = Trim(tagElements.innerText)
IE.Quit
Set IE = Nothing
End Sub

How can I pull data from website using vba

I am new at vba coding to pull data from website so generally, I use this code to connect and check item to pull data from website but this code cannot check data via watch in vba with my firm webapp. it show nothing when I add watch to the class so what should I do.HTML Code from my firm webapp 1
HTML Code from my firm webapp 2
Sub Connect_web()
Dim ie As InternetExplorer
Dim doc As HTMLdocument
Dim ele As IHTMLElement
Dim col As IHTMLElementCollection
Dim ele_tmp As IHTMLElement
Set ie = New InternetExplorer
URL = "" ' Cannot provide
ie.Visible = True
ie.navigate URL
Do While ie.readyState <> READYSTATE_COMPLETE
Application.StatusBar = "Loading Page..."
DoEvents
End If
Loop
Set doc = ie.Document
Set ele = doc.getElementByClassName("GDB3EHGDHLC")
end sub
Let's start with four things:
1) Instead of .Navigate use .Navigate2
2) Use a proper wait
While ie.Busy Or ie.readyState < 4: DoEvents: Wend
3) Correct the syntax of your Set ele line. You are using ByClassNamewhich returns a collection and therefore is plural. You are missing the s at the end of element.
As you have declared ele as singular (element), perhaps first set the collection into a separate variable and index into that collection.
Dim eles As Object, ele As Object
Set eles = doc.getElementsByClassName("GDB3EHGDHLC")
Set ele = eles(0)
4) You should always use id over other attributes, if possible, as id is usually quicker for retrieval. There is an id against that class name in your image (highlighted element). I am not going to try and type it all out. Please share your HTML using the snippet tool, by editing your question, so we can relate to your html in answer easily.
Set ele = doc.getElementById("gwt-debug-restOfIdStringGoesHere")

Excel VBA code to click web button

Need help how to create excel vba code for this
I'll be needing the codes so I can complete my macro.
Thanks in advance
First, you will need to create a reference to:
Microsoft Internet Controls
Microsoft HTML Object Library
In VBE, click Tools > References
Sub clickLink()
Dim ie As New InternetExplorer, Url$, doc As HTMLDocument
Url = "http://UrlToYourLink.com"
With ie
.navigate Url
Do While .Busy Or .readyState < READYSTATE_COMPLETE
DoEvents
Loop
doc = .document
.Visible = True
End With
Dim myBtn As Object
Set myBtn = doc.getElementsByClassName("button rounded")(0)
myBtn.Click
End Sub
The Internet control is used to browse the webpage and the HTML Objects are used to identify the username and password textboxes and submit the text using the control button.
Dim HTMLDoc As HTMLDocument
Dim oBrowser As InternetExplorer
Sub Login_2_Website()
Dim oHTML_Element As IHTMLElement
Dim sURL As String
On Error GoTo Err_Clear
sURL = "https://www.google.com/accounts/Login"
Set oBrowser = New InternetExplorer
oBrowser.Silent = True
oBrowser.timeout = 60
oBrowser.navigate sURL
oBrowser.Visible = True
Do
' Wait till the Browser is loaded
Loop Until oBrowser.readyState = READYSTATE_COMPLETE
Set HTMLDoc = oBrowser.Document
HTMLDoc.all.Email.Value = "sample#vbadud.com"
HTMLDoc.all.passwd.Value = "*****"
For Each oHTML_Element In HTMLDoc.getElementsByTagName("input")
If oHTML_Element.Type = "submit" Then oHTML_Element.Click: Exit For
Next
' oBrowser.Refresh ' Refresh If Needed
Err_Clear:
If Err <> 0 Then
Debug.Assert Err = 0
Err.Clear
Resume Next
End If
End Sub
The program requires references to the following:
1 Microsoft Internet Controls
2. Microsoft HTML Object Library
Microsoft internet controls are a great way to do this, but if you aren't allowed to add new references, here is another way to go about web scraping.
This methode ain't as 'clean' as Microsoft internet controls and HTML object but it gets the job done.
Sub GoogleSearch()
Dim ie As Object
Dim objSearchBnt As Object
Dim objCollection As Object
Dim i As Integer
'initialize counter
i = 0
'Create InternetExplorer Object
Set ie = CreateObject("InternetExplorer.Application")
ie.Visible = True
'navigate to the url
ie.navigate "Www.google.com"
'Statusbar shows in the buttom corner of excel
Application.StatusBar = "Loading, please wait..."
'Wait until page is ready
Do While ie.busy
Application.Wait DateAdd("s", 1, Now)
Loop
'Store all the elements with input tag
Set objCollection = ie.Document.getElementsByTagName("input")
'Go through all input elements
While i < objCollection.Length
'input search field
If objCollection(i).Name = "q" Then
objCollection(i).Value = "Hello World"
End If
'search button
If objCollection(i).Type = "submit" Then
Set objSearchBnt = objCollection(i)
End If
i = i + 1
Wend
objSearchBnt.Click
'Clean up
Set objSearchBnt = Nothing
Set objCollection = Nothing
Set ie = Nothing
'Give excel control over the status bar agian
Application.StatusBar = ""
End Sub

Getting information from HTML page via VBA

From VBA, I am trying to access to the "username" cell from a web page so that I could type in the appropriate username.
The problem is that in the HTML code from the page we have more than one element with the same name which is "LOGON_USERID" and I can't figure out how to access to the right one.
As you can see on the image "part of the HTML code", the line I'm trying to access to is the highlighted one, but there are also 2 other elements which have the same name above it.
part of the HTML code
I tried lots of different ways (using different methods or variable types etc), but since I'm not familiar with HTML I can't manage to get what I want.
Sub Pum()
Dim ie As New InternetExplorer
'Dim IEDoc As IHTMLElementCollection
Dim IEDoc As HTMLDocument
Dim name As Object
Dim nameList As HTMLInputElement
Dim WRONGS As DispHTMLElementCollection
Dim Elems As HTMLElementCollection
Dim i As Integer
ie.navigate "thewebsiteinquestion"
ie.Visible = False
WaitIE ie
Set IEDoc = ie.document
'MsgBox IEDoc.DocumentElement.
'Elems = IEDoc.getElementsByTagName("INPUT")
MsgBox TypeName(IEDoc.getElementById("LOGON_USERID").all)
Set Elems = IEDoc.getElementById("LOGON_USERID")
'For i = 0 To 5
MsgBox Elems.Length
'Next i
For Each name In Elems.Children
MsgBox name.nodeName
MsgBox name.Attributes
MsgBox name.all
Next
'If ((NameStr Isnot Nothing And (NameStr.Length <> 0)) Then
'If NameStr = "LOGON_USERID" Then
'If TypeName(IEDoc.all("LOGON_USERID")) = "HTMLInputElement" Then
'MsgBox TypeName(IEDoc.all("LOGON_USERID"))
'Set names = IEDoc.all.Item("text")
'TypeName (InputUsernameTextzone)
'Dim Question As IHTMLElement
'Question = InputUsernameTextzone.parentElement
'MsgBox TypeName(InputUsernameTextzone.parentElement.getAttribute("name"))
'InputUsernameTextzone.parentElement
'CELLULE.value = "qtc2464"
WaitIE ie
Set ie = Nothing
Set IEDoc = Nothing
End Sub
I tried two other similar codes using different methods but I still have no results. Hopefully you can help me.
If you need more information, let me know.
The other two input elements are of different type (they are hidden) so you could use querySelector with attribute type=text to find your desired element.
Dim userid As HTMLInputElement
Set userid = IEDoc.querySelector("input[name='LOGON_USERID'][type='text']")
If Not userid Is Nothing Then
' Continue with user id element
Else
MsgBox "LOGON_USERID not found on the page"
End If
I am a newbie at this but if this could help anyone, here's the simplified version of the macro I made :
Sub Access_Puma()
Dim ie As New InternetExplorer
Dim IEDoc As HTMLDocument
Dim userid As HTMLInputElement
Dim userpwd As HTMLInputElement
ie.navigate "thewebsitetoaccess"
ie.Visible = True
WaitIE ie
Set IEDoc = ie.document
Set userid = IEDoc.querySelector("input[name='LOGON_USERID'][type='text']")
If Not userid Is Nothing Then
userid.value = "myusername"
Else
MsgBox "LOGON_USERID not found on the page"
End If
Set userpwd = IEDoc.querySelector("input[name='LOGON_PASSWD'][type='password']")
If Not userpwd Is Nothing Then
userpwd.value = "mypassword"
Else
MsgBox "LOGON_PASSWD not found on the page"
End If
End Sub