I am trying to wrote some code that will lookup a word in an excel cell, the search for that word and, most importantly, click on the link that returns.
I have the search part down, but it's the click on the link I'm struggling with.
the HMTL extract is -
<span>
<div class="searchResult webResult ">
<div class="resultTitlePane">
<h3>
<a class="outbound" href="whatever" target ="" rel="nofollow" rev="lots of
text">..</a>
</h3>
</div>
I've not typed out the href and rel text, but I just want to be able to click the link that's returned and follow through to that site.
Any help please?
this is my code -
Sub test()
Dim ie As InternetExplorer
Dim RegEx As RegExp, RegMatch As MatchCollection
Dim MyStr As String
Dim pDisp As Object
Dim dtStartTime As Date
Set ie = New InternetExplorer
Set RegEx = New RegExp
Dim iedoc As Object
SearchEng = "http://easysearch.org.uk/search/?s="
LastRow = Range("A1").End(xlDown).Row
Do Until i = LastRow + 1
SearchMe = Range("A" & i)
ie.Navigate SearchEng & SearchMe
Do Until ie.ReadyState = READYSTATE_COMPLETE
Loop
MyStr = ie.document.body.innerText
Set RegMatch = RegEx.Execute(MyStr)
If RegMatch.Count > 0 Then
ie.Navigate RegMatch(0)
Do Until ie.ReadyState = READYSTATE_COMPLETE
Loop
ie.Visible = True
Set iedoc = ie.document
''NEED TO ADD SOMETHING HERE TO CLICK LINK''
End If
i = i + 1
Do Until ie.ReadyState = READYSTATE_COMPLETE
Loop
Loop
Set RegEx = Nothing
ie.Quit
Set ie = Nothing
End Sub
In your code, you can replace you placeholder: ''NEED TO ADD SOMETHING HERE TO CLICK LINK'' with:
iedoc.getElementsByClassName("resultTitlePane")(0).getElementsByTagName("a")(0).Click
In response to OP's comment of readystate not completing:
You can continue to use your readystate loop, and then loop the actual object until it becomes available, then click it:
Do Until ie.ReadyState = READYSTATE_COMPLETE
Loop
ie.Visible = True
Set iedoc = ie.document
' ^ ^ ^ ^ Already part of your code
Dim myBtn As Object
Set myBtn = Nothing
' Loop object while it's Nothing
On Error Resume Next
Do While myBtn Is Nothing
DoEvents
set myBtn = iedoc.getElementsByClassName("resultTitlePane")(0).getElementsByTagName("a")(0)
Loop
On Error GoTo 0
myBtn.Click
Related
My script runs for few row and then i a getting object variable or with block variable not set error.
I am using the below script to extract the 5,6,7 value from the NSEIndia website.
I get the value of a stock from the same Excel and update the same excel with the values from the nseindia website.
Sub Stock_Basic_Update_NSE()
Dim ie As InternetExplorer
Dim webpage As HTMLDocument
Dim ws As Worksheet
For Item = 23 To 1505
Set ws = ThisWorkbook.Worksheets("NSE Stocks Details")
sSearch = ws.Range("A" & Item).Value
'sSearch = Filestk.Worksheets("Sheet1").Range("E1").Value
Set ie = New InternetExplorer
'ie.Visible = True
ie.navigate ("https://www.nseindia.com/get-quotes/equity?symbol=" & sSearch)
Do While ie.readyState = 4: DoEvents: Loop
Do Until ie.readyState = 4: DoEvents: Loop
While ie.Busy
DoEvents
Wend
Set webpage = ie.document
ws.Cells(Item, 3).Value = webpage.getElementsByClassName("eq-series table-fullwidth w-100")(0).getElementsByTagName("td")(5).innerText
ws.Cells(Item, 4).Value = webpage.getElementsByClassName("eq-series table-fullwidth w-100")(0).getElementsByTagName("td")(6).innerText
ws.Cells(Item, 5).Value = webpage.getElementsByClassName("eq-series table-fullwidth w-100")(0).getElementsByTagName("td")(7).innerText
ie.Quit
Set ie = Nothing
Next Item
End Sub
You had some errors in your code and you hadn't wait for the full document to load. Try the following code. I have commented it. So you can see, what I have changed and why. I have tried it with the top 50 symbols.
Sub Stock_Basic_Update_NSE()
'Declare always all variables
Dim ie As Object 'I switched this from early to late binding (not required)
Dim nodeTable As Object
Dim ws As Worksheet
Dim item As Long
Dim sSearch As String
'Use this outside the loop. You only need it once
Set ws = ThisWorkbook.Worksheets("NSE Stocks Details")
For item = 23 To 1505
sSearch = ws.Range("A" & item).Value
Set ie = CreateObject("internetexplorer.application")
ie.Visible = False
'Encode symbols that are restricted for using in URLs. Like &, : or ?
ie.navigate ("https://www.nseindia.com/get-quotes/equity?symbol=" & WorksheetFunction.EncodeURL(sSearch))
'It's not "While = 4" because 4 stands for "readystate = complete"
'If you want to use "= 4" you must use "Until" instead of "While"
'It doesn't matter what you use
Do While ie.readyState <> 4: DoEvents: Loop
'Manual break to load dynamic content after the IE reports the page load was complete
'This was your main problem
Application.Wait (Now + TimeSerial(0, 0, 2))
'The needed html table has an ID. If possible use always that instead of class names
'because an html ID is unique if the standard is kept
'Also use a variable to save the elements
'So you don't need to shorten the html document string in most cases because
'it's only needed one time
Set nodeTable = ie.document.getElementByID("equityInfo")
ws.Cells(item, 3).Value = nodeTable.getElementsByTagName("td")(5).innerText
ws.Cells(item, 4).Value = nodeTable.getElementsByTagName("td")(6).innerText
ws.Cells(item, 5).Value = nodeTable.getElementsByTagName("td")(7).innerText
'Clean up
ie.Quit
Set ie = Nothing
Next item
End Sub
I have 15 different URLs, and I need to fetch price from the particular website in Excel a particular column, can you please help me out. It's my first VBA program and I try but it show my syntax error.
Sub myfile()
Dim IE As New InternetExplorer Dim url As String Dim item As
HTMLHtmlElement Dim Doc As HTMLDocument Dim tagElements As Object
Dim element As Object Dim lastRow Application.ScreenUpdating =
False Application.DisplayAlerts = False Application.EnableEvents =
False Application.Calculation = xlCalculationManual url =
"https://wtb.app.channeliq.com/buyonline/D_nhoFMJcUal_LOXlInI_g/TOA-60?html=true"
IE.navigate url IE.Visible = True Do DoEvents Loop Until
IE.readyState = READYSTATE_COMPLETE
Set Doc = IE.document
lastRow = Sheet1.UsedRange.Rows.Count + 1 Set tagElements =
Doc.all.tags("tr") For Each element In tagElements
If InStr(element.innerText, "ciq-price")> 0 And
InStr(element.className, "ciq-product-name") > 0 Then
Sheet1.Cells(lastRow, 1).Value = element.innerText
' Exit the for loop once you get the temperature to avoid unnecessary processing
Exit For End If Next
IE.Quit Set IE = Nothing Application.ScreenUpdating = True
Application.DisplayAlerts = True Application.EnableEvents = True
Application.Calculation = xlCalculationAutomatic
End Sub
You can't copy any web scraping macro for your purposes. Every page has it's own HTML code structure. So you must write for every page an own web scraping macro.
I can't explain all about web scraping with VBA here. Please start your recherche for information with "excel vba web scraping" and "document object model". Further you need knowlege about HTML and CSS. In best case also about JavaScript:
The error message user-defined type not defined ocours because you use early binding without a reference to the libraries Microsoft HTML Object Library and Microsoft Internet Controls. You can read here how to set a reference via Tools -> References... and about the differences between early and late binding Early Binding v/s Late Binding and here deeper information from Microsoft Using early binding and late binding in Automation
To get the prices from the shown url you can use the following macro. I use late binding:
Option Explicit
Sub myfile()
Dim IE As Object
Dim url As String
Dim tagElements As Object
Dim element As Object
Dim item As Object
Dim lastRow As Long
lastRow = ActiveSheet.UsedRange.Rows.Count + 1
url = "https://wtb.app.channeliq.com/buyonline/D_nhoFMJcUal_LOXlInI_g/TOA-60?html=true"
Set IE = CreateObject("internetexplorer.application")
IE.navigate url
IE.Visible = True
Do: DoEvents: Loop Until IE.readyState = 4
Set tagElements = IE.document.getElementsByClassName("ciq-online-offer-item ")
For Each element In tagElements
Set item = element.getElementsByTagName("td")(1)
ActiveSheet.Cells(lastRow, 1).Value = Trim(item.innerText)
lastRow = lastRow + 1
Next
IE.Quit
Set IE = Nothing
End Sub
Edit for a second Example:
The new link leads to an offer. I assume the price of the product is to be fetched. No loop is needed for this. You just have to find out in which HTML segment the price is and then you can decide how to get it. In the end there are only two lines of VBA that write the price into the Excel spreadsheet.
I'm in Germany and Excel has automatically set the currency sign from Dollar to Euro. This is of course wrong. Depending on where you are, this may have to be intercepted.
Sub myfile2()
Dim IE As Object
Dim url As String
Dim tagElements As Object
Dim lastRow As Long
lastRow = ActiveSheet.UsedRange.Rows.Count + 1
url = "https://www.wayfair.com/kitchen-tabletop/pdx/cuisinart-air-fryer-toaster-oven-cui3490.html"
Set IE = CreateObject("internetexplorer.application")
IE.navigate url
IE.Visible = True
Do: DoEvents: Loop Until IE.readyState = 4
'Break for 3 seconds
Application.Wait (Now + TimeSerial(0, 0, 3))
Set tagElements = IE.document.getElementsByClassName("BasePriceBlock BasePriceBlock--highlight")(0)
ActiveSheet.Cells(lastRow, 1).Value = Trim(tagElements.innerText)
IE.Quit
Set IE = Nothing
End Sub
I'm trying to write a simple code for studying vocabulary and want this code to look up the words in column "A" using my favorite online dictionary "Cambridge" automatically and then print the definitions to the cells next to the words. I have written the code below so far and it goes to the site and searches the word. The question is what code is needed to get the definitions and print them to the cells?
Sub SearchWords()
Dim IE As New SHDocVw.InternetExplorer
Dim HTMLDoc As MSHTML.HTMLDocument
Dim HTMLInput As MSHTML.IHTMLElement
Dim HTMLButtons As MSHTML.IHTMLElementCollection
Dim HTMLButton As MSHTML.IHTMLElement
IE.Visible = True
IE.Navigate "www.dictionary.cambridge.org"
Do While IE.ReadyState <> READYSTATE_COMPLETE
Loop
Set HTMLDoc = IE.Document
Set HTMLInput = HTMLDoc.getElementById("cdo-search-input")
HTMLInput.Value = ThisWorkbook.Sheets(1).Range("A1").Value
Set HTMLButtons = HTMLDoc.getElementsByClassName("cdo-search__button")
HTMLButtons(0).Click
End Sub
Thanks in advance.
The result appears to be in an element with classname entry. I read your column A search words in to an array and loop that to look up each word. The result is written back out to the sheet. I use css selectors mostly as a more flexible and faster method for selecting elements. css selectors, in this instance, are applied via querySelector method of HTMLDocument (i.e. ie.Document)
Proper page loads waits are used throughout.
Option Explicit
'entry
Public Sub SearchWords()
Dim IE As SHDocVw.InternetExplorer, lookups(), dataSheet As Worksheet, iRow As Long
Set dataSheet = ThisWorkbook.Worksheets("Sheet1")
Set IE = New SHDocVw.InternetExplorer
lookups = Application.Transpose(dataSheet.Range("A2:A3").Value) '<Read words to lookup into a 2d array and transpose into 1D
With IE
.Visible = True
.Navigate2 "www.dictionary.cambridge.org"
While .Busy Or .readyState <> 4: DoEvents: Wend
For iRow = LBound(lookups) To UBound(lookups)
.document.getElementById("cdo-search-input").Value = lookups(iRow) 'work off .document to avoid stale elements
.document.querySelector(".cdo-search__button").Click
While .Busy Or .readyState <> 4: DoEvents: Wend 'wait for page reload
Application.Wait Now + TimeSerial(0, 0, 1)
Do
Loop While .document.querySelectorAll(".entry").Length = 0
dataSheet.Cells(iRow + 1, 2) = .document.querySelector(".entry").innerText
Next
.Quit
End With
End Sub
Done! Perfectly working. (Since this post is too long for a comment, I had to post this as an answer) Now I am trying to get some more data from the page(since I need the other explanations and Turkish definitions as well). When I inspect the page, I see that full descriptions are placed in "di $ entry-body__el entry-body__el--smalltop clrd js-share-holder" class. I added "/turkish" to the URL and tried to get the related element using the class name I mentioned instead of ".def-block", but it didn't work. Then I tried a different way using this code:
Sub GetMeaningsFromCambridgeDictionary()
Dim ws As Worksheet
Set ws = ThisWorkbook.Worksheets("Meanings")
Dim sourceWord As String
sourceWord = ws.Range("A2").Value
Dim i As Integer
Dim çeviri As String
Dim ilkSatir As Integer
ilkSatir = ws.Cells(ws.Rows.Count, "B").End(xlUp).Row + 1
Dim IE As Object
Set IE = CreateObject("InternetExplorer.Application")
Dim URL As String
Dim countElement As Integer
Range("B2:B1000").Delete
IE.Visible = False
URL = "https://dictionary.cambridge.org/dictionary/turkish/" & sourceWord
IE.Navigate URL
Do While IE.Busy: DoEvents: Loop
Application.Wait (Now + TimeValue("0:00:01"))
Do While IE.readyState <> 4
Application.Wait (Now + TimeValue("0:00:01"))
Loop
countElement = IE.document.getElementsByClassName("di $ entry-body__el entry-body__el--smalltop clrd js-share-holder").Length
For i = 0 To countElement - 1
çeviri = IE.document.getElementsByClassName("di $ entry-body__el entry-body__el--smalltop clrd js-share-holder")(i).innerText
Range("B" & i + 2).Value = çeviri
Range("B" & i + 2).Rows.AutoFit
Next i
Columns(2).AutoFit
IE.Quit
MsgBox "All meanings have been copied."
End Sub
This code is also working, and I see all the definitions in detail, but this time the problem is only the first word is done. What should I do to do the same thing for the other words?
I'm trying to pull some information from a website after navigating to it but I can't seem to wait until it completely loads. I've been trying to loop until the class at (0) contains text. Anyone know what I'm doing wrong?
Sub test()
Dim IE As Object
Set IE = CreateObject("InternetExplorer.Application")
IE.Visible = True
Dim elements2 As IHTMLElementCollection
IE.Navigate "https://www.facebook.com/marketplace/item/955559644646354/"
Do While IE.Busy Or IE.readyState <> 4
DoEvents
Loop
Dim x
x = 0
Do Until x = 1
Set elements2 = IE.document.getElementsByClassName("_3cgd")
If WorksheetFunction.IsText(elements2(0).innerText) = True Then
MsgBox ((elements2(0).innerText))
x = 1
Else
Application.Wait Now + #12:00:01 AM#
End If
Loop
End Sub
Try something like this (untested)
Dim t, elements2, txt
t = Timer
txt = ""
Do
Set elements2 = IE.document.getElementsByClassName("_3cgd")
If elements2.length > 0 Then
txt = elements2(0).innerText
If Len(txt) > 0 Then Exit Do
End If
If (Timer - t) > 10 Then Exit Do 'exit if too long waiting
Application.Wait Now + TimeSerial(0, 0, 1)
Loop
You can try to add lines below to wait for loading the web page completely.
' Wait while IE loading...
'IE ReadyState = 4 signifies the webpage has loaded (the first loop is set to avoid inadvertently skipping over the second loop)
Do While IE.ReadyState = 4: DoEvents: Loop 'Do While
Do Until IE.ReadyState = 4: DoEvents: Loop 'Do Until
Below is the full working example:
Sub Automate_IE_Load_Page()
'This will load a webpage in IE
Dim i As Long
Dim URL As String
Dim IE As Object
Dim objElement As Object
Dim objCollection As Object
'Create InternetExplorer Object
Set IE = CreateObject("InternetExplorer.Application")
'Set IE.Visible = True to make IE visible, or False for IE to run in the background
IE.Visible = True
'Define URL
URL = "https://www.automateexcel.com/excel/"
'Navigate to URL
IE.Navigate URL
' Statusbar let's user know website is loading
Application.StatusBar = URL & " is loading. Please wait..."
' Wait while IE loading...
'IE ReadyState = 4 signifies the webpage has loaded (the first loop is set to avoid inadvertently skipping over the second loop)
Do While IE.ReadyState = 4: DoEvents: Loop 'Do While
Do Until IE.ReadyState = 4: DoEvents: Loop 'Do Until
'Webpage Loaded
Application.StatusBar = URL & " Loaded"
'Unload IE
Set IE = Nothing
Set objElement = Nothing
Set objCollection = Nothing
End Sub
Further, You can try to modify this code example as per your requirement.
Reference:
(1) Automate Internet Explorer (IE) Using VBA
Need help how to create excel vba code for this
I'll be needing the codes so I can complete my macro.
Thanks in advance
First, you will need to create a reference to:
Microsoft Internet Controls
Microsoft HTML Object Library
In VBE, click Tools > References
Sub clickLink()
Dim ie As New InternetExplorer, Url$, doc As HTMLDocument
Url = "http://UrlToYourLink.com"
With ie
.navigate Url
Do While .Busy Or .readyState < READYSTATE_COMPLETE
DoEvents
Loop
doc = .document
.Visible = True
End With
Dim myBtn As Object
Set myBtn = doc.getElementsByClassName("button rounded")(0)
myBtn.Click
End Sub
The Internet control is used to browse the webpage and the HTML Objects are used to identify the username and password textboxes and submit the text using the control button.
Dim HTMLDoc As HTMLDocument
Dim oBrowser As InternetExplorer
Sub Login_2_Website()
Dim oHTML_Element As IHTMLElement
Dim sURL As String
On Error GoTo Err_Clear
sURL = "https://www.google.com/accounts/Login"
Set oBrowser = New InternetExplorer
oBrowser.Silent = True
oBrowser.timeout = 60
oBrowser.navigate sURL
oBrowser.Visible = True
Do
' Wait till the Browser is loaded
Loop Until oBrowser.readyState = READYSTATE_COMPLETE
Set HTMLDoc = oBrowser.Document
HTMLDoc.all.Email.Value = "sample#vbadud.com"
HTMLDoc.all.passwd.Value = "*****"
For Each oHTML_Element In HTMLDoc.getElementsByTagName("input")
If oHTML_Element.Type = "submit" Then oHTML_Element.Click: Exit For
Next
' oBrowser.Refresh ' Refresh If Needed
Err_Clear:
If Err <> 0 Then
Debug.Assert Err = 0
Err.Clear
Resume Next
End If
End Sub
The program requires references to the following:
1 Microsoft Internet Controls
2. Microsoft HTML Object Library
Microsoft internet controls are a great way to do this, but if you aren't allowed to add new references, here is another way to go about web scraping.
This methode ain't as 'clean' as Microsoft internet controls and HTML object but it gets the job done.
Sub GoogleSearch()
Dim ie As Object
Dim objSearchBnt As Object
Dim objCollection As Object
Dim i As Integer
'initialize counter
i = 0
'Create InternetExplorer Object
Set ie = CreateObject("InternetExplorer.Application")
ie.Visible = True
'navigate to the url
ie.navigate "Www.google.com"
'Statusbar shows in the buttom corner of excel
Application.StatusBar = "Loading, please wait..."
'Wait until page is ready
Do While ie.busy
Application.Wait DateAdd("s", 1, Now)
Loop
'Store all the elements with input tag
Set objCollection = ie.Document.getElementsByTagName("input")
'Go through all input elements
While i < objCollection.Length
'input search field
If objCollection(i).Name = "q" Then
objCollection(i).Value = "Hello World"
End If
'search button
If objCollection(i).Type = "submit" Then
Set objSearchBnt = objCollection(i)
End If
i = i + 1
Wend
objSearchBnt.Click
'Clean up
Set objSearchBnt = Nothing
Set objCollection = Nothing
Set ie = Nothing
'Give excel control over the status bar agian
Application.StatusBar = ""
End Sub