I am using below mentioned code in Excel VBA for IE navigation.I am facing following error while fetching data from iframe.
Error detail:
Object does not support this property or method
Option Explicit
Public Sub Cgg_Click()
Dim Ie As New InternetExplorer
Dim WebURL
Dim Docx As HTMLDocument
Dim productDesc
Dim productTitle
Dim price
Dim RcdNum
Ie.Visible = True
WebURL = "https://www.google.com/maps/place/parlour+beauty+parlour+beauty/#40.7314166,-74.13182,11z/data=!4m8!1m2!2m1!1sParlour+NY!3m4!1s0x89c2599bd4c1d2e7:0x20873676f6334189!8m2!3d40.7314166!4d-73.9917443"
Ie.Navigate2 WebURL
Do Until Ie.readyState = READYSTATE_COMPLETE
DoEvents
Loop
Application.Wait (Now + TimeValue("00:00:25"))
For N = 0 To Ie.document.getElementsByClassName("section-subheader-header GLOBAL__gm2-subtitle-alt-1").Length - 1
If Ie.document.getElementsByClassName("section-subheader-header GLOBAL__gm2-subtitle-alt-1").Item(N).innerText = "Web results" Then
Ie.document.getElementsByClassName("section-subheader-header GLOBAL__gm2-subtitle-alt-1").Item(N).ScrollIntoView (False)
End If
Next N
Application.Wait (Now + TimeValue("00:00:25"))
Set Docx = Ie.document
productDesc = Docx.Window.frames("section-iframe-iframe").contentWindow.document.getElementsByClassName("trex")(0).outerHTML
End Sub
Here is the HTML:
Please help to resolve this error.
I want to extract "trex" ClassName HTML Contain from above url
Thanks.
You can change the line of extract "trex" element to one of the following, both of them can work well:
Use the getElementsbyTagName method to get the Iframe first , then according to the Iframe.contentDocument property to reach the element via the class name:
productDesc = Docx.getElementsByTagName("iframe")(0).contentDocument.getElementsByClassName("trex")(0).outerHTML
Use querySelector method to get the Iframe through class, then use the same as the above to reach the element:
productDesc = Docx.querySelector(".section-iframe-iframe").contentDocument.getElementsByClassName("trex")(0).outerHTML
Related
I have 15 different URLs, and I need to fetch price from the particular website in Excel a particular column, can you please help me out. It's my first VBA program and I try but it show my syntax error.
Sub myfile()
Dim IE As New InternetExplorer Dim url As String Dim item As
HTMLHtmlElement Dim Doc As HTMLDocument Dim tagElements As Object
Dim element As Object Dim lastRow Application.ScreenUpdating =
False Application.DisplayAlerts = False Application.EnableEvents =
False Application.Calculation = xlCalculationManual url =
"https://wtb.app.channeliq.com/buyonline/D_nhoFMJcUal_LOXlInI_g/TOA-60?html=true"
IE.navigate url IE.Visible = True Do DoEvents Loop Until
IE.readyState = READYSTATE_COMPLETE
Set Doc = IE.document
lastRow = Sheet1.UsedRange.Rows.Count + 1 Set tagElements =
Doc.all.tags("tr") For Each element In tagElements
If InStr(element.innerText, "ciq-price")> 0 And
InStr(element.className, "ciq-product-name") > 0 Then
Sheet1.Cells(lastRow, 1).Value = element.innerText
' Exit the for loop once you get the temperature to avoid unnecessary processing
Exit For End If Next
IE.Quit Set IE = Nothing Application.ScreenUpdating = True
Application.DisplayAlerts = True Application.EnableEvents = True
Application.Calculation = xlCalculationAutomatic
End Sub
You can't copy any web scraping macro for your purposes. Every page has it's own HTML code structure. So you must write for every page an own web scraping macro.
I can't explain all about web scraping with VBA here. Please start your recherche for information with "excel vba web scraping" and "document object model". Further you need knowlege about HTML and CSS. In best case also about JavaScript:
The error message user-defined type not defined ocours because you use early binding without a reference to the libraries Microsoft HTML Object Library and Microsoft Internet Controls. You can read here how to set a reference via Tools -> References... and about the differences between early and late binding Early Binding v/s Late Binding and here deeper information from Microsoft Using early binding and late binding in Automation
To get the prices from the shown url you can use the following macro. I use late binding:
Option Explicit
Sub myfile()
Dim IE As Object
Dim url As String
Dim tagElements As Object
Dim element As Object
Dim item As Object
Dim lastRow As Long
lastRow = ActiveSheet.UsedRange.Rows.Count + 1
url = "https://wtb.app.channeliq.com/buyonline/D_nhoFMJcUal_LOXlInI_g/TOA-60?html=true"
Set IE = CreateObject("internetexplorer.application")
IE.navigate url
IE.Visible = True
Do: DoEvents: Loop Until IE.readyState = 4
Set tagElements = IE.document.getElementsByClassName("ciq-online-offer-item ")
For Each element In tagElements
Set item = element.getElementsByTagName("td")(1)
ActiveSheet.Cells(lastRow, 1).Value = Trim(item.innerText)
lastRow = lastRow + 1
Next
IE.Quit
Set IE = Nothing
End Sub
Edit for a second Example:
The new link leads to an offer. I assume the price of the product is to be fetched. No loop is needed for this. You just have to find out in which HTML segment the price is and then you can decide how to get it. In the end there are only two lines of VBA that write the price into the Excel spreadsheet.
I'm in Germany and Excel has automatically set the currency sign from Dollar to Euro. This is of course wrong. Depending on where you are, this may have to be intercepted.
Sub myfile2()
Dim IE As Object
Dim url As String
Dim tagElements As Object
Dim lastRow As Long
lastRow = ActiveSheet.UsedRange.Rows.Count + 1
url = "https://www.wayfair.com/kitchen-tabletop/pdx/cuisinart-air-fryer-toaster-oven-cui3490.html"
Set IE = CreateObject("internetexplorer.application")
IE.navigate url
IE.Visible = True
Do: DoEvents: Loop Until IE.readyState = 4
'Break for 3 seconds
Application.Wait (Now + TimeSerial(0, 0, 3))
Set tagElements = IE.document.getElementsByClassName("BasePriceBlock BasePriceBlock--highlight")(0)
ActiveSheet.Cells(lastRow, 1).Value = Trim(tagElements.innerText)
IE.Quit
Set IE = Nothing
End Sub
I am new at vba coding to pull data from website so generally, I use this code to connect and check item to pull data from website but this code cannot check data via watch in vba with my firm webapp. it show nothing when I add watch to the class so what should I do.HTML Code from my firm webapp 1
HTML Code from my firm webapp 2
Sub Connect_web()
Dim ie As InternetExplorer
Dim doc As HTMLdocument
Dim ele As IHTMLElement
Dim col As IHTMLElementCollection
Dim ele_tmp As IHTMLElement
Set ie = New InternetExplorer
URL = "" ' Cannot provide
ie.Visible = True
ie.navigate URL
Do While ie.readyState <> READYSTATE_COMPLETE
Application.StatusBar = "Loading Page..."
DoEvents
End If
Loop
Set doc = ie.Document
Set ele = doc.getElementByClassName("GDB3EHGDHLC")
end sub
Let's start with four things:
1) Instead of .Navigate use .Navigate2
2) Use a proper wait
While ie.Busy Or ie.readyState < 4: DoEvents: Wend
3) Correct the syntax of your Set ele line. You are using ByClassNamewhich returns a collection and therefore is plural. You are missing the s at the end of element.
As you have declared ele as singular (element), perhaps first set the collection into a separate variable and index into that collection.
Dim eles As Object, ele As Object
Set eles = doc.getElementsByClassName("GDB3EHGDHLC")
Set ele = eles(0)
4) You should always use id over other attributes, if possible, as id is usually quicker for retrieval. There is an id against that class name in your image (highlighted element). I am not going to try and type it all out. Please share your HTML using the snippet tool, by editing your question, so we can relate to your html in answer easily.
Set ele = doc.getElementById("gwt-debug-restOfIdStringGoesHere")
My goal is to scrape the source code of a web page.
The site seems to have different Frames which is why my code won't work properly.
I tried to modify a code which I found online which should solve the Frame issue.
The following code creates an error (object required) at:
Set profileFrame .document.getElementById("profileFrame")
Public Sub IE_Automation()
'Needs references to Microsoft Internet Controls and Microsoft HTML Object Library
Dim baseURL As String
Dim IE As InternetExplorer
Dim HTMLdoc As HTMLDocument
Dim profileFrame As HTMLIFrame
Dim slotsDiv As HTMLDivElement
'example URL with multiple frames
baseURL = "https://www.xing.com/search/members?section=members&keywords=IT&filters%5Bcontact_level%5D=non_contact"
Set IE = New InternetExplorer
With IE
.Visible = True
'Navigate to the main page
.navigate baseURL & "/publictrophy/index.htm?onlinename=ace_anubis"
While .Busy Or .readyState <> READYSTATE_COMPLETE: DoEvents: Wend
'Get the profileFrame iframe and navigate to it
Set profileFrame = .document.getElementById("profileFrame")
.navigate baseURL & profileFrame.src
While .Busy Or .readyState <> READYSTATE_COMPLETE: DoEvents: Wend
Set HTMLdoc = .document
End With
'Display all the text in the profileFrame iframe
MsgBox HTMLdoc.body.innerText
'Display just the text in the slots_container div
Set slotsDiv = HTMLdoc.getElementById("slots_container")
MsgBox slotsDiv.innerText
End Sub
Hummmm, I'm not exactly sure what you are doing here, but can you try the code below?
Option Explicit
Sub Sample()
Dim ie As Object
Dim links As Variant, lnk As Variant
Dim rowcount As Long
Set ie = CreateObject("InternetExplorer.Application")
ie.Visible = True
ie.navigate "https://www.xing.com/search/members?section=members&keywords=IT&filters%5Bcontact_level%5D=non_contact"
'Wait for site to fully load
'ie.Navigate2 URL
Do While ie.Busy = True
DoEvents
Loop
Set links = ie.document.getElementsByTagName("a")
rowcount = 1
With Sheets("Sheet1")
For Each lnk In links
'Debug.Print lnk.innerText
'If lnk.classname Like "*Real Statistics Examples Part 1*" Then
.Range("A" & rowcount) = lnk.innerText
rowcount = rowcount + 1
'Exit For
'End If
Next
End With
End Sub
General:
I think in your research you may have come across this question and misunderstood how it relates/doesn't relate to your circumstance.
I don't think iFrames are relevant to your query. If you are after the list of names, their details and the URLs to their pages you can use the code below.
CSS Selectors
To target the elements of interest I use the following two CSS selectors. These use style infomation on the page to target the elements:
.SearchResults-link
.SearchResults-item
"." means class, which is like saying .getElementsByClassName. The first gets the links, and the second gets the description information on the first page.
With respect to the first CSS selector: The actual link required is dynamically constructed, but we can use the fact that the actual profile URLs have a common base string of "https://www.xing.com/profile/", which is then followed by the profileName. So, in function GetURL, we parse the outerHTML returned by the CSS selector to get the profileName and concatenate it with the BASESTRING constant to get our actual profile link.
Code:
Option Explicit
Public Sub GetInfo()
Dim IE As New InternetExplorer
With IE
.Visible = True
.navigate "https://www.xing.com/publicsearch/query?search%5Bq%5D=IT"
While .Busy Or .readyState < 4: DoEvents: Wend
Dim a As Object, exitTime As Date, linksNodeList As Object, profileNodeList As Object
' exitTime = Now + TimeSerial(0, 0, 5) '<== uncomment this section if timing problems
'
' Do
' DoEvents
' On Error Resume Next
' Set linksNodeList = .document.querySelectorAll(".SearchResults-link")
' On Error GoTo 0
' If Now > exitTime Then Exit Do
' Loop While linksNodeList Is Nothing
Set linksNodeList = .document.querySelectorAll(".SearchResults-link") '<== comment this out if uncommented section above
Set profileNodeList = .document.querySelectorAll(".SearchResults-item")
Dim i As Long
For i = 0 To profileNodeList.Length - 1
Debug.Print "Profile link: " & GetURL(linksNodeList.item(i).outerHTML)
Debug.Print "Basic info: " & profileNodeList.item(i).innerText
Next i
End With
End Sub
Public Function GetURL(ByVal htmlSection As String) As String
Const BASESTRING As String = "https://www.xing.com/profile/"
Dim arr() As String
arr = Split(htmlSection, "/")
GetURL = BASESTRING & Replace$(Split((arr(UBound(arr) - 1)), ">")(0), Chr$(34), vbNullString)
End Function
Example return information:
I have to download the source code of a several hundred websites to an Excel file (for example to cells(1, 1) in Worksheets 1) and then extract the content of of the META tag keywords in let's say cells(1, 2).
For downloading I use the following code in VBA:
Dim htm As Object
Set htm = CreateObject("HTMLfile")
URL = "https://www.insolvenzbekanntmachungen.de/cgi-bin/bl_aufruf.pl?PHPSESSID=8ecbeb942c887974468b9010531fc7ab&datei=gerichte/nw/agkoeln/16/0071_IN00181_16/2016_06_10__11_53_26_Anordnung_Sicherungsmassnahmen.htm"
With CreateObject("MSXML2.XMLHTTP")
.Open "GET", URL, False
.send
htm.body.innerHTML = .responseText
Cells(1, 1) = .responseText
End With
I've found the following code on this website but, unfortunately, I'm unable to adapt it to solve my problem:
Sub GetData()
Dim ie As New InternetExplorer
Dim str As String
Dim wk As Worksheet
Dim webpage As New HTMLDocument
Dim item As HTMLHtmlElement
Set wk = Worksheets(1)
str = "https://www.insolvenzbekanntmachungen.de/cgi-bin/bl_aufruf.pl?PHPSESSID=8ecbeb942c887974468b9010531fc7ab&datei=gerichte/nw/agkoeln/16/0071_IN00181_16/2016_06_10__11_53_26_Anordnung_Sicherungsmassnahmen.htm"
ie.Visible = True
ie.navigate str
Do
DoEvents
Loop Until ie.readyState = READYSTATE_COMPLETE
'Find the proper meta element --------------
Const META_TAG As String = "META"
Const META_NAME As String = "keywords"
Dim Doc As HTMLDocument
Dim metaElements As Object
Dim element As Object
Dim kwd As String
Set Doc = ie.Document
Set metaElements = Doc.all.tags(META_TAG)
For Each element In metaElements
If element.Name = META_NAME Then
kwd = element.Content
End If
Next
MsgBox kwd
End Sub
I think I have to modify this line, but don't know how:
Set Doc = ie.Document
Can you please help me out?
Embed a WebrowserControl into a Excel Spreadsheet or userform
How to add a Webrowser to Excel
Set up references to the HTML Object Library
How to add VBA References – Internet Controls, HTML Object Library
Grab Greg Truby's code from this post Webbroswer Control
You'll have access the Document Object Model (DOM). This will expose most of the HTMLElements properties and event's
Option Explicit
Private WithEvents htmDocument As HTMLDocument
Private WithEvents MyButton As HTMLButtonElement
Private Function MyButton_onclick() As Boolean
MsgBox "Sombody Click MyButton on WebBrowser1"
End Function
Private Sub WebBrowser1_NavigateComplete2(ByVal pDisp As Object, URL As Variant)
Dim aTags As Hyperlinks
Do Until .ReadyState = READYSTATE_COMPLETE
DoEvents
Loop
Set MyButton = htmDocument.getElementById("MyButtonID")
Set htmDocument = WebBrowser1.Document
Set aTags = htmDocument.getElementsByTagName("a")
End Sub
Google Web Api, HTA, (MDN){https://developer.mozilla.org/en-US/docs/Web/API} and if you get stuck try to refactor Javascript code to vbscript. It's
Given an url, how can I get the title of the html page in VBA in Excel?
For example suppose I have three urls like :
http://url1.com/somepage.html
http://url2.com/page.html
http://url3.com/page.html
Now I need to get the title of these html pages in another column. How do I do it?
Remou's answer was VERY helpful for me, but it caused a problem: It doesn't close the Internet Explorer process, so since I needed to run this dozens of times I ended up with too many IEs open, and my computer couldn't handle this.
So just add
wb.Quit
and everything will be fine.
This is the code that works for me:
Function GetTitleFromURL(sURL As String)
Dim wb As Object
Dim doc As Object
Set wb = CreateObject("InternetExplorer.Application")
wb.Navigate sURL
While wb.Busy
DoEvents
Wend
GetTitleFromURL = wb.Document.Title
wb.Quit
Set wb = Nothing
End Function
I am not sure what you mean by title, but here is an idea:
Dim wb As Object
Dim doc As Object
Dim sURL As String
Set wb = CreateObject("InternetExplorer.Application")
sURL = "http://lessthandot.com"
wb.Navigate sURL
While wb.Busy
DoEvents
Wend
''HTML Document
Set doc = wb.document
''Title
Debug.Print doc.Title
Set wb = Nothing
if you use Selenium :
Sub Get_Title()
Dim driver As New WebDriver
debug.print driver.Title
End Sub