I'm trying to click 'a' element on a webpage and I can't find out why it does not work.
Here is my VBA code.
Function answer1(ie3 As InternetExplorer, str_anwer As String, answerid As String)
Dim ie4 As New InternetExplorer
Dim a As Object
Set ie4 = ie3
ie4.Document.getElementbyId("view=" & answerid).Click
ie4.Document.getElementbyId("reply_cont").Value = str_anwer
End Function
Error: Property not found
Here's the HTML code from the webpage I think it is located in
<tr>
<td class="thm">208975260</td>
<td><pre>교환</pre></td>
<td class="subject">작동이안되서 교환 원합니다 어떻게 하면되나요?</td>
<td class="id"><span class="thm">st******</span><br>한혜진</td>
<td class="thm">2016.09.29 12:53:57</td>
<td id="date208975260"><span class="point2 ls1">미답변</span>
</td>
<td class="ansr">-</td>
</tr>
Sorry for my English
I'm not fluent English.
Please, let me know why it is not working
Without reference to Microsoft Internet Controls (SHDocVw) and Microsoft HTML Object Library the code could look something like the following. Note the IsNull call. When getElementbyId is called like this and the element is not found on the page this function returns Variant\Null.
In commented code a second example is shown. In this case the references were added and getElementbyId was called on variable of type HTMLDocument. Here in case the element was not found on the page this function returns Nothing.
Sub main()
Dim ie, url, readyStateComplete
readyStateComplete = 4
Set ie = CreateObject("InternetExplorer.Application")
ie.Visible = True
url = "your-url"
ie.navigate url
While ie.Busy Or ie.readyState <> readyStateComplete: DoEvents: Wend
answer1 ie, "<anwer>", "208975260"
ie.Quit
End Sub
Function answer1(ie As Variant, str_anwer As String, answerid As String)
Dim a As Object
If Not IsNull(ie.Document.getElementbyId("view" & answerid)) Then
ie.Document.getElementbyId("view" & answerid).Click
End If
If Not IsNull(ie.Document.getElementbyId("reply_cont")) Then
ie.Document.getElementbyId("reply_cont").Value = str_anwer
End If
' Dim htmlDoc As HTMLDocument
' Set htmlDoc = ie.document
' If Not htmlDoc.getElementbyId("reply_cont") Is Nothing Then
' htmlDoc.getElementbyId("reply_cont").Value = str_anwer
' End If
End Function
Read about difference between Early/Late Binding.
Related
I am using below mentioned code in Excel VBA for IE navigation.I am facing following error while fetching data from iframe.
Error detail:
Object does not support this property or method
Option Explicit
Public Sub Cgg_Click()
Dim Ie As New InternetExplorer
Dim WebURL
Dim Docx As HTMLDocument
Dim productDesc
Dim productTitle
Dim price
Dim RcdNum
Ie.Visible = True
WebURL = "https://www.google.com/maps/place/parlour+beauty+parlour+beauty/#40.7314166,-74.13182,11z/data=!4m8!1m2!2m1!1sParlour+NY!3m4!1s0x89c2599bd4c1d2e7:0x20873676f6334189!8m2!3d40.7314166!4d-73.9917443"
Ie.Navigate2 WebURL
Do Until Ie.readyState = READYSTATE_COMPLETE
DoEvents
Loop
Application.Wait (Now + TimeValue("00:00:25"))
For N = 0 To Ie.document.getElementsByClassName("section-subheader-header GLOBAL__gm2-subtitle-alt-1").Length - 1
If Ie.document.getElementsByClassName("section-subheader-header GLOBAL__gm2-subtitle-alt-1").Item(N).innerText = "Web results" Then
Ie.document.getElementsByClassName("section-subheader-header GLOBAL__gm2-subtitle-alt-1").Item(N).ScrollIntoView (False)
End If
Next N
Application.Wait (Now + TimeValue("00:00:25"))
Set Docx = Ie.document
productDesc = Docx.Window.frames("section-iframe-iframe").contentWindow.document.getElementsByClassName("trex")(0).outerHTML
End Sub
Here is the HTML:
Please help to resolve this error.
I want to extract "trex" ClassName HTML Contain from above url
Thanks.
You can change the line of extract "trex" element to one of the following, both of them can work well:
Use the getElementsbyTagName method to get the Iframe first , then according to the Iframe.contentDocument property to reach the element via the class name:
productDesc = Docx.getElementsByTagName("iframe")(0).contentDocument.getElementsByClassName("trex")(0).outerHTML
Use querySelector method to get the Iframe through class, then use the same as the above to reach the element:
productDesc = Docx.querySelector(".section-iframe-iframe").contentDocument.getElementsByClassName("trex")(0).outerHTML
I am a beginner to web scraping with excel vba and need some help.
I am trying to reference an element. If there was an id then I could use getElementByID but sometimes there is no id. I could use getElementByClassName but sometimes there are too many elements of the same class.
Is there some way to refer to an element by xpath?
(I can't post the actual website since there is personal info so let us say this is the html)
<!DOCTYPE html>
<html>
<body>
Link
</body>
</html>
is there something like ie.document.getElementByXPath.(/html/body/a).click?
I've searched all over the web and can't seem to find anything on the topic.
this is not meant to be an answer
here is a couple of subs that may give you some ideas
Sub google()
' add reference: Microsoft XML v6.0
Const url = "https://www.google.co.in"
Dim http As New XMLHTTP60
Dim html As New HTMLDocument
http.Open "GET", url, False
http.Send
html.body.innerHTML = http.responseText
Dim elem As Object
Set elem = html.getElementsByClassName("ctr-p") ' HTMLElementCollection
Debug.Print elem.Length
Set elem = html.getElementsByClassName("ctr-p")("viewport") ' HTMLDivElement <div class="ctr-p" id="viewport">
Debug.Print elem.Children.Length
Dim aaa As Object
Set aaa = elem.getElementsByTagName("div")("hplogo") ' HTMLDivElement
Debug.Print aaa.Children.Length
Debug.Print aaa.outerHTML
End Sub
.
' add references Microsoft HTML Object Library
' Microsoft Internet Controls
Sub ieGoogle()
Const url = "https://www.google.co.in"
Dim iE As InternetExplorer
Set iE = New InternetExplorer
iE.Navigate url
iE.Visible = True
Do While iE.ReadyState <> 4: DoEvents: Loop
Dim doc As HTMLDocument
Set doc = iE.Document
Debug.Print doc.ChildNodes.Length ' DOMChildrenCollection
Debug.Print doc.ChildNodes(1).ChildNodes.Item(0).nodeName ' HEAD
Debug.Print doc.ChildNodes(1).ChildNodes.Item(1).nodeName ' BODY
' for querySelector arguments see: https://www.w3schools.com/cssref/css_selectors.asp
Dim elm As HTMLInputElement
Set elm = doc.querySelector("*") ' all elements
Debug.Print Left(elm.outerHTML, 40)
Set elm = doc.querySelector("div.ctr-p#viewport") ' <div class="ctr-p" id="viewport">
Debug.Print Left(elm.outerHTML, 40)
Set elm = doc.querySelector(".ctr-p#viewport") ' <div class="ctr-p" id="viewport">
Debug.Print Left(elm.outerHTML, 40)
Debug.Print elm.ChildNodes.Length
Debug.Print elm.Children.Length
Set elm = doc.querySelector("#viewport") ' id="viewport"
Debug.Print Left(elm.outerHTML, 40)
Debug.Print elm.ID
Dim elem As HTMLInputElement
Set elem = doc.getElementsByClassName("ctr-p")("viewport")
Debug.Print elem.Children.Length
Dim aaa As Object
Set aaa = elem.getElementsByTagName("div")("hplogo")
Debug.Print aaa.Children.Length
Debug.Print aaa.outerHTML
iE.Quit
Set iE = Nothing
End Sub
You can do this in Excel VBA using Selenium Webdriver (https://www.selenium.dev/).
Webdriver does have a FindElementByXPath method. It has the advantage of providing control of other browser than Internet Explorer, but the disadvantage will be the need to install Selenium in every machine which will run your VBA script.
Here is a walkthrough for installing Selenium and adding its library reference to your project (this was the tutorial I used; it's a pt-br page, but I put into google for automatic translation): https://translate.google.com/translate?sl=pt&tl=en&u=https%3A%2F%2Fwww.tomasvasquez.com.br%2Fblog%2Fmicrosoft-office%2Fexcel%2Fvba-interagindo-com-paginas-web-com-o-selenium-webdriver%2F
And here is another quickstart from Coding is Love (it doesn't have the installation walkthrough): https://codingislove.com/browser-automation-in-excel-selenium/
My goal is to scrape the source code of a web page.
The site seems to have different Frames which is why my code won't work properly.
I tried to modify a code which I found online which should solve the Frame issue.
The following code creates an error (object required) at:
Set profileFrame .document.getElementById("profileFrame")
Public Sub IE_Automation()
'Needs references to Microsoft Internet Controls and Microsoft HTML Object Library
Dim baseURL As String
Dim IE As InternetExplorer
Dim HTMLdoc As HTMLDocument
Dim profileFrame As HTMLIFrame
Dim slotsDiv As HTMLDivElement
'example URL with multiple frames
baseURL = "https://www.xing.com/search/members?section=members&keywords=IT&filters%5Bcontact_level%5D=non_contact"
Set IE = New InternetExplorer
With IE
.Visible = True
'Navigate to the main page
.navigate baseURL & "/publictrophy/index.htm?onlinename=ace_anubis"
While .Busy Or .readyState <> READYSTATE_COMPLETE: DoEvents: Wend
'Get the profileFrame iframe and navigate to it
Set profileFrame = .document.getElementById("profileFrame")
.navigate baseURL & profileFrame.src
While .Busy Or .readyState <> READYSTATE_COMPLETE: DoEvents: Wend
Set HTMLdoc = .document
End With
'Display all the text in the profileFrame iframe
MsgBox HTMLdoc.body.innerText
'Display just the text in the slots_container div
Set slotsDiv = HTMLdoc.getElementById("slots_container")
MsgBox slotsDiv.innerText
End Sub
Hummmm, I'm not exactly sure what you are doing here, but can you try the code below?
Option Explicit
Sub Sample()
Dim ie As Object
Dim links As Variant, lnk As Variant
Dim rowcount As Long
Set ie = CreateObject("InternetExplorer.Application")
ie.Visible = True
ie.navigate "https://www.xing.com/search/members?section=members&keywords=IT&filters%5Bcontact_level%5D=non_contact"
'Wait for site to fully load
'ie.Navigate2 URL
Do While ie.Busy = True
DoEvents
Loop
Set links = ie.document.getElementsByTagName("a")
rowcount = 1
With Sheets("Sheet1")
For Each lnk In links
'Debug.Print lnk.innerText
'If lnk.classname Like "*Real Statistics Examples Part 1*" Then
.Range("A" & rowcount) = lnk.innerText
rowcount = rowcount + 1
'Exit For
'End If
Next
End With
End Sub
General:
I think in your research you may have come across this question and misunderstood how it relates/doesn't relate to your circumstance.
I don't think iFrames are relevant to your query. If you are after the list of names, their details and the URLs to their pages you can use the code below.
CSS Selectors
To target the elements of interest I use the following two CSS selectors. These use style infomation on the page to target the elements:
.SearchResults-link
.SearchResults-item
"." means class, which is like saying .getElementsByClassName. The first gets the links, and the second gets the description information on the first page.
With respect to the first CSS selector: The actual link required is dynamically constructed, but we can use the fact that the actual profile URLs have a common base string of "https://www.xing.com/profile/", which is then followed by the profileName. So, in function GetURL, we parse the outerHTML returned by the CSS selector to get the profileName and concatenate it with the BASESTRING constant to get our actual profile link.
Code:
Option Explicit
Public Sub GetInfo()
Dim IE As New InternetExplorer
With IE
.Visible = True
.navigate "https://www.xing.com/publicsearch/query?search%5Bq%5D=IT"
While .Busy Or .readyState < 4: DoEvents: Wend
Dim a As Object, exitTime As Date, linksNodeList As Object, profileNodeList As Object
' exitTime = Now + TimeSerial(0, 0, 5) '<== uncomment this section if timing problems
'
' Do
' DoEvents
' On Error Resume Next
' Set linksNodeList = .document.querySelectorAll(".SearchResults-link")
' On Error GoTo 0
' If Now > exitTime Then Exit Do
' Loop While linksNodeList Is Nothing
Set linksNodeList = .document.querySelectorAll(".SearchResults-link") '<== comment this out if uncommented section above
Set profileNodeList = .document.querySelectorAll(".SearchResults-item")
Dim i As Long
For i = 0 To profileNodeList.Length - 1
Debug.Print "Profile link: " & GetURL(linksNodeList.item(i).outerHTML)
Debug.Print "Basic info: " & profileNodeList.item(i).innerText
Next i
End With
End Sub
Public Function GetURL(ByVal htmlSection As String) As String
Const BASESTRING As String = "https://www.xing.com/profile/"
Dim arr() As String
arr = Split(htmlSection, "/")
GetURL = BASESTRING & Replace$(Split((arr(UBound(arr) - 1)), ">")(0), Chr$(34), vbNullString)
End Function
Example return information:
I've been banging my head against he wall trying to figure out why this VBA code will not work :(
I am simply trying to insert the value entered into the excel input box into a website's input box. I am a novice when it comes to HTML so I'm sure that has something to do with it.
Here is the HTML element from the website Zomato.com:
<input class="dark" id="location_input" role="combobox" aria-expanded="true" aria-labelledby="label_search_location" aria-owns="explore-location-suggest" aria-autocomplete="list" placeholder="Please type a location...">
Here is my VBA code:
Sub Merchant_Extraction()
Dim IE As Object
Dim form As Variant
Dim button As Variant
Set IE = CreateObject("internetexplorer.application")
merchantzip = InputBox("Enter Zip Code")
With IE
.Visible = True
.navigate ("http://www.zomato.com")
While IE.readystate <> 4
DoEvents
Wend
IE.Document.GetElementByID(“location_input_sp”).Item.innertext = merchantzip
Set form = IE.Document.getelementsbytagname("form")
Set button = form(0).onsubmit
form(0).get
End With
Set IE = Nothing
End Sub
I am unclear why it's not working - any help would be incredible!
API XMLHTTP GET request
The API was mentioned. The documention is here.
The basic free account allows access to restaurant information and search APIs (up to 1000 calls/day).
An example first 0-20 results call, with a city id specified (68 for Manchester,UK), is as follows; a JSON response is received. The response it parsed into a JSON object with JSONConverter.bas
Option Explicit
Public Sub GetInfo()
Dim URL As String, strJSON As String, json As Object
URL = "https://developers.zomato.com/api/v2.1/search?entity_id=68&entity_type=city&start=0&count=20"
With CreateObject("MSXML2.XMLHTTP")
.Open "GET", URL, False
.setRequestHeader "Content-Type", "application/json"
.setRequestHeader "user-key", "yourAPIkey"
.send
strJSON = .responseText
End With
Set json = JsonConverter.ParseJson(strJSON)
'other stuff with JSON object
End Sub
Example JSON response:
Zomato - Common APIs:
Finding your city ID:
The quickest way for me was to go to concatenate the city onto a base URL string e.g. https://www.zomato.com/manchester, then click search and right-click inspect HTML on first result. Then Ctrl+F to bring up search box, search for CITY_ID, and scan through find results for the HTML until city id found e.g.
As far as entering values into webpages the correct syntax would be:
IE.Document.all.Item("location_input").Value = ""
I've combined your routine with some code that I use so you can see an example. I have not been able to test however. In my environment, the IE object disconnects after the .navigate portion so I added in a loop to find and re-assign the object...
Option Explicit
Public Declare Sub Sleep Lib "kernel32" (ByVal dwMilliseconds As Long)
Sub Merchant_Extraction()
Dim IE As Object, objShellWindows As Object
Dim MerchantZip As String, strWebPath As String
Dim Form As Variant, Button As Variant
Dim X As Long
strWebPath = "http://www.zomato.com"
MerchantZip = InputBox("Enter Zip Code")
If MerchantZip = vbNullString Then Exit Sub
Set IE = CreateObject("InternetExplorer.Application")
With IE
.Visible = True
.Navigate strWebPath
End With
Do
Sleep 250
DoEvents
Loop While IE.Busy Or IE.ReadyState <> 4
If TypeName(IE) <> "IWebBrowser2" Or IE.Name <> "Internet Explorer" Then
Set objShellWindows = CreateObject("Shell.Application").Windows
For X = 0 To objShellWindows.Count - 1
Set IE = objShellWindows.Item(X)
If Not IE Is Nothing Then
If IE.Name = "Internet Explorer" Then
If InStr(1, IE.LocationURL, strWebPath, 1) > 0 Then
Do While IE.Busy Or IE.ReadyState <> 4
Sleep 250
DoEvents
Loop
Exit For
End If
End If
End If
Set IE = Nothing
Next
Set objShellWindows = Nothing
End If
If Not IE Is Nothing Then
IE.Document.all.Item("location_input").Value = MerchantZip
Sleep 250
For Each Button In IE.Document.getelementsbytagname("form")
If StrComp(Button.Type, "Button", 1) = 0 Then
Button.Click
End If
Next
Set IE = Nothing
End If
End Sub
Using this HTML code for example:
<table class="table-grid">
<tr>
<th>auto.model</th>
<td>
<pre>'Toyota Avensis Wagon'</pre>
</td>
</tr>
<tr>
<th>auto.year</th>
<td>
<pre>2005</pre>
</td>
</tr>
</table>
If I take the parameter "auto.model" between <th></th> tags and want to receive "Toyota Avensis Wagon", i.e. the next expression between <pre></pre>. Ideally I'd like to have function to do it.
Thank you #Jeeped, but code raise "Type mismatch" error and points to Set el = Param.PreviousSibling:
Sub Extract_TD_text()
Dim URL As String
Dim IE As InternetExplorer
Dim HTMLdoc As HTMLDocument
Dim Params As IHTMLElementCollection
Dim Param As HTMLTableCell
Dim Val As HTMLTableCell
Dim r As Long
Dim el As HTMLTableCell
URL = "My URL"
Set IE = New InternetExplorer
With IE
.navigate URL
.Visible = False
'Wait for page to load
While .Busy Or .READYSTATE <> READYSTATE_COMPLETE: DoEvents: Wend
Set HTMLdoc = .document
End With
Set Params = HTMLdoc.getElementsByTagName("tr")
For Each Param In Params
If Param.innerText Like "*auto.model*" Then
Set el = Param.PreviousSibling
Exit For
End If
Next
If Not el Is Nothing Then Debug.Print el.innerText
IE.Quit
Set IE = Nothing
End Sub
Instead of using previousSibling, I'd like to suggest nextElementSibling.
From the way your HTML and VBA codes are currently set up, the current 'param' value being passed should be the <th> tag. I think previousSibling would likely check the tag that comes before that, and since is the first element within the <tr> (the parent element), there shouldn't be anything (except maybe an invisible node- which previousSibling can find, but that we don't need).
I think nextElementSibling should be able to find your <td> tag, since it comes after your <th> tag.