VBA - getElementById works for simple website but not another? - html

So I'm currently stuck at getting a VBA script to retrieve the value of an input box from this Sudoku website. However, I was able to get the value from a paragraph element with the id of "contact" from my own simpler website, using the same code (after switching the url and id names, of course).
Any attempts to research further brings up articles/blogs that discuss what I've done correctly so far, so I suspect I am not researching it properly.
Here is my code:
Sub GetTable()
Dim ieApp As InternetExplorer
Dim ieDoc As Object
Dim sudokuCell As Object
Dim url, id, content As String
Dim i As Integer
Set ieApp = New InternetExplorer
ieApp.Visible = True
url = "http://www.websudoku.com/"
ieApp.navigate url
Do While ieApp.Busy: DoEvents: Loop
Do Until ieApp.READYSTATE = READYSTATE_COMPLETE: DoEvents: Loop
Set ieDoc = ieApp.document
If ieDoc Is Nothing Then
MsgBox ("Nothing")
'Else
' MsgBox ("Something")
End If
For i = 0 To 8
Set sudokuCell = ieDoc.getElementById("f00")
content = sudokuCell.innerText
MsgBox (content)
Next i
ieApp.Quit
Set ieApp = Nothing
End Sub
And here is an example of the html for a cell which is blank:
<td class="g0" id="c00"><input class="d0" size="2" autocomplete="off"
name="8iz6n11" maxlength="1" onblur="j8(this)" id="f00"></td>
And here is one for cell that is prefilled with a number:
<td class="f0" id="c10"><input class="s0" size="2" autocomplete="off"
name="s8iz6n21" readonly="" value="7" id="f10"></td>
I have tried both the "c00" an "f00" without success. Also, while I believe the problem at hand is I am not retrieving the element, I am concerned that the .innerText property won't retrieve the values.

First: The website is using FRAME, so you are not accessing the frame document in the VBA code actually. You need to navigate to the actual URL given below - change your url variable as the following (which is the frame's src property):
url = "http://view.websudoku.com/?"
Second: Those are INPUT elements you are trying to get values, you should be better using Value property instead innerText
content = sudokuCell.Value
Third and last: I have no idea what your code is supposed to do inside the loop as it will keep reading f00 element value as is. However I believe you'll loop through the input elements and just hit the wall here about the FRAME issue I explained above, so I assume loop is your part and have no trouble about it.

Related

Website scraping: website search box has no value

I am trying to crosscheck a large body of data with a specific website (https://icis.corp.delaware.gov/Ecorp/EntitySearch/NameSearch.aspx).
The goal is to search for many company names based on a larger list in Excel to get their founding dates. For now I am starting out with a single name to get it running. I am having trouble in my main code as there is no inherent input value in the HTML code:
<input name="ctl00$ContentPlaceHolder1$frmEntityName" type="text" id="ctl00_ContentPlaceHolder1_frmEntityName" tabindex="4" size="30" maxlength="120" class="txtNormal" onkeyup="KeyEvent1(this.id)">
I tried the following:
Sub click_search()
Dim i As SHDocVw.InternetExplorer
Set i = New InternetExplorer
i.Visible = True
i.Navigate "https://icis.corp.delaware.gov/Ecorp/EntitySearch/NameSearch.aspx"
Do While i.ReadyState <> READYSTATE_COMPLETE
Loop
Dim idoc As MSHTML.HTMLDocument
Set idock = i.Document
idoc.getElementsByTagName("input").Item("ctl00$ContentPlaceHolder1$frmEntityName").Value = "10X Genomics Inc"
End Sub
The problem I believe is the HTML code does not have inherent value = "" to begin with but it only comes up in the HTML code after you write it in.
How do I fix this and furthermore then click the search button?
The error is
"Object variable or With block variable not set"
Always use Option Explicit at the top of every VBA code file.
If the webpage in question contains ids for the elements you are interested in, use getElementById() to access them. This code works, however it does not find any records.
Option Explicit
Sub click_search()
Dim i As SHDocVw.InternetExplorer
Dim idoc As MSHTML.HTMLDocument
Set i = New InternetExplorer
i.Visible = True
i.Navigate "https://icis.corp.delaware.gov/Ecorp/EntitySearch/NameSearch.aspx"
Do While i.ReadyState <> READYSTATE_COMPLETE
Loop
Set idoc = i.Document
idoc.getElementById("ctl00_ContentPlaceHolder1_frmEntityName").Value = "10X Genomics Inc"
idoc.getElementById("ctl00_ContentPlaceHolder1_frmFileNumber").Value = "1"
idoc.getElementById("ctl00_ContentPlaceHolder1_btnSubmit").Click
End Sub

VBA Web search button - GetElementsbyClassName

I have a problem with the VBA code.
I would like to open this website: https://www.tnt.com/express/en_us/site/tracking.html and in Shipment numbers search box I would like to put active cells from Excel file. At the beginning I tried to put only a specific text for example: "777777".
I wrote the below code but unfortunately, the search button is empty and there is no error. I tried everything and I have no idea what should I change in my code.
Any clues? Thank you in advance.
HTML:
<input class="__c-form-field__text ng-touched ng-dirty ng-invalid" formcontrolname="query" pbconvertnewlinestocommasonpaste="" pbsearchhistorynavigation="" shamselectalltextonfocus="" type="search">
VBA:
Sub TNT2_tracker()
Dim objIE As InternetExplorer
Dim aEle As HTMLLinkElement
Dim y As Integer
Dim result As String
Set objIE = New InternetExplorer
objIE.Visible = True
objIE.navigate "https://www.tnt.com/express/en_us/site/tracking.html"
Do While objIE.Busy = True Or objIE.readyState <> 4: DoEvents: Loop
Dim webpageelement As Object
For Each webpageelement In objIE.document.getElementsByClassName("input")
If webpageelement.Class = "__c-form-field__text ng-pristine ng-invalid ng-touched" Then
webpageelement.Value = "777"
End If
Next webpageelement
End Sub
You could use the querySelector + class name to find an element.
something like
'Find the input box
objIE.document.querySelector("input.__c-form-field__text").value = "test"
'Find the search button and do a click
objIE.document.querySelector("button.__c-btn").Click
No need to loop through elements. Unless the site allows you to search multiple tracking numbers at the same time.
It seems automating this page is a litte tricky. If you change the value of the input field it doesn' t work. Nothing happen by clicking the submit button.
A look in the dom inspector shows several events for the input field. I checked them out and it seems we need to paste the value over the clipboard by trigger the paste event of the shipping field.
In order for this to work without Internet Explorer prompting, its security settings for the Internet zone must be set to allow pasting from the clipboard. I'm using a German version of IE, so I have problems explaining how to find the setting.
This macro works for me:
Sub TNT2_tracker()
Dim browser As Object
Dim url As String
Dim nodeDivWithInputField As Object
Dim nodeInputShipmentNumber As Object
Dim textToClipboard As Object
'Dataobject by late binding to use the clipboard
Set textToClipboard = CreateObject("New:{1C3B4210-F441-11CE-B9EA-00AA006B1A69}")
url = "https://www.tnt.com/express/en_us/site/tracking.html"
'Initialize Internet Explorer, set visibility,
'call URL and wait until page is fully loaded
Set browser = CreateObject("internetexplorer.application")
browser.Visible = True
browser.navigate url
Do Until browser.ReadyState = 4: DoEvents: Loop
'Manual break for loading the page complitly
'Application.Wait (Now + TimeSerial(pause_hours, pause_minutes, pause_seconds))
Application.Wait (Now + TimeSerial(0, 0, 3))
'Get div element with input field for shipment number
Set nodeDivWithInputField = browser.Document.getElementsByClassName("pb-search-form-input-group")(0)
If Not nodeDivWithInputField Is Nothing Then
'If we got the div element ...
'First child element is the input field
Set nodeInputShipmentNumber = nodeDivWithInputField.FirstChild
'Put shipment number to clipboard
textToClipboard.setText "7777777"
textToClipboard.PutInClipboard
'Insert value by trigger paste event of the input field
Call TriggerEvent(browser.Document, nodeInputShipmentNumber, "paste")
'Click button
browser.Document.getElementsByClassName("__c-btn")(0).Click
Else
MsgBox "No input field for shipment number found."
End If
End Sub
And this function to trigger a html event:
Private Sub TriggerEvent(htmlDocument As Object, htmlElementWithEvent As Object, eventType As String)
Dim theEvent As Object
htmlElementWithEvent.Focus
Set theEvent = htmlDocument.createEvent("HTMLEvents")
theEvent.initEvent eventType, True, False
htmlElementWithEvent.dispatchEvent theEvent
End Sub
As #Stavros Jon alludes to..... there is a browserless way using xhr GET request via API. It returns json and thus you ideally need to use a json parser to handle the response.
I use jsonconverter.bas as the json parser to handle the response. Download raw code from here and add to standard module called JsonConverter . You then need to go VBE > Tools > References > Add reference to Microsoft Scripting Runtime. Remove the top Attribute line from the copied code.
Example request with dummy tracking number (deliberately passed as string):
Option Explicit
Public Sub TntTracking()
Dim json As Object, ws As Worksheet, trackingNumber As String
trackingNumber = "1234567" 'test input value. Currently this is not a valid input but is for demo.
Set ws = ThisWorkbook.Worksheets("Sheet1") 'for later use if writing something specific out
With CreateObject("MSXML2.XMLHTTP")
.Open "GET", "https://www.tnt.com/api/v3/shipment?con=" & trackingNumber & "&searchType=CON&locale=en_US&channel=OPENTRACK", False
.send
Set json = JsonConverter.ParseJson(.responseText)
End With
'do something with results
Debug.Print json("tracker.output")("notFound").Count > 0
Debug.Print JsonConverter.ConvertToJson(json("tracker.output")("notFound"))
End Sub

Prompted with Object doesn't support his property or method

I am trying to update the 1 of the field on the right section after selecting 1 of the button on left section but was prompted with the run-time error 438 code.
I have tried changing the element and attribution of the last row of my code but nothing seems to work.
Below is part of my VBA script:
Sub BrowseToWebTest1()
Dim ie As Object
Dim the_button_elements As Object
Dim button_element As Object
Dim radioButton As Object
Dim radioButtons As Object
Dim doc As HTMLDocument
Set ie = New InternetExplorerMedium
ie.navigate "company system web"
ie.Visible = True
While ie.Busy
DoEvents
Wend
Set doc = ie.document
Set the_button_elements = doc.getElementsByTagName("button")
For Each button_element In the_button_elements
If button_element.getAttribute("onclick") = "CreateAcqCase();" Then
button_element.Click
Exit For
End If
Next button_element
Call doc.getElementByName(“TransactionID”).setAttribute(“value”, “test”)
Below is the DOM Explorer code:
<input name="$PAcqCaseCreation$pTransactionID" class="leftJustifyStyle" id="TransactionID" style="width: 175px;" type="text" maxlength="15" value="" data-ctl='["TextInput"]' minchars="15" validationtype="minchars" data-changed="false">
Hopefully someone call help so that i can update fields accordingly. By the way I am using IE11 and Window 10
1) You have a mistake here:
doc.getElementByName(“TransactionID”).setAttribute(“value”, “test”)
The method is getElementsByName , notice the s indicating plural - a collection is returned. As it is a collection, you will need to supply an appropriate index to target the element of interest.
2) Also, you have introduced smart “ where you want ".
3) Neither call keyword, nor parantheses are needed.
4) The name attribute is:
name="$PAcqCaseCreation$pTransactionID"
Whereas the id attribute is:
id="TransactionID"
id is likely unique and a better selector (and is singular, so no s or index):
doc.getElementId("TransactionID").setAttribute "value", "test"
Otherwise,
doc.getElementsByName("$PAcqCaseCreation$pTransactionID")(0).setAttribute "value", "test"
This would be assuming first element in collection is the correct; otherwise, change the index.
5) You can replace all this (and remove the associated declarations):
Set the_button_elements = doc.getElementsByTagName("button")
For Each button_element In the_button_elements
If button_element.getAttribute("onclick") = "CreateAcqCase();" Then
button_element.Click
Exit For
End If
Next button_element
With one line:
doc.querySelector("[onclick='CreateAcqCase();']").Click

HTML & VBA Visual Basic automatic input website

My company has a "link" I can go to... http://blablabla... where there is a box where I can enter an 8 digit number (next to the box it says lot name) and a button where I can click view report. That's it. That's all that's on the page. (input box and a view report button)
I have a list of numbers in excel in cell range B3:B7 for example (10378851, 10378857, 10488213, etc. )
I want to open excel and run a code that automatically goes to the website, automatically inputs the first number, automatically "clicks" the view report button.
Then there will be an "export data button" - I want to automatically click which will have a drop down (csv, txt, excel, etc.) I want to automatically select "excel"
I found a site that explains how to do something similar but I am stuck.
Sub GetTable()
Dim ieApp As InternetExplorer
Dim ieDoc As Object
Dim ieTable As Object
Dim clip As DataObject
'create a new instance of ie
Set ieApp = New InternetExplorer
'you don’t need this, but it’s good for debugging
ieApp.Visible = True
'assume we’re not logged in and just go directly to the login page
ieApp.Navigate "http://severe-frost-552.heroku.com/login"
Do While ieApp.Busy: DoEvents: Loop
Do Until ieApp.ReadyState = READYSTATE_COMPLETE: DoEvents: Loop
Set ieDoc = ieApp.Document
'fill in the login form – View Source from your browser to get the control names
With ieDoc.forms(0)
.login.Value = "dailydose"
.Password.Value = "password"
.submit
End With
Do While ieApp.Busy: DoEvents: Loop
Do Until ieApp.ReadyState = READYSTATE_COMPLETE: DoEvents: Loop
'now that we’re in, the rest of this code isn't relevant because it copied a table of data...
Here is the website...
http://dailydoseofexcel.com/archives/2011/03/08/get-data-from-website-that-requires-a-login/
As you can see, there source code says:
<form action="/session" method="post"><div style="margin:0;padding:0;display:in...
<input id="login" name="login" type="text" /></p>
I think that is where they are getting this part of their code:
With ieDoc.forms(0)
.login.Value = "dailydose"
Looking at my websites source code, it looks like I have:
<DIV id=ReportViewerControl_ctl04_ctl03><INPUT id=ReportViewerControl_ctl04_ctl03_txtValue class=null style= size=30 name=ReportViewerControl$ctl04$ctl03$txtValue> </DIV>
<INPUT id=ReportViewerControl_ctl04_ctl03_txtValue class=null style= size=30 name=ReportViewerControl$ctl04$ctl03$txtValue>
I think this is the right area of the code because when I type "hello" into the input box it shows up there like this:
<INPUT id=ReportViewerControl_ctl04_ctl03_txtValue class=null style= size=30 value=Hello name=ReportViewerControl$ctl04$ctl03$txtValue>
This is much different than the simple example... I am not sure how to code this input in my code...
you can try this:
ieDoc.ReportViewerControl_ctl04_ctl03_txt.Value = "Hello"
and if that does not work:
ieDoc.getElementById("ReportViewerControl_ctl04_ctl03_txt").Value = "Hello"
disclaimer: I have not tested the code
hint: what you are "playing" with is called "web-scraping". You can find decent examples here

HTML object library / pull

I have the following code in an HTML web page, and I am trying to use the html object library via vba engine to pull the value from within this tag:
<input name="txtAdd_Line1" disabled="disabled" size="30" maxLength="50" value="123 N 1ST ST"/>
I figure I have to use .getelementsbytagname or .getelementsbyname, but I am not sure how to grab the value. Does anyone have any ideas?
Here's an example with comments, subtitute in your actual address:
Sub Example()
'Declare needed variables
Dim ie, elements
Dim x As Long
'Create IE Applction
Set ie = CreateObject("InternetExplorer.Application")
'Navigate to the website
ie.navigate "C:\test.html" 'Substitute your actual address
'Wait for website to finish loading
Do While ie.ReadyState <> 4
Loop
'Find the elements
Set elements = ie.document.getelementsbyName("txtAdd_Line1")
'Display the value of each returned element
For x = 0 To elements.Length - 1
MsgBox elements(x).Value
Next
'Quit IE
ie.Quit
End Sub
Based on your comment most likely just looking at the document wasn't retrieving the actual layer of the tree you wanted, try this:
Set HTMLDoc = ie.document.frames("MainFrame").document
With HTMLDoc
'This returns an (object) which contains an array of all matching elements
a = .getElementsByName("txtAdd_Line1")
end with
For x = 0 to a.length
msgbox a(x).value
next
You can use a CSS selector of input[name='txtAdd_Line1'] . This says element with input tag having attribute name with value 'txtAdd_Line1'.
CSS selector:
You apply a CSS selector using the .querySelector method of document e.g.
Msgbox ie.document.querySelector("input[name='txtAdd_Line1']").innerText