VBA to scrape the value from a hidden field - html

I'm new to scraping HTML and have been trying for several days now to scrape the value from a hidden field using the class name and the id, but I still can't get the value.
I'm trying to get the value (4) from the following HTML;
<input id="collectionQuantity" type="hidden" value="4">
This is taken from larger extract below;
<div class="lg-24 md-12 cols">
<input id="selectedBranchCode" type="hidden" value="OT4">
<input id="selectedBranchName" type="hidden" value="Ashton-under-Lyne">
<input id="collectionQuantity" type="hidden" value="4">
<button id="add_for_collection_button_3730P" title="Click here to add this item to your basket for collection" class="btn btn--lg btn--col fill " data-content="add-to-basket">Click & Collect</button>
<p id="branch_collection_3730P">4 in stock in <strong>Ashton-under-Lyne</strong> Change store</p>
</div>
I've tried sooooo many ways of getting the value.
The one's I think I've got closest with are;
sh01.Cells(r, 5) = HTML.getElementsByClassName("lg-24 md-12 cols")(3).innertext 'product stock
sh01.Cells(r, 5) = HTML.getElementsByTagName("p")(7).innertext 'product stock
sh01.Cells(r, 5) = HTML.getElementById("branch_collection_" & z_sh01.Cells(y, 2)).innertext 'product stock
sh01.Cells(r, 5) = HTML.getElementsByClassName("lg-24 md-12 cols")(3).getElementById("collectionQuantity").Value 'product stock
sh01.Cells(r, 5) = HTML.querySelector("# branch_collection_" & z_sh01.Cells(y, 2)).innertext 'product stock
sh01.Cells(r, 5) = HTML.getElementById("collectionQuantity").innertext 'product stock
Thanks in advance for your help.
Ian

Try
HTML.querySelector("#collectionQuantity").Value
Or
HTML.getElementById("collectionQuantity").getAttribute("value")
Or even
HTML.getElementById("collectionQuantity").Value
You are after the value of the value attribute of the target element rather than the .innerText . The above shows 3 ways to do that.
So, I had to set a local store and then add a different navigate first to ensure the local store was set before going to the page of interest, otherwise the store is left blank and thus the .value of the element in question is also blank
Option Explicit
Public Sub GetInfo()
Dim IE As New InternetExplorer
With IE
.Visible = True
.Navigate2 "https://www.screwfix.com/jsp/tradeCounter/tradeCounterDetailsPage.jsp?id=460"
While .Busy Or .readyState < 4: DoEvents: Wend
.document.querySelector("input.btn").Click
While .Busy Or .readyState < 4: DoEvents: Wend
.Navigate2 "https://www.screwfix.com/p/hd907-9q-freestanding-oil-filled-radiator-2000w/3730p"
While .Busy Or .readyState < 4: DoEvents: Wend
With .document
Debug.Print .getElementById("collectionQuantity").Value
End With
.Quit
End With
End Sub

Related

Hidden Button Click in HTML through VBA

<tr>
**<td>**JB - Al Qouz**</td>**
**<td>**
<strong>0</strong>
<input type="hidden" name="ingredient[inventories_attributes][][id]" id="ingredient_inventories_attributes__id" value="5564069">
<small><span class="buyUnitValueText ">pc</span></small>
<small><b><span id="totalStockValue_1" class="hide paddingLeft15 updateTotalStockValue"></span></b></small>
**</td>**
**<td>**
<div class="align-left">
<input type="text" name="ingredient[inventories_attributes][][reorder_level]" id="ingredient_inventories_attributes__reorder_level" value="0" class="span5 menuItemInventory reOrderValue">
<small><span class="buyUnitValueText span7 labelText pull-right align-left">pc</span>
</small>
</div>
**</td>**
<td class="align-left">
<input type="hidden" name="ingredient[inventories_attributes][][track_inventory]" id="track_inventory_9971" **value="true"**>
**<div class="vCheckBox trackInventoryCheckbox active" id="9971"></div>**
I need to change 2 things (See Bold) if "id=9971" through vba.
Set class vCheckBox trackInventoryCheckbox active to class vCheckBox trackInventoryCheckbox
Set value="true" to value="false"
Reason being, I'm not able to click the button through VBA and the code change in html that occurs when I click manually are the 1. & 2. mentioned above.
Please advise, below is my code where I tried to get tag value through tr but couldn't, second part was where I could change the values.
With ie
.Visible = True
.navigate (EditIngredientURL)
While .Busy Or .readyState <> 4: DoEvents: Wend
While .document.readyState <> "complete": DoEvents: Wend
End With
Set HTMLdoc = ie.document
Set tr = HTMLdoc.getElementsByTagName("tr")
For Each trObj In tr
Set td = trObj.getElementsByTagName("td")
For Each tdObj In td
If InStr(tdObj.innerText, "JB - Al Qouz") Then
Debug.Print tdObj.innerText
Set CheckBox = tdObj.getElementsByTagName("class")
For Each idnumber In CheckBox
Debug.Print idnumber.innerText
Exit For
Next
End If
Next
Next
End Sub
If the ID is known and you have it in a variable idNo you can use it to locate the elements:
idNo = "9971"
HTMLdoc.getElementById("track_inventory_" & idNo).value = "false"
HTMLdoc.getElementById(idNo).className = "vCheckBox trackInventoryCheckbox"

Click on Checkbox in Website using Excel VBA

There is a website that has some data I want for my work. I login check some checkbox and submit, then the website will send me an email with my data. But it takes way too long to receive my information so I want to automate it so everyday first hour in the morning the information is requested, like that I will already have it on my email when I need it.
I modified some code and I already managed to login and go to the page where I need to click the checkbox (there are three of them) and then click on a submit button.
Sub GetTable()
Dim ieApp As InternetExplorer
Dim ieDoc As Object
Dim ieTable As Object
Dim chkBox As Object
'create a new instance of ie
Set ieApp = New InternetExplorer
'you don’t need this, but it’s good for debugging
ieApp.Visible = True
'assume we’re not logged in and just go directly to the login page
ieApp.Navigate "http://mydata/aspx/mydataLogon.aspx?Language=2"
Do While ieApp.Busy: DoEvents: Loop
Do Until ieApp.ReadyState = READYSTATE_COMPLETE: DoEvents: Loop
Set ieDoc = ieApp.Document
'fill in the login form – View Source from your browser to get the control names
With ieDoc.forms(0)
.txtUsername.Value = "usertest"
.txtPassword.Value = "test123"
.Submit
End With
Do While ieApp.Busy: DoEvents: Loop
Do Until ieApp.ReadyState = READYSTATE_COMPLETE: DoEvents: Loop
'now that we’re in, go to the page we want
ieApp.Navigate "http://mydata/InventoryTracking/aspx/rptDefault.aspx?rpt=../aspx/invRptDetailReport.aspx"
Set ieDoc = ieApp.Document
With ieDoc
'I tried with getElementbyId, I dont get any error but the chkbox still comes as with a nothing value
Set chkBox = ieDoc.getElementByid("cblPlants_0")
'therefore these give error too, I dont know why. A an object variable or with block variable not set error
chkBox.Click
chkBox.Checked = True
'here I get an object variable or with block variable not set error
ieDoc.getElementByid("cblPlants_0").Click
.Submit
End With
End sub
The checkbox control I want is inside of this:
'first there is a form and this is how the iframe is first called I think
<form name="_ctl1" method="post" action="rptDefault.aspx" id="_ctl1">
<table width="100%" HEIGHT="100%" cellpadding="0" cellspacing="0" border="0">
<TR>
<TD CLASS="Normal" COLSPAN="4"><IFRAME ID="IFRAME1" NAME="IFRAME1" SRC="../aspx/invRptDetailReport.aspx" frameborder="0" width="100%" height="100%"></IFRAME></TD>
</TR>
'Now the iframe1 goes like this
<form name="frmRptDetailReport" method="POST" action="invRptDetailReport.aspx" id="frmRptDetailReport">
<table id="htblMainBody" width="100%" cellpadding="0" cellspacing="0" border="0" bgcolor="#E5DBE2">
<tr><td <a id="lbtnSelect" href="javascript:__doPostBack('lbtnSelect','')"</a>
<div id="divChkboxlist" class="scrollingControlContainer scrollingCheckBoxList" onscroll="saveScrollPos();">
<table id="cblPlants" border="0" style="font-family:Arial;font-size:11px;width:275px;overflow: scroll"><tr>
<tr><td><input id="cblPlants_0" type="checkbox" name="cblPlants:0" onclick="javascript:setTimeout('__doPostBack(\'cblPlants$0\',\'\')', 0)" language="javascript" /><label for="cblPlants_0">1X1 -Confecciones</label></td>
</tr>
How can I check this check box on this page? After the checkbox is checked I will receive the email I want.
If you need more information please tell me
Try
ieDoc.querySelector("#cblPlants_0").click
Equivalent to:
ieDoc.getElementById("cblPlants_0").click
Remember to have proper page load waits after the .Navigate step before attempting the click
While ieApp.Busy Or ieApp.readyState < 4: DoEvents: Wend
Set ieDoc = ieApp.Document
ieDoc.getElementById("cblPlants_0").click
You may need an additional wait before the click. Step through with F8 to test.
You could also try executing the javascript
ieDoc.parentWindow.execScript "javascript:setTimeout('__doPostBack(\'cblPlants$0\',\'\')', 0)"

Cannot login to website using a macro and vba

I am trying to login to a website using the following which works on different url
Sub Mylogin()
Dim MyHTML_Element As IHTMLElement
Dim MyURL As String
On Error GoTo Err_Clear
MyURL = "MYURL"
ie.Silent = True
ie.Navigate MyURL
ie.Visible = True
Do
Loop Until ie.ReadyState = READYSTATE_COMPLETE
Set HTMLDoc1 = ie.Document
HTMLDoc1.all.Email.Value = "MYEMAIL" 'Enter your email id here
HTMLDoc1.all.Password.Value = "MYPASSWD" 'Enter your password here
For Each MyHTML_Element In HTMLDoc1.getElementsByTagName("input")
If MyHTML_Element.Type = "submit" Then MyHTML_Element.Click: Exit For
Next
Err_Clear:
If Err <> 0 Then
Err.Clear
Resume Next
End If
End Sub
The submit button is under div and I dont know if this is the problem. The html login part is the following
<div id="login-area-main">
<div id="user">
<div id="username">
<input name="p$lt$ctl01$LogonFormIndice$loginElem$UserName" type="text" maxlength="50" id="p_lt_ctl01_LogonFormIndice_loginElem_UserName" placeholder="E-mail" />
<span class="CMSValidator"><span id="p_lt_ctl01_LogonFormIndice_loginElem_rfvUserNameRequired" title="Please enter a user name." class="profile-validator validator error-message" style="display:none;">
</span></span>
</div>
</div>
<div id="pass">
<div id="password">
<input name="p$lt$ctl01$LogonFormIndice$loginElem$Password" type="password" maxlength="20" id="p_lt_ctl01_LogonFormIndice_loginElem_Password" placeholder="Password" />
<span class="CMSValidator"><span id="p_lt_ctl01_LogonFormIndice_loginElem_rfvPasswordRequired" class="profile-validator validator error-message" style="visibility:hidden;">
</span></span>
</div>
</div>
<div id="pass-forgot">
<p>Forgot Password</p>
</div>
<div id="submit-button">
<a id="p_lt_ctl01_LogonFormIndice_loginElem_btnLogon" class="buyBtn button" href="javascript:WebForm_DoPostBackWithOptions(new WebForm_PostBackOptions("p$lt$ctl01$LogonFormIndice$loginElem$btnLogon", "", true, "p_lt_ctl01_LogonFormIndice_MiniLogon", "", false, true))"></a>
</div>
Do I have to do something with
MyHTML_Element.Type = "submit"
And put inside the div id?
I notice that the HTMLDoc1 is Empty..! should it be empty?
I tried the following but nothing displayed or clicked
Const Url$ = "URL"
Dim UserName As String, Password As String, LoginData As Worksheet
Set LoginData = ThisWorkbook.Worksheets("MySheet")
UserName = LoginData.Cells(1, "K").Value
Password = LoginData.Cells(2, "K").Value
Dim iex As Object
Set iex = CreateObject("InternetExplorer.Application")
With iex
.Navigate Url
ieBusy iex
.Visible = True
Dim oLogin As Object, oPassword As Object
iex.Document.querySelector(".username [id='p_lt_ctl01_LogonFormIndice_loginElem_UserName']").Focus
Set oLogin = iex.Document.querySelector(".username [id='p_lt_ctl01_LogonFormIndice_loginElem_UserName']").Value = ""
Set oPassword = iex.Document.querySelector(".password [type=password]").Value = ""
oLogin.Value = UserName
oPassword.Value = Password
iex.Document.getElementById("submit-button").Click
End With
It asks about an Object on
iex.Document.querySelector("id='p_lt_ctl01_LogonFormIndice_loginElem_UserName']").Focus
I am confused by where your actual problem is.
There is an id for the submit
ie.document.getElementById("submit-button").click ' 0r .submit
For username:
ie.document.querySelector("[id='p_lt_ctl01_LogonFormIndice_loginElem_UserName']").value = ""
For password
ie.document.querySelector("[type=password]").value = ""
When entering values it sometimes helps to use .Focus on the element before assigning the .value.
Public Sub GetInfo()
Dim ie As New InternetExplorer
With ie
.Visible = True
.navigate URL
While .Busy Or .readyState < 4: DoEvents: Wend
With .document.querySelector("[id='p_lt_ctl01_LogonFormIndice_loginElem_UserName']")
.Focus
.value = ""
End With
With .document.querySelector("[type=password]")
.Focus
.value = ""
End With
.document.getElementById("submit-button").click
While .Busy Or .readyState < 4: DoEvents: Wend
Stop
'Quit
End With
End Sub

With VBA get Internet explorer to select item from list

Good Day,
I've searched for answers and solutions proveded on this site did not seem to help including selectedIndex and looping through arrays
I've got the following HTML code making up a table from which I want to select the second option "Vorige week"
<table cellspacing="0" cellpadding="0" title="" class="mstrListBlock"
id="id_mstr51" style="display: table; width: auto;">
<tbody>
<tr>
<td class="mstrListBlockCell">
<span class="">
<div class="mstrListBlockCaption" style="display: none;"/>
<div class="mstrListBlockHeader" style="display: none;">
<div style="" class="mstrListBlockContents"
id="ListBlockContents_id_mstr51">
<div oncontextmenu="return
mstr.behaviors.Generic.oncontextmenu(arguments[0], self, 'id_mstr51');"
onmouseup="try{mstr.$obj('id_mstr51').focus();}catch(localerr){}; return
mstr.behaviors.Generic.clearBrowserHighlights(self)" onmousedown="var retVal
= mstr.behaviors.ListView.onmousedown(arguments[0], self, 'id_mstr51');
try{mstr.$obj('id_mstr51').focus();}catch(localerr){}; return retVal"
ondblclick="return mstr.behaviors.ListView.ondblclick(arguments[0], self,
'id_mstr51')" class="mstrListBlockListContainer" id="id_mstr51ListContainer"
style="display: block;">
<div class="mstrListBlockItem" title="Huidige Week">
<div class="mstrListBlockItemSelected" title="Vorige Week">
<div class="mstrBGIcon_fi mstrListBlockItemName" style="background-position:
2px 50%; padding-left: 23px;">Vorige Week</div>
</div>
<div class="mstrListBlockItem" title="Afgesloten 4 Weken">
<div class="mstrListBlockItem" title="Afgesloten 8 Weken">
<div class="mstrListBlockItem" title="Huidige Periode">
<div class="mstrListBlockItem" title="Vorige Periode">
<div class="mstrListBlockItem" title="Afgesloten 2 Perioden">
<div class="mstrListBlockItem" title="Selectie Datum Hiërarchie. Aangepast
wegens IServer crash icm. Metric prompts.">
<div class="mstrListBlockItem" title="Gisteren">
I think my problem is in deciding which element I need to use to get the desired outcome
Sub JDWReport()
Dim objIE As InternetExplorer
Set objIE = New InternetExplorerMedium
objIE.Visible = True
objIE.navigate "URL"
Do While objIE.Busy = True Or objIE.readyState <> 4: DoEvents: Loop
objIE.document.getElementById("Uid").Value = "username"
objIE.document.getElementById("Pwd").Value = "password"
objIE.document.getElementById("3054").Click
Do While objIE.Busy = True Or objIE.readyState <> 4: DoEvents: Loop
objIE.navigate "URL2"
Do While objIE.Busy = True Or objIE.readyState <> 4: DoEvents: Loop
objIE.document.getElementsClassName("mstrBGIcon_fi
mstrListBlockItemName")(0).Click
objIE.Quit
End Sub
See code above which I'm currently using.
It gets stuck with the line objIE.document.getElementsClassName("mstrBGIcon_fi
mstrListBlockItemName")(0).Click
I tried changing this line to different elements based on the HTML code and use .click .selectedindex=2 but those won't work.
<div class="mstrListBlockItemSelected" title="Vorige Week">
Currently it says mstrListBlockItemSelected, however, when first navigating to the site, the class is defined as the rest, mstrListBlockItem.
It will only change to selected if you click on the item in question (from a list of items). My ultimate goal would be to get the class with title "Vorige Week" to change from mstrListBlockItem to mstrListBlockItemSelected.
I can see that you are using HTML Table and create DIV's in that.
I try to search and find that there is no any method or property is available to select the text in DIV.
I suggest you to use any HTML control to select its value. For example "Select option".
You can try to create drop down using select and then use the code below to select any value in it.
Sub Select_Item()
Dim post As Object, elem As Object
With CreateObject("InternetExplorer.Application")
.Visible = True
.navigate "C:\Users\WCS\Desktop\element.html"
While .Busy = True Or .ReadyState < 4: DoEvents: Wend
Set post = .Document.getElementById("ctl00_ContentPlaceHolder1_ddlCycleID")
For Each elem In post.getElementsByTagName("option")
If InStr(elem.Value, "10") > 0 Then elem.Selected = True: Exit For
Next elem
End With
End Sub
You can try an attribute = value CSS selector:
ie.document.querySelector("[title='Vorige Week']").Selected = True
Or
ie.document.querySelector("[title='Vorige Week']").Click

Webpage login not working Using excel VBA

I am working on project to download the dump from website and save it on the path specified using Excel vba.
Code is working perfectly when u do the debug or execute line by line by pressing "F8".
but when you execute the whole program by pressing "F5" or clicking on button after assigning macro to it. its not working.
need your precious advise to resolve this issue.
Thanks in Advance,
Prasanna
VBA Code used to login.
Sub Login()
Dim MyHTML_Element As IHTMLElement
Dim MyURL As String
Dim HTMLDoc As HTMLDocument
Dim MyBrowser As InternetExplorer
MyURL = "URL"
Set MyBrowser = New InternetExplorer
MyBrowser.Silent = True
MyBrowser.Navigate MyURL
MyBrowser.Visible = True
Do
Application.Wait DateAdd("s", 5, Now)
Loop Until MyBrowser.READYSTATE = READYSTATE_COMPLETE
Application.Wait DateAdd("s", 5, Now)
Set HTMLDoc = MyBrowser.document
HTMLDoc.all.Country_Code.Value = "Country_Code"
HTMLDoc.all.Login.Value = "UserName"
HTMLDoc.all.passwd.Value = "Password"
HTMLDoc.all.Item("B1").Click
For Each MyHTML_Element In HTMLDoc.getElementsByName("B1")
If MyHTML_Element.Type = "button" Then MyHTML_Element.Click: Exit For
Next
End sub
Sample HTML code of webpage for login.
<table border=0>
<tr>
<td>Country:</td>
<td>
<input type="text" name="country_code" maxlength=2
onblur="this.value=this.value.toUpperCase();Form1_action(this.value)">
</td>
</tr>
<tr>
<td>Language:</td>
<td>
<select name="idioma" disabled >
<option value="uk|es" onblur="document.Form1.login.focus()">ENGLISH</option>
<option value="sp|es" onblur="document.Form1.login.focus()">SPANISH</option>
<option value="fr|en-us" onblur="document.Form1.login.focus()">FRENCH</option>
<option value="it|en-us" onblur="document.Form1.login.focus()">ITALIAN</option>
<option value="de|de" onblur="document.Form1.login.focus()">GERMAN</option>
</select>
</td>
</tr>
<tr>
<td>Login:</td>
<td>
<input type="text" name="login" maxlength=10 value="" disabled >
</td>
</tr>
<tr>
<td>Password:</td>
<td>
<input type="password" autocomplete="off" name="passwd" maxlength=10 value="" disabled onkeypress="var okp=(event.which)?event.which:event.keyCode; if(okp==13) SiteRedirect(this.form)">
</td>
</tr>
</table>
<br>
<center>
<input type="button" name="B1" value="Sign In"
onclick="SiteRedirect()"
disabled
style="width:80pt"
>
</center>
This is the method I use to allow IE to load in an application that frequently works with webpages in IE. I have come to this after a lot of trial and error and it works consistently now - though I have seen many ways to accomplish this.
Public Declare PtrSafe Sub Sleep Lib "kernel32" (ByVal dwMilliseconds As LongPtr)
Sub IEWait(IE As Object)
'assumes IE is loaded as InternetExplorer.Application")
With IE
Do While .Busy Or .ReadyState <> 4: Sleep (100): Loop
End With
End Sub
You can add this to your code by
Placing the Public Declare at the top of your module window before any Sub are defined.
Incorporating it into your code as shown below.
Code:
With MyBrowser
.Silent = True
.Navigate MyURL
.Visible = True
Do While .Busy or .Readystate <> 4: Sleep (100): Loop
Set HTMLDoc = .document
'... rest of code
End With
Scott hit the nail on the head.
Sub Test()
Dim IE As Object
Set IE = CreateObject("InternetExplorer.Application")
With IE
.Visible = True
.Navigate "http://www.marketwatch.com/investing/stock/aapl/analystestimates" ' should work for any URL
Do Until .ReadyState = 4: DoEvents: Loop
. . . YOUR CODE HERE . . .
End With
End Sub
OR
Sub DumpData()
Set IE = CreateObject("InternetExplorer.Application")
IE.Visible = True
URL = "http://finance.yahoo.com/q?s=sbux&ql=1"
'Wait for site to fully load
IE.Navigate2 URL
Do While IE.Busy = True
DoEvents
Loop
. . . YOUR CODE HERE . . .
End Sub