HTML & VBA Visual Basic automatic input website - html

My company has a "link" I can go to... http://blablabla... where there is a box where I can enter an 8 digit number (next to the box it says lot name) and a button where I can click view report. That's it. That's all that's on the page. (input box and a view report button)
I have a list of numbers in excel in cell range B3:B7 for example (10378851, 10378857, 10488213, etc. )
I want to open excel and run a code that automatically goes to the website, automatically inputs the first number, automatically "clicks" the view report button.
Then there will be an "export data button" - I want to automatically click which will have a drop down (csv, txt, excel, etc.) I want to automatically select "excel"
I found a site that explains how to do something similar but I am stuck.
Sub GetTable()
Dim ieApp As InternetExplorer
Dim ieDoc As Object
Dim ieTable As Object
Dim clip As DataObject
'create a new instance of ie
Set ieApp = New InternetExplorer
'you don’t need this, but it’s good for debugging
ieApp.Visible = True
'assume we’re not logged in and just go directly to the login page
ieApp.Navigate "http://severe-frost-552.heroku.com/login"
Do While ieApp.Busy: DoEvents: Loop
Do Until ieApp.ReadyState = READYSTATE_COMPLETE: DoEvents: Loop
Set ieDoc = ieApp.Document
'fill in the login form – View Source from your browser to get the control names
With ieDoc.forms(0)
.login.Value = "dailydose"
.Password.Value = "password"
.submit
End With
Do While ieApp.Busy: DoEvents: Loop
Do Until ieApp.ReadyState = READYSTATE_COMPLETE: DoEvents: Loop
'now that we’re in, the rest of this code isn't relevant because it copied a table of data...
Here is the website...
http://dailydoseofexcel.com/archives/2011/03/08/get-data-from-website-that-requires-a-login/
As you can see, there source code says:
<form action="/session" method="post"><div style="margin:0;padding:0;display:in...
<input id="login" name="login" type="text" /></p>
I think that is where they are getting this part of their code:
With ieDoc.forms(0)
.login.Value = "dailydose"
Looking at my websites source code, it looks like I have:
<DIV id=ReportViewerControl_ctl04_ctl03><INPUT id=ReportViewerControl_ctl04_ctl03_txtValue class=null style= size=30 name=ReportViewerControl$ctl04$ctl03$txtValue> </DIV>
<INPUT id=ReportViewerControl_ctl04_ctl03_txtValue class=null style= size=30 name=ReportViewerControl$ctl04$ctl03$txtValue>
I think this is the right area of the code because when I type "hello" into the input box it shows up there like this:
<INPUT id=ReportViewerControl_ctl04_ctl03_txtValue class=null style= size=30 value=Hello name=ReportViewerControl$ctl04$ctl03$txtValue>
This is much different than the simple example... I am not sure how to code this input in my code...

you can try this:
ieDoc.ReportViewerControl_ctl04_ctl03_txt.Value = "Hello"
and if that does not work:
ieDoc.getElementById("ReportViewerControl_ctl04_ctl03_txt").Value = "Hello"
disclaimer: I have not tested the code
hint: what you are "playing" with is called "web-scraping". You can find decent examples here

Related

Website scraping: website search box has no value

I am trying to crosscheck a large body of data with a specific website (https://icis.corp.delaware.gov/Ecorp/EntitySearch/NameSearch.aspx).
The goal is to search for many company names based on a larger list in Excel to get their founding dates. For now I am starting out with a single name to get it running. I am having trouble in my main code as there is no inherent input value in the HTML code:
<input name="ctl00$ContentPlaceHolder1$frmEntityName" type="text" id="ctl00_ContentPlaceHolder1_frmEntityName" tabindex="4" size="30" maxlength="120" class="txtNormal" onkeyup="KeyEvent1(this.id)">
I tried the following:
Sub click_search()
Dim i As SHDocVw.InternetExplorer
Set i = New InternetExplorer
i.Visible = True
i.Navigate "https://icis.corp.delaware.gov/Ecorp/EntitySearch/NameSearch.aspx"
Do While i.ReadyState <> READYSTATE_COMPLETE
Loop
Dim idoc As MSHTML.HTMLDocument
Set idock = i.Document
idoc.getElementsByTagName("input").Item("ctl00$ContentPlaceHolder1$frmEntityName").Value = "10X Genomics Inc"
End Sub
The problem I believe is the HTML code does not have inherent value = "" to begin with but it only comes up in the HTML code after you write it in.
How do I fix this and furthermore then click the search button?
The error is
"Object variable or With block variable not set"
Always use Option Explicit at the top of every VBA code file.
If the webpage in question contains ids for the elements you are interested in, use getElementById() to access them. This code works, however it does not find any records.
Option Explicit
Sub click_search()
Dim i As SHDocVw.InternetExplorer
Dim idoc As MSHTML.HTMLDocument
Set i = New InternetExplorer
i.Visible = True
i.Navigate "https://icis.corp.delaware.gov/Ecorp/EntitySearch/NameSearch.aspx"
Do While i.ReadyState <> READYSTATE_COMPLETE
Loop
Set idoc = i.Document
idoc.getElementById("ctl00_ContentPlaceHolder1_frmEntityName").Value = "10X Genomics Inc"
idoc.getElementById("ctl00_ContentPlaceHolder1_frmFileNumber").Value = "1"
idoc.getElementById("ctl00_ContentPlaceHolder1_btnSubmit").Click
End Sub

Input text to search box on web page and click Search

I am trying to input data from Excel workbook to web pages search field and click search button. A lot of different approaches has been tested with success. Current solution gives 424 Error, Object required. Pointing to:
.Document.getElementById("crmGrid_findCriteria").Value = _
Sheets("Other data").Range("A2").Value
HTML for search field:
<input id="crmGrid_findCriteria" title="Search for records" hintlabelid="crmGrid_findHintText" type="text" class="ms-crm-Dialog-Lookup-QuickFind" maxlength="100" tabindex="0" value="">
HTML for Search button:
<img id="crmGrid_findCriteriaImg" src="/_imgs/imagestrips/transparent_spacer.gif" class="ms-crm-ImageStrip-search " imgbase="/_imgs/search" title="Start search" alt="Start search">
Here is structure:
Here is my VBA (link for CRM system so it is not public):
Sub GoToOpport()
'This will load a webpage in IE
Dim ieappIE As InternetExplorerObject
Dim HWNDSrc As Long
Dim elementsSet AsappIE Object= CreateObject("internetexplorer.application")
Set ie = Nothing With appIE
Set ie = New InternetExplorerMedium
ie.Visible = True
ie.Navigate "https://crm.dynamics.com/main.aspx?area=nav_oppts&etc=3&page=SFA&pageType=EntityList&web=true"
With ie
Do
DoEvents
Loop Until ie.ReadyState = READYSTATE_COMPLETE
End With
ie.Document.getElementById("contentIFrame0").contentDocument.getElementById("crmGrid_findCriteria").Value = "hello"
'Unload IE
Set ieappIE = Nothing
End Sub
Error:
It is in an iframe so you need to access that
ie.document.getElementById("contentIFrame0").contentDocument.getElementById("crmGrid_findCriteria")
You can use js to attempt to assign value
ie.document.parentWindow.execScript "document.getElementById('contentIFrame0').document.getElementById('scrmGrid_findCriteria').value = 'hello';"
You could also try navigating to the src of the iframe
ie.navigate2 ie.document.querySelector("#crmContentPanel").src
While ie.Busy Or ie.readyState < 4: DoEvents: Wend
With ie.document.getElementById("crmGrid_findCriteria")
.focus
.value = "abc"
End With
I'm getting a 301 Moved Permanently status when I send a GET request and then I'm redirected here:
https://dynamics.microsoft.com/en-us/crm/what-is-crm/
The id="crmGrid_findCriteria" is nowhere to be found in the HTML response, so my guess is that if you try:
debug.print ie.Document.getElementById("crmGrid_findCriteria") Is Nothing
you'll get TRUE as a result.
In other words, the element you're looking for doesn't exist in the HTML you downloaded.

vba: How to click on element within iframe

My goal is to click an element within a html Iframe, but nothing worked for me so far. Hope someone can advise how to approach this task correctly as I am running in circles for weeks now.
I have tried to click on a div Id, span title but nothing worked so far. I believe it is because a wrong syntex
Option Explicit
Sub it_will_work()
'make the app work faster?
Application.ScreenUpdating = False
Application.DisplayAlerts = False
'--------------------------------
Dim sht As Worksheet
Set sht = ThisWorkbook.Sheets("Fields") 'my data will be stored here
Dim LastRow As Long
LastRow = sht.Cells(sht.Rows.Count, "A").End(xlUp).Row 'range definition
Dim i As Long 'Will be used for a loop that navigate to different url
For i = 2 To LastRow 'First url starts at row 2 untill the last row
Dim IE As Object 'Internet Explorer declaration
Set IE = CreateObject("InternetExplorer.Application")
IE.Visible = True
IE.navigate sht.Range("A" & i).Value 'My url that I want to navigate to
While IE.readyState <> 4 Or IE.Busy: DoEvents: Wend
Dim Doc As New HTMLDocument 'Will be used for the main html page
Set Doc = IE.document
Doc.getElementById("tab7").Click 'data taht need to be updated is here
'Global workgroup data that will effect the workgroup data(dependency)
Doc.getElementById("mcdResourceGlobalWorkgroup_ddltxt").Value = sht.Range("W" & i).Value
Doc.getElementById("mcdResourceGlobalWorkgroup_ddltxt").Focus
Doc.getElementById("mcdResourceGlobalWorkgroup_ddlimg").Click
'Workgroup dropdown, that need to be choosen within the Iframe:
Doc.getElementById("ResourceWorkgroup").Value = sht.Range("X" & i).Value '1) worgroup that I want to insert
Doc.getElementById("ResourceWorkgroup").Focus
Doc.getElementById("_IB_imgResourceWorkgroup").Click '2) Cliking here will generate dropdown values according the value inserted above
Application.Wait Now + TimeValue("00:00:5") 'before refering to Iframe I let the values to be loaded
'***from this point I have the issue where I try to access Iframe and click on the desired element:***
'Here I declare Iframe
Dim objIFRAME As Object
Set objIFRAME = IE.document.getElementsByTagName("iframe")
Debug.Print TypeName(objIFRAME)
'Here I ask to click on a title within the Iframe where value = X
objIFRAME.getElementsByName("title").Value = sht.Range("X" & i).Value.Click
Next i
Application.DisplayAlerts = True
Application.ScreenUpdating = True
End Sub
After the url loads the following steps should happen:
Click on tab 7 -> this will open the correct tab to work on
inseart value from clumn "W" to "Global workgroup" field
focus on "Global workgroup" field
Click on an image that validate the "Global workgroup" field
(validates the instered value)
inseart value from clumn "X" to "Workgroup" field
focus on "Workgroup" field
Click on image that opens the drop down options, which is generated
according the inserted value to "Workgroup" field
Within the Iframe, Click on the title that is equal to value
which was inserted to "Workgroup" field
I have also tried to use Selenium IDE so I can see how the recorded macro access the Iframe and click the desired elemnt:
Command: Select frame | Target: Index=2
Click | Target: css=span[title="APAC"]
I have tried to mimic the stpes above in VBE, but couldn't find a way to write it properly. I event tried to download & apply selenium driver and run the code using the selenium library but got stuck as well.
Below image is the html code of the Iframe and the desired element I want to click on:
You should be able to use the following syntax
ie.document.querySelector("[id='_CPDDWRCC_ifr']").contentDocument.querySelector("span[title=APAC]").click
With selenium you can use
driver.SwitchToFrame driver.FindElementByCss("[id='_CPDDWRCC_ifr']")
driver.FindElementByCss("span[title=APAC]").click
With your existing tag solution you need to use an index. For example,
objIFRAME(0)
Then querySelector on the contentDocument of that.

VBA - getElementById works for simple website but not another?

So I'm currently stuck at getting a VBA script to retrieve the value of an input box from this Sudoku website. However, I was able to get the value from a paragraph element with the id of "contact" from my own simpler website, using the same code (after switching the url and id names, of course).
Any attempts to research further brings up articles/blogs that discuss what I've done correctly so far, so I suspect I am not researching it properly.
Here is my code:
Sub GetTable()
Dim ieApp As InternetExplorer
Dim ieDoc As Object
Dim sudokuCell As Object
Dim url, id, content As String
Dim i As Integer
Set ieApp = New InternetExplorer
ieApp.Visible = True
url = "http://www.websudoku.com/"
ieApp.navigate url
Do While ieApp.Busy: DoEvents: Loop
Do Until ieApp.READYSTATE = READYSTATE_COMPLETE: DoEvents: Loop
Set ieDoc = ieApp.document
If ieDoc Is Nothing Then
MsgBox ("Nothing")
'Else
' MsgBox ("Something")
End If
For i = 0 To 8
Set sudokuCell = ieDoc.getElementById("f00")
content = sudokuCell.innerText
MsgBox (content)
Next i
ieApp.Quit
Set ieApp = Nothing
End Sub
And here is an example of the html for a cell which is blank:
<td class="g0" id="c00"><input class="d0" size="2" autocomplete="off"
name="8iz6n11" maxlength="1" onblur="j8(this)" id="f00"></td>
And here is one for cell that is prefilled with a number:
<td class="f0" id="c10"><input class="s0" size="2" autocomplete="off"
name="s8iz6n21" readonly="" value="7" id="f10"></td>
I have tried both the "c00" an "f00" without success. Also, while I believe the problem at hand is I am not retrieving the element, I am concerned that the .innerText property won't retrieve the values.
First: The website is using FRAME, so you are not accessing the frame document in the VBA code actually. You need to navigate to the actual URL given below - change your url variable as the following (which is the frame's src property):
url = "http://view.websudoku.com/?"
Second: Those are INPUT elements you are trying to get values, you should be better using Value property instead innerText
content = sudokuCell.Value
Third and last: I have no idea what your code is supposed to do inside the loop as it will keep reading f00 element value as is. However I believe you'll loop through the input elements and just hit the wall here about the FRAME issue I explained above, so I assume loop is your part and have no trouble about it.

Problems Using VBA to Submit a Web Page - Using the click button function but web page won't submit

I am writing a VBA code to pull data from a website (https://app.buzzsumo.com/top-content). I have a functional code that runs without errors however I still can't get the webpage to actually submit the form when the click command runs. I have tried many different approaches and combinations of submitting the form/clicking the submit button but none have seemed to work so far. Below is my current code.
Sub clickFormButton()
Dim ie As Object
Dim form As Variant,
Dim button As Variant
'add the “Microsoft Internet Controls” reference in VBA Project
Set ie = CreateObject("InternetExplorer.Application")
'using input box to enter URL I am serching for
Search_URL = InputBox("Enter URL to Search For")
With ie
.Visible = True
.navigate ("https://app.buzzsumo.com/#/top-content")
'Ensure that the web page downloads completely
While ie.ReadyState <> 4
DoEvents
Wend
'assigning the input variables to the html elements of the form
ie.document.getElementsByName("q").Item.innertext = Search_URL
'finding and clicking the button
Set objInputs = ie.document.getElementsByTagName("input")
For Each ele In objInputs
If ele.Title Like "Press Enter to Search" Then
ele.Click
End If
End With
End Sub
I have also tried other methods to find and click the button such as:
'Dim i As Variant
'Set form = ie.document.getElementsByClassName("btn btn-highlight")
'For i = 1 To 5
'If form.Item(i).DefaultValue = "Search!" Then
'Set button = form.Item(i)
'button.Click
'End If
'Next i
Please provide any recomendations on what I may be missing or how I can get this code to actually submit the form and advance to the search results. Thanks in advance for any help you can provide!
Here are some additional details: Unfortunately the element I am trying to click (the "Search" button) does not have an ID or Name associated with it. This is why is was trying alternative approaches, such as looping through all of the object and trying to find the one with the right “Title”. Here is the code for the element from the DOM explorer:
<input title="Press Enter to search" class="btn btn-highlight" type="submit" ng-disabled="topContentSearchForm.$invalid" value="Search!"/>
The only attributes associated with it are:
class: btn btn-highlight
type: submit
ng-disabled: topContentSearchForm.$invalid
value: Search!
title: Press Enter to Search
Please let me know if there is another way to find the element ID/name? or if there is another way to click the button without these attributes? Thanks
I know this is an old post but... I have been using this effectively..
'click login
Set htmlDoc = .document
Set htmlColl = htmlDoc.getElementsByTagName("input")
Do While htmlDoc.readyState <> "complete": DoEvents: Loop
For Each htmlInput In htmlColl
If Trim(htmlInput.Type) = "submit" Then
htmlInput.Click
Exit For
End If
Next htmlInput
A couple of ideas:
While ie.ReadyState <> 4
DoEvents
Wend
If you have javascripts on the page use Application.Wait Now + TimeSerial(0, 0, 4) (basically wait for 4 seconds) instead.
Second I don't understand why you need to loop through all the objects on the web page. The easier way would be to go that webpage in IE, hit F12 and select element in DOM explorer, you can get the ID or Name of the button and then use ie.document.GetElementByID("buttonID").Click or ie.document.GetElementsByName("buttonName").Item.Click
Let me know if this helps.
Edit: After inspecting the particular webpage it appears that the ID and Name attributes for that button are missing. So I had to resort to the following:
Dim i As integer
Set form = ie.document.getElementsByClassName("btn btn-highlight")
On Error Resume Next
For i = 1 To 20
If form.Item(i).DefaultValue = "Search!" Then
form.Item(i).Click
End If
Next i
The relevant button is clicked for the fourth item (I had to manually go through the loop because 3rd item navigated away from the page to a pricing page, so i had to go back). Anyway the full code is the following, please note that you will need to go through this exercise again if there were changes to the webpage
Sub clickFormButton()
Dim ie As Object
Dim form As Variant
Dim button As Variant
'add the “Microsoft Internet Controls” reference in VBA Project
Set ie = CreateObject("InternetExplorer.Application")
'using input box to enter URL I am serching for
Search_URL = InputBox("Enter URL to Search For")
With ie
.Visible = True
.navigate ("https://app.buzzsumo.com/#/top-content")
End With
'wait for page to load
Application.Wait Now + TimeSerial(0, 0, 5)
'assigning the input variables to the html elements of the form
ie.document.getElementsByName("q").Item.InnerText = Search_URL
'finding and clicking the button
ie.document.getElementsByClassName("btn btn-highlight").Item(4).Click
End Sub
It looks like you could potentially just build the string URL, for example if you put "abcd" in the search field, the resulting URL will be:
https://app.buzzsumo.com/top-content?result_type=total&type=articles&num_days=360&tfc=false&general_article&infographic&video&page=1&guest_post&giveaway&interview&links_sitewide=true&unique_domains=true&backlinks=false&q=abcd&offset=0
Note the bolded portion which is the search query.
So, and this is just a quick idea that may work as long as you're not trying to abuse their system by sending 1000's of automated requests:
Sub FetchWebsite()
Dim ie As Object
Dim form As Variant
Dim button As Variant
Dim url As String
'add the “Microsoft Internet Controls” reference in VBA Project
Set ie = CreateObject("InternetExplorer.Application")
'using input box to enter URL I am serching for
Search_URL = InputBox("Enter URL to Search For")
'### BUILD THE FULL URL
url = "https://app.buzzsumo.com/top-content?result_type=total&type=articles&num_days=360&tfc=false&general_article&infographic&video&page=1&guest_post&giveaway&interview&links_sitewide=true&unique_domains=true&backlinks=false&q=" & Search_URL & "&offset=0"
With ie
.Visible = True
.navigate url
End With
'wait for page to load
Do
Loop While Not ie.ReadyState = 4 And Not ie.Busy
AppActivate "Internet Explorer"
End Sub
I did some poking around in the Locals window and this should also work, modified from your code. This would be the Form.Submit that I mentioned in comment on OP.
Sub clickFormButton()
Dim ie As InternetExplorer
Dim form As Variant
Dim button As Variant
Dim ele As HTMLFormElement
'add the “Microsoft Internet Controls” reference in VBA Project
Set ie = CreateObject("InternetExplorer.Application")
'using input box to enter URL I am serching for
Search_URL = InputBox("Enter URL to Search For")
With ie
.Visible = True
.navigate ("https://app.buzzsumo.com/#/top-content")
End With
'wait for page to load
Do
Loop While Not ie.ReadyState = 4 And Not ie.Busy
'assigning the input variables to the html elements of the form
ie.document.getElementsByName("q").Item.InnerText = Search_URL
'finding and clicking the button
ie.document.getElementsByClassName("btn btn-highlight").Item(4).form.submit
End Sub
CSS selector:
You can use CSS selector of #search-btn > div. Which is div within className search-btn. "#" means class.
VBA:
Use .querySelector method to apply CSS selector:
ie.document.querySelector("#search-btn > div").Click