Selenium: How to deal with web elements that get hidden when you try to inspect them? Python 3.x related - html

I'm about to finish developing an automated program on Python and the only thing that's stopping me it's a web element that gets immediately hidden when I try to inspect it from an OpenSea page (can't share the link here since it requires to be connected to my wallet):
So far, I have only managed to get the XPATH and the CSS_SELECTOR of this problematic element (div#tippy-298) which is the following:
button_problematic_xpath = driver.find_element(By.XPATH, '//*[#id="tippy-298"]')
button_problematic_css = driver.find_element(By.CSS_SELECTOR, '#tippy-298')
But what I need is to get the XPATH or CSS_SELECTOR of the Web Elements that button_problematic_css contains, more specifically the "Starting" and "Ending" Textboxes :
Also, just in case, the element that's on top of button_problematic_css is called "Duration", here's its XPATH:
button_duration = driver.find_element(By.XPATH, '//*[#id="duration"]').click #after clicked it deploys div#tippy-298 element
I was wondering if there could be any method to force the visibility of this button_problematic_css using the Chrome Console in the Inspection of Web Elements, or by using an specific method from Selenium on Python3.x to inmediately download all of the web elements this page has right after button_problematic_css is visible.
What I have tried so far is to manually inspect the element, by pressing Ctrl + Shift + C and and then clicking the elements contained in button_problematic_css which just ends up hiding this element before even clicking its content, the same happens when trying to display the contents of this element in the DOM of this page, it just dissapears.
I just wish my program could be capable of editing the dates in the "Starting" and "Ending" Textboxes

I guess you need to click the button to show the element. why did I guess? you need to show the url
or u can also try this code:
clickmore=WebDriverWait(driver, 2).until(EC.element_to_be_clickable((By.XPATH, 'blabalabla')))
self.driver.execute_script("arguments[0].click();", clickmore)
button_duration = driver.find_element(By.XPATH, '//*[#id="duration"]').click()

After several days of working, I managed to solve this, it happened that in order to get the XPATH from these particular elements, I had to use the Chrome Console and type $('#duration').click();.
In this case, #duration is a Selector that can be used to get the web element button that allows you to edit the period of time you want to set when clicked.
I guess the general statement would be something like this:
$('#your_css_selector').click();
Now, this way I could managed to look deeper in the DOM without making them hide again.

Related

Why would links in an object or iframe (I've tried both) start opening in a new window

I'm working on an application that has many links. They all open in the same window, until today. All of a sudden, in all browsers I'm testing in, 3 links in an iframe or object (I've tried both) start opening in a new window. I can't seem to stop this.
An object example follows. The dolnks program generates 3 simple links like the one following the object example and these links open in a new window.
<OBJECT ID='fixed' DATA='dolnks.cgi?str=$params' TARGET='dynamic' NORESIZE></OBJECT>
darea.cgi?str=$dogstr
Can someone help me understand this and how to get these links to open in the same window.
All links that now open in the same window or should be opening in the same window are from
the same domain.
I now have to close the link instead of using the back button.
Thanks,
craigt
I imagine it's because of your TARGET attribute, although it's not using one of the special target values:
target
Where to display the linked URL, as the name for a browsing context (a tab, window, or ). The following keywords have special meanings for where to load the URL:
_self: the current browsing context. (Default)
_blank: usually a new tab, but users can configure browsers to open a new window instead.
_parent: the parent browsing context of the current one. If no parent, behaves as _self.
_top: the topmost browsing context (the "highest" context that’s an ancestor of the current one). If no ancestors, behaves as _self.
From the MDN Docs: https://developer.mozilla.org/en-US/docs/Web/HTML/Element/a#attributes
Note though that the docs for object do not list a target attribute, so that behavior is apparently undefined, and probably varies depending on the browser, and the plugin displaying the object.
Check to see if it's the items with a target attribute (case does not matter) that are working "wrong", and see if removing that attribute fixes it. If that's not it, next check to see if there are javascript being loaded. Try turning javascript off (hopefully the relevant links are not generated with javascript) and see if that fixes the behavior. If turning javascript off is too heavy handed, you can use the javascript console to see what listeners are attached to the links.

How to check which web elements that are truly clickable?

I am experimenting with web-crawling techniques with python and selenium webdriver in win10 and firefox.
I encountered the following typical page with a pop-up box asking for consent to allowing for cookies.
https://irs.thsrc.com.tw/IMINT/
There are only the following two clickable tags in the page with the following xpaths.
//html/body/div[2]/form[1]/div[2]/div[1]/div[1]/div[1]/div[1]/span[1]/a[1]
//html/body/div[2]/form[1]/div[2]/div[1]/div[1]/div[2]/table[1]/tr[1]/td[1]/input[1]
People here suggested that I can first get an element via webdriver and then use the two methods of is_enabled() and is_displayed() to together check if the element is clickable.
However, when I get elements with the following statement:
e = WebDriverWait(driver, 5).until(EC.element_to_be_clickable((By.XPATH, xpath)))
I found that all elements (even those in the shadow) that I got via the above statement are both enabled and displayed.
But in fact in the browser, only the two actionable elements in the pop-up message box were clickable.
All the other elements were in the shadow and cannot be clicked (or actionable).
Can someone tell me how to get the correct information which elements are actually actionable via some API ?
Thank you very much.
Farn

How to append the Smooch web-messenger-container element to a custom element

I want to be able to append the #web-messenger-container element to an element instead of the document.body.
I've tried using Smooch.render without setting the embedded attribute to true and it works fine on some browsers, but on others, the smooch CDN calls are cancelled. It is related with this issue: https://github.com/smooch/smooch-web/issues/666
I don't want to use embedded mode, I just want to move the Smooch button to another container. I've tried moving the iframe element with JS but it causes the iframe to reload and the Smooch window disappears.
I would say if you want to append #web-messenger-container to a different element, embedded mode is the way to do that. Without embedded mode Smooch's host JS will call render() to append the iframe on it's own so trying to call it second time will give you nothing but grief I think.
If all you want is to render your own open/close button, that's pretty straightforward. You can specify a custom button width and height of '0', and trigger Smooch.open() / Smooch.close() however you like.
For example:
https://jsfiddle.net/alavers/ve5uhjnd/

Always getting - Exception in thread "main" org.openqa.selenium.ElementNotVisibleException: element not visible

I'm trying to automate the logging in for one of our applications and trying to click on 'Reporting' (as highlighted in yellow). When i inspect it in firefox, it says its an anchor web element (with this version of firefox i'm not able to install firebug/firepath). This page loads immediately after i enter the username/password and click the 'Submit' button on the previous page. The script always gives me error that 'element not visible'. I tried even explicit wait for that element by using the xpath "//ul[#id='mainnav-menu']/li[2]/a" or "//ul[#id='mainnav-menu' and #class='list-group']/li[2]/a". Could you please help to determine why i'm getting this exception all the time. Basically what i'm trying to do is click on this element which will show 2 links once that arrow beside it expands. i need to click on the 1st child element after it shows up.
Please see the HTML code portion for the element from the application in the image below
The error means that element exists (html is loaded), but not visible. It may be out of visible part of the screen or because of java script effects.

how to inspect jquery code on chrome developer tools?

I use web developer tools to inspect html and subsequently see the css that is attached to the html element. This is proving to be a great process for learning from other websites (and the debugging my own)
Is there a way to inspect the javascript as well? So when I select the element, to be able to see the javascript related to the element?
In the element panel you can find all information related to an element including events attached to it.
You can use console in the element panel to inspect an element using dir(elementId)which dumps the object with the given id, as a JavaScript object with its properties.
You can see javascript code that is attached as event to any element on the page. In Developer tools its in the Elements tab and each element has "Event Listeners" - there you see what events and what javascript will be catched and executed.
I had the same problem and have been wandering a bit until I found how to do it on Chrome:
1. Open the Inspector Ctrl + Shift + i
2. Select the element that you would like to inspect
3. Click on the Event Listeners tab
4. Click on the link next to the event listener you would like to check
5. Click on Pretty Print (Symbolized as {} at the bottom of the window)