I perform a simple search in devtool, but it drops drastically without a reason:
What's more, if I view the source and do the same search, the number of results of <link rel is just 58, not 184. Do you know why?
Here is the page if you need to examine.
For these "complex" queries you'll have to use xPath selectors:
//link[#rel]
//link[contains(#rel,'style')]
or CSS selectors:
link[rel]
link[rel*="style"]
For a trivial CSS selector like a use html a instead to ensure it doesn't match as literal text.
List of supported queries
Devtools uses CDP command DOM.performSearch and judging by the implementation it tries to match these types of queries:
text - inside #text nodes (like textContent in js)
text - inside tag names
text - inside attribute names
text - inside attribute values
<tag - matching at the start of a tag name
</tag - matching a closing tag
tag> - matching at the end of a tag name
<tag> - matching an entire tag name
"text - matching at the start of an attribute value
text" - matching at the end of an attribute value
text - matching an entire attribute value
//a[contains(., 'foo')] - XPath selector
a#foo.class[attr] - CSS selector
As you can see the literal text matching is limited to the first four types, and it won't find things that span more than one type like attr="value" that spans two types.
Related
Let's say I have a piece of HTML like this:
<a>Ask Question<other/>more text</a>
I can match this piece of XPath:
//a[text() = 'Ask Question']
Or...
//a[text() = 'more text']
Or I can use dot to match the whole thing:
//a[. = 'Ask Questionmore text']
This post describes this difference between . (dot) and text(), but in short the first returns a single element, where the latter returns a list of elements. But this is where it gets a bit weird to me. Because while text() can be used to match either of the elements on the list, this is not the case when it comes to the XPath function contains(). If I do this:
//a[contains(text(), 'Ask Question')]
...I get the following error:
Error: Required cardinality of first argument of contains() is one or zero
How can it be that text() works when using a full match (equals), but doesn't work on partial matches (contains)?
For this markup,
<a>Ask Question<other/>more text</a>
notice that the a element has a text node child ("Ask Question"), an empty element child (other), and a second text node child ("more text").
Here's how to reason through what's happening when evaluating //a[contains(text(),'Ask Question')] against that markup:
contains(x,y) expects x to be a string, but text() matches two text nodes.
In XPath 1.0, the rule for converting multiple nodes to a string is this:
A node-set is converted to a string by returning the string-value of
the node in the node-set that is first in document order. If the
node-set is empty, an empty string is returned. [Emphasis added]
In XPath 2.0+, it is an error to provide a sequence of text nodes to a function expecting a string, so contains(text(),'substr') will cause an error for more than one matching text node.
In your case...
XPath 1.0 would treat contains(text(),'Ask Question') as
contains('Ask Question','Ask Question')
which is true. On the other hand, be sure to notice that contains(text(),'more text') will evaluate to false in XPath 1.0. Without knowing the (1)-(3) above, this can be counter-intuitive.
XPath 2.0 would treat it as an error.
Better alternatives
If the goal is to find all a elements whose string value contains the substring, "Ask Question":
//a[contains(.,'Ask Question')]
This is the most common requirement.
If the goal is to find all a elements with an immediate text node child equal to "Ask Question":
//a[text()='Ask Question']
This can be useful when wishing to exclude strings from descendent elements in a such as if you want this a,
<a>Ask Question<other/>more text</a>
but not this a:
<a>more text before <not>Ask Question</not> more text after</a>
See also
How contains() handles a nodeset first arg
How to use XPath contains() for specific text?
Testing text() nodes vs string values in XPath
The reason for this is that the contains function doesn't accept a nodeset as input - it only accepts a string. (Well, it may be engine dependent, because it works for Python's lxml module. According to the specification, it should convert the value of the first node in the set to a string and act on that. See also XPath contains(text(),'some string') doesn't work when used with node with more than one Text subnode)
//a[text() = 'Ask Question'] is matching any a elements which contain a text node which equals Ask Question.
//a[text() = 'more text'] is matching any a elements which contain a text node which equals more text.
So both of these expressions match the same a element.
You can re-work your query to //a[text()[contains(., 'Ask Question')]] so that the contains method will only act on a single text node at a time.
I perform a simple search in devtool, but it drops drastically without a reason:
What's more, if I view the source and do the same search, the number of results of <link rel is just 58, not 184. Do you know why?
Here is the page if you need to examine.
For these "complex" queries you'll have to use xPath selectors:
//link[#rel]
//link[contains(#rel,'style')]
or CSS selectors:
link[rel]
link[rel*="style"]
For a trivial CSS selector like a use html a instead to ensure it doesn't match as literal text.
List of supported queries
Devtools uses CDP command DOM.performSearch and judging by the implementation it tries to match these types of queries:
text - inside #text nodes (like textContent in js)
text - inside tag names
text - inside attribute names
text - inside attribute values
<tag - matching at the start of a tag name
</tag - matching a closing tag
tag> - matching at the end of a tag name
<tag> - matching an entire tag name
"text - matching at the start of an attribute value
text" - matching at the end of an attribute value
text - matching an entire attribute value
//a[contains(., 'foo')] - XPath selector
a#foo.class[attr] - CSS selector
As you can see the literal text matching is limited to the first four types, and it won't find things that span more than one type like attr="value" that spans two types.
<a href="javascript:void(0)" title="home">
<span class="menu_icon">Maybe more text here</span>
Home
</a>
So for above code when I write //a as XPath, it gets highlighted, but when I write //a[contains(text(), 'Home')], it is not getting highlighted. I think this is simple and should have worked.
Where's my mistake?
Other answers have missed the actual problem here:
Yes, you could match on #title instead, but that's not why OP's
XPath is failing where it may have worked previously.
Yes, XML and XPath are case sensitive, so Home is not the same as
home, but there is a Home text node as a child of a, so OP is
right to use Home if he doesn't trust #title to be present.
Real Problem
OP's XPath,
//a[contains(text(), 'Home')]
says to select all a elements whose first text node contains the substring Home. Yet, the first text node contains nothing but whitespace.
Explanation: text() selects all child text nodes of the context node, a. When contains() is given multiple nodes as its first argument, it takes the string value of the first node, but Home appears in the second text node, not the first.
Instead, OP should use this XPath,
//a[text()[contains(., 'Home')]]
which says to select all a elements with any text child whose string value contains the substring Home.
If there weren't surrounding whitespace, this XPath could be used to test for equality rather than substring containment:
//a[text()[.='Home']]
Or, with surrounding whitespace, this XPath could be used to trim it away:
//a[text()[normalize-space()= 'Home']]
See also:
Testing text() nodes vs string values in XPath
Why is XPath unclean constructed? Why is text() not needed in predicate?
XPath: difference between dot and text()
yes you are doing 2 mistakes, you're writing Home with an uppercase H when you want to match home with a lowercase h. also you're trying to check the text content, when you want to check check the "title" attribute. correct those 2, and you get:
//a[contains(#title, 'home')]
however, if you want to match the exact string home, instead of any a that has home anywhere in the title attribute, use #zsbappa's code.
You can try this XPath..Its just select element by attribute
//a[#title,'home']
Is it possible for one XPath expression to match all the following <a> elements using the text in the element, in this case "Link"?
Examples:
Link
<span>Link</span>
<div>Link</div>
<div><span>Link</span></div>
This simple XPath expression,
//a[contains(., 'Link')]
will select the a elements of all of your examples because . represents the current node (a), and contains() will check the string value of a to see if it contains 'Link'. The string value of a already conveniently abstracts away from any descendent elements.
This even simpler XPath expression,
//a[. = 'Link']
will also select the a elements in all of your examples. It's appropriate to use if the string value of a will exactly equal, rather than just contain, "Link".
Note: The above expressions will also select Li<br/>nk, which may or may not be desirable.
You could use the following:
//a[(.//*|.)[contains(text(), "Link")]]
This will select a elements that contain the text "Link" or a elements that have a descendant element that contains the text "Link".
//a - Select all a elements
( - Open OR grouping
.//* Select all the descendant nodes
| - Or..
. - Select the current node
) - Close OR grouping
[contains(text(), "Link")] - If they contain the text "Link"
Alternatively, you could also use:
//a[(.//*|.)[.="Link"]]
Maybe I am missing something, but they seem similar. If you use for example...
a[alt~="thumb"]
or...
a[alt*="thumb"]
What can I narrow my selection down to differently? I am at the understanding that ~ gives you a partial match in the quotes while the * gives you a partial match. I am going to fiddle with the code a little, but since I could not find a question on the subject here, thought it would make a good topic either way.
From the JQuery help (which supports the standard selectors):
a[alt~="thumb"]
Description: Selects elements that have the specified attribute with a
value containing a given word, delimited by spaces. This selector
matches the test string against each word in the attribute value,
where a "word" is defined as a string delimited by whitespace. The
selector matches if the test string is exactly equal to any of the
words.
a[alt*="thumb"]
Description: Selects elements that have the specified attribute with a
value containing the a given substring. This is the most generous of
the jQuery attribute selectors that match against a value. It will
select an element if the selector's string appears anywhere within the
element's attribute value. Compare this selector with the Attribute
Contains Word selector (e.g. [attr~="word"]), which is more
appropriate in many cases.
Basically the selector ~= only matches if the value is found surrounded by white space. The selector *= matches if the value is found anywhere.
<div alt='heading navigation'>
<div alt='head'>
div[alt~='head'] would match only the second div, but div[alt*='head'] would match both.
[att~=value] is a contains word selector.
So a [alt="foo"] selector will match <a alt="foo bar"> but will not match <a alt="foobar">.
[alt*="foo"] will match both though, because this doesn't discriminate on words or whatever. As long as it's in the value, it hits.