CsQuery : Finding Prices on a page - html

Question

I am trying to scrape prices from any given URL. I am using CsQuery and for the life of me, I cannot figure out the best way to find all items on a page that might be a price. A bonus would be figuring out the most likely price by size / color of the test and how close it is to the top of the page. I was thinking maybe looking at a Regex solution, but I am not sure if that is the correct way to go with CsQuery.

Answer 1

Well, if a currency sign is present, You might do something like.
(?:\$|\£)(\d+(?!\d*,\d)|\d{1,3}((, ?)\d{3}?)?(\3\d{3}?){0,4})(\.\d{1,2})?(?=[^\d,]|, (?!\d{3,})|$)
(?:\$|\£) -- matches literal currency simbols. You can remove this
if you can't count on the presence of currency symbols,
but it's a great anchor if you can
(\d+ -- matches any number of digits
(?!\d*,\d) as long as not followed by comma digit
|
\d{1,3} -- otherwise matches betweein 1 and 3 digits
(
(, ?) -- looks for a comma followed by a possible space
captures as \3
\d{3}?) -- followed by 3 digits
? -- zero or one times
(\3 -- looks for the same pattern of comma with or without space
\d{3}? -- followed by 3 digits
){0,4}) -- between 0 and 4 times, more on that below
(\. -- literal period
\d{1,2} -- followed by one or two digits
)? -- zero or one times (so, optional)
(?=[^\d,]|, (?!\d{3,})|$)
Another thing you might do is to limit how many repetitions of comma groups there can be, it might help weed out high numbers that aren't likely prices. If you're not expecting anything over 999,999, you might do this (but if you're dealing with foreign currencies, inflation has made some astronomically high--a loaf of bread in Zimbabwe costs fifty million).
For easy reading, I'll show you how to limit the repetitions to 7
Change the 4, (the only 4 in the whole regex) to 6, (the number you want -1, because we look for 1 beforehand to establish comma pattern).
(?:\$|\£)(\d+(?!\d*,\d)|\d{1,3}((, ?)\d{3}?)?(\3\d{3}?){0,6})(\.\d{1,2})?(?=[^\d,]|, (?!\d{3,})|$)
You can see this in action at: https://regex101.com/r/oU2nW2/1