Although I know a lot of email clients will pre-fetch or otherwise cache images. I am unaware of any that pre-fetch regular links like some link
Is this a practice done by some emails? If it is, is there a sort of no-follow type of rel attribute that can be added to the link to help prevent this?
As of Feb 2017 Outlook (https://outlook.live.com/) scans emails arriving in your inbox and it sends all found URLs to Bing, to be indexed by Bing crawler.
This effectively makes all one-time use links like login/pass-reset/etc useless.
(Users of my service were complaining that one-time login links don't work for some of them and it appeared that BingPreview/1.0b is hitting the URL before the user even opens the inbox)
Drupal seems to be experiencing the same problem: https://www.drupal.org/node/2828034
Although I know a lot of email clients will pre-fetch or otherwise cache images.
That is not even a given already.
Many email clients – be they web-based, or standalone applications – have privacy controls that prevent images from being automatically loaded, to prevent tracking of who read a (specific) email.
On the other hand, there’s clients like f.e. gmail’s web interface, that tries to establish the standard of downloading all referenced external images, presumably to mitigate/invalidate such attempts at user tracking – if a large majority of gmail users have those images downloaded automatically, whether they actually opened the email or not, the data that can be gained for analytical purposes becomes watered down.
I am unaware of any that pre-fetch regular links like some link
Let’s stay on gmail for example purposes, but others will behave similarly: Since Google is always interested in “what’s out there on the web”, it is highly likely that their crawlers will follow that link to see what it contains/leads to – for their own indexing purposes.
If it is, is there a sort of no-follow type of rel attribute that can be added to the link to help prevent this?
rel=no-follow concerns ranking rather than crawling, and a no-index (either in robots.txt or via meta element/rel attribute) also won’t keep nosy bots from at least requesting the URL.
Plus, other clients involved – such as a firewall/anti-virus/anti-madware – might also request it for analytical purposes without any user actively triggering it.
If you want to be (relatively) sure that any action is triggered only by a (specific) human user, then use URLs in emails or other kind of messages over the internet only to lead them to a website where they confirm an action to be taken via a form, using method=POST; whether some kind of authentication or CSRF protection might also be needed, might go a little beyond the context of this question.
All Common email clients do not have crawlers to search or pre-build <a> tag related documents if that is what you're asking, as trying to pre-build and cache a web location could be an immense task if the page is dynamic or of large enough size.
Images are stored locally to reduce load time of the email which is a convenience factor and network load reduction, but when you open an email hyperlink it will load it in your web browser rather than email client.
I just ran a test using analytics to report any server traffic, and an email containing just
linktomysite
did not throw any resulting crawls to the site from outlook07, outlook10, thunderbird, or apple mail(yosemite). You could try using a wireshark scan to check for network traffic from the client to specific outgoing IP's if you're really interested
You won't find any native email clients that do that, but you could come across some "web accelerators" that, when using a web-based email, could try to pre-fetch links. I've never seen anything to prevent it.
Links (GETs) aren't supposed to "do" anything, only a POST is. For example, your "unsubscribe me" link in your email should not directly unsubscribe th subscriber. It should "GET" a page the subscriber can then post from.
W3 does a good job of how you should expect a GET to work (caching, etc.)
http://www.w3schools.com/tags/ref_httpmethods.asp
Related
I'm working for a forum owner who allows users to submit hotlinked images from other domains in their posts. If they choose to use an http version of the URL, the otherwise clean page becomes insecure in the eyes of a browser, which some percentage of the time triggers a worried email from certain users.
I can't rewrite the urls, since I can't code against the assumption that future off site images will have https available. For the same reason, I can't use protocol relative src attributes. I'm unwilling to fetch and cache the images on our server just so that they can be served over https, because of the computational expense involved.
What can I do? Is there some piece of HTML syntax or some similar which I can use to tell the browser "This image doesn't matter, and doesn't constitute a security threat"?
This isn't possible. The image may not constitute a security threat but MITM attacks could still lead to images other than the intended one being loaded over the network, and who knows what an attacker may want to supplant that image with. My suggestion would be to pass the annoyance on to your users and tell them they can only use https URLs.
I have been asked to add some email addresses to individuals personal portfolio galleries on a site I designed.
There is already a very visible email form that customers can use, but it only goes to one address. Then the master email recipient has to forward the email to the intended recipient. When the business started there was only one master recipient. But now there are 5 or 6 artists and the shop is growing.
My first thought was to put href="mailto:" links on everyone's personal page. This would not be convenient for customers without a configured email client, but copy and paste would be fine. Also, The customer is worried about spam, but I think email filters would be more effective then some obfuscation technique.
Then I read this which says using mailto "screams 'this web site has been developed by a beginner.'I don't really agree. The author advocates using server side forms. I already have made a PHP form and could easily add a drop down of recipients (maybe not so easy, but it could be done) or have the recipient field auto fill on a certain persons page.
My real question(s) is/are threefold:
Does using mailto demonstrate some kind of beginners ignorance?
What would be the best practice to easily allow an average website user to email an individual from a small group?
And is email obfuscation critical?
BTW I've already ready a found a little data that suggests that obfuscation is (sort of effective), but the article seems dated.
Disadvantages of using mailto: links:
Show the recipients email address (unless obfuscation is used)
Users need to have a working email client installed on their computer. Nowadays more and more users only use a web based email client, which won't work with mailto: links.
Personally I'd go with a server created contact form.
Does using mailto demonstrate some kind of beginners ignorance?
Not at all, mailto: and a mail icon is state of the art if you display a mail address. It's a UX and accessibility fail if you declare it as regular text. Especially for mobile users.
Using a contact form instead provides some benefits, but may also make your page vulnerable to mail spamming or even getting hacked if you don't watch out and update regularly.
What would be the best practice to easily allow an average website user to email an individual from a small group?
That's hard to say, it depends on the design and use case. I would recommend using a simple AJAX contact form with as few fields as possible ("Ask me a question:") and a ReCaptcha v3, but a simple mailto: link with a predefined subject is also fine.
And is email obfuscation critical?
I had to research this today and it seems to still be necessary. Most bots search for common patterns in the page source and don't use JavaScript.
You could add the href and insert the # symbol using JS after the DOMContentLoaded event: <a class="uncrypt">user???example.com</a>. But this is just one way of many.
Bots that use JS could still get the address, but that's hard to avoid anyway and I personally think it's more important to create a good UX for all users than making your page bullet proof to spammers.
But you don't have to protect generic addresses (like info#<domain>) as spammers will try those anyway.
Follow the leaders
Having simply an email link on a site isn't an especially good indicator of a high quality site. Quite simply, this is because high powered sites, sites with millions of users per day (microsoft.com, stackoverflow.com), or multinational corporations like Amazon do not do that. If they did, that persons email would probably be buried in a day.
Pros of the mailto: link:
Simple
Trivially understandable
Cons:
Exposes contact information directly
Not flexible
Not what the pros do
Email mailto links cause bad user experience *
Here are a few additional approaches that provide more professionalism:
- If anonymity is a goal, a generic contact form. No user is exposed, could be 10 it people ready to respond to it, could be 1, could be 100. Submit to a back-end, forward to 10 people on the back-end if you like.
- If you want to put a face to the contact, or you want to allow users to contact specific individuals, use user profiles with avatars of their faces, clicking a button also displays a contact form, perhaps in a modal.
When you click a link and your email spins all 8 of your cpus as you wait for the world to end, you can see how the user experience suffers with having to open an unrelated email client
What security holes can appear on my site by including external images via img tag and how to avoid them?
I'm currently only checking the extension and mime-type of image on submission (that can be changed after URL is submitted) and URL is sanitized before putting it in src attribute.
There's probably a differentiation to be made here between who is at risk.
If all you're doing is storing URLs, and not uploading images to your server, then your site is probably safe, and any potential risk is to your users who view your site.
In essence, you're putting your trust in the reliability of the browser manufacturers. Things might be fine, but if a security hole in some browser one of your users uses were to arise that involved incorrectly parsing images that contain malicious code, then it's your users who will end up paying for it (you might find GIFAR interesting).
It comes down to whether you trust the browser manufacturers to make secure software, and whether you trust your users to not upload URLs to images that might contain exploits for certain browsers. What might be secure now might not be secure come the next release.
The primary holes that can be exposed are those where corrupted images cause buffer overflows within the browser, allowing arbitrary code execution.
If you're only putting the images into an <img> tag there shoudln't be any vulnerabilities relating to sending alternative MIME types, but never underestimate the stupidity of some web browser developers...
Well, obviously, you're not doing any checks on the data, so the data can be anything (the mime-type reported by the remote server doesn't necessarily tell the truth). Plus, as you said, the data on the remote server can be changed since you're never looking at it after submission.
As such, if the link is put into lets say an <img src="..."/>, then any vulnerability that a browser might have in the image handling can be exploited.
"Sanitizing" the URL doesn't help with anything: somebody submitting a link that points to a 'bad' image isn't going to attack his own server.
It is impossible to identify a user or request as unique since duping is trivial.
However, there are a handful of methods that, combined, can hamper cheating attempts and give a user quasi-unique status.
I know of the following:
IP Address - store the IP address of each visitor in a database of some sort
Can be faked
Multiple computers/users can have the same address
Users with dynamic IP addresses (some ISP issue them)
Cookie tracking - store a cookie per visitor. Visitors that don't have it are considered "unique"
Can be faked
Cookies can be blocked or cleared via browser
Are there more ways to track non-authorized (non-login, non-authentication) website visitors?
There are actually many ways you can detect a "unique" user. Many of these methods are used by our marketing friends. It get's even easier when you have plugins enabled such as Java, Flash etc.
Currently my favorite presentation of cookie based tracking is evercookie (http://samy.pl/evercookie/). It creates a "permanent" cookie via multiple storage mechanisms, the average user is not able to flush, specifically it uses:
Standard HTTP Cookies
Local Shared Objects (Flash Cookies)
Silverlight Isolated Storage
Storing cookies in RGB values of
auto-generated, force-cached PNGs
using HTML5 Canvas tag to read pixels
(cookies) back out
Storing cookies in Web History
Storing cookies in HTTP ETags
Storing cookies in Web cache
window.name caching
Internet Explorer userData storage
HTML5 Session Storage
HTML5 Local Storage
HTML5 Global Storage
HTML5 Database Storage via SQLite
I can't remember the URL, but there is also a site which tells you how "anonymous" you are based on everything it can gather from your web browser: What plugins you have loaded, what version, what language, screensize, ... Then you can leverage the plugins I was talking about earlier (Flash, Java, ...) to find out even more about the user. I'll edit this post when I find the page whcih showed you "how unique you are" or maybe somebody knows »» actually it looks as if every user is in a way unique!
--EDIT--
Found the page I was talking about: Panopticlick - "How Unique and trackable is your browser".
It collects stuff like User Agent, HTTP_ACCEPT headers, Browser Plugins, Time Zone, Screen Size and Depth, System Fonts (via Java?), Cookies...
My result: Your browser fingerprint appears to be unique among the 1,221,154 tested so far.
Panopticlick has a quite refined method for checking for unique users using fingerprinting. Apart from IP-adress and user-agent it used things like timezone, screen resolution, fonts installed on the system and plugins installed in the browser etc, so it comes up with a very distinct ID for each and every user without storing anything in their computers. False negatives (finding two different users with the exact same fingerprint) are very rare.
A problem with that approach is that it can yield some false positive, i.e. it considers the same user to be a new one if they've installed a new font for example. If this is ok or not depends on your application I suppose.
Yes, it's impossible to tell anonymous visitors apart with 100% certainty. The best that you can do is to gather the information that you have, and try to tell as many visitors apart as you can.
There is one more piece of infomration that you can use:
Browser string
It's not unique, but in combination with the other information it increases the resolution.
If you need to tell the visitors apart with 100% certainty, then you need to make them log in.
There is no sure-fire way to achieve this, in my view. Of your options, cookies are the most likely to yield a reasonably realistic number. NATing and proxy servers can mask the IP addresses of a large number of users, and dynamic IP address allocation will confuse the results for a lot of others
Have you considered using e.g Google Analytics or similar? They do unique visitor tracking as part of their service, and they probably have a lot more money to throw at finding heuristic solutions to this problem than you or I. Just a thought!
I am relatively new to web development, and I was hoping I could get some pointers about the feasibility of a feature I would like to implement. Is it possible to have a url link that you can click on, that can contain login credentials for the website it is linking to, so as to bypass that websites login screen?
In other words, can I make a link from my website to facebook, that would allow me to login right in to my facebook, from any computer? Meaning, if I don't have cookies to store my login info in, is it possible to login still?
This is just a conceptual question, so any help would be appreciated! Thanks!
One reason why this is generally avoided, is because web servers often store the query string parameters in the access logs. And normally, you wouldn't want files on your server with a long list of usernames and passwords in clear text.
In addition, a query string containing a username and password could be used with a dictionary attack to guess valid login credentials.
Apart from those issues, as long as the request is made via HTTPS, it would have been safe during the transit.
It is possible to pass parameters in the URL through a GET request on the server, but one has to understand that the request would likely be made in clear text and thus isn't likely to be secure. There was a time where I did have to program a "silent" log-in using tokens, so it can be done in enterprise applications.
You used to be able to do this, but most browsers don't allow it anymore. You would never be able to do this using facebook only something that uses browser auth (the browser pops up a username/pass dialog)
it was like this:
http://username:pass#myprotectedresource.com
What you might be able to do is whip up some javascript in a link that posts your username and password to the login page of facebook. Not sure if it will work because you might need to scrape the cookie/hidden fields from the login page itself.
It is possible for the site to block you on account of no cookies, or invalid nonce or wrong HTTP referrer, but it may work if their security is low.
While it is possible, it is up to the site (in this case Facebook) to accept these values in the query string. There are some security issues to consider certainly, and isn't done generally.
Though, there are different options out there for single sign on. This web site uses OpenID for that.