Validate legitimacy of email address before sending email? - smtp

We're sending out emails to people when they sign up for our service, but a lot of them are bouncing (partially our fault for using a bogus email in some end-to-end tests), and AWS has informed us that additional bouncing could be bad for our instances' health.
So, I'm trying to come up with a means of validation (or at least a way to stop us from sending to bad addresses).
It looks like there's a means of asking SMTP servers if they know of an address, but there's some controversy around the technique (it isn't guaranteed to work, and doing too much of it makes you look like a spammer; could get you blacklisted in the worst case).
I guess the other option here is to keep track of what we've seen bounce and just dump 'bad' emails into a database collection, although there are problems with this too (if someone signs up for that email address later, they're unable to sign up for the service; we still send at least one bounced email even in the best case).
My question is: is there a better option, one that I'm missing? This seems like the sort of issue that should be A Solved Problemâ„¢, but I'm not finding any applicable patterns.

This is a solved problem. Step 1 is just to validate the syntax of the email address (using something like EmailValidation) and then step 2 is to send an email to the address with a link for the user to click to verify their email address. Until the user clicks the link in their email, don't add the address to your database.

Related

How do HTML forms protect against email spoofing?

I can't find anything on this but it seems like a big problem. My concern is with HTML forms which contain an email field which the recipient can use to reply. What stops malicious users from inputting any old email address resulting in the spoofed email address receiving the reply? Isn't this quite a big issue and why isn't it abused?
Because there is no sense. Hackers do something either for benefits (better if its a financial profit) or fun. There is no benefit or fun if someone else receives an e-mail. For you, as a hacker, what is the purpose of such "spoofing"?
Of course, you can fill the form at some web-site and specify someone else's e-mail as a reply address. He will receive the e-mail, see "if you think that you have received this e-mail by mistake - ignore it" and he will follow this instructions.
If you are a developer of such feedback form, then you would probably want to implement a limitation on the amount of such requests on one e-mail per hour; or IP address; or whatever else to prevent SMTP server overloading and making your actual or potential clients angry.

Possible to make a one account per person website?

Is it possible to make a website with only one account per person?
Any suggestion is good.
Thanks
If you don't want people to go generating 100 accounts a minute, you'll need something like captcha, which is very easy to add on to your website.
You can do other things, like associate each account with an email address, and make the user verify that that email actually exists by sending a link out to that email address so that when they click on it, it verifies the connection.
To associate exactly one account per person, you are going to need to use some sort of official identity, and usually for smaller websites that doesn't make sense. By official identity I mean verify their credit card or government identification (social security?), but then you run into a lot of problems because people won't want to do this, and it is going to cost money to make sure that these identities are real. Also, if you really need something like this, you're going to have to beef up the security of your website.
An alternative is to require a user to put in a verification number which you send to them via SMS, and ensure the phone number they enter is unique, phones/simcards are relatively cheap these days, but most people wouldn't go through the effort of spending $5 on a new simcard to get a duplicate account on your site.
(and if they would, sell accounts for $3 and undercut the cost of a sim ;)
Ask for identifying information for example credit card data that you can verify and allow registrations only with this identifying information. Of course credit card data can be stolen, so you cannot be 100% sure about anything in internet or in world generally.
Whatever you do, people will still try to buck the system. If you use Email, then people can have multiple free emails, if you block free emails, lots of people dont have any other emails so you block them. If you use IP you block anyone with a shared IP such as ISPs who enforce proxy servers for their clients.
Unless you start asking for social security numbers or NI or whatever your country has, and you start alienating people as a rule because they would consider that irrelevant personal information.
You just have to hope people treat your site fairly, and know some wont.
One potential solution these days is to only create user accounts via federation of identity from a much stricter service provider. For example, verify your users via a facebook account using oauth. I believe facebook is pretty good at detecting shills/spammers and you can harness their resources in your service too.
I think you have 2 options, neither of which will solve the whole problem:
Log the IP address and prevent multiple logins from one IP. Not great, as this will probably backfire if college kids use your site and the dorm uses shared IPs.
Log the MAC address and prevent multiple logins from one MAC. Better, but it will prevent multiple people from the same household from using your site, and most houses have more than one computer/mobile device.
You could always combine the two options, but again with multiple mobile devices it's possible to circumnavigate that.
you could make a code that only lets 1 account be made per computer used. you have to be able to get the computers IP address and block it from making more than 1 account.

How to defend against users with Multiple Accounts?

We have a service where we literally give away free money.
Naturally said service is ripe for abuse. To defend against this we do the following:
log ip address
use unique email addresses (only 1 acct/email addy)
collect more info like st. address, phone number, etc.
use signup captcha
BHOs (I've seen poker rooms use these)
Now, let's get real here -- NONE of this will stop a determined user.
Obviously ip addresses can be changed via a proxy (which could be blacklisted via akismet) but change anyways if the user has a dynamic ip or if more than one user is behind a NAT'd network (can we say almost everyone?)
I can sign up for thousands of unique email addresses each hour -- this is no defense.
I can put in fake information taken from lists for street addresses and phone numbers.
I can buy captchas from captcha solving services (1k for $5).
bhos seem only effective for downloadable software -- this is a website
What are some other ways to prevent multiple users from abusing the service? How do all the PPC people control click fraud?
I know we could actually call the person but I don't think we are trying to do that anytime soon.
Thanks,
It's pretty difficult to generate lots of fake phone numbers that can send and receive SMS messages. SMS verification could go a long way towards cutting down on fraud. Of course, it also limits you to giving away free money to cell phone owners.
I think only way is to bind your users accounts to 'real world' information, like his/her passport number, for instance. Of course, you'll need to make sure that information is securely stored and to find some way to validate it.
Re: signing up for new email accounts...
A user doesn't even need to do that. Please feel free to send your mail to brian_s#mailinator.com, or feydr.asks.a.question#spamherelots.com, or stackoverflow#safetymail.info, or my_arbitrary_username#zippymail.info. I haven't registered any of those email addresses, but all of them will work.
Those domains are owned by ManyBrain, and they (and probably others as well) set the domain to accept any email user. ManyBrain in particular then makes the inboxes for those emails publicly accessible without any registration (stripping everything by text from the email and deleting old mail). Check it out: admin#mailinator.com's email inbox!
Others have mentioned ways to try and keep user identities unique. This is just one more reason to not trust email addresses.
First, I suppose (hope) that you don't literally give away free money but rather give it to use your service or something like that.
That matters as there is a big difference between users trying to just get free money from you they can spend on buying expensive cars vs only spending on your service which would be much more limited.
Obviously many more user will try to fool the system in the former than in the latter case.
Why it matters? Because it is all about the balance between your control vs your user annoyance. I see many answers concentrating on the control part, so let's go through annoyance, shall we?
Log IP address. What if I am the next guy on the computer in say internet shop and the guy before me already used that IP? The other guy left your hot page that I now see but I am screwed because the IP is blocked. Yes, I can go to another computer but it is annoyance and I may have other things to do.
Collecting physical Adresses. For what??? Are you going to visit me? Or start sending me spam letters? Let me guess, more often than not you get addresses with misprints at best and fake ones at worst. In fact, it is much less hassle for me to give you fake address and not dealing with whatever possible spam letters I'll have to recycle in environment-friendly way. :)
Collecting phone numbers. Again, why shall I trust your site? This is the real story. I gave my phone nr to obscure site, then later I started receiving occasional messages full of nonsense like "hit the fly". That I simply deleted. Only later and by accident to discover that I was actually charged 2 euros to receive each of those messages!!! Do I want to get those hassles? Obviously not! So no, buddy, sorry to disappoint but I will not give your site my phone number unless your company is called Facebook or Google. :)
Use signup captcha. I love that :). So what are we trying to achieve here? Will the user who is determined to abuse your service, have problems to type in a couple of captchas? I doubt it. But what about the "good user"? Are you aware how annoying captchas are for many users??? What about users with impaired vision? But even without it, most captchas are so bad that they make you feel like you have impaired vision! The best advice I can give - if you care about user experience, avoid captchas as plague! If you have any doubts, do your online research first!
See here more discussion about control vs annoyance and here some more thoughts about being user-friendly.
You have to bind their information to something that is 'real world', as Rubens says. Of course, you also need to be able to verify this information (I can just make up passport numbers all day if you don't check to make sure they're correct).
How do you deliver the money? Perhaps you can index this off the paypal account, mailing address, or whatever you're sending the money to?
Sometimes the only way to prevent people abusing a system is to not have the system in the first place.
If you're doing what you say you're doing, "giving away money to people", then surprise surprise, there will be tons of people with more time available to try to find ways to game the system than you will have to fix it.
I guess it will never be possible to have an identification system which identifies fake identities that is:
cheap to run (I think it's called "operational cost"?)
cheap to implement (ideally one time cost - how do you call that?)
has no Type-I/Type-II errors
is scalable
But I think you could prevent users from having too many (to say a quite random number: more than 50) accounts.
You might combine the following approaches:
IP address: can be bypassed with VPN
CAPTCHA: can be bypassed with human farms (see this article, for example - although they claim that their test can't be that easily passed to other humans, I doubt this is true)
Ability-based identification: can be faked when you know what is stored and how exactly the identification works by randomly (but with a given distribution) acting (example: brainauth.com)
Real-world interaction: Although this might be the best one, but I guess it is expensive and not many users will accept it. Also, for some users/countries it might not be possible. (example: Postident in Germany, where the Post wants to see your identity card. I guess this can only be faced in massive scale by the government.)
Other sites/resources: This basically transforms the problem for other sites. You can use services, where it is not allowed/uncommon/expensive to have much more than 1 account
Email
Phone number: e.g. by using SMS, see Multi-factor authentication
Bank account: PayPal; transfer not much money or ask them to transfer a random (small) amount to you (which you will send back).
Social based
When you take the social graph (vertices are people, edges are connections), you will expect some distribution. You know that you are a single human and you know some other people. So you have a "network of trust" (in quotes, because I think this might be used in other context as well). Now you might not trust people / networks how interact heavily with your service, but are either isolated (no connection) or who connect a large group with another large group ("articulation points"). You also might not trust fast growing, heavily interacting new, isolated graphs.
When a user provides content that is liked by many other users (who you trust), this might be an indicator that there is a real human creating it.
We had a similar issue recently on our website, it is really a hassle to solve this issue if you are providing a business over one time or monthly recurring free credits system.
We are using a fraud detection solution https://fraudradar.io for a while and that helped us a lot to clean out most of the spam activities. It is pretty customizable with:
IP checks
Email domain validity
Regex rules
Whitelisting options per IP, email domain etc.
Simple API to communicate through
I would suggest to check that out.

Do we really need email confirmation?

I've gotten into a habit of using the standard register->send activation email->activate account process for every site that supports user authentication and free registration without questioning if I really need this.
What are your thoughts on this? If I have captcha on the registration form is the email confirmation process really necessary?
EDIT:
OK, so the general consensus seems to be that by getting the users to confirm the email they entered I'll keep them away from putting someone else's email in there.
What about when I let users edit their profile/settings and they enter another email?
If I need to keep them away from entering other people's addresses then I'd need to confirm that email address (by temporarily deactivating their accoun)t every time they change it.
Captcha+activation prevents bots AND spoofed people
Well basically it is since each part prevents one problematic scenario:
Captcha prevents (if you use strong captcha like reCaptcha) bots from registering new users
Email activation prevents people from registering other people (by their email address)
I guess this is a valid everyday pattern for registration that's widely acknowledged by IT community.
EDIT
Yes. When you want to prevent users from changing their email address, you'd have to repeat email activation procedure to make it robust.
But you don't have to deactivate their account while doing it. All you have to do is having a pending email-change email activation active. If it gets activated, you change email address at that point (not when they change it), otherwise the old one is still used.
If you don't confirm an e-mail, you're supposing that the user registering that service owns that email account. How can you start sending a lot of system e-mails, reset passwords and etc to a person that has nothing to do with your system? I would be really pissed of if it was my e-mail.
Another scenario: what if the register mispelled his e-mail when registering? Suppose he doesn't check his "account settins" in your application, doesn't change his email, and needs to reset his password. If the e-mail is registered in a wrong way, it's your fault for not checking it before.
Of course, I'm just saying this to services that would REALLY demand an account to be created. Avoid the login barrier when possible, or use openid when your service isn't so critical.
You should give serious consideration to supporting OpenID. http://openid.net/get-an-openid/what-is-openid/
The key benefit for OpenID is that it reduces the complexity for your user. There is no reason to force people to remember login credentials for hundreds of sites when a viable alternative exists. There is no worldwide netizen database - and there likely never will be - but OpenID simplifies the situation greatly.
I know that as a user I found the registration process for Stack Overflow to be painless and easy. I wish more sites used OpenID.
It's the lowest-level attempt at identity validation. It encourages users to re-use the same account when they return (by having a common, shared identifier you and they can use to reconnect), and it prevents impersonation, because it requires access to the claimed identity as proof.
It's not perfect, but something by definition works infinitely better than nothing.
If identity doesn't matter on your site (e.g. your service is throwaway after each use) then you don't need email activation. Otherwise, you probably want it.
On my site, I let users sign-up and do everything non-public until they confirm their email address. Because I run a gaming website, it means users can earn medals, post scores, just not post in the forum or post comments in the blog until they verify their email address.
I find it works pretty well. I have 16,000 registered users.
I find it both unnecessary and annoying. If I can, I avoid doing this.
However, I do do this if 1) email will be sent by the program, so I can test if the email address is valid, or 2) this is a very large, public-facing website, in which case I want to filter out as many potential problems as possible.
For most basic sites, I don't bother with either. Both email activation and captcha are relatively easy for dedicated spammers to bypass and overcome and do little but cause an annoyance to most of the users, driving away at least a certain percentage who might have otherwise signed up. I've found in my experience, focusing more on spam filters for member posted content has a better ROI overall.
For sites with more serious content, you'll typically have more serious users. In cases like that, I'll throw everything I've reasonably got available at it to counter the spam.
I find it useful when an email is sent for confirmation. This makes sure that I am the one who has registered with that email address.
Even with captcha you can register someone else email address although he may or may not approve that confirmation.
You only seem to need e-mail confirmation to confirm identity, not to send useful content by e-mail. But e-mail confirmation is only one means to an end. You may consider others, preferablly less intrusive ones.
Generally you can check something that
you are (e.g. fingerprint, iris scan)
you have (e.g. token, creditcard, key, access to an e-mail account)
you know (e.g. PIN, password, your mom's weight, name of your favorite deceased pet, the optimistic length of your most private bodyparts measured in inches)
Also, you can delegate the check to others; the creditcard company, phone company, someone's friends.
Example: GoogleMail could not ask
for a confirmation e-mail address
upon creation of your GMail account.
Instead, the early adopters had a limited supply of
"invites" to share with friends.
So - unless you actually need me to receive information you'd e-mail, which I generally hate anyway - you might be inclined to resort to more creative/fun means.

Is a "Confirm Email" input good practice when user changes email address?

My organization has a form to allow users to update their email address with us.
It's suggested that we have two input boxes for email: the second as an email confirmation.
I always copy/paste my email address when faced with the confirmation.
I'm assuming most of our users are not so savvy.
Regardless, is this considered a good practice?
I can't stand it personally, but I also realize it probably isn't meant for me.
If someone screws up their email, they can't login, and they must call to sort things out.
I've seen plenty of people type their email address wrong and I've also looked through user databases full of invalid email address.
The way I see it you've got two options. Use a second box to confirm the input, or send an authentication/activation email.
Both are annoyances so you get to choose which you think will annoy your users less.
Most would argue that having to find an email and click on a link is more annoying, but it avoids the copy/paste a bad address issue, and it allows you to do things like delete or roll back users if they don't activate after say 48 hours.
I would just use one input box. The "Confirm" input is a remnant form the "Confirm Password" method.
With passwords, this is useful because they are usually typed as little circles. So, you can't just look at it to make sure that you typed it correctly.
With a regular text box, you can visually check your input. So, there is no need for a confirmation input box.
I agree with you in that it is quite an annoyance to me (I also copy and paste my address into the second input).
That being said, for less savvy users, it is probably a good idea. Watching my mother type is affirmation that many users do not look at the screen when they type (when she's using her laptop she resembles Linus from Peanuts when he's playing the piano). If it's important for you to have the user's correct email address then I would say having a confirmation input is a very good idea (one of these days I'll probably type my email address wrong in the first box and paste it wrong into the second box and then feel like a complete idiot).
While the more tech-savvy people tend to copy and paste, not technical people find it just as annoying to have to type something twice. During a lot of user testing I've down, the less tech-savvy - the more annoyed they seem with something like this... They struggle to type as it is, when they see they have to type their email in again it's usually greeted with a strong sign.
I would suggested a few things.
Next to the input box write the style of the information you are looking for so something like (i.e. user#domain.com). The reason this is important is you would be surprised how many of the less tech-savvy don't really understand the different between a website and an email address, so let them know visually the format you want.
Run strong formatting test in real time, and visually show a user that the format is good or bad. A green check box if everything is okay comes to mind.
Lastly, depending on your system architecture I often use a library to actually wrong a domain in the background. I don't necessarily try to run a VRFY on the server - I often use a library to check to make sure the domain they entered has MX records in it's DNS record.
I agree with Justin, while most technical folks will use the copy, paste method, for the less savvy users it is a good practice.
One more thing that I would add is that the second field should have the auto-complete feature disabled. This ensures that there is human input from either method on at least one of the fields.
Typing things twice is frustrating and doesn't prevent copy&paste errors or even some typos.
I would use an authenticate/activate schema with a roll back to the old address if the activation is not met within 48 hours or if the email bounces.
As long as a field is viewable, you do not need a confirm box. As long as you do some form validation to be sure that it is at least in valid format for an email address let the user manage the rest of the issues.
I'd say that this is ok but should only be reserved for forms where the email is essential. If you mistype your email for your flight booking then you have severed the two-way link between yourself and the other party and risk not getting the confirmation number, here on StackOverflow it would only mean your Gravatar would not be loaded ...
I'd consider myself fairly techie but I always fill in both fields /wo cut-paste if I regard it to be important enough.
I tend to have it send a verification code to the email address specified (and only ask for it once), and not change the email address until the user has entered the code I sent them.
This has the advantage that if they try to set it to a dozen different addresses in quick succession, you'll know which ones work by which verification code they put in.
Plus, if I am presented with a "confirm email address" box, I just copy and paste from the previous one, and if I'm guilty of that, I'm sure that other less careful users will do the same.