Related
Apologies for the super generic title, as I couldn't find a proper one for the questions I have.
In my company, we are now investigating the communications we have with our own employees. I've done a small test send with an HTML email (we are using SFMC) to a super small group, and I've seen a couple of minor issues. Since everyone in my company (I'm talking about +5000 employees) uses Outlook, all the issues I'm going to mention are Outlook specific:
1- Some of the emails ended up in the 'Other' instead of the 'Focused'. I don't know the pattern here, why some ended up in one while the rest ended up in the other. I'd like all of them to end up in 'Focused'.
2- People have to press 'Download Images' or 'add the sender to the safe list' or something like that, in order to see the whole HTML email.
How do you think we can overcome those issues? We have never investigated those in our B2C communications as the people who use Outlook in our Consumer audience was very low; but if we can solve this for the internal communications, it can also help with our B2C audience.
I've seen in some newsletters from other companies where they mention 'add our domain to the safe sender list in order to see the messages properly'; but I'd rather not have that; and fix it internally.
One option I'm thinking about is, maybe I can get in touch with our IT department, and see if they can do something about the issue I've mentioned in #2.
What do you think?
For #2, you need to embed images as attachments instead of providing external links.
We're sending out emails to people when they sign up for our service, but a lot of them are bouncing (partially our fault for using a bogus email in some end-to-end tests), and AWS has informed us that additional bouncing could be bad for our instances' health.
So, I'm trying to come up with a means of validation (or at least a way to stop us from sending to bad addresses).
It looks like there's a means of asking SMTP servers if they know of an address, but there's some controversy around the technique (it isn't guaranteed to work, and doing too much of it makes you look like a spammer; could get you blacklisted in the worst case).
I guess the other option here is to keep track of what we've seen bounce and just dump 'bad' emails into a database collection, although there are problems with this too (if someone signs up for that email address later, they're unable to sign up for the service; we still send at least one bounced email even in the best case).
My question is: is there a better option, one that I'm missing? This seems like the sort of issue that should be A Solved Problem™, but I'm not finding any applicable patterns.
This is a solved problem. Step 1 is just to validate the syntax of the email address (using something like EmailValidation) and then step 2 is to send an email to the address with a link for the user to click to verify their email address. Until the user clicks the link in their email, don't add the address to your database.
We have a service where we literally give away free money.
Naturally said service is ripe for abuse. To defend against this we do the following:
log ip address
use unique email addresses (only 1 acct/email addy)
collect more info like st. address, phone number, etc.
use signup captcha
BHOs (I've seen poker rooms use these)
Now, let's get real here -- NONE of this will stop a determined user.
Obviously ip addresses can be changed via a proxy (which could be blacklisted via akismet) but change anyways if the user has a dynamic ip or if more than one user is behind a NAT'd network (can we say almost everyone?)
I can sign up for thousands of unique email addresses each hour -- this is no defense.
I can put in fake information taken from lists for street addresses and phone numbers.
I can buy captchas from captcha solving services (1k for $5).
bhos seem only effective for downloadable software -- this is a website
What are some other ways to prevent multiple users from abusing the service? How do all the PPC people control click fraud?
I know we could actually call the person but I don't think we are trying to do that anytime soon.
Thanks,
It's pretty difficult to generate lots of fake phone numbers that can send and receive SMS messages. SMS verification could go a long way towards cutting down on fraud. Of course, it also limits you to giving away free money to cell phone owners.
I think only way is to bind your users accounts to 'real world' information, like his/her passport number, for instance. Of course, you'll need to make sure that information is securely stored and to find some way to validate it.
Re: signing up for new email accounts...
A user doesn't even need to do that. Please feel free to send your mail to brian_s#mailinator.com, or feydr.asks.a.question#spamherelots.com, or stackoverflow#safetymail.info, or my_arbitrary_username#zippymail.info. I haven't registered any of those email addresses, but all of them will work.
Those domains are owned by ManyBrain, and they (and probably others as well) set the domain to accept any email user. ManyBrain in particular then makes the inboxes for those emails publicly accessible without any registration (stripping everything by text from the email and deleting old mail). Check it out: admin#mailinator.com's email inbox!
Others have mentioned ways to try and keep user identities unique. This is just one more reason to not trust email addresses.
First, I suppose (hope) that you don't literally give away free money but rather give it to use your service or something like that.
That matters as there is a big difference between users trying to just get free money from you they can spend on buying expensive cars vs only spending on your service which would be much more limited.
Obviously many more user will try to fool the system in the former than in the latter case.
Why it matters? Because it is all about the balance between your control vs your user annoyance. I see many answers concentrating on the control part, so let's go through annoyance, shall we?
Log IP address. What if I am the next guy on the computer in say internet shop and the guy before me already used that IP? The other guy left your hot page that I now see but I am screwed because the IP is blocked. Yes, I can go to another computer but it is annoyance and I may have other things to do.
Collecting physical Adresses. For what??? Are you going to visit me? Or start sending me spam letters? Let me guess, more often than not you get addresses with misprints at best and fake ones at worst. In fact, it is much less hassle for me to give you fake address and not dealing with whatever possible spam letters I'll have to recycle in environment-friendly way. :)
Collecting phone numbers. Again, why shall I trust your site? This is the real story. I gave my phone nr to obscure site, then later I started receiving occasional messages full of nonsense like "hit the fly". That I simply deleted. Only later and by accident to discover that I was actually charged 2 euros to receive each of those messages!!! Do I want to get those hassles? Obviously not! So no, buddy, sorry to disappoint but I will not give your site my phone number unless your company is called Facebook or Google. :)
Use signup captcha. I love that :). So what are we trying to achieve here? Will the user who is determined to abuse your service, have problems to type in a couple of captchas? I doubt it. But what about the "good user"? Are you aware how annoying captchas are for many users??? What about users with impaired vision? But even without it, most captchas are so bad that they make you feel like you have impaired vision! The best advice I can give - if you care about user experience, avoid captchas as plague! If you have any doubts, do your online research first!
See here more discussion about control vs annoyance and here some more thoughts about being user-friendly.
You have to bind their information to something that is 'real world', as Rubens says. Of course, you also need to be able to verify this information (I can just make up passport numbers all day if you don't check to make sure they're correct).
How do you deliver the money? Perhaps you can index this off the paypal account, mailing address, or whatever you're sending the money to?
Sometimes the only way to prevent people abusing a system is to not have the system in the first place.
If you're doing what you say you're doing, "giving away money to people", then surprise surprise, there will be tons of people with more time available to try to find ways to game the system than you will have to fix it.
I guess it will never be possible to have an identification system which identifies fake identities that is:
cheap to run (I think it's called "operational cost"?)
cheap to implement (ideally one time cost - how do you call that?)
has no Type-I/Type-II errors
is scalable
But I think you could prevent users from having too many (to say a quite random number: more than 50) accounts.
You might combine the following approaches:
IP address: can be bypassed with VPN
CAPTCHA: can be bypassed with human farms (see this article, for example - although they claim that their test can't be that easily passed to other humans, I doubt this is true)
Ability-based identification: can be faked when you know what is stored and how exactly the identification works by randomly (but with a given distribution) acting (example: brainauth.com)
Real-world interaction: Although this might be the best one, but I guess it is expensive and not many users will accept it. Also, for some users/countries it might not be possible. (example: Postident in Germany, where the Post wants to see your identity card. I guess this can only be faced in massive scale by the government.)
Other sites/resources: This basically transforms the problem for other sites. You can use services, where it is not allowed/uncommon/expensive to have much more than 1 account
Email
Phone number: e.g. by using SMS, see Multi-factor authentication
Bank account: PayPal; transfer not much money or ask them to transfer a random (small) amount to you (which you will send back).
Social based
When you take the social graph (vertices are people, edges are connections), you will expect some distribution. You know that you are a single human and you know some other people. So you have a "network of trust" (in quotes, because I think this might be used in other context as well). Now you might not trust people / networks how interact heavily with your service, but are either isolated (no connection) or who connect a large group with another large group ("articulation points"). You also might not trust fast growing, heavily interacting new, isolated graphs.
When a user provides content that is liked by many other users (who you trust), this might be an indicator that there is a real human creating it.
We had a similar issue recently on our website, it is really a hassle to solve this issue if you are providing a business over one time or monthly recurring free credits system.
We are using a fraud detection solution https://fraudradar.io for a while and that helped us a lot to clean out most of the spam activities. It is pretty customizable with:
IP checks
Email domain validity
Regex rules
Whitelisting options per IP, email domain etc.
Simple API to communicate through
I would suggest to check that out.
I've gotten into a habit of using the standard register->send activation email->activate account process for every site that supports user authentication and free registration without questioning if I really need this.
What are your thoughts on this? If I have captcha on the registration form is the email confirmation process really necessary?
EDIT:
OK, so the general consensus seems to be that by getting the users to confirm the email they entered I'll keep them away from putting someone else's email in there.
What about when I let users edit their profile/settings and they enter another email?
If I need to keep them away from entering other people's addresses then I'd need to confirm that email address (by temporarily deactivating their accoun)t every time they change it.
Captcha+activation prevents bots AND spoofed people
Well basically it is since each part prevents one problematic scenario:
Captcha prevents (if you use strong captcha like reCaptcha) bots from registering new users
Email activation prevents people from registering other people (by their email address)
I guess this is a valid everyday pattern for registration that's widely acknowledged by IT community.
EDIT
Yes. When you want to prevent users from changing their email address, you'd have to repeat email activation procedure to make it robust.
But you don't have to deactivate their account while doing it. All you have to do is having a pending email-change email activation active. If it gets activated, you change email address at that point (not when they change it), otherwise the old one is still used.
If you don't confirm an e-mail, you're supposing that the user registering that service owns that email account. How can you start sending a lot of system e-mails, reset passwords and etc to a person that has nothing to do with your system? I would be really pissed of if it was my e-mail.
Another scenario: what if the register mispelled his e-mail when registering? Suppose he doesn't check his "account settins" in your application, doesn't change his email, and needs to reset his password. If the e-mail is registered in a wrong way, it's your fault for not checking it before.
Of course, I'm just saying this to services that would REALLY demand an account to be created. Avoid the login barrier when possible, or use openid when your service isn't so critical.
You should give serious consideration to supporting OpenID. http://openid.net/get-an-openid/what-is-openid/
The key benefit for OpenID is that it reduces the complexity for your user. There is no reason to force people to remember login credentials for hundreds of sites when a viable alternative exists. There is no worldwide netizen database - and there likely never will be - but OpenID simplifies the situation greatly.
I know that as a user I found the registration process for Stack Overflow to be painless and easy. I wish more sites used OpenID.
It's the lowest-level attempt at identity validation. It encourages users to re-use the same account when they return (by having a common, shared identifier you and they can use to reconnect), and it prevents impersonation, because it requires access to the claimed identity as proof.
It's not perfect, but something by definition works infinitely better than nothing.
If identity doesn't matter on your site (e.g. your service is throwaway after each use) then you don't need email activation. Otherwise, you probably want it.
On my site, I let users sign-up and do everything non-public until they confirm their email address. Because I run a gaming website, it means users can earn medals, post scores, just not post in the forum or post comments in the blog until they verify their email address.
I find it works pretty well. I have 16,000 registered users.
I find it both unnecessary and annoying. If I can, I avoid doing this.
However, I do do this if 1) email will be sent by the program, so I can test if the email address is valid, or 2) this is a very large, public-facing website, in which case I want to filter out as many potential problems as possible.
For most basic sites, I don't bother with either. Both email activation and captcha are relatively easy for dedicated spammers to bypass and overcome and do little but cause an annoyance to most of the users, driving away at least a certain percentage who might have otherwise signed up. I've found in my experience, focusing more on spam filters for member posted content has a better ROI overall.
For sites with more serious content, you'll typically have more serious users. In cases like that, I'll throw everything I've reasonably got available at it to counter the spam.
I find it useful when an email is sent for confirmation. This makes sure that I am the one who has registered with that email address.
Even with captcha you can register someone else email address although he may or may not approve that confirmation.
You only seem to need e-mail confirmation to confirm identity, not to send useful content by e-mail. But e-mail confirmation is only one means to an end. You may consider others, preferablly less intrusive ones.
Generally you can check something that
you are (e.g. fingerprint, iris scan)
you have (e.g. token, creditcard, key, access to an e-mail account)
you know (e.g. PIN, password, your mom's weight, name of your favorite deceased pet, the optimistic length of your most private bodyparts measured in inches)
Also, you can delegate the check to others; the creditcard company, phone company, someone's friends.
Example: GoogleMail could not ask
for a confirmation e-mail address
upon creation of your GMail account.
Instead, the early adopters had a limited supply of
"invites" to share with friends.
So - unless you actually need me to receive information you'd e-mail, which I generally hate anyway - you might be inclined to resort to more creative/fun means.
My organization has a form to allow users to update their email address with us.
It's suggested that we have two input boxes for email: the second as an email confirmation.
I always copy/paste my email address when faced with the confirmation.
I'm assuming most of our users are not so savvy.
Regardless, is this considered a good practice?
I can't stand it personally, but I also realize it probably isn't meant for me.
If someone screws up their email, they can't login, and they must call to sort things out.
I've seen plenty of people type their email address wrong and I've also looked through user databases full of invalid email address.
The way I see it you've got two options. Use a second box to confirm the input, or send an authentication/activation email.
Both are annoyances so you get to choose which you think will annoy your users less.
Most would argue that having to find an email and click on a link is more annoying, but it avoids the copy/paste a bad address issue, and it allows you to do things like delete or roll back users if they don't activate after say 48 hours.
I would just use one input box. The "Confirm" input is a remnant form the "Confirm Password" method.
With passwords, this is useful because they are usually typed as little circles. So, you can't just look at it to make sure that you typed it correctly.
With a regular text box, you can visually check your input. So, there is no need for a confirmation input box.
I agree with you in that it is quite an annoyance to me (I also copy and paste my address into the second input).
That being said, for less savvy users, it is probably a good idea. Watching my mother type is affirmation that many users do not look at the screen when they type (when she's using her laptop she resembles Linus from Peanuts when he's playing the piano). If it's important for you to have the user's correct email address then I would say having a confirmation input is a very good idea (one of these days I'll probably type my email address wrong in the first box and paste it wrong into the second box and then feel like a complete idiot).
While the more tech-savvy people tend to copy and paste, not technical people find it just as annoying to have to type something twice. During a lot of user testing I've down, the less tech-savvy - the more annoyed they seem with something like this... They struggle to type as it is, when they see they have to type their email in again it's usually greeted with a strong sign.
I would suggested a few things.
Next to the input box write the style of the information you are looking for so something like (i.e. user#domain.com). The reason this is important is you would be surprised how many of the less tech-savvy don't really understand the different between a website and an email address, so let them know visually the format you want.
Run strong formatting test in real time, and visually show a user that the format is good or bad. A green check box if everything is okay comes to mind.
Lastly, depending on your system architecture I often use a library to actually wrong a domain in the background. I don't necessarily try to run a VRFY on the server - I often use a library to check to make sure the domain they entered has MX records in it's DNS record.
I agree with Justin, while most technical folks will use the copy, paste method, for the less savvy users it is a good practice.
One more thing that I would add is that the second field should have the auto-complete feature disabled. This ensures that there is human input from either method on at least one of the fields.
Typing things twice is frustrating and doesn't prevent copy&paste errors or even some typos.
I would use an authenticate/activate schema with a roll back to the old address if the activation is not met within 48 hours or if the email bounces.
As long as a field is viewable, you do not need a confirm box. As long as you do some form validation to be sure that it is at least in valid format for an email address let the user manage the rest of the issues.
I'd say that this is ok but should only be reserved for forms where the email is essential. If you mistype your email for your flight booking then you have severed the two-way link between yourself and the other party and risk not getting the confirmation number, here on StackOverflow it would only mean your Gravatar would not be loaded ...
I'd consider myself fairly techie but I always fill in both fields /wo cut-paste if I regard it to be important enough.
I tend to have it send a verification code to the email address specified (and only ask for it once), and not change the email address until the user has entered the code I sent them.
This has the advantage that if they try to set it to a dozen different addresses in quick succession, you'll know which ones work by which verification code they put in.
Plus, if I am presented with a "confirm email address" box, I just copy and paste from the previous one, and if I'm guilty of that, I'm sure that other less careful users will do the same.