I've got a PHP app with an invitation system where users can invite other users to try the service. Internally we use google apps for our domain to send/receive emails (mydomain.com).
1) My question is, can I send emails from my server with the from address being invite#mydomain.com? I am worried about the emails being blocked/ignored by the destination server. I am aware that it is possible to send the emails by configuring my php installation to use google smtp server, but there is a limit of 500 emails a day, which is not very scalable.
I don't really know that much about sending emails and why/how they are blocked/considered spam. I'd appreciate any good advice/tips you can give me.
2) What is a good way to test to see if the email portion of my app is working without installing it on my live server. Can I just setup an smtp server on my desktop and send mails this way? Can you recommend any other good ideas for testing. I'll basically be sending just a few emails to my personal webmail accounts to make sure that everything works.
Thanks,
Bill
1) My question is, can I send emails
from my server with the from address
being invite#mydomain.com? I am
worried about the emails being
blocked/ignored by the destination
server. I am aware that it is possible
to send the emails by configuring my
php installation to use google smtp
server, but there is a limit of 500
emails a day, which is not very
scalable.
I don't really know that much about
sending emails and why/how they are
blocked/considered spam. I'd
appreciate any good advice/tips you
can give me.
There is a way track if mail has been bounced (there are more than 10 possible bounce reasons!). You can set the return-path header in your outgoing emails. Best practice is to specify a different mail address in the return-path. When e-mails are getting bounced for whatever reason, a notification will be sent to this address. Additionally you can have for example a (PHP) cron job that connects using IMAP to the bounced email account and do something with the bounced e-mails. This is a pretty reliable way to track the status of your sent emails.
Additionally, in order to minimize the chance your e-mail will get blacklisted you could think about signing your e-mails using a certificate (you can get one for free for personal usage. A commercial one may cost you around 25 dollars a year)
2) What is a good way to test to see
if the email portion of my app is
working without installing it on my
live server. Can I just setup an smtp
server on my desktop and send mails
this way? Can you recommend any other
good ideas for testing. I'll basically
be sending just a few emails to my
personal webmail accounts to make sure
that everything works.
You can actually send a test email from everywhere as long as the outgoing SMTP port (25) is not blocked. If you have an own smtp server with username/passwd authentication enabled, you will be able to send e-mails from everywhere using the these credentials/settings. In all other cases, you will have to use the smtp of your internet provider to send emails.
To address the second part (as Eric pointed out, you'll have better luck at serverfault.com with the first part), any locally hosted SMTP server should be able to do the trick, and there are plenty available for any given OS. Google can help you there.
The main thing you'll want from a local SMTP server is detailed logging. It's entirely possible that the local server could fail/refuse to deliver the message to its intended destination for any number of reasons (again, serverfault.com), but that's outside the scope of testing the code's delivery of the email to the SMTP server.
If it does properly forward the test message to you, great. But if it doesn't, you just want to be able to see in the server's logs that it received the message correctly and was able to process it. Whatever that processing accomplished is a separate issue.
For email testing I use Pappercut. It's easy to use but some antivirus may not like you opening port 25.
I use Dumbster for testing. I will catch the emails, then my test code can check the content.
To avoid spam, there are a number of things you have to do, and I'm not sure I've found them all. Make sure that your IP is registered, and that a reverse lookup returns the right domain.
1) Sending:
This is a good article describing some of the pitfalls around sending email http://www.codinghorror.com/blog/2010/04/so-youd-like-to-send-some-email-through-code.html
Check out the comments too.
2) Testing:
Disclaimer - I work for the company behind the service linked to below.
If you would rather not set up your own smtp server you can use a hosted email testing service like Clickity
You can create as many test email addresses as you like or configure your app to point directly at our smtp server. You can then view the complete email on our site as part of your manual testing or automate the tests it via our API.
Related
I have a gmail account that I want to use for one of our internal services. We have a server running and we want this gmail account to be used to send out an automated email to people who register.
Now, here's the problem. I have correctly configured my gmail id in the server and know that I am able to correctly ping the gmail smtp server. However,I always get stuck because gmail by default expects you to verify any new device you log in from using an OTP. In my case, this is the first time login from this server machine and hence this happens. I can see it in the logs as well - it brings up this device verification page.
Some questions:
Is there a way I can turn off this feature in gmail? I don't think this is the same as 2 step verification. I have turned it off. I have also enabled my gmail account to be accessible from non secure apps.
If not, is there a way to do the verification step via command line on my Linux server?
Any other smtp based email service I could use that does not have so many security features?
Lastly, I am doing this for a quick prototype. Therefore I am ok bypassing some of these aspects. However, I would definitely want to ensure that the access to my account is secure once this is productized. Any links on how we can do this from non google apps or services?
Thanks in advance!!
It sounds to me you have an application that needs to work on your behalf and do some gmail-related tasks.
If that's the case, why not use the Gmail API with Service Account impersonation?
In short, a service account can impersonate a real user. This is a process that does not require manual input after it's setup, so it's ideal for server-side usage, where you might not even have a UI to interact with.
Also, the Gmail API is much easier to set up than Gmail SMTP, and there are examples out there on how to do this with this setup. You can look at this answer for some references on what it would take.
Google offers two systems for accessing Gmail. IMAP and SMTP and a the Gmail rest api Gmail - Scope for SMTP is https://mail.google.com/. However with Gmail rest API, just the required scope (like send, modify) can be used.
What are the main differences between the implementation of these two for sending an email? I've been using SMTP to send the mails without any issues but since that involves having a bigger scope for OAuth2, I want to know if there are any possible risks involved in moving to the API approach.
Users.messages: send says there's a restriction on attachment size.
This method supports an /upload URI and accepts uploaded media with
the following characteristics:
Maximum file size: 35MB Accepted Media MIME types: message/rfc822
Are there any other differences that I should know about if I start using Gmail APIs instead of using SMTP connection for OAuth2.
Also, what is the reasoning behind providing full access as the only possible scope for SMTP/IMAP?
Note: I only requirement is the ability to send emails.
Using SMTP you are directly accessing the mail server located at mail.google.com. SMTP servers have been around since the 60's they don't have the ability to limit what access you have. When you log in you have full access to do what ever the mail server in question is capable of. To login to the SMTP server you need the login (most often email address) and password of the account you wish to access. Drawback to using the SMTP to connect to Gmail is that if the user changed the password you would then loose access. This day in age it is also considered by most to be bad practice for third party developers to be storing a users login and password in your system. For example: I would never give any application access to my login and password to Google. How could you ever prove to me that your system is secure? If your hacked so am I.
Now on to Oauth2. Oauth came about sometime around 2005 when people wanted to be able to access APIs without having to do something stupid like
http://awsom.api.com?login=xxx&password=XXX
If memory services it was originally created for the twitter API developers wanted to be able to access their users twitter account without having to store their login in and password. Again the main problem with this was the developer in question would then have full access to a users twitter account and if the user or the developer changed the password things would break.
So they created OAuth. The main features with OAuth are:
You can limit access you give an application: (readonly, read write)
Password change does not affect access
No sharing account credentials with developers of third party apps
So the main point for me as a developer using Oauth with any Google API would be not having to store the login and password of my users and not being affected by a password change. My users would probably say not having to share their login with me and being able to give my application limited access to their account.
Now back to Gmail. Google made a change about two months ago any refresh token(oauth2) that was created using a Gmail scope will automatically expire when the user changes their password. To my knowledge this is only Gmail. so that removes point number two from the features of oauth.
Which should you use is really up to you, assuming you need to be able to send emails. Then limiting access to read only in your application isn't something you need (point one). However in my opinion from a security standpoint I would never ask my users to give me their login and password and would always choose oauth2. Yes SMTP works, will Google shut it down, probably not users have always been given access to the direct SMTP server of their email provider its how applications like outlook work.
as for OAuth support with SMTP unfortunately I haven't done much research into that guess I need to read RFC 4422 . If you can use OAuth with SMTP servers then again I guess the question would have to come down to speed is it faster to access the SMTP server or the REST API server? I can really think of no differences. Attachments with the Rest API can be tricky. I may do a bit more digging on the subject.
Usually, when you send an e-mail (with Thunderbird or Outlook), you don't send it directly.
example: I have a gmail address and I want to send an e-mail to a myopera address. The process will be:
user->gmail server(gmail-smtp-in.l.google.com)->myopera server(in1.smtp.messagingengine.com)->final user who'll download it's email with pop/imap.
One of the inconvenient is the size: Imagine you have attachment of 50Mb: the limit of myopera is 60MB; but the limit of gmail is 25MB,So the mail will be refused whereas it would be accepted if it was send directly to myopera.
But I saw with telnet that, it is possible to send mail directly with SMTP commands.
I want to write a client-side Only web application which would convert a mail in a set of SMTP commands for sending it. I should be very basic and not support encryption
I don't know how to create a TCP connection from a client, so, here's my questions: Does a library already exist? If not, what I should use? I've read about the existence of WebSockets but that Ajax would be more universal.
Also, most of the actuals implementations of WebSocket I saw, don't work in my latests versions of web browser despite the fact they 'support it'. There's also the raw Socket API from the W3c (I've no idea of the web browsers which actually support it).So, I would like to not avoid statements telling it is impossible to create near raw TCP/UDP session. Since it is possible, I can't imagine nobody created a kind of library for dealing with protocols
You should take an alternative route.
If i had that issue i would still use a server side component of some sort, and just have the server contact to receiving mail server directly.
Given the email: "someuser#somedomain.tld" you could do a DNS MX record lookup on "somedomain.tld" and find the receiving mail server say "mail.somedomain.tld", then you could tell your mail send component to send the email directly to "mail.somedomain.tld", that way you would have an immediate feedback on whatever the mail went through or not.
For Objective-C you may use https://github.com/jetseven/skpsmtpmessage
By looking at the source you see how SMTP works.
is there any way i install SMTP service on my Linux server and use php to send emails through it ?
so that emails doesn't show up as spam ?
maybe through ssl ?
my mail() function is perfect , with perfect headers , but still sending to spam :(
Thanks .
Cross-posting myself from another answer:
A few bullet points (Assuming that mail() returns true and there are no errors in the error log) :
Does the sender address ("From") belong to a domain on your server? If not, make it so.
Is your server on a blacklist (e.g. check IP on spamhaus.org)? This is a remote possibility with shared hosting.
Are mails filtered by a spam filter? Open an account with a freemailer that has a spam folder and find out. Also, try sending mail to an address without a spam filter.
Do you possibly need the fifth parameter "-f" of mail() to add a sender address? (See mail() command in the PHP manual)
If you have access to log files, check those, of course, as suggested above.
Do you check the "from:" address for possible bounce mails ("Returned to sender")? You can also set up a separate "errors-to" address.
There are plenty of variables that could potentially be involved here.
Your idea of a perfect header seems subjective. I would recommend you use something like PHPMailer or Swiftmailer to handle your mailing needs.
Some other variables you might need to take into account are the processing of bounced emails, and making sure you send your mail in chunks as opposed to one big send off.
You can recheck your headers, maybe there might a space or something in the header from address, you can also check if the ip of your server is blacklisted (http://www.anti-abuse.org/multi-rbl-check/)
Is it possible to retrieve the SMTP response of a mail. For example, I am sending a mail to non existing email id. Surely our server will send us a mailer daemon failure mail to our mail id. I need to capture that failure mail.
How its possible? please explain me. Some time we may enter more than one non existing email id, so i have to retrieve all the failure mail alone for every corresponding emails
Please guide me!
Thanks in advance,
Praveen J
I think I understood your question correctly now. As I understand, you are writing an application to send mail. And in your application, whenever you send a mail, you also want verify that if mail was delivered and also if it was not delivered then you want to get hold of the failure message in your application. Is that how you mean?
Well, if that is how you mean, then I think it is impossible to track the mail status with your apllication code. For instance if you are using java sendMail in your apllication you can only ensure that the send happened from your code successfully(without any send exceptions like java.net.SocketException or javax.mail.MessagingException). But, you can never ensure if the mail really reached the recepient. i.e. you can never track in your application if the mail was rejected due to wrong recepient address or any other error like illegal attachment at receipient mail server or errors like blocked sender id etc.
That is because any such error condition will be communicated by the receipient mail server to the sending mail server the information of which is present in the sent mail's header.
Does that answer your question? (Or did I understand your question correctly? ;-))
I am not sure if I am getting your question right. If you send an email to any non existent address say xxx#gmail.com from your address yyy#yourhost.com, the mail server at gmail.com replies to the mail server at yourhost.com with failure message and reason, with your delivery address and you receive the fialure mail automatically. you don't have to do anything extra in this.
If you are talking about seeing mail headers, then it depends on which client you are using. For instace, if you are using MS outlook, you can right-click on the message and click options and then see internet headers section to get mail headers. If you are using some web based mail then i am sure there will some option to view detailed mail headers.
The bounced messages are going to return to a mailbox. You should be able to configure that mailbox by properly setting the headers on the messages you send out. You would then need to monitor that mailbox, or have that mailbox deliver the messages to your program.
I would suggest you consider using VERP for all messages you send out. It will make it much easier for you to identify which email address a particular bounce belongs too. To do this you would need control of your mail server though. It takes some work configuring things.
To answer the question with more detail you need to tell us how your are sending messages, what type of mail server you are running, and how much control you have over the mail server.
On Unix, you can use "procmail" for this. Procmail is a service which can intercept your mails and process them following rules.
If you can access your mail my IMAP, I suggest to look at the Python module imaplib.