I need to implement an intelligent mailing list/relay (on Linux). The idea is that:
The server receives emails to a list address
It parses the mail, and confirms that it's from a trusted source
It looks up a list of recipients in a local database
It does some minor processing on the incoming mail, and sends it out to the list
It returns any bounce messages to the original sender
The server already has sendmail installed, but I can use another MTA if necessary.
This sounds straightforward, and sendmail already has a database look-up capability. However, I don't think this is particularly flexible, and I don't want to give sendmail independent access to my database.
The closest I've come to an existing solution is E-MailRelay, which looks good, but I don't want the overhead of integrating it if I can avoid it.
I'd appreciate a sanity check on my Plan B before starting it, or alternative suggestions. I haven't found any useful docs on this and the Sendmail book doesn't seem to have anything relevant in it.
What I'm thinking about is:
Implement an SMTP delivery agent for sendmail, and have sendmail and the DA running on the same server, with the DA listening on some unspecified port (which?)
Sendmail presumably acts as an SMTP client when connecting to the DA, and my DA will respond to MAIL/RCPT/DATA commands
The DA processes the received mail, which will be either a message out to the mailing list, or a bounce, or possibly a response
The DA then switches to client mode, connects to sendmail, and issues MAIL/RCT/DATA commands to return the response to the original sender
Does this make sense? Thanks.
This turned out to be pretty straightforward, although I didn't use a sendmail delivery agent - I just did everthing in SMTP. The server has two IP addresses, and sendmail is already listening on port 25 on IP#1.
I wrote an SMTP proxy which listens on port 25 on IP#2. This runs an SMTP server, which accepts incoming messages, and re-writes them. It then connects (as a client) to port 25 on IP#1, sending the re-written message to sendmail. sendmail then handles transmission to the re-written destination addresses. This is all transparent to the original mail client.
Not sure how I'd do this if the server only had one IP address, though.
Related
I am using PHP's mail function to send messages to the users of an application.
But I have been getting reports that some people don't receive them, and that others have arrived up to a couple of hours later.
So I was looking through /var/log/maillog, where I came across this message a lot:
postfix/smtp[22966]: connect to mx1.hotmail.com[65.55.37.104]:25: Connection timed out.
There were a lot of delays over 1000 sec in the log.
I have checked the firewall settings, and there are no rules stopping port 25, and this problem is not consistent, some emails go out fine, while others are sent with a long delay, and others are not sent at all.
Any suggestions for what to try next?
Running on CentOS 6.5.
From what you describe, it sounds like some of the remote mail servers that you are sending to could be greylisting your mail server, perhaps because it thinks that your mail server could be a spammer.
To see if your mail server has a glaring problem that would cause other mail servers to think it's a spammer, try sending a message from your mail server to check-auth#verifier.port25.com. This service will do a bunch of checks, and you'll get a report back with ton of information, such whether or not your mail server's DNS is setup correctly, whether your mail server's IP is on any black lists, if you have a problem with your SPF records, etc.
I know we can send email from php using smtp servers on different hosts or if there is local smtp server installed. What I want to know is can php send email without any local or remote smtp servers? I have heard about sendmail program but can it function without any mail server installed in the server?
At some point you have to talk to a SMTP server. Sending via a SMTP server on the local host is the cleanest option and the most likely to succeed at getting through spam filters.
What a mail server does is quite complex. Let's take your average e-mail as it arrives from your e-mail client to your e-mail server with an outbound host as the destination:
The server checks your user account and makes sure it is valid.
The e-mail goes into a queue either separately for each recipient or as one message (depends on the server).
The server finds the e-mail in the queue and processes each recipient address. This requires a DNS lookup for a MX record for each target domain.
The e-mail server connects to the address specified by the MX record and delivers the e-mail to it as one does over SMTP.
On success, the e-mail is removed from the queue. On failure, the e-mail may remain in the queue and the server will try again later (exponential backoff - see greylisting) or be put in the mail queue to be returned to you when you check your e-mail via POP3 later.
The next e-mail server in the queue then repeats the above until the final server receives the e-mail and sits in the recipient's mailbox.
Doing that within PHP is possible, but I don't recommend it. MX record lookup can be tricky because people do all sorts of non-compliant things that mail servers tolerate. Plus, your script might time out while attempting to connect directly to the target SMTP server. Some servers are also configured to "greylist" e-mail, which means the e-mail will initially be rejected but would be accepted later (e.g. 30 minutes is not unusual). The average PHP script won't be able to handle that scenario.
I want to write both client and server sides of sending an e-mail also I should attend to the standard format of SMTP protocol which is specified in RFC 821.
Would you please help and guide me that how can I start it?
Why would you reinvent the wheel? There are many implementations for both sides that do a very good job and are well tested.
What feature do you miss in let's say Postfix (server) and Outlook (client)?
If you still want to start, ask yourself
how to create TCP Sockets and read/write to them
how to do connection and thread pooling
how to implement a state machine
what are you doing with related topics? (SPF, DKIM, mbox storage, Maildir storage)
what are you going to use for user authentication and authorization
(... enter hundreds of other things here ...)
I was wondering if anyone might be able to explain to me why it's taking nearly 30 seconds each time my Java web app sends an email using Gmail's SMTP server? See the following timestamps:
13/04/2010-22:24:27:281 DEBUG test.service.impl.SynchronousEmailService - Before sending mail.
13/04/2010-22:24:52:625 DEBUG test.service.impl.SynchronousEmailService - After sending mail.
I'm using spring's JavaMailSender class with the following settings:
email.host=smtp.gmail.com
email.username=myaccount#gmail.com
email.password=mypassword
email.port=465
mail.smtp.auth.required=true
Note that the mail is getting sent and I'm receiving it fine, there's just this delay which is resulting in a slow experience for the application user.
If you know how I can diagnose the problem myself that would be good too :)
It might be that gmail uses this delay to prevent spammers from using their SMTP server from the "outside": if the SMTP is called from the actual webmail client it would not use this delay.
In order to test this, you could open a telnet session to port 25 and do a manual SMTP session
(Search the web for SMTP HELO for examples)
GMail uses SSL for connection. To validate SSL certificate properly certain time is needed. Exact time depends on complexity of the certificate chain and use and accessibility of CRLs and OCSP checking. It's very likely that Java checks OCSPs for the certificate(s) presented, and this can be slow.
Try running your application on different system (preferably in other subnet) to exclude influence of firewalls. Maybe some part of communication (either SMTP itself, or CRL retrieval or OCSP checking) is slowed down by the firewall.
Depending on your OS, you should have a proper mail system that takes care of actually sending mails outside your network.
For instance, Postfix can do the job nicely. The configuration of Postfix would be on another thread, but typically Postfix would accept mails from your LAN (accepting to be a relay for your LAN) and would transfer the mails to gmail.
This is a exemple as to how setup postfix to have gmail as relay.
Having a mail service taking care of the relay of Gmail mails allows your Java application to be freed almost instantly (Postfix absorbs the mail, store it in its queue and forward it to Gmail itself).
I am trying to setup windows server 2008 smtp server to relay emails to gmail smtp. Everything appears to be setup but it is not sending emails. Could you please help me figure out whats wrong.
Below is the setup:
Windows server 2008 with SMTP server
feature installed. Need SMTP server
to forward all messages to gmail smtp
server to send.
I have google apps setup for my
domain, also I can send emails
throught my test app using
gmail smtp.
SMTP Server Configuration: By default has default smtp server virtual directory.
In Properties of that virtual smtp server changed following.
Fully qualified domain name = mydomain.com
smart host = smtp.gmail.com
TCP Port = 587
Out Bound Security = Basic Authentication(my username password for google apps email account)
In domains list under virtual smtp server. I have one default domain that's server dns. I added another one for my domain name.
With above setup i am trying to redirect all email to gmail smtp.
I tested connection to smtp.gmail.com from server on port 587 through telnet and it works.
I am trying to use above server from my web application also by just dropping emails in pickup directory. It get's picked up and also accepts request form web application but never sends an email.
I can see that it adds those emails in queue folder but it stays there forever.
When i try to send emails from web app to above server it rejects if To address is other than my domain.(Am i missing something in list of domains)
Thanks for all answers, finally found solution there is a property for maximum sessions which value was 0 by default. Changed it to 100 and it send all pending emails immediately.
Possible reasons are that some SMTP servers block the outgoing messages if there domain name mismatch, possible to prevent spam mails from being sent. So for example, I will not be able to send my email with an address abc#mydomain.com from my domain yourdomain.com.
Hope that helps.
Ensure your sending domain is the same as the google apps domain
Ensure your sending address is a real address and not just an alias
IIRC you need to use STARTTLS (SSL) not basic authentication
This souds like a DNS issue. Check your /badmail directory. It will have .bad and .bdp files in there. You can open these in notepad (there will be some binary in there).
However, it may point to the possible problem.
You may also want to try and enable logging on the SMTP service. There may be something in there.