ColdFusion/Railo bulk email - Is there a better way? - mysql

My apologies if the answer is here on SO and I missed it. Anyway, I will give as much information as possible about my question.
I have a Railo CF server running on RHEL 6.6 that hosts an application to send out notifications and alerts via email. The emails are sent from the application to our internal cluster of Exchange servers using cfmail. For neatness and a few other reasons, all emails are sent via BCC.
The application was implemented to standardize the way important information is sent to the thousands of employees at my company. To do this, the application uses multiple form fields that require the end-user to enter specific information in each field. Once submitted the application then formats it into different templates depending on the content of the form fields. The end-user also has the capability to send file attachments.
The application works well, that is until the list of email addresses gets too large. Since the cfexchange tags do not currently have the capability to pull email distributions directly from Exchange, users must select email lists from either a pre-populated drop down or enter in individual email addresses in CSV to another form field. This for the most part is OK since the most used distributions are already in the drop down list. Unfortunately, some notifications sent by the application must be sent in CSV which can number in the thousands. When that happens, I get this error:
nested exception is:
class com.sun.mail.smtp.SMTPAddressFailedException: 452 #4.5.3 Too many recipients.
(If anyone knows if there is a fix for this via the admin or config files this may solve my issue without reading further.)
Thus I began my search for a better way. I first discovered this: need to slow mass email sending in coldfusion 4.5 however, the emails need to go out quickly and setting cfschedule to check constantly put a bit of a strain on the server. I could not find anything else here on SO that was relevant.
My google searches have not resulted in anything great. I found one suggestion of using cffile to write to the spool directory of either qmail or postfix on the application server. That method just seemed inefficient and unreliable to me. (However, if someone thinks this may work, please advise.)
I had one idea of; instead of immediately sending all those emails out through cfmail, to first insert the entire list into the local MySQL database and then writing CF code to query that database and somehow cfloop the cfmail tag. I couldn't come up with any code to do this without using cfschedule or forcing the user to wait until the process finished (Which could take a rather long time especially since the emails had to be relayed to our Exchange servers).
So, any help with this problem would be most welcome. Thank you!
EDIT
In response to just looping over the emails: This is something I considered. I know how to loop over a list of items but my concern was each email that is sent requires a login via the cfmail tag to Exchange since the application server does not log directly into our Windows domain. I could not figure out a reasonable way to test this since it would require sending out bulk emails.
The Exchange documentation on microsoft.com is vague at best. Unless I missed it, there does not seem to be a definitive answer on how many emails it can receive at a time before the 452 error code pops up. I could not find if this is something set from within the Exchange servers admin panel. I do not administrate the Exchange servers so I would have to contact that team at our company to find out.
EDIT #2
Adding in a code example for review. However, I am thinking the isValid() piece will fail if more than 1 email is passed at a time. Thoughts?
<cfloop index="bcc" list="#FORM.email_bcc#" delimiters = ", ">
<cfif isValid("email", bcc)>
<cfmail bcc="#bcc#"
from="#email_from#"
subject="#FORM.pre_email_subject# #Trim(FORM.subject)#"
type="html"
to="email#example.com">
Message goes here.
</cfmail>
<cfelse>
<cfmail
from="#email_from#"
failto="#fail_to#"
subject="FAILED #Trim(FORM.subject)#"
type="html"
to="#fail_to#">
</cfmail>
</cfif>
</cfloop>

Related

NetSuite SuiteScript 2.0 Integration with external hook

I need to setup EPL2 label printing from Netsuite. Unfortunately the company this is for is very small and they don't have much money to spend, hence they cannot buy a $1000 label printing solution.
The current system uses a linux server that then sends a file to one of the CUPS print server queues using the linux cat command. From there it goes to a Intel NetportExpress 10/100 Print Server and then to the Argox V1000+ label printer. This is via a corporate network ip address.
Instead I started looking at some cheap options:
Popup a browser window with content type text/plain and use a suitelet to populate that browser window with the EPL2 label printer codes. Then open a print dialog window so that the user can print to the label printer driver. This requires installation of the label printer driver for all users. Sadly I could not get this to print a label.
Integration from Netsuite via a Restlet to an external python application (on Linux) that can then perform the linux cat command needed to print the label. The Restlet works nice, but unfortunately there does not seem to be a way to have some sort of hook that fires when a new label custom record arrives. Therefore I have to keep on polling the Restlet from Python every 2 seconds to see if a new label is waiting to be printed. I started running this about an hour ago and so far I have made about 2500 requests without errors. My concurrency limit is 5 and I'm using 2 so that seems ok. The script does very little so I don't think there will be size limit issues. The problem is just that I wonder whether NetSuite will eventually terminate my script for doing so many requests. Not sure whether there is such a governance issue, but can't imagine that they won't eventually stop that sort of thing.
Use the http module to send data in an ajax type manner. This should be able to pickup when new data arrives instead of having to poll (not sure). The problem with this is that I assume I will need a static IP address which is sadly an expensive option.
Use Netsuite SOAP web services which might have a hook instead of polling (not sure). I think this would not be free (like Restlets) either.
So my question is whether there is a better option that I'm missing or what would you recommend. Also would I hit some sort of governance limit if I poll every 2 seconds with option 2?
Update: The polling mysteriously stopped working after 7395 requests and about 3 hours. It did not return an error that I'm aware of. The rejected requests on Integration Governance shows 0.
I used to do the emailing thing quite a bit and it works pretty well. Volume may be an issue.
Another thing to do is just get a static IP address with something like ngrok.
ngrok runs on linux/mac/windows so you'd be able to write an app that listens on a particular port. Netsuite would send an https post to that app at (for instance) https://printing.mycompany.ngrok.io and the app would handle local printing.
I believe ngrok runs about $US60/year.
the app can verify identity with some sort of timestamp and hash so that if someone does get the https address they couldn't easily use all your paper or cause a DoS situation.
We got bamboozled by a printer vender (Zebra) before we found out that we could HTTP post to most printers using PRINTER_IP:9100 and just sending the RAW ZPL/EPL as the body.
Look into: IPP enabled printers. most are these days. saves you 1000's in longrun if you have a large warehouse operation like we do
Instead of polling I would have NetSuite initiate the connection in an afterSubmit User Event script.
I've automated label printing by having NetSuite email attachments to a dedicated mail box which is monitored by a Linux server. My setup is documented here:
https://gist.github.com/michoelchaikin/80af08856144d340b335d69aa383dbe7

verify recaptcha html code

From the google site, I cam across a code that lets me display the recaptcha box. Although I must admit I have only started html today and hence, have no basics at all. The site does not have a verify code in html although it does in jsp and I dont know how to link it all since data from the html went to the servlet.
https://developers.google.com/recaptcha/docs/verify
specifis a list of parameters to be 'posted'
I am using Tomcat 7 and no Spring.
My doubts:
Can I have multiple forms in an html file
How do i send parameters from the precious form to the second form?
How do I link a jsp file that will verify it to be run through the java class invoked by the html file
How do I get the user (accessing my app through a local server) remote IP address?
Got sorted out. Since I did not understand the problem in that, I proceeded to simple captcha itself. I must admit I feel like a fool saying I din't know html/jsp. The second day it was all so clear. I must have been positively psyched. Unwilling to see the not working code again, I started fresh.
Answers :
I can have multiple forms in a html
as set and get parameter as a session and request functions respectively.
through a form itself or response.sendredirect("")
Remote address class and methods.

What is the simplest way to set up "reply to this email directly"?

I have a site that people communicate via private message.
I send npemail notifications to these people.
In the email notifications, I want the user to be able to hit reply on the email. My system would then analyze the subject line and write the message to the database for that user.
I have an email server set up (followed instructions to install dovecot and postfix). However, its a very basic set up and I'm completely clueless how to implement the system I just described.
I'm looking for a good overview answer or a link to a tutorial that goes through this.
There's a trick you can make use of: Mails to user+extension#domain go to the same mailbox as mails to user#domain. I'd use that instead of the subject line to find out which message the reply is about – this lets the user choose a custom subject line.
To let mails to a certain user go directly into a program, the probably easiest way is to put a .forward file into his home directory. Into that file, write | <path to executable>. The full mail (including headers) will be handed to the specified executable via stdin and a few environment variables with metadata (e.g. EXTENSION) will be set.
You might want to do man 8 local.

Importing csv-style text attachment from gmail into any kind of db/spreadsheet automatically?

Every hour, I get a csv style file (delimited by | (pipe)) delivered via email to a gmail address with a few rows of stuff like 12X98XJ|75.00|0.00||0.00|23.15
I'd like to automatically import to/update a database. I was thinking Google Docs "email to docs" functionality. Except helpfully they seem to have disabled that now.
I feel there MUST be a simple method in existence that does what I want.
Once it's in something where I can get at it with an API, it's plain sailing from then on.
Even something as simple as importing to Amazon SimpleDB would do.
But a good half a day of Googling just leads down disappointing paths.
Two notes:
All email functions are disabled on my server, so the .py scripts I found to retrieve from a local mail store file aren't going to work.
Don't ask me why the data is given to me in such a cack-handed way. It's historical.
I seem to be working with people getting ready to migrate to windows 3.1
What about using the pair of fetchmail + procmail

Disable page cache on a specific page

Not sure if I really am on the right forum, but if not, just tell me. I have a page that is coded in ASP (not .net) which is used to send email. We are currently having a problem in which the page seem to be sent twice sometime. Upon checking, we found out that those who have this problem are coming from big organisation, so it was suggested that their server might cache the file for some reason.
I would like to know, is there a way in HTML (or ASP ) to prevent that from happening ? Or is it in IIS that we must set this up ?
EDIT : I forgot to mention is that sometime, the time between the two mails can be in hours, not mere seconds
Thanks,
I don't see any cache problem here.
The only solution i see is to store somewhere server side(db, file system) the list of emails sent and check the list before send them.
With this approach, you will be sure to send just one mail to the specified address avoiding double submit or other possible problem.
I do not see how this could have anything to do with caching. After all, a cached page contains the generated html, and thus it would not trigger another execution of the code that sends the email.
However, my best guess is that it has to do with users that refresh the page. To avoid this, you could implement the post/redirect pattern, where after sending the mail you redirect to another page (or the same page but with different form parameters). This way the user can refresh the page as many times as he/she wants without triggering another email being sent.
If your problem is caching, there's really nothing you can do to keep an organization from caching it.
You can try to add this code to see if it makes a difference:
Response.Expires = 0
Response.Expiresabsolute = Now() - 1
Response.AddHeader "pragma","no-cache"
Response.AddHeader "cache-control","private"
Response.CacheControl = "no-cache"
If this doesn't work, you may need to contact that organization's IT department and ask them to add a caching exception for your page/site.