I am building a chat component(where all discussions are saved) for an application and I planned to have a long-polling server as an interface between the client machines and the rabbitmq server, that will parse format and enqueue messages in a job queue(on rabbitmq) to be handled by celery, which will then check if posting is allowed via checking the db for some information, etc, save the messages to a db, and enqueue them in rabbitmq again, this time to be consumed by the long-polling server to be pushed out to the client machines.
I planned to have one connection from the rabbit server to the long-polling server, containing many channels, that will each correspond to a nio socket channel, where a client machine is listening in on the other end.
The long polling server will be written using the Java Nio libraries. My question is, what would be an efficient way of sending the messages from the AMQP channel to the socketchannel to be pushed out to the clients?
From my understanding of NIO, the only time a selectionkey should be registered for OP_WRITE, is when a previously attempted write() call returned 0. In this case though, I am interested in writing to a socketchannel when information is present on the corresponding AMQP channel. How can I alert the socket channel once data comes in on its corresponding amqp channel so that I may send data through the socketchannel to the client?
Just call write()! No need to 'tell' the SocketChannel anything. Then if write() returns zero, do the OP_WRITE stuff.
Related
I am currently running a Django site on ec2. The site sends a csv back to the client. The CSV is of varying sizes. If it is small the site works fine and client is able to download the file. However, if the file gets large, I get an ERR_EMPTY_RESPONSE. I am guessing this is because the connection is aborting without giving adequate time for the process to run fully. Is there a way to increase this time span?
Here's what my site is returning to the client.
with open('//home/ubuntu/Fantasy-Fire/website/optimizer/lineups.csv') as myfile:
response = HttpResponse(myfile, content_type='text/csv')
response['Content-Disposition'] = 'attachment; filename=lineups.csv'
return response
Is there some other argument that can allow me to ignore this error and keep generating the file even if it is taking awhile or is large?
I believe that you have any sort of backend proxy server which resets the connection to the Django backend and returns ERR_EMPTY_RESPONSE for the case. You should re-configure timeouts on your backend proxy. Usually that is nginx or apache used as a reverse proxy server.
What is Reverse Proxy Server
A reverse proxy server is an intermediate connection point positioned at a network’s edge. It receives initial HTTP connection requests, acting like the actual endpoint.
Essentially your network’s traffic cop, the reverse proxy serves as a gateway between users and your application origin server. In so doing it handles all policy management and traffic routing.
A reverse proxy operates by:
Receiving a user connection request
Completing a TCP three-way handshake, terminating the initial connection
Connecting with the origin server and forwarding the original request
More info at https://www.imperva.com/learn/performance/reverse-proxy/
One more possible case - your reverse proxy backend server doesn't have enough free space to process response from Django and aborts the request. You can also check free space on your reverse proxy balancer.
Within gunicorn, there is an argument for timeout, -t. When you run gunicorn, the default timeout is 30 seconds. Increase that to something your comfortable with like 90 or 120 seconds, whatever you think fits your application.
I have a EJB application running on Glassfish server which stores data on MySQL DB which I call as Global DB.
I have two exact remote Swing applications which are stand alone applications accessing EJB's using RMI. They have their own local DB in case of lost connection.
My aim is to implement two phase commit protocol i.e to make one participant as coordinator and others as participants.
One method which I could think of was to implement using JMS i.e send a message across queue and make remote clients listen to these messages and take appropriate action.
I do this my sending a message on Buttonclick of one of the Swing application.
Problem is, even tough I have implemented MessageListener, onMessage() method does not receive any message for the other client.
Each Remote client has following properties set:
props.setProperty("java.naming.factory.initial", "com.sun.enterprise.naming.SerialInitContextFactory");
props.setProperty("java.naming.factory.url.pkgs", "com.sun.enterprise.naming");
props.setProperty("java.naming.factory.state", "com.sun.corba.ee.impl.presentation.rmi.JNDIStateFactoryImpl");
props.setProperty("org.omg.CORBA.ORBInitialHost", "localhost");
props.setProperty("org.omg.CORBA.ORBInitialPort", "3700");
This is to connect to Glassfish server and access the connectionFactory and Queue which I have already configured.
Is it because only application running on server are allowed to receive messages and not remote applications?
Any suggestions for topology for 2 PC are welcome.
For this, we used JMS for exchanging the messages between these systems i.e one acting as coordinator who will initiate the process by sending message on the queue and others will respond accordingly by sending back again a message on the queue.
Since you are using EJB,you can use JTA to manage transcation,it a standard implementation of two-phased commit protocal,and JMS support JTA too.
Here are my steps:
config the trans-attribute to Required/Mandatory /Supports, depends on you need.
in your client get UserTransaction by lookup jndi from the EJB server.
start the transaction from client.
commit/rollback the transaction at client side
This is the so called "Client owner tranaction design pattern". I suggest you to read the book javatransactionsbook
Is it possible for multiple low-end computers to each make CUDA calls to a GPU located on a central server in a request/response over the cloud scenario? To make it as if these low-end computers possess a "virtual" GPU.
I had a similiar problem to solve.
The database was living in the low end machine and I had a cluster of GPUs in my disposal on the local network.
I made a small client (on the low end machine) to parse the database, serialize the data with google protocol buffers and send them to the server with zmq sockets. For data distribution you can have asynchrouns publisher/subscriber sockets.
On the server side you deserialize the data and you have the CUDA program to run the calculations (it can also be a daemonized application so you dont have to fire it up yourself every time).
Once the data is ready on the server you can issue a synchronous message (request/reply socket) from the client and when the server receives the message it calls a function wrapper to the CUDA kernel.
If you need to process the results back on the client you can follow the reverse route to send the data back to the client.
If the data is already in the server, its even easier. You only need the request/reply socket to send a message and call the function.
Check the zmq manual, they have a lot of examples in many programming languages.
I already have a server with port and want to write a web app to get the information form the port. Will this be possible with WebPorts?
The Client doesn't even need to talk back to the server, which is the whole point of websockets I would imagine, but since I already have the ports setup, I might be easier and cleaner to just connect and get the info without having to refresh.
WebSockets are not intended as clear TCP channels over which other existing protocols can be implemented.
WebSockets are designed to allow messages to be sent between a client and server, where an event is raised each time a message is received.
Hence a WebSocket client cannot simply connect to an existing TCP server - that server also has to speak the WebSocket protocol.
You could of course write a WebSocket-based server that does nothing but act as a proxy to existing network services.
I think you want websockify which is a WebSocket to plain TCP socket bridge/proxy. It also allows sending and receiving of binary data with the older version of the WebSocket protocol which hadn't yet added direct binary data support.
Disclaimer: I created websockify.
I was wondering if anyone might be able to explain to me why it's taking nearly 30 seconds each time my Java web app sends an email using Gmail's SMTP server? See the following timestamps:
13/04/2010-22:24:27:281 DEBUG test.service.impl.SynchronousEmailService - Before sending mail.
13/04/2010-22:24:52:625 DEBUG test.service.impl.SynchronousEmailService - After sending mail.
I'm using spring's JavaMailSender class with the following settings:
email.host=smtp.gmail.com
email.username=myaccount#gmail.com
email.password=mypassword
email.port=465
mail.smtp.auth.required=true
Note that the mail is getting sent and I'm receiving it fine, there's just this delay which is resulting in a slow experience for the application user.
If you know how I can diagnose the problem myself that would be good too :)
It might be that gmail uses this delay to prevent spammers from using their SMTP server from the "outside": if the SMTP is called from the actual webmail client it would not use this delay.
In order to test this, you could open a telnet session to port 25 and do a manual SMTP session
(Search the web for SMTP HELO for examples)
GMail uses SSL for connection. To validate SSL certificate properly certain time is needed. Exact time depends on complexity of the certificate chain and use and accessibility of CRLs and OCSP checking. It's very likely that Java checks OCSPs for the certificate(s) presented, and this can be slow.
Try running your application on different system (preferably in other subnet) to exclude influence of firewalls. Maybe some part of communication (either SMTP itself, or CRL retrieval or OCSP checking) is slowed down by the firewall.
Depending on your OS, you should have a proper mail system that takes care of actually sending mails outside your network.
For instance, Postfix can do the job nicely. The configuration of Postfix would be on another thread, but typically Postfix would accept mails from your LAN (accepting to be a relay for your LAN) and would transfer the mails to gmail.
This is a exemple as to how setup postfix to have gmail as relay.
Having a mail service taking care of the relay of Gmail mails allows your Java application to be freed almost instantly (Postfix absorbs the mail, store it in its queue and forward it to Gmail itself).