How to set up Tomcat for one Database Connection per Request - mysql

I have a Sparkjava app which I have deployed on a Tomcat server. It uses SQL2O to interface with the MySQL-database. After some time I start to have trouble connecting to the database. I've tried connecting directly from SQL2O, connecting through HikariCP and connecting through JNDI. They all work for about a day, before I start getting Communications link failure. This app gets hit a handful of times a day at best, so performance is a complete non issue. I want to configure the app to use one database connection per request. How do I go about that?
The app doesn't come online again afterwards until I redeploy it (overwrite ROOT.war again). Restarting tomcat or the entire server does nothing.
Currently every request creates a new Sql2o object and executes the query using withConnection. I'd be highly surprised if I was leaking any connections.
Here's some example code (simplified).
public class UserRepositry {
static {
try {
Class.forName("com.mysql.jdbc.Driver");
} catch (ClassNotFoundException e) {
e.printStackTrace();
}
}
protected Sql2o sql2o = new Sql2o("jdbc:mysql://mysql.server.name/dbname?serverTimezone=UTC", "username", "password");
public List<Users> getUsers() {
return sql2o.withConnection((c, o) -> {
return c.createQuery(
"SELECT\n" +
" id,\n" +
" name\n" +
"FROM users"
)
.executeAndFetch(User.class);
});
}
}
public class Main {
public static void main(String[] args) {
val gson = new Gson();
port(8080);
get("/users", (req, res) -> {
return new UserRepository().getUsers();
}, gson::toJson);
}
}

If you rely on Tomcat to provide the connection to you: It's coming from a pool. Just go with plain old JDBC and open that connection yourself (and make sure to close it as well) if you don't like that.
So much for the answer to your question, to the letter. Now for the spirit: There's nothing wrong with connections coming from a pool. In all cases, it's your responsibility to handle it properly: Get access to a connection and free it up (close) when you're done with it. It doesn't make a difference if the connection is coming from a pool or has been created manually.
As you say performance is not an issue: Note that the creation of a connection may take some time, so even if the computer is largely idle, creating a new connection per request may have a notable effect on the performance. Your server won't overheat, but it might add a second or two to the request turnaround time.
Check configurations for your pool - e.g. validationQuery (to detect communication failures) or limits for use per connection. And make sure that you don't run into those issues because of bugs in your code. You'll need to handle communication errors anyways. And, again, that handling doesn't differ whether you use pools or not.
Edit: And finally: Are you extra extra sure that there indeed is no communication link failure? Like: Database or router unplugged every night to connect the vacuum cleaner? (no pun intended), Firewall dropping/resetting connections etc?

Related

akka.net first published message ends up in the dead letter queue, handshake problem

I have an issue with an akka.net message send/Tell that ends up in the dead letter queue.
I developed a cluster based application using Akka.Cluster.Tools.PublishSubscribe with two ActorSystems each running in a 'console.application' on the same machine.
I start up one actor system with some actors. Then I start up my 2nd. application and immediatelly after I initialized the Actor system I publish the first Message Mediator.Tell(new Publish(Topics.Backend.SomeName, new MyInitialMessage())) to a Topic where the receiving actor is hosted in the 1st. application.
This message ends up in the dead letter queue always.
Ok now, instead of sending the message immediatelly I put in a delay of e.g 5sec. Then the message could be delivered properly.
This seems to me as a handshake problem.
Question: How do I find out when the 2nd. actor system is ready to receive any messages??
My current workaround is: I send scheduler based for each second a MyInitialMessage and wait for the first response message from my 2nd. application. Then I know my 2nd. app is now ready, handshake done.
But this seems to me just as a workaround. What would be a proper solution to this issue?
chris
Akka.Cluster.Tools.PublishSubscribe works over cluster. You need to await for cluster to become initialized before you'll be able to publish any messages. All of cluster operations are encapsulated in Cluster class that can be created from any actor system using Cluster.Get(actorSystem). In order to wait for cluster to initialize:
You can join to cluster programmatically by using await cluster.JoinAsync(address, cancellationToken) - you can use it to initialize both seed nodes (just make actor system join to itself) and new nodes. This will require to leave seed-nodes in your HOCON configuration empty.
If you're initializing cluster from configuration (using HOCON config file), you can register a callback function using cluster.RegisterOnMemberUp(callback) to postpone the rest of processing until local actor system successfully joined the cluster.
The fastest (in terms of performance and resource usage) way is to subscribe to cluster membership events from within a particular actor. In fact this is how other solutions described above are actually implemented under the hood.
class MyActor : ReceiveActor
{
readonly Cluster cluster = Akka.Cluster.Cluster.Get(Context.System);
public MyActor()
{
Receive<ClusterEvent.MemberUp>(up =>
{
if (up.Member.Address == cluster.SelfAddress)
{
Become(Ready);
}
});
}
protected override void PreStart()
{
cluster.Subscribe(Self, new[]{ typeof(ClusterEvent.IMemberEvent) });
}
protected override void PostStop()
{
// rember to unsubscribe once actor is stopping
cluster.Subscribe(Self);
}
void Ready()
{
// other receiver handlers
}
}

JDBC Statement cancelled due to time out or client request

com.mysql.jdbc.exceptions.MySQLTimeoutException: Statement cancelled due to time out or client request
com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.java:1442)
Some code:
#Override
public synchronized ResultSet query(String s) throws SQLException {
try {
Statement statement = connection.createStatement();
statement.setQueryTimeout(3);
So I'm saving user data on SQL and this sometimes happens, it's very rare, but still. I would like to know why this could be happening.
Your client Open a connection to your Service or database.
By default, there is a timeout set, sometimes about 30 sec, in your apache server or Php... depends of what you use. Maybe mysql itself.
So, if your request is longer that 30 sec, the connection is stop, and you get this answer.
Hope this help you ;)
For more information, please give us your configuration : client , server type, apache or not, services...

EWS API Streaming subscription stops working

Created an windows service which saves all received and sent emails to my local drive and my service successfully does that.I have also resubscribed my streaming subscription onDisconnect event and Onerror event also.But my service stops responding after some time and there is no exception catched even though i have handled everything properly.Saw other forum and found the same issue people facing but there is not proper solution.
static private void OnDisconnect(object sender, SubscriptionErrorEventArgs args)
{
try
{
// Cast the sender as a StreamingSubscriptionConnection object.
StreamingSubscriptionConnection connection = (StreamingSubscriptionConnection)sender;
if (!connection.IsOpen)
connection.Open();
}
static void OnError(object sender, SubscriptionErrorEventArgs args)
{
// Cast the sender as a StreamingSubscriptionConnection object.
StreamingSubscriptionConnection connection = (StreamingSubscriptionConnection)sender;
if (!connection.IsOpen)
connection.Open();
}
Is this something to do with the Microsoft bug or it requires any settings on Exchange server for changing the limits for EWS subscription.
Even i checked below something related to throttling limit but no success:
http://msdn.microsoft.com/en-us/library/exchange/hh881884(v=exchg.140).aspx
Thanks a million in advance.
We have exactly same issue. And we do re-create whole subscription in OnError event just in case. It is also interesting that multiple application instances running on separate boxes exhibit identical behavior: at some point they just stop receiving notifications. Restarting any and all of them doesn't help; they do successfully subscribe but still no notifications other than OnDisconnect. Restarting Exchange Server is what really helps, though for a while.
I can see that the problem here is that you are trying to open the connection in the OnError handler. The problem here is that when OnError happen, the connection normally loses all the subscriptions, so you might need to consider creating the subscriptions again before opening them.

WebAPI and HTML5 SSE

was trying to encapsulate a partial view to show feedback that i can push back to the client.
This Article shows a method of pushing back data using HTML5 Server-Sent events (SSE).
I noticed that if i opened up several browser tabs and then closed one i got exceptions as the logic didn't remove the respective stream from the ConcurrentQueue. I amended the code as below
private static void TimerCallback(object state)
{
StreamWriter data;
Random randNum = new Random();
// foreach (var data in _streammessage)
for (int x = 0; x < _streammessage.Count; x++)
{
_streammessage.TryDequeue(out data);
data.WriteLine("data:" + randNum.Next(30, 100) + "\n");
try
{
data.Flush();
_streammessage.Enqueue(data);
}
catch (Exception ex)
{
// dont re-add the stream as an error ocurred presumable the client has lost connection
}
}
//To set timer with random interval
_timer.Value.Change(TimeSpan.FromMilliseconds(randNum.Next(1, 3) * 500), TimeSpan.FromMilliseconds(-1));
}
I also had to amend the OnStreamAvailable member as the framework syntax had changed to the second parameter being a HttpContent rather than HttpContentHeaders
public static void OnStreamAvailable(Stream stream, HttpContent headers, TransportContext context)
The problem now is i am still getting inconsistant behaviour if i add or remove clients i.e it times out when trying to initialise a new client. Does anyone have any ideas or more examples of using SSE with WinAPI and the correct "framework of methods" to handle disconnected clients
Cheers
Tim
This article is actually an adaptation of my original article from May - http://www.strathweb.com/2012/05/native-html5-push-notifications-with-asp-net-web-api-and-knockout-js/ (notice even variable names and port numbers are the same :-).
It is a very valid point that you are raising, and detecting a broken connection is something that's not very easy with this setup. The main reason is that while ASP.NET (the host) allows you to check a broken connection, there is no notification mechanism between ASP.NET (host) and Web API informing about that.
That is why in order to detect a broken connection (disconnected client) you should really try writing to the stream, and catch any error - this would mean the client has been disconnected.
I asked the same question to Brad Wilson/Marcin Dobosz/Damien Edwards at aspconf, and Damien suggested using HttpContext.Current.Response.IsClientConnected - so basically bypassing Web API and obtaining the connectivity info from the underlying host directly (however there is still a race condition involved anyway). That is really .NET 4. He also pointed an interesting way in which this problem could be avoided in .NET 4.5 using an async cancellation token. Frankly, I have never got around to test it, but perhaps this is something you should explore.
You can see their response to this problem in this video - http://channel9.msdn.com/Events/aspConf/aspConf/Ask-The-Experts - fast forward to 48:00

Is NServiceBus (AsA_Server) without DTC possible?

I am using NServiceBus for the first time and have a small, simple application where a user submits a form, the form fields are then sent to the queue, and the handler collects this data and writes it to the database using linq-to-sql.
Any changes within Component Services is a complete no-no as far as the DBA is concerned, so I'm now looking for an alternative to DTC (which is not enabled on the DB server), but using AsA_Server so that messages do not get purged.
I have tried removing AsA_Server after IConfigureThisEndpoint and specifying the configuration myself, but this doesn't seem to work (the console appears, page loads but nothing happens, it doesn't even stop at breakpoints.) AsA_Client does work, but as I understand it the messages will be purged at startup which I need to avoid.
Any suggestions?
Thanks,
OMK
EDIT: This has now been resolved by using wrapping the call to the database in a suppress transaction scope, which allows the database work to be done with no ambient transaction to enlist in:
using (TransactionScope sc = new TransactionScope(TransactionScopeOption.Suppress))
{
// code here
sc.Complete();
}
When you use AsA_Server, you are specifying you want durable queues and you will need to configure transactional queues.
With a transactional send/receive MSMQ requires you to send, transmit, receive, and process as part of one transaction. However, actually all these stages take place in their own transactions.
For example, the send transaction is complete when the sender sends a message onto their local MSMQ subsystem (even if the queue address is remote, the sender still sends to a local queue which acts as a kind of proxy to the remote queue).
The transmit transaction is complete when the MSMQ subsystem on the senders machine successfully transmits the message to the MSMQ subsystem on the receivers machine.
Even though this may all happen on one machine, I am guessing that your Handle() method is writing to a database on a different machine.
The problem here is that for the receive operation to complete satisfactorily from a transaction perspective, your call to the database must be successful. Only then will the message be de-queued from your input queue. This prevents any chance that the message is lost during processing failure.
However, in order to enforce that across the network you need to involve DTC to coordinate the distributed transaction to the database.
Bottom line, if you want durable queues in a distributed environment then you will need to use MSDTC.
Hope this helps.
There is an alternative. In your connection string you can add the option to not enlist in a distributed transaction and this will have your DB connection ignored in the DTC.
Of course, if this is set in the config then all database transactions for the application are ignored by the DTC rather than just a specific one.
Example:
<add key="DatabaseConnectionString" value="Data Source=SERVERNAME;Initial Catalog=DBNAME;Integrated Security=True;Enlist=False"/>
With NServiceBus 4.0 you can now do the following, which finally worked for me:
Configure.Transactions.Advanced(t =>
{
t.DisableDistributedTransactions();
t.DoNotWrapHandlersExecutionInATransactionScope();
});
When you use the As (AsA_Client, AsA_Server) interfaces, the configuration is applied after Init() so all the settings that you make there regarding MsmqTransport and UnicastBus are overriden.
It's possible to override those settings using IWantTheConfiguration in a IHandleProfile implementation. You get the Configuration after the default roles are applied but before the bus is started.
This way you can change the default profile settings and tailor them to your needs: deactivate transactions, enable impersonation...
Example:
public class DeactivateTransactions : IHandleProfile<Lite>, IWantTheEndpointConfig
{
private IConfigureThisEndpoint configure;
public IConfigureThisEndpoint Config
{
get { return configure; }
set
{
this.configure = value;
Configure.Instance.MsmqTransport()
.PurgeOnStartup(false)
.IsTransactional(false); // Or other changes
}
}
public void ProfileActivated()
{
}
}