We all know that logging is very important, and that there are a multitude of potential places to log to. (e.g. a file, a database, the event log, ...)
However, what do you do when the logging itself throws an exception? If we try to log to a file, but don't have permissions, or the file is locked, we can log to the event log. I don't know how it would happen, but I assume that there is some scenario that could cause logging to the event log to also fail. How would you handle exceptions that occur while logging to ensure that it is reported somewhere?
What methods of logging are least likely to fail?
What methods of logging are most likely to fail?
My current scheme is for all logging to go to a file, with a little bit also going to the event log. If logging to the file fails, I would log that to the event log. I don't really have a good plan for the event log failing.
I'd say you're going too deep. Logging frameworks should take care of themselves. That means: no exceptions, unless something really horrid is going on behind the scenes.
File locking exceptions should never appear. If they do, your logging framework has a flaw. That means you're using a wrong framework, since that would be a very fundamental flaw.
Secondly, file permissions. It is YOU who decides where logged files will appear. If you don't take into account file and directory permissions, it is your fault. You must make sure your logger can log where you tell it to log.
Bottom line: log to files. This is the most convenient way. Also, the fastest way. Logging to a database can always fail. If the framework is robust enough, it won't throw any exceptions. You must ensure that file permissions are set up properly. And that's all there is to it.
Related
I'm finishing a system at work that makes calls to mysql server. Those calls' arguments reveal information that I need to keep private, like vote(idUser, idCandidate). There's no information in the db that relates those two of course, nor in "the visible part" of the back end, but even though I think this can't be done, I wanted to make sure that it is impossible to trace this sort of calls, with a log or something (calls that were made, or calls being made at the moment), as it is impossible in most languages, unless you specifically "debug" in a certain way, while the system is in production and being used. I hope the questions is clear enough. Thanks.
How do I log thee? Let me count the ways.
MySQL query log. I can enable this per-session and send everything to a log file.
I can set up a slave server and have insertions sent to me by the master. This is a significant intervention and would leave a wide trace.
On the server, unbeknownst to either Web app and MySQL log, I can intercept communications between the two. I need administrative access to the machine, of course.
On the server, again with administrative access, I can both log the query calls and inject a logging instrumentation into the SQL interface (the legitimate one is the MySQL Audit Plugin, but there are several alternatives, developed for various purposes by developers over the years)
What can you do? You can have the applications use a secure protocol, just for starters.
Then, you need to secure your machine so that administrator tricks do not work, and even if the logs are activated, nobody can read them and you can be advised of any new and modified file to delete it promptly.
I have a Yii application with many concurrent console jobs writing to one database. Due to the high concurrency sometimes I get MySQL deadlock errors. Sometimes these can be too many. The console.log file becomes too big, and it translates to more expenses.
I want to prevent logging of specific CDbException instances, or at least suppress them altogether (I am handling the exceptions and can generate more compact log sentences from there).
YII__DEBUG is already commented out.
Can anyone please help me figure out how to do this?
Thanks a lot!!
Regards.
I decided to modify the log statement in yii/framwework/db/CDbCommand.php that was logging the failed SQL. I converted it into a trace statement:
Yii::trace(Yii::t('yii','CDbCommand::{method}() failed: {error}. The SQL statement executed was: {sql}.', array('{method}'=>$method, '{error}'=>$message, '{sql}'=>$this->getText().$par)),CLogger::LEVEL_ERROR,'system.db.CDbCommand');
I am anyway catching the exception and logging a more compact version of the sentence, so it is OK for me to do it.
This was the easiest way I could find. We don't upgrade Yii very often, so if and when we go to the next version I'll probably repeat the change.
When receiving so called IPN message from PayPal, I need to update a row in my database.
The issue is that I need perfect reliability.
Currently I use InnoDB. I am afraid that the transaction may fail due a race condition.
Should I use LOCK TABLES? Any other reliable solution?
Should I check for a failure and repeat the transaction several (how many?) times?
You cannot reliably make a distributed process (like adding a row locally and notifying the server remotely) perfectly reliable, no matter the order. This is a lot like the Two General's Problem: there is no single event which can denote the successful completion of the transaction on both sides simultaneously, as any message might get lost along the way.
I'm not sure I understand your issue correctly, but perhaps the following would work: Write a line to some table noting the fact that you are going to verify a given message. Then do the verification, and afterwards write a line to the database about the result of that verification. In the unlikely but important scenario that something broke in between, you will have an intent line with no matching result line. You can then detect such situations and recover from them manually.
On your local database, you'd have single row updates, which you may execute in their own transaction, probably even with autocommit turned on. You have to make sure that the first write is actually committed to disk (and preferrably a binary log on some other disk as well) before you start talking to the PayPal server, but I see no need for locking or similar. You migt want to retry failed transactions, I'd say up to three times, but the important thing is that in the end you can have admin intervention to fix anything your code can't handle.
In any application, we can do error logging using flat file system.
How do we handle a scenario when there are multiple users having exceptions which are logged in the same flat file?
Many Thanks.
You'd acquire a write lock to the logfile before logging an exception. If someone else tries to log while you're in the middle of writing, their lock request will wait until you release it.
Alternatively, if such a thing is available, use an atomic file write operation to log the entire exception.
You could create one thread for writing exceptions, reading from a queue. The actual exception handling code would write the exception to the queue. Since there's only one thread writing the file, everything is serialized. Of course, you need a thread-safe queue implementation, but your language or framework probably provides one.
I'm writing some client/server software and I'm facing the following design issue. Normally, I use a VERIFY macro very liberally - if something is wrong in an user's machine, I want the software to fail and log the error so it can be fixed. I was never a fan of ignoring any kind of errors.
However, I'm now writing a server. If the server dies, many clients go down, so the server should die as little as possible. Therefore, I don't know how to treat some conditions that I'd treat as fatal exceptions otherwise.
For example, I get a network packet from an user who isn't logged in. Even though it shouldn't happen, I have enough experience to know "impossible" errors do happen from time to time. So I'm pretty sure if I do a fatal error on these cases, the server WILL crash eventually. On the other hand, I could log and ignore the error and continue, but I'm afraid some bugs may go undetected this way.
What would you do in a situation like this one?
If you can recover from the error, than obviously it wasn't fatal. I can't see the benefit of failing if you can log the error and continue execution - the most important thing is that you've captured the error on log. If you can recover and continue to operate as normal, than that is the best course.
You should implement in addition a notification system (server monitoring) that depending on the error level would notify you in varying degrees of urgency so you'd pick up as soon as possible on something time critical. There are generic system like that for servers, such as Nagios and Munin. You should have look at what they do and see if you can take something from them and implement / integrate it into your system.
Regardless, you should try to make sure client instances are as sandboxed as possible. A client thread going down shouldn't take down the entire server - ever (at least in theory).