I have a LabView data acquisition system that is writing data to a MySQL Database. It is writing data every second. The LabView system recently froze around the time when I was playing with the SQL queries.
I have a client computer, which is supposed to send queries to that MySQL Database every hour. This client computer set up a cron job to send the command to query the database every hour.
I recently added an index to my time_stamp, in order to optimize my query.
This may be a shot in the dark, but could there be any deleterious interaction between the fact that I had created an index on our time_stamp (to optimize the query), and set up a cron job to send the query every hour? Around that time, I think I may have also sent a query and aborted quickly before it completed, so I was wondering if something like that may cause the LabView system to freeze?
It doesn't appear to be an issue on the MySQL side, because the server was still running.
Check if you currently have deadlocks at your MySQL server, the query from your LabVIEW application might be suspended and is waiting for completion forever. Hint, look to the query execution time. Why it is happened should be investigated separately, but there is a good chance that if you just kill that suspended query the system will unfreeze and keeps running normally.
Related
I'm atm working to create a huge mySQL database by parsing XML files released on a FTP.
On a single computer, it takes ages, because of the huge amount of SQL INSERT INTO to make.
Thus, I modified my code to build it on AWS by creating a cluster, launching a database, build everything and download back the dump.
However, I got a question. Is there a "queue" for SQL requests sent ? I mean, if every of my nodes are sending requests at the same time to the database, what's going to happen ?
Thanks
On MySQL you can use SHOW FULL PROCESSLIST to see the open connections and what query they are running at the moment.
There is no queue of requests but some requests waits for others to complete, before starting, because they attempt to use rows or tables that are locked by the requests that are currently running.
Only one request is executed at a time for each connection.
I have a weird issue with one of the data driven subscriptions on SSRS.
The subscription is a timed subscription that generates invoices (pdf/excel) and gets triggered by a stored procedure.
The issue we are facing is that the first run always takes 30-60 minutes regardless of how many invoices are being generated. Once the first run has completed the subsequent runs are completed under a minute throughout the day.
There is a second version of the same report that is run manually and it runs fine(ruling out any delays with the data extraction bit).
I have looked at some other questions here but that didnt help identify the problem:
SQL Reporting services: First call is very slow
SSRS report subscription not working sometime
Without knowing more about the query, data, database setup, other process, etc.; it will be quite difficult to say for sure. But if I had to guess, based on your description, it sounds like the query plan cache is lost and is rebuilt on the first run of the day. Without the plan the query can be less efficient. Each subsequent run will use the plan created on the first run, and will therefor run more quickly. There are a number of reasons that could cause the query plan to be wiped from cache. A recompile, other queries using too much memory, not enough system memory to begin with etc.
Hope that helps!
I want to be able to track the data changes in the DB of my app so I'm thinking about storing, in a dedicated DB table, all INSERT, UPDATE and DELETE queries that my app executes.
Would that be a bad idea?
We do that in debug mode - we output all the queries to a file. Of course, this does not make sense (and is a huge performance hit) on a production server, but we can turn it on there too, for short period of debugging.
Any way - mysql has query log you can turn on, it will record every single thing it does.
is there any software that does "lazy" deletion of the rows from the table. I would like to do maintenance of my tables when my server is idle, and ideally i should be able to define what "idle" is (num of database connections/system load/ requests per second). Is there anything remotely similar to this?
If you are on a linux server, you can make your table cleanup scripts only run based on the output of the command "w" which will show you a system load. If your system load is under say .25 you can run your script. Do this with shell scripting.
To some degree, from an internal perspective InnoDB already does this. Rows are initially marked as deleted, but only made free as part of a background operation.
My advice: You can get in to needlessly complicated problems if you try and first check if the server is idle. i.e.
What if it was idle, but the cleanup takes 2 minutes. During that 2 minutes the server load peaks?
What if the server never becomes idle enough? Now you just have an unlimited backlog.
If you just background the task you might improve performance enough, since now at least no users will be sitting in front of web pages waiting for it to complete. Look at activity graphs as to what is the best time to schedule it (3am, 5am etc).
I'm using R to read some data from a MySQL database using the RODBC package. The data is then processed and some results are sent back to the database. The problem is that the server closes the connection after about a minute due to inactivity, which is the time needed to process the data locally. It's a shared server, so the host won't bump up the timeout time.
I think there are two possibilities to get around this:
Open a connection before every database transaction and close it immediately after
Send some small 'ping' command to the server every 30 seconds or so to let the server know that I'm still there.
I can implement the first fairly easily, but it seems pretty slow to constantly open and close connections. Does anyone know an efficient command for the second? Or is a better way altogether?
The first solution is the one I prefer. It's really hard to do the latter with a single threaded program like R. If R is busy running analysis there's no way for it to handle the ping. Unless you are doing hundreds of reads/writes the method of opening and closing the connection should not introduce an extreme amount of overhead.