I want to create a monitoring script that will check my servers. Right now I'm stuck on a problem, I need to find out a way to get uptime percentage. Basically all data is stored in MySQL server, for me the easiest way to get uptime is to create a function that will add a new record to mysql server every minute with date, time, information is it online etc. but if I will use this method and I will have for example 1000 servers to monitor, I will end up with 518 400 000 records in MySQL server per year.
Another idea was to create one record per server with two rows online and offline, but without any date and time I'm not able to get uptime...
Any ideas how to design database for monitoring system ?
The MySQL information_schema contains uptime information (expressed in seconds) for each server. I am not sure how accurate your figure has to be, but you could get this value at a set interval and compare it to the previous value. Depending on the interval, this could give you a good approximation.
SELECT Variable_Value FROM SESSION_STATUS S WHERE Variable_Name = 'UPTIME';
Also, the MySQL error log contains a date and time stamp when the server starts. Scrape this info periodically and add to your server table.
Related
I am using MySQL workbench 6.3 CE. I want to take the snapshots of MySQL status variables.
I want to store the values of status variables after every 1 second during the execution of query.
I can simply show the variables using 'show global status'. But I want to execute it automatically after every 1 second.
You can run a procedure and a query at the same time by having two separate connections. Workbench is a handy tool, but you should learn to use the mysql commandline tool, too.
The query is rather simple. INDEX(l_shipdate) is likely to be the best for it.
The real way to speed up the query (assuming that is your ultimate goal) is to build and maintain a "Summary table" of daily or monthly subtotals. Then sum the sums and sum the counts. Avg is (SUM(sums)/SUM(counts)).
More discussion: http://mysql.rjweb.org/doc.php/summarytables
Be cautious about running (via EVENT or cron) any code that might take longer than the interval time. If it gets behind, it is likely to cascade and bring the server down, or at least slow things down severely. For that reason, I much prefer the WHILE loop.
I have a database which has a timestamp column and I want outdated data to be deleted.
So my idea is to write a MySQL query to a .php file which deletes every row where timestamp < current_timestamp - const. As there will be a lot of rows where this has to be checked, I am going to set an index to the timestamp column.
So how can I run this script automatically every n seconds? I heard about Linux crontab - can I use this on my webserver (not the db server) to execute the .php file periodically and is this overall a good technique to delete outdated rows from a database?
The database is set on a RDS instance on Amazon Web Services. My webserver is a EC2 instance (also Amazon Web Services).
Doing such a thing requires setting up an event or job. Such efforts keep the database very busy.
I would strongly recommend a different approach. Use a view to access the data you want:
create view v_t as
select t.*
from t
where timestamp > CURRENT_TIMESTAMP - ??;
Then use this view to access the data.
Then, periodically, go in an clean the table to get rid of the rows that you don't don't want. You can do this once a day, once a week, once an hour -- the deletions can occur at times when the database load is lighter, so it doesn't affect users.
I think you should check out lambda service on AWS.
It allows you to run commands against AWS services without another instance running.
Here's an example on how to set it up.
http://docs.aws.amazon.com/lambda/latest/dg/vpc-rds-deployment-pkg.html
Good luck
Eugene
Gordon Linoff's approach is ideal, but if you want to go the route of scheduled jobs, MySQL Event Scheduler is something you can try. The following example, runs daily and delete records older than a week.
CREATE EVENT
clean_my_table
ON SCHEDULE EVERY 1 DAY
DO
DELETE FROM my_table
WHERE time_stamp < date_sub(now(), INTERVAL 1 WEEK);
MySQL Event Reference page
https://dev.mysql.com/doc/refman/5.7/en/create-event.html
As per my research I thought of using the mysqldumpslow utility to parse the log and extract the results, but not able to figure out how to use it. I want to get the count of number of queries logged in the slow query log for an interval of 10 minutes, so that the values can be compared for analysis.
Thanks
You could use logrotate to create a new slow.log every 10 minutes and analyze them one after another. Implying you are using Linux. Be aware that your example shows that your mysql instance is configured to "log-queries-not-using-indexes" hence you will also get those SELECT's that dont use an index in your log file too.
Update :
Since i still dont know what OS you are using, a more general aproach to your problem would be redirecting the slow log into mysql itself following the mysql docs and get all records from the slow log table like :
SELECT COUNT(*) FROM slow_log;
Which gives you the total amount of Querys logged. Follwed by a :
TRUNCATE TABLE slow_log;
Having a script in place doing this every 10 minutes would output the desired information.
I am developing a mobile application. From the application calls are made to a web service which runs different queries based on mode (?mode=xx)
In some of those queries I use date functions like DATE(NOW()).
The data stored in the MySQL database is stored in GMT-7 (Mountain Time Canada).
I have yet to register a domain/host for this web service but when I do lets say it is hosted in a different city such as Toronto (which is GMT-5 - 2 hours ahead). Then at 10:05pm Mountain Time Canada a user uses the application to send a web request call which has a query like:
SELECT DATE(NOW())
Because the server is hosted in Toronto, that will return tomorrow's date, even though where the user is it is the day before and the application shows data based on the current day.
Anyone have any ideas on this?
Edit:
SYSTEM
2015-01-29 16:19:48
2015-01-29 23:19:48
is the result of running the query select ##time_zone, now(), utc_timestamp()
The queries deal with date (yyyy-mm-dd) and time (hh:mm:ss) column type.
You ran this time-diagnostic query on your MySQL server.
select ##time_zone, now(), utc_timestamp()
It's clear from your local time and utc time that your server machine's system time zone setting is 'Canada/Mountain', and the MySQL server software doesn't have its own timezone setting.
If you pick up your tables and move them unchanged to a server in some nearby timezone, you can update your software always to issue the command
set time_zone = 'Canada/Mountain';
right after you connect from your software. This will make your new MySQL connection behave like your current one does time-zone-wise. If you own the MySQL server you can set its default time zone according to the directions on this page. http://dev.mysql.com/doc/refman/5.5/en/time-zone-support.html
Now, here's the story about time data types. DATE, TIME, and DATETIME are all timezone-ignorant. Once you've stored a date/time value you'll get it back the same value even if you change your timezone settings.
The TIMESTAMP data type is timezone-sensitive. Those data items are always stored in UTC, also known as Z, time, formerly known as Greenwich Mean Time. They're always converted to UTC upon being stored, and always converted back upon being retrieved.
The builtin functions for getting current date and time (NOW() and friends) are timezone-sensitive. They'll yield values in local time. The exceptions are the three functions starting with UTC_ which yield values in UTC time.
Many MySQL multi-time-zone applications use the following operational discipline:
Ask each user for a user-preference time zone, or figure it out from some other bit of personal data about the user. (Telephones have this information provisioned into them from the network.) Store that as a zoneinfo-friendly time zone descriptor ('America/New_York', 'Canada/Mountain', 'Europe/Vienna', etc) on the user's behalf.
Upon establishing a MySQL session on behalf of the user, set the user's time zone with a set time_zone query like the one shown above. You should do this right after your connect operation.
Store dates and times for users into TIMESTAMP data types. They'll get converted to UTC as they're stored.
Retrieve them as needed. They'll get converted back to local time.
The idea is that your user's timezone is part of her context. This works well, because if user A is in Vancouver and user B in Halifax, and for some reason user B views user A's time data, it will be shown to B in Atlantic time more-or-less automatically.
It's also good because it deals transparently with the global vagaries of daylight-to-standard time changing. A timestamp from last summer will be displayed in last summer's local time.
Many managers of servers for global use set their system server time, or their MySQL default time zone, to UTC. (Yours doesn't.)
Another way to handle all this is the way in which you've started. Pick a time zone and store your timestamps with respect to that time zone. It's best if you pick a timezone that doesn't alternate between daylight and standard time in that case. Then, when storing times into the database, convert explicity. You'd store times from users in Ottawa by doing something like this.
INSERT INTO tbl (appt) VALUES ( 'whatever-time' - INTERVAL 120 MINUTE)
and you'd get the values out the same way. This is error-prone but you can make it work.
Finally, you can do your conversions yourself.
If you want to know how many minutes of offset there are between some arbitary timezone and UTC, try these two queries.
set time_zone = 'Canada/Atlantic';
select timestampdiff(minute, utc_timestamp(), now());
At this time of year that gives back -240, which is -4:00. You need to use minutes rather than hours because of half-hour or quarter-hour timezone offsets in some countries.
Finally, watch out. TIMESTAMP data types don't represent times before 1970. And, on my MariaDB 10.0 instance it appears to go to hell in a bucket right after 2038-01-19T03:14:07 UTC when the time rolls over out of 32 bits.
I have a resource table for a game I'm trying to code, and each resource has a fixed income rate over time. But I can't find any description on how to increase the stored values of the MySQL Table over time automatically.
I'm using NetBeans to connect the program with the database, but I want the values to be updated on the server without the need to run the program. Otherwise I would just have had the time recorded and just add the time difference value.
Is there a way of doing this?
Table:
Player ID: 1
Gold: 100
Wood: 100
Increase rate: 50 per hour
One way of doing this is using Cron jobs and schedule some script to run.
Otherwise you can simply calculate the time elapsed from the beginning and (whithout updating your DB) calculate values based on the time when your program is running.
You can define a cron job on the server, that runs a query to update the values.
Yes, you can by adding a scheduled event like this. However, if you update the value in the database, the value/variable stored by the program will not be updated in real-time: you have to query the database for the updated value.