I have a lobby page that goes to the MySql database every second and checks each timestamp variable(belongs to the users) from a table and if the timestamp is older than (NOW() - 3) seconds, it sets the 'connection'(bool) variable to false. Basically it checks all currently connected users.
I haven't tested on a real server yet, but I have a feeling that it's going to be really intensive process. Because every user has access to the lobby area and each user will send a request to the MySql database and update the table. That means if I have 1000 users in the lobby area that means 1000 requests per second.
My question is, is there any other way to do the same thing without sending so many requests? I looked into Cron jobs, but cron doesn't let you to run a specific script every 1 second. I think the minimum is 1 minute.
I think this will help you run your script twice every 1 minute:
function for_cron() {
//database update code
}
function check_up() {
//assuming you dont have anything to echo
//call the function
foo_cron();
sleep(30); //sleep 30 seconds
for_cron();
}
Then setup your "check_up" function to run on cron for every 1 minute
Hopt it helps
Related
I have to run a command every day at a different time. The times are known in advance and saved in a MySQL database in the familiar YYYY-MM-DD HH:MM:SS format.
What I thought of:
cron schedule the job for the exact time the first day, then make the script itself modify the crontab entry with the correct time for the next day.
cron scheduling the job at approximately the right time, then make it read the exact time from the database and sleep until then.
cron schedule the job execution every minute, and leave it to the script to determine whether the current date/time corresponds to the right execution time; proceded if it is, exit if not.
at submit the job the first day with at, then make it read the next day's time from the database and resubmit itself for then with at.
Additional info:
The command is a PHP script that composes the message of the day and sends it to all users registered to the website. I can consider other technologies if they solve this problem better. I would like to retain the ability of rebooting the server (outside of the intended execution hour) without worrying too much about jobs getting lost, therefore solutions 1. and 3. look better under this aspect. I'm starting with two commands to be run at two different times of the day, but I could soon end up with dozens more of similar jobs to be scheduled at different times every day, so I would prefer to avoid clutter as much as possible. I'd probably go with option 3 at this point.
The question(s):
Is there a better / preferred / established way of accomplishing this task? Solutions other that those mentioned above are welcome. What are the main drawbacks (of your recommended solution) I should be aware of?
I do believe you need to build your custom application for implementing the logic you want to implement.
Eventually you can use the cron system to start the process or to make sure that the process is running (in case it died or it was killed).
In your place what I would do, is to write a custom PHP program (or python or you name it) that performs the following:
Opens a connection to DB
Checks when the next execution is scheduled
Calculates if it is time to run
if not, it sleeps for X seconds (this depends on your preference)
it it is time to run, it performs its duty
sleeps again, and the loop begins
An alternative would be to check the every time the execution schedule, to check for changes in the schedule.
Another one would be read once and sleep until the execution time, but in this other case you would not catch changes in the schedule
This all depends on you, all in the all the program is an extremely easy one
I ended up using solution 3. above and am quite satisfied with it so far.
All the logic is in the .php file, which is responsible to:
save the current date/time in a variable (e.g. $now)
perform any considerations on it
scan the database in search of a matching date/time
This actually allows for a reasonable degree of flexibility:
I can choose not to run any commands if a certain semaphore file exists:
if (file_exists($filename)) {exit;}
I can set parameters in an option file enabling e.g. debug or test modes:
include parameters.php
if ($debug === true) {error_reporting(E_ALL);}
I can avoid bothering users if it is, let's say, new year's day:
if (date('m-d') == '01-01') {exit;}
I can introduce delays based on custom logic:
if (date('w', strtotime($now)) === '0') {$now = date('Y-m-d H:i:s', strtotime($now . ' +15 minutes'));}
I have a database which has a timestamp column and I want outdated data to be deleted.
So my idea is to write a MySQL query to a .php file which deletes every row where timestamp < current_timestamp - const. As there will be a lot of rows where this has to be checked, I am going to set an index to the timestamp column.
So how can I run this script automatically every n seconds? I heard about Linux crontab - can I use this on my webserver (not the db server) to execute the .php file periodically and is this overall a good technique to delete outdated rows from a database?
The database is set on a RDS instance on Amazon Web Services. My webserver is a EC2 instance (also Amazon Web Services).
Doing such a thing requires setting up an event or job. Such efforts keep the database very busy.
I would strongly recommend a different approach. Use a view to access the data you want:
create view v_t as
select t.*
from t
where timestamp > CURRENT_TIMESTAMP - ??;
Then use this view to access the data.
Then, periodically, go in an clean the table to get rid of the rows that you don't don't want. You can do this once a day, once a week, once an hour -- the deletions can occur at times when the database load is lighter, so it doesn't affect users.
I think you should check out lambda service on AWS.
It allows you to run commands against AWS services without another instance running.
Here's an example on how to set it up.
http://docs.aws.amazon.com/lambda/latest/dg/vpc-rds-deployment-pkg.html
Good luck
Eugene
Gordon Linoff's approach is ideal, but if you want to go the route of scheduled jobs, MySQL Event Scheduler is something you can try. The following example, runs daily and delete records older than a week.
CREATE EVENT
clean_my_table
ON SCHEDULE EVERY 1 DAY
DO
DELETE FROM my_table
WHERE time_stamp < date_sub(now(), INTERVAL 1 WEEK);
MySQL Event Reference page
https://dev.mysql.com/doc/refman/5.7/en/create-event.html
We've got some data driven subscriptions running on SSRS.
Sometimes they take an unusually long time to complete, if I check the activity on the server I find that things are relatively quite.
What I did notice is that in the ReportServer database on the Notification table there's a column called ProcessAfter.
Sometimes this value is set about 15 minutes into the future, and the subscription only completes after the time stated in that column.
What is setting this value? Since this behaviour is relatively rare.
After a few days I posted this question here, and got an answer:
When a subscription runs, there are several things happen: The SQL
Server Agent job fires and puts a row in the Event table in the RS
catalog with the settings necessary to process the subscription. The
RS server service has a limited number of threads (2 per CPU) that
poll the Event table every few seconds looking for subscriptions to
process. When it finds an event, it puts a row in the Notifications
table and starts processing the subscription.
The only reason that rows would stay in the Notification table is that
the RS service event processing threads are not processing the events.
As per my understanding, the NotificationEntered column stores the
time when the notification enters. Delivery extension provide some
settings for specifies the number of times a report server will retry
a delivery if the first attempt does not succeed (MaxRetries property)
and specifies the interval of time (in seconds) between each retry
attempt (SecondsBeforeRetry property). The default value for
SecondsBeforeRetry is 900 seconds, means 15 minutes. When the delivery
fails, it retry attempts every 15 minutes.
Reference: Monitoring and Troubleshooting Subscriptions Delivery
Extension(s) General configuration
If there are any other questions, please feel free to let me know.
Thanks, Katherine Xiong
I found the Extension(s) General Configuration link especially helpful
This question already has an answer here:
Delete MySQL Row after 30 minutes using Cron Jobs/Event Scheduler
(1 answer)
Closed 9 years ago.
I want to have a script, that counts how many users are online on my site, but this script should count guests, so I have created a database for session, this script gives user an ID and set 30 minutes session, and now I have a problem, because if he is not active more than 30 minutes, he should be deleted from the database, because I want to count by ID how many users are online, and I have headache how can I do this.
Is there a simple way to do this?
As stated by Barmar in their answer here:
DELETE FROM my_table
WHERE timestamp < NOW() - INTERVAL 30 MINUTE
Write a PHP script that executes this SQL, and add a crontab entry
that runs it every 30 minutes. Or use the MySQL Event Scheduler to run
it periodically; it is described here.
Since you have not mentioned any database in specific, I give you a general idea of how to implement it:
You can have a table like this:
USER_SESSIONS
(
USER_ID //UNIQUE_ID
LAST_ACTIVE //TIMESTAMP
)
Here is the functionality you can associate to implement the session:
When a user logs in, you create an entry in this table.
When a user logs out, you delete the corresponding entry from this table.
When user does some activity (depends on how you want to track activity in the front end), update the corresponding TIMESTAMP of the user.
Create a DB Schedule (a continuously running process) that monitors the TIMESTAMP column. All good databases have a built in scheduler. Here is some pseudo code for the scheduler:
FOR each entry in USER_SESSIONS
If (CURRENT_TIMESTAMP - LAST_ACTIVE) > [30 mins] then
delete the entry of the corresponding user from this table.
//This will essentially cause a session timeout.
End If
LOOP;
I hope you got a fair idea of how to implement it.
I have a resource table for a game I'm trying to code, and each resource has a fixed income rate over time. But I can't find any description on how to increase the stored values of the MySQL Table over time automatically.
I'm using NetBeans to connect the program with the database, but I want the values to be updated on the server without the need to run the program. Otherwise I would just have had the time recorded and just add the time difference value.
Is there a way of doing this?
Table:
Player ID: 1
Gold: 100
Wood: 100
Increase rate: 50 per hour
One way of doing this is using Cron jobs and schedule some script to run.
Otherwise you can simply calculate the time elapsed from the beginning and (whithout updating your DB) calculate values based on the time when your program is running.
You can define a cron job on the server, that runs a query to update the values.
Yes, you can by adding a scheduled event like this. However, if you update the value in the database, the value/variable stored by the program will not be updated in real-time: you have to query the database for the updated value.