I'm having a 'Tournament' sql table that contains start_time and end_time for my tournaments. I also have another table which has playerId and tournamentIds so I can tell which players playes in which tournament.
What I'm trying to do is to run a cron task to check my tournament table and see if tournament has ended so it can check players results from an external api. The problem is the external API has rate limit and I have to send my requestes every 1.5 sec.
What I tried to do is to write a cron job for every 10 seconds to check my tournament table (I couldn't come up with anyother solution rather than keep checking db):
cron.job("*/10 * * * * *", function(){
result = Query tournament table Where EndTime=<Now && EndTime+10second>=Now
if(result is not empty)
{
cron.job("*/1.5 * * * * *",function(){
send API requests for that userId
parse & store result in db
});
}
});
I don't feel right about this and it seems so buggy to me. Because the inner cron job might take longer than 10 seconds. Is there any good solution to do this. I'm using ExpressJS & MySQL.
The problem you are facing can be solved with event emitters. There is a very useful module node-schedule in npm which can help you in this scenario that you are telling. What you have to do is is to schedule a job to fire at the deadline of the project, that job will hit the 3rd party api and check for results.You an schedule a job like this
var schedule = require('node-schedule');
schedule.scheduleJob("jobName", "TimeToFire", function () {
//Put your api hit here.
//finally remove the schedule
if ("jobName" in schedule.scheduledJobs) {
schedule.scheduledJobs["jobName"].cancel();
delete schedule.scheduledJobs["jobName"];
}
})
Make sure you store all the jobs scheduled in the database also as a server crash will invalidate all the schedules that you have scheduled and will have to reschedule them again.
Related
I'm new at this guys(Laravel) and I need to store my data collected from my DB each six hours. I thought of using laravel's task scheduling, but in the kernel where I'm supposed to create my cron job, I don't think that I can return my data as an array
I use 6 arrays each one of them in a different page.
Here is an exemple of one of them :
$dataGlobal = DB::select("select * from articles order by articleDate ASC");
PS: the code bellow is in my controller and I return the array with its view
Kernel.php
protected function schedule(Schedule $schedule)
{
// $schedule->command('inspire')->hourly();
$schedule->call(function(){
DB::select("select * from articles order by articleDate ASC");
})->cron('* */6 * * *');
}
Any ideas guys ? How can I store my data in an array where m going to use it in my pages (It's already working when I stop manually the project and relaunch it with php artisan serve)
You can't return data from cron job.
I think what you need is:
Create cron job that collect data each 6 hours and store it on table to your database. so you could use that data from the table you have created.
Since I solved my issue, I'm sharing it here if someone faced the same thing.
Laravel is a PHP's Framework, and php is a scripting language. Each time a person naviguate through my site each row is executed at that time.
By doing so, my array will always contain latest rows of my database. I don't really need a cron-job for this.
I have problems is: Auto-update order status if that status doesn't have any update in 30 days.
For example, the current status is " processing" and during 30days later, no more update on this. So the MySQL auto-update order status to "on Hold
I found and guest it something related to Hook, but I don't know how to implement it
depends on how you run you app. I'll suggest using e.g. some cron to get such instances and update it's status.(e.g. https://www.npmjs.com/package/node-cron)
Set it for example to run once a day, query such instances depends on build in column updatedAt. Or just to add some other column which will be updated with current date, depends what rules you want to apply
you can use cronjob.as npm says Cron is a tool that allows you to execute something on a schedule.
for install cron - 'npm i cron'
the (*) shows in order given details.
Seconds: 0-59 Minutes: 0-59 Hours: 0-23 Day of Month: 1-31 Months:
0-11 (Jan-Dec) Day of Week: 0-6 (Sun-Sat)
how to initialize cronjob:
var CronJob = require('cron').CronJob;
var job = new CronJob(
'* * * */30 * *',
function() {
console.log('every 30 days it is auto updated!');
},
null,
true,
'America/Los_Angeles'
);
make sure other parameters pass according to your application
and also cron use different port then your application.
As suggested by Damian you need to use a kind of CRON job which runs everyday to check for expired orders(created more than 30 days ago without any update).
You can use Bull with PM2
Create a processor in Bull and a queue and use the CRON functionality of Bull to set this job to run everyday usually at 12:05 AM or any time you see suitable.
The job would fetch all orders which have status "processing" and were created more than 30 days ago and update them.
Use PM2 to run Bull
I need to perform some actions in my app based on user's input. For e.g. if user selects 17:00-18:00, the timeframe is updated in the MySQL db and something should happen in that specific time.
Could someone tell me how can I achieve that?
You can use cron jobs for it.
For example, if you need to do some actions based on time value in your DB, you can write some code, that gets the current date and selects from MySQL all raws if your current date is between the date range of that raw and process them.
Then just make some cronjob that calls your code every minute, or an hour or whatever you need.
You can use cron package for that.
Here is an example:
const { CronJob } = require('cron');
// example cron job
const yourFunction = require('./yourFile.js');
// setting cron for example cronJob
const cron = new CronJob('0 * * * * *', yourFunction, null, true);
cron.start();
Or you can use Linux's crontab, you can read about how to use it here.
Configure frequency of cron with cron scheduler, you can play with it here.
I have a model in Django that holds some data which is irrelevant after a month.
Is there a way to automatically delete it after a certain period of time?
The DB is MySQL if it matters, thing is I can't tell whether this is done in the DB side (perhaps there's a way to configure this via MySQL?), or in my back-end code.
Is there a quick solution, or do I have to write code that does this, and have it run every day, deleting anything that wasn't added a month ago?
Thanks
I'd suggest creating a management command that queries for all the records in your model that are older than one month and delete those records. Then throw that management command into a daily cronjob. This should suit your needs.
you can solve this issue depends on your case,
if this data become with no value and you want to delete it
you can do that by
1- from database & using crontab
DELETE FROM mytable
WHERE date_field < UNIX_TIMESTAMP(DATE_SUB(NOW(), INTERVAL 30 DAY));
2- using managment command with crontab
import datetime
samples = Sample.objects.filter(sampledate__gt=datetime.date(2011, 1,
1), sampledate__lt=datetime.date(2011, 1, 31))
3- using celery with periodic task
http://celery.readthedocs.org/en/latest/userguide/periodic-tasks.html
You can always let the manager filter for you:
class RecentManager(models.Manager):
def get_queryset(self):
return super(
RecentManager,
self
).get_queryset().filter(
your_timestamp__gt=datetime.datetime.now()-datetime.timedelta(30)
)
class YourModel(models.Model):
#your fields, including your_timestamp
objects = RecentManager()
unrestricted = models.Manager()
#static
def delete_old():
YourModel.unrestricted.filter(
your_timestamp__lt=datetime.datetime.now()-datetime.timedelta(30)
).delete()
Hook up the delete to a management command which you can run in a cronjob or Celery task or whichever other infrastructure you have handy for async execution.
I'm on a shared hosting platform and would like to throttle the queries in my app so that if the total execution time exceeds a certain amount over a variable time period then I can make the app cool off and then resume later on.
To do this I would like to find out how long each of my queries take in real time and manage it within the app and not by profiling it externally.
I've seen examples in PHP where the time is recorded before and after the query (even phpMyAdmin does this), but this won't work in NodeJS or anything that runs the query asynchronously.
So the question is: how would I go about getting the actual execution time of a query in NodeJS?
For reference I am using this module to query the MySQL db: https://github.com/felixge/node-mysql/
One option is just to timestamp before the query and timestamp after the query and check the difference like this:
// get a timestamp before running the query
var pre_query = new Date().getTime();
// run the job
connection.query(query, function(err, rows, fields) {
// get a timestamp after running the query
var post_query = new Date().getTime();
// calculate the duration in seconds
var duration = (post_query - pre_query) / 1000;
});
If you are using rest api then you can use postman. Run the query in postman and then look for Time: on middle right side of section as show in image attached.
console.time('100-elements');
for (let i = 0; i < 100; i++) {}
console.timeEnd('100-elements');
// prints 100-elements: 225.438ms
Label is must be unique and same as start and End time
Working well in nodejs.
here is documentation on nodejs