I have a model in Django that holds some data which is irrelevant after a month.
Is there a way to automatically delete it after a certain period of time?
The DB is MySQL if it matters, thing is I can't tell whether this is done in the DB side (perhaps there's a way to configure this via MySQL?), or in my back-end code.
Is there a quick solution, or do I have to write code that does this, and have it run every day, deleting anything that wasn't added a month ago?
Thanks
I'd suggest creating a management command that queries for all the records in your model that are older than one month and delete those records. Then throw that management command into a daily cronjob. This should suit your needs.
you can solve this issue depends on your case,
if this data become with no value and you want to delete it
you can do that by
1- from database & using crontab
DELETE FROM mytable
WHERE date_field < UNIX_TIMESTAMP(DATE_SUB(NOW(), INTERVAL 30 DAY));
2- using managment command with crontab
import datetime
samples = Sample.objects.filter(sampledate__gt=datetime.date(2011, 1,
1), sampledate__lt=datetime.date(2011, 1, 31))
3- using celery with periodic task
http://celery.readthedocs.org/en/latest/userguide/periodic-tasks.html
You can always let the manager filter for you:
class RecentManager(models.Manager):
def get_queryset(self):
return super(
RecentManager,
self
).get_queryset().filter(
your_timestamp__gt=datetime.datetime.now()-datetime.timedelta(30)
)
class YourModel(models.Model):
#your fields, including your_timestamp
objects = RecentManager()
unrestricted = models.Manager()
#static
def delete_old():
YourModel.unrestricted.filter(
your_timestamp__lt=datetime.datetime.now()-datetime.timedelta(30)
).delete()
Hook up the delete to a management command which you can run in a cronjob or Celery task or whichever other infrastructure you have handy for async execution.
Related
I'm new at this guys(Laravel) and I need to store my data collected from my DB each six hours. I thought of using laravel's task scheduling, but in the kernel where I'm supposed to create my cron job, I don't think that I can return my data as an array
I use 6 arrays each one of them in a different page.
Here is an exemple of one of them :
$dataGlobal = DB::select("select * from articles order by articleDate ASC");
PS: the code bellow is in my controller and I return the array with its view
Kernel.php
protected function schedule(Schedule $schedule)
{
// $schedule->command('inspire')->hourly();
$schedule->call(function(){
DB::select("select * from articles order by articleDate ASC");
})->cron('* */6 * * *');
}
Any ideas guys ? How can I store my data in an array where m going to use it in my pages (It's already working when I stop manually the project and relaunch it with php artisan serve)
You can't return data from cron job.
I think what you need is:
Create cron job that collect data each 6 hours and store it on table to your database. so you could use that data from the table you have created.
Since I solved my issue, I'm sharing it here if someone faced the same thing.
Laravel is a PHP's Framework, and php is a scripting language. Each time a person naviguate through my site each row is executed at that time.
By doing so, my array will always contain latest rows of my database. I don't really need a cron-job for this.
I need to perform some actions in my app based on user's input. For e.g. if user selects 17:00-18:00, the timeframe is updated in the MySQL db and something should happen in that specific time.
Could someone tell me how can I achieve that?
You can use cron jobs for it.
For example, if you need to do some actions based on time value in your DB, you can write some code, that gets the current date and selects from MySQL all raws if your current date is between the date range of that raw and process them.
Then just make some cronjob that calls your code every minute, or an hour or whatever you need.
You can use cron package for that.
Here is an example:
const { CronJob } = require('cron');
// example cron job
const yourFunction = require('./yourFile.js');
// setting cron for example cronJob
const cron = new CronJob('0 * * * * *', yourFunction, null, true);
cron.start();
Or you can use Linux's crontab, you can read about how to use it here.
Configure frequency of cron with cron scheduler, you can play with it here.
I'm having a 'Tournament' sql table that contains start_time and end_time for my tournaments. I also have another table which has playerId and tournamentIds so I can tell which players playes in which tournament.
What I'm trying to do is to run a cron task to check my tournament table and see if tournament has ended so it can check players results from an external api. The problem is the external API has rate limit and I have to send my requestes every 1.5 sec.
What I tried to do is to write a cron job for every 10 seconds to check my tournament table (I couldn't come up with anyother solution rather than keep checking db):
cron.job("*/10 * * * * *", function(){
result = Query tournament table Where EndTime=<Now && EndTime+10second>=Now
if(result is not empty)
{
cron.job("*/1.5 * * * * *",function(){
send API requests for that userId
parse & store result in db
});
}
});
I don't feel right about this and it seems so buggy to me. Because the inner cron job might take longer than 10 seconds. Is there any good solution to do this. I'm using ExpressJS & MySQL.
The problem you are facing can be solved with event emitters. There is a very useful module node-schedule in npm which can help you in this scenario that you are telling. What you have to do is is to schedule a job to fire at the deadline of the project, that job will hit the 3rd party api and check for results.You an schedule a job like this
var schedule = require('node-schedule');
schedule.scheduleJob("jobName", "TimeToFire", function () {
//Put your api hit here.
//finally remove the schedule
if ("jobName" in schedule.scheduledJobs) {
schedule.scheduledJobs["jobName"].cancel();
delete schedule.scheduledJobs["jobName"];
}
})
Make sure you store all the jobs scheduled in the database also as a server crash will invalidate all the schedules that you have scheduled and will have to reschedule them again.
For fun, I'd like to see the set of users (in a Rails database) who were created during a specific hour range (2AM - 5AM to be specific), but on any day. They all have the typical created_at field. I think I know how to extract the hour from this field for one user, and then see if it falls in a range--but how I do do this for all of them? Should I just loop through them? (Even as I write it, that sounds naive).
The first part of Sontyas answer is the easy solution in rails.
I would however move that part to it's own place inside your class to separate your code from the framework a bit more.
# app/models/user.rb
class User < ActiveRecord::Base
# ...
def self.get_users_created_between(start_time, end_time)
User.where("TIME(created_at) BETWEEN TIME(?) AND TIME(?)", start_time, end_time)
end
# ...
end
And use it like this:
irb> User.get_users_created_between(Time.parse("2pm"), Time.parse("5pm"))
This provides you with a couple of benefits:
You can reuse it all over your code without ever having to worry about the syntax of the where or time ranges.
If for some weird reason rails decides to change the interface for this, you only need to edit one method and not code in a thousand places all over your project.
You can easily move this piece of code out of the user.rb if you feel that user.rb gets to big. Maybe to some dedicated finder or query class. Or to something like a repository pattern.
PS: Time functions may vary between different DBMS like MySQL, Postgresql, MSSQL etc. I don't know, if there is generic way to do this. This answer is for MySQL.
Try this,
User.where(created_at: Time.parse("2pm")..Time.parse("5pm"))
Or something like this
User.select { |user| user.created_at.hour.between?(2, 5) }
To return users that where created between two hours on any given day, use this:
User.where('HOUR(created_at) BETWEEN ? AND ?', 2, 5)
Please note that HOUR(created_at) only works for MySQL. The syntax in Postgresql is extract(hour from timestamp created_at) and strftime('%H' created_at) in SQLite.
The table has following fields: id, date, user_id, city, price
I would like the rows that are older than 1 year to be automaticaly deleted, but I am not really sure about how to do this. Should it be done by application or database? How?
Another issue here is that there are going to be ~50k inserts every year. What will happen to 'id' field in a couple of years? Won't the number get too large? What could be done in this case?
Many thanks in advance.
I would suggest creating a basic shell that handles deleting the records, and schedule a cron job that runs whenever you wish to check for records that should be deleted. The shell could be very simple, something like this:
class JanitorShell extends AppShell {
// put the model you wish to delete from here
public $uses = array('Model');
public function main() {
$this->Model->deleteAll(array(
'Model.created <' => date('Y-m-d', '-1 year')
));
$this->out('Records deleted.');
}
}
Then your cron job would run your shell. To run your shell, call:
cake -app /path/to/app janitor
This assumes cake is in your PATH. Of course, this is a very basic shell and could be easily expanded, keeping logs of what's been deleted or even just moving deleted records to a new table, a sort of 'soft delete'. You should probably not put such destructive code in main() since it runs each time you run the shell, but this will get you started.
To answer your second question, 50k inserts a year is nothing to fret about. To know the limits of your primary keys, read up on MySQL datatypes.
actually I think it should be
date('Y-m-d', strtotime('-1 year'))