I am currently getting products from one site, storing them in a database, and then having their prices display on another site. I am trying to get the prices from the one site to update daily in my database so the new updated prices can be displayed onto my other site.
Right now I am getting the products using an item number but have to manually go in and update any prices that have changed.
I am guessing I am going to have to use some kind of cronjob but not sure how to do this. I have no experience with cronjobs and am a noob with php.
Any ideas?
Thanks!
I have done some reading on the foreach loop and have written some code. But my foreach loop is only running once for the first item number. The foreach loop runs then goes to the "api.php" page but then stops. It doesn't continually loop for each item number. How do I tell it to go through all of item numbers in my database?
Also if you see anything else wrong in my code please let me know.
Thanks
....
$itemnumber = array("".$result['item_number']."");
foreach ($itemnumber as $item_number) {
echo "<form method=\"post\" action=\"api.php\" name=\"ChangeSubmit\" id=\"ChangeSubmit\">";
echo "<input type=\"text\" name=\"item_number\" value=\"{$item_number}\" />";
echo "<script type=\"text/javascript\">
function myfunc () {
var frm = document.getElementById(\"ChangeSubmit\");
frm.submit();
}
window.onload = myfunc;
</script></form>";
}
}
If you already retrieve the product data from an external site and store it in a local database, updating the prices from the same source should be no problem to you. Just retrieve the data, iterate through it in a foreach loop or similar and update the prices to the database based on the item number.
Once you have created the update script and run it manually, adding it as a cronjob will be as simple as running the command `crontab -e´ and adding this row to execute your script every midnight:
0 0 * * * /usr/local/bin/php /path/to/your/script.php
Don't forget to use the correct path for PHP for your system, running which php in the shell will tell you the path.
If you have cronjob's on your server, it'll be very apparent- You make a PHP script that updates it, and throw it in a daily cronjob.
However, I do it this way:
Method 1: At the beginning of every page request, check the last "update" time (you choose how to store it). If it's been more than a day, do the update and set the "update" time to the current time.
This way, every time someone loads a page and it's been a day since the last update, it updates for them. However, this means it's slower for random users, once a day. If this isn't acceptable, there's a little change:
Method 2: If you need to update (via the above method of checking), start an asyncronous request for the data, handle the rest of the page, flush it to the user, then in a while loop wait until the request finishes and update it.
The downside to method 2 is that the user won't see the updated values, but, the benefit is that it won't be any more of a wait for them.
Related
I'm new at this guys(Laravel) and I need to store my data collected from my DB each six hours. I thought of using laravel's task scheduling, but in the kernel where I'm supposed to create my cron job, I don't think that I can return my data as an array
I use 6 arrays each one of them in a different page.
Here is an exemple of one of them :
$dataGlobal = DB::select("select * from articles order by articleDate ASC");
PS: the code bellow is in my controller and I return the array with its view
Kernel.php
protected function schedule(Schedule $schedule)
{
// $schedule->command('inspire')->hourly();
$schedule->call(function(){
DB::select("select * from articles order by articleDate ASC");
})->cron('* */6 * * *');
}
Any ideas guys ? How can I store my data in an array where m going to use it in my pages (It's already working when I stop manually the project and relaunch it with php artisan serve)
You can't return data from cron job.
I think what you need is:
Create cron job that collect data each 6 hours and store it on table to your database. so you could use that data from the table you have created.
Since I solved my issue, I'm sharing it here if someone faced the same thing.
Laravel is a PHP's Framework, and php is a scripting language. Each time a person naviguate through my site each row is executed at that time.
By doing so, my array will always contain latest rows of my database. I don't really need a cron-job for this.
Thanks in advance for attempting to asssist me with this issue.
I'm using CakePHP 2 (2.10.22).
I have a system which creates applications. Each application that gets created has a unique application number. The MySQL database column that stores this application number is set to 'Not null' and 'Unique'. I'm using CakePHP to get the last used application number from the database to then build the next application number for the new application that needs to be created. The process that I have written works without any problem when a single request is received at a given point in time. The problem arises when two requests are received to create an application at the exact same time. The behaviour that I have observed is that the the request that gets picked up first gets the last application number - e.g. ABC001233 and assigns ABC001234 as the application number for the new application it needs to create. It successfully saves this application into the database. The second request which is running concurrently also gets ABC001233 as the last application number and tries to create a new application with ABC001234 as the application number. The MySQL database returns an error saying that the application number is not unique. I then put the second request to sleep for 2 seconds by which time the first application has successfully saved to the database. I then re-attempt the application creation process which first gets the last application number which should be ABC001234 but instead each database read keeps returning ABC001233 even though the first request has long been completed. Both requests have transactions in the controller. What I have noticed is that when I remove these transactions, the process works correctly where for the second request after the first attempt fails, the second attempt works correctly as the system correctly gets ABC001234 as the last application number and assigns ABC001235 as the new application number. I want to know what I need to be doing so as to ensure the process works correctly even with the transaction directives in the controller.
Please find below some basic information on how the code is structured -
Database
The last application number is ABC001233
Controller file
function create_application(){
$db_source->begin(); //The process works correctly if I remove this line.
$result = $Application->create_new();
if($result === true){
$db_source->commit();
)else{
$db_source->rollback();
}
}
Application model file
function get_new_application_number(){
$application_record = $this->find('first',[
'order'=>[
$this->name.'.application_number DESC'
],
'fields'=>[
$this->name.'.application_number'
]
]);
$old_application_number = $application_record[$this->name]['application_number'];
$new_application_number = $old_application_number+1;
return $new_application_number;
}
The above is where I feel the problem originates. For the first request that gets picked up, this find correctly finds that ABC001233 is the last application number and this function then returns ABC001234 as the next application number. For the second request, it also picks up ABC001233 as the last application number but will fail when it tries to save ABC001234 as the application number as the first request has already saved an application with that number. As a part of the second attempt for the second request (which occurs because of the do/while loop) this find is requested again, but instead of returning ABC001234 as the last application number (per the successfuly save of the first request), it keeps returning ABC001233 resulting in a failure to correctly save. If I remove the transaction from the controller, this then works correctly where it will return ABC001234 in the second attempt. I couldn't find any documentation as to why that is and what can be done about the same and is where I need some assistance. Thank you!
function create_new(){
$new_application_number = $this->get_new_application_number();
$save_attempts = 0;
do{
$save_exception = false;
try{
$result = $this->save([$this->name=>['application_number'=>$new_application_number]], [
'atomic'=>false
]);
}catch(Exception $e){
$save_exception = true;
sleep(2);
$new_application_number = $this->get_new_application_number();
}
}while($save_exception === true && $save_attempts++<5);
return !$save_exception;
}
You just have to lock the row with the previous number in a transaction using SELECT ... FOR UPDATE. It's much better than the whole table lock as said in the comments.
According to documentation https://book.cakephp.org/2/en/models/retrieving-your-data.html you just have to add 'lock' => true to get_new_application_number function:
function get_new_application_number(){
$application_record = $this->find('first',[
'order'=>[
$this->name.'.application_number DESC'
],
'fields'=>[
$this->name.'.application_number'
],
'lock'=>true
]);
$old_application_number = $application_record[$this->name]['application_number'];
$new_application_number = $old_application_number+1;
return $new_application_number;
}
How does it work:
The second transaction will wait on that request while the first transaction is ended.
P.S. According to documentation lock option was added in the 2.10.0 version of CakePHP.
Okay, I know this question has been half answered around here, but I'll explain in full and see if there maybe a better way to handle this then in just two parts.
I'm building an HTML5 voting site, that will close voting every half hour. And then open a new batch of voting.
What I need to do, is have the URL of this site be 'masked' ie:
have a random URL for it be generated that leads to the same location- so say I'm directing users to voting.com, I want them to only access it via the url votingstuff.com/hg67, and have that end portion be changed every half hour- and once the new url is generated, remove access to the site via the old URL.
I might add that after the user votes, I'd also like to close their voting access until the new URL is posted. AKA- everyone only gets one chance to vote.
Thoughts? Libraries to check out? Sites that do this? I know the randomization has been answered a few times, but usually it's a generator led to by a page, I want to START with a randomly generated URL, and not have a page that creates it as a function. I may have to write code that generates the page I guess?
Here is an approach I can think of -
You don't need to handle multiple URL's, you can achieve this on the same URL itself by enabling/disabling access to the URL content every hour.
You will need some action on your application/server side as well. For reference lets assume you have a PHP server running. You will need some table that holds a status setting for you, something like this -
Settings Table
id settingCode value
=============================
1 votingOpen 1
You will also need to setup a cron job on your server that will run every half an hour. The cron job's task would be to run every half an hour & change the value of the votingOpen to its inverse. So assume open your votings, initial value of the votingOpen field would be 1 i.e the first half an hour of voting starts.
Along with this you will start the cron job as well. Now what will happen is that after half an hour the cron job will fetch the value of votingOpen read it & if its 1, it will change it to 0 & if its 0 it will change it to 1. This will happen every half an hour on your server.
Now you need to make this votingOpen variable accessible on your client side code using a simple query to fetch its value. On each render of your voting page, on top somewhere you can check this value & if its 1 i.e true, you will write the logic to go ahead & show the voting page. But if its 0, you can redirect the user or show a different message on that page.
Lets say your voting happens on http://somedomain.com/voting, on the render logic of this page you can do something like this -
Voting View Logic
<?php
$votingOpen = {some database query that fetches the value from the database};
if($votingOpen) {
// your voting view resides here
} else {
// either redirect user to some other page
// or show some message like voting is closed on this same page.
}
?>
Limit multiple votes from one person
Normally the best way to handle this is a combination or registered user only vote + cookie/localStorage + IP tracking
But if you don't have registered user voting, you can still capture IP of user & allow only one vote per IP and you can also set a cookie or a localStorage variable in the users computer so he can only vote once.
Hope this helps.
Ref for Cron Job - http://code.tutsplus.com/tutorials/managing-cron-jobs-with-php--net-19428
The table has following fields: id, date, user_id, city, price
I would like the rows that are older than 1 year to be automaticaly deleted, but I am not really sure about how to do this. Should it be done by application or database? How?
Another issue here is that there are going to be ~50k inserts every year. What will happen to 'id' field in a couple of years? Won't the number get too large? What could be done in this case?
Many thanks in advance.
I would suggest creating a basic shell that handles deleting the records, and schedule a cron job that runs whenever you wish to check for records that should be deleted. The shell could be very simple, something like this:
class JanitorShell extends AppShell {
// put the model you wish to delete from here
public $uses = array('Model');
public function main() {
$this->Model->deleteAll(array(
'Model.created <' => date('Y-m-d', '-1 year')
));
$this->out('Records deleted.');
}
}
Then your cron job would run your shell. To run your shell, call:
cake -app /path/to/app janitor
This assumes cake is in your PATH. Of course, this is a very basic shell and could be easily expanded, keeping logs of what's been deleted or even just moving deleted records to a new table, a sort of 'soft delete'. You should probably not put such destructive code in main() since it runs each time you run the shell, but this will get you started.
To answer your second question, 50k inserts a year is nothing to fret about. To know the limits of your primary keys, read up on MySQL datatypes.
actually I think it should be
date('Y-m-d', strtotime('-1 year'))
I have an app that allows you to upload a recipe to the db, as part of the app i would like to have a "Featured Recipe" that is called randomly from the recipe db. I am using this method to call a random record
#featurerecipe = Recipe.offset(rand(Recipe.count)).limit(1)
As expected when i refresh the page the record changes, however I would like the random record to stay in place for say 24 hours, even when the page is refreshed. I'm looking for some advice on how to go about this or perhaps someone else has done this in a project of their own? I was thinking that you could have a db column called feature and then set a "true" value to it, set everything else as false and then somewhere a loop needs to occur to check the time spent at value "true", then after 24 hours pick another random recipe?
I may be well off here, any help appreciated
As you mentioned, add a "featured" flag to the table. Setup resque/resque-scheduler or delayed job to run every 24 hours and select a new item at random.