Freaking behaviour with Zend_Date and Cronjob - mysql

We have an Cron-Script, which detects - if some users got kicked out of our application.
We can detect this, if a specific value is 1 - but in the the stream, no new entries get set.
Scripts run every hour. Mostly non are detected. But since 2012-10-31 23:59:03 every user got detected. If i run the script on my local maschine or even on the same machine as the cron runs. Everything got handled as it should.
First things first, our script:
require_once ('cron_init.php');
ini_set('date.timezone', 'Europe/Berlin');
ini_set('max_execution_time', 30);
ini_set('memory_limit', -1);
error_reporting(E_ALL);
ini_set("display_errors", 1);
Zend_Date::setOptions(array('fix_dst' => true));
$userinfos = new Repricing_Dbservices_Userinfos();
$users = $userinfos->getUsersForRepricing();
$repricingstream = new Repricing_Dbservices_Repricingstream();
$error = new Repricing_Dbservices_Error();
if($users!==false AND count($users)>0){
$counter = 0;
$errCounter = 0;
$jetzt = new Zend_Date();
$jetzt->setTimezone('Europe/Berlin');
$jetzt = $jetzt->get(Zend_Date::TIMESTAMP);
foreach($users as $user){
$stream = $repricingstream->getStreamLimit($user);
$last = new Zend_Date($stream);
$last->setTimezone('Europe/Berlin');
$last = $last->get(Zend_Date::TIMESTAMP);
$diff = (($jetzt-$last)/60);
$error->setError(1, 'DIED', $diff, $user);
if($diff > 50 ){
$errCounter++;
$userinfos->setUserFree($user);
$error->setError(1, 'DIED', 'ANSTOSSEN', $user);
}
$counter++;
}
$error->setError(1, $errCounter, 'ANSTOSSEN_ALL', 'ALL');
}
Usually $diff >= 0 AND $diff <= 4 but, we detected, that $diff is always round about 381595. If we run it out of cron $diff is, as it should.
We also detected, that $jetzt is now ( as it should ) only $last is much more later. 381595 later. But that shouldnt be. The last stream-date is fully normal. We cant understand this behaviour of. Zend_Date with cron. Bevor 2012-10-21 23:59:03 the script run 2 weeks as it should. We cant explain, how come. Can you?

Consider this:
$right = new Zend_Date('2012-11-01 12:12:12', Zend_Date::ISO_8601);
var_dump( $right->getIso() ); // 2012-11-01T12:12:12+00:00
var_dump( $right->getTimestamp() ); // 1351771932
$wrong = new Zend_Date('2012-11-01 12:12:12', null, 'en_US');
var_dump( $wrong->getIso() ); // 2012-01-11T12:12:12+00:00
var_dump( $wrong->getTimestamp() ); // 1326283932
Now the real freaky part: on my PC it's the second behavior that is default - i.e., when no additional params are given to Zend_Date constructor.
The point is, Zend_Date is a bit... too helpful when trying to parse datetime strings. For example, it's taking the locale into account - but the locale both of server and client! And if the string cannot be parsed within this locale's rules, it silently gives up - and tries to use another rule.
That's why 2012-10-29 was parsed as October, 29 (despite of what locale suggested, as there's no 29th month) - but 2012-11-01 became January, 11 - and messed up your script big time. )

Related

Loop in column name MYSQL

I am using MYSQL.My table contains column name as Revenue2000,Revenue2001,Revenue2002,....,Revenue 2016,Revenue 2017
Traditional way(to select all column manually):
select Revenue2005,
Revenue2006,
Revenue2007,
Revenue2008,
Revenue2009,
Revenue2010
from table_name
Desired Way:
I want to write a Dynamic select statement .There should 2 variables "start" and "end" so that i can make it dynamic.User has the option to specify the starting year and ending year and can view the desired result.
In above case, Start year =2005
End Year=2010
Yes, it's bad database design, and the best answer would be "don't do this at all, just fix your table." Unfortunately, sometimes you're stuck with something someone else made, and can't change it for whatever reason, but you still need to accomplish something (welcome to my life). I would do it like this:
Get the years from user input and convert them to integers in case someone enters something silly/naughty. Don't depend on client-side validation. Prepared statements won't help you here because these will be used as parts of column names.
$start = (int) $_POST['start'];
$end = (int) $_POST['end'];
Do a quick sanity check to make sure that the range makes sense and should work with what's in your database.
if ($start > $end
|| $start < $lowest_year_in_your_db
|| $end > $highest_year_in_your_db) {
// quit with error
}
Then you can generate a list of columns to use in your query. Here's one way with range and array_map, but you could also just build a string with a for loop.
$columns = implode(', ', array_map(function($year) {
return "Revenue$year";
}, range($start, $end)));
$sql = "SELECT $columns FROM table_name";
Theoretically, the worst thing that should be able to happen with this is that you'd get a column that didn't exist, and your query would fail.
But really, if you have any choice about it, don't do this. Normalize your database as people have stated in the comments, or find whoever keeps adding more year columns to the database and make them do it.
As already pointed out the database design is horrible. You should really normalize it, it's worth the effort.
However if that is not possible at the moment the follow code should do exactly what you need:
// Connect to DB
$mysqli = new mysqli("localhost", "USERNAME", "PASSWORD", "DATABASE");
// Get column names
$columns = $mysqli->query('SHOW COLUMNS FROM revenue')->fetch_all();
$columnNames = array_column($columns, 0);
// Extract years from column names
$years = array_map(function($columnName) {
return (int) substr($columnName, -4);
}, $columnNames);
// Get max and min year
$maxYear = max($years);
$minYear = min($years);
// Input year start and end
$start = (int) $_POST['start']; // User-input
$end = (int) $_POST['end']; // User-input
// Avoid wrong inputs
if($start > $end || $start < $minYear || $end > $maxYear) {
die('Error');
}
// Create the SQL-query
$selectColumns = [];
for ($i = $start; $i <= $end; $i++) {
$selectColumns[] = "revenue" . $i;
}
$queryString = "SELECT " . implode(", ", $selectColumns) . " FROM TABLE";
// Run the query
// ...

Too many sql connections error : due to long polling

I have designed a coding platform just like Spoj and Codeforces for competitions to be organised in my college on LAN.
I have used long polling there so that any announcements from the Admin can be broadcasted to all users with a JavaScript alert message. When anything is posted on the forum then the admin also gets a notification.
But for just 16 users (including the 1 Admin) accessing the site, the server went down showing too many sql connections. I restarted my laptop (server) and it continued for a while, then again went down; giving the same error message as before.
When I removed both long-poll processes everything continued smoothly.
Server-side code for long-poll:
include 'dbconnect.php';
$old_ann_id = $_GET['old_ann_id'];
$resultann = mysqli_query($con,"SELECT cmntid FROM announcements ORDER BY cmntid DESC LIMIT 1");
while($rowann = mysqli_fetch_array($resultann)){
$last_ann_id = $rowann['cmntid'];
}
while($last_ann_id <= $old_ann_id){
usleep(10000000);
clearstatcache();
$resultann = mysqli_query($con,"SELECT cmntid FROM announcements ORDER BY cmntid DESC LIMIT 1");
while($rowann = mysqli_fetch_array($resultann)){
$last_ann_id = $rowann['cmntid'];
}
}
$response = array();
$response['msg'] = 'new';
$response['old_ann_id'] = $last_ann_id;
$resultann = mysqli_query($con, "Select announcements from announcements where cmntid = $last_ann_id");
while($rowann = mysqli_fetch_array($resultann)){
$response['announcement'] = $rowann['announcements'];
}
echo json_encode($response);
Max connections is defined. Think the default is 100 or 151 connections depending on the version of MySQL. You can see the value in "Server variables and settings" in phpmyadmin (or directly by executing *show variables like "max_connections";* ).
If that is set to something very low (say 10) and you have (say) 15 users you will hit the limit rapidly. You are giving each long polling script its own connection, and that connection is probably sitting open until that long polling script ends. You could likely reduce this by having the script disconnect after each time it checks the database, then reconnect the next time it checks (ie, if your long polling script checks the db every 5 seconds you probably have well over 4.5 seconds of that 5 seconds currently where there is a connection to the db but where the connection is not being used)
However you could have a larger number of connections, but if you trigger the ajax polling multiple times per user, each could have several simultaneous connections. This is probably quite easy to do with a minor bug in your javascript.
Possibly worse if you are using a persistent connections you might leave connections open after the user has left the page that calls the long polling script.
EDIT - update based on your script.
Note I am not sure exactly what your dbconnect.php include is doing. I might be possible to easily call a connect / disconnect function in that include, but I have just put it in this example code as using the mysqlu_close and mysqli_connect functions.
<?php
include 'dbconnect.php';
$old_ann_id = $_GET['old_ann_id'];
$resultann = mysqli_query($con,"SELECT MAX(cmntid) FROM announcements");
if($rowann = mysqli_fetch_array($resultann))
{
$last_ann_id = $rowann['cmntid'];
}
$timeout = 0;
while($last_ann_id <=$old_ann_id and $timeout < 6)
{
$timeout++;
mysqli_close($con);
usleep(10000000);
clearstatcache();
$con = mysqli_connect("myhost","myuser","mypassw","mybd");
$resultann = mysqli_query($con,"SELECT MAX(cmntid) FROM announcements");
if($rowann = mysqli_fetch_array($resultann))
{
$last_ann_id = $rowann['cmntid'];
}
}
if ($last_ann_id >$old_ann_id)
{
$response = array();
$response['msg'] = 'new';
$response['old_ann_id'] = $last_ann_id;
$resultann=mysqli_query($con,"SELECT cmntid, announcements FROM announcements WHERE cmntid>$old_ann_id ORDER BY cmntid");
while($rowann = mysqli_fetch_array($resultann))
{
$response['announcement'][]=$rowann['announcements'];
$response['old_ann_id'] = $rowann['cmntid'];
}
mysqli_close($con);
echo json_encode($response);
}
else
{
echo "No annoucements - resubmit";
}
?>
I have added a count to the main loop. But it will drop out of the loop whether anything is found once it has executed 6 times. This way even if someone leaves the page the script will only be running for a short time afterwards (max a minute). You will have to amend you javascript to catch this and resubmit the ajax call.
Also I have changed the announcement in the response to be an array. This way if there are several announcements while the script is running all will be brought back.

mysql_affected_rows sometimes returns 0 instead of 1

I have a strange problem with php scripts - mysql_affected_rows() sometimes returns "0" for no reason.
There is a similar question #stackoverflow and answer to this question is:
MySQL only actually updates a row if there would be a noticeable difference before and after the updat.
But this is not my case. For example, if value before update is 1320402744 and value after update is 1320402944 mysql_affected_rows() anyway return "0". Is this difference not enough noticable?
Below are 3 files. As you can see, all files include file "functions.inc.php" which calls function "online()".
File "login.php" is working fine. It inserts a new row in "session" table correctly.
File "content.php" is working fine - it displays content and correctly runs function "online() in "functions.inc.php".
Then I call file "test.php". It deletes "something from sometable" correctly. Then it refreshes itself (Header("Location: /test.php");). After refreshing I am logged off.
I added this to "online()" function:
echo "affected_rows";
It returns 0.
I added more code to "online() function:
$checkuser = mysql_query("SELECT userid FROM session WHERE userid = '" . $_SESSION['id'] . "'") or die('Error');
$found = mysql_num_rows($checkuser);
echo $found;
$result = mysql_query("UPDATE session SET time='$ctime' WHERE userid='".$_SESSION['id']."'") or die('Error');
$affected_rows = mysql_affected_rows();
if ($affected_rows != 1) #session_destroy();
echo $affected_rows;
The result is 1 and 0.
I checked the database. "time" field in session table has been updated.
So, I can't understand how is it possible that the row exists, it updates correctly but mysql_affected_rows(); returns 0, and why this happends only if te same page has been refreshed.
functions.inc.php
<?php
#ob_start();#session_start();
#mysql_connect(C_HOST, C_USER, C_PASS) or die('Cant connect');
#mysql_select_db(C_BASE) or die('Cant select DB');
function online() {
$ctime = time()+1800;
if((isset($_SESSION['id']))&&(is_numeric($_SESSION['id']))) {
$query = mysql_query("UPDATE session SET time='$ctime2' WHERE userid='".$_SESSION['id']."'") or die('Error');
$affected_rows = mysql_affected_rows();
if ($affected_rows != 1) #session_destroy();
}
}
//many other functions go here
online();
?>
login.php
<?php
include_once 'configuration.inc.php';
include_once 'functions.inc.php';
//many things go here
$upd = mysql_query("INSERT INTO session VALUES ('" . $i['id'] . "','$ctime')") or die('Error2');
Header("Location: /content.php?justlogged=1");
die;
?>
content.php
<?php
include_once 'configuration.inc.php';
include_once 'functions.inc.php';
//many thing go here
echo "content";
?>
test.php
<?php
include_once 'configuration.inc.php';
include_once 'functions.inc.php';
if (isset($_GET['tid'])&&(is_numeric($_GET['tid']))){
$result = mysql_query("delete from some_table where something = '" . $_GET['tid'] . "'") or die('Error123a');
Header("Location: /test.php");
die;
}
//file content
?>
In your function.inc.php you call online() - session time is changed every second. But can it be that you're switching between pages (login, content, test) more faster than 1 second? In that case time would be the same and you'd get session destroy because of unaffected rows
Edit:
Yes. As I thought.
See how it comes:
you call login.php: after successful login it creates new session with time X. After this you're immediately redirected to content.php (time is still X) which calls online again. And of course, as you redirected immediately - time is the same.. so already at point of content.php session is already destroyed, because time wasn't changed.

Mysql Server has gone away error on PHP script

I've wrote a script to batch process domains and retrieve data on each one. For each domain retrieved, it connects to a remote page via curl and retrieves the data required for 30 domains at a time.
This page typical takes between 2 - 3 mins to load and return the curl result, at this point, the details are parsed and placed into an array (page rank tools function).
Upon running this script via CRON, I keep getting the error 'MySQL server has gone away'.
Can anyone tell me if I'm missing something obvious that could be causing this?
// script dies after 4 mins in time for next cron to start
set_time_limit(240);
include('../include_prehead.php');
$sql = "SELECT id, url FROM domains WHERE (provider_id = 9 OR provider_id = 10) AND google_page_rank IS NULL LIMIT 30";
$result = mysql_query($sql);
$row = mysql_fetch_assoc($result);
do {
$url_list[$row['id']] = $row['url'];
} while ($row = mysql_fetch_assoc($result));
// curl domain information page - typically takes about 3 minutes
$pr = page_rank_tools($url_list);
foreach ($pr AS $p) {
// each domain
if (isset($p['google_page_rank']) && isset($p['alexa_rank']) && isset($p['links_in_yahoo']) && isset($p['links_in_google'])) {
$sql = "UPDATE domains SET google_page_rank = '".$p['google_page_rank']."' , alexa_rank = '".$p['alexa_rank']."' , links_in_yahoo = '".$p['links_in_yahoo']."' , links_in_google = '".$p['links_in_google']."' WHERE id = '".$p['id']."'";
mysql_query($sql) or die(mysql_error());
}
}
Thanks
CJ
This happens because MySQL connection has its own timeout and while you are parsing your pages, well, it ends. You can try to increase this timeout with
ini_set('mysql.connect_timeout', 300);
ini_set('default_socket_timeout', 300);
(as mentioned in MySQL server has gone away - in exactly 60 seconds)
Or just call mysql_connect() again.
Because the curl take too long time, you can consider to connect again your database before entering the LOOP for update
There are many reasons why this error occurs. See a list here, it may be something you can fix quite easily
MySQL Server has gone away

Multidimensional Array insert into Mysql rows

I have an Array (twodimensional) and i insert it into my database.
My Code:
$yourArr = $_POST;
$action = $yourArr['action'];
$mysql = $yourArr['mysql'];
$total = $yourArr['total'];
unset( $yourArr['action'] , $yourArr['mysql'] , $yourArr['total'] );
foreach ($yourArr as $k => $v) {
list($type,$num) = explode('_item_',$k);
$items[$num][$type] = $v;
$pnr= $items[$num][pnr];
$pkt= $items[$num][pkt];
$desc= $items[$num][desc];
$qty= $items[$num][qty];
$price= $items[$num][price];
$eintragen = mysql_query("INSERT INTO rechnungspositionen (artikelnummer, menge, artikel, beschreibung,preis) VALUES ('$pnr', '$qty', '$pkt', '$desc', '$price')");
}
I get 5 inserts in the Database but only the 5th have the informations i want. The firsts are incomplete.
Can someone help me?
Sorry for my english.
check if You have sent vars from browser in array (like
input name="some_name[]" ...
also You can check, what You get at any time by putting var_dump($your_var) in any place in script.
good luck:)
You probably want to have your query and the 5 assignments above that outside of the foreach. Instead in a new loop which only executes once for every item instead of 5 times. Your indentation even suggests the same however your brackets do not.
Currently it is only assigning one value each time and executing a new query. After 5 times all the variables are assigned and the last inserted row finally has everything proper.
error_reporting(E_ALL);
$items = array();
foreach($yourArr as $k => $v) {
// check here if the variable is one you need
list($type, $num) = explode('_item_', $k);
$items[$num][$type] = $v;
}
foreach($items as $item) {
$pnr = mysql_real_escape_string($item['pnr']);
$pkt = mysql_real_escape_string($item['pkt']);
$desc = mysql_real_escape_string($item['desc']);
$qty = mysql_real_escape_string($item['qty']);
$price = mysql_real_escape_string($item['price']);
$eintragen = mysql_query("INSERT INTO rechnungspositionen (artikelnummer, menge, artikel, beschreibung,preis) VALUES ('$pnr', '$qty', '$pkt', '$desc', '$price')");
}
Switching on your error level to E_ALL would have hinted in such a direction, among else:
unquoted array-keys: if a constant of
the same name exists your script will
be unpredictable.
unescaped variables: malformed values
or even just containing a quote which
needs to be there will fail your
query or worse.
naïve exploding: not each $_POST-key
variable will contain the string
item and your list will fail, including subsequent use of $num