I created a php script that takes values from 1 table and inserts them into another.
This executes pretty quickly (1000 rows per second).
Everything went well. There are no duplicates in the table and I think all the values from the other table are inserted.
But something strange caught my attention:
There are a total of 903388 rows in the table but the highest ID that has auto-increment enabled is 898582.
That's a difference of 4806.
The other table has almost the same amount of items as the ID but that is always off by a bit due to the original table always getting new values.
Due to such a large database a MRE could not be produced
So now my question is:
How can the auto-increment value be lower than the total amount of rows?
The datatype of the original tables ID:
bigint(20)
The datatype of the IMAGES table ID:
int(11)
Show create table IMAGES;
CREATE TABLE `IMAGES` (
`ID` int(11) unsigned NOT NULL AUTO_INCREMENT,
`SRC` varchar(300) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_520_ci DEFAULT NULL COMMENT 'Pad naar afbeelding',
`VERWIJDERD` tinyint(1) NOT NULL DEFAULT 0 COMMENT '0=image is online, 1=image is verwijderd',
`DATUM` datetime NOT NULL DEFAULT current_timestamp() COMMENT 'Datum van upload',
`IP` varchar(15) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_520_ci DEFAULT NULL COMMENT 'ip van uploader',
`SITE` tinyint(1) DEFAULT NULL COMMENT 'site waar image is geupload',
`OID` int(11) DEFAULT NULL COMMENT 'occasion id',
`POSITIE` int(11) NOT NULL DEFAULT 0 COMMENT 'sorteer id',
`TYPE` tinyint(1) NOT NULL DEFAULT 0 COMMENT '0=normaal 1=schade',
PRIMARY KEY (`ID`),
KEY `OID` (`OID`)
) ENGINE=InnoDB AUTO_INCREMENT=898583 DEFAULT CHARSET=latin1
My PHP script that fills the table:
<form action="" method="post">
<label for="fname">Rijen:</label><br/>
offset<input type="number" name="offset"><br/>
amount<input type="number" name="amount"><br/>
<button type="submit" name="submit">Go</button>
</form><br/>
<?php
$per_insert = 100;
if(isset($_POST['submit'])){
echo "Offset: ".$_POST['offset']."<br/>";
echo "Limit: ".$_POST['amount']."<br/>";
$msc = microtime(true);
$count = (is_numeric($_POST['offset']) ? (int)$_POST['offset'] : 0);
$amount = (is_numeric($_POST['amount']) ? (int)$_POST['amount'] : 0);
$qcount = 0;
$filter_array = array('images/occ/', null, '');
for ($i = $count+$per_insert; $i <= $amount; $i+=$per_insert){
$valuesq = array();
$olddataq = $mysqli->query("SELECT `ID`,SITE,DATUM,IP,`IMG_1`,`IMG_2`,`IMG_3`,`IMG_4`,`IMG_5`,`IMG_6`,`IMG_7`,`IMG_8`,`IMG_9`,`IMG_10`,`IMG_11`,`IMG_12`,`IMG_13`,`IMG_14`,`IMG_15`,`IMG_16`,`IMG_17`,`IMG_18`,`IMG_19`,`IMG_20`,`IMGS_1`,`IMGS_2`,`IMGS_3`,`IMGS_4`,`IMGS_5`,`IMGS_6`,`IMGS_7`,`IMGS_8`,`IMGS_9`,`IMGS_10`,`IMGS_11`,`IMGS_12`,`IMGS_13`,`IMGS_14`,`IMGS_15`,`IMGS_16`,`IMGS_17`,`IMGS_18`,`IMGS_19`,`IMGS_20` FROM `OCCASIONS` LIMIT ".$per_insert." OFFSET ".$count.";");
$qcount++;
$schade = $normaal = 0;
while($olddata = $olddataq->fetch_assoc()){
$olddata = array_diff($olddata, $filter_array);
$id = $olddata['ID'];
$datum = $olddata['DATUM'];
$ip = $olddata['IP'];
$site = $olddata['SITE'];
unset($olddata['DATUM']);
unset($olddata['ID']);
unset($olddata['IP']);
unset($olddata['SITE']);
while ($data = current($olddata)) {
$key = explode('_',key($olddata));
if($key[0] == 'IMG'){
//normale image
$datacheck = check_fotodata($data, $id, $key[1], 0);
if($datacheck === false){
$valuesq[] = "('".$data."','".$datum."','".$ip."',".$site.",".$id.", ".$key[1].", 0,0)";
}else{
$valuesq[] = $datacheck;
}
}else{
//schade image
$datacheck = check_fotodata($data, $id, $key[1], 1);
if($datacheck === false){
$valuesq[] = "('".$data."','".$datum."','".$ip."',".$site.",".$id.", ".$key[1].", 1,0)";
}else{
$valuesq[] = $datacheck;
}
}
next($olddata);
}
}
$count += $per_insert;
//var_dump($valuesq);
$mysqli->query("INSERT INTO IMAGES (SRC, DATUM, IP, SITE, OID, POSITIE, TYPE, VERWIJDERD) VALUES ". implode(",", $valuesq));
$qcount++;
}
$msc = microtime(true)-$msc;
echo "buildtime: <br/>";
echo $msc . ' s<br/>'; // in seconds
echo ($msc * 1000) . ' ms<br/>'; // in millseconds
echo $qcount . "<br/>";
$msc = microtime(true);
}
function check_fotodata($image, $oid, $pos, $type){
global $qcount, $mysqli;
$checkdataq = $mysqli->query("SELECT * FROM FOTODATA WHERE KID = ". $oid ." AND IMG = '". $image ."'");
$qcount++;
if($checkdataq->num_rows > 0){
$checkdata = $checkdataq->fetch_assoc();
if($checkdata['INFO'] == 'Verwijderd'){
$del = 1;
}else{
$del = 0;
}
return "('".$checkdata['IMG']."', '".$checkdata['DATUM']."', '".$checkdata['IP']."', '".$checkdata['SITE']."', '".$checkdata['KID']."',".$pos.",".$type.",".$del.")";
}else{
return false;
}
}
Please check count of table by below query first then compare. Count of table and max id should be equal if you use auto_increment_offset 1 and auto_increment_increment 1 in MySQL.
select count(*) from IMAGES;
Please comment what you find.
I am parsing currency rates from a rss.xml feed that all works great. I am now trying to insert that data into a database called rates with a table called tblRates. I keep getting this error and do not know why. Here is the function in the model I am using to try to batch insert into the database.
function addIQDRates($Data){
if($this->db->insert_batch('tblRates', $Data, 'Currency'))
{
return $this->db->affected_rows();
}else{
return FALSE;
}
}
Also here is the foreach statement I am using in my controller to sort the data from the xml file and to insert it into the database.
$Data = array();
$Data = array();
$Count = 0;
foreach ($xml->channel->item as $currencyInfo) {
$Data[$Count]['Currency'] = trim(str_replace("/USD", "", $currencyInfo->title)); // UNIQUE
$Data[$Count]['PubDate'] = date('Y-m-d H:i:s', strtotime(trim($currencyInfo->pubDate)));
$Data['CXRate'] = trim(preg_replace("/[^0-9,.]/", "", str_replace("1 United States Dollar = ", "", $currencyInfo->description)));
$Data[$Count]['DateCreated'] = date('Y-m-d H:i:s');
$Count++;
}
$TotalRows = $this->mycron_model->addIQDRates($Data);
Also here is my Create Table statement
CREATE TABLE IF NOT EXISTS `tblRates` (
`RateID` int(11) NOT NULL AUTO_INCREMENT,
`Currency` varchar(50) NOT NULL,
`PubDate` datetime NOT NULL,
`CXRate` int(11) NOT NULL,
`DateCreated` datetime NOT NULL,
PRIMARY KEY (`RateID`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 AUTO_INCREMENT=1 ;
all help greatly appreciated.
I am not sure, you might have written $Data['CXRate'] instead of $Data[$Count]['CXRate'].
So the loop should like like below:
foreach ($xml->channel->item as $currencyInfo) {
$Data[$Count]['Currency'] = trim(str_replace("/USD", "", $currencyInfo->title)); // UNIQUE
$Data[$Count]['PubDate'] = date('Y-m-d H:i:s', strtotime(trim($currencyInfo->pubDate)));
$Data[$Count]['CXRate'] = trim(preg_replace("/[^0-9,.]/", "", str_replace("1 United States Dollar = ", "", $currencyInfo->description)));
$Data[$Count]['DateCreated'] = date('Y-m-d H:i:s');
$Count++;
}
I want to insert similar strings in a page in to the database at once! for example, I want to insert each line in to a table row:
$flag = 'AD.png'; $title = 'Andorra';
$flag = 'AE.png'; $title = 'United Arab Emirates';
$flag = 'AF.png'; $title = 'Afghanistan';
$flag = 'AG.png'; $title = 'Antigua and Barbuda';
$flag = 'AI.png'; $title = 'Anguilla';
$flag = 'AL.png'; $title = 'Albania';
$flag = 'AM.png'; $title = 'Armenia';
$flag = 'AN.png'; $title = 'Netherlands Antilles';
My database rows are:
`flag_Id` int(11) NOT NULL AUTO_INCREMENT,
`flag_Title` varchar(250) NOT NULL,
`flag_ImageId` varchar(250) NOT NULL,
PRIMARY KEY (`flag_Id`)
How should I do that?!
You can insert multiple records in one query:
INSERT INTO tablename (flag_ImageId, flag_Title) VALUES
('AD.png', 'Andorra'),
('AI.png', 'Anguilla),
...
('US.png', 'USA');
this foreach item:
insert into tablename flag_Title, flag_ImageId values ('AD.png', 'Andorra');
If you want to do it in 1 transaction add START TRANSACTION; and COMMIT; like:
START TRANSACTION;
insert into tablename flag_Title, flag_ImageId values ('AD.png', 'Andorra');
insert into tablename flag_Title, flag_ImageId values ('AD.png', 'Andorra');
COMMIT;
I'm outputting a list of cruises to show availability status, but I'm getting each cruise several times with each cabins status, I only want the cruise, then if all the cabins are sold out the outputted status should be C, otherwise A
here are my queries
<?php
$maketemp = "CREATE TEMPORARY TABLE temp (
`sailingId` varchar(5) NOT NULL,
marketCurrency varchar(3),
`Status` varchar(1)
)";
mysql_query( $maketemp, $connection ) or
die ( "Sql error : " . mysql_error ( ) );
$inserttemp = "INSERT INTO
temp
SELECT
code AS sailingId,
'USD' AS marketCurrency,
CASE
WHEN fares_usa.status = 'sold_out' THEN 'C'
END AS Status
FROM
cruises,
fares_usa
WHERE
live ='Y' AND
cruises.id = fares_usa.cruise_id";
mysql_query( $inserttemp, $connection ) or
die ( "Sql error : " . mysql_error ( ) );
$select = "SELECT sailingId, marketCurrency, Status FROM temp";
$export = mysql_query ( $select, $connection ) or
die ( "Sql error : " . mysql_error( ) );
$fields = mysql_num_fields ( $export );
So this is how I did it...
$inserttemp = "INSERT INTO temp SELECT code AS sailingId, 'USD' AS marketCurrency, CASE WHEN GROUP_CONCAT(DISTINCT status) = 'Sold Out' THEN 'C' ELSE 'A' END AS Status FROM cruises, fares_usa WHERE live ='Y' AND cruises.id = fares_usa.cruise_id GROUP BY cruises.code";
result !
session_start();
if(!$_SESSION['user_id'])
{
$_SESSION['user_id'] = rand(1, 1000000);
include 'database_connect.php';
mysql_query('INSERT INTO product_views (user_session_id)
VALUES
('.$_SESSION['user_id'].')');
}
$productid = $_GET['name'];
$query = 'SELECT * FROM product_views WHERE user_session_id = '.$_SESSION['user_id'].'';
$result = mysql_query($query);
while ($row = mysql_fetch_array($result))
{
mysql_query('UPDATE product_views SET modelNumber="'.$productid.'" WHERE user_session_id="'.$_SESSION['user_id'].'"');
}
My field modelNumber is set to null, and I am performing an Update via the last query.
Do you think that since the default value is null, it is therefore not allowing an insertion?
My table structure:
CREATE TABLE `product_views` (
`id` int(10) DEFAULT NULL,
`user_session_id` int(11) DEFAULT NULL,
`product_id` varchar(100) DEFAULT NULL,
`view_date` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`modelNumber` varchar(...
I'm confused:
$query = 'SELECT * FROM product_views WHERE user_session_id = '.$_SESSION['user_id'].'';
$result = mysql_query($query);
while ($row = mysql_fetch_array($result))
{
mysql_query('UPDATE product_views SET modelNumber="'.$productid.'" WHERE user_session_id="'.$_SESSION['user_id'].'"');
}
Why are you looping through this result set if you're not even using $row?
Edit: I think this is what you're really trying to do:
session_start();
if(!$_SESSION['user_id'])
{
// Get the user ID
$_SESSION['user_id'] = rand(1, 1000000);
require_once('database_connect.php');
// Get the model number and escape it to prevent SQL injection
$model_number = mysql_real_escape_string($_GET['name']);
// Insert a row that associates the user_id with the model number
mysql_query("INSERT INTO product_views (user_session_id,modelNumber) VALUES('{$_SESSION['user_id']}', '$model_number')");
}