I have a query running on a large table that worked when I did limit 100. When I remove the limit I get:
[Err] 126 - Incorrect key file for table '/tmp/#sql_5e2d_6.MYI'; try
to repair it
I checked with server admin basically the /tmp file fills up quickly.
Is there a way to set up the query to flush the table as it goes along? Or run say 100 records, stop, re-run? The query is pretty simple:
select distinct a,
min(b) N_b
from K
group by a;
At the end of the day what I am trying to do is delete from a large table duplicate records, keeping the record with the lowest value in b. This was the initial select statement.
You can use something like,
*** Edit according to the language you use.
$sql = true;
int i = 0;
While($sql){
$sql = "select distinct a,
min(b) N_b
from K
group by a LIMIT i, i+99";
//Do whatever you want.
i=i+100;
}
Related
I have this query
SELECT * FROM outbox where Status=0 ;
then I need to update the selected records so Status should be equal 1
i.e (UPDATE outbox(selected records from SELECT query) SET Status =1 )
any help ?
This is a much harder problem than it sounds. Yes, in the simplistic case where you are only thinking of one user and a few records, it seems easy. But, databases are designed to be ACID-compliant, with multiple users and multiple concurrent transactions that can all be affecting the data at the same time. And there is no single statement in MySQL that does what you want (other databases support an OUTPUT clause, RETURNING or something similar).
One structure that will work in MySQL is to place the items in a temporary table, then do the update, then return them. The following shows the semantics using transactions:
start transaction;
create temporary table TempOutboxStatus0 as
select *
from outbox
where status = 0;
update outbox o
set status = 1
where status = 0;
select *
from TempOutboxStatus0;
commit;
For the update, I actually prefer:
where exists (select 1 from TempOutboxStatus0 t where t.outboxid = o.outboxid);
because its intention is clearer -- and the code is safer in case the conditions subtly change.
Note: you may want to use explicit table locks. Such considerations depend on the storage engine you are using.
BEGIN
Start transaction;
SELECT *
FROM
outbox
where
Status = 0 and
Is_Expired = 0 and
Service_ID=p_service_id
order by
Next_Try_Date asc FOR Update;
update outbox
set
Status=1
where
Status = 0 and
Is_Expired = 0 and
Service_ID=p_service_id;
commit;
END
is this possible .. it seems it works with me
You can do something like that, the outbox is your table:
update outbox set Status = 1 where Status = 0
you can do it like below
$sql=mysql_query("SELECT * FROM outbox where `Status`=0");
while($result=mysql_fetch_array($sql))
{
$update="UPDATE `outbox` SET `Status` =1 where
'your column name'='your previous fetched value');
}
I have a MySQL table that grows very rapidly. I would like to have an ongoing process (e.g. running every 30 minutes) that selects all the rows that are "new" since the last time the process was run. The table has an AUTO_INCREMENT column called id, and I thought I could use it to make sure I'm getting all the records. Here was the approach:
Initialize
LAST_ID_SELECTED = 0
Run repeatedly, forever:
startID = 1 + LAST_ID_SELECTED
nextAutoIncrementID = getQuery("SELECT AUTO_INCREMENT FROM information_schema.tables WHERE TABLE_NAME = 'myTable'")
lastID = nextAutoIncrementID - 1
data = SELECT * FROM myTable WHERE id BETWEEN startID AND lastID
LAST_ID_SELECTED = lastID
However, I don't think this will actually work: I'm worried that I can get my (startID, lastID) range and make my select query before all the inserts for the ids <= lastID have been completed. So I won't actually get all the rows in the range (startID,lastID).
What's the right way to do this?
Are there any solutions on MySQL script to filter the results with specific interval number.
For example, if I have 100,000 records in database and I'd like to get only the record number 1000, 2000, 3000, etc. (step by 1000).
I could do this on server side script by getting the entire results (e.g. 100,000) and use syntax like:
for($i=0, $i <= 100,000, $i = $i+1000) $filterResult[] = $record[$i];
However, as you may see, it would pull stress to the system as 100,000 records will need to generated first.
Are there any solutions that could complete from database script? Please note that, primary key may not start with 1 - 100,000 as the results based on some condition in where clause.
Your help would be really appreciated.
You can do:
SELECT *
FROM tbl
WHERE id % 1000 = 0
But it seems like you don't want to rely on the primary key value, but rather the row ranking of a result set.
In that case, you can do:
SELECT *
FROM (
SELECT *, #rn:=#rn+1 AS rank
FROM tbl
CROSS JOIN (SELECT #rn:=0) var_init
WHERE column1 = value AND
column2 = value
) a
WHERE a.rank % 1000 = 0
Where column1 = value AND column2 = value is just a placeholder for whatever filtration you're doing in your query.
I have a mysql table of data and I need to only return the rows that do not have a status of "Deactive" and do not have a Total of 0 (but there can be rows where status is deactive and total is not 0 and vice versa). Before I needed this requirement I was just doing the standard query to select all rows:
SELECT * FROM tableName WHERE uid = id;
However now when I change the query to add the constraints above:
SELECT * FROM tableName WHERE uid = id AND (status != "Deactive" OR Total != 0);
This bottom query is taking much much longer to complete. Why is this happening and is there a better way I can do this query?
The first query is looking up based on an index (I'm assuming by 'uid'). The second query is filtering on other values. Run this, and it will help you figure out how you can best optimize it:
EXPLAIN EXTENDED SELECT * FROM tableName WHERE uid = id AND status != "Deactive" OR Total != 0;
It's dirty, but this would probably be a quick way to speed it up:
SELECT
*
FROM
(
SELECT
*
FROM
tableName
WHERE
uid = id
) as temp
WHERE
temp.status != "Deactive"
OR temp.Total != 0;
This would force the query to just get the rows with a matching uid first, and then filter them down, instead of having to do a big table scan. Like I said before though, EXPLAIN EXTENDED is your friend
Do you need to return all the data in each row? ie picking only the columns you need will make the query run faster.
You also say you need to 'return the rows that do not have a status of "Deactive" and do not have a Total of 0'. Your query should then read:
SELECT * (or column names) FROM tableName WHERE uid = id AND status != "Deactive" AND Total != 0;
I'm trying to write a function to SELECT the least-recently fetched value from a table in my database. I do this by SELECTing a row and then immediately changing the last_used field.
Because this involves a SELECT and UPDATE, I'm trying to do this with locks. The locks are to ensure that concurrent executions of this query won't operate on the same row.
The query runs perfectly fine in phpMyAdmin, but fails in Magento. I get the following error:
SQLSTATE[HY000]: General error
Error occurs here:
#0 /var/www/virtual/magentodev.com/htdocs/lib/Varien/Db/Adapter/Pdo/Mysql.php(249): PDOStatement->fetch(2)
Here is my model's code, including the SQL query:
$write = Mage::getSingleton('core/resource')->getConnection('core_write');
$sql = "LOCK TABLES mytable AS mytable_write WRITE, mytable AS mytable_read READ;
SELECT #val := unique_field_to_grab FROM mytable AS mytable_read ORDER BY last_used ASC LIMIT 1;
UPDATE mytable AS mytable_write SET last_used = unix_timestamp() WHERE unique_field_to_grab = #val LIMIT 1;
UNLOCK TABLES;
SELECT #val AS val;";
$result = $write->raw_fetchrow($sql, 'val');
I've also tried using raw_query and query instead of raw_fetchrow with no luck.
Any thoughts on why this doesn't work? Or is there a better way to accomplish this?
EDIT: I'm starting to think this may be related to the PDO driver, which Magento is definitely using. I think phpMyAdmin is using mysqli, but I can't confirm that.
Probably a function that Magento uses doesn't support multiple sql statements.
Call each statement separately.
exec("LOCK TABLES mytable AS mytable_write WRITE, mytable AS mytable_read READ");
exec("SELECT #val := unique_field_to_grab FROM mytable AS mytable_read ORDER BY last_used ASC LIMIT 1");
exec("UPDATE mytable AS mytable_write SET last_used = unix_timestamp() WHERE unique_field_to_grab = #val LIMIT 1");
exec("UNLOCK TABLES");
exec("SELECT #val AS val");
Use appropriate functions instead of exec().