Is there a way to update all records in the table by a specific custom order? I specifically mean a situation, when the actual order comes from the 'outside' (eg as POST value).
For example, have a table
id | title | order_idx
----------------------
1 | lorem | 1
2 | ipsum | 2
3 | dolor | 3
I have a form that submits a hidden field, carrying ID values in this order: 2, 3, 1
I want to update the table to add incremental number to order_idx in each next row, going by the ID order served by the form field. So in this case, end result should look like this:
id | title | order_idx
----------------------
1 | lorem | 3
2 | ipsum | 1
3 | dolor | 2
Can this be done in a single UPDATE query somehow as opposed to running 3 queries (each including WHERE clause) in a php loop
You can use conditional expressions in assignment statements, like so:
UPDATE t
SET x = CASE
WHEN 2 THEN 1
WHEN 3 THEN 2
WHEN 1 THEN 3
ELSE x
WHERE ....
parameterized:
UPDATE t
SET x = CASE
WHEN ? THEN 1
WHEN ? THEN 2
WHEN ? THEN 3
ELSE x
WHERE ....
In either case, the query will most likely need constructed dynamically to account for a varying number of items to order.
Since your comment indicates potentially hundreds...
MySQL has a limit to query length (reference).
For a large number, I would start recommending a different approach.
Step 1) CREATE TEMPORARY TABLE `newOrder` (id INT, new_order_idx INT);
Step 2) INSERT id's and their new order into the temp table.
Step 3) UPDATE t INNER JOIN newOrder AS n ON t.id = n.id SET t.order_idx = n.new_order_idx WHERE ...
Step 4) DROP TEMPORARY TABLE newOrder;
The process itself is no longer a single query; but the UPDATE is.
Note: If you have a unique key involving order_idx I am entirely sure either of these would work. Occasions when I have needed to maintain uniqueness, the usual solution is to shift the records to be adjusted to a completely different range in one step, and then to their new positions in a second one. (Something like UPDATE t SET order_idx = -1 * order_idx WHERE ... would work as a pre-Step 3 range shift in the second part of this answer.)
Use ON DUPLICATE KEY UPDATE like below:
INSERT INTO table (id, order_idx) VALUES (1,3),(2,1),(3,2)
ON DUPLICATE KEY UPDATE order_idx=VALUES(order_idx);
So you can set order_idx dynamically for every row:
INSERT INTO table (id, order_idx)
VALUES (1,order_list[0]),(2,order_list[1]),(3,order_list[2])
ON DUPLICATE KEY UPDATE order_idx=VALUES(order_idx);
Related
What I mean by literal order is that, altough the IDs are auto-increment, through business logic, it might end up that 8 comes after 4 when 5 should've been there. That is to say, if a deletion if ID happens, there's no re-indexing
This is how my rows look (table name is wp_posts):
+-----+-------------+----+--+--+--+
| ID | post_author | .. | | | |
+-----+-------------+----+--+--+--+
| 4 | .. | | | | |
+-----+-------------+----+--+--+--+
| 8 | .. | | | | |
+-----+-------------+----+--+--+--+
| 124 | .. | | | | |
+-----+-------------+----+--+--+--+
| 672 | .. | | | | |
+-----+-------------+----+--+--+--+
| 673 | .. | | | | |
+-----+-------------+----+--+--+--+
| 674 | .. | | | | |
+-----+-------------+----+--+--+--+
ID is an int that has the auto-increment characteristic, but when a post is deleted, there is no re-assignment of IDs. It will just simply get deleted and because it's auto-increment, you can still assume that, vertically, the items that come after the one you're looking at are always bigger than the ones before.
I'm querying for ID: SELECT ID FROM wp_posts to get a list of all the IDs I need. Now, it just so happens that I need to batch all of this, using AJAX requests because once I retrieve the IDs, I need to operate on them.
Thing is, I don't really understand how to pass my data back to AJAX. What LIMIT does is, if I provide 2 arguments, such as: SELECT ID FROM wp_posts LIMIT 1,3, it'll return back 4,8,124 because it looks at row number. But what do I do on the next call? Yes, the first call always starts with 1, but once I need to launch the second AJAX request to perform yet another SELECT, how do I know where I should start? In my case, I'd want to start again at 4, so, my second query would be SELECT ID FROM wp_posts LIMIT 4, 7 and so on.
Do I really need to send that counter (even if I can automate it, since, you see, it's an increment of 3) back?
Is there no way for SQL to handle this automatically?
You have many confusions in your question. Let me try to clear up some basic ones.
First, the auto-incremented key is the primary key for the table. You do not need to worry about gaps. In fact, the key should basically be meaningless. It fulfills the following:
It is guaranteed to be unique.
It is guaranteed to be in insertion order.
Gaps are allowed and of no concern. There is no re-indexing. It is a bad idea because:
Primary keys uniquely identify each row and this mapping should be consistent across time.
Primary keys are used in other tables to refer to values, so re-indexing would either invalidate those relationships or require massive changes to many tables.
Re-indexes pre-supposes that the value means something, when it doesn't.
Second, a query such as:
SELECT ID
FROM wp_posts
LIMIT 1, 3;
Can return any three rows. Why? Because you have no specified an ORDER BY and SQL result sets without ORDER BY are unordered. There are no guarantees. So you should always be in the habit of using an ORDER BY.
Third, if you want to essentially "page" through results, then use the OFFSET feature in LIMIT (as you have above):
SELECT ID
FROM wp_posts
ORDER BY ID
LIMIT #offset, 3;
This will allow you to reset the #offset value and go to which rows you want.
First query:
SELECT ID FROM wp_posts ORDER BY ID LIMIT 3
This returns 4,8,124 as you said. In your client, save the largest ID value in a variable.
Subsequent queries:
SELECT ID FROM wp_posts WHERE ID > ? ORDER BY ID LIMIT 3
Send a parameter into this query using the greated ID value from the previous result. It's still in a variable.
This also helps make the query faster, because it doesn't have to skip all those initial rows every time. Paging through a large dataset using LIMIT/OFFSET is pretty inefficient. SQL has to actually read all those rows even though it's not going to return them.
But if you use WHERE ID > ? then SQL can efficiently start the scan in the right place, on the first row that would be included in the result.
Seems, you want to return the first three rows of your query ordered by currently existing ID values(whatever they're after all DML statement's applied on the table wp_posts).
Then, Consider using an auxiliary iteration variable #i to provide an ordered integer value set starting from 1 and increasing as 2,3,... without any gaps :
select t.*
from
(
select #i := #i + 1 as rownum, t1.*
from tab t1
join (select #i:=0) t2
) t
order by rownum
limit 0,3;
Demo
How are the rules in context of table row encountering when UPDATE with WHERE is performed on non-unique indexed column ?
I have a test table with col column as non-unique index:
id | col
----------
1 | 1
----------
2 | 2
----------
3 | 2
----------
22 | 3
UPDATE tab SET col=1 WHERE col=1;
// OR
UPDATE tab SET col=3 WHERE col=3;
// OR
UPDATE tab SET col=2 WHERE col=2;
// These updates encounter ONLY rows where col=1, col=3 or col=2
Same table and same updates, but with one more record in the table where col=2:
id | col
----------
1 | 1
----------
2 | 2
----------
3 | 2
----------
4 | 2
----------
22 | 3
UPDATE tab SET col=1 WHERE col=1;
// OR
UPDATE tab SET col=3 WHERE col=3;
// Both updates encounter ONLY rows where col=1 or col=3.
UPDATE tab SET col=2 WHERE col=2;
// This update encounters ALL the rows in the table even those where col IS NOT 2.
// WHY ?
In short, every row encountered in the processing of an UPDATE is exclusively row-locked. This means that the locking impact of an UPDATE depends on how the query is processed to read the rows to be updated. If your UPDATE query uses no index, or a bad index, it may lock many or all rows. (Note that the order in which rows are locked also depends on the index used.) In your case, since your table is very small and you're materially changing the distribution of the rows in your index, it is choosing to use a full table scan for the query in question.
You can test the performance and behavior of most UPDATE queries by converting them to a SELECT and using EXPLAIN SELECT on them (in newer versions you can even EXPLAIN UPDATE).
In short, though: You should have tables with a realistic distribution of data bInefore testing performance or locking behavior, not a very small table with a few test rows.
There is a wonderful article out there.I believe this would answer your queries.
http://www.mysqlperformanceblog.com/2012/11/23/full-table-scan-vs-full-index-scan-performance/
is it possible to always respect an expected number of element constraint by filling the remaining of a SQL dataset with previous written data, keeping the data insertion in order? Using MySQL?
Edit
In a web store, I always want to show n elements. I update the show elements every w seconds and I want to loop indefinitely.
By example, using table myTable:
+----+
| id |
+----+
| 1 |
| 2 |
| 3 |
| 4 |
| 5 |
+----+
Something like
SELECT id FROM myTable WHERE id > 3 ORDER BY id ALWAYS_RETURN_THIS_NUMBER_OF_ELEMENTS 5
would actually return (where ALWAYS_RETURN_THIS_NUMBER_OF_ELEMENTS doesn't exist)
+----+
| id |
+----+
| 4 |
| 5 |
| 4 |
| 5 |
| 4 |
+----+
This is a very strange need. Here is a method:
select id
from (SELECT id
FROM myTable
WHERE id > 3
ORDER BY id
LIMIT 5
) t cross join
(select 1 as n union all select 2 union all select 3 union all select 4 union all select 5
) n
order by n.n, id
limit 5;
You may need to extend the list of numbers in n to be sure you have enough rows for the final limit.
No, that's not what LIMIT does. The LIMIT clause is applied as the last step in the statement execution, after aggregation, after the HAVING clause, and after ordering.
I can't fathom a use case that would require the type of functionality you describe.
FOLLOWUP
The query that Gordon Linoff provided will return the specified result, as long as there is at least one row in myTable that satisfies the predicate. Otherwise, it will return zero rows.
Here's the EXPLAIN output for Gordon's query:
id select_type table type key rows Extra
-- ------------ ---------------- ----- ------- ---- -------------------------------
1 PRIMARY <derived2> ALL 5 Using temporary; Using filesort
1 PRIMARY <derived3> ALL 5 Using join buffer
3 DERIVED No tables used
4 UNION No tables used
5 UNION No tables used
6 UNION No tables used
7 UNION No tables used
UNION RESULT <union3,4,5,6,7> ALL
2 DERIVED myTable range PRIMARY 10 Using where; Using index
Here's the EXPLAIN output for the original query:
id select_type table type key rows Extra
-- ----------- ----------------- ----- ------- ---- -------------------------------
1 SIMPLE myTable range PRIMARY 10 Using where; Using index
It just seems like it would be a whole lot more efficient to reprocess the resultset from the original query, if that resultset contains fewer than five (and more than zero) rows. (When that number of rows goes from 5 to 1,000 or 150,000, it would be even stranger.)
The code to get multiple copies of rows from a resultset is quite simple: fetch the rows, and if the end of the result set is reached before you've fetched five (or N) rows, then just reset the row pointer back to the first row, so the next fetch will return the first row again. In PHP using mysqli, for example, you could use:
$result->data_seek(0);
Or, for those still using the deprecated mysql_ interface:
mysql_data_seek($result,0);
But if you're returning only five rows, it's likely you aren't even looping through the result at all, and you already stuffed all the rows into an array. Just loop back through the beginning of the array.
For MySQL interfaces that don't support a scrollable cursor, we'd just store the whole resultset and process it multiple times. With Perl DBI, using the fetchall_arrayref, with JDBC (which is going to store the whole result set in memory anyway without special settings on the connection), we'd store the resultset as an object.
Bottom line, squeezing this requirement (to produce a resultset of exactly five rows) back to the database server, and pulling back duplicate copies of a row and/or storing duplicate copies of a row in memory just seems like the wrong way to satisfy the use case. (If there's rationale for storing duplicate copies of a row in memory, then that can be achieved without pulling duplicate copies of rows back from the database.)
It's just very odd that you say you're using/implementing a "circular buffer", but that you choose not to "circle" back around to the beginning of a resultset which contains fewer than five rows, and instead need to have MySQL return you duplicate rows. Just very, very strange.
I have a table with unique index on two columns, id_parent and sort_order to be precise
+----+-----------+------------+-------------+-------------+-------------+
| id | id_parent | sort_order | some_data | other_data | more_data |
+----+-----------+------------+-------------+-------------+-------------+
| 1 | 1 | 1 | lorem ipsum | lorem ipsum | lorem ipsum |
| 2 | 1 | 2 | lorem ipsum | lorem ipsum | lorem ipsum |
| 3 | 1 | 3 | lorem ipsum | lorem ipsum | lorem ipsum |
+----+-----------+------------+-------------+-------------+-------------+
Now I want to update them, their data and their sort_order in one-go. sort_order would change from 1 - 2 - 3 to, for example 2 - 3 - 1.
But when I start running update statements, unique index block me, just as expected, saying that I can't have two rows with id_parent = 1 and sort_order = 2.
Well, I could set it 4 for now, update other rows in correct order, and then set this one.
But then, I would have to run an extra statement, and most probably add additional logic to my scripting language to determine correct order of updates.
I also use ORM, and it becomes even more inconvinient.
My question now, is there some method to make mysql temporarily ignore this index? Like starting a special transaction, in which indexes would be calculated only right before commiting it?
As far as I know that isn't possible.
The only time I've seen anything like that is that you can disable non unique keys on myisam tables. But not on InnoDB and not on unique keys.
However, to save you an update or two, there is no need to have the exact number 1, 2 and 3. You could as well have 4, 5 and 6. Right? You would use it in a order by and nothing else so the exact numbers aren't important. It will even save you an update if you're clever. From your example
update table set sort_order = 4 where sort_order = 1 and id = 1 and id_parent = 1;
New sort order is 2, 3, 1. And in just one update.
‘But when I start running update statements…’ – I understand, you tried updating the values using multiple UPDATE statement, like in a loop. Is that so? How about updating them in one go? Like this, for example:
UPDATE atable
SET sort_order = CASE sort_order WHEN 3 THEN 1 ELSE sort_order + 1 END
WHERE id_parent = 1
AND sort_order BETWEEN 1 AND 3
A single statement is atomic, so, by the time this update ends, the values of sort_order, although changed, remain unique.
I can't test this in MySQL, sorry, but it definitely works in SQL Server, and I believe the behaviour respects the standards.
MyISAM
For MyISAM tables, you can simply add this line at the start of your script:
SET UNIQUE_CHECKS=0;
It's common to use this in conjunction with:
SET FOREIGN_KEY_CHECKS=0;
The UNIQUE_CHECKS variable is mentioned in the docs here:
http://dev.mysql.com/doc/refman/5.0/en/converting-tables-to-innodb.html
InnoDB
There seem to be reports that the above commands don't work with the InnoDB engine. If so, then you can try dropping the UNIQUE index temporarily and then adding it back.
See: How to remove unique key from mysql table
Feel free to edit this post and improve it with a code example if you have one.
My problem is: I have a table with an auto_increment column. When I insert some values, all is right.
Insert first row : ID 1
Insert second row : ID 2
Now I want to insert a row at ID 10.
My problem is, that after this there are only rows inserted after ID 10 (which is the normal behaviour ).
But I want that the database first fills up ID 3-9 before making that.
Any suggestions?
EDIT:
To clarify: this is for an URL shortener I want to build for myself.
I convert the id to a word(a-zA-z0-9) for searching, and for saving in the database I convert it to a number which is the ID of the table.
The Problem is now:
I shorten the first link (without a name) -> ID is 1 and the automatically name is 1 converted to a-zA-Z0-9 which is a
Next the same happenes -> ID is 2 and the name is b, which is 2 converted.
Next interesting, somebody want to name the link test -> ID is 4597691 which is the converted test
Now if somebody adds another link with no name -> ID is 4597692 which would be tesu because the number is converted.
I want that new rows will be automatically inserted at the last gap that was made (here 3)
You could have another integer column for URL IDs.
Your process then might look like this:
If a default name is generated for a link, then you simply insert a new row, fill the URL ID column with the auto-increment value, then convert the result to the corresponding name.
If a custom name is specified for a URL, then, after inserting a row, the URL ID column would be filled with the number obtained from converting the chosen name to an integer.
And so on. When looking up for integer IDs, you would then use the URL ID column, not the table auto-increment column.
If I'm missing something, please let me know.
You could do 6 dummy inserts and delete/update them later as you need. The concept of the auto increment, by design, is meant to limit the application's or user's control over the number to ensure a unique value for every single record entered into the table.
ALTER TABLE MY_TABLE AUTO_INCREMENT = 3;
You would have to find first unused id, store it as user variable, use as id for insert.
SELECT #id := t1.id +1
FROM sometable t1 LEFT JOIN sometable t2
ON t2.id = t1.id +1 WHERE t2.id IS NULL LIMIT 1;
INSERT INTO sometable(id, col1, col2, ... ) VALUES(#id, 'aaa', 'bbb', ... );
You will have to run both queries for every insert if you still have gaps, its up to you to decide whether it is worth doing it.
not 100% sure what you're trying to achieve but something like this might work:
drop table if exists foo;
create table foo
(
id int unsigned not null auto_increment primary key,
row_id tinyint unsigned unique not null default 0
)
engine=innodb;
insert into foo (row_id) values (1),(2),(10),(3),(7),(5);
select * from foo order by row_id;
+----+--------+
| id | row_id |
+----+--------+
| 1 | 1 |
| 2 | 2 |
| 4 | 3 |
| 6 | 5 |
| 5 | 7 |
| 3 | 10 |
+----+--------+
6 rows in set (0.00 sec)