I have a table of items that need to be displayed in a certain order, but that order can be changed. Items can be added at the beginning, end, or in the middle, and items can be rearranged. How can I set up the table to keep track of that order in such a way that it's easy to modify but the list can also be fetched in order with a single query?
For example, I could have a "NEXT_ID" column to do it linked list-style, but then how would I run a SELECT query to get the rows in order of the NEXT_ID chain?
Apologies in advance for the super-obvious solution I'm probably missing.
I have this problem often, and I solved it with a simple solution : an extra column called Sort Order (or DisplayOrder, whatever floats your boat really) . This allows me the flexibility to use auto-generated, auto-incremented ID column and have a special pre-defined sort.
In my case, I need them to come out of the database with an alphabetical order except that some items like "Other" and "N/A" are always last.
ProdID ProdText SortOrder
2 "Anchovies" 1
3 "Rivet" 2
4 "N/A" 4
5 "Other" 3
SELECT ProdID, ProdText ORDER BY Sort Order
Create a column in the table that represents the sort order. Put an index on this column so that the MySQL engine can retrieve based on this column quickly. When you change the order, update the values in this field for all records to keep it consistent.
For example, when you insert a new record in the middle:
UPDATE table SET sort_order = sort_order + 1 WHERE sort_order >= 5;
INSERT INTO table (sort_order, column1, column2) VALUES (5, 'value1', 'value2');
Something more complicated, like moving #3 down to #6 and sliding all others up:
UPDATE table
SET sort_order = Case sort_order When 3 Then 6 Else sort_order - 1 End
WHERE sort_order BETWEEN 3 AND 6;
If you want to avoid the sort order, you can try a "parent" column and consider the linked list to be a special case of a tree like structure. This article may help.
http://mikehillyer.com/articles/managing-hierarchical-data-in-mysql/
There are other articles out there:
http://ferdychristant.com/blog//archive/DOMM-7QJPM7
Keep in mind that selecting a long linked list may degrade performance though.
Unless I am misunderstanding what you are looking for it would seem you could just add a DISPLAY_ORDER column that is a numeric index of how things should be returned. This can easily be changed and rearranged. Plus value can be used in order by.
easiest solution: use a column named "display_order" in which you set 1,2,3 and so on. The query would be sorted with "ORDER BY display_order".
To edit this (if youre in a website related environment) use javascript with + and - buttons for example. everytime you do + the number increments and if something with the number exists itll get decreased so they switch positions.
Related
I would like to show a table of items, where user can add, delete and reorder the items. The first way I thought to store these items in MySQL DB was to have an order column in the table, but it seems pretty inefficient because when user deletes an item, I need to update the order of all the following items. And if I just use ORDER BY with id or something, I don't know how to let user reorder the items as they want. What do you think is the best way to handle this?
The stack I use: React JS -> Node JS / Express -> MySQL
A way I have seen is to have a sperate column which states the ID of the item it is behind. Then in your Query you can order from that column when selecting to make it display in that order.
This means that if you delete a field, you only have to update the one behind it to then state its behind the one ahead of what you just deleted (That was a jumple of words...)
Does this help?
You'll definitely need a new column to do this ordering. It requires the storage of more information than you can represent with just the item ID, name, and all that.
I have done this successfully with the kind of order column you mention. Here are some tricks I used.
Don't worry about what happens when you delete a row. Things work just fine if you have gaps in the values of your order column.
Use the same data type for your order column as your autoincrementing id column (BIGINT UNSIGNED probably) but allow NULL values.
Use ORDER BY COALESCE(order, id * 1000), id so you'll use the id value for ordering newly inserted rows that have never had their order set. New rows come at the end. Intutitive enough default behavior.
To put item with id a right after the item with id b, update the order value in row a to COALESCE(b.order, id * 1000) + 1.
Like this
id name order COALESCE(order, id*1000)
1 orange NULL 1000
2 banana NULL 2000
3 apple NULL 3000
4 fig NULL 4000
This makes your order values be 1000, 2000, 3000, 4000 and so forth. To order the fourth item right after the second item you'll set its order to 2001. Like this
id name order COALESCE(order, id*1000)
1 orange NULL 1000
2 banana NULL 2000
4 fig 2001 2001
3 apple NULL 3000
If you do enough reordering that you insert more than 1000 items after any given item, you'll have to clean up your order column by rewriting it for the whole table. It's OK if you have some identical order values. The ORDER BY clause I suggested will keep them in their original ID order.
I have a REST service which return rows from a database table depending on the current page and results per page.
When not filtering the results, it's pretty easy to do, I just do a SELECT WHERE id >= (page - 1) * perPage + 1 and LIMIT to perPage.
The problem is when trying to use pagination on filtered results, e.g. if I choose to filter only the rows WHERE type = someType.
In that case, the first match of the first page can start in id 7, and the last can be in id 5046. Then the first match of the second page can start at 7302 and end at 12430, and so on.
For the first page of filtered results, I'd be able to simply start from id 1 and LIMIT to perPage, but for the second page, etc, I need to know the index of the last matched row in the previous page, or even better - the first matched row in the current page, or some other indication.
How do I do it efficiently? I need to be able to do it on tables with millions of rows, so obviously fetching all the rows and taking it from there is not an option.
The idea is something like this:
SELECT ... FROM ... WHERE filterKey = filterValue AND id >= id_of_first_match_in_current_page
with id_of_first_match_in_current_page being the mystery.
You can't know what the first id on a given page is, because id numbers are not necessarily sequential. In other words, there could be gaps in the sequence, so rows on the fifth page of 100 rows doesn't necessarily start at id 500. It could start on id 527 for example, It's impossible to know.
Stated yet another way: id is a value, not a row number.
One possible solution if your client is advancing through pages in ascending order is that each REST request fetches data, notes the greatest id value on that page, then uses that in the next REST request so it queries id values that are larger.
SELECT ... FROM ... WHERE filterKey = filterValue
AND id > id_of_last_match_of_previous_page
But if your REST request can fetch any random page, this solution doesn't work. It depends on having fetched the prior page already.
Another solution is to use the LIMIT <x> OFFSET <y> syntax. This allows you to request any arbitrary page. LIMIT <y>, <x> works the same, but for some reason x and y are reversed in the two different syntax forms, so keep that in mind.
Using LIMIT...OFFSET isn't very efficient when you request a page that is many pages into the result. Say you request the 5,000th page. MySQL has to generate a result on the server-side of 5,000 pages, then discard 4,999 of them and return the last page in the result. Sorry, but that's how it works.
Re your comment:
You must understand that WHERE applies conditions on values in rows, but pages are defined by the position of rows. These are two different ways of determining rows!
If you have a column that is guaranteed to be a row-number, then you can use that value like a row position. You can even put an index on it, or use it as the primary key.
But primary key values may change, and may not be consecutive, for example if you update or delete rows, or rollback some transactions, and so on. Renumbering primary key values is a bad idea because other tables or external data may reference primary key values.
So you could add another column that is not the primary key, but only a row-number.
ALTER TABLE MyTable ADD COLUMN row_number BIGINT UNSIGNED, ADD KEY (row_number);
Then fill the values when you need to renumber the rows.
SET #row := 0;
UPDATE MyTable SET row_number = (#row := #row + 1) ORDER BY id;
You'd have to re-number the rows if you ever delete some, for example. It's not efficient to do this frequently, depending on the size of the table.
Also, new inserts cannot create correct row number values without locking the table. This is necessary to prevent race conditions.
If you have a guarantee that row_number is a sequence of consecutive values, then it's both a value and a row position, so you can use it for high-performance index lookups for any arbitrary page of rows.
SELECT * FROM MyTable WHERE row_number BETWEEN 401 AND 500;
At least until the next time the sequence of row numbers is put into doubt by a delete or by new inserts.
You're using the ID column for the wrong purpose. ID is the identifier of a record, not the sequence number of a record for any given set of results.
The LIMIT keyword extends to basic pagination. If you just wanted the first 10 records, you'd do something like:
LIMIT 10
To paginate, if you wanted the second 10 records, you'd do:
LIMIT 10,10
The 10 after that:
LIMIT 20,10
And so on.
The LIMIT clause is independent of the WHERE clause. Use WHERE to filter your results, use LIMIT to paginate them.
I have two columns named priority and state:
The column priority will only contain field values: urgent and normal.
The column state will only contain field values: wait, executed and done.
I'm trying to sort these using my dataset CommandText property, so the dbgrid connected to the dataset displays the data in the sort order I've set there.
The sort order should be like this:
Rows containing urgent in the priority column should start the DBGrid list.
Then the list should continue with the ones marked as normal in the priority column,
followed by the ones marked as wait in the state column,
followed by the ones marked as executed in the state column,
and finally the list ends with the ones marked as done in the state column.
I can't figure out how to write the SQL statement for this, so I can use it in the CommantText property for the dataset.
I am using this currently:
SELECT *
FROM table_name
ORDER BY FIELD(column_name, "normal", "urgent") DESC
This works with the first column priority but doesn't take into account the second column state.
You are very close. You just need a second key in the order by:
SELECT *
FROM table_name
ORDER BY FIELD(priority, 'urgent', 'normal'),
FIELD(state, 'wait', 'executed', 'done')
Notes:
I removed the DESC from the first key. You are using field(), so put things in the right order.
This will order the "urgent" by the same three keys. This seems consistent with your question.
I've searched and searched and can't find an answer to this question, I'm probably asking it in the wrong way.
I am querying an employee database.
I need to get details based on a position id, however there could be multiple records for that position id as this organisation has permanent employees and temporary employees that act against the same position.
So, in order to get the CURRENT occupant of the position id, I need my query to SELECT the FIRST record that matches the position string, from the TOP DOWN.
will this select the first matched record from the top?
SELECT * WHERE `position_id`="00000000" LIMIT 1;
Thanks in advance.
You need an ORDER BY clause to define the ordering between the individual records your table. If you do not use ORDER BY you can assume no fixed order between the records, and you could get a new order each time you executed the query.
From the manual:
With one argument, the value specifies the number of rows to return from the beginning of the result set
So with LIMIT 1 you get the first row from the result set. What the result set is depends on engine used and which indexes you have. If you want the first row added, you need to create another column to define that.
It just gets one at random*. There's no way to tell which one it will be, unless you add an ORDER BY clause.
* Not really at random, of course. It depends on the way the records are stored and repeated queries will probably return the same result every time, at least as long as you don't modify the table or its contents. I actually mean, you cannot be sure.
What is the best way to assign orders to mysql table rows? If I have rows 1 to 5 and I want to remove 4, how can I make 5 the new 4? If I make make 3 the new 1, I need to add 1 to each of the other rows. There could be 1 or 2 rows or a hundred.
Is there a simpler way than manually programming each contingency?
Thanks in advance.
additional:
I have an interface where I add packages for customers to see. They are automatically ordered ascending by id. I can reorder them by price, package name, or whatever, but I want to arbitrarily order them by my own preference from time to time.
Thanks again.
Assuming you have a unique order_column column in your database:
To add a new row at position x:
Lock tables
update all rows where position >= x and add 1
Then insert the new row at position x
Unlock tables
To swap positions x and y:
UPDATE table SET x=(#temp:=x), x = y, y = #temp;
(source)
To remove a row at position x:
Lock tables
Remove row at position x
update all rows where position > x and subtract 1
Unlock tables
To display data:
Just ORDER BY by the order_column column.
Basic approaches to ordering usually amount to ordering on a specific column either alphabetically or numerically. Alternatively if the the order that you need can not be created based on the data you have then you need to add a column exculisevly for ordering purposes, then create an interface to populate that with data. Sounds like you need to do the later.
Unless I completely misunderstood the question:
Leave the table alone and just use ORDER BY to determine sorting when SELECTing rows.