I'm using MySQL database to store student projects (every single project has its own unique ID, as the first screenshot shows).
The 'ID' column is set as auto_increment and PRIMARY KEY.
All projects must be ordered (only) by their ID ascending. But every time I insert a new project into my database and set lower value in the 'Year' field (lower value than I entered last time, when I was inserting my previous project), my projects become automatically ordered by this 'Year' field (as shown on the second screenshot).
Is there any way, how to set my table for automatic ordering all newly added projects only by the 'ID' column? Yeah, I know that I can change the ordering with ALTER TABLE tablename ORDER BY columnname ASC;after I place every new record, but can it be done automatically?
Thx to everyone who helps.
Q: Is there any way, how to set my table for automatic ordering all newly added projects only by the 'ID' column?
A: There is no "automatic ordering" in a MySQL table. (Some storage engines, such as InnoDB, are "index organized" structures, and do store rows in order by the cluster key.) But this organization does not define or specify the order of rows returned by a SELECT statement.
Without an ORDER BY clause on a SELECT statement, then the MySQL server can return rows in any order it chooses to. There is no guarantee of any "automatic" or "default" ordering.
When we run a SQL SELECT statement without an ORDER BY clause, we do observe that rows tend to be returned in a consistent order. This behavior isn't guaranteed, and it isn't "automatic".
This consistent "ordering" behavior we observe is due to the MySQL server performing a consistent set of operations, on a consistent set of data.
Performing an ALTER TABLE statement to rebuild the entire table is not the solution to the "automatic ordering" issue.
If you want the MySQL server to return rows in a specific order, then the solution is to add an ORDER BY clause to the SELECT statement.
The client that executes the SELECT statement is free to do whatever it wants with the rows it retrieves. The client can perform operations such as filtering, ordering, or whatever, to come up with what gets returned in the user interface.
Some clients (like the mysql command line) don't implement any functions for "filtering" or "ordering" rows, so the only way for the client to specify an order that rows should be returned in is the ORDER BY clause on statement itself. (The MySQL command line client returns rows in the user interface in the same order that they are retrieved.)
I expect that phpMyAdmin does the same thing, it displays the rows in the same order that they are returned from the MySQL server.
Order in query results should be determined by the ORDER clause. Don't rely on default order applied by phpmyadmin (or some other tool).
You have to distinct retrieving from inserting/updating.
About automatic order INTO phpmyadmin, maybe using bookmarks on queries : How can I set a default sort for tables in PHPMyAdmin (i.e. always "Primary key - Descending")
Related
In a ColdFusion application I have a query with several joins and I need one MAX number from one of the tables. I am getting all tangled up in 'only_full_group_by'. I cannot disable it in etc or in the Administrator, because I am on a shared system and have no access to those. I'd like to disable it at run time. Here's a simplified version of the query:
<cfquery name = 'test' datasource = 'whatever'>
select PersActExt,PersActPaid,PersActMdate,PersActbl,
PersTrId,PersTrMas,PersTrSub,PersTrCode,
MAX(PersTrPaid), MAX(PersTrDate)
from PersActiv
left join PersTrans on
PersTrId = PersActId and
PersTrMas = PersActMas and
PersTrSub = PersActSub and
PersTrCode = PersActCode
where PersActMas = 'bill'
group by PersTrId,PersTrMas,PersTrSub,PersTrCode
</cfquery>
The fact is the condition PersActMas = 'bill' reduces this to one record per Person in the PersAct table. I suppose I could put a MAX on all those Persact type fields, but there are other tables that also have to be joined in; and frankly I don't want a MAX on everything just to pick up one Max number in PersTr.
Is there a way to disable the ONLY_FULL_GROUP_BY mode within the query? Or better yet within the application? Or can anyone suggest another way around this problem. I know I can split this up into several queries and then stitch everything back together in ColdFusion, but that's a lot of code to get around this bad MySql mode.
Thinking laterally about the problem, could you restructure your query to join the PersAct table to a pre-aggregated subquery? This removes the need to perform a grouping on the outer query and running up against the limitation of your setup.
SELECT
PersActExt,
PersActPaid,
PersActMdate,
PersActbl,
PersTrId,
PersTrMas,
PersTrSub,
PersTrCode,
MaxPersTrPaid,
MaxPersTrDate
FROM
PersActiv
LEFT JOIN
(
SELECT
PersTrId,
PersTrMas,
PersTrSub,
PersTrCode,
MAX(PersTrPaid) AS MaxPersTrPaid,
MAX(PersTrDate) AS MaxPersTrDate
FROM
PersTrans
GROUP BY
PersTrId,
PersTrMas,
PersTrSub,
PersTrCode
) AS PersTr ON
PersTrId = PersActId and
PersTrMas = PersActMas and
PersTrSub = PersActSub and
PersTrCode = PersActCode
WHERE PersActMas = 'bill'
Edit:
For those wanting to know how to remove the 'ONLY_FULL_GROUP_BY' on an ad hoc basis, you need to add the below snippet to your query.
SET SESSION sql_mode = sys.list_drop(##session.sql_mode, 'ONLY_FULL_GROUP_BY');
This will only work if the DB user used in ColdFusion has the correct permissions and the option to allow multiple queries in a cfquery call is set.
Tested the query on MySQL 5.7 here: https://dbfiddle.uk/E1em1_k2
I tested with your query, with the last line as you wrote it:
...
group by PersTrId,PersTrMas,PersTrSub,PersTrCode;
This resulted in the error due to the ONLY_FULL_GROUP_BY:
Expression #1 of SELECT list is not in GROUP BY clause and contains nonaggregated column 'test.PersActiv.PersActExt' which is not functionally dependent on columns in GROUP BY clause; this is incompatible with sql_mode=only_full_group_by
Then I tested with an alternative last line:
...
group by PersActId,PersActMas,PersActSub,PersActCode;
This works with the ONLY_FULL_GROUP_BY at the default value. See the demo for proof. Note that I had to do my alternative query first, because the dbfiddle aborts when it gets an error.
The reason is that MySQL 5.7 introduced the capability to infer functional dependencies when using GROUP BY on the primary key.
https://dev.mysql.com/doc/refman/5.7/en/group-by-handling.html
SQL:1999 and later permits such nonaggregates per optional feature T301 if they are functionally dependent on GROUP BY columns: If such a relationship exists between name and custid, the query is legal. This would be the case, for example, were custid a primary key of customers.
MySQL 5.7.5 and later implements detection of functional dependence.
Since the four columns in the GROUP BY are in fact the primary key of the PersActive table, MySQL can tell that other columns of that table are guaranteed to have one value per grouping.
There are multiple rows in PersTrans, but the functional dependencies are still analyzed, so the columns that are known to be equal to primary key columns of PersActiv are still okay to use in the query without aggregation. The other columns of PersTrans that are not functionally dependent are already in MAX() expressions in your query, which also satisfies the requirement.
I conclude that functional dependency analysis works in MySQL 5.7, and this would be an effective workaround in the case you show.
You wrote:
I'm used to a system where if there is an aggregated field and some fields are not aggregated, the first or last value of that the non-aggregated field is automatically used.
In my experience, this includes only MySQL prior to 5.7, or SQLite. It's illegitimate in the ANSI/ISO SQL standard, and other SQL implementations conform to the standard.
MySQL 5.7 does not have a "bad sql mode" — it has been corrected to enforce the rule per the SQL standard, like most other brands of SQL also do.
Back to your original question:
Is there a way to disable the ONLY_FULL_GROUP_BY mode within the query? Or better yet within the application?
You can change the sql mode globally, either in the my.cnf options file, or using SET GLOBAL sql_mode='...';. I would do both, because if you change options dynamically with SET GLOBAL, they are reset to whatever is in the options file the next time you restart the MySQL Server.
You can also change the sql mode in a session with SET SESSION sql_mode='...'; and it takes effect for the remainder of the current session. But SET SESSION is a separate statement that you would have to execute in the session prior to your query. It can't be combined with a query.
There is an option in MySQL 8.0 to use the "hint" syntax to set certain options for the scope of a single SQL query. But in your case, you're using MySQL 5.7, so you'd have to upgrade to get this feature.
A similar feature exists in Percona Server 5.6 and later to set a variable for one statement. This does not apply if you use stock MySQL. You'd have to use the Percona branch of MySQL.
Regardless, I recommend you do not change the sql mode. It's proper that it behaves the way it does, because it prevents queries that may inadvertently produce wrong results.
I am working with an application which needs to function with any of 300+ different MySQL databases on the same server. The databases all have nearly identical table structures, with slight variations. For example, a particular column might be present in a table for only some of the databases.
I'm wondering if there is a way that, when performing an update on a table, I can update a specific column if it exists, but still successfully execute if the column does not exist.
For example, say I have a basic update statement like this:
UPDATE some_table
SET col1 = "some value",
col2 = "another value",
col3 = "a third value"
WHERE id = 567
What can I do to make it so that, if col3 doesn't actually exist when that query is run, the statement still executes and col1 and col2 are still updated with the new values?
I have tried using IF and CASE, but those seem to only allow changing the value based on some condition, not whether or not a column actually gets updated.
I know I can query the database for the existence of the column, then use a simple if condition in the application code use a different query. However, that requires me to query the database twice: once to see if the column exists, and again to actually update it. I'd prefer to do it with one SQL query if possible. I feel like that application code might start to get unwieldy with lots of extra code to check the existence of this-or-that column and conditionally build queries, instead of just having one query which works regardless of which database the application happens to be running against at the time.
To clarify, any given instance of the application is ever only running against one database; there is a different application instance for each database, but the instances will all be running the same code. These are legacy databases that legacy code is also relying on, so I don't want to modify the actual structures in the database to make them more consistent, for fear of breaking the legacy code.
No, the syntax of your SQL query, including all column identifiers you reference, must be fixed at the time it is parsed, before it validates that the columns exist.
A given UPDATE will either succeed fully or fail fully. There is no way to update some of the columns if the query fails to update all of them.
You have two choices:
Query INFORMATION_SCHEMA.COLUMNS first, to check what columns exist in the table for a given schema. Then format your UPDATE query, including clauses to set each column only if the column exists in that instance of the table.
Or...
Run several UPDATE statements, one for each column you want to update. Each statement will succeed or fail independently, but you can catch the error and continue on to the remaining statements. You can put all these statements in a transaction, so the set of changes is committed atomically, regardless of how many succeed (a single failed statement does not roll back a transaction).
Either way, it requires you to write more code. That's the unavoidable cost of supporting such variable table structure.
I can successfully delete records manually by click-selecting & deleting row(s) but executing delete queries result in 'Query Interrupted'.
My deletion queries are in the form:
DELETE FROM table where column = value;
The select statement uses the same values:
SELECT * FROM table WHERE column = value;
and returns desired results.
What could be causing the delete statement to fail? Are there limits on the amount of records you can delete at once in workbench?
If you wish to delete the entire contents of a table you can use Truncate.
TRUNCATE [TABLE] tbl_name
Please see the docs: https://dev.mysql.com/doc/refman/5.7/en/truncate-table.html
Using the DELETE function is usually used for deleting single rows.
According to the documentation, in the Preferences >> SQL Editor >> Other, the Safe Updates setting is on by default.
Safe Updates (rejects UPDATEs and DELETEs with no restrictions)
Enabled by default. Prevents UPDATE and DELETE queries that lack a corresponding key in a WHERE clause, or lack a LIMIT clause, from executing. This option requires a MySQL server reconnection.
When selected, this preference makes it possible to catch UPDATE and DELETE statements with keys that are not used properly and that can probably accidentally change or delete a large number of rows.
I think what this says is that if the setting is on, then the column you are filtering by in the DELETE or UPDATE statement must be the primary key, it cannot be just any column.
If you change the setting to off, then you might need to restart MySQL Workbench for the change to take effect (at least under Linux).
There is a default thousand-row limit in MySQL-Workbench. The SELECT query will return results but DELETE will fail if the number of records to be deleted exceeds one thousand. One option is to limit the results in the query itself or you can adjust the settings as stated in the documentation.
I have MySQL tables that are all InnoDB.
We have so many copies of various databases spread across multiple servers (trust me we're talking hundreds here), and many of them are not being queried at all.
How can I get a list of the MAX(LastAccessDate) for example for all tables within a specific database? Esp. considering that they are InnoDB tables.
I would prefer knowing even where the "select" query was run, but would settle for "insert/update" as well, since, if a db hasn't changed in a long time, it's probably dead.
If you have a table that always gets values inserted you can add a trigger to the update/insert. Inside this trigger you can set the current timestamp in a dedicated database, including the name of the database from which the insert took place.
This way the only requirement of your database is that it supports triggers.
Alternatively you could take a look this link:
odify date and create date for a table can be retrieved from sys.tables catalog view. When any structural changes are made the modify date is updated. It can be queried as follows:
USE [SqlAndMe]
GO
SELECT [TableName] = name,
create_date,
modify_date
FROM sys.tables
WHERE name = 'TransactionHistoryArchive'
GO
sys.tables only shows modify date for structural changes. If we need to check when was the tables last updated or accessed, we can use dynamic management view sys.dm_db_index_usage_stats. This DMV returns counts of different types of index operations and last time the operation was performed.
It can be used as follows:
USE [SqlAndMe]
GO
SELECT [TableName] = OBJECT_NAME(object_id),
last_user_update, last_user_seek, last_user_scan, last_user_lookup
FROM sys.dm_db_index_usage_stats
WHERE database_id = DB_ID('SqlAndMe')
AND OBJECT_NAME(object_id) = 'TransactionHistoryArchive'
GO
last_user_update – provides time of last user update
last_user_* – provides time of last scan/seek/lookup
It is important to note that sys.dm_db_index_usage_stats counters are reset when SQL Server service is restarted.
Hope This Helps!
What I try to accomplish seems simple,
Db type: MyISAM
Table Structure: card_id, status
Query: select an unused card_id from a table, and set the row as "used".
Is it race condition that when two queries running at the same time, and before status is updated, the same card_id is fetched twice?
I did some search already. It seems Lock table is a solution, but it's overkill to me and need Lock Privilege.
Any Idea?
Thanks!
It really depends on what statements you are running.
For plain old UPDATE statements against a MyISAM table, MySQL will obtain a lock on the entire table, so there is no "race" condition between two sessions there. One session will wait until the lock is released, and then proceed with it's own update (or will wait for a specified period, and abort with a "timeout".)
BUT, if what you are asking about is two sessions both running a SELECT against a table, to retrieve an identifier for a row to be updated, and both sessions retrieving the same row identifier, and then both sessions attempting to update the same row, then yes, that's a definite possibility, and one which really does have to be considered.
If that condition is not addressed, then it's basically going to be a matter of "last update wins", the second session will (potentially) overwrite the changes made by a previous update.
If that's an untenable situation for your application, then that does need to be addressed, either with a different design, or with some mechanism that prevents the second update from overwriting the update applied by the first update.
One approach, as you mentioned, is to avoid this situation by first obtaining an exclusive lock on the table (using a LOCK TABLES statement), then running a SELECT to obtain an identifier, and then running an UPDATE to update the identified row, and then finally, releasing the lock (using an UNLOCK TABLES statement.)
That's a workable approach for some low volume, low concurrency applications. But it does have some significant drawbacks. Of primary concern is reduced concurrency, due to the exclusive locks obtained on a single resource, which has the potential to cause a performance bottleneck.
Another alternative is an strategy called "optimistic locking". (As opposed to the previously described approach, which could be described as "pessimistic locking".)
For an "optimistic locking" strategy, an additional "counter" column is added to the table. Whenever an update is applied to a row in the table, the counter for that row is incremented by one.
To make use of this "counter" column, when a query retrieves a row that will (or might) be updated later, that query also retrieves the value of the counter column.
When an UPDATE is attempted, the statement also compares the current value of the "counter" column in the row with the previously retrieved value of the counter column. (We just include a predicate (e.g. in the WHERE clause) of the UPDATE statement. For example,
UPDATE mytable
SET counter = counter + 1
, col = :some_new_value
WHERE id = :previously_fetched_row_identifier
AND counter = :previously_fetched_row_counter
If some other session has applied an update to the row we are attempting to update (sometime between the time our session retrieved the row and before our session is attempting to do the update), then the value of the "counter" column on that row will have been changed.
The predicate on our UPDATE statement checks for that, and if the "counter" has been changed, that will cause our update to NOT be applied. We can then detect this condition (i.e. the affected rows count will be a 0 rather than a 1) and our session can take some appropriate action. ("Hey! Some other session updated a row we were intending to update!")
There are some good write-ups on how to implement an "optimistic locking" strategy.
Some ORM frameworks (e.g. Hibernate, JPA) provide support for this type of locking strategy.
Unfortunately, MySQL does NOT provide support for a RETURNING clause in an UPDATE statement, such as:
UPDATE ...
SET status = 'used'
WHERE status = 'unused'
AND ROWNUM = 1
RETURNING card_id INTO ...
Other RDBMS (e.g. Oracle) do provide that kind of functionality. With that feature of the UPDATE statement available, we can simply run the UPDATE statement to both 1) locate a row with status = 'unused', 2) change the value of status = 'used', and 3) return the card_id (or whatever columns we want) of the row the we just updated.
That gets around the problem of having to run a SELECT and then running a separate UPDATE, with the potential of some other session updating the row between our SELECT and our UPDATE.
But the RETURNING clause is not supported in MySQL. And I've not found any reliable way of emulating this type functionality from within MySQL.
This may work for you
I'm not entirely sure why I previously abandoned this approach using user variables (I mentioned above that I had played around with this. I think maybe I needed something more general, which would update more than one row and return a set of of id values. Or, maybe there was something that wasn't guaranteed about the behavior of user variables. (Then again, I only reference user variables in carefully constructed SELECT statements; I don't use user variables in DML; it may be because I don't have a guarantee of their behavior.)
Since you are interested in exactly ONE row, this sequence of three statements may work for you:
SELECT #id := NULL ;
UPDATE mytable
SET card_id = (#id := card_id)
, status = 'used'
WHERE status = 'unused'
LIMIT 1 ;
SELECT ROW_COUNT(), #id AS updated_card_id ;
It's IMPORTANT that these three statements run in the SAME database session (i.e. keep a hold of the database session; don't let go of it and get a new one.)
First, we initialize a user variable (#id) to a value which we won't confuse with a real card_id value from the table. (A SET #id := NULL statement would work as well, without returning a result, like the SELECT statement does.)
Next, we run the UPDATE statement to 1) find one row where status = 'unused', 2) change the value of the status column to 'used', and 3) set the value of the #id user variable to the card_id value of the row we changed. (We'd want that card_id column to be integer type, not character, to avoid any possible character set translation issues.)
Next, we run a query get the number of rows changed by the previous UPDATE statement, using the ROW_COUNT() function (we are going to need to verify that this is 1 on the client side), and retrieve the value of the #id user variable, which will be the card_id value from the row that was changed.
After I post this questions, I thought of a solution which is exactly the same as the one you mentioned at the end. I used update statement, which is "update TABLE set status ='used' where status = 'unused' limit 1", which returns the primary Id of the TABLE, and then I can use this primary ID to get cart_id. Says there are two update statements occurs at the same time, as you said, "MySQL will obtain a lock on the entire table, so there is no "race" condition between two sessions there", so this should solve my issue. But I am not sure why you said, "MySQL does NOT provide support an style statement".