I have a view which queries from 2 tables that don't change often (they are updated once or twice a day) and have a maximum of 2000 and 1000 rows).
Which algorithm should perform better, MERGE or TEMPTABLE?
Wondering, will MySQL cache the query result, making TEMPTABLE the best choice in my case?
Reading https://dev.mysql.com/doc/refman/5.7/en/view-algorithms.html I understood that basically, the MERGE algorithm will inject the view code in the query that is calling it, then run. The TEMPTABLE algorithm will make the view run first, store its result into a temporary table then used. But no mention to cache.
I know I have the option to implement Materialized Views myself (http://www.fromdual.com/mysql-materialized-views). Can MySQL automatically cache the TEMPTABLE result and use it instead?
Generally speaking the MERGE algorithm is preferred as it allows your view to utilize table indexes, and doesn't introduce a delay in creating temporary tables (as TEMPTABLE does).
In fact this is what the MySQL Optimizer does by default - when a view's algorithm UNDEFINED (as it is by default) MySQL will use MERGE if it can, otherwise it'll use TEMPTABLE.
One thing to note (which has caused me a lot of pain) is that MySQL will not use the MERGE algorithm if your view contains any of the following constructs:
Constructs that prevent merging are the same for derived tables and view references:
Aggregate functions (SUM(), MIN(), MAX(), COUNT(), and so forth)
DISTINCT
GROUP BY
HAVING
LIMIT
UNION or UNION ALL
Subqueries in the select list
Assignments to user variables
Refererences only to literal values (in this case, there is no underlying table)
In this case, TEMPTABLE will be used, which can cause performance issues without any clear reason why. In this case it's best to use a stored procedure, or subquery instead of a view
Thank's MySQL ðŸ˜
Which algorithm? It depends on the particular query and schema. Usually the Optimizer picks the better approach, and you should not specify.
But... Sometimes the Optimizer picks really bad approach. At that point, the only real solution is not to use Views. That is, some Views cannot be optimized as well as the equivalent SELECT.
If you want to discuss a particular case, please provide the SHOW CREATE VIEW and SHOW CREATE TABLEs, plus a SELECT calling the view. And construct the equivalent SELECT. Also include EXPLAIN for both SELECTs.
Related
I have a view (say 'v') that is the combination of 10 tables using several Joins and complex calculations. In that view, there are around 10 Thousand rows.
And then I select 1 row based on row as WHERE id = 23456.
Another possible way to use a larger query in which I can cut short the dataset to 1% before the complex calculation starts.
Question: Are SQL views optimized in some form?
MySQL Views are just syntactic sugar. There is not special optimization. Think of views as being textually merged; then optimized. That is, you could get the same optimizations (or not) by manually writing the equivalent SELECT.
If you would like to discuss the particular query further, please provide SHOW CREATE TABLE/VIEW and EXPLAIN SELECT .... It may be that you are missing a useful 'composite' index.
I am renaming multiple tables in a large application. I need to preserve the old table name because some parts of the application will take longer to be updated, we can have no downtime.
My idea is to create a view that selects all from the new table, like this:
create view old_table_name as select a as x, b as y, c as z from new_table_name;
According to this article (http://dev.mysql.com/doc/refman/5.7/en/view-updatability.html) I will be able to make inserts and updates and deletes with this view.
My question is (considering that this is only a temporary solution in the mean time until we are able to migrate all legacy code to use this new table) will I be able to pull this off?
Will I have a decent enough performance in joins and things alike?
Will I be able to make complex updates or deletes (involving joins) with this approach?
Is there a better way to approach this problem?
Thanks in advance for your help.
The performance should be essentially identical.
For simple views without aggregate functions/group by/having, distinct, limit, unions, scalar subqueries, and views that return literals only, MySQL uses the MERGE algorithm by default, which effectively rewites a query referencing such a view as if you had used the columns in the base tables directly.
See View Algorithms in the documentation.
Determining what algorithm MySQL view is using may be informative as well.
I would like to know what is the best practice for better performance when you have a select query that can have any combination of a number (20+) of parameters in the where clause that is passed to a stored procedure.
let's say I have a query that should return the list of people and their addresses (maybe more than 1 address per person). The user wants to search by any possible combination of fields from the person/address tables. The search could be on one field or all 20 or anything in between.
The way I use to handle this is by creating one cursor like this
(for simplicity I am listing 2 variables only a varchar and an int)
create procedure dynasp for (
in in_name varchar(40),
in in_age int
..... rest of parameters here...
declare cursor cs for
select .... from person join address....
where
(in_age = 0 or in_age = person_age) and
(in_name is null or rtrim(in_name)='' or in_name = person.name)
and...
I believe since the value of an input variable is constant, the query should not evaluate it on each row or does it?
The other option that I use is using dynamic cursor built from string in the sp. this way it will contain only the fields that are not empty in the where clause, but I believe this means that the sql needs to be constructed and recompiled on every call to the SP.
My question is for best practices which method above is more recommended, and is there any other better way than the 2 methods mentioned above?
Thank you
The question of performance basically hinges on one simple thing: does your table have any indexes that you intend to use to improve the performance of the query?
If indexes aren't an issue, then your approach is fine. Well, let me add: assuming a cursor is necessary for the additional processing that you are doing. If you can just return the result set and do set-based processing, that is superior to using cursors.
If indexes are an issue, then a long complex where statement with a bunch of constant expressions might confuse the MySQL compiler. The documentation on using indexes for where clauses is here. MySQL definitely removes constant expressions. However, in a very complex expression, I'm not sure how well this interacts with choosing the right index. (I am assuming you are using MySQL based on the syntax).
For this latter case, a dynamic cursor would be beneficial, because it would encourage MySQL to choose an execution plan that uses indexes.
So, if you are not using indexing (or partitioning), then your current approach is fine. If you are, look at the execution plan for your queries. If they use the appropriate indexes, then your current approach is fine. If they are not, consider dynamic cursors.
It depends on the size of the tables you're searching. If you have 20+ criteria in the where clause, all containing or operators, the query optimizer will not be able to choose a good index to use and will likely just scan the entire table(s). For small tables, this won't matter but for very large tables, it will be slow.
The other alternative, constructing a dynamic query, will occur some overhead in parsing and choosing a query plan, but when executed, the query will likely be more efficient. (Make sure you're protecting against SQL injection vulnerabilities).
So the best practice is to benchmark both and see what's best in your situation.
Basically, I have multiple queries like this:
SELECT a, b, c FROM (LONG QUERY) X WHERE ...
The thing is, I am using this LONG QUERY really frequently. I am looking to give this subquery an alias which would primarily:
Shorten & simplify queries (also reduce errors and code duplication)
Possibly optimize performance. (I believe this is done by default by mysql query caching)
Till now, I have been doing it this way to store:
variable = LONG QUERY;
Query("SELECT a, b, c FROM ("+variable+") X WHERE ...");
Which is not bad. I am looking for a way to do this with mysql internally.
Is it possible to create a simple, read-only view that would generate NO overhead, so I could do everywhere? I believe this is more propper & readable way of doing it.
SELECT a, b, c FROM myTable WHERE ...
Typically these are called views. For example:
CREATE VIEW vMyLongQuery
AS
SELECT a, b, c FROM (LONG QUERY) X WHERE ...
Which can then be referenced like this:
SELECT a, b, c FROM vMyLongQuery
See http://dev.mysql.com/doc/refman/5.0/en/create-view.html for more info on syntax.
As far as performance goes, best case performance will be near enough exactly the same as what you're doing now and worst case it will kill your app. It depends on what you do with the views and how MySQL processes them.
MySQL implements views two ways merge and temptable. The merge option is pretty much exactly what you're doing now, your view is merged into your query as a subquery. With a temptable it will actually spool all the data into a temptable and then select/join to that temptable. You also lose index benefits when data is joined to the temptable.
As a heads up, the merge query plan doesn't support any of the following in your view.
Aggregate functions (SUM(), MIN(), MAX(), COUNT(), and so forth)
DISTINCT
GROUP BY
HAVING
LIMIT
UNION or UNION ALL
Subquery in the select list
Reference to literals without an underlying table
So if your subquery uses these, you're likely going to hurt performance.
Also, heed OMG Ponies' advice carefully, a view is NOT the same as a base class. Views have their place in a DB but can easily be misused. When an engineer comes to the database from an OO background, views seem like a convenient way to promote inheritance and reusability of code. Often people eventually find themselves in a position where they have nested views joined to nested views of nested views. SQL processes nested views by essentially taking the definition of each individual view and expanding that into a beast of a query that will make your DBA cry.
Also, you followed excellent practice in your example and I encourage you to continue this. You specified all your columns individually, never ever use SELECT * to specify the results of your view. It will, eventually, ruin your day.
Im not sure if this is what you are looking for but you can use stored procedures to recall mysql queries. Im not sure if you can use it inside another query though?
http://www.mysqltutorial.org/getting-started-with-mysql-stored-procedures.aspx
Suppose I have a table A, creating a view V from that table.
Then I do several queries from V. I wonder if V will be re-constructed each time I query? or it will be constructed only 1 time, and being saved somewhere in memory by DBMS for next queries (which I think similar to query from a table)?
In general, no. V is a transient set of rows that is computed when requested by a query. Because you can apply additional WHERE and ORDER BY criteria when querying from a view, the execution plan for two queries against the same view could conceivably be quite different. The database generally cannot reuse the results of a previous query against a view to satisfy the next query against that view.
That said, there is a relatively new technology in some engines called Materialized Views. I have never used them myself, but my understanding is that these views are pre-computed based on updates that are made to the underlying tables. So with Materialize Views you do get improved SELECT performance, but at the expense of decrease INSERT, UPDATE, and DELETE performance.
You should also be aware that multi-column indexes can be used to precompute certain selections and sort orders involving individual tables. If you issue a query against a table that can be satisfied using a compound index (only the columns in the index are required by the query, and the sort order matches the index) then the table itself need never be read, only the index.
Views in MySQL are not a de facto caching solution.
MySQL runs the query against the base tables every time you query a view on those base tables. The results of the query are not stored for the view.
As a result, there is no need to "refresh" the view as there is when using materialized views in Oracle Microsoft SQL Server. Even the SQL in a MySQL view definition is re-evaluated every time you query the view.
If you need something like materialized views in MySQL, one tool that might help is FlexViews. This stores the results of a query in an ordinary base table, and then monitors changes recorded in MySQL's binary log, applying relevant changes to the base table. This tool can be quite useful, but it has some caveats:
FlexViews is written in PHP, and as such it has some performance limitations. Depending on your write traffic load, FlexViews may not be able to keep up.
It doesn't support every possible type of SELECT query.
FlexViews-managed materialized view tables are not updateable. That is, you can UPDATE this view table, but the change will not apply to the base tables.
According to Pinal Dave, a view must be refreshed in order to reflect changes made to its referenced table(s). I'm not sure this makes a view of a simple 1-table query any more efficient than querying the table directly (it probably doesn't) but I think it means that views containing complex joins and subqueries may be more efficient than their non-view counterparts.
Pinal Dave has more to say about the other limitations of SQL views (or features, if you like). Maybe you can learn something useful there.
Mysql Views do not support Indexes. (as like in Oracle, where you can create index in Oracle Views) But mysql views can use the indexes in underlying table when created with Merge Algorithm.
If you have to use views, then adjust your JOIN BUFFER.
Using, Something like this
set global join_buffer_size=314572800;
Do profile the differences before and after changing the buffer size.
I have seen after increasing join buffers, the view query executes in same time (in ms) as the table of the same size will do.