I have some questions on views -
Where views are created/stored in Mysql? Or they are only virtual and deleted after some time period?
When the data of views is refresh? (It refresh automatically when we insert data in actual table or we have to update view each time?)
Use of views is good or we should fire the queries each time?
Views are pure metadata. MySQL doesn't copy any data during the creating of a view, and it's also it is not deleted after a time.
When you run a select on a view, mysql (or any other database) runs the query defined at creation time.
There's no performance difference (or almos not different) between running a query on a table or on a view.
Some databases, such as oracle, support something called materialised views. These views, do copy the data, so they have to be refreshed, so the data doesn't become stale.
Leaving this as it turned up in Google results.
To see view definitions in MySQL you can use this query:
SELECT * FROM information_schema.VIEWS;
Regards,
James
Related
I have a SQL Server 2008 DB that has a set of views that are accessed by a program. Our goal is to optimize the access speed of the program (which pulls in data on a user request), to minimize end-user impact.
Right now we are writing all of our views to tables, and passing those mappings to the application (we found the application performed better reading from tables as opposed to views). We are soon going to implement indexes (still need to discuss with the application vendor what indexes will speed up their import), but for now we're trying to figure out the best way to optimize the import.
The plan currently is to write the views to tables, add the proper indexes and then run a (select *) statement to force them into memory. My question is whether A) writing them to tables is necessary once we have the indices and the select * and B) what are some methods that we are missing?
Edited to clarify question goal.
OK I think I follow
Select into implies you are dropping the table and let select into create
You are probably better off with a truncate
If it is a FK then you need to delete but they tend to be smaller
And then just do an insert into
This way you also don't need to drop and recreate views
If you can take the hit you are better off taking a tablock hit the whole table
If the linked is slow then insert into some local staging tables
From the staging table load the production table
I totally don't get why you would materialize a view into table.
If you have performance issue with a view then first optimize the view.
What is going on in the view that is slow?
At my work my colleagues always build report cronjobs for heavy tables. With the cronjob we get all data from 1 day per user and insert the totals in a report table. The report overview page is not correct because it has a delay for at most 1 hour.
The cronjob runs 24 times a day (every hour).
Is it better to use a MySQL view? When a record has been added to the master table the MySQL view will updated, right? This is a very though action. Will that affect the users using the dashboard?
Kind regards,
Joost
Okay so some terminology first.
The cron jobs are most likely appending data to existing tables (perhaps using an upsert method like INSERT ... ON DUPLICATE KEY UPDATE). These data you are writing to the existing tables may be indexed, just like normal MySQL tables, and they are also persistent on disk
Views, on the other hand, are really nothing more than saved queries in MySQL. Every time you open a view, you run the query again. Views aren't really useful for performance optimization as much as they are useful for small, efficient queries that otherwise might be a pain to remember. Views cannot have indices (although they are effectively saved queries, so the query itself can make use of the indices on the tables it's referencing) and they are not persistent to disk. Every time you load the view, you will be running the query that makes up the view again
Now, in between views and tables populated by Cron jobs, you also could install a plugin for MySQL called Flexviews (https://github.com/greenlion/swanhart-tools). Flexviews allows MySQL to use what are called materialized views (eg http://en.wikipedia.org/wiki/Materialized_view). Materialized views are basically views that are persisted to disk as tables. And, since they are tables, they can also use indices.
Materialized views are not native to MySQL, but the developer who maintains that plugin is well known in the MySQL community, and he tends to write good, reliable SQL tools . Obviously it would be a mistake to test the plugin in a production environment, or without using backups. But there are plenty of folks who use Flexviews in production to accomplish exactly what it seems like you'd like to do... obtain near real time updates of dashboard/summary tables in a way that doesn't murder DB performance.
I'd definitely check Flexviews out... you can learn more about it
here: http://www.percona.com/blog/2011/03/23/using-flexviews-part-one-introduction-to-materialized-views/
and here: http://www.percona.com/blog/2011/03/25/using-flexviews-part-two-change-data-capture/
I got multiple tables where I have to join, subquery,pagination, grouping, ordering . Keeping hibernate limitation in mind, sometime native SQL is required and during this time hibernate cache is helpless. Also the data is stored in hibernate second level cache is not automatic, since its stored only when DB is accessed. So first time second level cache is empty.
My problem is I used native sql to get data with multiple joins and grouping,ordering, finally ending up in the performance issue.
My thoughts: I like sql VIEW to pull data with all those joins ,ordering , grouping. But the sql VIEW is like a normal select statement and executes every time on access. Is there any live result set as table where I can just say fetch data as select * from ONE_LIVE_RESULT_SET where condition.
Is there any concept like LIVE_RESULT_SET IN sql world? Any comments.
Use a materialized view
Extract from Wikipedia: http://en.wikipedia.org/wiki/Materialized_view
A materialized view is a database object that contains the results of
a query. For example, it may be a local copy of data located remotely,
or may be a subset of the rows and/or columns of a table or join
result, or may be a summary based on aggregations of a table's data.
Materialized views, which store data based on remote tables, are also
known as snapshots. A snapshot can be redefined as a materialized
view.
Example syntax to create a materialized view in Oracle:
CREATE MATERIALIZED VIEW MV_MY_VIEW REFRESH FAST START WITH SYSDATE
NEXT SYSDATE + 1
AS SELECT * FROM ;
Regards
But this MATERIALIZED VIEW is not a live data(sync up with table) but inorder to make it live data it has to be REFRESH. Here the question will be When to REFRESH OR during such refresh again one has to wait. Also frequent data changing is another use case to suffer. Is there any ways where the refresh can be done for specific row?
Any hibernate experts!!! Does HIBERNATE persist the data on multiple joins, complex joins?
I have seen hibernate persisting second level cache session.get(id), but I am not sure about the hql or native sql having multiple/complex join. Is it possible to get from hibernate second level cache for multiple/comples joins ?
I have a mysql query that is taking 8 seconds to execute/fetch (in workbench).
I won't go into the details of why it may be slow (I think GROUPBY isnt helping though).
What I really want to know is, how I can basically cache it to work more quickly because the tables only change like 5-10 times/hr, while users access the site 1000s times/hour.
Is there a way to just have the results regenerated/cached when the db changes so results are not constantly regenerated?
I'm quite new to sql so any basic thought may go a long way.
I am not familiar with such a caching facility in MySQL. There are alternatives.
One mechanism would be to use application level caching. The application would store the previous result and use that if possible. Note this wouldn't really work well for multiple users.
What you might want to do is store the report in a separate table. Then you can run that every five minutes or so. This would be a simple mechanism using a job scheduler to run the job.
A variation on this would be to have a stored procedure that first checks if the data has changed. If the underlying data has changed, then the stored procedure would regenerate the report table. When the stored procedure is done, the report table would be up-to-date.
An alternative would be to use triggers, whenever the underlying data changes. The trigger could run the query, storing the results in a table (as above). Alternatively, the trigger could just update the rows in the report that would have changed (harder, because it involves understanding the business logic behind the report).
All of these require some change to the application. If your application query is stored in a view (something like vw_FetchReport1) then the change is trivial and all on the server side. If the query is embedded in the application, then you need to replace it with something else. I strongly advocate using views (or in other databases user defined functions or stored procedures) for database access. This defines the API for the database application and greatly facilitates changes such as the ones described here.
EDIT: (in response to comment)
More information about scheduling jobs in MySQL is here. I would expect the SQL code to be something like:
truncate table ReportTable;
insert into ReportTable
select * from <ReportQuery>;
(In practice, you would include column lists in the select and insert statements.)
A simple solution that can be used to speed-up the response time for long running queries is to periodically generate summarized tables, based on underlying data refreshing or business needs.
For example, if your business don't care about sub-minute "accuracy", you can run the process once each minute and make your user interface to query this calculated table, instead of summarizing raw data online.
How to implement Materialized Views?
If not, how can I implement Materialized View with MySQL?
Update:
Would the following work? This doesn't occur in a transaction, is that a problem?
DROP TABLE IF EXISTS `myDatabase`.`myMaterializedView`;
CREATE TABLE `myDatabase`.`myMaterializedView` SELECT * from `myDatabase`.`myRegularView`;
I maintain LeapDB (http://www.leapdb.com) which adds incrementally refreshable materialized views to MySQL (aka fast refresh), even for views that use joins and aggregation. I've been working on this project for 13 years. It includes a change data capture utility to read the database logs. No triggers are used.
It includes two refresh methods. The first is similar to your method, except a new version is built, and then RENAME TABLE is used to swap the new for the old. At no point is the view unavailable for querying, but 2x the space is used for a short time.
The second method is true "fast refresh", it even has support for aggregation and joins.
LeapDB is significantly more advanced than the FromDual example referenced by astander.
Your example approximates a "full refresh" materialized view. You may need a "fast refresh" view, often used in a data warehouse setting, if the source tables include millions or billions of rows.
You would approximate a fast refresh by instead using insert / update (upsert) joining the existing "view table" against the primary keys of the source views (assuming they can be key preserved) or keeping a date_time of the last update, and using that in the criteria of the refresh SQL to reduce the refresh time.
Also, consider using table renaming, rather than drop/create, so the new view can be built and put in place with nearly no gap of unavailability. Build a new table 'mview_new' first, then rename the 'mview' to 'mview_old' (or drop it), and rename 'mview_new' to 'mview'. In your above sample, your view will be unavailable while your SQL populate is running.
This thread is rather old, so I will try to re-fresh it a bit:
I've been experimenting and even deployed in production several methods for having materialized views in MySQL. Basically all methods assume that you create a normal view and transfer the data to a normal table - the actual materialized view. Then, it's only a question of how you refresh the materialized view.
Here's what I've success with so far:
Using triggers - you can set triggers on the source tables on which you build the view. This minimizes the resource usage as the refresh is only done when needed. Also, data in the materialized view is realtime-ish
Using cron jobs with stored procedures or SQL scripts - refresh is done on a regular basis. You have more control as to when resources are used. Obviously you data is only as fresh as the refresh-rate allows.
Using MySQL scheduled events - similar to 2, but runs inside the database
Flexviews - using FlexDC mentioned by Justin. The closest thing to real materialized
I've been collecting and analyzing these methods, their pros and cons in my article Creating MySQL materialized views
looking forwards for feedback or proposals for other methods for creating materialized views in MySQL
According to the mySQL docs and comments at the bottom of the page, it just seems like people are creating views then creating tables from those views. Not sure if this solution is the equivalent of creating a materialized view, but it seems to be the only avenue available at this time.