I have created a view on a simple table. My problem is that my average execution time of a select on that view is about 29 seconds.
However, if I run the select statement which describes the view directly, the query executes in about 0.015 seconds.
Now, I have looked up some info, and here and here, people basically say that it should be roughly the same since a view is just a stored query.
Is it possible that I have this much of a difference in time? I have tried using SQL_NO_CACHE to make sure no cache is used so I get representative data when testing both options.
I would prefer to keep my view unless I have no option in reducing costs.
After a lot of research and trial and error I have concluded that on even simple queries and views, the performance can be a huge difference when selecting * from a view or just running the select query that is described in the creation of the view.
Related
I have two queries whose timing parameters I want to analyze.
The first query is taking much longer than the second one while in my opinion it should be the other way round.Any Explanations?
First query:
select mrn
from EncounterInformation
limit 20;
Second query:
select enc.mrn, fl.fileallocationid
from EncounterInformation as enc inner join
FileAllocation as fl
on enc.encounterIndexId = fl.encounterid
limit 20;
The first query runs in 0.760 seconds on MYSQL while second one runs in 0.509 seconds surprisingly.
There are many reasons why measured performance between two queries might be different:
The execution plans for the queries (the dominant factor)
The size of the data being returned (perhaps mrn is a string that is really long for the result set in the first query but not the second)
Other database activity, that locks tables and indexes
Other server activity
Data and index caches that are pre-loaded -- either in the database itself or in the underlying OS components
Your observation is correct. The first should be faster than the second. More importantly though is the observation that this simply does not make sense for your simple query:
The first query runs in 0.760 seconds
select mrn
from EncounterInformation
limit 20;
The work done for this would typically be to load one data page (or maybe a handful). That would only consistently take 0.760 seconds if:
You had really slow data storage (think "carrier pigeons").
EncounterInformation is a view and not a table.
You don't understand timings.
If the difference is between 0.760 milliseconds and 0.509 milliseconds, then the difference is really small and likely due to other issues -- warm caches, other activity on the server, other database activity.
It is also possible that you are measuring elapsed time and not database time, so network congestion could potentially be an issue.
If you are querying views, all bets are off without knowing what the views are. In fact, if you care about performance you should be including the execution plan in the question.
I can't explain the difference. What I can say is that your observation is reasonable, but your question lacks lots of information that suggests you need to learn more about how to understand timings. I would suggest that you start with learning about explain.
This is a general question that applies to MySQL, Oracle DB or whatever else might be out there.
I know for MySQL there is LIMIT offset,size; and for Oracle there is 'ROW_NUMBER' or something like that.
But when such 'paginated' queries are called back to back, does the database engine actually do the entire 'select' all over again and then retrieve a different subset of results each time? Or does it do the overall fetching of results only once, keeps the results in memory or something, and then serves subsets of results from it for subsequent queries based on offset and size?
If it does the full fetch every time, then it seems quite inefficient.
If it does full fetch only once, it must be 'storing' the query somewhere somehow, so that the next time that query comes in, it knows that it has already fetched all the data and just needs to extract next page from it.
In that case, how will the database engine handle multiple threads? Two threads executing the same query?
I am very confused :(
I desagree with #Bill Karwin. First of all, do not make assumptions in advance whether something will be quick or slow without taking measurements, and complicate the code in advance to download 12 pages at once and cache them because "it seems to me that it will be faster".
YAGNI principle - the programmer should not add functionality until deemed necessary.
Do it in the simplest way (ordinary pagination of one page), measure how it works on production, if it is slow, then try a different method, if the speed is satisfactory, leave it as it is.
From my own practice - an application that retrieves data from a table containing about 80,000 records, the main table is joined with 4-5 additional lookup tables, the whole query is paginated, about 25-30 records per page, about 2500-3000 pages in total. Database is Oracle 12c, there are indexes on a few columns, queries are generated by Hibernate.
Measurements on production system at the server side show that an average time (median - 50% percentile) of retrieving one page is about 300 ms. 95% percentile is less than 800 ms - this means that 95% of requests for retrieving a single page is less that 800ms, when we add a transfer time from the server to the user and a rendering time of about 0.5-1 seconds, the total time is less than 2 seconds. That's enough, users are happy.
And some theory - see this answer to know what is purpose of Pagination pattern
Yes, the query is executed over again when you run it with a different OFFSET.
Yes, this is inefficient. Don't do that if you have a need to paginate through a large result set.
I'd suggest doing the query once, with a large LIMIT — enough for 10 or 12 pages. Then save the result in a cache. When the user wants to advance through several pages, then your application can fetch the 10-12 pages you saved in the cache and display the page the user wants to see. That is usually much faster than running the SQL query for each page.
This works well if, like most users, your user reads only a few pages and then changes their query.
Re your comment:
By cache I mean something like Memcached or Redis. A high-speed, in-memory key/value store.
MySQL views don't store anything, they're more like a macro that runs a predefined query for you.
Oracle supports materialized views, so that might work better, but querying the view would have the overhead of interpreting an SQL query.
A simpler in-memory cache should be much faster.
Just interested, maybe someone might know that. If I use lazy load to get all attributes, relations and so on it makes ~350 queries to database it takes about 2 sec to fully render the page. If i make one big query with multiple joins to get all relations I need it makes ~20 queries one is really big, and problem is that this big query first time takes about 10 sec to execute, after that it gets cached and it goes much faster and whole page loads in ~1.5 sec, but problem is that every user has different parameters to that query so for every user first time it goes for 10 sec.. why it goes so long for first time?
May I ask, if you are using a stored procedure? I have added a link with some advantages of using a stored procedure https://docs.oracle.com/cd/F49540_01/DOC/java.815/a64686/01_intr3.htm . Can you give some examples of your parameters for different users?
Thanks
As you gave no information on the data base schema, the data size and other parameters it is very difficult to determine the root cause of the bad performance. However, there is another answer here on StackOverflow that might be a great starting point for further investigation.
In general consider the following questions to start investigating / optimizing:
Do you really need all the information you fetch from the DB (at once)?
Is the database optimized for the queries you execute?
How often do you need to execute the queries and if you cache them, how often does the cache outdate?
So I have a real estate website. I have a page that runs about 5 queries to build a statistics page. I am wondering if there is a way to speed this up or optimize or combine the queries so that it runs faster. Right now its take up to 5 seconds to run the page.
Query:
SELECT COUNT(`listing_num`) as `count`,
AVG(`price`),
AVG(`square_feet`),
AVG(`bedroom_total`),
AVG(`bathroom_total`),
MIN(`price`),
MAX(`price`),
MIN(`square_feet`),
MAX(`square_feet`),
MIN(`bathroom_total`),
MAX(`bathroom_total`),
MIN(`bedroom_total`),
MAX(`bedroom_total`),
MIN(`psf`),
MAX(`psf`),
AVG(`psf`)
FROM `Res_Active2`
WHERE `status` != 'S'
So i run this query about 6 different times on the page with the WHERE clause changed in each so that I can display stats for sold properties, active properties, under contract properties, etc.
What is the right way and fast way to do this? Can i use cache, combine the sql, anything? I need to speed this page up. Thanks.
Try just setup mysql query cache. It will do only once and reuse result in all other queries.
To enable mysql cache see mysql cache
I am pretty sure for you will be enought just add in your /etc/my.conf
query_cache_size=30M
If that not help, you can create special table which have hold result of that query and update that result every X minutes by external script.
I have a complex query using many joins (8 in fact). I was thinking of simplifying it into a view. After a little research I can see the benefits in simplicity and security. But I didn't see any mentions of speed.
Do views work like prepared statements, where the query is pre-compiled? Is there any notable performance gain from using views?
No, a view is simply a stored text query. You can apply WHERE and ORDER against it, the execution plan will be calculated with those clauses taken into consideration.
A view's basically a stored subquery. There's essentially no difference between:
SELECT *
FROM someview
and
SELECT *
FROM (
SELECT somestuff
FROM underlying table
);
except the view's a bit more portable, as you don't have to write out the underlying query each time you want to work with whatever data it returns.
It can sometimes help, but it's no silver bullet. I've seen a view help performance, but I've also seen it hurt it. A view can force materialization, which can sometimes get you a better access path if MySQL isn't choosing a good one for you.