I have created a complex view which gives me output within a second on Oracle 10g DBMS.. but the same view takes 2,3 minutes on MYSQL DBMS.. I have created indexes on all the fields which are included in the view definition and also increased the query_cache_size but still failed to get answer in less time. My query is given below
select * from results where beltno<1000;
And my view is:
create view results as select person_biodata.*,current_rank.*,current_posting.* from person_biodata,current_rank,current_posting where person_biodata.belt_no=current_rank.belt_no and person_biodata.belt_no=current_posting.belt_no ;
The current_posting view is defined as follows:
select p.belt_no belt_no,ps.ps_name police_station,pl.pl_name posting_as,p.st_date from p_posting p,post_list pl,police_station ps where p.ps_id=ps.ps_id and p.pl_id=pl.pl_id and (p.belt_no,p.st_date) IN(select belt_no,max(st_date) from p_posting group by belt_no);
The current_rank view is defined as follows:
select p.belt_no belt_no,r.r_name from p_rank p,rank r where p.r_id=r.r_id and (p.belt_no,p.st_date) IN (select belt_no,max(st_date) from p_rank group by belt_no)
Some versions of MySQL have a particular problem with in and subqueries, which you have in this view:
select p.belt_no belt_no,ps.ps_name police_station,pl.pl_name posting_as,p.st_date
from p_posting p,post_list pl,police_station ps
where p.ps_id=ps.ps_id and p.pl_id=pl.pl_id and
(p.belt_no,p.st_date) IN(select belt_no,max(st_date) from p_posting group by belt_no)
Try changing that to:
where exists (select 1
from posting
group by belt_no
having belt_no = p.belt_no and p.st_date = max(st_date)
)
There may be other issues, of course. At the very least, you could format your queries so they are readable and use ANSI standard join syntax. Being able to read the queries would be the first step to improving their performance. Then you should use explain in MySQL to see what the query plans are like.
Muhammad Jawad it's so simple. you have already created indexes on table that allow database application to find data fast, but if you change/update the (indexes tables) table (e.g: inserst,update,delete) then it take more time that of which have no indexes applied on table because the indexes also need updation so each index will be updated that take too much time. So you should apply indexes on columns or tables that we use it only for search purposes only. hope this will help u. thank u.
Related
I am creating a booking application and have normalised the database as much as is possible. My question is the following, should I create Views in the database that combine the tables again before I do selective selects on the combined view in the WHERE clause or is it better to filter the tables with selects before joining them in a view?
EDIT: Included example.
The first scenario creates the combined view first, and then performs the SELECT on the combined view (this view may have thousands of records):
CREATE VIEW appc as
SELECT * FROM appointment
LEFT OUTER JOIN chair
ON appointment.chair_idchair = chair.idchair
SELECT * FROM appc
WHERE chair_idchair = 1;
The second scenario will first filter the table in the left of the join and then create a view based on the filtered table, then the join will be made using this much smaller view:
CREATE VIEW appf as
SELECT * FROM appointment
WHERE chair_idchair = 1;
SELECT * FROM appf
LEFT OUTER JOIN chair
ON appf.chair_idchair = chair.idchair
It makes little difference to MySQL. MySQL (unlike some other RDBMS brands) does not store anything in the view itself. Think of a MySQL view more like a macro. In most cases, querying the view is exactly like executing the query you defined as the view.
Applying subsequent conditions in the WHERE clause when querying the view is combined pretty transparently with the view's query, as if you had written the full query and given it one more condition.
Exception: you can optionally create the view with ALGORITHM=TEMPTABLE, to force it to store the results of the view in a temp table, and then apply extra conditions you specified in your query. In this case, it would be better to build the conditions into the view's query, to reduce the size of the resulting temp table.
See http://dev.mysql.com/doc/refman/5.6/en/view-algorithms.html for more details.
As a general rule, the optimizer is pretty smart, and can see right through the views when constructing a query plan. If you try to "help" the optimizer by pre-selecting some data, you may be hiding information from the optimizer that would have allowed it to create a smarter, more-optimal plan.
There is a strange rule in MySql which basically says if you want to create a view and you want to use constructs such as GROUP BY, DISTINCT, UNION, etc. in your view, you cannot create your view with ALGORITHM = MERGE! Read more here.
This basically means that if you create a view in MySql and you have something like GROUP BY in your view, if you run a query like:
select * from MyView
where foo = 2
then MySql will read all the rows from MyView and put it in a temp table and after that it will apply the where clause (foo = 2) to the temp table and return you the result.
The problem is that if your view has 10 Million rows it will get all the 10M rows and then apply the where clause, which is insanely inefficient.
So, does anyone know what is the rationale behind this? I know that MS Sql Server is not doing this and I bet Oracle is not doing it either.
A similar question was asked on the MySQL forums about union views, years ago. A MySQL developer agreed that it should be easy to implement the MERGE algorithm on views using UNION, but AFAIK it still has not been implemented!
I have a very large number of rows in my table, table_1. Sometimes I just need to retrieve a particular row.
I assume, when I use SELECT query with WHERE clause, it loops through the very first row until it matches my requirement.
Is there any way to make the query jump to a particular row and then start from that row?
Example:
Suppose there are 50,000,000 rows and the id which I want to search for is 53750. What I need is: the search can start from 50000 so that it can save time for searching 49999 rows.
I don't know the exact term since I am not expert of SQL!
You need to create an index : http://dev.mysql.com/doc/refman/5.1/en/create-index.html
ALTER TABLE_1 ADD UNIQUE INDEX (ID);
The way I understand it, you want to select a row with id 53750. If you have a field named id you could do this:
SELECT * FROM table_1 WHERE id = 53750
Along with indexing the id field. That's the fastest way to do so. As far as I know.
ALTER table_1 ADD UNIQUE INDEX (<collumn>)
Would be a great first step if it has not been generated automatically. You can also use:
EXPLAIN <your query here>
To see which kind of query works best in this case. Note that if you want to change the where statement (anywhere in the future) but see a returning value in there it will be a good idea to put an index on that aswell.
Create an index on the column you want to do the SELECT on:
CREATE INDEX index_1 ON table_1 (id);
Then, select the row just like you would before.
But also, please read up on databases, database design and optimization. Your question is full of false assumptions. Don't just copy and paste our answers verbatim. Get educated!
There are several things to know about optimizing select queries like Range and Where clause Optimization, the documentation is pretty informative baout this issue, read the section: Optimizing SELECT Statements. Creating an index on the column you evaluate is very helpfull regarding performance too.
One possible solution You can create View then query from view. here is details of creating view and obtain data from view
http://www.w3schools.com/sql/sql_view.asp
now you just split that huge number of rows into many view (i. e row 1-10000 in one view then 10001-20000 another view )
then query from view.
I am pretty sure that any SQL database with a little respect for themselves does not start looping from the first row to get the desired row. But I am also not sure how they makes it work, so I can't give an exact answer.
You could check out what's in your WHERE-clause and how the table is indexed. Do you have a proper primary key? Like using a numeric data type for that. Do you have indexes on more columns, that is used in your queries?
There is also alot to concider when installing the database server, like where to put the data and log files, how much memory to give the server and setting the growth. There's a lot you can do to tune your server.
You could try and split your tables in partitions
More about alter tables to add partitions
Selecting from a specific partition
In your case you could create a partition on ID for every 50.000 rows and when you want to skip the first 50.000 you just select from partition 2. How to do this ies explained quite well in the MySQL documentation.
You may try simple as this one.
query = "SELECT * FROM tblname LIMIT 50000,0
i just tried it with phpmyadmin. WHERE the "50,000" is the starting row to look up.
EDIT :
But if i we're you i wouldn't use this one, because it will lapses the 1 - 49999 records to search.
So my expertise is not in MySQL so I wrote this query and it is starting to run increasingly slow as in 5 minutes or so with 100k rows in EquipmentData and 30k or so in EquipmentDataStaging (which to me is very little data):
CREATE TEMPORARY TABLE dataCompareTemp
SELECT eds.eds_id FROM equipmentdatastaging eds
INNER JOIN equipment e ON e.e_id_string = eds.eds_e_id_string
INNER JOIN equipmentdata ed ON e.e_id = ed.ed_e_id
AND eds.eds_ed_log_time=ed.ed_log_time
AND eds.eds_ed_unit_type=ed.ed_unit_type
AND eds.eds_ed_value = ed.ed_value
I am using this query to compare data rows pulled from a clients device to current data sitting within their database. From here I take the temp table and use the ID's off it to make conditional decisions. I have the e_id_string indexed and I have e_id indexed and everything else is not. I know that it looks stupid that I have to compare all this information, but the clients system is spitting out redundant data and I am using this query to find it. Any type of help on this would be greatly appreciated whether it be a different approach by SQL or MySql Management. I feel like when I do stuff like this in MSSQL it handles the requests much better, but that is probably because I have something set up incorrectly.
TIPS
index all necessary columns which are using with ON or WHERE condition
here you need to index eds_ed_log_time,eds_e_id_string, eds_ed_unit_type, eds_ed_value,ed_e_id,ed_log_time,ed_unit_type,ed_value
change syntax to SELECT STRAIGHT JOIN ... see more reference
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 5 years ago.
Improve this question
I wanted to ask if View are really worth using.
From my understanding a view is really just a query and each time you query the view the view then runs its own query again to get fresh/uptodate data.
This sounds to me like 2 queries are run.
wouldn't it be faster to just run the query required and skip the view?
Please note: I would be using simple views but even it they were quite complex I assume the same principle applies.
My type of view - say 3 tables with 6 columns each - and 2 columns of each time is added into the view with a couple of maths equations to refine the data a touch.
What do others do? Skip or use them?
Typically Views are set up to make selects easier to understand and at the same time give guidance to the database engine on how to optimize the query. By creating a view you tell the database engine that you're going to be selecting from this frequently and to spend more time optimizing the query plan so selects from the view will be faster. The upside of this is that when it comes time to parse the query and plan the query you'll save some execution time because the optimization has already been performed. It could be as little as a few miliseconds you save, or potentially very large (for very large result sets)
You're correct that views are not designed to be a performance benefit in MySQL.
What they are designed to do is make other queries built on them to be simpler to read, and to make sure that other users and programmers have a better chance at using the data correctly. Think of them as a way to virtually de-normalize the data without taking the size/performance hit of actually de-normalizing the data.
Just as the most simple case, let's just take orders and line items. Each order has a line item.
The orders table might have the following columns:
ID
Status
Created_at
Paid_on
And the line_items table might have the following columns:
LI_ID
order_id
sku_id
quantity
price
What you'll find, when writing code and queries is that you are going to be doing the following join all the time -
orders
join line_items on line_items.order_id = orders.id
This could be simplified by creating a view:
create view 'order_lines' as
select * from orders
join line_items on line_items.order_id = orders.id
So your query would go from:
select orders.id, sum(price) from orders
join line_items on line_items.order_id = orders.id
where created_at >= '2011-12-01' and created_at < '2012-01-01
group by orders.id;
to:
select id, sum(price) from order_lines
where created_at >= '2011-12-01' and created_at < '2012-01-01
group by id;
The DB will execute both of these exactly the same way, but one is easier to read. Admittedly in this case, not MUCH easier to read, but easier to read and code.
The query optimizer is usually able to combine the view query with the query that uses the view in such a way that only a single query is run, so the objection you have to views doesn't really apply.
See also:
MySQL VIEW as performance troublemaker
View vs. Table Valued Function vs. Multi-Statement Table Valued Function
Should I use a view, a stored procedure, or a user-defined function?
Regards
Views can be provided to applications or users that needs a straight-forward view of data that isn't necessarily in one table (or limited fields from one table). That means they don't have to understand the data and how it relates -- they just get the data they need. You create the complex query, optimize it, and they just use the resulting view.