I have query like this
UPDATE linksupload as lu SET lu.progress = (SELECT SUM(l.submitted)
FROM links l
WHERE l.upload_id = lu.id)
It takes 10 sec to execute. linksupload contains 10 rows, links contains 700k rows.
Query:
UPDATE linksupload as lu SET lu.progress = (SELECT count(*)
FROM links l
WHERE l.upload_id = lu.id)
takes 0,0003 sek to execute. Also select with sum with group by from first query is fast. upload_id and id are indexes. Why first query takes so long time to execute? How to speed it up?
Indexes allow the database application to find data fast; without reading the whole table.
second query should be just count so it is not reading table. But first query should be sum of submitted column.So it is reading table.
First query should be slow.
Related
I have a table with about 2m records that I'm joining with a much smaller table, producing a set of about 2k records. This joining takes 0.13 second to process. If I apply a WHERE condition referencing fields in the 2m record table however, the query takes 17+ seconds to process. It appears to me that the MySQL engine is deciding to apply the WHERE clause before the JOIN even though in this case that is much less efficient. Can anyone help me understand why MySQL is doing this and how I can get it to apply the WHERE clause after the JOIN? I've put example tables below to better help illustrate the problem:
Table BigTable {
SmallTableId: int,
result: varchar
}
Table SmallTable {
id: int
}
SELECT * FROM SmallTable
LEFT JOIN BigTable on BigTable.smallTableId = SmallTable.id
-- This takes 0.13 seconds to process and produces ~2k rows
SELECT * FROM SmallTable
LEFT JOIN BigTable on BigTable.smallTableId = SmallTable.id
WHERE result = 'ERROR'
-- This takes 17 seconds to process
My uninformed naive idea about the optimal way to complete the second query would be to generate the result set of the first query (about 2k rows in 0.13 seconds), and then apply the WHERE condition to that, which would be trivial in terms of process time to apply to 2k records. Clearly it's not doing that though.
I've tried making the first query a sub-query to try to get MySQL to generate it first but that has no effect:
SELECT * FROM (
SELECT * FROM SmallTable
LEFT JOIN BigTable on BigTable.smallTableId = SmallTable.id
) as t
WHERE t.result = 'ERROR'
-- Still takes 17 seconds
Thanks in advance for any help
I have table of 10 million records and it will still grow and there are more 1000 thousand users who will be executing this query at the same time...
Here is how my table looks like :
So I want when :
select * from tbl_bol group by categorie order by categorie desc
query is executed by those users the execution should not exceed 1 sec and at least from each categorie(from categorie column) on record must be in the result.
The current problem I have is the query execution is taking long is bring only the top categorie(column) records .
How should I write the query to give that result I want and improve the performance? I have index on categorie.
I have a table having 200K rows. When I execute a query it's taking too much time; approximately 2 minutes.
This is my query:
SELECT a,b,c,d,#row:="tag1" as tag
FROM tableName
WHERE tagName like "%helloworld|%"
ORDER BY viewCount desc
LIMIT 20;
helloworld| occurred only in 2 rows.
I want to change the query so if the data is present more than 20 times, it should return 20 rows else whatever rows present.
How to optimize this query?
You cannot speed this up as written.
The WHERE clause with the LIKE requires that you scan each and every row. It's O(N), where N = # of rows in the table. It will run more slowly as your table size increases.
You can make the query run faster if you can find a way to parse that string into tokens that you can INSERT as columns and index.
Try these
Set index on your search field of your table & then check the query execution time
Not sure what's viewCount here but i guess you are getting this with subquery try to remove order statement & then check the query execution time
I have a large query with many nested SELECT statements. A simplified version might look like this:
SELECT * FROM tableA WHERE x IN(
SELECT * FROM tableB WHERE x IN(
SELECT * FROM tableC WHERE user_id = y
)
)
Crucially, the innermost statement starts off by looking at the user_id and selecting a list of id numbers to use in the rest of the query.
The problem I'm having is that even if two users have the same data in tableC, the rest of the query doesn't seem to be cached.
For example if SELECT * FROM tableC WHERE user_id = 1 returns (1,2,3,4,5)
and SELECT * FROM tableC WHERE user_id = 2 also returns (1,2,3,4,5)
If I run the full query with user_id = 1 the execution time is about 0.007 seconds. If I re-run the query, I get a reduced execution time of 0.002. If I change the user_id to 2 and run the query, the execution time goes back to 0.007 for the first time the query is run. Is it possible for mySQL to cache the result of the individual parts of a query?
It seems that you use mysql. So when you run query 'SELECT * FROM tableC WHERE user_id = 1' first time you get the result '1,2,3,4,5' and you query goes to query cache. Therefore the time execution after the second running is less than the first one. In this case your result is associated with your first query.
When you run the second query your server doesn't know anything about it. So it runs it and returns something(in your case results are identical). Next time when you run the second query you will get it from query cache and it will be significantly fast. Anyway the server will store two different records in query cache.
I am using
SELECT *
FROM picture
INNER JOIN user ON picture.fbid = user.fbid
WHERE hide = 0
AND userhide = 0
ORDER BY gender, RAND( ) LIMIT 0 , 1
This executes and gives me the link to a picture of a random row where the gender is female. It takes around 15 seconds to execute.
How could I change the SQL so that it is calculated fast? Should I add a gender column in the Picture table so that it doesn't combine both the table and then gets a random row or is there some other way to optimise the SQL?
[EDIT] You can do the same with a MySQL stored procedure, I just coded it here: http://sqlfiddle.com/#!2/d0e6a/2
(Note: Not having the count done in every query would aliviate the load, otherwise in this particular case it can be even slower and I would recommend to keep your query, but with index optimization. Or check this out: Is cross-table indexing possible?)
[EDIT 2] Here's another example with index optimization using an average of 50000 random entries in each table: http://sqlfiddle.com/#!2/bfbe1/1 The query takes less than 100ms (Without indexes I left it running for several minutes and still no result!). You can create indexes on existing tables using CREATE INDEX
Here's my solution using 2 queries from a PHP script:
$sql="SELECT COUNT(*)
FROM picture
INNER JOIN user ON picture.fbid = user.fbid
WHERE hide = 0
AND userhide = 0
AND gender = 'female'
GROUP BY gender";
$result=mysql_query($sql);
$row=mysql_fetch_array($result);
$sql="SELECT *
FROM picture
INNER JOIN user ON picture.fbid = user.fbid
WHERE hide = 0
AND userhide = 0
AND gender = 'female'
LIMIT FLOOR(RAND()*$row['COUNT(*)']),1";
$result=mysql_query($sql);
$picture=mysql_fetch_array($result);
This should make you query a lot faster.
Also, if your tables are big (which I guess it's the case, since the query takes 15 secs) it would help if the conditional/join fields are indexed.
I'd recommend creating an INDEX on picture.fbid = user.fbid this will help to speed the JOIN on the table.
More Mysql specific information here: http://dev.mysql.com/doc/refman/5.0/en/mysql-indexes.html