I am running the following insert query in MS Access 2010, and I need a way for the query to run faster. I do need to get all of the columns from the Source_1_table into the temp_Table.
Source_1_table has 505K records
Source_2_table has 3959 records
INSERT INTO [temp_Table]
SELECT *
FROM Source_1_table
WHERE
((dbo_Source_1_table.Field_1)) Not IN (SELECT [Source_2_table].[ID_Field] FROM [Source_2_table]));
Question 1. How do I see how long it is taking the query to run (similar to SSMS)?
Question 2. The query seems to be taking longer than 10 minutes to run. Is there any way speed up this query?
I've never worked with that many records in an Access db before, so I'm not surprised it's slow.
I solved a similar problem recently in some legacy code. I think a big problem is that you are running your subquery once for every record in the main query (someone please correct me if that's not correct). Try this instead:
INSERT INTO [temp_Table]
SELECT s1.*
FROM Source_1_table s1
Left Join Source_1_table s2 on s1.Field_1=s2.ID_Field
where s2.ID_Field is null
Related
I found a strange behavior on following query:
UPDATE llx_socpeople SET no_email=1 WHERE rowid IN (SELECT source_id FROM llx_mailing_cibles where tag = "68d74c3bc618ebed67919ed5646d0ffb");
takes 1 min and 30 seconds.
When I split up the commands to 2 queries:
SELECT source_id FROM llx_mailing_cibles where tag = "68d74c3bc618ebed67919ed5646d0ffb";
Result is 10842
UPDATE llx_socpeople SET no_email=1 WHERE rowid = 10842;
Result is shown in milliseconds.
Table llx_socpeople has about 7.000 records, llx_mailing_cibles has about 10.000 records.
MySQL Version is: 5.7.20-0ubuntu0.16.04.1
I already tried to optimize/repair both tables with no effect.
Any ideas?
Currently, as the subquery is being run for each row of the main query, we can expect a longer execution time.
What I would suggest would be to rely on a inner join for performing the update:
UPDATE llx_socpeople AS t1
INNER JOIN llx_mailing_cibles AS t2
ON t1.rowid = t2.source_id
SET t1.no_email=1
WHERE t2.tag = "68d74c3bc618ebed67919ed5646d0ffb";
This way you will definitely get far better performance.
You can troubleshoot your slow queries using the EXPLAIN MySql statement. Find more details about it on it's dedicated page from the official documentation. It might help you discover any missing indexes.
I have table with documents nammed: z_web_dok it has 133,369 rows, the other table with products nammed: z_web_dok_art has 693,930 rows, all fields needed for querying are indexed, and connection between those tables is auto increment field of z_web_dok.oid and z_web_dok_art.oid_id.
What is the problem?
I were made simple sql query for returning old prices (by buyer, place, country). The query is:
SELECT (COALESCE(NULLIF(TNew.cijena_e,''), TNew.cijena)*(TN.kurs_iznos/1))
FROM z_web_dok_art As TNew
INNER JOIN z_web_dok As TN ON TN.oid=TNew.oid_id
WHERE (TN.Drzava='BiH' AND TNew.aid='SOME_PRODUCT_ID' AND TN.vrsta_dok='pri'
AND TN.kup_id='1047' AND TN.mag_id='5' AND TN.oid<>'151967')
ORDER BY TN.dat_zavrsena DESC
LIMIT 1
and it worked, for each product, so i used it inside another query, which gets products their new prices and with the query above (subquery) i tried to pull the old prices (Connection was between tables: TNew.aid=WArt.aid )
SELECT WArt.aid, WArt.nc, WArt.isp, COALESCE(NULLIF(WArt.cijena_e,''), WArt.cijena) As cijena, WArt.rab, WArt.vpc, WArt.mpc, C.EAN, C.Model, C.Naziv, C.JM, C.WebBiH, C.DostupnostBiH,
(
SELECT (COALESCE(NULLIF(TNew.cijena_e,''), TNew.cijena)*(TN.kurs_iznos/1))
FROM z_web_dok_art As TNew
INNER JOIN z_web_dok As TN ON TN.oid=TNew.oid_id
WHERE (TN.Drzava='BiH' AND TNew.aid=WArt.aid AND TN.vrsta_dok='pri'
AND TN.kup_id='1047' AND TN.mag_id='5' AND TN.oid<>'151967')
ORDER BY TN.dat_zavrsena DESC
LIMIT 1
) As s_cijena
FROM
z_web_dok_art As WArt
LEFT JOIN Cjenovnik As C ON C.ID=WArt.aid
WHERE WArt.oid_id='151967'
ORDER BY CASE WHEN WArt.isp='0' THEN 1 ELSE 0 END, WArt.id_dok_art
It worked, for a long period, but today we discovered it returned 6 times higher result for old price for two products (on a order that has 19 products).
So i were query TN.oid with same query to see what does it pulls from database and it was oid that has following fields same: (kup_id, vrsta_dok) but all others are different even if the query asked for Drzava='BiH' AND TN.mag_id='5' it doesn't return results for that.
But the other problem is, if i execute subquery as query alone, with that product (or more them) it returns RIGHT RESULTS. But above one mixed results right and wrong one.
It's not problem in solving this!
I can solve it, but i wanted to know why this QUERY didn't work, i mean, what's the problem with it? Did anyone had problem with this?
This is my first bad experience with similar queries...
I were thinking about, it's happening because of subquery JOINS with another table inside of query (because the subquery works perfect without it's parent query).
Thank you for your time!
Best regards,
Nedžad.
IMPORTANT UPDATE (25.10.2016):
The server mysql version: 5.5.52-cll
Local version mysql: 5.6.17 - MySQL Community Server (GPL)
Localhost returned the proper results, with NO MISTAKE,
while the server mysql still returns wrong results..
Is it bug at mysql 5.5.52-cll or what?
Image of results:
UPDATE: (SOLVED)
And i solved it by using group in subquery:
GROUP BY TNew.oid_id, TNew.aid
(grouping by, documents, and products inside of it), and it returned the right results - the performance was good, 0.1704 (which is good because it's always one document at opening). Once again, thanks everyone which were lead me to right path
From my debian terminal I try to execute in mysql client a query like:
SELECT *
FROM stop_times_lazio_set2_withtime2 AS B
WHERE EXISTS
(SELECT *
FROM stop_times_lazio_set2_emptytime2 AS A
WHERE B.trip_id=A.trip_id);
table A contains around 3 million records.
table B is a sub set of A of around 400000 records.
I'd like to select every records of A thats have a row "parent" with the same id (yes its not an unique/primary id)
Now it takes more than hours...now I'm around 2h and i still seen just a blinking pointer... is it the query correct? Even I can't access to others mysql client like phpmyadmin.
There is any way to speed up the process?
There is a way to check how many records are processed at running times?
I guess you have already indexed trip_id? There is another way writing the query, maybe it helps:
SELECT *
FROM stop_times_lazio_set2_withtime
WHERE trip_id IN (SELECT trip_id FROM stop_times_lazio_set2_emptytime2)
I would expect a straight JOIN to be much much faster...
SELECT B.*
FROM stop_times_lazio_set2_withtime2 AS B
JOIN stop_times_lazio_set2_emptytime2 AS A ON B.trip_id=A.trip_id
Why not using a simpler request ?
SELECT A.*
FROM stop_times_lazio_set2_emptytime2 AS A, stop_times_lazio_set_withtime2 AS B
WHERE B.trip_id=A.trip_id;
With that many records, it will obviously take time.
You can actually prevent it by processing only a few at a time by adding this at the end of the request
LIMIT <beginning>, <number of records>
Did you tried "left join"??
sample:
select columns from withtime
left join emptytime on withtime.tripid=emptytime.tripid;
I have a big Access database I am working on, designed by someone else. I am trying to improve it. The way it was set up, first you run Query 1 which makes Table A. Then, run Query 2, which used Table A among others, and made Table B. Then run Query 3, which uses Table B and makes Table C. Then run Query 4, which uses Table C and makes Table D. The final output we use is Table D.
My improvement here was to change Query 1, Query 2, and Query 3 to Select queries, instead of make table queries, and changing the SQL simply by find and replace (for example, in Query 2, replace all the instances of Table A with Query 1 instead). Query 4 is still a make table query and it makes Table D which I export to Excel later.
My question: Can I just run Query 4? That is, will that automatically run Query 3, which will automatically run Query 2, which will automatically run Query 1? Or, do I need to run Query 1 first, then run Query 2, then run Query 3, then run Query 4?
Perhaps beyond that, can I make Query 4 a Select query also? Then, I wouldn't even run Query 4. Instead, I would just export Query 4 itself to Excel and I'm wondering if that would automatically run Query 4, which would run Query 3, and so on.
Just to be clear: I'm not looking for any SQL tips here. I want to keep it just using Access (which is a front-end for SQL, I know) for now.
Note: I realize that one answer here is, "Why don't you try it?" I have and it seems to work as I think it should. The problem is, I have already run all the queries many times before, so I don't know if that makes everything work for now but maybe it won't work the same way later if I haven't already run the previous queries.
Thanks for any help
As Olivier already mentioned, yes selecting from SELECT queries works as you expect. You can actually test this by creating this function in a module
Function LogQueryCall(ByVal query As String)
Debug.Print query & " " & Now
End Function
an then calling it from your queries e.g.
Query1
SELECT *, LogQueryCall("Query1") FROM Table1 ;
Query2
SELECT *, LogQueryCall("Query2")
FROM Query1
INNER JOIN Table2
ON Query1.Field = table2.Field
Query3
SELECT *, LogQueryCall("Query3") FROM Query2
Then you'll see the results in the Immediate Window
Query1 01/03/2012 5:54:46 PM
Query2 01/03/2012 5:54:46 PM
Query3 01/03/2012 5:54:46 PM
Note the function will only get called once per query not once per row
Another option is to create 1 big query. This would use the contents of each query in the from clause like so.
SELECT *
FROM
(Select * FROM Table1) A
INNER JOIN Table2
ON a.Field = table2.Field
The short answer is, "Yes, you can nest SELECT queries and just export to Excel by just exporting the outer-most query result".
Queries can be used exactly like tables in many situations. The nested queries will execute their own SELECT statement and pass their result to the surrounding query as if it was a table.
If query2 calls query1 and
query3 calls query2 and
qyery4 calls query3
... then the queries will automatically be executed in the order shown below by passing their result to the next query when you execute query4:
table -> query1 -> query2 -> query3 -> query4
I assume that your queries look something like this:
query1: SELECT * FROM table;
query2: SELECT * FROM query1;
query3: SELECT * FROM query2;
query4: SELECT * FROM query3;
Here's the query (the largest table has about 40,000 rows)
SELECT
Course.CourseID,
Course.Description,
UserCourse.UserID,
UserCourse.TimeAllowed,
UserCourse.CreatedOn,
UserCourse.PassedOn,
UserCourse.IssuedOn,
C.LessonCnt
FROM
UserCourse
INNER JOIN
Course
USING(CourseID)
INNER JOIN
(
SELECT CourseID, COUNT(*) AS LessonCnt FROM CourseSection GROUP BY CourseID
) C
USING(CourseID)
WHERE
UserCourse.UserID = 8810
If I run this, it executes very quickly (.05 seconds roughly). It returns 13 rows.
When I add an ORDER BY clause at the end of the query (ordering by any column) the query takes about 10 seconds.
I'm using this database in production now, and everything is working fine. All my other queries are speedy.
Any ideas of what it could be? I ran the query in MySQL's Query Browser, and from the command line. Both places it was dead slow with the ORDER BY.
EDIT: Tolgahan ALBAYRAK solution works, but can anyone explain why it works?
maybe this helps:
SELECT * FROM (
SELECT
Course.CourseID,
Course.Description,
UserCourse.UserID,
UserCourse.TimeAllowed,
UserCourse.CreatedOn,
UserCourse.PassedOn,
UserCourse.IssuedOn,
C.LessonCnt
FROM
UserCourse
INNER JOIN
Course
USING(CourseID)
INNER JOIN
(
SELECT CourseID, COUNT(*) AS LessonCnt FROM CourseSection GROUP BY CourseID
) C
USING(CourseID)
WHERE
UserCourse.UserID = 8810
) ORDER BY CourseID
Is the column you're ordering by indexed?
Indexing drastically speeds up ordering and filtering.
You are selecting from "UserCourse" which I assume is a joining table between courses and users (Many to Many).
You should index the column that you need to order by, in the "UserCourse" table.
Suppose you want to "order by CourseID", then you need to index it on UserCourse table.
Ordering by any other column that is not present in the joining table (i.e. UserCourse) may require further denormalization and indexing on the joining table to be optimized for speed;
In other words, you need to have a copy of that column in the joining table and index it.
P.S.
The answer given by Tolgahan Albayrak, although correct for this question, would not produce the desired result, in cases where one is doing a "LIMIT x" query.
Have you updated the statistics on your database? I ran into something similar on mine where I had 2 identical queries where the only difference was a capital letter and one returned in 1/2 a second and the other took nearly 5 minutes. Updating the statistics resolved the issue
Realise answer is too late, however I have just had a similar problem, adding order by increased the query time from seconds to 5 minutes and having tried most other suggestions for speeding it up, noticed that the /tmp files where getting to be 12G for this query. Changed the query such that a varchar(20000) field being returned was "trim("ed and performance dramatically improved (back to seconds). So I guess its worth checking whether you are returning large varchars as part of your query and if so, process them (maybe substring(x, 1, length(x))?? if you dont want to trim them.
Query was returning 500k rows and the /tmp file indicated that each row was using about 20k of data.
A similar question was asked before here.
It might help you as well. Basically it describes using composite indexes and how order by works.
Today I was running into a same kind of problem. As soon as I was sorting the resultset by a field from a joined table, the whole query was horribly slow and took more than a hundred seconds.
The server was running MySQL 5.0.51a and by chance I noticed that the same query was running as fast as it should have always done on a server with MySQL 5.1. When comparing the explains for that query I saw that obviously the usage and handling of indexes has changed a lot (at least from 5.0 -> 5.1).
So if you encounter such a problem, maybe your resolution is to simply upgrade your MySQL