I am tying to execute this query but it is taking more than 5 hours, but the data base size is just 20mb. this is my code. Here I am joining 11 tables with reg_id. I need all columns with distinct values. Please guide me how to rearrange the query.
SELECT *
FROM degree
JOIN diploma
ON degree.reg_id = diploma.reg_id
JOIN further_studies
ON diploma.reg_id = further_studies.reg_id
JOIN iti
ON further_studies.reg_id = iti.reg_id
JOIN personal_info
ON iti.reg_id = personal_info.reg_id
JOIN postgraduation
ON personal_info.reg_id = postgraduation.reg_id
JOIN puc
ON postgraduation.reg_id = puc.reg_id
JOIN skills
ON puc.reg_id = skills.reg_id
JOIN sslc
ON skills.reg_id = sslc.reg_id
JOIN license
ON sslc.reg_id = license.reg_id
JOIN passport
ON license.reg_id = passport.reg_id
GROUP BY fullname
Please help me if I did any mistake
This is a bit long for a comment.
The first problem with your query is that you are using select * with group by fullname. You have zillions of columns in the select that are not in the group by. Unless you really, really, really know what you are doing (which I doubt), this is the wrong way to write a query.
Your performance problem is undoubtedly due to cartesian products and lack of indexes. You are joining across different dimensions -- such as skills and degrees. The result is a product of all the possibilities. For some people, the data size can grow and grow and grow.
And then, the question is: do you have indexes on the keys used in the joins? For performance, you generally want such indexes.
I thought the problem is in the query.First make sure group by fullname and try to give some column names instead of *.
I need help with optimizing some of my SQL queries. I'm not good in SQL performance. I have a SQL Server 2008 RS Express and I can't use DTA.
May be can help me with optimising and manually creating indexes for these two queries:
SELECT tblBlogs.RecordID, tblBlogs.RecordText, tblBlogs.CDate, tblBlogs.UserID, tblBlogs.Comments, tblUsers.Username, tblUserpics.UserpicName
FROM (
SELECT tblBlogs_2.RecordID, tblBlogs_2.RecordText, tblBlogs_2.CDate, tblBlogs_2.UserID, COUNT(dbo.tblBlogComments.CommentID) AS Comments
FROM (
SELECT TOP (150) RecordID, RecordText, CDate, UserID
FROM dbo.tblBlogs AS tblBlogs_1
ORDER BY RecordID DESC
) AS tblBlogs_2
LEFT OUTER JOIN dbo.tblBlogComments ON tblBlogs_2.RecordID = tblBlogComments.RecordID
GROUP BY tblBlogs_2.RecordID, tblBlogs_2.RecordText, tblBlogs_2.CDate, tblBlogs_2.UserID
) AS tblBlogs
INNER JOIN dbo.tblUsers ON tblBlogs.UserID = tblUsers.UserID
LEFT OUTER JOIN dbo.tblUserpics ON tblBlogs.UserID = tblUserpics.UserID
ORDER BY tblBlogs.CDate DESC
This must select top 150 ros from Blogs table with User details + Comments for every single Blog entry.
SELECT f.ForumID, f.ForumName, t.ThreadName, m.MsgID, m.MsgName, m.MsgBody, m.UserID, m.CDate, m.IP, u.Username, tblCities.CityName,
t.IsClosed, ISNULL(u.Msgs, 0) AS Posts, ISNULL(tblUserpics.UserpicName, '') AS UserpicName, t.IsPoll,
t.IsPollMultiple, ISNULL(u.Crashes, 0) AS Crashes, 0 AS LastMsgID, m.IsFlood, ISNULL(u.RepaGood, 0) AS RepaGood, ISNULL(u.RepaBad, 0)
AS RepaBad, ISNULL(dbo.vMsgsRepaGood.RepaGood, 0) AS MsgRepaGood, ISNULL(dbo.vMsgsRepaBad.RepaBad, 0) AS MsgRepaBad, t.ThreadID,
tblUserPrivateStatuses.StatusName AS PrivateStatus
FROM tblMsgs AS m
INNER JOIN tblThreads AS t ON m.ThreadID = t.ThreadID
INNER JOIN tblForums AS f ON t.ForumID = f.ForumID
INNER JOIN tblUsers AS u ON m.UserID = u.UserID
LEFT OUTER JOIN tblUserPrivateStatuses ON u.UserID = dbo.tblUserPrivateStatuses.UserID
LEFT OUTER JOIN tblCities ON u.CityID = dbo.tblCities.CityID
LEFT OUTER JOIN tblUserpics ON u.UserID = dbo.tblUserpics.UserID
LEFT OUTER JOIN vMsgsRepaGood ON m.MsgID = vMsgsRepaGood.MsgID
LEFT OUTER JOIN vMsgsRepaBad ON m.MsgID = vMsgsRepaBad.MsgID
WHERE m.ThreadID = "& ThreadID & " AND IsFlood = 0
GROUP BY f.ForumID, f.ForumName, t.ThreadName, m.MsgID, m.MsgName, m.MsgBody, m.UserID, m.CDate, m.IP, u.Username, tblCities.CityName, t.IsClosed, u.Msgs, dbo.tblUserpics.UserpicName, t.IsPoll, t.IsPollMultiple, u.Crashes, m.IsFlood, u.RepaGood, u.RepaBad, vMsgsRepaGood.RepaGood, vMsgsRepaBad.RepaBad, t.ThreadID, tblUserPrivateStatuses.StatusName
ORDER BY m.CDate</pre>
This query selects all not flood messages from specific Thread from specific Forum with User details (Registration date, number of good/bad reputation, number of crashes, number of post on this whole forum, city, userpic).
Or maybe somebody can tell me about free tools for optimizing queries and creating indexes?
There is a lot to talk about here, and without significantly more information, it's going to be impossible for anyone to help with your query fully.
Precaution: If you have a DBA for your system, check with them before indexing anything, especially on a live system. They can even help, if you're nice to them. If the system is used by many others, be careful before changing anything like indexes.
A basic tip on indexing, if you don't want to dive deep into the problem, is: index by the following, in this order:
Join predicates
Filter
Order by / Group By / etc.
Also:
Make sure whatever columns possible are non-null.
Use data types that make sense - store nothing as varchar if it's an integer or date. (Column width matters. Use the smallest data type you can, if possible.)
Make sure your joins are the same data type - int to int, varchar to varchar, and so on.
If possible, use unique, non-null indexes on each join predicate in each tables.
Do all of this, and you'll be well on your way. But if you need this stuff regularly, learn it! There is a lot out there, and it is a deep topic, but you can make queries MUCH better if you know what you are doing.
Edit: The syntax for building indexes is here: How do I index a database column. The How/Why is here: How does database indexing work?
I have a real mindbender of a MySQL problem which I am now thinking there is no answer to. Please help me, you are my only hope!
Stripping it down to the basics, I have two tables, "People" and "Activity". It is possible (long story and lots of data involved) for these two tables to be joined by two different relationship tables: people_activity and entity_activity
I need to do a query on the activity table which gets the people record/s linked to activity records based on both relationship tables.
This is what I have, but it is massively slow on lots of data:
select * from activity
left join peopleactivity on peopleactivity.activityid = activity.activityid
left join entityactivity on entityactivity.activityid = activity.activityid
left join people on (peopleactivity.peopleid = people.peopleid OR
entityactivity.entityid = people.peopleid)
Some more notes - I have also tried creating a view to combine the results of the two relationship tables and instead joining people and activity via this view. This also works, but is also still massively slow
Changing how the relationship/s work to consolodate to one table is a major headache
I have also tried a union -like this -
select * from activity
left join peopleactivity on peopleactivity.activityid = activity.activityid
left join people on (peopleactivity.peopleid = people.peopleid)
union
select * from activity
left join peopleactivity on peopleactivity.activityid = activity.activityid
left join people on (entityactivity.entityid= people.peopleid)
which also works, but for other reasons causes me problems. I really need to do this in one query without changing too much underlying.
Has anyone got any super amazing ideas that I have missed??!
You may try to replace OR with IN
left join people on people.peopleid IN (peopleactivity.peopleid, entityactivity.entityid)
1.) Try setting the id of the tables as the primary key on each table
2.) Use inner joins instead of left joins. Not sure why you are using left joins here as you will get all the results of the other tables left joined on the activity table and get basically all records whether or not they have a join value in another table. I think this might also help you. Can you post a describe of your tables.
I think you should keep the UNION query but making those INNER joins. Do you really need LEFT joins?
You could also change it into UNION ALL, which will have some performance gain:
SELECT activity.*, people.*, 'PA' AS joining_table
FROM activity
JOIN peopleactivity ON peopleactivity.activityid = activity.activityid
JOIN people ON peopleactivity.peopleid = people.peopleid
UNION ALL
SELECT activity.*, people.*, 'EA'
FROM activity
JOIN entityactivity ON entityactivity.activityid = activity.activityid
JOIN people ON entityactivity.entityid = people.peopleid
Thanks for the comments. I had tried various incarnations of the above. My answer was to set up a new table, copy all the existing links into that table, and then use triggers to add/remove links to that table whenever the links were added removed in the two separate link tables. This works well and also allows me to use indexes on this new table to keep things nice and snappy. Many thanks for those that took the time to post the ideas though!
Let's say that I have these tables on my db:
items
categories
weapons
shields
swords
And I need to create a join like this:
SELECT items.*, {swords}.*
FROM items
INNER JOIN categories
ON categories.id = items.category_id # <-- here I know that the item is a sword
INNER JOIN {swords}
ON {swords}.item_id = item.id
WHERE items.id = 12
But the {swords} part is dynamic since I found that an item is a sword checkgin the categories.name field.
The query will change if the categories.name is "shield" to this:
SELECT items.*, shields.*
FROM items
INNER JOIN categories
ON categories.id = items.category_id
INNER JOIN shields
ON shields.item_id = item.id
WHERE items.id = 13
I used { and } around the swords to show it like a variable
Thank you for your answer and sorry about my english! :)
Technically, you can only do this with dynamic SQL - which means MySQL's Prepared Statement syntax, or string concatenation to create the query prior to submitting it to the database. Prepared Statements are the preferable choice, due to better SQL injection defense.
You could use LEFT JOINs to the various tables, but it would mean including numerous columns from each of the tables (weapons, shields, swords...) that would be null if the category didn't match. It'd be horrible to try to pull out data from a single query...
PS: Why is there a SWORD table? Wouldn't that be covered by the WEAPON table?
I have database with schema on picture below and I need to select everything related to one row (one id) of [letaky]. That means the related [zamestnanci], every related [obsah] and every [knihy] in it.
This is the first time i used relations in database and i have no idea how to make such a select.
Use JOIN ... ON:
SELECT *
FROM zamestnanci
JOIN lekaty ON lekaty.zamestnanciid = zamestnanci.id
JOIN obsah ON obsah.idletaku = lekaty.id
JOIN knihy ON knihy.id = obsah.idknihy
WHERE letaky.id = 123
You may also want to consider whether you need INNER JOIN, LEFT JOIN or RIGHT JOIN for each of these joins. The difference between these JOINs is described in many other questions on StackOverflow, for example this one:
SQL Join Differences