I am creating a user profile plugin in Joomla 2.5 to extend the standard user fields.
The issue is I need an efficient way to retrieve the users (along with the extra fields) via a MySQL query. Currently the best method I can think of is querying the #__user_profiles table and processing that data to determine the extra fields to load and then producing a query that creates a separate join on the #__user_profiles table for each extra user field.
Obviously that isn't very efficient and on a large user base the query is quite slow.
Is there a better way to combine extra user fields that are separate records in another table into one query?
EDIT:
I have an external script that needs to grab all the users and their extended fields so I need to combine the #_users and #_user_profiles tables
This is simply a join between the two tables
select u.*, p.*
from #__users u
left join #__user_profiles p on p.user_id = u.id
This retrieves all user and associated profile entries.
Depending on what rows or profile entries you really need, you can restrict this query with an additional where clause.
If you want the user and associated profile entries in one row, you can combine the profile entries with group_concat
select u.*, group_concat(p.profile_key, '=', p.profile_value)
from #__users u
left join #__user_profiles p on p.user_id = u.id
group by u.id
In the end I decided to go for a pivot table approach
example:
SELECT
#__users.*,
MAX(CASE WHEN profile_key = "example.firstname" THEN profile_value END) AS FirstName,
MAX(CASE WHEN profile_key = "example.lastname" THEN profile_value END) AS LastName,
MAX(CASE WHEN profile_key = "example.company" THEN profile_value END) AS Company
FROM #__users LEFT JOIN #__user_profiles ON #__users.id = #__user_profiles.user_id
GROUP BY #__users.id
This allows me to have all the extra user data in one row so I can easily order and filter.
Related
I have a products table where I include 3 columns, created_user_id, updated_user_id and in_charge_user_id, all of which are related to my user table, where I store the id and name of the users.
I want to build an efficient query to obtain the names of the corresponding user_id's.
The query that I build so far is the following:
SELECT products.*,
(SELECT name FROM user WHERE user_id = products.created_user_id) as created_user,
(SELECT name FROM user WHERE user_id = products.updated_user_id) as updated_user,
(SELECT name FROM user WHERE user_id = products.in_charge_user_id) as in_charge_user
FROM products
The problem with this query is that if I have 30,000 records, I am executing 3 more queries per row.
What would be a more efficient way of achieving this? I am using mysql.
For each type of user id (created, updated, in_charge) you would JOIN the users table once:
SELECT
products.*,
u1.username AS created_username,
u2.username AS updated_username,,
u3.username AS in_charge_username,
FROM products
JOIN user u1 ON products.created_user_id = u1.user_id
JOIN user u2 ON products.updated_user_id = u2.user_id
LEFT JOIN user u3 ON products.in_charge_user_id = u3.user_id
This is the best practice method to obtain the data.
It is similiar to your query with sub-selects but a more modern approach which I think the database can optimize and utilize better.
Important:
You need foreign key index on all the user_id fields in both tables!
Then the query will be very fast no matter how many rows are in the table. This requires an engine which supports foreign keys, like InnoDB.
LEFT JOIN or INNER JOIN ?
As the other answers suggest a LEFT JOIN, I would not do a left join.
If you have an user id in the products table, there MUST be a linked user_id in the user table, except for the in_charge_user which is only present some times. If not, the data would be semantically corrupt. The foreign keys assure that you always have a linked user_id and a user_id can only be deleted when there is no linked product left.
JOIN is equivalent to INNER JOIN.
You can use LEFT JOIN instead of subselects.
Your query should be like:
SELECT
P.*,
[CU].[name],
[UU].[name],
[CU].[name]
FROM products AS [P]
LEFT JOIN user AS [CU] ON [CU].[user_id] = [P].[created_user_id]
LEFT JOIN user AS [UU] ON [UU].[user_id] = [P].[updated_user_id]
LEFT JOIN user AS [CU] ON [CU].[user_id] = [P].[in_charge_user_id]
First, your query should be fine. You only need an index on user(user_id) or better yet user(user_id, name) for performance. I imagine that the first exists.
Second, you can write this using LEFT JOIN:
SELECT p.*, uc.name as created_user,
uu.name as updated_user, uin.name as in_charge_user
FROM products p LEFT JOIN
user uc
ON uc.user_id = p.created_user_id LEFT JOIN
user uu
ON uu.user_id = p.updated_user_id LEFT JOIN
user uin
ON uin.user_id = p.in_charge_user_id;
With one of the above indexes, the two methods should have very similar performance.
Also note the use of LEFT JOIN. This handles the case where one or more of the user ids is missing.
Try this below query
SELECT products.*, c.name as created_user,u.name as updated_user,i.name as in_charge_user
FROM products left join user c on(products.created_user_id=c.user_id ) left join user u on(products.updated_user_id=u.user_id ) left join user u on(products.in_charge_user_id=i.user_id )
Also as Gordon Linoff mentioned create index on user table will fetch your data faster.
I currently am trying to join two tables but prevent duplication of information from one of the tables.
The user's table has 4 columns, uid, name, email and status.
The stats table has 4 columns, uid, date, follows, views
What I would like to be able to do is pull every record from the stats table and only the name, email and status values from the user table. The issue I have with the below SQL is that it duplicates the data from the user table, is there a way around this?
SELECT u.name
, u.email
, u.status
, s.date
, s.follows
, s.views
FROM users u
JOIN stats s
ON u.id = s.uid
WHERE name = :name
If you don't want every matching stats row to be accompanied by its matching users row, then you have to run two queries:
SELECT u.id, u.name, u.email, u.status FROM users u WHERE name = :name
Note the result of u.id because you'll use it as a parameter for the second query:
SELECT s.date, s.follows, s.views FROM stats s WHERE s.uid = :uid
You have to understand that the relational model works because every query result is itself a relation. The matching data is returned in every row, and this is what allows JOIN to be part of an algebra, where the result can be used as the operand of another JOIN, or a GROUP BY, or some other relational operation.
You should read SQL and Relational Theory: How to Write Accurate SQL Code by C. J. Date.
I have a schema with table
user (username, password, fullname, usertype)
there are 4 types of users and for each there is a table with additional attributes for specific type:
individual(username (FOREIGN), education, work_since)
corporation*(username (FOREIGN), headquarters, office, num_employees)
and a couple more...
there could be only 1 record in all the additional tables for user.
I need to display all of the user information from user table and additional attribute table based on user type.
The first thing that came to mind was to first query user table, and then, based on type returned, query one of the related tables... but that would be too many queries, so I was wondering, is it possible to do it in a single query?
Use left join:
select u.*,
(case when i.username is not null then 'individual'
when c.username is not null then 'corporation'
end) as usertype,
i.education, i.work_since,
c.headquarters, c.office, c.num_employees
from users u left join
individual i
on i.username = u.username left join
corporation c
on c.username = u.username;
I manage a property website. I have a table with banned users (small table) and a table called advert_views which keeps track of each listing that each user views (currently 1.3m lines and growing). The advert_views table alsio takes note of the IP address for every advert viewed).
I want to get the IP addresses used by the banned users and check if any of these banned users have opened new accounts. I ran the following query:
SELECT adviews.user_id AS 'banned user_id',
adviews.client_ip AS 'IPs used by banned users',
adviews2.user_id AS 'banned users that opened a new account'
FROM banned_users
LEFT JOIN users on users.email_address = banned_users.email_address #since I don't store the user_id in banned_users
LEFT JOIN advert_views adviews ON adviews.user_id = users.id AND adviews.user_id IS NOT NULL # users may view listings when not logged in but they have restricted access to the information on the listing
LEFT JOIN (SELECT client_ip,
user_id
FROM advert_views
WHERE user_id IS NOT NULL
) adviews2
ON adviews2.client_ip = adviews.client_ip
WHERE banned_users.rec_status = 1 and adviews.user_id <> adviews2.user_id
GROUP BY adviews2.user_id
I applied an index on the advert_views table and the users table as per below:
enter image description here
My query takes half an hour to execute. Is there a way how to improve my query speed?
Thanks!
Chris
First of all: Why do you outer join the tables? Or better: Why do you try to outer join the tables? A left join is meant to get data from a table even when there is no match. But then your results could contain rows with all values null. (That doesn't happen though, because adviews.user_id <> adviews2.user_id in your where clause dismisses all outer-joined rows.) Don't give the DBMS more work to do than necessary. If you want inner joins, then don't outer join. (Though the difference in execution time won't be huge.)
Next: You select from banned_users, but you only use it to check existence. You shouldn't do this. Use an EXISTS or IN clause instead. (This is mainly for readability and in order not to produce duplicate results. This probably won't speed things up.)
SELECT av1.user_id AS 'banned user_id',
av2.client_ip AS 'IPs used by banned users',
av2.user_id AS 'banned users that opened a new account'
FROM adviews av1
JOIN adviews av2 ON av2.client_ip = av1.client_ip AND av2.user_id <> av1.user_id
WHERE av1.user_id IN
(
SELECT user_id
FROM users
WHERE email_address IN (select email_address from banned_users where rec_status = 1)
)
GROUP BY av2.user_id;
You may replace the inner IN clause with a join. It's mostly a matter of personal preference, but it is also that in the past MySQL sometimes didn't perform well on IN clauses, so many people made it a habit to join instead.
WHERE av1.user_id IN
(
SELECT u.user_id
FROM users u
JOIN banned_users bu ON bu.email_address = u.email_address
WHERE bu.rec_status = 1
)
At last consider removing the GROUP BY clause. It reduces your results to one row per reusing user_id, showing one of its related banned user_ids (arbitrarily chosen in case there is more than one). I don't know your tables. Are you getting many records per reusing user_id? If not, remove the clause.
As to indexes I suggest:
banned_users(rec_status, email_address)
users(email_address, user_id)
adviews(user_id, client_ip)
adviews(client_ip, user_id)
Suppose I have 3 different tables relationships as following
1st is tbl_users(id,gender,name)
2nd is tbl_feeds(id,user_id,feed_value)
3rd is tbl_favs(id,user_id,feed_id)
where id is primary key for every table.
Now suppose I want to get data where those feeds should come which is uploaded by Gender=Male users with one field in every row that should say either the user who is calling this query marked that particular feed as favourite or not.
So final data of result should be like following :
where lets say the person who is calling this query have user_id=2 then is_favourite column should contain 1 if that user marked favourite that particular feed otherwise is_favourite should contain 0.
user_id feed_id feed_value is_favourite gender
1 2 xyz 1 M
2 3 abc 0 M
3 4 mno 0 M
I hope you getting my question , I m able to get feeds as per gender but problem is I m facing problem to get is_favourite flag as per particular user for every feed entry.
I hope some one have these problem before and I can get help from those for sure.
I would be so thankful if some one can resolve my this issue.
Thanks
Something like this should work:
SELECT
u.id AS user_id.
fe.id AS feed_id,
fe.feed_value,
IFNULL(fa.is_favourite, 0),
u.gender
FROM
tbl_users u
JOIN
tbl_feeds fe ON (fe.user_id = u.id)
LEFT JOIN
tbl_favs fa ON (
fa.user_id = u.id
AND
fa.feed_id = fe.id
)
In order to link your tables, you need to find the most common link between them all. This link is user_id. You'll want to create a relationship between all tables with JOIN in order to make sure each and every user has data.
Now I don't know if you're planning on making sure all tables have data with the user_id. But I would use INNER JOIN as it will ONLY show records of that user_id without nulls. If the other tables could POSSIBLY (Not always guaranteed) you should use a LEFT JOIN based on the tables that is it possible with.
Here is an SQLFiddle as an example. However, I recommend you name your ID fields as appropriate to your table's name so that way, there is no confusion!
To get your isFavorite I would use a subquery in order to validate and verify if the user has it selected as a favorite.
SELECT
u.userid,
u.gender,
f.feedsid,
f.feedvalue,
(
SELECT
COUNT(*)
FROM
tbl_favs a
WHERE
a.userid = u.userid AND
a.feedsid = f.feedsid
) as isFavorite
FROM
tbl_users u
INNER JOIN
tbl_feeds f
ON
u.userid = f.userid
~~~~EDIT 1~~~~
In response to your comment, I have updated the SQLFiddle and the query. I don't believe you really need a join now based on the information given. If you were to do a join you would get unexpected results since you would be trying to make a common link between two tables that you do not want. Instead you'll want to just combine the tables together and do a subquery to determine from the favs if it is a favorite of the user's.
SQLFiddle:
SELECT
u.userid,
f.feedsid,
u.name,
u.gender,
f.feedvalue,
(
SELECT
COUNT(*)
FROM
tbl_favs a
WHERE
a.userid = u.userid AND
a.feedsid = f.feedsid
) as isFavorite
FROM
tbl_users u,
tbl_feeds f
ORDER BY
u.userid,
f.feedsid