I am creating an application in PHP of Power Meter Analysis. I have following table structure:
table: 'feeds'
| feed_id | device_no | current1 | voltage1 | power_factor_1 | vc1 | ic1 | date_added
-------------------------------------------------------------------------------------
| 36752 | 2 | 36.048 | 196.01 | 0.9 | 1 | 1 | 2014-06-23 14:14:44
| 36753 | 2 | 35.963 | 195.59 | 0.9 | 1 | 1 | 2014-06-23 14:15:34
and so on.
table: 'machine'
| machine_id | machine_phone | machine_name | company_id |
----------------------------------------------------------
| 1 | 2 | ABC Machine | 1 |
| 2 | 093 | DEF Machine | 1 |
I need records on hourly basis and I have written the following query for this purpose:
$sql = "
SELECT
SUM(t.power1) AS 'power1'
, HOUR(t.date) AS 'pulse_hour'
FROM (
SELECT
IF(#diff = 0, 0, (((f.voltage1*f.vc1)*(f.current1*f.ic1)*(f.power_factor_1))/1000) * (#diff/3600)) AS 'power1'
, IF(#diff = 0,0, TIME_TO_SEC(f.date_added) - #diff) AS 'deltaT'
, #diff := TIME_TO_SEC(f.date_added)
, f.date_added AS 'date'
FROM
feeds f,
(SELECT #diff := 0) AS X
left join
machine m
on
f.device_no = m.machine_phone
left join
company c
on
c.company_id = m.company_id
";
$sql .= $params['machine_id'] ? " where f.device_no = '".$params['machine_id']."'" : " where f.device_no > 0";
$sql .= $params['machine_pulse_datetime_from'] ? " and f.date_added >= '".$params['machine_pulse_datetime_from']."'" : "";
$sql .= $params['machine_pulse_datetime_to'] ? " and f.date_added <= '".$params['machine_pulse_datetime_to']."'" : "";
$sql .= $params['company_id'] ? " and c.company_id = '".$params['company_id']."'" : "";
$sql .= "
ORDER BY
f.date_added ASC
) t
GROUP BY HOUR(t.date)
ORDER BY HOUR(t.date) ASC
";
The query is running OK if I remove the following part from the query:
left join
machine m
on
f.device_no = m.machine_phone
left join
company c
on
c.company_id = m.company_id
But with this part It is giving me following error:
Error Code : 1054
Unknown column 'f.device_no' in 'on clause'
can you please help me to sort this out... I have spent an hour with this query :(
This is your join:
FROM feeds f,
(SELECT #diff := 0) AS X left join
machine m
on f.device_no = m.machine_phone left join
company c
on c.company_id = m.company_id
The problem is that you are mixing explicit and implicit joins. You can fix this by replacing the comma with cross join:
FROM feeds f cross join
(SELECT #diff := 0) AS X left join
machine m
on f.device_no = m.machine_phone left join
company c
on c.company_id = m.company_id
The issue, which is buried deep in the documentation for select, is that , is a lot like a cross join with the exception of scoping rules -- that is, when the table aliases are recognized. With a comma, the table aliases are not recognized as you expect. They are with a cross join.
Here is the reference:
INNER [CROSS] JOIN and , (comma) are semantically equivalent in the
absence of a join condition: both produce a Cartesian product between
the specified tables (that is, each and every row in the first table
is joined to each and every row in the second table).
However, the precedence of the comma operator is less than of INNER
JOIN, CROSS JOIN, LEFT JOIN, and so on. If you mix comma joins
with the other join types when there is a join condition, an error of
the form Unknown column 'col_name' in 'on clause' may occur.
Information about dealing with this problem is given later in this
section.
Related
I have simple but long query which count the content of the result it takes about 14 seconds. the count itself on the main table takes less than a second but after multiple join the delay is too high as follow
Select Count(Distinct visits.id) As Count_id
From visits
Left Join clients_locations ON visits.client_location_id = clients_locations.id
Left Join clients ON clients_locations.client_id = clients.id
Left Join locations ON clients_locations.location_id = locations.id
Left Join users ON visits.user_id = users.id
Left Join potentialities ON clients_locations.potentiality = potentialities.id
Left Join classes ON clients_locations.class = classes.id
Left Join professions ON clients.profession_id = professions.id
Inner Join specialties ON clients.specialty_id = specialties.id
Left Join districts ON locations.district_id = districts.id
Left Join provinces ON districts.province_id = provinces.id
Left Join locations_types ON locations.location_type_id = locations_types.id
Left Join areas ON clients_locations.area_id = areas.id
Left Join calls ON calls.visit_id = visits.id
The output of explain is
+---+---+---+---+---+---+---+---+---+---+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+---+---+---+---+---+---+---+---+---+---+
| 1 | SIMPLE | specialties | index | PRIMARY | specialty_name | 52 | NULL | 53 | Using index |
| 1 | SIMPLE | clients | ref | PRIMARY,specialty | specialty | 4 | crm_db.specialties.id | 143 | |
| 1 | SIMPLE | clients_locations | ref | PRIMARY,client_id | client_id | 4 | crm_db.clients.id | 1 | |
| 1 | SIMPLE | locations | eq_ref | PRIMARY | PRIMARY | 4 | crm_db.clients_locations.location_id | 1 | |
| 1 | SIMPLE | districts | eq_ref | PRIMARY | PRIMARY | 4 | crm_db.locations.district_id | 1 | Using where |
| 1 | SIMPLE | visits | ref | unique_visit,client_location_id | unique_visit | 4 | crm_db.clients_locations.id | 4 | Using index |
| 1 | SIMPLE | calls | ref | call_unique,visit_id | call_unique | 4 | crm_db.visits.id | 1 | Using index |
+---+---+---+---+---+---+---+---+---+---+
Update 1
The above query used with dynamic where statement $sql = $sql . "Where ". $whereFilter but the i submitted it in simple form . So do not consider the answer just eleminate the joins :)
Update 2
Here is example of dynamic filtering
$temp = $this->province_id;
if ($temp != null) {
$whereFilter = $whereFilter . " and provinces.id In ($temp) ";
}
But in startup case which is our case no where statement
Left joins always return a row from the first table, but may return multiple rows if there are multiple matching rows. But because you are counting distinct visit rows, left joining to another table while counting distinct visits is the same as just counting the rows of visits. Thus the only joins that affect the result are inner joins, so you can remove all "completely" left joined tables without affecting the result.
What I mean by "completely" is that some left joined tables are effectively inner joined; the inner join to specialty requires the join to clients to succeed and thus also be an inner join, which in turn requires the join to clients_locations to succeed and thus also be an inner join.
Your query (as posted) can be reduced to:
Select Count(Distinct visits.id) As Count_id
From visits
Join clients_locations ON visits.client_location_id = clients_locations.id
Join clients ON clients_locations.client_id = clients.id
Join specialties ON clients.specialty_id = specialties.id
Removing all those unnecessary joins will however greatly improve the runtime of your query, not only because there are less joins to make but also because the resulting rowset size could be enormous when you consider that the size is the product of the matches in all the tables (not the sum.
For maximum performance, create a covering indexes on all id-and-fk columns:
create index visits_id_client_location_id on visits(id, client_location_id);
create index clients_locations_id_client_id on clients_locations(id, client_id);
create index clients_id_specialty_id on clients(id, specialty_id);
so index-only scans can be used where possible. I assume there are indexes on the PK columns.
You don't seem to have any (or much) intentional filtering. If you want to know the number of visits referred to in calls, I would propose:
select count(distinct c.visit_id)
from calls c;
in order to optimize the whole process you can dynamically construct the pre-where SQL according to the filters you are going to apply. Like:
// base select and left join
$preSQL = "Select Count(Distinct visits.id) As Count_id From visits ";
$preSQL .= "Left Join clients_locations ON visits.client_location_id = clients_locations.id ";
// filtering by province_id
$temp = $this->province_id;
if ($temp != null) {
$preSQL .= "Left Join locations ON clients_locations.location_id = locations.id ";
$preSQL .= "Left Join districts ON locations.district_id = districts.id ";
$preSQL .= "Left Join provinces ON districts.province_id = provinces.id ";
$whereFilter = "provinces.id In ($temp) ";
}
$sql = $preSQL . "Where ". $whereFilter;
// ...
If you are using multiple filters you can put all inner/left-join strings in an array and then after analysing the request, you can construct your $preSQL using the minimum of joins.
Use COUNT(CASE WHEN visit_id!="" THEN 1 END) as visit.
Hope this will help
Isn't it just:
SELECT COUNT(id)
FROM visits
because all the left outer joins also return a visits.id when theres no matching clients, ..., calls and id's ought to be unique?
Different hint: The one inner join also is only effective when a client exists. Generally when needing inner joins they must be put as high/near as possible to the source table, so in your example it would have been best in the line after "left join clients".
I didn't understand too much your idea, specially your INNER JOIN that will tranform some LEFT in INNER JOINs, it seems strange, but lets try a solution:
Usually the LEFT JOINs has a very bad performance, and I think you'll need them only if you'll use them in WHERE clause, then you can include them with INNER JOIN only if you'll use them.
For example:
$query = "Select Count(Distinct visits.id) As Count_id From visits ";
if($temp != null){
$query .= " INNER JOIN clients_locations ON visits.client_location_id = clients_locations.id ";
$query .= " INNER JOIN locations ON clients_locations.location_id = locations.id ";
$query .= " INNER JOIN locations ON clients_locations.location_id = locations.id ";
$query .= " INNER JOIN districts ON locations.district_id = districts.id "
$query .= " INNER JOIN provinces ON districts.province_id = provinces.id ";
$whereFilter .= " and provinces.id In ($temp) ";
}
I think it'll help your performance and it'll works as you need.
What's wrong here ? i just want to display all the item in item_tb with 2 different group , vicma and branch but it returns nothing. It only works in one inner join but when i join the other one it display nothing.
|-------------|-------------------------|---------------|
|item_tb | vicma_tb | branch_tb |
| | vID - PK | id-PK |
|branchID-FK | | |
|vicma - FK | | |
|-------------|-------------------------|---------------|
$sql = "
SELECT item_tb.*
, branch_tb.*
, vicma_tb.*
from item_tb
JOIN branch_tb
on item_tb.branchID = branch_tb.id
JOIN vicma_tb
on item_tb.vicma = vicma_tb.vID ";
Seems like you need to do a LEFT JOIN instead of INNER JOIN. LEFT JOIN will return all values from your original table and NULL if there is no match. Try:
SELECT item_tb.*, branch_tb.* , vicma_tb.* from item_tb
LEFT JOIN branch_tb on item_tb.branchID = branch_tb.id
LEFT JOIN vicma_tb on item_tb.vicma = vicma_tb.vID
Ok I have a few tables tables. I am only showing relevant fields:
items:
----------------------------------------------------------------
name | owner_id | location_id | cab_id | description |
----------------------------------------------------------------
itm_A | 11 | 23 | 100 | Blah |
----------------------------------------------------------------
.
.
.
users:
-------------------------
id | name |
-------------------------
11 | John |
-------------------------
.
.
.
locations
-------------------------
id | name |
-------------------------
23 | Seattle |
-------------------------
.
.
.
cabs
id | location_id | name
-----------------------------------
100 | 23 | Cool |
-----------------------------------
101 | 24 | Cool |
-----------------------------------
102 | 24 |thecab |
-----------------------------------
I am trying to SELECT all items (and their owner info) that are from Seattle OR Denver, but if they are in Seattle they can only be in the cab NAMED Cool and if they are in Denver they can only be in the cab named 'thecab' (not Denver AND cool).
This query doesn't work but I hope it explains what I am trying to accomplish:
SELECT DISTINCT
`item`.`name`,
`item`.`owner_id`,
`item`.`description`,
`user`.`name`,
IF(`loc`.`name` = 'Seattle' AND `cab`.`name` = 'Cool',1,0) AS `cab_test_1`,
IF(`loc`.`name` = 'Denver' AND `cab`.`name` = 'thecab',1,0) AS `cab_test_2`,
FROM `items` AS `item`
LEFT JOIN `users` AS `user` ON `item`.`owner_id` = `user`.`id`
LEFT JOIN `locations` AS `loc` ON `item`.`location_id` = `loc`.`location_id`
LEFT JOIN `cabs` AS `cab` ON `item`.`cab_id` = `cabs`.`id`
WHERE (`loc`.`name` IN ("Seattle","Denver")) AND `cab_test_1` = 1 AND `cab_test_2` = 1
I'd rather get rid of the IFs is possible. It seems inefficent, looks clunky, and is not scalable if I have a lot of location\name pairs
Try this:
SELECT DISTINCT
item.name,
item.owner_id,
item.description,
user.name
FROM items AS item
LEFT JOIN users AS user ON item.owner_id = user.id
LEFT JOIN locations AS loc ON item.location_id = loc.id
LEFT JOIN cabs AS cab ON item.cab_id = cabs.id
WHERE ((loc.name = 'Seattle' AND cab.name = 'Cool')
OR (loc.name = 'Denver' AND cab.name = 'thecab'))
My first thought is to store the pairs of locations and cab names in a separate table. Well not quite a table, but a derived table generated by a subquery.
You still have the problem of pivoting the test results into separate columns. The code can be simplified by making use of mysql boolean expressions, which get rid of the need for a case or if.
So, the approach is to use the same joins you have (although left join is not needed because the comparison on cab.name turns them in to inner joins). Then add a table of the pairs you are looking for, along with the "test name" for the pair. The final step is an explicit group by and a check whether conditions are met for each test:
SELECT i.`name`, i.`owner_id`, i.`description`, u.`name`,
max(pairs.test_name = 'test_1') as cab_test_1,
max(pairs.test_name = 'test_2') as cab_test_2
FROM `items` i LEFT JOIN
`users` u
ON i.`owner_id` = u.`id` LEFT JOIN
`locations` l`
ON i.`location_id` = l.`location_id` left join
`cabs` c
ON i.`cab_id` = c.`id` join
(select 'test_1' as testname, 'Seattle' as loc, 'cool' as cabname union all
select 'test_2', 'Denver', 'thecab'
) pairs
on l.name = pairs.name and
l.cabname = c.name
group by i.`name`, i.`owner_id`, i.`description`, u.`name`;
To add in additional pairs, add them into the pairs table along, and add an appropriate line in the select for the test flag.
By using COALESCE, I can create a temporary variable called comment_votes like so:
SELECT comments.*, COALESCE(rs_reputations.value, 0) AS comment_votes FROM `comments`
LEFT JOIN rs_reputations ON comments.id = rs_reputations.target_id AND
rs_reputations.target_type = 'Comment' AND rs_reputations.reputation_name =
'comment_votes' AND rs_reputations.active = 1 WHERE (impression_id = 1)
I want to create a second variable called impression_votes in the came query. I attempted to do this with:
SELECT comments.*, COALESCE(rs_reputations.value, 0) AS comment_votes
FROM 'comments'
LEFT JOIN rs_reputations ON
comments.id = rs_reputations.target_id AND
rs_reputations.target_type = 'Comment' AND
rs_reputations.reputation_name = 'comment_votes' AND
rs_reputations.active = 1
SELECT comments.*, COALESCE(rs_reputations.value, 0) AS impression_votes
FROM 'comments'
LEFT JOIN rs_reputations ON
comments.id = rs_reputations.target_id AND
rs_reputations.target_type = 'Comment' AND
rs_reputations.reputation_name = 'impression_votes' AND
rs_reputations.active = 1
WHERE
This leads to the error:
You have an error in your SQL syntax
Is what I'm attempting even possible? If so, I seem to be bridging the two SELECT/COALESCE statements improperly. How should I write this?
The MySQL COALESCE function is actually an inbuilt function that returns the first non-null value - it's not a variable, it's a function that is actually supported across a wide variety of database systems.
For example, with the following table:
| Id | Name | Counter |
| 1 | lolcat | NULL |
| 2 | codez | 1 |
The sql statement:
SELECT Id, Name, COALESCE(counter, 0) AS NonNullCounter FROM table
will return the results:
| Id | Name | NonNullCounter |
| 1 | lolcat | 0 |
| 2 | codez | 1 |
In this instance, the NULL value has been replaced by 0.
This is useful for you as, if you don't yet have any matching rows in rs_reputations for the row in comments, the LEFT JOIN will return NULL for the column rs_repuations.value, which is then replaced by 0 by COALESCE.
If you are new to JOINs then there is a great visual guide by Jeff Atwood.
Your first query can is actually:
SELECT comments.*,
COALESCE(rs_reputations.value, 0) AS comment_votes
FROM comments
LEFT JOIN rs_reputations ON comments.id = rs_reputations.target_id
AND rs_reputations.reputation_name = 'comment_votes'
WHERE impression_id = 1;
CHOICE 1 - UNION
You have a couple of choices - you can either UNION your results together like this:
SELECT comments.*,
COALESCE(rs_reputations.value, 0) AS votes,
'comment_votes' AS vote_type
FROM comments
LEFT JOIN rs_reputations ON comments.id = rs_reputations.target_id
AND rs_reputations.reputation_name = 'comment_votes'
WHERE impression_id = 1
UNION
SELECT comments.*,
COALESCE(rs_reputations.value, 0) AS votes,
'impression_votes' as vote_type
FROM comments
LEFT JOIN rs_reputations ON comments.id = rs_reputations.target_id
AND rs_reputations.reputation_name = 'impression_votes'
WHERE impression_id = 1;
In this instance your results will look like this:
|comments_columns|votes|vote_type |
| * |12 |comment_vote |
| * |2 |impression_vote |
CHOICE 2 - JOIN ON TO THE SAME TABLE TWICE
Or you can self join onto the same table twice by using the same table name but a different alias:
SELECT comments.*,
COALESCE(CommentRep.value, 0) AS comment_votes,
COALESCE(ImpressionRep.value, 0) AS impression_votes,
FROM comments
LEFT JOIN rs_reputations AS CommentRep ON comments.id = CommentRep.target_id
AND CommentRep.reputation_name = 'comment_votes'
LEFT JOIN rs_reputations AS ImpressionRep ON comments.id = ImpressionRep.target_id
AND ImpressionRep.reputation_name = 'impression_votes'
WHERE CommentRep.impression_id = 1
AND ImpressionRep.impression_id = 1
In this instance your results will look like this:
|comments_columns|comment_votes|impression_votes|
| * |12 |0 |
| * |2 |6 |
Finally (phew) the reason you have an error in your original SQL is that you are chaining two SELECT statements together without actually relating them - the SQL doesn't really make sense in this instance as you need to logically relate them (either via a UNION or a repeated join as per above.
The Schema:
I have 3 Tables:
User
Feature
User_has_Feature:
initially all users has no features
Example data:
User:
| id | name |
| 1 | Rex |
| 2 | Job |
Feature:
| id | name |
| 1 | Eat |
| 2 | Walk |
User_has_Feature:
| id | user_id | feature_id | have_feature |
| 1 | 1 | 1 | true |
| 2 | 1 | 1 | true |
| 3 | 2 | 2 | true |
| 4 | 2 | 2 | false |
The questions are:
¿How to get only the records that have all features? (explicitly)
Example:
| user_name | feature_name | feature_status |
| Rex | Eat | true |
| Rex | Walk | true |
How to get records that do not have all the features? (again explicitly)
Example:
| user_name | feature_name | feature_status |
| Job | Eat | true |
| Job | Walk | false |
Some conditions have to be attended
I need the Users list with all features (true or false) in both queries like examples
User have 650k records (for now)
Feature have 45 records (for now)
Is one time query.
The idea is to export the result to a CSV file
Early Solution
thanks to the answers of (#RolandoMySQLDBA, #Tom Ingram, #DRapp) I found a solution:
SELECT u.name, f.name, IF(uhf.status=1,'YES','NO') as status
FROM user u
JOIN user_has_feature uhf ON u.id = uhf.user_id
JOIN feature f ON f.id = uhf.feature_id
JOIN
(
SELECT u.id as id
FROM user u
JOIN user_has_feature uhf ON uhf.user_id = u.id
WHERE uhf.status = 1
GROUP BY u.id
HAVING count(u.id) <= (SELECT COUNT(1) FROM feature)
) as `condition` ON `condition`.id = u.id
ORDER BY u.name, f.id, uhf.status
For get records that do not have all the features and for get all record that have all features change:
WHERE uhf.status = 1 by WHERE uhf.status = 2
HAVING count(u.id) <= (SELECT COUNT(1) FROM feature) by HAVING count(u.id) = (SELECT COUNT(1) FROM feature)
but I want to know if this is an optimal solution?
SELECT
UNF.*,
IF(
(LENGTH(UNF.FeatureList) - LENGTH(REPLACE(UNF.FeatureList,',','')))
= (FC.FeatureCount - 1),'Has All Features','Does Not Have All Features'
) HasAllFeatures
FROM
(SELECT
U.name user_name
GROUP_CONCAT(F.name) Features
FROM
(SELECT user_id,feature_id FROM User_has_Feature
WHERE feature_status = true) UHF
INNER JOIN User U ON UHF.user_id = U.id
INNER JOIN Feature F ON UHF.feature_id = F.id
GROUP BY
U.name
) UNF,
(SELECT COUNT(1) FeatureCount FROM Feature) FC
;
The UNF subquery returns with all users listed in User_has_Feature and a comma-separated list of the features. The column HasAllFeatures is determined by the number of columns in UNF.FeatureList. In your case, there are two features. If the number of commas in UNF.FeatureList is FeatureCount - 1, then the user has all features. Otherwise, user does not have all features.
Here is a better version that shows all users and whether or not they have all, some or no features
SELECT
U.name user_name,
IFNULL(UsersAndFeatures.HasAllFeatures,
'Does Not Have Any Features')
WhatFeaturesDoesThisUserHave
FROM
User U LEFT JOIN
(
SELECT
UHF.user_id id,
IF(
(LENGTH(UHF.FeatureList) - LENGTH(REPLACE(UHF.FeatureList,',','')))
= (FC.FeatureCount - 1),
'Has All Features',
'Does Not Have All Features'
) HasAllFeatures
FROM
(
SELECT user_id,GROUP_CONCAT(Feature.name) FeatureList
FROM User_has_Feature INNER JOIN Feature
ON User_has_Feature.feature_id = Feature.id
GROUP BY user_id
) UHF,
(SELECT COUNT(1) FeatureCount FROM Feature) FC
) UsersAndFeatures
USING (id);
select
u.id,
u.name as User_Name,
f.name as Feature_Name,
uhf.feature_Status
from
( select uhf.user_id,
sum( if( uhf.feature_status, 1, 0 ) ) as UserFeatureCount
from user_has_feature uhf
group by uhf.user_id ) AllUsersWithCounts
join
( select count(*) as AllFeaturesCount
from Feature ) AllFeatures
on AllUsersWithCounts.UserFeatureCount = AllFeatures.AllFeaturesCount
join user u
on AllUsersWithCounts.user_id = u.ID
join user_has_feature uhf
on AllUsersWithCounts.User_id = uhf.user_id
join feature f
on uhf.feature_id = f.id
The above query should get all people that explicitly have ALL features. In order to get those that do NOT have all features, just change the one join from = to <
on AllUsersWithCounts.UserFeatureCount < AllFeatures.AllFeaturesCount
Here's my bash at it
create a view of the general information
CREATE VIEW v_users_have_features AS
SELECT usr.id, usr.name, feature.name, has_feature.status
FROM usr
JOIN has_feature ON usr.id = has_feature.user_id
JOIN feature ON has_feature.feature_id = feature.id;
use the view for other queries
SELECT v_users_have_features.id, v_users_have_features.u_name, v_users_have_features.f_name
FROM v_users_have_features
GROUP BY v_users_have_features.id
HAVING COUNT( v_users_have_features.id ) = (SELECT COUNT( feature.id )
FROM feature
WHERE feature.name = v_users_have_features.f_name )
p.s. you may need to adapt (particularly the latter) to your exact requirements you could also omit creating the view and nest it in the FROM clause like in another answer it just seemed handier to create the view
Count the number of features. Write a query over users that uses a correlated subquery to find all the features a user has and count them. Make the restriction criterion in the top query the equality of that count and the global number of features.
Can MySQL do correlated subqueries? If not, you might need to use a better database.