I'm trying to create a temp table or view that I can use for charting league members changing handicaps over the season. This requires creating a pivot table.
Even before that, however, I need to create an ordinal column to represent the nth match the person played that season. I can't use date_played since players can play on all different dates, and I'd like each player's 2nd match to line up vertically, and their third to do so as well, etc.
I used code from the answer to this question which seemed like it should work. Here's a copy of the relevant section:
SELECT books.*,
if( #libId = libraryId,
#var_record := #var_record + 1,
if(#var_record := 1 and #libId := libraryId, #var_record, #var_record)
) AS Ordinal
FROM books
JOIN (SELECT #var_record := 0, #libId := 0) tmp
ORDER BY libraryId;
`lwljhb_lwl_matches` # One record for each match played
id
date_played
other fields about the match
id date_played
1 2017-08-23
2 2017-08-29
3 2017-09-26
4 2017-08-24
5 2017-09-02
6 2017-09-21
7 2017-08-24
8 2017-08-31
9 2017-09-05
10 2017-09-15
11 2017-09-17
`lwljhb_users`
id
display_name
id display_name
1 Alan
2 Bill
3 Dave
`lwljhb_lwl_players` # One record per player per match
id
match_id # Foreign key to matches.id
player_id # Foreign key to users.id
end_hcp
other fields about the player's performance in the linked match
id match_id player_id end_hcp
1 1 1 720
2 2 1 692
3 3 1 694
4 4 2 865
5 5 2 868
6 6 2 842
7 7 3 363
8 8 3 339
9 9 3 332
10 10 3 348
11 11 3 374
Normally there would be two records in PLAYERS for each MATCH record, but I didn't add them here. The fact that the id and match_id are the same in every record of PLAYERS is artificial because this isn't real data.
Before I used this snippet, my code looked like this:
SELECT u.display_name,
m.date_played,
p.end_hcp
FROM `lwljhb_lwl_matches` AS m
INNER JOIN `lwljhb_lwl_players` AS p
INNER JOIN `lwljhb_users` AS u
ON m.id = p.match_id AND
p.player_id = u.id
WHERE league_seasons_id = 12 AND
playoff_round = 0
ORDER BY u.display_name, m.date_played
and generated data that looks like this:
display_name date_played end_hcp
Alan 2017-08-23 720
Alan 2017-08-29 692
Alan 2017-09-26 694
Bill 2017-08-24 865
Bill 2017-09-02 868
Bill 2017-09-21 842
Dave 2017-08-24 363
Dave 2017-08-31 339
Dave 2017-09-05 332
Dave 2017-09-15 348
Dave 2017-09-17 374
At any time during the season there will be different numbers of matches played by each of the players, but by the end of the season, they will have all played the same number of matches.
I tried to incorporate Alin Stoian's code into mine like so, changing a variable name and field names as I thought appropriate.
SET #var_record = 1;
SELECT u.display_name,
m.date_played,
p.end_hcp,
if( #player = u.display_name,
#var_record := #var_record + 1,
if(#var_record := 1 and #player := u.display_name, #var_record, #var_record)
) AS Ordinal
FROM `lwljhb_lwl_matches` AS m
INNER JOIN `lwljhb_lwl_players` AS p
INNER JOIN `lwljhb_users` AS u
JOIN (SELECT #var_record := 0, #player := 0) tmp
ON m.id = p.match_id AND
p.player_id = u.id
WHERE league_seasons_id = 12 AND
playoff_round = 0
ORDER BY u.display_name, m.date_played
I was hoping for a new column with the ordinals, but the new column is all zeros. Before I move on to trying the pivot table, I have to get these ordinals, so I hope someone here can show me my mistake.
Try this:
SELECT u.display_name,
m.date_played,
p.end_hcp,
#var_record := if ( #player = u.display_name,
#var_record + 1,
if(#player := u.display_name, 1, 1)
) AS Ordinal
FROM `lwljhb_lwl_matches` AS m
INNER JOIN `lwljhb_lwl_players` AS p
INNER JOIN `lwljhb_users` AS u
JOIN (SELECT #var_record := 0, #player := 0) tmp
ON m.id = p.match_id AND
p.player_id = u.id
WHERE league_seasons_id = 12 AND
playoff_round = 0
ORDER BY u.display_name, m.date_played
Demo
OK, it turns out that I wasn't getting all 1s in my Ordinal column after all, just in the 1st 25 records. Since it was obviously wrong I didn't look further.
The sample data I provided is in physical order as well as logical order, but MY ACTUAL data is not. That was the only difference I could think of, so I investigated further and found that in the 1st 1,000 records of output I got 7 records with 2 as the ordinal, not coincidentally where the m.ids were consecutive.
I created a view to put the data in order like in the sample and the JOINed that to the rest of the tables but still got bad data for the Ordinal.
When I swapped out the view for a temporary table it worked. It turns out that an ORDER BY in a view will be ignored if there's an ORDER by joining to it.
Bill Karwin pointed this out to me in a different question (46892912).
Let say I have these two tables in mysql.
table1:
date staff_no
2016-06-10 1
2016-06-09 1
2016-05-09 1
2016-04-09 1
table2:
staff_no name
1 David
Then, I have this query to get analysis for the staff for each month:
SELECT DATE_FORMAT(table1.date,'%b %Y') as month,COUNT(table1.date) as total_records,table2.name as name
FROM table1 as table1
LEFT JOIN table2 as table2 on table2.staff_no = table1.staff_no
WHERE table1.staff_no = "1" and date(table1.date) between = "2016-04-01" and "2016-06-30"
GROUP BY table2.name,DATE_FORMAT(table1.date,'%Y-%m')
ORDER BY DATE_FORMAT(table1.date,'%Y-%m-%d')
This query will output:
month total_records name
Apr 2016 1 David
May 2016 1 David
Jun 2016 2 David
But, if I replace the date between "2016-04-01" and "2016-07-31" from the query,it wont show me the July record because it is not exist in table1 which is not what I want. I still want to get result like this:
month total_records name
Apr 2016 1 David
May 2016 1 David
Jun 2016 2 David
Jul 2016 0 David
Anyone expert on this? Kindly help me into this. Thanks!
Consider the following schema with the 3rd table being the year/month Helper Table mentioned. Helper tables are very common and can be re-used throughout your code naturally. I will leave it to you to load it up with substantial date data. Note however the way the end date for each month was put together for those of us that want to do less work, while allowing the db engine to figure out leap years for us.
You could have just one column in that helper table. But that would require the use of function calls for end dates in some of your functions and that means more slowness. We like fast.
Schema
create table workerRecords
( id int auto_increment primary key,
the_date date not null,
staff_no int not null
);
-- truncate workerRecords;
insert workerRecords(the_date,staff_no) values
('2016-06-10',1),
('2016-06-09',1),
('2016-05-09',1),
('2016-04-09',1),
('2016-03-02',2),
('2016-07-02',2);
create table workers
( staff_no int primary key,
full_name varchar(100) not null
);
-- truncate workers;
insert workers(staff_no,full_name) values
(1,'David Higgins'),(2,"Sally O'Riordan");
Helper table below
create table ymHelper
( -- Year Month helper table. Used for left joins to pick up all dates.
-- PK is programmer's choice.
dtBegin date primary key, -- by definition not null
dtEnd date null
);
-- truncate ymHelper;
insert ymHelper (dtBegin,dtEnd) values
('2015-01-01',null),('2015-02-01',null),('2015-03-01',null),('2015-04-01',null),('2015-05-01',null),('2015-06-01',null),('2015-07-01',null),('2015-08-01',null),('2015-09-01',null),('2015-10-01',null),('2015-11-01',null),('2015-12-01',null),
('2016-01-01',null),('2016-02-01',null),('2016-03-01',null),('2016-04-01',null),('2016-05-01',null),('2016-06-01',null),('2016-07-01',null),('2016-08-01',null),('2016-09-01',null),('2016-10-01',null),('2016-11-01',null),('2016-12-01',null),
('2017-01-01',null),('2017-02-01',null),('2017-03-01',null),('2017-04-01',null),('2017-05-01',null),('2017-06-01',null),('2017-07-01',null),('2017-08-01',null),('2017-09-01',null),('2017-10-01',null),('2017-11-01',null),('2017-12-01',null),
('2018-01-01',null),('2018-02-01',null),('2018-03-01',null),('2018-04-01',null),('2018-05-01',null),('2018-06-01',null),('2018-07-01',null),('2018-08-01',null),('2018-09-01',null),('2018-10-01',null),('2018-11-01',null),('2018-12-01',null),
('2019-01-01',null),('2019-02-01',null),('2019-03-01',null),('2019-04-01',null),('2019-05-01',null),('2019-06-01',null),('2019-07-01',null),('2019-08-01',null),('2019-09-01',null),('2019-10-01',null),('2019-11-01',null),('2019-12-01',null);
-- will leave as an exercise for you to add more years. Good idea to start, 10 in either direction, at least.
update ymHelper set dtEnd=LAST_DAY(dtBegin); -- data patch. Confirmed leap years.
alter table ymHelper modify dtEnd date not null; -- there, ugly patch above worked fine. Can forget it ever happened (until you add rows)
-- show create table ymHelper; -- this confirms that dtEnd is not null
So that is a helper table. Set it up once and forget about it for a few years
Note: Don't forget to run the above update stmt
Quick Test for your Query
SELECT DATE_FORMAT(ymH.dtBegin,'%b %Y') as month,
ifnull(COUNT(wr.the_date),0) as total_records,#soloName as full_name
FROM ymHelper ymH
left join workerRecords wr
on wr.the_date between ymH.dtBegin and ymH.dtEnd
and wr.staff_no = 1 and wr.the_date between '2016-04-01' and '2016-07-31'
LEFT JOIN workers w on w.staff_no = wr.staff_no
cross join (select #soloName:=full_name from workers where staff_no=1) xDerived
WHERE ymH.dtBegin between '2016-04-01' and '2016-07-31'
GROUP BY ymH.dtBegin
order by ymH.dtBegin;
+----------+---------------+---------------+
| month | total_records | full_name |
+----------+---------------+---------------+
| Apr 2016 | 1 | David Higgins |
| May 2016 | 1 | David Higgins |
| Jun 2016 | 2 | David Higgins |
| Jul 2016 | 0 | David Higgins |
+----------+---------------+---------------+
It works fine. The first mysql table is the Helper table. A left join to bring in the worker records (allowing for null). Let's pause here. That was afterall the point of your question: missing data. Finally the worker table in a cross join.
The cross join is to initialize a variable (#soloName) that is the worker's name. Whereas the null status of missing dates as you requested is picked up fine via the ifnull() function returning 0, we don't have that luxury for a worker's name. That forces the cross join.
A cross join is a cartesian product. But since it is a single row, we don't suffer from the normal problems one gets with cartesians causing way to many rows in the result set. Anyway, it works.
But here is one problem: it is too hard to maintain and plug in values in 6 places as can be seen. So consider below a stored proc for it.
Stored Proc
drop procedure if exists getOneWorkersRecCount;
DELIMITER $$
create procedure getOneWorkersRecCount
(pStaffNo int, pBeginDt date, pEndDt date)
BEGIN
SELECT DATE_FORMAT(ymH.dtBegin,'%b %Y') as month,ifnull(COUNT(wr.the_date),0) as total_records,#soloName as full_name
FROM ymHelper ymH
left join workerRecords wr
on wr.the_date between ymH.dtBegin and ymH.dtEnd
and wr.staff_no = pStaffNo and wr.the_date between pBeginDt and pEndDt
LEFT JOIN workers w on w.staff_no = wr.staff_no
cross join (select #soloName:=full_name from workers where staff_no=pStaffNo) xDerived
WHERE ymH.dtBegin between pBeginDt and pEndDt
GROUP BY ymH.dtBegin
order by ymH.dtBegin;
END$$
DELIMITER ;
Test the stored proc a number of times
call getOneWorkersRecCount(1,'2016-04-01','2016-06-09');
call getOneWorkersRecCount(1,'2016-04-01','2016-06-10');
call getOneWorkersRecCount(1,'2016-04-01','2016-07-01');
call getOneWorkersRecCount(2,'2016-02-01','2016-11-01');
Ah, much easier to work with (in PHP, c#, Java, you name it). Choice is yours, stored proc or not.
Bonus Stored Proc
drop procedure if exists getAllWorkersRecCount;
DELIMITER $$
create procedure getAllWorkersRecCount
(pBeginDt date, pEndDt date)
BEGIN
SELECT DATE_FORMAT(ymH.dtBegin,'%b %Y') as month,ifnull(COUNT(wr.the_date),0) as total_records,w.staff_no,w.full_name
FROM ymHelper ymH
cross join workers w
left join workerRecords wr
on wr.the_date between ymH.dtBegin and ymH.dtEnd
and wr.staff_no = w.staff_no and wr.the_date between pBeginDt and pEndDt
-- LEFT JOIN workers w on w.staff_no = wr.staff_no
-- cross join (select #soloName:=full_name from workers ) xDerived
WHERE ymH.dtBegin between pBeginDt and pEndDt
GROUP BY ymH.dtBegin,w.staff_no,w.full_name
order by ymH.dtBegin,w.staff_no;
END$$
DELIMITER ;
Quick test of it
call getAllWorkersRecCount('2016-03-01','2016-08-01');
+----------+---------------+----------+-----------------+
| month | total_records | staff_no | full_name |
+----------+---------------+----------+-----------------+
| Mar 2016 | 0 | 1 | David Higgins |
| Mar 2016 | 1 | 2 | Sally O'Riordan |
| Apr 2016 | 1 | 1 | David Higgins |
| Apr 2016 | 0 | 2 | Sally O'Riordan |
| May 2016 | 1 | 1 | David Higgins |
| May 2016 | 0 | 2 | Sally O'Riordan |
| Jun 2016 | 2 | 1 | David Higgins |
| Jun 2016 | 0 | 2 | Sally O'Riordan |
| Jul 2016 | 0 | 1 | David Higgins |
| Jul 2016 | 1 | 2 | Sally O'Riordan |
| Aug 2016 | 0 | 1 | David Higgins |
| Aug 2016 | 0 | 2 | Sally O'Riordan |
+----------+---------------+----------+-----------------+
The Takeaway
Helper Tables have been used for decades. Don't be afraid or embarrassed to use them. In fact, trying to get some specialty work done without them is nearly impossible at times.
You can build an inline set of variables representing all the dates you want by using any other table in your system that has AT LEAST the number of months you are trying to represent even though the data does not have to have dates. Just has records that you can put a limit on.
TRY the following statement that uses MySql variables. The FROM clause declares a variable inline to the SQL statement "#Date1". I am starting it with MARCH 1 of 2016. Now, the select fields list takes that variable and keeps adding 1 month at a time to it. Since it is combined with the "AnyTableWithAtLeast12Records" (literally any table in your system with at least X records), it will create a result showing the dates. This is one way of forcing a calendar type of list.
But notice the SECOND column in this select does not change the #Date1 via the := assignment. So, it takes the date as it now stands and adds another month to it for the END Date. If you need a smaller or larger date range, just change the limit of records to create the calendar spread...
select
#Date1 := date_add( #Date1, interval 1 month ) StartDate,
date_add( #Date1, interval 1 month ) EndDate
from
AnyTableWithAtLeast12Records,
( select #Date1 := '2016-03-01' ) sqlvars
limit 12;
The result is something like...
StartDate EndDate
2016-04-01 2016-05-01
2016-05-01 2016-06-01
2016-06-01 2016-07-01
2016-07-01 2016-08-01
2016-08-01 2016-09-01
2016-09-01 2016-10-01
2016-10-01 2016-11-01
2016-11-01 2016-12-01
2016-12-01 2017-01-01
2017-01-01 2017-02-01
2017-02-01 2017-03-01
2017-03-01 2017-04-01
Now you have your dynamic "Calendar" completed in one simple query. Now, use that as a basis for all the records you need counts for and format as you had. So take the entire query above as a JOIN to find records within those date ranges... No other queries or stored procedures required. Now, a simple LEFT JOIN will keep all dates, but only show those with staff when WITHIN the between range of per start/end. So ex: greater or equal to 04/01/2016, but LESS THEN 05/01/2016 which includes 04/30/2016 # 11:59:59pm.
SELECT
DATE_FORMAT(MyCalendar.StartDate,'%b %Y') as month,
COALESCE(COUNT(T1.Staff_no),0) as total_records,
COALESCE(T2.name,"") as name
FROM
( select #Date1 := date_add( #Date1, interval 1 month ) StartDate,
date_add( #Date1, interval 1 month ) EndDate
from
AnyTableWithAtLeast12Records,
( select #Date1 := '2016-03-01' ) sqlvars
limit 12 ) MyCalendar
LEFT JOIN table1 T1
ON T1.Date >= MyCalendar.StartDate
AND T1.Date < MyCalendar.EndDate
AND T1.Staff_No = 1
LEFT JOIN table2 T2
ON T1.staff_no = T2.StaffNo
GROUP BY
T2.name,
DATE_FORMAT(MyCalendar.StartDate,'%Y-%m')
ORDER BY
DATE_FORMAT(MyCalendar.StartDate,'%Y-%m-%d')
I would say you need to have RIGHT JOIN here to include staff from second table
I have a table like this :
id | user_id | param_id | param_value
1 1 44 google
2 1 45 adTest
3 1 46 Campaign
4 1 47 null
5 1 48 null
6 2 44 google
7 2 45 adAnotherTest
8 2 46 Campaign2
9 2 47 null
10 2 48 null
I want to fetch all the user_ids where (param_id = 44 AND param_value=google) AND (param_id= 45 AND param_value = adTest) . So the above where clause should give only user_id = 1 and not user_id = 2 . They both have google at param_id 44 but only user 1 has param_value adTest at param_id = 45 .
The problem is the n the future more params could be added . I need to find a dynamic query . Here what i have tryed :
SELECT DISTINCT up.user_id FROM user_params AS up
LEFT JOIN user_params AS upp ON up.id = upp.id
WHERE up.param_id IN (?,?)
AND upp.param_value IN (?,?)
SELECT DISTINCT up.user_id
FROM user_params AS up
LEFT JOIN user_params AS upp ON up.id = upp.id
group by up.user_id
having sum(param_id = 44 AND param_value = 'google') >= 1
and sum(param_id = 45 AND param_value = 'adTest') >= 1
Another way:
SELECT -- DISTINCT
up1.user_id
FROM
user_params AS up1
JOIN
user_params AS up2
ON up1.user_id = up2.user_id
WHERE
up1.param_id = 44 AND up1.param_value = 'google'
AND
up2.param_id = 45 AND up2.param_value = 'adTest' ;
You do not need the DISTINCT, if there is a UNIQUE constraint on (user_id, param_id)
For efficiency, add an index on (param_id, param_value, user_id)
The problem you are dealing with is called "Relational Division" and there is a great answer by #Erwin Brandstetter here: How to filter SQL results in a has-many-through relation, with a lot of ways to write such a query, along with performance tests.
The tests were done in Postgres so some of the queries do not even run in MySQL but at least half of them do run and efficiency would be similar in many of them.
If you want to optimize this should give the same results without the need an LEFT JOIN table scans (thanks to juergen d for having part)
SELECT
user_id
FROM
user_params
WHERE
param_id IN(44, 45)
AND
param_value IN('google', 'adTest')
GROUP BY
user_id
HAVING
sum(param_id = 44 AND param_value = 'google') >= 1
AND
sum(param_id = 45 AND param_value = 'adTest') >= 1
;
see http://sqlfiddle.com/#!2/17b65/4 for demo
I have a question about joining the data from 2 tables from a MySQL Database. First I will explain what I currently have and then what I want just to be as clear as possible.
I have 2 tables in the Database which look like this:
Table: Subscriptions
Columns:
ID int(11) PK AI
Klant ID int(11)
Mail ID int(11)
Status varchar(15)
Datum varchar(15)
ID Klant_ID Mail_ID Status Datum
123 6 6 90 21-03-2013
124 6 6 10 21-03-2013
125 6 5 90 21-03-2013
126 6 5 10 21-03-2013
127 6 1 90 20-03-2013
128 6 1 10 20-03-2013
129 6 2 10 21-03-2013
130 6 2 90 21-03-2013
131 6 4 90 21-03-2013
132 6 4 10 21-03-2013
And:
Table: Mail
Columns:
ID int(11) PK AI
Content longtext
Datum varchar(15)
Titel varchar(150)
ID Content Datum Titel
1 (alot of encoded html) 18-03-13 test
2 (alot of encoded html) 18-03-13 test2
4 (alot of encoded html) 18-03-13 alles weer testen
5 (alot of encoded html) 20-03-13 testje
6 (alot of encoded html) 21-03-13 Statusupdate week 6
I am using these 2 queries to select the data from the tables now:
SELECT ID, Titel FROM Mail
SELECT * FROM Subscriptions,
(SELECT MAX(ID) as ids, Mail_ID FROM Subscriptions
WHERE Klant_ID = '".$_GET["ID"]."' GROUP BY Mail_ID) table2
WHERE ID=table2.ids
I want to get a query using JOIN to be able to create this table using html:
I personaly haven't got much experience using JOIN since I first used it yesterday, I am able to make a simple JOIN query but I just don't know how to do this. If you have more questions ask them in the comments. If anyone could help me with this it would be great!
this is a simple Join between your tables
SELECT Mail_ID, Titel, Status, Subscriptions.Datum FROM Subscriptions
JOIN Mail ON (Subscription.Mail_ID=Mail.ID)
WHERE Klant_ID = '".$_GET["ID"]."' GROUP BY Mail_ID
the order of the rows is random by a join,
if you want to get the last data, your query is right.
Try:
select mail.id, mail.titel, subscriptions.status, subscriptions.datum
from mail join subscriptions on mail.id = subscriptions.mail_id
Tyvm #kolonel peteruk, #Kaii and #JaMaBing for your answers!
With your help I was able to merge my query with yours. I finally got it to work using this query:
SELECT Mail.ID, Mail.Titel, Subscriptions.ID, Subscriptions.Status, Subscriptions.Datum, Subscriptions.Mail_ID, Subscriptions.Mail_ID, Subscriptions.Klant_ID
FROM NAW.Subscriptions
JOIN NAW.Mail
ON Mail.ID = Subscriptions.Mail_ID,
(SELECT MAX(Subscriptions.ID) as ids, Mail_ID
FROM NAW.Subscriptions
WHERE Klant_ID =6
GROUP BY Mail_ID) table2
WHERE Subscriptions.ID=table2.ids
SELECT e.ID ,e.Titel,ea.Status,ea.Datum
FROM Mail e
LEFT JOIN Subscriptions ea
ON e.ID = ea.Mail_ID
WHERE ea.Klant_ID = '".$_GET["ID"]."'
GROUP BY Mail_ID)
ORDER BY e.ID ASC