Related
I need some help to do it right in one query (if it possible).
(this is a theoretical example and I assume the presence of events in event_name(like registration/action etc)
I have 3 colums:
-user_id
-event_timestamp
-event_name
From this 3 columns we need to create new table with 4 new columns:
-user year and month registration time
-number of new user registration in this month
-number of users who returned to the second calendar month after registration
-return probability
Result must be looks like this:
2019-1 | 1 | 1 | 100%
2019-2 | 3 | 2 | 67%
2019-3 | 2 | 0 | 0%
What I've done now:
I'm use this toy example of my possible main table:
CREATE TABLE `main` (
`event_timestamp` timestamp,
`user_id` int(10),
`event_name` char(12)
) DEFAULT CHARSET=utf8;
INSERT INTO `main` (`event_timestamp`, `user_id`, `event_name`) VALUES
('2019-01-23 20:02:21.550', '1', 'registration'),
('2019-01-24 20:03:21.550', '2', 'action'),
('2019-02-21 20:04:21.550', '3', 'registration'),
('2019-02-22 20:05:21.550', '4', 'registration'),
('2019-02-23 20:06:21.550', '5', 'registration'),
('2019-02-23 20:06:21.550', '1', 'action'),
('2019-02-24 20:07:21.550', '6', 'action'),
('2019-03-20 20:08:21.550', '3', 'action'),
('2019-03-21 20:09:21.550', '4', 'action'),
('2019-03-22 20:10:21.550', '9', 'action'),
('2019-03-23 20:11:21.550', '10', 'registration'),
('2019-03-22 20:10:21.550', '4', 'action'),
('2019-03-22 20:10:21.550', '5', 'action'),
('2019-03-24 20:11:21.550', '11', 'registration');
I'm trying to test some queries to create 4 new columns:
This is for column #1, we select month and year from timestamp where action is registration (as I guess), but I need to sum it for month (like 2019-11, 2019-12)
SELECT DATE_FORMAT(event_timestamp, '%Y-%m') AS column_1 FROM main
WHERE event_name='registration';
For column #2 we need to sum users with even_name registration in this month for every month, or.. we can trying for searching first time activity by user_id, but I don't know how to do this.
Here is some thinks about it...
SELECT COUNT(DISTINCT user_id) AS user_count
FROM main
GROUP BY MONTH(event_timestamp);
SELECT COUNT(DISTINCT user_id) AS user_count FROM main
WHERE event_name='registration';
For column #3 we need to compare user_id with the event_name registration and last month event with any event of the second month so we get users who returned for the next month.
Any idea how to create this query?
This is how to calc column #4
SELECT *,
ROUND ((column_3/column_2)*100) AS column_4
FROM main;
I hope you will find the following answer helpful.
The first column is the extraction of year and month. The new_users column is the COUNT of the unique user ids when the action is 'registration' since the user can be duplicated from the JOIN as a result of taking multiple actions the following month. The returned_users column is the number of users who have an action in the next month from the registration. The returned_users column needs a DISTINCT clause since a user can have multiple actions during one month. The final column is the probability that you asked from the two previous columns.
The JOIN clause is a self-join to bring the users that had at least one action the next month of their registration.
SELECT CONCAT(YEAR(A.event_timestamp),'-',MONTH(A.event_timestamp)),
COUNT(DISTINCT(CASE WHEN A.event_name LIKE 'registration' THEN A.user_id END)) AS new_users,
COUNT(DISTINCT B.user_id) AS returned_users,
CASE WHEN COUNT(DISTINCT(CASE WHEN A.event_name LIKE 'registration' THEN A.user_id END))=0 THEN 0 ELSE COUNT(DISTINCT B.user_id)/COUNT(DISTINCT(CASE WHEN A.event_name LIKE 'registration' THEN A.user_id END))*100 END AS My_Ratio
FROM main AS A
LEFT JOIN main AS B
ON A.user_id=B.user_id AND MONTH(A.event_timestamp)+1=MONTH(B.event_timestamp)
AND A.event_name='registration' AND B.event_name='action'
GROUP BY CONCAT(YEAR(A.event_timestamp),'-',MONTH(A.event_timestamp))
What we will do is to use window functions and aggregation -- window functions to get the earliest registration date. Then some conditional aggregation.
One challenge is the handling of calendar months. To handle this, we will truncate the dates to the beginning of the month to facilitate the date arithmetic:
select yyyymm_reg, count(*) as regs_in_month,
sum( month_2 > 0 ) as visits_2months,
avg( month_2 > 0 ) as return_rate_2months
from (select m.user_id, m.yyyymm_reg,
max( (timestampdiff(month, m.yyyymm_reg, m.yyyymm) = 1) ) as month_1,
max( (timestampdiff(month, m.yyyymm_reg, m.yyyymm) = 2) ) as month_2,
max( (timestampdiff(month, m.yyyymm_reg, m.yyyymm) = 3) ) as month_3
from (select m.*,
cast(concat(extract(year_month from event_timestamp), '01') as date) as yyyymm,
cast(concat(extract(year_month from min(case when event_name = 'registration' then event_timestamp end) over (partition by user_id)), '01') as date) as yyyymm_reg
from main m
) m
where m.yyyymm_reg is not null
group by m.user_id, m.yyyymm_reg
) u
group by u.yyyymm_reg;
Here is a db<>fiddle.
Here you go, done in T-SQL:
;with cte as(
select a.* from (
select form,user_id,sum(count_regs) as count_regs,sum(count_action) as count_action from (
select FORMAT(event_timestamp,'yyyy-MM') as form,user_id,event_name,
CASE WHEN event_name = 'registration' THEN 1 ELSE 0 END as count_regs,
CASE WHEN event_name = 'action' THEN 1 ELSE 0 END as count_action from main) a
group by form,user_id) a)
select final.form,final.count_regs,final.count_action,((CAST(final.count_action as float)/(CASE WHEN final.count_regs = '0' THEN '1' ELSE final.count_regs END))*100) as probability from (
select a.form,sum(a.count_regs) count_regs,CASE WHEN sum(b.count_action) is null then '0' else sum(b.count_action) end count_action from cte a
left join
cte b
ON a.user_id = b.user_id and
DATEADD(month,1,CONVERT(date,a.form+'-01')) = CONVERT(date,b.form+'-01')
group by a.form ) final where final.count_regs != '0' or final.count_action != '0'
I'm trying to retrive the records from a table in my MySQL database, where:
the timestamp is the closest to a variable I provide; and,
grouped by the fields keyA, keyB, keyC and keyD
I've hard coded the variable as below to test this, however can not get the query to work.
SQLFiddle
My current schema is:
CREATE TABLE dataHistory (
timestamp datetime NOT NULL,
keyA varchar(10) NOT NULL,
keyB varchar(10) NOT NULL,
keyC varchar(25) NOT NULL,
keyD varchar(10) NOT NULL,
value int NOT NULL,
PRIMARY KEY (timestamp,keyA,keyB,keyC,keyD)
);
INSERT INTO dataHistory
(timestamp, keyA, keyB, keyC, keyD, value)
VALUES
('2016-05-12 04:15:00', 'value1', 'all', 'value2', 'domestic', 96921),
('2016-05-12 04:05:00', 'value1', 'all', 'value2', 'domestic', 96947),
('2016-05-12 04:20:00', 'value1', 'all', 'value2', 'domestic', 96954),
('2016-05-12 04:15:00', 'value1', 'all', 'value3', 'domestic', 2732),
('2016-05-12 04:10:00', 'value1', 'all', 'value3', 'domestic', 2819),
('2016-05-12 04:20:00', 'value1', 'all', 'value3', 'domestic', 2802);
and the query I currently have is:
SELECT e.difference, e.timestamp, e.keyA, e.keyB, e.keyC, e.keyD, e.value
FROM (SELECT TIMESTAMPDIFF(minute, '2016-05-12 04:11:00', d.timestamp) as difference, d.timestamp, d.keyA, d.keyB, d.keyC, d.keyD, d.value
FROM dataHistory d
GROUP BY d.keyA, d.keyB, d.keyC, d.keyD) as e;
All I can seem to extract from the sample data is the earliest two records and not the two closest to the datetime.
What I receive:
difference timestamp keyA keyB keyC keyD value
-10 May, 12 2016 04:05:00 value1 all value2 domestic 96947
-5 May, 12 2016 04:10:00 value1 all value3 domestic 2819
I am expecting to see:
timestamp keyA keyB keyC keyD value
May, 12 2016 04:15:00 value1 all value2 domestic 96921
May, 12 2016 04:10:00 value1 all value3 domestic 2819
Any assistance would be appreciated!
SELECT e.difference, e.timestamp, e.keyA, e.keyB, e.keyC, e.keyD, e.value
FROM (SELECT ABS(TIMESTAMPDIFF(minute, '2016-05-12 04:11:00', d.timestamp)) as difference, d.timestamp, d.keyA, d.keyB, d.keyC, d.keyD, d.value
FROM dataHistory d
ORDER BY difference) as e
GROUP BY e.keyA, e.keyB, e.keyC, e.keyD;
This query is returning the values you want.
Does this help?
SELECT
TIMESTAMPDIFF (MINUTE , '2016-05-12 04:15:00' , MainTable.timestamp) AS Difference ,
MainTable.timestamp ,
MainTable.KeyA ,
MainTable.KeyB ,
MainTable.KeyC ,
MainTable.KeyD ,
MainTable.value
FROM
dataHistory AS MainTable
LEFT OUTER JOIN
dataHistory AS SecondaryTable
ON
MainTable.KeyA = SecondaryTable.KeyA
AND
MainTable.KeyB = SecondaryTable.KeyB
AND
MainTable.KeyC = SecondaryTable.KeyC
AND
MainTable.KeyD = SecondaryTable.KeyD
AND
ABS (TIMESTAMPDIFF (MINUTE , '2016-05-12 04:15:00' , MainTable.timestamp)) > ABS (TIMESTAMPDIFF (MINUTE , '2016-05-12 04:15:00' , SecondaryTable.timestamp))
WHERE
SecondaryTable.timestamp IS NULL;
Guy Glantser,
Data Professional,
Madeira - Data Solutions,
http://www.madeiradata.com
You are obviously expecting some magic to happen here. You group by some fields and select the column timestamp and its difference to the current time. And somehow you think you should get the closest time to now. Why? Why should this happen? You are not telling the DBMS to do that. You are simply letting it pick one of the matching timestamps arbitrarily. To pick a particular value per group, you need an aggregate function, e.g. MIN to get a minimum value.
You need two steps:
First step: Find the minimum timestamp difference to now per group.
select
keya,
keyb,
keyc,
keyd,
min(abs(timestampdiff(minute, '2016-05-12 04:11:00', d.timestamp))) as difference
from datahistory
group by keya, keyb, keyc, keyd;
Second step: With the query from the first step, find the matching records for each of these minimum differences.
select
best.difference,
dh.timestamp,
best.keyA,
best.keyB,
best.keyC,
best.keyD,
dh.value
from
(
select
keya, keyb, keyc, keyd,
min(abs(timestampdiff(minute, '2016-05-12 04:11:00', timestamp))) as difference
from datahistory
group by keya, keyb, keyc, keyd
) best
join datahistory dh
on dh.keya = best.keya and dh.keyb = best.keyb
and dh.keyc = best.keyc and dh.keyd = best.keyd
and abs(timestampdiff(minute, '2016-05-12 04:11:00', dh.timestamp)) = best.difference
order by best.keyA, best.keyB, best.keyC, best.keyD;
SQL fiddle: http://sqlfiddle.com/#!9/a6004b/10
(Replace '2016-05-12 04:11:00' with now() in your real query.)
I have entry table:
I need to do a SELECT to receive 'Date', 'Number of entries' (in that date), 'Total number of entries until that date'.
When I do the SELECT:
SELECT e1.*,
(select count(*) from entry where date(dateCreated) <= e1.date) as Total
from (
SELECT
DATE(e.dateCreated) as "Date",
count(e.dateCreated) as "No of Entries",
sum( case when e.premium='Y' then 1 else 0 end ) as Premium,
sum( case when e.free='Y' then 1 else 0 end ) as Free,
sum( case when e.affiliateID IS NOT NULL then 1 else 0 end) as Affiliate
FROM entry e
WHERE e.competitionID=166
GROUP BY DATE(e.dateCreated)
) as e1
ORDER BY Date DESC
I've got a result table
but the column 'Total' has a wrong data.
How the correct select should be? Is this logic of select is the best and more efficient one?
Here is a demo
If it is just the 5 vs 7 that is off I think it is because that subquery in your select list, which accesses the inline view e1 (which is filtered to competitionID = 166), is not itself filtered when also utilizing the original entry table (unfiltered). You have to filter the original table to that competitionID as well.
Notice line 3 in sql below (only change)
SELECT e1.*,
(select count(*) from entry where date(dateCreated) <= e1.date
and competitionID=166) as Total
from (
SELECT
DATE(e.dateCreated) as "Date",
count(e.dateCreated) as "No of Entries",
sum( case when e.premium='Y' then 1 else 0 end ) as Premium,
sum( case when e.free='Y' then 1 else 0 end ) as Free,
sum( case when e.affiliateID IS NOT NULL then 1 else 0 end) as Affiliate
FROM entry e
WHERE e.competitionID=166
GROUP BY DATE(e.dateCreated)
) as e1
ORDER BY Date DESC
Fiddle - http://sqlfiddle.com/#!9/e5e88/22/0
Hy guys, sometimes my subquery return null which is ok, it should return null, but in those cases i would like my "parent select" to return null.
Is that possible?
And if yes, then how?
Heres the code:
SELECT
`company`.`companyID`,
`company`.`companyName`,
`company`.`companyName`,
`company`.`companyEmail`,
`company`.`contactEmail`,
`company`.`companyTel`,
(
SELECT
`package_map`.`szekhely_endDate`
FROM
`package_map`
WHERE
`package_map`.`companyID` = `company`.`companyID`
AND
`package_map`.`active` = 1
AND
`package_map`.`szekhely_endDate` > NOW()
ORDER BY
`package_map`.`szekhely_endDate` DESC
LIMIT 1
) as endDate,
CASE
WHEN endDate = NULL
FROM
`company`
WHERE
`company`.`companyBase` = 'some address'
AND
`company`.`szekhely_check_out` = 0
Use an ordinary INNER JOIN between the two tables. If there's no matching rows in the package_map table, there won't be a row in the result. To get the latest endDate, use the MAX() function.
SELECT
`company`.`companyID`,
`company`.`companyName`,
`company`.`companyName`,
`company`.`companyEmail`,
`company`.`contactEmail`,
`company`.`companyTel`,
MAX(package_map.szekhely_endDate) AS endDate
FROM company
INNER JOIN package_map ON `package_map`.`companyID` = `company`.`companyID`
WHERE
`company`.`companyBase` = 'some address'
AND
`company`.`szekhely_check_out` = 0
AND
`package_map`.`active` = 1
AND
`package_map`.`szekhely_endDate` > NOW()
GROUP BY `company`.`companyID`
I've read a lot of the examples on self join, but they don't seem to cover the case where some fields are not in some rows.
For eg, I have a database with:
testId, testItem, testResult
And the rows:
1,test1,1
1,test2,0
1,test3,1
2,test1,0
2,test4,1
2,test5,1
I would like the output:
testItem,a.testId,b.testId,a.testResult,b.testResult
test1,1,2,1,0
test2,1,NULL,0,NULL
test3,1,NULL,1,NULL
test4,NULL,2,NULL,1
test5,NULL,2,NULL,1
Essentially, I want to compare each testItem (test1->test5) from two different testIds (1 and 2) and compare their testResult values, factoring in testIds that may not have the same test Items.
Given your exact requirement, you can try this:
select testItem
, max(case when testID = 1 then testID else null end) as testID1
, max(case when testID = 2 then testID else null end) as testID2
, max(case when testID = 1 then testResult else null end) as testResult1
, max(case when testID = 2 then testResult else null end) as testResult2
from mytable
where testID in (1,2)
group by testItem
This makes a lot of assumptions about your data, so take it with a grain of salt.
It looks like you want a FULL OUTER JOIN, which is not supported in MySQL. You can emulate this with a UNION of two queries: a LEFT JOIN query and RIGHT JOIN which throws out matching rows.
Something like this will return the specified resultset:
SELECT a.testItem
, a.testId AS `a.testId`
, b.testId AS `b.testId`
, a.testResult AS `a.testResult`
, b.testResult AS `b.testResult`
FROM mytable a
LEFT
JOIN mytable b
ON b.testItem = a.testItem
AND b.testId = 2
WHERE a.testId = 1
AND a.testItem IN ('test1','test2','test3','test4','test5')
UNION ALL
SELECT d.testItem
, c.testId
, d.testId
, c.testResult
, d.testResult
FROM mytable d
LEFT
JOIN mytable c
ON c.testItem = d.testItem
AND c.testId = 1
WHERE d.testId = 2
AND d.testItem IN ('test1','test2','test3','test4','test5')
AND c.testId IS NULL
ORDER
BY 1,2,4
(I included the predicates on testItem IN ('test1' thru 'test5') because you specified that as a requirement; those predicates could be removed if you want all values for testItem included.)
SQLFiddle Demo
select testItem,
group_concat(IFNULL(testId,'null') separator ', ') testIds,
group_concat(IFNULL(testResult, 'null') separator ', ') testResults
from table_name group by testItem;