First record combined with GROUP BY - mysql

Let's say I got a table "values" which contains the fields
id (int)
name (varchar)
value (float)
timestamp (int)
Now I want to to calculate the highest lowest and first value (timestamp based) for each name on the entire values table.
Is this possible to be achieved in one single performant query? I stumbled upon the 'first_value' function, but that one doesn't seem to work. I tried the following query, using joins, but also without success.
SELECT
a.name,
b.value as open,
MIN(a.value) as low,
MAX(a.value) as high
FROM values a
LEFT JOIN values b
ON a.name = b.name AND b.id = MIN(a.id)
GROUP BY a.name;
Isn't there some sort of function which would make something similar as this possible?
SELECT
name,
FIRST_VALUE(value) as open,
MIN(value) as low,
MAX(value) as high
FROM values
GROUP BY name
ORDER BY timestamp ASC;
Example data
id name value timestamp
1 USD 3 16540
2 EUR 5 16540
3 GBP 4 16540
4 EUR 2 16600
5 USD 4 16600
6 GBP 5 16600
7 USD 6 16660
8 EUR 7 16660
9 GBP 6 16660
10 USD 5 16720
11 EUR 5 16720
12 GBP 7 16720
13 EUR 8 16780
14 USD 7 16780
15 GBP 8 16780
Example output
name open low high
USD 3 3 7
EUR 5 2 8
GBP 4 4 8
I'm using MySQL-client version: 5.6.39
A tie should not be possible, if it does, I don't care which value gets picked.

If you are running MySQL 8.0, this can be quite easily solved with window functions:
select name, value open, low, high
from (
select
name,
value,
min(value) over(partition by name) low,
max(value) over(partition by name) high,
row_number() over(partition by name order by timestamp) rn
from mytable
) x
where rn = 1
Demo on DB Fiddle:
| name | open | low | high |
| ---- | ---- | --- | ---- |
| EUR | 5 | 2 | 8 |
| GBP | 4 | 4 | 8 |
| USD | 3 | 3 | 7 |
In earlier versions, you could:
use a correlated subquery to filter on the first record for each name
join the table with an aggregate query that computes the min and max of each name
Query:
select
t.name,
t.value open,
t0.low,
t0.high
from
mytable t
inner join (
select name, min(value) low, max(value) high from mytable group by name
) t0 on t0.name = t.name
where t.timestamp = (
select min(t1.timestamp) from mytable t1 where t1.name = t.name
);
Demo on MySQL 5.6 DB Fiddle: same results as above
This could also be achieved using inline subqueries (which may actually perform better):
select
t.name,
t.value open,
(select min(value) from mytable t1 where t1.name = t.name) low,
(select max(value) from mytable t1 where t1.name = t.name) high
from
mytable t
where timestamp = (
select min(t1.timestamp) from mytable t1 where t1.name = t.name
)
Demo on MySQL 5.6 DB Fiddle

in one single performant query
Do it logically and let the DBMS worry about performance. If that isn't fast enough, check your indexes.
The value associated with the first timestamp requires a join. You can find the first timestamp easily enough. Getting a value from a row associated with a given row: that's what joins are for.
So, we have:
SELECT
name,
value as open,
v1.low
v1.high
FROM values as v join (
select name,
min(timestamp) as timestamp,
min(value) as low,
max(value) as high
FROM values
GROUP BY name
) as v1
on v.name = v1.name and v.timestamp = v1.timestamp

This solution seems to have the best performance.
SELECT
name,
CAST(SUBSTRING_INDEX(GROUP_CONCAT(CAST(value AS CHAR) ORDER BY TIMESTAMP ASC), ',', 1) AS DECIMAL(10, 6)) AS open,
MIN(value) AS low,
MAX(value) AS high
FROM mytable
GROUP BY name
ORDER BY name ASC

Related

mysql query lowest option sum

I've got a query returning the following:
ID | Price
---------------
1 | 20
1 | 30
1 | 15
2 | 10
2 | 12
2 | 20
3 | 1
3 | 0
3 | 0
4 | 0
4 | 0
4 | 7
I'm wondering if there's a way I can get the sum of the lowest value for each ID. So in this case it would return 25.
15+10+0+0
You can use a subquery selecting the min price for each id, then sum those values:
select sum(minprice) as overallprice
from (
select min(price) minprice
from yourtable
group by id) t
You can create a sub-query that finds the lowest price per id and take the results from that and sum them together. In pseudo-code:
select
sum(lowest_price)
from (select id, min(price) as lowest_price from prices group by id) lowest_prices
You can do a query like below
Select sum (a) from
(
Select min (price) as a from yourtable
Group by id
) t
Another approach using partition without using group by statement
select sum(price.min_price) from
(select distinct id,min(price) over(partition by id) as min_price from prices) price
Some other approaches would be to use MySQL user variables or a self left join..
MySQL user variable solution
Query
SELECT
SUM(prices.Price)
FROM (
SELECT
prices.Price
, CASE
WHEN #id != prices.id
THEN 1
ELSE 0
END AS isMinGroupValue
, (#id := prices.id)
FROM
prices
CROSS JOIN (
SELECT
#id := 0
) AS init_user_params
ORDER BY
prices.ID ASC
, prices.price ASC
) AS prices
WHERE
prices.isMinGroupValue = 1
see demo https://www.db-fiddle.com/f/nzWqMQAxd7mvq589R7WuZ8/0
Self left join solution
Query
SELECT
SUM(prices1.Price)
FROM
prices prices1
LEFT JOIN
prices prices2
ON
prices1.ID = prices2.ID
AND
prices1.price > prices2.price
WHERE
prices2.ID IS NULL
see demo https://www.db-fiddle.com/f/nzWqMQAxd7mvq589R7WuZ8/1
I would use correlation subquery :
select sum(t.price) as overallprice
from table t
where price = (select min(price) from table t1 where t1.id = t.id);

How to get average per group and figure out outliers in SQL

This is what my data looks like:
id | value | group
------------------
1 | 4 | abc
2 | 8 | def
3 | 100 | abc
4 | 8 | ghi
5 | 7 | abc
6 | 10 | ghi
I need to figure out the averages per group where outliers (for e.g. id = 3 for group = abc) are excluded. Then display the ouliers next to averages. For above data I am expecting something like this as result:
group = 'abc'
average = '5.5'
outlier = '100'
One method creates a subquery containing the stats for each group (mean and standard deviation), and then joins this back to the original table to determine which records are outliers, for which group.
SELECT t1.id,
t1.group AS `group`,
t2.valAvg AS average,
t1.value AS outlier
FROM yourTable t1
INNER JOIN
(
SELECT `group`, AVG(value) AS valAvg, STDDEV(value) AS valStd
FROM yourTable
GROUP BY `group`
) t2
ON t1.group = t2.group
WHERE ABS(t1.value - t2.valAvg) > t2.valStd -- any record whose value is MORE
-- than one standard deviation from
-- the mean is an outlier
Update:
It appears that, for some reason, your value column is actual varchar rather than a numeric type. This means you won't be able to do any math on it. So first, convert that column to integer via:
ALTER TABLE yourTable MODIFY value INTEGER;
If you only want outliers which are greater than the average then use the following WHERE clause:
WHERE t1.value - t2.valAvg > t2.valStd
You can exclude the value you don't need with a subquery
select `group`, avg/value) from my_table
where (group, value) not in (select `group`, max(value)
from my_table
group by `group`)
from my_table
group by `group`

Divide with the rollup value

I have this code where it sums up the hours of the employee and uses rollup to get the total of the hours:
SELECT IFNULL(users, 'Total') AS Employee,
SUM(actual) AS Amount
FROM table1
WHERE name = "ProjectName"
GROUP BY users
WITH ROLLUP
Employee | Amount
A | 15
B | 10
C | 10
Total | 35
What I would like to do for my third column (Percent) is to divide the sum(actual) to the value of the total to get the percentage.
But for that Percent column I don't need to get the Total Percent.
The total value is not constant to just 35.
Employee | Amount | Percent
A | 15 | 42.85
B | 10 | 28.57
C | 10 | 28.57
Total | 35 |
How can I do that?
Here's the sqlfiddle: http://sqlfiddle.com/#!2/4543b/5
This works as desired:
SET #project_name = 'ProjectName';
SELECT IFNULL(users, 'Total') AS Employee, SUM(actual) AS Amount,
IF(ISNULL(users), '', TRUNCATE(SUM(actual) / sum_table.amount_sum * 100, 2)
) AS Percent
FROM Table1
INNER JOIN (
SELECT SUM(actual) AS amount_sum
FROM Table1
WHERE name = #project_name
) AS sum_table
WHERE name = #project_name
GROUP BY users
WITH ROLLUP;
DEMO # SQL Fiddle
Perhaps a job best left to the logic tier of your application, but if you absolutely must do it in the data tier then you merely need to join your query with another that finds the overall total:
SELECT IFNULL(users, 'Total') AS Employee,
SUM(actual) AS Amount,
SUM(actual)/t.Total AS Percent
FROM Table1, (
SELECT SUM(actual) AS Total
FROM Table1
WHERE name = 'ProjectName'
) t
WHERE name = 'ProjectName'
GROUP BY users WITH ROLLUP
SELECT if(users is NULL,'Total',users) as Employee, sum(actual) as Amount,
(CASE
WHEN users is not null THEN CAST(sum(actual)/sum.sumAmt * 100 as DECIMAL(10,2))
END) as Percent
FROM Table1, (SELECT sum(actual) as sumAmt FROM Table1
WHERE name = 'ProjectName') sum
WHERE name = "ProjectName"
GROUP BY users
WITH ROLLUP
DEMO

same column calculation / percentage

How does someone in MYSQL compare a users percentage from a dates entry and score to another dates entry and score, effectively returning a users percentage increase from one date to another?
I have been trying to wrap my head around this question for a few days and am running out of ideas and feel my sql knowledge is limited. Not sure if I'm supposed to use a join or a subquery? The MYSQL tables consist of 3 fields, name, score, and date.
TABLE: userData
name score date
joe 5 2014-01-01
bob 10 2014-01-01
joe 15 2014-01-08
bob 12 2014-01-08
returned query idea
user %inc last date
joe 33% 2014-01-08
bob 17% 2014-01-08
It seems like such a simple function a database would serve yet trying to understand this is out of my grasp?
You need to use SUBQUERIES. Something like:
SELECT name,
((SELECT score
FROM userData as u2
WHERE u2.name = u1.name
ORDER BY date desc
LIMIT 1
)
/
(
SELECT score
FROM userData as u3
WHERE u3.name = u1.name
ORDER BY date desc
LIMIT 1,1
)
* 100.0
) as inc_perc,
max(date) as last_date
FROM userData as u1
GROUP BY name
Simple solution assuming that the formula for %Inc column = total/sum *100
select name,total/sum * 100, date from (
select name,sum(score) as total,count(*) as num,date from table group by name
)as resultTable
select a.name as [user],(cast(cast(b.score as float)-a.score as float)/cast(a.score as float))*100 as '% Inc',b.[date] as lastdate
from userdata a inner join userdata b on a.name = b.name and a.date < b.date
I guess you are looking for the % increse in the score compared to past date
Another way (and note, that I have another result. Based on the name "percinc", percentage increase, I calculated it in my eyes correctly. If you want your result, just calculate it with t1.score / t2.score * 100):
Sample data:
CREATE TABLE t
(`name` varchar(3), `score` int, `date` varchar(10))
;
INSERT INTO t
(`name`, `score`, `date`)
VALUES
('joe', 5, '2014-01-01'),
('bob', 10, '2014-01-01'),
('joe', 15, '2014-01-08'),
('bob', 12, '2014-01-08')
;
Query:
select
t1.name,
t1.score first_score,
t1.date first_date,
t2.score last_score,
t2.date last_date,
t2.score / t1.score * 100 percinc
from
t t1
join t t2 on t1.name = t2.name
where
t1.date = (select min(date) from t where t.name = t1.name)
and t2.date = (select max(date) from t where t.name = t1.name);
Result:
| NAME | FIRST_SCORE | FIRST_DATE | LAST_SCORE | LAST_DATE | PERCINC |
|------|-------------|------------|------------|------------|---------|
| joe | 5 | 2014-01-01 | 15 | 2014-01-08 | 300 |
| bob | 10 | 2014-01-01 | 12 | 2014-01-08 | 120 |
live demo

MySql select next lower number without using limit

Is it possible to select the next lower number from a table without using limit.
Eg: If my table had 10, 3, 2 , 1 I'm trying to select * from table where col > 10.
The result I'm expecting is 3. I know I can use limit 1, but can it be done without that?
Try
SELECT MAX(no) no
FROM table1
WHERE no < 10
Output:
| NO |
------
| 3 |
SQLFiddle
Try this query
SELECT
*
FROM
(SELECT
#rid:=#rid+1 as rId,
a.*
FROM
tbl a
JOIN
(SELECT #rid:=0) b
ORDER BY
id DESC)tmp
WHERE rId=2;
SQL FIDDLE:
| RID | ID | TYPE | DETAILS |
------------------------------------
| 2 | 28 | Twitter | #sqlfiddle5 |
Another approach
select a.* from supportContacts a inner join
(select max(id) as id
from supportContacts
where
id in (select id from supportContacts where id not in
(select max(id) from supportContacts)))b
on a.id=b.id
SQL FIDDLE:
| ID | TYPE | DETAILS |
------------------------------
| 28 | Twitter | #sqlfiddle5 |
Alternatively, this query will always get the second highest number based on the inner where clause.
SELECT *
FROM
(
SELECT t.col,
(
SELECT COUNT(distinct t2.col)
FROM tableName t2
WHERE t2.col >= t.col
) as rank
FROM tablename t
WHERE col <= 10
) xx
WHERE rank = 2 -- <<== means second highest
SQLFiddle Demo
SQLFiddle Demo (supports duplicate values)
If you want to get next lower number from table
you can get it with this query:
SELECT distinct col FROM table1 a
WHERE 2 = (SELECT count(DISTINCT(b.col)) FROM table1 b WHERE a.col >= b.col);
later again if you want to get third lower number you can just pass 3 in place of 2 in where clause
again if you want to get second higher number, just change the condition of where clause in inner query with
a.col <= b.col