mySQL group numeric data together with last image from same table - mysql

I have a table with cell viability data containing both numeric data and images (dose-response curves). The table can contain multiple rows for the same compound (uniqued by Batch ID). each row has a unique ID, as well as a date field. Now I want to group the data by Batch ID and produce the average EC50 values, but show this together with the last dose-response Curve generated for each compound Batch_ID. The code below will select the first Curve encountered for a particular compound Batch_ID. How can I select the last one instead, but still show it together with average EC50? Any tips appreciated!
SELECT Batch_ID, avg(EC50), Curve FROM CELL_VIABILITY GROUP BY Batch_ID
Example data:
ID Batch_ID EC50 Curve Date
1 ABC123 6.72 blob_1 18-06-15
2 ABC123 4.74 blob_2 18-07-10
3 ABC123 8.72 blob_3 18-08-22
4 DEF456 1.95 blob_4 18-06-15
5 DEF456 1.66 blob_5 18-07-10
6 DEF456 3.06 blob_6 18-08-22
Expected outcome:
Batch_ID EC50 Curve
ABC123 6.73 blob_3
DEF456 2.22 blob_6

Remember that data is unordered set. Without defining a specific order, we cannot determine what is last, first etc.
We can use the Date column to define the Order. Latest updated entry (Maximum Date value) can be considered as "Last".
We can then use Correlated Subquery to determine the Last Curve for a specific Batch_ID.
SELECT cv1.Batch_ID,
AVG(cv1.EC50),
MAX((SELECT cv2.Curve
FROM CELL_VIABILITY AS cv2
WHERE cv2.Batch_ID = cv1.Batch_ID
ORDER BY cv2.Date DESC LIMIT 1)) AS Last_Curve
FROM CELL_VIABILITY AS cv1
GROUP BY Batch_ID
Another approach would be using a Derived Table. We can get the last Date value for every Batch_ID. Then "Self-Join" to the table using the maximum value of the Date to get the Last Curve:
SELECT
cv1.Batch_ID,
cv1.average,
cv2.Curve
FROM
(
SELECT Batch_ID,
AVG(EC50) AS average,
MAX(Date) AS last_date
FROM CELL_VIABILITY
GROUP BY Batch_ID
) AS cv1
JOIN CELL_VIABILITY AS cv2
ON cv2.Batch_ID = cv1.Batch_ID AND
cv2.Date = cv1.last_date

Related

I need to create a query with a dynamic range of value in mysql

I have a table with a column called "Points", the value of this column have a min value of 0, and a max value of 100.000. I have to do an analysis per range of this column, so I wrote a query like this:
select case when Points between 0 and 5000 then '0 - 5000'
when Points between 5001 and 20000 then '50001 - 20000'
when Points > 20000 then '> 20000'
else 0 end RangeP
from Sales
The problem is that the customer wants to see for each 2.000 points from 0 to 100.000
If I wrote the query using the case, the query will get really big, so I'd like one way to get dynamically this range.
It is possible? Thank you very much
You may create a table which contains the ranges, and their text labels, and then join to it, e.g.
SELECT
s.Points,
COALESCE(r.label, 'range not found') AS range
FROM Sales s
LEFT JOIN
(
SELECT 0 AS start, 2000 AS end, '0 - 2000' AS label UNION ALL
SELECT 2001, 4000, '2001 - 4000' UNION ALL
...
SELECT 98000, 100000, '98000 - 100000'
) r
ON s.Points BETWEEN r.start AND r.end;
I have inlined the table of ranges, but you may create a formal table (or maybe a temp table) instead, and then replace the above subquery with just a reference to that table.

SQL SELECT ORDER BY multiple columns depending on value of other column

I have a table with the following columns:
id | revisit (bool) | FL (decimal) | FR (decimal) | RL (decimal) | RR (decimal) | date
I need to write a SELECT statement that will ORDER BY on multiple columns, depending on the value of the 'revisit' field.
ORDER BY 'revisit' DESC - records with this field having the value 1 will be first, and 0 will be after
If 'revisit' = 1 order by the lowest value that exists in FL, FR, RL and RR. So if record 1 has values 4.6, 4.6, 3.0, 5.0 in these fields, and record 2 has values 4.0, 3.1, 3.9, and 2.8 then record 2 will be returned first as it holds a lowest value within these four columns.
If 'revisit' = 0 then order by date - oldest date will be first.
So far I have the 'revisit' alone ordering correctly, and ordering by date if 'revisit' = 0, but ordering by the four columns simultaneously when 'revisit' = 1 does not.
SELECT *
FROM vehicle
ORDER BY
`revisit` DESC,
CASE WHEN `revisit` = 1 THEN `FL` + `FR` + `RR` + `RL` END ASC,
CASE WHEN `revisit` = 0 THEN `date` END ASC
Instead it seems to be ordering by the total of the four columns (which would make sense given addition symbols), so how do I ORDER BY these columns simultaneously, as individual columns, rather than a sum.
I hope this makes sense and thanks!
In your current query, you order by the sum of the four columns. You can use least to get the lowest value, so your order by clause could look like:
SELECT *
FROM vehicle
ORDER BY
`revisit` DESC,
CASE WHEN `revisit` = 1 THEN LEAST(`FL`, `FR`, `RR`, `RL`) END ASC,
CASE WHEN `revisit` = 0 THEN `date` END ASC
Of course this would sort only by the lowest value. If two rows would both share the same lowest value, there is no sorting on the second-lowest value. To do that is quite a bit harder, and I didn't really get from your question whether you need that.

Checking consecutive values at a MySQL query

I have a MySQL table like this:
ID - Time - Value
And I'm getting every pair of ID, Time (grouped by ID) where Value is greater than a certain threshold. So basicaly, I'm getting every ID which has at least one time a value greater than the threshold. The query looks like this:
SELECT ID, Time FROM mydb.MYTABLE
WHERE Value>%s AND Time>=%s AND Time<=%s
GROUP BY ID
EDIT: The Time checks allow to operate in a time range of my choice between all the data which is into the table; it has nothing else to do with what I am asking.
It works perfectly, but now I want to add some filtering: I want it to avoid those times the value is greater than the threshold (let's call it alarms) if the alarm hasn't happened also the Time just before or just after. I mean: if the alarm accurs at a single, isolated instant of time instead of two consecutive instants of time, I'll consider it is a false alarm and avoid it to be returned at the query response.
Of course I can do this with a call for each Id to check for this, but I'd like to do this in a single query to make it faster. I guess I could use conditionals, but I don't have that expertise at MySQL.
Any help?
EDIT2: Example for Threshold = 10
ID - Time - Value
1 - 2004 - 9
1 - 2005 - 11
1 - 2006 - 8
2 - 2107 - 12
2 - 2109 - 13
3 - 3402 - 11
3 - 3403 - 12
In this example, only ID 3 should be a valid alarm, since 2 consecutive time values for this ID have their value > threshold. ID 1 has a single, isolated alarm, so it should be filteres. For ID 2 there are 2 alarms, but not consecutive, so it should be also filtered.
Something like this:
10 - is a threshold
0 - minimum of the time period
100000 - maximum of the time period
select ID, min(Time)
from
(
SELECT ID, Time,
(select max(time) from t
where Time<t1.Time
and Id=t1.Id
and Value>10) LAG_G,
(select max(time) from t
where Time<t1.Time
and Id=t1.Id
and Value<=10) LAG_L,
(select min(time) from t
where Time>t1.Time
and Id=t1.Id
and Value>10) LEAD_G,
(select min(time) from t
where Time>t1.Time
and Id=t1.Id
and Value<=10) LEAD_L
FROM t as t1
WHERE Value>10 AND Time>=0 AND Time<=100000
) t3
where ifnull(LAG_G,0)>ifnull(LAG_L,0)
OR
ifnull(LEAD_G,100000)<ifnull(LEAD_L,100000)
GROUP BY ID
SQLFiddle demo
This query works for searching near records.
If you need to search records by Time (+1, -1 ) as you've mentioned in the comment try this query:
select ID, min(Time) from t as t1
where Value>10
AND Time>=%s2 AND Time<=%s1
and
(
Exists(select 1 from t where Value>10
and Id=t1.Id
and Time=t1.Time-1)
OR
Exists(select 1 from t where Value>10
and Id=t1.Id
and Time=t1.Time+1)
)
group by ID
SQLFiddle demo
such alarm ?
SELECT ID, Time , count(if(value>%treshold ,1,0)) alert_active
FROM mydb.MYTABLE
WHERE Value>%s3 AND Time>=%s2 AND Time<=%s1
GROUP BY ID;
i don't understand exactly:
In this example, only ID 3 should be a valid alarm, since 2
consecutive time values for this ID have their value > threshold. ID 1
has a single, isolated alarm, so it should be filteres. For ID 2 there
are 2 alarms, but not consecutive, so it should be also filtered.
I guess that You want filter alerts:
SELECT ID, Time
FROM mydb.MYTABLE
WHERE Value>%s3 AND Time>=%s2 AND Time<=%s1
GROUP BY ID
having value<%treshold;

MS Access Crosstab query - sum of columns reported by month

I am putting together a Crosstab and I want a report for multiple column values (all numbers) grouped by month. Here is the SQL I used. I understand that this won't bring back the desired results. every "timex" column has a different number in it. I want a query that will return the sum of the column grouped by month.
TRANSFORM Sum(tblTimeTracking.time1+ tblTimeTracking.time2+ tblTimeTracking.time3+ tblTimeTracking.time4+ tblTimeTracking.time5+ tblTimeTracking.time6+ tblTimeTracking.time7+ tblTimeTracking.time8+ tblTimeTracking.time9+ tblTimeTracking.time10+ tblTimeTracking.time11+ tblTimeTracking.time12+ tblTimeTracking.time13+ tblTimeTracking.time14+ tblTimeTracking.time15+ tblTimeTracking.time16+ tblTimeTracking.time17+ tblTimeTracking.time18+ tblTimeTracking.time19+ tblTimeTracking.time20+ tblTimeTracking.time21+ tblTimeTracking.time22 ) AS Total
SELECT tbl_vlookup.Manager AS Manager
FROM tbl_vlookup INNER JOIN tblTimeTracking ON tbl_vlookup.[Associate Name] = tblTimeTracking.Associate
GROUP BY tbl_vlookup.Manager
PIVOT Format([Day],"yyyy-mm");
Associate Day Time 1 Time 2 Time 3 Time 4 Time 5 Time 6 Time 7
John Smith 12/1/9999 1 0 0 5.5 1 0.25 0.25
Something like this:
TRANSFORM Sum(q.Time1) AS SumOfTime1
SELECT q.Associate, q.Day
FROM (SELECT t.Associate, t.Day, t.Time1,"Time1" As TimeType
FROM tbl t
UNION ALL
SELECT t.Associate, t.Day, t.Time2,"Time2" As TimeType
FROM tbl t
UNION ALL
SELECT t.Associate, t.Day, t.Time3,"Time3" As TimeType
FROM tbl t) AS q
GROUP BY q.Associate, q.Day
PIVOT q.TimeType;
As I mentioned, you need to flatten the table. It only seems to be different types of data :)

Break Numbers List Into Min and Max Ranges

Brain is not working today and my google skills are failing me.
I have a column of numbers ranging from 1 - 1000. I want to dump the min and max values for 100 (or whatever I chose) record ranges into a temp table. The plan is to use this temp table to process ranges of records (in this example 100 at a time) in a larger table.
Swear I have done this before with a CTE but then I had something to group on. Here I just want to break up a single list of numbers into ranges of X.
The output from the temp table should look like:
Min Max
0 99
100 199
200 299
300 399
etc.
Thanks!
You can use this trick from Stuart Ainsworth:
http://codegumbo.com/index.php/2009/01/25/building-ranges-using-a-dynamically-generated-numbers-table/
Numbers tables are awesome, but he uses a dynamically generated numbers table, which is even awesome...r.
If you know all numbers are present in the source table, you can use a recursive CTE to generate the number ranges:
; with numbers as
(
select 0 as a
, 99 as b
union all
select a+100
, b+100
from numbers
where a < 900
)
select *
from numbers
If the source table is sparsely populated, you can limit it to numbers that are actually present like:
... insert CTE from above here ...
select min(ot.NumberColumn)
, max(ot.NumberColumn)
from numbers
left join
OtherTable ot
on ot.NumberColumn between numbers.a and numbers.b
group by
numbers.a
enter code hereI have been having a play with a CTE after you posted this and came up with the following, I would be interested to hear if it works for you at all.
DECLARE #segment int = 100
;
WITH _CTE
(rowNum, value)
AS
(
SELECT ROW_NUMBER() OVER(ORDER BY col01) -1, col01
FROM dbo.testTable
)
SELECT rowNum/#segment AS Bucket, MIN(Value) AS MinVal, MAX(Value) AS MaxVal
FROM _CTE
group by rowNum/#segment
ORDER BY Bucket
;
col01 in this case is the column that you want the min/max range values from, as is TestTable.