I am working on queries which are based on time and I would like to get the optimal way to calculate opened cases during the day. I do have table task_interval which has 2 columns start and end.
JSON example:
[
{
"start" : "2019-10-15 20:41:38",
"end" : "2019-10-16 01:44:03"
},
{
"start" : "2019-10-15 20:43:52",
"end" : "2019-10-15 22:18:54"
},
{
"start" : "2019-10-16 20:21:38",
"end" : null,
},
{
"start" : "2019-10-17 01:42:35",
"end" : null
},
{
"create_time" : "2019-10-17 03:15:57",
"end_time" : "2019-10-17 04:14:17"
},
{
"start" : "2019-10-17 03:16:44",
"end" : "2019-10-17 04:14:31"
},
{
"start" : "2019-10-17 04:15:23",
"end" : "2019-10-17 04:53:28"
},
{
"start" : "2019-10-17 04:15:23",
"end" : null,
},
]
The result of query should return:
[
{ time: '2019-10-15', value: 1 },
{ time: '2019-10-16', value: 1 }, // Not 2! One task from 15th has ended
{ time: '2019-10-17', value: 3 }, // We take 1 continues task from 16th and add 2 from 17th which has no end in same day
]
I have written the query which will return cumulative sum of started tasks which end date is not same as started date:
SELECT
time,
#running_total:=#running_total + tickets_number AS cumulative_sum
FROM
(SELECT
CAST(ti.start AS DATE) start,
COUNT(*) AS tickets_number
FROM
ticket_interval ti
WHERE
DATEDIFF(ti.start, ti.end) != 0
OR ti.end IS NULL
GROUP BY CAST(ti.start AS DATE)) X
JOIN
(SELECT #running_total:=0) total;
If you are running MySQL 8.0, one option is to unpivot, then aggregate and perform a window sum to compute the running count:
select
date(dt) dt_day,
sum(sum(no_tasks)) over(order by date(dt)) no_tasks
from (
select start_dt dt, 1 no_tasks from mytable
union all select end_dt, -1 from mytable where end_dt is not null
) t
group by date(dt)
order by dt_day
Side note: start and end are reserved words, hence not good choices for column names. I renamed those to start_dt and end_dt.
In earlier versions, we can emulate the window sum with a user variable, like so:
select
dt_day,
#no_tasks := #no_tasks + no_tasks no_tasks
from (
select date(dt) dt_day, sum(no_tasks) no_tasks
from (
select start_dt dt, 1 no_tasks from mytable
union all select end_dt, -1 from mytable where end_dt is not null
) t
group by dt_day
order by dt_day
) t
cross join (select #no_tasks := 0) x
order by dt_day
Demo on DB Fiddle - both queries yield:
dt_day | no_tasks
:--------- | -------:
2019-10-15 | 1
2019-10-16 | 1
2019-10-17 | 3
Related
I am trying to calculate a cumulative sum of values that can occur more than once on a given day. Therefore i only want to keep the max value for each given day.
currently i have:
SELECT
DATE_FORMAT(created_at, '%d/%m') AS day1,
SUM(principal) over (order by created_at) AS tot
FROM loan;
OUTPUT:
[
{
"day1" : "21/08",
"tot" : 200
},
{
"day1" : "21/08",
"tot" : 1200
},
{
"day1" : "21/08",
"tot" : 2200
},
{
"day1" : "21/08",
"tot" : 2500
},
{
"day1" : "25/08",
"tot" : 4500
},
{
"day1" : "25/08",
"tot" : 6500
},
{
"day1" : "27/08",
"tot" : 7000
},
{
"day1" : "27/08",
"tot" : 7600
}
]
I know i need to group it by the day, but it needs to take the max value for each group of days. any ideas?
When i group by like so:
SELECT
DATE_FORMAT(created_at, '%d/%m') AS day1,
SUM(principal) over (order by created_at) AS tot
FROM loan
GROUP BY day1;
I Get:
[
{
"day1" : "21/08",
"tot" : 1000
},
{
"day1" : "25/08",
"tot" : 3000
},
{
"day1" : "27/08",
"tot" : 3500
},
{
"day1" : "30/08",
"tot" : 4200
}
]
Which is obviously wrong. Im not sure why it gives that output. But how could i get it to be correct?
You need to nest the sums:
SELECT DATE_FORMAT(created_at, '%d/%m') AS day1,
SUM(SUM(principal)) OVER (ORDER BY MIN(created_at)) AS tot
FROM loan
GROUP BY day1;
I am surprised that your version works at all. It would fail in most databases, because neither principal nor created_at are aggregation keys.
Presumably, you want:
select
date_format(created_at, '%d/%m') as day1,
max(sum(principal) over (order by created_at)) as tot
from loan
group by day1
Actually, you could also express this with distinct, and a slightly different order by clause:
select distinct
date_format(created_at, '%d/%m') as day1,
sum(principal) over (order by date(created_at)) as tot
from loan
I am not a big fan of the way you represent dates; if your data spreads over more than one year, the result will become ambiguous (in the second option), or even wrong (in the first query). I would recommend adding the year to the date representation, or simply date(created_at), which basically truncates the time portion.
Gleaning several articles online, including this one with a CTE, and this one WITHOUT a CTE, I have been successful in getting the data I need, including a count of the results. However, I need this count to be in a specific place in the JSON object... Basically, I know how to get a rowset into a specific JSON structure with FOR JSON PATH, ROOT ('data'), etc.
However, I do not know how to get the "recordsFiltered" into the root of my JSON output. This count is is derived using COUNT(*) OVER () AS recordsFiltered
Basically, I need my structure to look like this (see below)... How do I get "recordsFiltered" into the root $. of the JSON result without it repeating a billion times under the "data":[] section?
The best idea I can come up with is to create a temporary table, and then use that to structure the JSON. But, I want to do it the fancy SQL way, if one exists, using SELECT statements or CTEs where applicable.
{
"draw": 1,
"recordsTotal": 57,
"recordsFiltered": 57, // <<<--- need records filtered HERE
"data": [
{
"DT_RowId": "row_3",
"recordsFiltered": "69,420", // <<<---- NOT HERE!!!
"first_name": "Angelica",
"last_name": "Ramos",
"position": "System Architect",
"office": "London",
"start_date": "9th Oct 09",
"salary": "$2,875"
},
...
]
}
Here is the example SQL code:
SELECT
COUNT(*) OVER () AS recordsFiltered,
id,
a,
b
FROM t1
WHERE
(#Search IS NULL OR
id LIKE '%'+#Search+'%' OR
a LIKE '%'+#Search+'%' OR
b LIKE '%'+#Search+'%')
ORDER BY
CASE
WHEN #SortDir = 'ASC' THEN
CASE #SortCol
WHEN 0 THEN id
WHEN 1 THEN a
WHEN 2 THEN b
END
END desc,
CASE
WHEN #SortDir = 'desc' THEN
CASE #SortCol
WHEN 0 THEN id
WHEN 1 THEN a
WHEN 2 THEN b
END
END DESC
OFFSET #DisplayStart ROWS
FETCH NEXT #DisplayLength ROWS ONLY
for json path, root ('data')
Looks like you need to generate your table results, then use two (or more?) sub-queries
Here's a simplified example:
declare #tbl table (ID int identity, Col1 varchar(50), Col2 int)
insert into #tbl (Col1, Col2) values ('A',1),('B',2),('C',3)
select
(select count(1) from #tbl) as 'total',
(select * from #tbl for json path) as 'data'
for json path
produces:
[
{
"total": 3,
"data": [
{
"ID": 1,
"Col1": "A",
"Col2": 1
},
{
"ID": 2,
"Col1": "B",
"Col2": 2
},
{
"ID": 3,
"Col1": "C",
"Col2": 3
}
]
}
]
Without knowing the rest of your code/schema, here's my guess at your needed query:
select
*
into
#MyTable
from
t1
WHERE
(#Search IS NULL OR
id LIKE '%'+#Search+'%' OR
a LIKE '%'+#Search+'%' OR
b LIKE '%'+#Search+'%')
select
(select count(*) from #MyTable) as recordsFiltered,
(
select
id,
a,
b
from
#MyTable
ORDER BY
CASE
WHEN #SortDir = 'ASC' THEN
CASE #SortCol
WHEN 0 THEN id
WHEN 1 THEN a
WHEN 2 THEN b
END
END desc,
CASE
WHEN #SortDir = 'desc' THEN
CASE #SortCol
WHEN 0 THEN id
WHEN 1 THEN a
WHEN 2 THEN b
END
END DESC
OFFSET #DisplayStart ROWS
FETCH NEXT #DisplayLength ROWS ONLY
for json path
) as [data]
for json path
Using a CTE:
with cte as ()
select
*
from
t1
WHERE
(#Search IS NULL OR
id LIKE '%'+#Search+'%' OR
a LIKE '%'+#Search+'%' OR
b LIKE '%'+#Search+'%')
)
select
(select count(*) from cte) as recordsFiltered,
(
select
id,
a,
b
from
cte
ORDER BY
CASE
WHEN #SortDir = 'ASC' THEN
CASE #SortCol
WHEN 0 THEN id
WHEN 1 THEN a
WHEN 2 THEN b
END
END desc,
CASE
WHEN #SortDir = 'desc' THEN
CASE #SortCol
WHEN 0 THEN id
WHEN 1 THEN a
WHEN 2 THEN b
END
END DESC
OFFSET #DisplayStart ROWS
FETCH NEXT #DisplayLength ROWS ONLY
for json path
) as [data]
for json path
I just installed MySQL 5.7.27 and I would like to use some Json fields so, I created some records, for example this value in a field:
{
"Intitule": {
"name": "Intitule de la formation",
"stats": false,
"is_array": false,
"is_filter": true,
"chart": "pie",
"col": "6"
},
"Fin": {
"name": "Date de fin",
"stats": false,
"is_array": false,
"is_filter": false,
"chart": "pie",
"col": "6"
}
}
I would like to know how can I retrieve keys of elements which contain "is_filter":true, dans the associated "name"
I tried with JSON_SEARCH, JSON_EXTRACT but I think I don't use it well.
For example, for this value, I expect the output will be:
Key: Intitule
Name: Intitule de la formation
Because is_filter is true, but not for "Fin"
Thank you for your help!
Like i said, it can be a challenge parsing JSON in MySQL as you are dealing with text based keys here.
So you would need to use JSON_KEYS() to get those in combination with a number generator a dynamic JSON path is generated to be used in JSON_EXTRACT()
MySQL 's 8 function JSON_TABLE() makes it much much more easy..
Keep in mind this answer is purely meant for educational fun
Bill Karwin 's comment
You should either structure your JSON differently to support the
search you need to do, or else you should not use JSON. Just store the
data in multiple rows, and then SELECT * FROM mytable WHERE is_filter = true -
Query
SELECT
JSON_UNQUOTE(
JSON_EXTRACT(json , CONCAT('$.', SUBSTRING_INDEX(
SUBSTRING_INDEX(json_parsed, ',', number_generator.number)
, ','
, -1
), '.name'))) AS name
FROM (
SELECT
#row := #row + 1 AS number
FROM (
SELECT 0 UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9
) row1
CROSS JOIN (
SELECT 0 UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9
) row2
CROSS JOIN (
SELECT #row := 0
) init_user_params
) AS number_generator
CROSS JOIN (
SELECT
SUBSTRING(json_keys, 2, json_keys_length - 2) AS json_parsed
, json_keys
, json
, JSON_LENGTH(json_keys) AS json_array_length
FROM (
SELECT
JSON_KEYS(record.json) AS json_keys
, json
, LENGTH(JSON_KEYS(record.json)) AS json_keys_length
FROM (
SELECT
'{
"Intitule": {
"name": "Intitule de la formation",
"stats": false,
"is_array": false,
"is_filter": true,
"chart": "pie",
"col": "6"
},
"Fin": {
"name": "Date de fin",
"stats": false,
"is_array": false,
"is_filter": false,
"chart": "pie",
"col": "6"
}
}' AS json
FROM
DUAL
) AS record
) AS json_information
) AS json_init
WHERE
number_generator.number BETWEEN 0 AND json_array_length
AND
JSON_EXTRACT(json , CONCAT('$.', SUBSTRING_INDEX(
SUBSTRING_INDEX(json_parsed, ',', number_generator.number)
, ','
, -1
), '.is_filter')) = true
Result
| name |
| ------------------------ |
| Intitule de la formation |
see demo
This answer is to expand on the comment I made above.
When you store data in non-relational format, you should understand that the structure of the data must serve the queries you need to run. You can't just store any data and expect it to be easy to query.
For example, if you need to search for the subdocument where is_filter is true, then store your JSON like this:
{
"filter": {
"label": "Intitule",
"name": "Intitule de la formation",
"stats": false,
"is_array": false,
"chart": "pie",
"col": "6"
},
"not_filter": {
"label": "Fin",
"name": "Date de fin",
"stats": false,
"is_array": false,
"chart": "pie",
"col": "6"
}
}
If you need to use a variety of queries against the data, then using non-relational data like JSON is probably not as useful as storing the data in a relational manner.
mysql> select * from mytable;
+----------+--------------------------+-------+----------+-----------+-------+------+
| label | name | stats | is_array | is_filter | chart | col |
+----------+--------------------------+-------+----------+-----------+-------+------+
| Intitule | Intitule de la formation | 0 | 0 | 1 | pie | 6 |
| Fin | Date de fin | 0 | 0 | 0 | pie | 6 |
+----------+--------------------------+-------+----------+-----------+-------+------+
Then you can query by any attribute:
SELECT label FROM mytable WHERE is_filter = true;
NoSQL is more flexible than SQL for storing data.
SQL is more flexible than NoSQL for querying data.
I have the following document model in my couchbase db
{
type:"account"
id : "123",
transactions: [
{
type : "credit",
value : 100
},
{
type : "debit",
value : 10
}
]
}
How do i query all the account Ids and their sum of all credits ?
Using AS ARRAY functions https://docs.couchbase.com/server/6.0/n1ql/n1ql-language-reference/arrayfun.html
SELECT d.id,
ARRAY_SUM(ARRAY v.`value` FOR v IN d.transactions WHEN v.type = "credit" END) AS s
FROM default AS d
WHERE d.type = "account";
OR
Using subquery expression https://docs.couchbase.com/server/6.0/n1ql/n1ql-language-reference/subqueries.html
SELECT d.id,
(SELECT RAW SUM(d1.`value`)
FROM d.transactions AS d1
WHERE d1.type = "credit")[0] AS s
FROM default AS d
WHERE d.type = "account";
I would like to convert below mysql query to mongodb query.
SELECT substring(o.schedule_datetime,1,4) 'Year',
SUM(IF(o.order_status in ('SUCCESS','#SUCCESS'),1,0)) 'SUCCESS'
FROM (
select group_concat(distinct ifnull(os.order_status,'') order by os.order_status
separator '#') 'order_status',schedule_datetime
from order_summary os group by order_number
)o group by 1 desc;
For Example: I have sample table
id order_number product_number order_status schedule_datetime
1 001 001.1 SUCCESS 20180103
2 001 001.2 SUCCESS 20180102
3 111 111.1 SUCCESS 20171225
4 111 111.2 SUCCESS 20171224
5 222 222.1 INPROGRESS 20171122
6 222 222.2 SUCCESS 20171121
I get the output using above mysql query for order status SUCCESS
Year SUCCESS
2018 1
2017 1
I have used separator(#) to combine multiple statues as string and get the desired result by status, to get INPROGRESS i will be just changing SUM funtion as shown below :
SUM(IF(o.order_status in ('INPROGRESS','INPROGRESS#SUCCESS', '#INPROGRESS','#INPROGRESS#SUCCESS'),1,0)) 'INPROGRESS'
I have tried to write the mongodb query, but got stuck how to combine sum and if condition as well group_concat with seperator as i used in mysql query.
db.order_summary.aggregate([
{ "$project" :
{ "orderDate" : 1 , "subOrderDate" : { "$substr" : [ "$order_date" , 0 , 4]},
"order_number":"$order_number"
},
} ,
{ "$group":{
"_id": { "order_number" : "$order_number", "Year": "$subOrderDate", "order_status":{"$addToSet":{"$ifNull":["$order_status",'']}}}
}
},
{ "$group": {
"_id": "$_id.Year", "count": { "$sum": 1 }
}
},
{ "$sort" : { "_id" : -1}}
])
Anyone help will be much appreciated, thanks
There is no Group_Concat kind of functionality in mongodb.
You can compare arrays for matching values in last group with $in operator in 3.4 version.
First $group to get all the distinct order status for a combination for order number and order status.
$sort to sort the order statuses.
Second $group to push all the sorted status values by order number.
Final $group to compare the statuses for each year against the input list of status and output total count for all matches.
db.order_summary.aggregate([{"$project":{
"schedule_datetime":1,
"order_number":1,
"order_status":{"$ifNull":["$order_status",""]}
}},
{"$group":{
"_id":{
"order_number":"$order_number",
"order_status":"$order_status"
},
"schedule_datetime":{"$first": "$schedule_datetime"}
}},
{"$sort":{"_id.order_status": 1}},
{"$group":{
"_id":{
"order_number":"$_id.order_number"
},
"schedule_datetime":{"$first": "$schedule_datetime"},
"order_status":{"$push": "$_id.order_status"}
}},
{"$group":{
"_id":{"$substr":["$schedule_datetime",0,4]},
"count":{
"$sum":{
"$cond": [
{"$in": ["$order_status",[["SUCCESS"], ["","SUCCESS"]]]},
1,
0]
}
}
}},
{"$sort":{"_id":-1}}])