GROUP_CONCAT limiting to 100 values is not working - mysql

I am trying to extract a sum of modified history field limited by 100 and grouped by each account, but the query below extracts sum of all rows from CDR_Accounts with charged_quantity = 60 and does not limit to 100.
table1 = table2 and these are the temporary tables. I have simplified the query and filtered all data in temporary tables.
Nevertheless, the query still gives me the total sum of all 'history' fields, however I need the sum for only 100 of them.
table1/table2 format:
+-----------+----------+------------+-------------+----------------------+
| i_account | id | account_id | CLD | history |
+-----------+----------+------------+-------------+----------------------+
| 10272 | 46479968 | 0814781293 | 27815212963 | +1x60#1.32d100%=1.32 |
| 6316 | 46480100 | 0813741427 | 27780233136 | +1x60#1.32d100%=1.32 |
| 6316 | 46480107 | 0813741427 | 27780233136 | +1x60#1.32d100%=1.32 |
| 13830 | 46480435 | 0814346396 | 27781356515 | +1x60#1.32d100%=1.32 |
| 13830 | 46480436 | 0814346396 | 27781356515 | +1x60#1.32d100%=1.32 |
+-----------+----------+------------+-------------+----------------------+
Account
SELECT sum(SUBSTRING_INDEX(history,'=',-1)),
cdr.i_account,
cdr.account_id
FROM table1 cdr
WHERE cdr.id IN
(SELECT SUBSTRING_INDEX(group_concat(s.id SEPARATOR ','), ',', 100) id
FROM table2 s
GROUP BY id
)
GROUP BY i_account;

I'm not clear about what your query does is. However limit applies to the final result which includes the grouping as well. if you need to limit the result of GROUP_CONCAT, first apply the limit and then apply the grouping

Related

MySQL concat columns and rows from multiple tables

I'm trying to concatenate data from three related tables according to:
orders orderrow orderrow_op
+----+ +----+----------+ +----+-------------+
| id | | id | id_order | | id | id_orderrow |
+----+ +----+----------+ +----+-------------+
| 1 | | 1 | 1 | | 1 | 1 |
| 2 | | 2 | 1 | | 2 | 1 |
| 3 | | 3 | 2 | | 3 | 2 |
+----+ | 4 | 3 | | 4 | 3 |
+----+----------+ | 5 | 3 |
| 6 | 3 |
+----+-------------+
The result i'm looking for is something like:
orderops (Desired Result)
+----------+-----------------+
| id_order | id_row:id_ops |
+----------+-----------------+
| 1 | 1:(1,2); 2:(3); |
| 2 | 3:(4,5,6) |
| 3 | 4:NULL |
+----------+-----------------+
I.e i want the operations and rows all be displayed on one row related to the order. So far i've tried things like:
SELECT
db.orders.id AS orderid,
db.orderrow.id AS rowids,
GROUP_CONCAT(DISTINCT db.orderrow.id) AS a,
GROUP_CONCAT(db.orderrow.id, ':', db.orderrow_op.id) AS b
FROM
db.orders
LEFT JOIN db.orderrow ON db.orders.id = db.orderrow.id_order
LEFT JOIN db.orderrow_op ON db.orderrow.id = db.orderrow_op.id_orderrow
GROUP BY orderid
Where in column 'a' i get the row ids and in column 'b' i get the operation_ids with corresponding row_id prepended. I'd like to combine the two into a single column such that related values in 'b' will start of with id from 'a' and only show once.
I'm fairly new to MySQL so i don't know if this is even possible or if i'ts a good idea at all? The aim is to structure the data into JSON for delivery via REST application so perhaps it's better to deliver the rows directly to the webserver and handle json parsing over there? I just figured that this approach might be faster.
This is not the nicest query but it's working for your example table setup.
SELECT
o.id AS id_order,
group_concat(sub.ops
SEPARATOR ' ') AS id_row_id_ops
FROM
(SELECT
orderrow.id_order,
IF(isnull(l3.ops), concat(orderrow.id, ':', 'NULL'), concat(orderrow.id, ':', l3.ops)) as ops
FROM
orderrow
LEFT JOIN (SELECT
orderrow_op.id_orderrow,
concat('(', group_concat(orderrow_op.id), '); ') as ops
FROM
orderrow_op
GROUP BY orderrow_op.id_orderrow) l3 ON l3.id_orderrow = orderrow.id) sub
LEFT JOIN
orders o ON o.id = sub.id_order
GROUP BY o.id;
One of the things to mind is the LEFT JOIN and that you need to cast a "null" value to a "null" text (otherwise your element 4 will vanish).
The output:

MySQL getting maximum of column with fallback to second column

+------+---------+--------+---------+---------+---------+
| id | user_id | obj_id | created | applied | content |
+------+---------+--------+---------+---------+---------+
| 1 | 1 | 1 | 1 | 1 | ... |
| 2 | 1 | 2 | 1 | 1 | ... |
| 3 | 1 | 1 | 1 | 2 | ... |
| 4 | 1 | 2 | 2 | 2 | ... |
| 5 | 2 | 1 | 1 | 1 | ... |
| 6 | 2 | 2 | 1 | 1 | ... |
+------+---------+--------+---------+---------+---------+
I have a table similar to the one above. id, user_id and obj_id are foreign keys; created and applied are timestamps stored as integers. I need to get the entire row, grouped by user_id and obj_id, with the maximum value of applied. If two rows have the same applied value, I need to favour the maximum value of created. So for the above data, my desired output is:
+------+---------+--------+---------+---------+---------+
| id | user_id | obj_id | created | applied | content |
+------+---------+--------+---------+---------+---------+
| 1 | 1 | 1 | 1 | 1 | ... |
| 4 | 1 | 2 | 2 | 2 | ... |
| 5 | 2 | 1 | 1 | 1 | ... |
| 6 | 2 | 2 | 1 | 1 | ... |
+------+---------+--------+---------+---------+---------+
My current solution is to get everything ordered by applied then created:
select * from data order by applied desc created desc;
and sort things out in the code, but this table gets pretty big and I'd like an SQL solution that just gets the data I need.
select *
from my_table
where id in (
/* inner subquery b */
select max(id)
from my_table where
(user_id, obj_id, applied, created) in (
/* inner subquery A */
select user_id, obj_id, max(applied), max(created)
from my_table
group by user_id, obj_id
)
);
Then inner subquery A return the (distinct) rows having user_id, obj_id, max(applied), max(created). Using these with in clause the subquery B retrive a list of single ID each realated the a row with a proper value of user_id, obj_id, max(applied), max(created). so you have a collection of valid id for getting your result.
The main select use these ID for select the result you need.
Thanks to Mark Heintz in the comments, this answer got me to where I need to be.
SELECT
data.id,
data.user_id,
data.obj_id,
data.created,
data.applied,
data.content
FROM data
LEFT JOIN data next_max_applied ON
next_max_applied.user_id = data.user_id AND
next_max_applied.obj_id = data.obj_id AND (
next_max_applied.applied > data.applied OR (
next_max_applied.applied = data.applied AND
next_max_applied.created > data.created
)
)
WHERE next_max_applied.applied IS NULL
GROUP BY user_id, obj_id;
Go read the answer for details on how it works; the left join tries to find a more recently applied row for the same user and object. If there isn't one, it will find a row applied at the same time, but created more recently.
The above means that any row without a more recent row to replace it will have a next_max_applied.applied value of null. These rows are filtered for by the IS NULL clause.
Finally, the group by clause handles any rows that have identical user, object, applied and created columns.

MySQL: join grouped data to table - only first row joined

I have problem joining tables with following content:
Table RingOrderItem:
+----+-------------+--------+
| ID | ID_RingType | Amount |
+----+-------------+--------+
| 1 | A | 100 |
| 2 | B | 50 |
| 3 | A | 500 |
| 4 | C | 100 |
+----+-------------+--------+
Grouped table Rings - result of SELECT min(Rings.Number) AS Number, ID_RingType FROM Rings GROUP BY ID_RingType statement:
+--------+-------------+
| Number | ID_RingType |
+--------+-------------+
| 1 | A |
| 1 | B |
+--------+-------------+
I want to retrieve all records from RingOrderItem and join number from grouped table Rings to them, for which I used this query:
SELECT
roi.ID,
roi.ID_RingOrder,
roi.ID_RingType,
roi.Amount,
min(r.Number) AS `FromValue`,
min(r.Number) + roi.Amount - 1 AS `ToValue`
FROM
RingOrderItem AS roi
LEFT JOIN
(SELECT min(Rings.Number) AS Number, ID_RingType FROM Rings
GROUP BY ID_RingType)
AS r ON r.ID_RingType = roi.ID_RingType;
For some reason, I get only the first row from RingOrderItem table:
+----+--------------+-------------+--------+-----------+---------+
| ID | ID_RingOrder | ID_RingType | Amount | FromValue | ToValue |
+----+--------------+-------------+--------+-----------+---------+
| 1 | 1 | A | 100 | 1 | 100 |
+----+--------------+-------------+--------+-----------+---------+
I want all rows, and if the data can not be joined (value C in ID_RingType), than return simply NULL.
Thanks,
Zbynek
I don't think you need the two min() functions on the main query since you are already getting the min values in the sub query.
Also, it's not really a good idea to do math to a column that might be NULL
Try this:
SELECT
roi.ID,
roi.ID_RingOrder,
roi.ID_RingType,
roi.Amount,
r.Number AS FromValue,
COALESCE(r.Number, 0) + roi.Amount - 1 AS ToValue
FROM
RingOrderItem AS roi
LEFT JOIN
(
SELECT
MIN(Rings.Number) AS Number,
ID_RingType
FROM
Rings
GROUP BY
ID_RingType
) AS r ON roi.ID_RingType = r.ID_RingType;
Also, switch your left join ON clause to have the first table listed first.

LIMIT results to n unique column values?

I have some MySQL results like this:
---------------------------
| name | something_random |
---------------------------
| john | ekjalsdjalfjkldd |
| alex | akjsldfjaekallee |
| alex | jkjlkjslakjfjflj |
| alex | kajslejajejjaddd |
| bob | ekakdie33kkd93ld |
| bob | 33kd993kakakl3ll |
| paul | 3k309dki595k3lkd |
| paul | 3k399kkfkg93lk3l |
etc...
This goes on for 1000's of rows of results. I need to limit the number of results to the first 50 unique names. I think there is a simple solution to this but I'm not sure.
I've tried using derived tables and variables but can't quite get there. If I could figure out how to increment a variable once every time a name is different I think I could say WHERE variable <= 50.
UPDATED
I've tried the Inner Join approach(es) suggested below. The problem is this:
The subselect SELECT DISTINCT name FROM testTable LIMIT 50 grabs the first 50 distinct names. Perhaps I wasn't clear enough in my original post, but this limits my query too much. In my query, not every name in the table is returned in the result. Let me modify my original example:
----------------------------------
| id | name | something_random |
----------------------------------
| 1 | john | ekjalsdjalfjkldd |
| 4 | alex | akjsldfjaekallee |
| 4 | alex | jkjlkjslakjfjflj |
| 4 | alex | kajslejajejjaddd |
| 6 | bob | ekakdie33kkd93ld |
| 6 | bob | 33kd993kakakl3ll |
| 12 | paul | 3k309dki595k3lkd |
| 12 | paul | 3k399kkfkg93lk3l |
etc...
So I added in some id numbers here. These ID numbers pertain to the people's names in the tables. So you can see in the results, not every single person/name in the table is necessarily in the result (due to some WHERE condition). So the 50th distinct name in the list will always have an ID number higher than 49. The 50th person could be id 79, 234, 4954 etc...
So back to the problem. The subselect SELECT DISTINCT name FROM testTable LIMIT 50 selects the first 50 names in the table. That means that my search results will be limited to names that have ID <=50, which is too constricting. If there are certain names that don't show up in the query (due to some WHERE condition), then they are still counted as one of the 50 distinct names. So you end up with too few results.
UPDATE 2
To #trapper: This is a basic simplification of what my query looks like:
SELECT
t1.id,
t1.name,
t2.details
FROM t1
LEFT JOIN t2 ON t1.id = t2.some_id
INNER JOIN
(SELECT DISTINCT name FROM t1 ORDER BY id LIMIT 0,50) s ON s.name = t1.name
WHERE
SOME CONDITIONS
ORDER BY
t1.id,
t1.name
And my results look like this:
----------------------------------
| id | name | details |
----------------------------------
| 1 | john | ekjalsdjalfjkldd |
| 3 | alex | akjsldfjaekallee |
| 3 | alex | jkjlkjslakjfjflj |
| 4 | alex | kajslejajejjaddd |
| 6 | bob | ekakdie33kkd93ld |
| 6 | bob | 33kd993kakakl3ll |
| 12 | paul | 3k309dki595k3lkd |
| 12 | paul | 3k399kkfkg93lk3l |
...
| 37 | bill | kajslejajejjaddd |
| 37 | bill | ekakdie33kkd93ld |
| 41 | matt | 33kd993kakakl3ll |
| 50 | jake | 3k309dki595k3lkd |
| 50 | jake | 3k399kkfkg93lk3l |
----------------------------------
The results stop at id=50. There are NOT 50 distinct names in the list. There are only roughly 23 distinct names.
My MySql syntax may be rusty, but the idea is to use a query to select the top 50 distinct names, then do a self-join on name and select the name and other information from the join.
select a.name, b.something_random
from Table b
inner join (select distinct name from Table order by RAND() limit 0,50) a
on a.name = b.name
SELECT DISTINCT name FROM table LIMIT 0,50
Edited: Ahh yes I misread question first time, this should do the trick though :)
SELECT a.name, b.something_random
FROM `table` b
INNER JOIN (SELECT DISTINCT name FROM `table` ORDER BY RAND() LIMIT 0,50) a
ON a.name = b.name ORDER BY a.name
How this work is the (SELECT DISTINCT name FROMtableORDER BY RAND() LIMIT 0,50) part is what pulls out the names to include in the join. So here I am taking 50 unique names at random, but you can change this to any other selection criteria if you want.
Then you join those results back into your table. This links each of those 50 selected names back to all of the rows with a matching name for your final results. Finally ORDER BY a.name just to be sure all the rows for each name end up grouped together.
This should do it:
SELECT tA.*
FROM
testTable tA
INNER JOIN
(SELECT distinct name FROM testTable LIMIT 50) tB ON tA.name = tB.name
;

How to calculated multiple moving average in MySQL

Using table below, How would get a column for 5 period moving average, 10 period moving average, 5 period exponential moving average.
+--------+------------+
| price | data_date |
+--------+------------+
| 122.29 | 2009-10-08 |
| 122.78 | 2009-10-07 |
| 121.35 | 2009-10-06 |
| 119.75 | 2009-10-05 |
| 119.02 | 2009-10-02 |
| 117.90 | 2009-10-01 |
| 119.61 | 2009-09-30 |
| 118.81 | 2009-09-29 |
| 119.33 | 2009-09-28 |
| 121.08 | 2009-09-25 |
+--------+------------+
The 5-row moving average in your example won't work. The LIMIT operator applies to the return set, not the rows being considered for the aggregates, so changing it makes no difference to the aggregate values.
SELECT AVG(a.price) FROM (SELECT price FROM t1 WHERE data_date <= ? ORDER BY data_date DESC LIMIT 5) AS a;
Replace ? with the date whose MA you need.
SELECT t1.data_date,
( SELECT SUM(t2.price) / COUNT(t2.price) as MA5 FROM mytable AS t2 WHERE DATEDIFF(t1.data_date, t2.data_date) BETWEEN 0 AND 6 )
FROM mytable AS t1 ORDER BY t1.data_date;
Change 6 to 13 for 10-day MA