MySql sum of custom column - mysql

I have a query in which I create custom column names.
At the end of that same table I want a row that will sum all the entries of these custom columns.
For example I have:
SELECT ... IFNULL(bl.amount, '---') AS BudgetAmount,
IFNULL((bl.amount *1) - ( + bal1.amount ), '---') AS Difference
FROM...
I want a column with the sum of BudgetAmount and DIfference columns.
How do I do that?

I think you can use this:
SELECT ... IFNULL(bl.amount, '---') AS BudgetAmount,
IFNULL((bl.amount *1) - ( + bal1.amount ), '---') AS Difference,
IFNULL(bl.amount, 0) + IFNULL((bl.amount *1) - ( + bal1.amount ), 0) AS NewCol
FROM...
Or you could explain more

One easy way to do that is to use your original query as an inline view and write a SELECT statement using the inline view as a rowsource. (In MySQL parlance, the inline view is called a derived table):
SELECT v.BudgetAmount
, v.Difference
, v.BudgetAmount + v.Difference AS Total
FROM (
-- original query here
SELECT ... IFNULL(bl.amount, '---') AS BudgetAmount,
IFNULL((bl.amount *1) - ( + bal1.amount ), '---') AS Difference
FROM...
) v
Unlike other relational database systems, MySQL will actually force the inline view to be materialized (i.e. the query in the inline view is executed and the results are stored as a MyISAM table), and that has performance implications.
But it is a way that you can reference the column aliases for use in other expressions.
(The aliases assigned to the columns can't be referenced in SELECT list where they are assigned... they can only be referenced in the HAVING and ORDER BY clauses of the query.)
To get the values for those columns added into another column in a single query, you can't reference the aliases, you have to to repeat the expressions, like this:
SELECT expr1 AS BudgetAmount
, expr2 AS Difference
, expr1 + expr2 AS Total
FROM ...
To reference the aliases, they need to come from a row source referenced by the query, such as an inline view:
SELECT v.BudgetAmount
, v.Difference
, v.foo
, v.BudgetAmount + v.Difference AS Total
FROM ( SELECT expr1 AS BudgetAmount
, expr2 AS Difference
, foo
FROM ...
) v
I'm not sure that's the answer you wanted to hear, but that's the way it is.
EDIT:
I misunderstood what you were asking. Your question said: "I want a column with the sum of ...". I took that to mean you wanted a query that returned the same number of rows, not that you wanted an additional ROW appended to the result set.

This sounds like a good candidate for app-side summarizing. However, if you really want to do it in MySQL only, there are essentially three options I think.
Write a stored procedure.
Do it in a UNION as others have suggested, which requires putting basically the same subquery twice in one statement.
Try the GROUP BY ... WITH ROLLUP syntax. I'm assuming you have a unique key included in the SELECT, e.g., someUniqueID. You need to group on that ID (so there will be only one row in each group) then use the rollup clause to append the additional row. The ID column will be NULL in the rollup row.
The statement will look something like this, depending on your full query and table structure:
SELECT someUniqueID, ... , IFNULL(SUM(bl.amount), '---') AS BudgetAmount,
IFNULL( SUM( (bl.amount *1) - ( + bal1.amount ) ), '---') AS Difference
FROM ...
GROUP BY someUniqueID
WITH ROLLUP

Sounds like you should do this on the application side. But you can just use UNION ALL at the end to stack the "sum of all entries." Just make sure your columns are aligned.
SELECT amount, balance
FROM table
UNION ALL
SELECT IFNULL(SUM(amount), '---') AS BudgetAmount,
IFNULL(SUM(...)) AS Difference
FROM table

Related

MYSQL error causing calculated result to be written to next row

Using MYSQL 5.7
This query creates a new table with the columns order_month and SKU plus the calculated columns qty_mth, count_mth and avg_month. The resulting table correctly reflects the first 4 columns but the final column (avg_month), while being correctly calculated is written on the next row.
CREATE TABLE tbl_temp_si_trans4
(temp_si_trans4id INT NOT NULL AUTO_INCREMENT,
PRIMARY KEY temp_trans4idkey (temp_si_trans4id),
INDEX index1 (order_month,SKU))
SELECT order_month, SKU,
#qty_mth := SUM(net_qty_after_refund) AS qty_mth,
#count_mth := COUNT(DISTINCT(order_year)) AS count_mth,
#avg_month := #qty_mth/#count_mth AS avg_month
FROM order_trans4
GROUP BY order_month, SKU
Please see below example of result:
I have tried to following modifications to the avg_month calculation line with the same result.
(#qty_mth/#count_mth) AS avg_month and
#qty_mth/#count_mth AS avg_month
From the documentation:
As a general rule, you should never assign a value to a user variable
and read the value within the same statement. You might get the
results you expect, but this is not guaranteed.
You can use subqueries without using user defined variables to achieve the effect you are looking for however. Something similar to the following
SELECT x.total_sale,
x.f1 / x.total_sale AS f1_percent
FROM (
SELECT s.f1,
s.f1 + s.f2 AS total_sale,
FROM sales s
) x

sql COALESCE result (12300.4567) to 12,300

How to select COALESCE result to format( , 0)
my query is
SELECT (COALESCE((SELECT SUM(`invoices`.`paid_amount`) FROM `invoices`
WHERE DATE(`invoices`.`date`)=CURDATE()),0) +
COALESCE((SELECT SUM(`other_incomes`.`other_income_amount`) FROM `other_incomes`
WHERE DATE(`other_incomes`.`date`)=CURDATE()),0))
AS total
FROM
....
Primarily, COALESCE doesn't change the formatting. It only returns the first non-null value passed to it.
Also, instead of trying to join or do two different queries and adding, and handling all the sums and coalesces separately (not to mention the rounding), I would probably UNION all the relevant results together, then handle the coalesce/sum/round all at the end.
Try this:
SELECT round(sum(coalesce(amt, 0)), 0) as total
FROM (
SELECT paid_amount as amt
FROM invoices i
WHERE date(i.date) = CURDATE()
union all
SELECT other_income_amount
FROM other_incomes o
WHERE date(o.date) = CURDATE()
) z
Here I COALESCE first, to make nulls be 0 instead. I wrap that in a SUM to add up the values, and finally a ROUND to get the format. It was unclear from the question is you wanted to ROUND or FLOOR. If you are looking to get it with that comma, use FORMAT. Here's the mySQL documentation for that. You didn't specify your SQL flavor.
https://dev.mysql.com/doc/refman/8.0/en/string-functions.html#function_format
Additionally, you should include your sql platform and version, the create statements for your tables, along with some insert statements that will provide sample data, along with the results you are looking for. It will help people answer your question. If you can include a fiddle, like https://dbfiddle.uk/, that would be nice.

How to use FIND_IN_SET using list of data

I have used FIND_IN_SET multiple times before but this case is a bit different.
Earlier I was searching a single value in the table like
SELECT * FROM tbl_name where find_in_set('1212121212', sku)
But now I have the list of SKUs which I want to search in the table. E.g
'3698520147','088586004490','868332000057','081308003405','088394000028','089541300893','0732511000148','009191711092','752830528161'
I have two columns in the table SKU LIKE 081308003405 and SKU Variation
In SKU column I am saving single value but in variation column I am saving the value in the comma-separated format LIKE 081308003405,088394000028,089541300893
SELECT * FROM tbl_name
WHERE 1
AND upc IN ('3698520147','088586004490','868332000057','081308003405','088394000028',
'089541300893','0732511000148','009191711092','752830528161')
I am using IN function to search UPC value now I want to search variation as well in the variation column. This is my concern is how to search using SKU list in variation column
For now, I have to check in the loop for UPC variation which is taking too much time. Below is the query
SELECT id FROM products
WHERE 1 AND upcVariation AND FIND_IN_SET('88076164444',upc_variation) > 0
First of all consider to store the data in a normalized way. Here is a good read: Is storing a delimited list in a database column really that bad?
Now - Assumng the following schema and data:
create table products (
id int auto_increment,
upc varchar(50),
upc_variation text,
primary key (id),
index (upc)
);
insert into products (upc, upc_variation) values
('01234', '01234,12345,23456'),
('56789', '45678,34567'),
('056789', '045678,034567');
We want to find products with variations '12345' and '34567'. The expected result is the 1st and the 2nd rows.
Normalized schema - many-to-many relation
Instead of storing the values in a comma separated list, create a new table, which maps product IDs with variations:
create table products_upc_variations (
product_id int,
upc_variation varchar(50),
primary key (product_id, upc_variation),
index (upc_variation, product_id)
);
insert into products_upc_variations (product_id, upc_variation) values
(1, '01234'),
(1, '12345'),
(1, '23456'),
(2, '45678'),
(2, '34567'),
(3, '045678'),
(3, '034567');
The select query would be:
select distinct p.*
from products p
join products_upc_variations v on v.product_id = p.id
where v.upc_variation in ('12345', '34567');
As you see - With a normalized schema the problem can be solved with a quite basic query. And we can effectively use indices.
"Exploiting" a FULLTEXT INDEX
With a FULLTEXT INDEX on (upc_variation) you can use:
select p.*
from products p
where match (upc_variation) against ('12345 34567');
This looks quite "pretty" and is probably efficient. But though it works for this example, I wouldn't feel comfortable with this solution, because I can't say exactly, when it doesn't work.
Using JSON_OVERLAPS()
Since MySQL 8.0.17 you can use JSON_OVERLAPS(). You should either store the values as a JSON array, or convert the list to JSON "on the fly":
select p.*
from products p
where json_overlaps(
'["12345","34567"]',
concat('["', replace(upc_variation, ',', '","'), '"]')
);
No index can be used for this. But neither can for FIND_IN_SET().
Using JSON_TABLE()
Since MySQL 8.0.4 you can use JSON_TABLE() to generate a normalized representation of the data "on the fly". Here again you would either store the data in a JSON array, or convert the list to JSON in the query:
select distinct p.*
from products p
join json_table(
concat('["', replace(p.upc_variation, ',', '","'), '"]'),
'$[*]' columns (upcv text path '$')
) v
where v.upcv in ('12345', '34567');
No index can be used here. And this is probably the slowest solution of all presented in this answer.
RLIKE / REGEXP
You can also use a regular expression:
select p.*
from products p
where p.upc_variation rlike '(^|,)(12345|34567)(,|$)'
See demo of all queries on dbfiddle.uk
You can try with below example:
SELECT * FROM TABLENAME
WHERE 1 AND ( FIND_IN_SET('3698520147', SKU)
OR UPC IN ('3698520147') )
I have a solution for you, you can consider this solution:
1: Create a temporary table example here: Sql Fiddle
select
tablename.id,
SUBSTRING_INDEX(SUBSTRING_INDEX(tablename.name, ',', numbers.n), ',', -1) sku_variation
from
numbers inner join tablename
on CHAR_LENGTH(tablename.sku_split)
-CHAR_LENGTH(REPLACE(tablename.sku_split, ',', ''))>=numbers.n-1
order by id, n
2: Use the temporary table to filter. find in set with your data
Performance considerations. The main thing that matters for performance is whether some index can be used. The complexity of the expression has only a minuscule impact on overall performance.
Step 1 is to learn what can be optimized, and in what way:
Equal: WHERE x = 1 -- can use index
IN/1: WHERE x IN (1) -- Turned into the Equal case by Optimizer
IN/many: WHERE x IN (22,33,44) -- Usually worse than Equal and better than "range"
Easy OR: WHERE (x = 22 OR x = 33) -- Turned into IN if possible
General OR: WHERE (sku = 22 OR upc = 33) -- not sargable (cf UNION)
Easy LIKE: WHERE x LIKE 'abc' -- turned into Equal
Range LIKE: WHERE x LIKE 'abc%' -- equivalent to "range" test
Wild LIKE: WHERE x LIKE '%abc%' -- not sargable
REGEXP: WHERE x RLIKE 'aaa|bbb|ccc' -- not sargable
FIND_IN_SET: WHERE FIND_IN_SET(x, '22,33,44') -- not sargable, even for single item
JSON: -- not sargable
FULLTEXT: WHERE MATCH(x) AGAINST('aaa bbb ccc') -- fast, but not equivalent
NOT: WHERE NOT ((any of the above)) -- usually poor performance
"Sargable" -- able to use index. Phrased differently "Hiding the column in a function call" prevents using an index.
FULLTEXT: There are many restrictions: "word-oriented", min word size, stopwords, etc. But it is very fast when it applies. Note: When used with outer tests, MATCH comes first (if possible), then further filtering will be done without the benefit of indexes, but on a smaller set of rows.
Even when an expression "can" use an index, it "may not". Whether a WHERE clause makes good use of an index is a much longer discussion than can be put here.
Step 2 Learn how to build composite indexes when you have multiple tests (WHERE ... AND ...):
When constructing a composite (multi-column) index, include columns in this order:
'Equal' -- any number of such columns.
'IN/many' column(s)
One range test (BETWEEN, <, etc)
(A couple of side notes.) The Optimizer is smart enough to clean up WHERE 1 AND .... But there are not many things that the Optimizer will handle. In particular, this is not sargable: `AND DATE(x) = '2020-02-20', but this does optimize as a "range":
AND x >= '2020-02-20'
AND x < '2020-02-20' + INTERVAL 1 DAY
Reading
Building indexes: http://mysql.rjweb.org/doc.php/index_cookbook_mysql
Sargable: https://en.wikipedia.org/wiki/Sargable
Tips on Many-to-many: http://mysql.rjweb.org/doc.php/index_cookbook_mysql#many_to_many_mapping_table
This depends on how you use it. In MySQL I found that find_in_set is way faster than using JSON when tested on the following commands, so much faster it wasn't even a competition (to be clear, the speed test did not include the set command line):
Fastest
set #ids = (select group_concat(`ID`) from `table`);
select count(*) from `table` where find_in_set(`ID`, #ids);
10 x slower
set #ids = (select json_arrayagg(`ID`) from `table`);
select count(*) from `table` where `ID` member of( #ids );
34 x slower
set #ids = (select json_arrayagg(`ID`) from `table`);
select count(*) from `table` where JSON_CONTAINS(#ids, convert(`ID`, char));
34 x slower
set #ids = (select json_arrayagg(`ID`) from `table`);
select count(*) from `table` where json_overlaps(#ids, json_array(`ID`));
SELECT * FROM tbl_name t1,(select
group_concat('3698520147',',','088586004490',',','868332000057',',',
'081308003405',',','088394000028',',','089541300893',',','0732511000148',',','009191711092',
',','752830528161') as skuid)t
WHERE FIND_IN_SET(t1.sku,t.skuid)>0

How can I use column relative position in if statement in order by?

I run a SQL query like below in MySQL:
select *
from (
select 2 as o,1 as t from dual
union
select 1 as o,2 as t from dual
) x
order by if((select 1),o,t);
It works well, but when I use column relative position in if statement, it doesn't work.
How can I use column relative position in if in ORDER BY statement?
select *
from (
select 2 as o,1 as t from dual
union
select 1 as o,2 as t from dual
) x
order by if((select 0),1,2);
I'm not sure what your real confusion is. When an integer appears in an order by, then this is treated as a column number. Any other use of an integer is interpreted as an expression.
The use of column numbers has been removed from the SQL standard. Hence, its use in any particular database is not guaranteed in future releases. It is really better to use the column names.
I think you want to sort your query based on a criteria over two columns, if I'm correct, you can use something like this:
...
order by
case when (your criteria)
then column1
else column2
end;
Note: use union all instead union when you don't want to remove duplicate values as performance issue ;).

SQL: Order by a list of foreign_key numbers

A simplified example:
I have a SQL table called things. Things by themselves have an id and a name. Things are part of a tree, e.g. a thing can have a parent; Exactly how to this is stored is not important, important is however that it is possible to obtain a list of thing ids from the root node to the current thing.
I have another table, called properties. A property has a thing_id column, a name column and a value column.
I now want, for the current thing, to obtain all properties, ordered by thing_id, in order of the paths from root thing to current thing.
e.g., if the current thing is nested like this: Root(1) > Vehicle(4) > Car(2) > Hybrid(3), I would want the list of properties be returned with the properties that have a thing_id==1 first, followed by the ones with thing_id == 4, then thing_id==2 and finally thing_id==3.
How can this be done using SQL? (without using N+1 selects)
In SQL this can be achieved with use of recursive query. Here is an example
DECLARE #item as varchar(10)
with CTE (main_part, sub_part, NestingLevel)
as
(
select main_part, sub_part, 1 from tblParts
where main_part = #item
union all
select tblParts.main_part, tblParts.sub_part, (NestingLevel + 1) from tblParts
inner join CTE on tblParts.main_part = CTE.sub_part
)
select * from CTE
In order to address this in MySQL you can try temporary table approach. Here is a good example of it: How to do the Recursive SELECT query in MySQL?