MySQL Sum() multiple columns - mysql

I have a table of student scorecard.
here is the table,
subject | mark1 | mark2 | mark3 |......|markn
stud1 | 99 | 87 | 92 | | 46
stud2 |....................................
.
.
studn |....................................|
Now, I need to sum it for each student of total marks. I got it by using sum(mark1+mark2+...+markn) group by stud. I want to know how to sum it without adding each column name,it will be huge when in case up to marks26. so could anyone know how to fix it.

SELECT student, (SUM(mark1)+SUM(mark2)+SUM(mark3)....+SUM(markn)) AS Total
FROM your_table
GROUP BY student

Another way of doing this is by generating the select query. Play with this fiddle.
SELECT CONCAT('SELECT ', group_concat(`COLUMN_NAME` SEPARATOR '+'), ' FROM scorecard')
FROM `INFORMATION_SCHEMA`.`COLUMNS`
WHERE `TABLE_SCHEMA` = (select database())
AND `TABLE_NAME` = 'scorecard'
AND `COLUMN_NAME` LIKE 'mark%';
The query above will generate another query that will do the selecting for you.
Run the above query.
Get the result and run that resulting query.
Sample result:
SELECT mark1+mark2+mark3 FROM scorecard
You won't have to manually add all the columns anymore.

If any of your markn columns are "AllowNull" then you will need to do a little extra to insure the correct result is returned, this is because 1 NULL value will result in a NULL total.
This is what i would consider to be the correct answer.
SUM(IFNULL(`mark1`, 0) + IFNULL(`mark2`, 0) + IFNULL(`mark3`, 0)) AS `total_marks`
IFNULL will return the 2nd parameter if the 1st is NULL.
COALESCE could be used but i prefer to only use it if it is required.
See What is the difference bewteen ifnull and coalesce in mysql?
SUM-ing the entire calculation is tidier than SUM-ing each column individually.
SELECT `student`, SUM(IFNULL(`mark1`, 0) + IFNULL(`mark2`, 0) + IFNULL(`mark3`, 0)) AS `total_marks`
FROM student_scorecard
GROUP BY `student`
i want to know how to sum it without adding each column name,it will be huge when in case up to marks26
To generate and execute this statement dynamically in sql you would need to use the INFORMATION_SCHEMA.COLUMNS table to create a query string then execute it using a prepared statement saved in a variable.
SELECT CONCAT('SELECT `student`, SUM(IFNULL(`', group_concat(`COLUMN_NAME` SEPARATOR '`, 0) + IFNULL(`'), '`, 0) AS `total_marks` FROM `student_scorecard` GROUP BY `student`')
FROM `INFORMATION_SCHEMA`.`COLUMNS`
WHERE `TABLE_SCHEMA` = (select DATABASE())
AND `TABLE_NAME` = 'student_scorecard'
AND `COLUMN_NAME` LIKE 'mark%'
# adapted from https://stackoverflow.com/a/22369767/2273611
# insert statement sql into a variable
INTO #statement_var;
#prepare the statement string
PREPARE stmt_name FROM #statement_var;
#execute the prepared statement/query
EXECUTE stmt_name;
# release statement
DEALLOCATE PREPARE stmt_name;

SELECT student, SUM(mark1+mark2+mark3+....+markn) AS Total FROM your_table

The short answer is there's no great way to do this given the design you have. Here's a related question on the topic: Sum values of a single row?
If you normalized your schema and created a separate table called "Marks" which had a subject_id and a mark column this would allow you to take advantage of the SUM function as intended by a relational model.
Then your query would be
SELECT subject, SUM(mark) total
FROM Subjects s
INNER JOIN Marks m ON m.subject_id = s.id
GROUP BY s.id

//Mysql sum of multiple rows
Hi Here is the simple way to do sum of columns
SELECT sum(IF(day_1 = 1,1,0)+IF(day_3 = 1,1,0)++IF(day_4 = 1,1,0)) from attendence WHERE class_period_id='1' and student_id='1'

You could change the database structure such that all subject rows become a column variable (like spreadsheet). This makes such analysis much easier

Related

Add another column on Dynamic Pivot Table and count the null/0 values

i have a working dynamic pivot code. i'm stuck on this almost 1 week since i was trying to find a way to add another column that counts the 0's or null.
SET #sql_dynamic:= (SELECT GROUP_CONCAT
(DISTINCT
CONCAT('if(sum(if(attendance_date = "',
date_format(attendance_date, '%Y-%m-%d'),
'",1,0))=0,0,attendance_status) AS `',
date_format(attendance_date, '%Y-%m-%d'),'`'
)
) from attendance
WHERE subject_id=1 AND attendance_month = "January"
);
SET #sql = CONCAT('SELECT studentidnumber, student_fullname,
subject_id, attendance_month, ', #sql_dynamic,'
FROM attendance
WHERE subject_id=1 AND attendance_month = "January"
GROUP BY studentidnumber'
);
PREPARE stmt FROM #sql;
EXECUTE stmt;`
which results to this:
pivot
now i want to add another column to the dynamic table which counts the 0 or null values.
pls help.
You need a separate table with all the dates in it. (Or at least enough dates to handle your date range.) MariaDB (not MySQL) has a neat feature of sequence tables. For example, seq_1_to_100 is a virtual table that contains all the integers from 1 to 100. Use something like that, together with + INTERVAL seq to generate all the dates. Then LEFT JOIN with our table to get the "missing" dates to generate NULL values. Change those to 0 with IFNULL(), if desired.

How to select all the 'not' part against the 'in' set in MySQL?

Here is the problem:
I've got a set A: (1,2,3,4,5), and a mysql table B which looks like:
| id |
1
2
3
6
7
What I want is to select all the elements in A which are not in the table B's id field. Thus, the result should be (4, 5).
I've tried select * from B where id not in (1,2,3,4,5), obviously, the result would be (6, 7).
I just can't get it through how to achieve what I intend to do. Could anyone give me some idea? Thanks a lot!
You can do this using prepared statement. This requires many queries but I think it's a clean solution(IMO).
I am using it in a stored procedure where I get list of comma saperated customer ids and retrieve customer information.
DROP TEMPORARY TABLE IF EXISTS tempSplitValues;
CREATE TEMPORARY TABLE tempSplitValues(tempText TEXT);
SET #insertStatement = CONCAT('INSERT INTO tempSplitValues VALUES', REPLACE(REPLACE(CONCAT('(', REPLACE("1,2,3,4,5", ',', '), ('), ')'), '(', '("'), ')', '")')); -- Build INSERT statement like this -> INSERT INTO customerIdsTable VALUES("cusId1"),("cusId2")
PREPARE stmt FROM #insertStatement; -- parse and prepare insert statement from the above string
EXECUTE stmt; -- execute statement
DEALLOCATE PREPARE stmt; -- release the statement memory
SELECT * FROM tempSplitValues tsv WHERE tsv.tempText NOT IN(SELECT id FROM B);
Use left join instead - it's much more effective.
SELECT A.* FROM A
LEFT JOIN B ON A.ID = B.ID
WHERE B.ID IS NULL
First - left join records from table B, then filter by B.ID IS NULL - which means table A rows, that doesn't have corresponding records in table B.
There's only two options I can see and neither is particularly elegant (I'd love to see better solutions though).
The first is to use unions to generate the list of numbers:
SELECT *
FROM (SELECT 1 AS id UNION SELECT 2 AS id UNION SELECT 3 AS id UNION SELECT 4 AS id UNION SELECT 5 AS id) AS a
WHERE a.id NOT IN (SELECT id FROM B);
This is pretty hideous however. The second option is to create a temporary table for A and use one of the other answers.

Creating Temp Variables within Queries

I would like to be able to create a temp variable within a query--not a stored proc nor function-- which will not need to be declared and set so that I don't need to pass the query parameters when I call it.
Trying to work toward this:
Select field1,
tempvariable=2+2,
newlycreatedfield=tempvariable*existingfield
From
table
Away from this:
DECLARE #tempvariable
SET #tempvariable = 2+2
Select field1,
newlycreatedfield=#tempvariable*existingfield
From
table
Thank you for your time
I may have overcomplicated the example; more simply, the following gives the Invalid Column Name QID
Select
QID = 1+1
THN = QID + 1
If this is housed in a query, is there a workaround?
You can avoid derived tables and subqueries if you do a "hidden" assignment as a part of a complex concat_ws expression
Since the assignment is part of the expression of the ultimate desired value for the column, as opposed to sitting in its own column, you don't have to worry about whether MySQL will evaluate it in the correct order. Needless to say, if you want to use the temp var in multiple columns, then all bets are off :-/
caveat: I did this in MySQL 5.1.73; things might have changed in later versions
I wrap everything in concat_ws because it coalesces null args to empty strings, whereas concat does not.
I wrap the assignment to the var #stamp in an if so that it is "consumed" instead of becoming an arg to be concatenated. As a side note, I have guaranteed elsewhere that u.status_timestamp is populated when the user record is first created. Then #stamp is used in two places in date_format, both as the date to be formatted and in the nested if to select which format to use. The final concat is an hour range "h-h" which I have guaranteed elsewhere to exist if the c record exists, otherwise its null return is coalesced by the outer concat_ws as mentioned above.
SELECT
concat_ws( '', if( #stamp := ifnull( cs.checkin_stamp, u.status_timestamp ), '', '' ),
date_format( #stamp, if( timestampdiff( day, #stamp, now() )<120, '%a %b %e', "%b %e %Y" )),
concat( ' ', time_format( cs.start, '%l' ), '-', time_format( cs.end, '%l' ))
) AS as_of
FROM dbi_user AS u LEFT JOIN
(SELECT c.u_id, c.checkin_stamp, s.start, s.end FROM dbi_claim AS c LEFT JOIN
dbi_shift AS s ON(c.shift_id=s.id) ORDER BY c.u_id, c.checkin_stamp DESC) AS cs
ON (cs.u_id=u.id) WHERE u.status='active' GROUP BY u.id ;
A final note: while I happen to be using a derived table in this example, it is only because of the requirement to get the latest claim record and its associated shift record for each user. You probably won't need a derived table if a complex join is not involved in the computation of your temp var. This can be demonstrated by going to the first fiddle in #Fabien TheSolution's answer and changing the right hand query to
Select field1, concat_ws( '', if(#tempvariable := 2+2,'','') ,
#tempvariable*existingfield ) as newlycreatedfield
from table1
Likewise the second fiddle (which appears to be broken) would have a right hand side of
SELECT concat_ws( '', if(#QID := 2+2,'',''), #QID + 1) AS THN
You can do this with subqueries:
Select field1, tempvariable,
(tempvariable*existingfield) as newlycreatedfield
from (select t.*, (2+2) as tempvariable
from table t
) t;
Unfortunately, MySQL has a tendency to actually instantiate (i.e. create) a derived table for the subquery. Most other databases are smart enough to avoid this.
You can gamble that the following will work:
Select field1, (#tempvariable := 2+2) as tempvariable,
(#tempvariable*existingfield) as newlycreatedfield
From table t;
This is a gamble, because MySQL does not guarantee that the second argument is evaluated before the third. It seems to work in practice, but it is not guaranteed.
Why not just:
SET #sum = 4 + 7;
SELECT #sum;
Output:
+------+
| #sum |
+------+
| 11 |
+------+
source
You can do something like this :
SELECT field1, tv.tempvariable,
(tv.tempvariable*existingfield) AS newlycreatedfield
FROM table1
INNER JOIN (SELECT 2+2 AS tempvariable) AS tv
See SQLFIDDLE : http://www.sqlfiddle.com/#!2/8b0724/8/0
And to refer at your simplified example :
SELECT var.QID,
(var.QID + 1) AS THN
FROM (SELECT 1+1 as QID) AS var
See SQLFIDDLE : http://www.sqlfiddle.com/#!2/d41d8/19140/0

How do I run a CONCAT on query stored in a database

I know this is a bad example but I'm trying to simplify things so bear with me in the round-about way this code is written. Say I have queries stored in a database such as
queries table
id query
1 concat('SELECT * FROM table1 WHERE year = ', _year, 'order by name')
2 concat('SELECT * FROM table2 WHERE year = ', _year, 'order by name')
and I want to run the following routine
DECLARE _year;
SET _year= "2013";
SET #SQL = (SELECT query FROM queries WHERE id = 1);
PREPARE stmt FROM #SQL;
EXECUTE stmt;
this is what I currently have but it's not working. I'm trying to select a query from the database, pass a few variables into it and then run the query.
If it is possible to determine your query in advance, except for specific parameters, then you could consider using a UNION query with a discriminant. Then, use your queries table to select specific queries in the union and apply parameters to them. The queries themselves are defined in a view in the database.
SQL is intended in most DBMS to not be dynamic and the attempt to subvert this will almost certainly result in reduced performance and potential security problems.
CREATE VIEW UnionView
AS SELECT *, 1 as Type, Value1 AS Param FROM Table1
UNION ALL SELECT *, 2 as Type, Value2 AS Param FROM Table1
UNION ALL SELECT *, 3 as Type, Value3 AS Param FROM Table1;
SELECT * FROM UnionView WHERE Type = 2 AND Param = 2;
See SqlFiddle for an example to demonstrate the behaviour.

How to count all NULL values in a table?

Just wondering, is there any quick way to count all the NULL values (from all columns) in a MySQL table?
Thanks for any idea!
If you want this done exclusively by MYSQL and without enumerating all of the columns take a look at this solution.
In this method you don't have to maintain the number of database columns by hard coding them. If your table schema will get modified this method will work, and won't require code change.
SET #db = 'testing'; -- database
SET #tb = 'fuzzysearch'; -- table
SET #x = ''; -- will hold the column names with ASCII method applied to retrieve the number of the first char
SET #numcolumns = 0; -- will hold the number of columns in the table
-- figure out how many columns we have
SELECT count(*) into #numcolumns FROM information_schema.columns where table_name=#tb and table_schema=#db;
-- we have to prepare some query from all columns of the table
SELECT group_concat(CONCAT('ASCII(',column_name,')') SEPARATOR ",") into #x from information_schema.columns where table_name=#tb and table_schema=#db;
-- after this query we have a variable separated with comma like
-- ASCII(col1),ASCII(col2),ASCII(col3)
-- we now generate a query to concat the columns using comma as separator (null values are omitted from concat)
-- then figgure out how many times the comma is in that substring (this is done by using length(value)-length(replace(value,',',''))
-- the number returned is how many non null columns we have in that column
-- then we deduct the number from the known number of columns, calculated previously
-- the +1 is added because there is no comma for single value
SET #s = CONCAT('SELECT #numcolumns - (length(CONCAT_WS(\',\',', #x, '))-length(replace(CONCAT_WS(\',\',', #x, '),\',\',\'\')) + 1) FROM ',#db,'.',#tb,';');
PREPARE stmt FROM #s;
EXECUTE stmt;
-- after this execution we have returned for each row the number of null columns
-- I will leave to you to add a sum() group call if you want to find the null values for the whole table
DEALLOCATE PREPARE stmt;
The ASCII is used to avoid reading, concatenating very long columns for nothing, also ASCII makes us safe for values where the first char is a comma(,).
Since you are working with reports, you may find this helpful as this can be reused for each table if you put in a method.
I tried to let as many comments as possible.
Let's split on pieces the above compact way (reverse way):
I wanted to end up having a query like this
SELECT totalcolumns - notnullcolumns from table; -- to return null columns for each row
While the first one is easy to calcule by running:
SELECT count(*) FROM information_schema.columns where table_name=#tb and table_schema=#db;
The second one the notnullcolumns is a bit of pain.
After a piece of examination of the functions available in MySQL, we detect that CONCAT_WS does not CONCAT null values
So running a query like this:
SELECT CONCAT_WS(",","First name",NULL,"Last Name");
returns: 'First name,Last Name'
This is good, we take rid of the null values from the enumeration.
But how do we get how many columns were actually concatenated?
Well that is tricky. We have to calculate the number of commas+1 to get the actually concatenated columns.
For this trick we used the following SQL notation
select length(value)-length(replace(value,',','')) +1 from table
Ok, so we have now the number of concatenated columns.
But the harder part is coming next.
We have to enumerate for CONCAT_WS() all values.
We need to have something like this:
SELECT CONCAT_WS(",",col1,col2,col3,col4,col5);
This is where we have to take use of the prepared statements, as we have to prepare an SQL query dynamically from yet unknown columns. We don't know how many columns will be in our table.
So for this we use data from information_schema columns table. We need to pass the table name, but also the database name, as we might have the same table name in separate databases.
We need a query that returns col1,col2,col3,col4,col5 to us on the CONCAT_WS "string"
So for this we run a query
SELECT group_concat(column_name SEPARATOR ",") into #x from information_schema.columns where table_name=#tb and table_schema=#db;
One more thing to mention. When we used the length() and replace() method to find out how many columns were concatenated, we have to make sure we do not have commas among the values. But also take note that we can have really long values in our database cells. For both of this trick we use method ASCII('value'), which will return the ASCII char of the first char, which cannot be comma and will return null for null columns.
That being said we can compact all this in the above comprehensive solution.
Something like
select id
, sum ( case when col1 is null then 1 else 0 end case ) col1
, sum ( case when col2 is null then 1 else 0 end case ) col2
, sum ( case when col3 is null then 1 else 0 end case ) col3
from contacts
group by id
Something like this (substitute COL_COUNT as appropriate):
select count(*) * COL_COUNT - count(col1) - count(col2) - ... - count(col_n) from table;
You should really do this using not only SQL, but the language which is at your disposal:
Obtain the metadata of each table - either using DESCRIBE table, or using a built-in metadata functionality in your db access technology
Create queries of the following type in a loop for each column. (in pseudo-code)
int nulls = 0;
for (String colmnName : columNames) {
query = "SELECT COUNT(*) FROM tableName WHERE " + columnName + " IS NULL";
Result result = executeQuery(query);
nulls += result.size();
}