Average length of table column - mysql

Question

I have a table with 50+ VARCHAR(255) columns.
The moderators report that some of the content is cut of after 250 characters in few of the fields.
As checked this is expected behavior for VARCHAR(255) and I have to update some of the fields to text. But the problem is they can not give me details/instruction which fields are making problems ..
So my best guess is to analyse the current data and find the columns that usually store long content.
Is there a good query structure I can use to get:
- AVG length for each column.
- Max length for each column.
- Count of rows with length 200+ for this column.

Answer 1

SELECT AVG(CHAR_LENGTH(col)) avg_length,
MAX(CHAR_LENGTH(col)) max_length,
COUNT(CASE WHEN CHAR_LENGTH(col) >= 200 THEN 1 ELSE NULL END) 200_plus_count
FROM tbl;

Answer 2

For average select AVG(length(column_name)) and for maximum select MAX(length(column_name)) for count 200+ select COUNT(column_name) from table WHERE len(rows)=>200. This site should help you with other sql related questions, hope I answered your question :)

Answer 3

I recently had reason to implement exactly this. Using similar logic as Arth's and VMai's answers, I built a stored procedure to get all column sizes for a table.
DELIMITER //
CREATE PROCEDURE ColumnSizeForTable(TableName varchar(64), SchemaName varchar(64))
BEGIN
SELECT ##group_concat_max_len INTO #group_concat_max;
SET SESSION group_concat_max_len = 100000;
SELECT
CONCAT('SELECT TRIM(TRAILING \' UNION ALL \' FROM CAST(CONCAT(',
GROUP_CONCAT(
CONCAT(
CONCAT('\'SELECT \\\'', COLUMN_NAME, '\\\' ColName,\''),
', ',
CONCAT('IFNULL(AVG(CHAR_LENGTH(',COLUMN_NAME,')),\'0\'), \' ColAverage,\''),
', ',
CONCAT('IFNULL(MAX(CHAR_LENGTH(',COLUMN_NAME,')),\'0\'), \' ColMaximum,\''),
', ',
CONCAT('IFNULL(COUNT(CASE WHEN CHAR_LENGTH(',COLUMN_NAME,') >= 200 THEN 1 ELSE NULL END),\'0\'), \' Col200Plus UNION ALL \'')
)
),
') AS CHAR)) INTO #unionquery FROM ',
TABLE_NAME,
';')
INTO #columnquery FROM
INFORMATION_SCHEMA.COLUMNS
WHERE
TABLE_NAME = TableName
AND TABLE_SCHEMA = SchemaName
GROUP BY TABLE_NAME;
PREPARE columnsizestmnt FROM #columnquery;
EXECUTE columnsizestmnt;
PREPARE unionstmnt FROM #unionquery;
EXECUTE unionstmnt;
SET SESSION group_concat_max_len = #group_concat_max;
END //
DELIMITER ;
CALL ColumnSizeForTable('TABLENAME','SCHEMANAME');
How this works:
Increase the session variable group_concat_max_len so we don't have to worry about GROUP_CONCAT being cut off.
Execute a concat query that builds another query.
This first query populates the column names.
The resulting query is put into #columnquery
Execute #columnquery. This builds the query to put the data into a readable format
This second query gets the column data, including average, max, and the 200+ count.
Similar to VMai's answer, if we weren't building another query from this, it would be a flattened result set.
The resulting query is put into #unionquery
Execute #unionquery. This SELECT is outputted to the user. It returns the column details that we were trying to collect in a single, readable format.

Answer 4

I'm building on Arths good answer and integrating my comment. This query should build the query to be executed:
SELECT
CONCAT(
-- The `SELECT` keyword
'SELECT ',
-- we build our list of analyzing columns with a GROUP_CONCAT
GROUP_CONCAT(
CONCAT (
-- of the columns from Arths answer
CONCAT ('AVG(CHAR_LENGTH(', COLUMN_NAME, ')) AVG_', COLUMN_NAME), ', ',
CONCAT ('MAX(CHAR_LENGTH(', COLUMN_NAME, ')) MAX_', COLUMN_NAME), ', ',
CONCAT ('COUNT((CASE WHEN CHAR_LENGTH(', COLUMN_NAME, ') >= 200 THEN 1 ELSE NULL END) 200_plus_', COLUMN_NAME)
), ' '),
-- and add the FROM clause
' FROM ',
TABLE_NAME,
';' )
FROM
INFORMATION_SCHEMA.COLUMNS
WHERE
-- replace it by your own
TABLE_NAME = 'example10'
GROUP BY
TABLE_NAME;
Note
It's not nicely formatted, but a search by , and replace by ,\n with an editor like notepad++ will make the statement readable. So you'll missing the annoying task of writing this by hand and not making mistakes.