MySQL: Summarize all table row-counts in a single query - mysql

When reviewing a database, it's highly useful to get an overview of all the tables, including their row counts:
TableName Count
t1 1234
t2 37
... ...
The MySQL TABLES table in the information_schema database provides a table_rows field:
SELECT table_name, table_rows
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_SCHEMA = '<your db>';
But table_rows is only valid for some database engines, whereas for INNODB it is either NULL or not accurate.
Hence it's necessary to compose a method that does an explicit SELECT Count(*)... for each table.
Across many repetitions of this question on stackoverflow, there are numerous answers that involve a two-step process. One query to create a result set with rows containing the individual select count(*) statements, followed by a textediting procedure to turn that into the actual statement that can produce the desired output.
I had not seen this turned into a single step, so below I post that answer. It's not rocket science, but it's convenient to have it spelled out.

The first example code here is a stored procedure which performs the entire process in one step, so far as the user is concerned.
BEGIN
# zgwp_tables_rowcounts
# TableName RowCount
# Outputs a result set listing all tables and their row counts
# for the current database
SET SESSION group_concat_max_len = 1000000;
SET #sql = NULL;
SET #dbname = DATABASE();
SELECT
GROUP_CONCAT(
CONCAT (
'SELECT ''',table_name,''' as TableName, COUNT(*) as RowCount FROM ',
table_name, ' '
)
SEPARATOR 'UNION '
) AS Qry
FROM
information_schema.`TABLES` AS t
WHERE
t.TABLE_SCHEMA = #dbname AND
t.TABLE_TYPE = "BASE TABLE"
ORDER BY
t.TABLE_NAME ASC
INTO #sql
;
PREPARE stmt FROM #sql;
EXECUTE stmt;
END
Notes:
The SELECT..INTO #sql creates the necessary query, and the PREPARE... EXECUTE runs it.
Sets the group_concat_max_len variable in order to allow a long enough result string from GROUP_CONCAT.
The above procedure is useful for a quick look in an admin environment like Navicat, or on the command line. However, despite returning a result set, so far as I am aware it can't be referenced in another View or Query, presumably because MySQL is unable to determine, before running it, what result sets it produces, let alone what columns they have.
So, it is still useful to be able to quickly produce, without manual editing, the separate SELECT...UNION statement that can be used as a View. That is useful if you want to join the row counts to some other per-table info from another table. Herewith another stored procedure:
BEGIN
# zgwp_tables_rowcounts_view_statement
# Output: SelectStatement
# Outputs a single row and column, containing a (possibly lengthy)
# SELECT...UNION statement that, if used as a View, will output
# TableName RowCount for all tables in the current database.
SET SESSION group_concat_max_len = 1000000;
SET #dbname = DATABASE();
SELECT
GROUP_CONCAT(
CONCAT (
'SELECT ''',table_name,''' as TableName, COUNT(*) as RowCount FROM ',
table_name, ' ', CHAR(10))
SEPARATOR 'UNION '
) AS SelectStatement
FROM
information_schema.`TABLES` AS t
WHERE
t.TABLE_SCHEMA = #dbname AND
t.TABLE_TYPE = "BASE TABLE"
ORDER BY
t.TABLE_NAME ASC
;
END
Notes
Very similar to the first procedure in concept. I added a linebreak (CHAR(10)) to each subsidiary "SELECT...UNION" statement, for convenience in viewing or editing the statement.
You could create this as a function and return the SelectStatement, if that's more convenient for your environment.
Hope that helps.

Related

Please explain these SQL statements

Please explain me the below example
SELECT GROUP_CONCAT(COLUMN_NAME)
FROM information_schema.`COLUMNS` C
WHERE table_name = 'table_name'
AND COLUMN_NAME =('columns_name') INTO #COLUMNS;
SET #table = 'table_name';
SET #s = CONCAT('SELECT ',#columns,' FROM ', #table);
PREPARE stmt FROM #s;
This pattern is all about creating dynamic (prepared in MySQL parlance) queries based on the names of columns in a particular table. INFORMATION_SCHEMA is a built-in database with read-only tables describing all the tables in all databases on the MySQL server.
The first query in your sequence retrieves a text string in the local variable #COLUMNS with a value like
id,name,value,description
for a table named table_name with those four columns.
The third one retrieves a string in the local variable #s with a value containing a query like
SELECT id,name,value,description FROM table_name
The fourth one, PREPARE, gets ready to do EXECUTE stmt, which runs the query. You can read about PREPARE and EXECUTE here.
The whole sequence of queries in your question does almost exactly the same thing as SELECT * FROM table_name.
There's a defect in your first query. You should add AND TABLE_SCHEMA = DATABASE() to its WHERE clause. Otherwise, you may pick up columns from tables named table_name in multiple databases.

How to delete all MySQL tables beginning with a certain prefix?

I've found another thread on this question, but I wasn't able to use its solutions, so I thought I'd ask with more clarity and detail.
I have a large MySQL database representing a vBulletin forum. For several years, this forum has had an error generated on each view, each time creating a new table named aagregate_temp_1251634200, aagregate_temp_1251734400, etc etc. There are about 20,000 of these tables in the database, and I wish to delete them all.
I want to issue a command that says the equivalent of DROP TABLE WHERE TABLE_NAME LIKE 'aggregate_temp%';.
Unfortunately this command doesn't work, and the Google results for this problem are full of elaborate stored procedures beyond my understanding and all seemingly tailored to the more complex problems of different posters.
Is it possible to write a simple statement that drops multiple tables based on a name like match?
There's no single statement to do that.
The simplest approach is to generate a set of statements, and execute them individually.
We can write a simple query that will generate the statements for us:
SELECT CONCAT('DROP TABLE `',t.table_schema,'`.`',t.table_name,'`;') AS stmt
FROM information_schema.tables t
WHERE t.table_schema = 'mydatabase'
AND t.table_name LIKE 'aggregate\_temp%' ESCAPE '\\'
ORDER BY t.table_name
The SELECT statement returns a rowset, but each row conveniently contains the exact SQL statement we need to execute to drop a table. (Note that information_schema is a builtin database that contains metadata. We'd need to replace mydatabase with the name of the database we want to drop tables from.
We can save the resultset from this query as a plain text file, remove any heading line, and voila, we've got a script we can execute in our SQL client.
There's no need for an elaborate stored procedure.
A little googling found this:
SELECT 'DROP TABLE "' + TABLE_NAME + '"'
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_NAME LIKE 'prefix%'
This should generate a script.
Source: Drop all tables whose names begin with a certain string
From memory you have to use prepared statements, for example: plenty of samples on stack exchange
I would recommend this example:
SQL: deleting tables with prefix
The SQL from above, this one includes the specific databasename - it builds it for you
SELECT CONCAT( 'DROP TABLE ', GROUP_CONCAT(table_name) , ';' )
AS statement FROM information_schema.tables
WHERE table_schema = 'database_name' AND table_name LIKE 'myprefix_%';
Here is a different way to do it:
MySQL bulk drop table where table like?
This will delete all tables with prefix "mg_"
No need to copy and paste rowsets and in phpadmin copying and pasting is problematic as it will cut off long table names and replace them with '...' ruining set of sql commands.
Also note that '_' is a special character so thats why 'mg_' should be encoded as 'mg\_'
(and FOREIGN_KEY_CHECKS needs to be disabled in order to avoid error messages)
SET FOREIGN_KEY_CHECKS = 0;
SET GROUP_CONCAT_MAX_LEN=32768;
SET #tables = NULL;
SELECT GROUP_CONCAT('`', table_name, '`') INTO #tables
FROM information_schema.tables
WHERE table_schema = (SELECT DATABASE()) and table_name like 'mg\_%';
SELECT IFNULL(#tables,'dummy') INTO #tables;
SET #tables = CONCAT('DROP TABLE IF EXISTS ', #tables);
PREPARE stmt FROM #tables;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
SET FOREIGN_KEY_CHECKS = 1;

How would I delete a single column in all tables in MYSQL

I have a column 'seq' in every table of my database that I would like to delete easily.
I have to do this on occasion in MySQL and am hoping this can be automated.
There isn't a simple magical expression to just do this. You need to generate a list of SQL statements and then run them, somehow.
(Most database folks don't routinely drop columns from a database in production; it takes a lot of time during which the tables are inaccessible, and it's destructive. A fat-finger error could really mess you up.)
You might start by using the information_schema in MySQL to discover which of your tables have a seq column in them. This query will return that list of tables for the database you're currently using.
SELECT DISTINCT TABLE_NAME
FROM information_schema.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND COLUMN_NAME = 'seq'
You could then adapt that query to, for example, create a list of statements like this.
SELECT DISTINCT
CONCAT('UPDATE ',TABLE_NAME, ' SET seq = 0;') AS stmt
FROM information_schema.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND COLUMN_NAME = 'seq'
This will produce a result set like this:
UPDATE table_a SET seq = 0;
UPDATE table_b SET seq = 0;
UPDATE user SET seq = 0;
Then you could run these statements one by one. These statements will zero out your seq columns.
Edit
You can also do
CONCAT('ALTER TABLE ',TABLE_NAME, ' DROP COLUMN seq;') AS stmt
to get a drop column statement for each table.
But, you might consider creating views of your tables that don't contain the seq columns, and then exporting to PostgreSQL using those views. If your tables are significant in size, this will save you a lot of time.

SQL: convert all columns to VarChar(255)

After some searching here on stackoverflow and on the web, I couldn't find the answer to my question. I'm not a real SQL talent, but I'm trying to covert all the columns in my table to varchar (255). It has about 600 columns which are all varchar, but the size limit varies. I would like them all to be 255. Is there a way to not having to do this manually? I work with MySQL.
Thanks!
You need to generate the alter table statement by pulling the data from the database.
select 'alter table MyTableName modify column ' + column_name + ' varchar(255);'
from information_schema where table_name = 'MyTableName'
And then paste the results of this command into the query window and run it -- making sure it does what you want it to do. Do a backup first.
Or you could make one big alter statement (if MySql wouldn't choke on it) by replacing the semicolon with a comma.
This isn't what you really need to do. You have something more important to do: NORMALIZE YOUR DATABASE
Now, It's impossible that you have a normalized table with 600 columns. Split your entities in that table correctly, following at least the 3rd normal form rules. After that, you'll have a much better database which is easier to mantain.
To do this, you'll need to drop your current table, therefore, you don't need to change all the types to varchar(255) because you'll fix them during the creation of other tables.
This would be a good start to read: http://en.wikipedia.org/wiki/Database_normalization (thanks to #Tim Schmelter from question's comments)
First of all as mentioned by others you better off normalize you data.
In the meantime you can achieve your goal with dynamic SQL like this
DELIMITER $$
CREATE PROCEDURE change_to_varchar255(IN _tname VARCHAR(64))
BEGIN
SET #sql = NULL;
SELECT GROUP_CONCAT(
CONCAT_WS(' ', 'CHANGE', COLUMN_NAME, COLUMN_NAME, 'VARCHAR(255)'))
INTO #sql
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = _tname
AND DATA_TYPE = 'varchar'
AND CHARACTER_MAXIMUM_LENGTH < 255
AND TABLE_SCHEMA = SCHEMA();
SET #sql = CONCAT_WS(' ', 'ALTER TABLE', _tname, #sql);
PREPARE stmt FROM #sql;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
END$$
DELIMITER ;
Sample usage:
CALL change_to_varchar255('table1');
Here is SQLFiddle demo
If you are using PhpMyAdmin or other, you can also click on the button to modify the table.
When you are on the web page, press Ctrl+Shift+J under Windows or Cmd+Opt+J under Mac to open the console window in the Chrome Developer tools. Now enter the following command to replace all occurrences of the number 255 with 100 :
document.body.innerHTML = document.body.innerHTML.replace(/255/g, "100").
Finally, click on the button to execute the query.

How can I "select *" from a table in MySQL but omit certain columns?

I have a table with the following columns:
id,name,age,surname,lastname,catgory,active
Instead of: SELECT name,age,surname,lastname,catgory FROM table
How can I make something like this: SELECT * FROM table [but not select id,active]
While many say it is best practice to explicitly list every column you want returned, there are situations where you might want to save time and omit certain columns from the results (e.g. testing). Below I have given two options that solve this problem.
1. Create a Function that retrieves all of the desired column names: ( I created a schema called functions to hold this function)
DELIMITER $$
CREATE DEFINER=`root`#`%` FUNCTION `getTableColumns`(_schemaName varchar(100), _tableName varchar(100), _omitColumns varchar(200)) RETURNS varchar(5000) CHARSET latin1
BEGIN
SELECT GROUP_CONCAT(COLUMN_NAME) FROM information_schema.columns
WHERE table_schema = _schemaName AND table_name = _tableName AND FIND_IN_SET(COLUMN_NAME,_omitColumns) = 0 ORDER BY ORDINAL_POSITION;
END
Create and execute select statement:
SET #sql = concat('SELECT ', (SELECT
functions.getTableColumns('test', 'employees', 'age,dateOfHire')), ' FROM test.employees');
PREPARE stmt1 FROM #sql;
EXECUTE stmt1;
2. OR without writing a function you could:
SET #sql = CONCAT('SELECT ', (SELECT GROUP_CONCAT(COLUMN_NAME) FROM
information_schema.columns WHERE table_schema = 'test' AND table_name =
'employees' AND column_name NOT IN ('age', 'dateOfHire')),
' from test.eployees');
PREPARE stmt1 FROM #sql;
EXECUTE stmt1;
*Replace test with your own schema name
**Replace employees with your own table name
***Replace age,dateOfHire with the columns you want to omit (you can leave it blank to return all columns or just enter one column name to omit)
** **You can adjust the lengths of the varchars in the function to meet your needs
The only way to do that that I know if is to enumerate each column you do want... no negative filters that I'm aware of.
select name, age, surname, lastname, category from table
you can't do that, sorry. Actually you shouln't have done it if you could - specifying these things explicitly is always better, assume other developer adds new field and your application will fail
You are too advanced.
The only data language that I have seen that supports your syntax is the D language with its "...ALL BUT ..." construct:
Wikipedia - D Language Specification
There are some reference implementations available, but mostly for teaching purposes.
Unless there's some special extension in MySql you cannot do that. You either get all, or have to explicitly state what you want. It is best practice to always name columns, as this will not alter the query behaviour even if the underlying table changes.
There is no SQL syntax to support:
select * from table but not select id,active
If you want all but one or more columns, you have to explicitly define the list of columns you want.
You should not be using select * anyway. Enumerate the columns you want and only the columns you want, that is the best practice.
SET #sql = CONCAT('SELECT ',
(SELECT REPLACE(GROUP_CONCAT(COLUMN_NAME), '<columns_to_delete>,', '')
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = '<table>'
AND TABLE_SCHEMA = '<database>'),
' FROM <table>');
PREPARE stmt1 FROM #sql;
EXECUTE stmt1;
I'm fairly certain you can't. Probably the best way I can think of is to create SELECT name, age, surname, lastname, category FROM table as a view, then just SELECT * FROM view. I prefer to always select from a view anyway.
However, as others have pointed out, if another column gets added to the view your application could fail. On some systems as well (PostgreSQL is a candidate) you cannot alter the table without first dropping the view so it becomes a bit cumbersome.
If the reason is to avoid column duplication error without having to specify a long list of columns:
temporarily change the name of column that is a duplicate to enable the view to be created.
delete the duplicate column from the select and save view
rename the changed column name
If the reason is simply to omit a one or more columns:
create view and delete column/s from select