TRIGGERs can be used to log changes to individual DB columns as described at https://stackoverflow.com/a/779250/569976 but that technique requires you have an IF statement for each column. It's not a huge issue if you're just interested in changes to one column BUT if you're interested in changes to all columns it becomes a bit more unweildy.
I can get all the column names of a table, dynamically, by querying the INFORMATION_SCHEMA.COLUMNS table. My question is... can I use that to dynamically reference the column names? Like in the TRIGGER you'd do OLD.columnName <> NEW.columnName but I don't think you can really make a column name dynamic like that.
In PHP you could use variable variables. eg. $obj->$var. But if MySQL has anything remotely similar that'd be news to me.
Any ideas? Or am I just going to go with the old fashioned approach of writing an IF statement for each of the 100s of columns this table has?
The trigger can only reference identifiers directly. You can't use a variable or an expression to name an identifier.
That would require dynamic SQL with PREPARE and EXECUTE so you could have the statement parsed at runtime from a string, but you can't PREPARE a new statement inside a trigger, because the trigger is already executing in the context of the currently executing statement.
The simplest solution is to write a trigger that references each column directly, with as many IF statements as there are columns in the table (I wonder why you have hundreds of columns in your table; that sounds like a different problem of bad design).
The comments above mention a binary log parser. Debezium is an example of an open-source binlog parser.
MySQL also supports an audit plugin architecture, but frankly the existing implementations of audit plugins are pretty clumsy.
https://www.mysql.com/products/enterprise/audit.html
https://mariadb.com/resources/blog/introducing-the-mariadb-audit-plugin/
https://github.com/mcafee/mysql-audit
Related
So I'm kind of stumped.
I have a MySql project that involves a database table that is being manipulated and altered by scripts on a regular basis. This isn't so unusual, but I need to automate a script to run (after hours, when changes aren't happening) that would save the result of the following:
SHOW CREATE TABLE [table-name];
This command generates the ready-to-run script that would create the (empty) table in it's current state.
In SqlWorkbench and Navicat it displays the result of this SHOW command in a field in a result set, as if it was the result of a SELECT statement.
Ideally, I want to take into a variable in a procedure, and change the table name; adding a '-mm-dd-yyyy' to end of it, so I could show the day-to-day changes in the table schema on an active server.
However, I can't seem to be able to do that. Unlike a Select result set, I can't use it like that. I can't get it in a variable, or save it to a temporary, or physical table or anything. I even tried to return this as a value in a function, from which I got the error that a function cannot return a result set - which explains why it's displayed like one in the db clients.
I suspect that this is a security thing in MySql? If so, I can totally understand why and see the dangers exposed to a hacker, but this isn't a public-facing box at all, and I have full root/admin access to it. Hopefully somebody has already tackled this problem before.
This is on MySql 8, btw.
[Edit] After my first initial comments, I need to add; I'm not concerned about the data with this question whatsoever, but rather just these schema changes.
What I'd really -like- to do is this:
SELECT `Create Table` FROM ( SHOW CREATE TABLE carts )
But this seems to be mixing apples and oranges, as SHOW and SELECT aren't created equal, although they both seem to return the same sort of object
You cannot do it in the MySQL stored procedure language.
https://dev.mysql.com/doc/refman/8.0/en/show.html says:
Many MySQL APIs (such as PHP) enable you to treat the result returned from a SHOW statement as you would a result set from a SELECT; see Chapter 29, Connectors and APIs, or your API documentation for more information. In addition, you can work in SQL with results from queries on tables in the INFORMATION_SCHEMA database, which you cannot easily do with results from SHOW statements. See Chapter 26, INFORMATION_SCHEMA Tables.
What is absent from this paragraph is any mention of treating the results of SHOW commands like the results of SELECT queries in other contexts. There is no support for setting a variable to the result of a SHOW command, or using INTO, or running SHOW in a subquery.
So you can capture the result returned by a SHOW command in a client programming language (Java, Python, PHP, etc.), and I suggest you do this.
In theory, all the information used by SHOW CREATE TABLE is accessible in the INFORMATION_SCHEMA tables (mostly TABLES and COLUMNS), but formatting a complete CREATE TABLE statement is a non-trivial exercise, and I wouldn't attempt it. For one thing, there are new features in every release of MySQL, e.g. new data types and table options, etc. So even if you could come up with the right query to produce this output, in a couple of years it would be out of date and it would be a thankless code maintenance chore to update it.
The closest solution I can think of, in pure MySQL, is to regularly clone the table structure (no data), like so:
CREATE TABLE backup_20220618 LIKE my_table;
As far as I know, to get your hands on the full explicit CREATE TABLE statement, as a string, would require the use of an external tool like mysqldump which was designed specifically for that purpose.
I am trying to match two MySQL Queries (for now, the target is "Create VIEW") to analyze if the result of execution would result in the same effect to Database.
The source of the queries is not the same, making the syntax across the queries inconsistent.
To further simplify the question, let me add more details:
Let's say there is an already existing View in the database.
This View was created using a Create VIEW ... SQL statement.
There is a possibility that the Create VIEW ... statement get's updated, hence to reflect the changes in the database currently this statement is executed at the time of migration.
But, I want to avoid this situation, if the statement Create VIEW ... will result in the same structure as of the existing View in the database, I want to avoid executing it.
To generate the CREATE VIEW from database I am using SHOW CREATE VIEW... (comparing this with the query originally used to create the VIEW).
The primary restriction is I need to make this decision only at the time of migration and cannot presume any conclusions (say, using git diff or commit history...).
I have already done some search to look for a solution for this:
Found no direct solution for this problem (like a SQL engine to which I can feed both queries and know if the result would be the same).
Decided to Parse the queries and to achieve that ended up looking into ANTLR (also used by MYSQL WorkBench)
ANTLR's approach looks promising but, this will require an extensive rule-based parsing and creating a query match program from scratch.
I realized that just parsing queries is not enough, I have to create my own POJOs to store the atomic lexers from queries and then compare the queries based on some rules.
Even if I could find predefined POJOs, that would allow to quickly create a solution for this problem.
I know that I can view the properties and do some amazing things to learn my db/table/field structure using INFORMATION_SCHEMA.
However, is it possible to update a field value in for example the COLUMNS table and thus update the actual column? For example setting nullable from NO to YES.
If this is not directly possible, I realize that I could use the query to CONCAT an ALTER string and then run those strings. However is there in that case a way to instead run an eval() command to do this in one operation? Thanks.
To learn INFORMATION_SCHEMA: I'd start from reading documentation - INFORMATION_SCHEMA Tables. Then I'd try to edit objects and see what happens in these tables.
INFORMATION_SCHEMA tables are system tables, they cannot be modified. ALTER TABLE is the only way to change a table.
It was a beautiful idea, but unfortunately, it won't work.
The tables in INFORMATION_SCHEMA are read-only views, and are not actually tables. So there aren't any files or directories associated with them. You can only read their contents and can't run INSERT, UPDATE, or DELETE operations on them.
See https://dev.mysql.com/doc/refman/5.7/en/information-schema-introduction.html#information-schema-usage-notes for the lowdown.
And FYI, there's a Database Administrators Stack Exchange site if you want more targeted answers to database questions.
So I don't want this thread to be marked as spam, as a previous thread was on this topic was, so I will explain what I have done so far and my issue is and ask if there are any solutions.
I have a MySQL database on my laptop that I need to migrate to DB2 on iSeries. I'm using a tool, I won't say which one because of the spam issue, which allows me to "copy" a table in my MySQL database and "paste" it into my DB2 database.
The issue that I'm having is because the table names and column names contain spaces in the MySQL db, the tool is failing on the paste. I confirmed this by altering one table by replacing the spaces with underscores and the copy worked perfectly. I have over a hundred tables I need to copy over and don't want to have to manually edit every table and column name.
Is there a way to globally replace spaces with underscores in MySQL table names and columns?
Any other ideas? I'm also researching a way to force the query the tool creates to enclose the object names in quotes, but have had no luck so far.
Thanks for any help and suggestions you can provide.
Since Stack Overflow is about helping to solve programming problems, I'm going to ignore the issue of deficiencies in the chosen tool and propose a programming solution to the larger problem - DB2 does not allow spaces in table and column names. You did ask for any suggestions...
Write code that reads the MySQL catalog tables. In DB2 they'd be SYSTABLES, SYSVIEWS, SYSINDEXES, SYSCOLUMNS, etc. Read SYSTABLES and use that as the 'primary' source for the rest of the code. Check the table name; if it has an embedded space, replace it with an underscore. Use SYSCOLUMNS to generate a CREATE TABLE statement that will create the new table (in a new MySQL database?) - also performing space to underscore replacement. After issuing the CREATE TABLE, generate an SQL statement that will INSERT INTO the new table the columns from the old table; again doing the space to underscore substitutions. Once the new table is populated, generate SQL statements to CREATE VIEW, CREATE PROCEDURE, CREATE FUNCTION, etc.
The general idea is that you will completely re-create your MySQL database with table, view and column names that are immediately compatible with DB2 for i so that your tool can do it's thing.
Of course, if you go to that much trouble it'll probably be just as easy to directly CREATE TABLE, etc on the IBM i side rather than go through an intermediate tool that isn't quite what you need.
I have a dataset with a lot of columns I want to import into a MySQL database, so I want to be able to create tables without specifying the column headers by hand. Rather I want to supply a filename with the column labels in it to (presumably) the MySQL CREATE TABLE command. I'm using standard MySQL Query Browser tools in Ubuntu, but I didn't see in option for this in the create table dialog, nor could I figure out how to write a query to do this from the CREATE TABLE documentation page. But there must be a way...
A CREATE TABLE statement includes more than just column names
Table name*
Column names*
Column data types*
Column constraints, like NOT NULL
Column options, like DEFAULT, character set
Table constraints, like PRIMARY KEY* and FOREIGN KEY
Indexes
Table options, like storage engine, default character set
* mandatory
You can't get all this just from a list of column names. You should write the CREATE TABLE statement yourself.
Re your comment: Many software development frameworks support ways to declare tables without using SQL DDL. E.g. Hibernate uses XML files. YAML is supported by Rails ActiveRecord, PHP Doctrine and Perl's SQLFairy. There are probably other tools that use other format such as JSON, but I don't know one offhand.
But eventually, all these "simplified" interfaces are no less complex to learn as SQL, while failing to represent exactly what SQL does. See also The Law of Leaky Abstractions.
Check out SQLFairy, because that tool might already convert from files to SQL in a way that can help you. And FWIW MySQL Query Browser (or under its current name, MySQL Workbench) can read SQL files. So you probably don't have to copy & paste manually.