Running mysqlbinlog to load up binary logs from one server to another.
Consistently get message:
WARNING: The option --database has been used. It may filter parts of transactions, but will include the GTIDs in any case.
Yah -- OK? So??
Well maybe this is a stupid question, but how am I supposed to distinguish the "GTID" of the database I want from the "GTID" of the database I don't want? In other words, how do I specify transactions to a particular database while shutting off this annoying warning?
Tried adding "--include-gtids" parameter, but I think it wants a list of GTIDs. I don't have a list of GTIDs. I have a database. Why is that complicated?
It's complicated because --database doesn't mean what you probably think it means.
It does NOT mean only include changes to the named database.
It means:
This option causes mysqlbinlog to output entries from the binary log (local log only) that occur while db_name is been selected as the default database by USE.
For example:
USE db1;
INSERT INTO db2.mytable ...
This will NOT be included if you use --database db2, because db2 wasn't the default database when this transaction was written to the binary log.
Another example:
USE db3;
UPDATE db1.mytable JOIN db2.myothertable
SET db1.col1 = ...,
db2.col2 = ...;
Should the changes to db1.mytable resulting from this be included if you use --database db2?
Trick question: the changes to neither table will be included, because the current default database was db3.
Related
I want to write a set of small shell scripts to help me keep MySQL schema in git. But one crux is that mysqldump -uuser -ppass -d devdb > schema.sql includes two things that change over time even if there are no schema changes:
AUTO_INCREMENT=[some number] at the end of the definition of any table that has an auto-incremented column
-- Dump completed on [date and time] at the end
I have scoured the web for ways to get a dump without those things, to no avail. Can you advise? Or should I be using a different command or tool to get a clean schema for use in version control?
EDIT: I just now found the --skip-dump-date option, so that solves point #2, but I still can't get rid of the auto-increment number without losing the other table attributes (or whatever you call those things) like the engine and the default character set.
There is no way to bypass #1, check https://bugs.mysql.com/bug.php?id=20786.
As noted in the comments section, you can use that sed command to filter it out (unless you have some CREATE TABLE statements that use it).
When I create a view, I create it in the context of the default database. So none of my references to table have a prefix which explicitly specify a database. However, when I edit a view in Workbench it automatically adds the database prefix!
I don't want the database prefix because when I restore a database under a different name it causes the restore to fail.
Is this possible to stop the prefixing in a view edit or there another way to get round the restore issue?
see https://bugs.mysql.com/bug.php?id=85176
The mysql 8.0.3 or above has been fixed
That's not possible. Views are stored in specific databases, not in some space "above" all databases. Consider following...
use playground_a; /*or whatever database*/
create view view_whatever as
select * from table_whatever;
use playground_b;
select * from view_whatever; /*here you will get an error that view_whatever does not exist*/
select * from playground_a.view_whatever; /*this works*/
That's why there will always be database prefixes in the view definition.
The only possibility I see, would be to use a stored procedure with a database name as parameter. In the procedure you'd use a prepared statement to execute a concated string of your query and the database name parameter. Of course this comes with downsides, like i.e. you can't add a where clause easily.
Creating the view without explicitely specifying a schema is a convenience feature. Behind the scenes the view is still saved in a specific schema (the default one in this case). When editing the source code is retrieved from the server which returns the real code (including the schema qualification). Hence already when you send the view code the association happens and cannot be removed again later.
Here is the command I use to create the backup:
mysqldump -u xxxxxx -pxxxxxx --routines database_a | gzip -9 > $FULLGZIPPATH
If you aren't easily able to update to MySQL 8.X then a workaround I've implemented was a post-processing step performed on the dump file itself prior to importing. I just remove the explicit prefixed db name, since the import process / view creation doesn't need it.
PowerShell -Command ^
"filter replace-dbname { $_ -replace '`<DB_NAME>`.`', '`' }"^
"Get-Content dump.sql -ReadCount 10 | replace-dbname | Add-Content replaced_dump.sql"
I've used PowerShell since I'm on Windows, but any scripting language will do. The only notes are that:
You'll need to do the replacement a-few-lines-at-a-time if you can't afford to read the entire dump into memory. Our dumps are about 11GB, which'd be a strain on our testing server's resources.
In my script I'm not doing an in-place string replacement, so it'll create a new dump file replaced_dump.sql alongside the original dump.sql. For me this was useful for diagnostics, because it meant if there was an issue, I didn't have to dump it again. Again, depending on your dump/disk size this might be an issue.
If your database happens to have `<DB_NAME>`.` as content in something like a text-field, this basic approach will also remove the string there as well.
We have a database in which one specific table gets very large very fast. When taking a dump and then restoring this database on another server, it takes a lot of time due to the large size. I want to exclude this specific table when taking the dump. Any ideas ?
mysqldump has a --ignore-table option, something like:
mysqldump -p -u username nameofdatabase --ignore-table=verybigtable
Use the --ignore-table flag.
--ignore-table=name Do not dump the specified table. To specify more than one
table to ignore, use the directive multiple times, once
for each table. Each table must be specified with both
database and table names, e.g.,
--ignore-table=database.table.
Most graphical clients (like phpmyadmin) give you exclusion options.
I would like to write a script that supports the following workflow.
given: a defined set of queries (select statements with table joins) that return sets of data from a single MySQL database
create: a SQLite database that contains the information (tables, data) required to returned the same results to the same set of queries sent in step 1.
Outside of select, delete, and update, I am relatively unfamiliar with SQL, so I would appreciate specific command line or SQL syntax... anything required beyond installing SQLite.
This's an half answer but can be usefull also for other kind of DB.
mysqldump has the option "compatible" to control the standard compliance of the sql dumped.
From the "mysqldump --help" execution:
--compatible=name Change the dump to be compatible with a given mode. By
default tables are dumped in a format optimized for
MySQL. Legal modes are: ansi, mysql323, mysql40,
postgresql, oracle, mssql, db2, maxdb, no_key_options,
no_table_options, no_field_options. One can use several
modes separated by commas. Note: Requires MySQL server
version 4.1.0 or higher. This option is ignored with
earlier server versions.
Abe, this doesn't answer your question but might help you get started. You can export the database using mysqldump with --complete-insert (since sqlite does not support multi-row / compound inserts), then use sqlite3_exec() to import the dump to SQLite
In the same way SHOW CREATE TABLE tblname; brings back what was previously executed, is there anyway to view the SQL of an ALTER TABLE query?
Please help?
One small clarification: show create table does not actually "bring back what was previously executed". It just shows you the DDL that would create the table from scratch. The table may have been created and then altered many times, but show create table reflects the current state of the table.
As for finding any alter table statements that ran on the table recently, the best bet is the binary log.
First check to see if binary logging is enabled:
show variable like 'log_bin';
If it is, find the binary log for the relevant time period, use mysqlbinlog to convert it to SQL, then grep for the relevant table name to find the alter table statement you are looking for.
Tools:
Maatkit.
Red-Gate's MySQL Schema & Data Compare
Toad
SQLYog
MySQL Diff
Manual:
First check to see if binary logging is enabled:
show variable like 'log_bin';
Then,
mysqlbinlog /var/log/mysql/mysql-bin.000001 | grep 'alter table *tablename*' > test.file
Go to test.file to see the alter statements.
If your MySQL instance has logging turned on, you can view the logs. This would be the best option.
If your system has history turned on, you can start a client back up from the same system and try the up arrow. You might be able to see the command there.
If you know the user that ran the command and they happened to run it directly from a command line, the same history trick might work.