I am preparing to make some changes to a database I manage and was unsure that what I wanted to do would replicate properly so I ran some tests in a test environment and it turns out they will but only as long as I do not run the commands from the MySQL Workbench.
For example if have a database named db_test and a table in that database named test_a having only a single column id and I try to execute this from the workbench: INSERT INTO db_test.test_a (id) VALUES (114);
I get the expected row in the master database, but it never replicates to the slave.
When I perform a SHOW SLAVE STATUS - it shows everything is fine, and current. If I then use a different SQL client such as SequelPro and insert another row the same way (but obviously a different id) it shows in the master and replicates to the slave.
This has me baffled, and concerned as I want to understand what the difference is so I can avoid performing actions that never replicate.
If you have set --replicate-do-db on the slave to filter replication for database db_test, replication is filtered based on the default database, so make sure that you issue USE db_test. Your client may be working differently in this manner, or you may be issuing different statements between clients.
Using --replicate-do-db set to db_test on the slave, this will replicate:
USE db_test;
INSERT INTO test_a (id) VALUES (114);
but this will not:
USE other_db;
INSERT INTO db_test.test_a (id) VALUES (114);
To get replication to work regardless of the current default database, use --replicate-wild-do-table to configure the database and table to replicate or don't filter at all.
Also, make sure that you are connected to the Master database server.
Related
I want to replicate certain table from one database into another database in the same server. This tables contain exactly the same fields.
I was considering to use MySQL Replication to replicate that table but some people said that it will increase IO so i find another way to create 3 Trigger (Insert, update and Delete) that will perform exactly the same thing like what i expect.
My Question is, which way is better? Is it using MySQL replication is better even though it's in the same server or using Trigger to replicate the data is better.
Thanks.
I don't know what is your goal, but I got mine getting use of the VIEW functionality.
I had two different applications with separate databases but in the same Mysql server. Application2 needed to get a few data from Application1. In general, this is a trivial situation that you can handle with USE DB1; or USE DB2; as your needing, but my programming framework does not work very well with multiple DBs.
So, lets see my solution...
Here is my select query to retrieve this data:
SELECT id, name FROM DB1.customers;
So, using DB2 as default schema, I've created a VIEW:
USE DB2;
CREATE VIEW app1_customers AS SELECT id, name FROM DB1.customers;
Now I can retrieve this data in DB2 as a regular table with a regular SELECT statement.
SELECT * FROM DB2.app1_customers;
Hope ts useful. BR
Assuming you have two databases on the same server i.e DB1 and DB2 and the table is called tbl1 and it is sitting in DB1 you can query the table like this:
USE DB1;
SELECT * FROM tbl1;
USE DB2;
SELECT * FROM DB1.tbl1;
This way you wont need to copy the data and worry about extra space and extra code. You can query a table in another database on the same server. Replication and triggers are not your answer here. You could also create a view to encapsulate the SQL statement.
Definitely triggers is the way to go. Having another server (slave) will need to spare several MB for installation, logs, cpu and memory usage.
I'd use triggers to keep both tables equal. If you want to create a table with the same columns definition and data use:
USE db2;
CREATE TABLE t1 AS SELECT * FROM db1.t1;
After that, go ahead and create the triggers for Update, Insert and Delete statemetns.
Also you could ALTER the new table to a different engine like MEMORY or add indexes to see if you can improve something.
I just configure 2 mySQL servers for a MASTER-MASTER replication.
I choose the RBR replication for some reason.
My congifuration on the server ONE :
#replication
server-id=1
binlog_do_db = db1
binlog_ignore_db = db2
log-bin="C:/ProgramData/MySQL/my56"
auto_increment_increment = 2
auto_increment_offset = 1
binlog_format=ROW
replicate_do_db=db1
and on the server TWO :
#replication
server-id=2
binlog_do_db = db1
log-bin="C:/ProgramData/MySQL/my56"
auto_increment_increment = 2
auto_increment_offset = 2
binlog_format=ROW
replicate_do_db=db1
With this, the replication works.
For example, on server ONE, if I execute :
USE db1;
INSERT INTO db1.table1 values (foo,bar);
It's works on the server TWO.
If on server ONE I execute :
USE db1;
INSERT INTO second_db.table2 values (foo,bar);
The insert is not execute on the server TWO, it's good.
If on server ONE I execute :
CREATE table db1.tableFoo(id INT NOT NULL AUTO_INCREMENT PRIMARY KEY);
The create table is not execute on the server TWO, it's good because I choose the Row-Based replication, so I have to manualy execute the CREATE STATEMENT on server TWO. It's what I want.
Now, there is my problem :
If on the server ONE I execute :
USE db1;
CREATE table db1.tableFoo(id INT NOT NULL AUTO_INCREMENT PRIMARY KEY);
The CREATE TABLE is execute on the server TWO, it's NOT good ! Normally with the Row-Based Replication the CREATE ORDER is not replicated.
Worst, if after the USE db1; if create a table in another database, the CREATE TABLE is replicated on my server TWO, and my slave is aborted on the server TWO because de database doesn't exist...
Do you have any idea ? I dont want any CREATE / ALTER / CREATE USER ... send to my replication even if I use à USE db1;
I based my work on the mySQL documentation, especially this one : http://dev.mysql.com/doc/refman/5.6/en/replication-options-binary-log.html
Thank you and merry Xmas !
DDL statements are always logged using statement-based replication, regardless of whether you've chosen RBR or not. As a result, the default (current) database is important when you execute CREATE TABLE statements.
From http://dev.mysql.com/doc/refman/5.6/en/replication-options-binary-log.html#option_mysqld_binlog-do-db :
DDL statements such as CREATE TABLE and ALTER TABLE are always logged as statements, without regard to the logging format in effect, so the following statement-based rules for --binlog-do-db always apply in determining whether or not the statement is logged.
...Only those statements are written to the binary log where the default database (that is, the one selected by USE) is db_name.
This suggests that the behavior you observe is expected, although it is a bit odd.
If possible, I'd suggest that you USE an unreplicated DB (for example mysql) before executing DDL that you do not want to replicate in your application.
In my prior job, I was able to copy data from our production environment in a breeze by using the following statements:
from tablename#UNIXPROD2
INSERT INTO tablename#UNIXTEST2
My current job's databases aren't setup in this fashion.
So, I did some research on MySQL 5.0+ because that's what we are using for one of our customers. And I came across FEDERATED tables, so as I was reading, I found this (here):
As of MySQL 5.0.46, FEDERATED performs bulk-insert handling such that multiple rows are sent to the remote table in a batch. This provides a performance improvement. Also, if the remote table is transactional, it enables the remote storage engine to perform statement rollback properly should an error occur. This capability has the following limitations:
To me, this indicates that (A) I can copy the data from our prod database to our test database; (B) any actions performed on the federated table will also be processed on the source table, which is not what I want to do. I have some scripts that I need to run and I want to run it against actual prod data to make sure it works before I use it in the prod environment.
My question: Is my interpretation correct?
Assuming it is, I've tried:
select * from database.tablename#ipaddress, but received an error message that told me to check the MySQL manual for the version I'm running, which is what I'm going to do after I hit "Post Your Question."
I would appreciate any help in this matter.
EDIT: After further research, I think might be able to do what I need using OUTFILE and INFILE whereby I would use OUTFILE on the prod table(s) and then INFILE those rows on the test table(s). Thoughts?
My answer:
A - correct
B - correct.
You could set the user permission to read-only, but in your situation I would not use federated tables, instead dump the whole db into file and then restore it on the other server. Easiest way - use MySql Workbench.
and some info about federated tables:
You need federated enabled just on server B
You can access a view on A by making a federated table on B
You can do INSERT UPDATE DELETE on federated table
If you need read-only access you can limit the user privileges
BUT! You can't do any aggregate func. on a view which will be federated (ex. COUNT(), MAX(), UNION...) (you can, however it will lag)
Remember to set the KEY's on the federated table you are creating. (or it will lag horr.)
Remember to use ALGORITHM=MERGE on views
Remember to grant acces to USERNAME(from connection string) on server A
example of a federated table on server B:
delimiter $$
CREATE TABLE `schemaName`.`tableName`(
`keyName` VARCHAR(10) NOT NULL,
`key2Name` DATE DEFAULT '2012-01-01',
KEY `keyName` (`keyName`)
)
ENGINE=FEDERATED
DEFAULT CHARSET=utf8
CONNECTION='mysql://USERNAME:PASSWORD#IPADDRESS:PORTNUMBER/baseSchema/baseTable'
$$
And the view on server A:
CREATE
ALGORITHM = MERGE
DEFINER = `ANOTHERUSERNAME`#`%`
SQL SECURITY DEFINER
VIEW `baseSchema`.`baseTable` AS
SELECT
... AS `keyName`,
... AS `key2Name`
FROM
...
Is it possible to use any sort of logic in MySQL without using any procedures? My web hosting does not let me create any procedures so I'm looking for a workaround.
The type of thing I want to do is only add an item to a table if it doesn't already exist. Or add a column to a table if it's not already there. There are some operations that can be done such as CREATE TABLE IF NOT EXISTS and so on, but some operations I require do not have such luxuries :(
I realised late on that my lovely procs won't work and so I tried writing IF/ELSE logic as top-level queries, but for MySQL, IF ELSE blocks only seem to work inside functions/procs and not at the global scope.
Any workarounds greatfully received - I've already asked the hosting to grant me privileges to create procedures but no reply as yet...
I suppose you don't have access to the INFORMATION_SCHEMA either. You can possibly find solutions but it would be better, in my oninion, to:
Change your hosting provider. Seriously. Pay more - if needed - for a MySQL instance that you can configure to your needs. You only have a crippled DBMS if you are not allowed to create procedures and functions.
Posible workarounds for the specific task: You want to add a column if it doesn't exist.
1) Just ALTER TABLE and add the column. If it already exists, you'll get an error. You can catch that error, in your application.
2) (If you have no access to the INFORMATION_SCHEMA) maintain a version of the schema, for your database.
The best solution that I can think of would be to use an additional language with SQL. For example, you can run a query for a specific record, and based on the response that you get, you can conditionally run an INSERT statement.
For inserting a table if it doesn't exist, try using the SHOW TABLES statement and testing whether or not a name exists in the result set.
MySQL supports INSERT IGNORE. and INSERT ... ON DUPLICATE KEY UPDATE.
The following will insert a new row, but only if there is no existing row with id=10. (This assumes that id is defined as a unique or primary key).
INSERT IGNORE INTO my_table (id, col1, col2) values (10, "abc", "def");
The following will insert a new row, but if there is an existing row with id=10 (again, assuming id is unique or primary), the existing row will be updated to hold the new values, instead of inserting a new row.
INSERT INTO my_table (id, col1, col2) values (10, "abc", "def")
ON DUPLICATE KEY UPDATE col1=VALUES(col1), col2=VALUES(col2)
Also, CREATE TABLE supports the IF NOT EXISTS modifier. So you can do something like:
CREATE TABLE IF NOT EXISTS my_table ...
There are many other similar options and modifiers available in MySQL. Check the docs for more.
Originally I created a big script to create or update the database schema, to make it easier to deploy database changes from my local machine to the server.
My script was doing a lot of "if table 'abc' exists and it doesn't have a FK constraint called 'blah'" then create an FK constraint called 'blah' on table 'abc'... and so on.
I now realise it's not actually necessary to check whether a table has a certain column or constraint etc, because I can just maintain a schema-versioning system, and query the DB schema-version when my app starts, or when I navigate to a certain page.
e.g. let's say I want to add a new column to a table. It works like this:
Add a new migration script to the app code, containing the SQL required to add the column to the existing table
Increment the app's schema-version by 1
On app startup, the app queries the DB for the DB's schema-version
If DB schema-version < app schema-version, execute the SQL migration scripts between the two schema-versions, and then update the DB schema-version to be the same as the app
e.g. if the DB's schema-version is 5 and the app version is 8, the app will apply migration scripts 5-6, 6-7 and 7-8 to the DB. These can just be run without having to check anything on the DB side.
The app is therefore solely responsible for updating the DB schema and there's no need for me to ever have to execute schema change scripts on the local or remote DB.
I think it's a better system than the one I was trying to implement for my question.
We have mysql v5.0.77 running on a server collecting some measurement data.
On the mysql server, we have the following databases:
raw_data_db
config_tables_db
processed_data_db
We ONLY want to replicate the 'processed_data_db' which is constructed using information from the 'raw_data_db' and 'config_tables_db'.
We keep getting errors on our slave server when it tries to duplicate the statements that are constructing the processed data.
Example:
[ERROR] Slave: Error 'Table 'raw_data_db.s253' doesn't exist' on query. Default database: 'data'. Query: 'CREATE TEMPORARY TABLE temp SELECT * FROM raw_data_db.s253 WHERE DateTimeVal>='2011/04/21 17:00:00' AND DateTimeVal<='2011/04/21 17:10:00'', Error_code: 1146
What I am assuming is happening is that the cross-db selects can't find the raw database because we aren't replicating it, and the data do not exist on the slave...or something along those lines?
So I tried using ignores, but we're still getting the errors
replicate-wild-ignore-table = raw_data_db.*
replicate-wild-ignore-table = data.temp*
Other configuration information:
replicate-rewrite-db = processed_data_db->data
replicate-do-db = data
Is it possible to replicate just the one database if all the tables are created from references to other databases? Any ideas on how to get around this error?
I looked in to row-based replication which seemed like it might do the trick, but it's only available in v5.1 or greater....is there anything similar in earlier versions?
I fixed the ignore table statements to "data.%temp%", and it seems to be ignoring just fine, but I still can't replicate the tables I want because the insert statement is now referencing a table that doesn't exist.
ex.
Error 'Table 'data.temp' doesn't exist' on query. Default database: 'data'. Query: 'INSERT INTO abc SELECT FROM_UNIXTIME(AVG(UNIX_TIMESTAMP(DateTimeVal))), ROUND(AVG(Difference),3), ROUND(STDDEV(Difference),3), ROUND(AVG(Frequency),0), ROUND(AVG(SignalPower),1) FROM temp WHERE ABS(Difference)<'10000.0' AND Difference!='0''
The processing is creating temporary tables from the raw database and then averaging all the values in the temporary table and inserting the result in to the processed_data_db, but since I'm ignoring the create statements, it doesn't have access to those tables, but the reason I'm ignoring them in the first place is because they reference tables outside of what I want to replicate...so I'm not sure how I should approach this....any suggestions would be greatly appreciated.
Temporary tables and replication
options. By default, all temporary
tables are replicated; this happens
whether or not there are any matching
--replicate-do-db, --replicate-do-table, or --replicate-wild-do-table options in effect. However, the
--replicate-ignore-table and --replicate-wild-ignore-table options are honored for temporary tables.
http://dev.mysql.com/doc/refman/5.0/en/replication-features-temptables.html
edit:
replicate raw_data_db and config_tables_db tables which using
in you insert query
use drbd protocol
http://www.mysql.com/why-mysql/drbd/