I am trying to replicate a specific view on mysql slave while ignoring the base table.
I have created a view that select * from a specific table on a specific DB.
In the slave my.cnf I have restricted the replication to the following:
replicate-do-db=DBNAME
replicate-ignore-table=TABLENAME
When I start the replication on the slave, I get an sql error :
Last_SQL_Error: Error 'TABLENAME 'DBNAME.TABLENAME' doesn't exist' on query. Default database: 'DBNAME'. Query: 'CREATE ALGORITHM=UNDEFINED DEFINER=`root`#`localhost` SQL SECURITY DEFINER VIEW `TABLENAMEVIEW` AS SELECT * FROM TABLENAME'
I am using mysql 5.5.1 and as per the following http://dev.mysql.com/doc/refman/5.5/en/replication-features-views.html a view can be replicated even if the table is ignored.
Any idea how I can solve this ?
Thanks,
Without the base table, the VIEW won't work. You can ignore the base table in replication so that new data doesn't come in but that table must exist for the VIEW to function.
VIEWs in MySQL are nothing more than simple aliases for SELECT statements. They are not materialized, no data is stored within, and thus execute their underlying SELECT each time you SELECT from the VIEW.
Related
I'm trying to run the following MySQL command:
USE database_name;
DROP TEMPORARY TABLE IF EXISTS only_with_balance;
DROP TEMPORARY TABLE IF EXISTS keys_to_match;
CREATE TEMPORARY TABLE only_with_balance as (
SELECT
*
FROM
transactions t
WHERE
t.balance is not NULL
and (t.transaction_status_id = 4 or t.transaction_status_id = 5)
and (t.date between "2022-05-01" and "2022-08-24" )
);
But I'm getting a syntax error while trying to run the all the commands at once.
You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'DROP TEMPORARY TABLE IF EXISTS only_with_balance;
DROP TEMPORARY TABLE IF EXIST' at line 2
When I run each command separately, the result is the expected.
Can someone help me here?
What am I forgetting?
In MySQL, by default the query interface only allows one SQL statement per call.
There's an option to enable multi-query per call, but it must be set at connect time. Some MySQL connectors do this by default, or allow it as an option, but some do not. You didn't say if you're writing code or if you're submitting this set of queries through a client (though you tag the question 'dbeaver' you don't say anything else about that). So I can't guess what interface you're using for these queries.
Anyway, there's no advantage to using multi-query. The default mode is one SQL statement per call. That's what I do.
Using the default mode of a single SQL statement per call has some advantages:
Supports prepared statements and bound parameters (you can't run multiple statements in a single prepare call, even if you enable multi-query).
Simplifies processing errors and warnings.
Simplifies processing result sets.
I'm running mysql 5.6 onn AWS RDS. I have a mysql slave via a readreplica in RDS as well. Replication to the slave gets an error when running a stored proc that inserts from a temporary table into a non-temporary table.
My reading of the mysql documentation is that this case is handled as long as we use mixed mode binlogging (which we do). http://dev.mysql.com/doc/refman/5.6/en/binary-log-mixed.html
Is this a mysql bug or am I missing something? Is this approach simply not supported when using mysql slaves?
The stored proc that is causing trouble is doing something like this, where MySummaryTable is the non-temp table and tmp_locations_table is the temp one:
CREATE TEMPORARY TABLE tmp_locations_table
INSERT INTO MySummaryTable (
accountID,
locationID
)
SELECT
row_data.accountID,
row_data.locationID
FROM tmp_locations_table row_data
GROUP BY
row_data.accountID,
row_data.locationID
ON DUPLICATE KEY UPDATE
locationID=123
The exact mysql error I'm seeing isn't particularly helpful: Could not execute Update_rows event on table myschema.MySummaryTable; Can't find record in 'MySummaryTable', Error_code: 1032; handler error HA_ERR_KEY_NOT_FOUND; the event's master log blahblahblah
I want be able to debug easily my scripts in Mysql, like in MSSQL (run a chunk of the script then verify the tables and so on), but the temporary tables are not persisted on the server.
For example :
CREATE temporary table a(i int);
INSERT INTO a VALUE (1);
SELECT * FROM a;
If I run the whole script it returns me the right result, but if I run it statement by statement on the insert I get the following error:
SQL.sql: Error (2,13): Table 'test.a' doesn't exist
I suppose this is a server configuration problem.
Temporary tables are dropped when the transaction is over.
from dev.mysql:
Temporary Tables:
You can use the TEMPORARY keyword when creating a
table. A TEMPORARY table is visible only to the current connection,
and is dropped automatically when the connection is closed. This means
that two different connections can use the same temporary table name
without conflicting with each other or with an existing non-TEMPORARY
table of the same name. (The existing table is hidden until the
temporary table is dropped.) To create temporary tables, you must have
the CREATE TEMPORARY TABLES privilege.
Note CREATE TABLE does not automatically commit the current active
transaction if you use the TEMPORARY keyword.
So if you run all these sql in deferent transactions you temporary table wont exist when you run the insert statement.
If these executions are executed in diferent transactions depend on what interface you use. Thats wy if you "run the whole script it returns me the right result" because its all in the same transaction.
You can try to force it to run on the same transaction with:
START TRANSACTION;
<SQL QUERYS>
COMMIT;
anyway i recomend you MySQL Workbench as interface.
my best regards, i hope this help you.
In my prior job, I was able to copy data from our production environment in a breeze by using the following statements:
from tablename#UNIXPROD2
INSERT INTO tablename#UNIXTEST2
My current job's databases aren't setup in this fashion.
So, I did some research on MySQL 5.0+ because that's what we are using for one of our customers. And I came across FEDERATED tables, so as I was reading, I found this (here):
As of MySQL 5.0.46, FEDERATED performs bulk-insert handling such that multiple rows are sent to the remote table in a batch. This provides a performance improvement. Also, if the remote table is transactional, it enables the remote storage engine to perform statement rollback properly should an error occur. This capability has the following limitations:
To me, this indicates that (A) I can copy the data from our prod database to our test database; (B) any actions performed on the federated table will also be processed on the source table, which is not what I want to do. I have some scripts that I need to run and I want to run it against actual prod data to make sure it works before I use it in the prod environment.
My question: Is my interpretation correct?
Assuming it is, I've tried:
select * from database.tablename#ipaddress, but received an error message that told me to check the MySQL manual for the version I'm running, which is what I'm going to do after I hit "Post Your Question."
I would appreciate any help in this matter.
EDIT: After further research, I think might be able to do what I need using OUTFILE and INFILE whereby I would use OUTFILE on the prod table(s) and then INFILE those rows on the test table(s). Thoughts?
My answer:
A - correct
B - correct.
You could set the user permission to read-only, but in your situation I would not use federated tables, instead dump the whole db into file and then restore it on the other server. Easiest way - use MySql Workbench.
and some info about federated tables:
You need federated enabled just on server B
You can access a view on A by making a federated table on B
You can do INSERT UPDATE DELETE on federated table
If you need read-only access you can limit the user privileges
BUT! You can't do any aggregate func. on a view which will be federated (ex. COUNT(), MAX(), UNION...) (you can, however it will lag)
Remember to set the KEY's on the federated table you are creating. (or it will lag horr.)
Remember to use ALGORITHM=MERGE on views
Remember to grant acces to USERNAME(from connection string) on server A
example of a federated table on server B:
delimiter $$
CREATE TABLE `schemaName`.`tableName`(
`keyName` VARCHAR(10) NOT NULL,
`key2Name` DATE DEFAULT '2012-01-01',
KEY `keyName` (`keyName`)
)
ENGINE=FEDERATED
DEFAULT CHARSET=utf8
CONNECTION='mysql://USERNAME:PASSWORD#IPADDRESS:PORTNUMBER/baseSchema/baseTable'
$$
And the view on server A:
CREATE
ALGORITHM = MERGE
DEFINER = `ANOTHERUSERNAME`#`%`
SQL SECURITY DEFINER
VIEW `baseSchema`.`baseTable` AS
SELECT
... AS `keyName`,
... AS `key2Name`
FROM
...
We have mysql v5.0.77 running on a server collecting some measurement data.
On the mysql server, we have the following databases:
raw_data_db
config_tables_db
processed_data_db
We ONLY want to replicate the 'processed_data_db' which is constructed using information from the 'raw_data_db' and 'config_tables_db'.
We keep getting errors on our slave server when it tries to duplicate the statements that are constructing the processed data.
Example:
[ERROR] Slave: Error 'Table 'raw_data_db.s253' doesn't exist' on query. Default database: 'data'. Query: 'CREATE TEMPORARY TABLE temp SELECT * FROM raw_data_db.s253 WHERE DateTimeVal>='2011/04/21 17:00:00' AND DateTimeVal<='2011/04/21 17:10:00'', Error_code: 1146
What I am assuming is happening is that the cross-db selects can't find the raw database because we aren't replicating it, and the data do not exist on the slave...or something along those lines?
So I tried using ignores, but we're still getting the errors
replicate-wild-ignore-table = raw_data_db.*
replicate-wild-ignore-table = data.temp*
Other configuration information:
replicate-rewrite-db = processed_data_db->data
replicate-do-db = data
Is it possible to replicate just the one database if all the tables are created from references to other databases? Any ideas on how to get around this error?
I looked in to row-based replication which seemed like it might do the trick, but it's only available in v5.1 or greater....is there anything similar in earlier versions?
I fixed the ignore table statements to "data.%temp%", and it seems to be ignoring just fine, but I still can't replicate the tables I want because the insert statement is now referencing a table that doesn't exist.
ex.
Error 'Table 'data.temp' doesn't exist' on query. Default database: 'data'. Query: 'INSERT INTO abc SELECT FROM_UNIXTIME(AVG(UNIX_TIMESTAMP(DateTimeVal))), ROUND(AVG(Difference),3), ROUND(STDDEV(Difference),3), ROUND(AVG(Frequency),0), ROUND(AVG(SignalPower),1) FROM temp WHERE ABS(Difference)<'10000.0' AND Difference!='0''
The processing is creating temporary tables from the raw database and then averaging all the values in the temporary table and inserting the result in to the processed_data_db, but since I'm ignoring the create statements, it doesn't have access to those tables, but the reason I'm ignoring them in the first place is because they reference tables outside of what I want to replicate...so I'm not sure how I should approach this....any suggestions would be greatly appreciated.
Temporary tables and replication
options. By default, all temporary
tables are replicated; this happens
whether or not there are any matching
--replicate-do-db, --replicate-do-table, or --replicate-wild-do-table options in effect. However, the
--replicate-ignore-table and --replicate-wild-ignore-table options are honored for temporary tables.
http://dev.mysql.com/doc/refman/5.0/en/replication-features-temptables.html
edit:
replicate raw_data_db and config_tables_db tables which using
in you insert query
use drbd protocol
http://www.mysql.com/why-mysql/drbd/