is it possible to execute a SELECT command on a mysql host db2, being logined to mysql host db1?
e.g. i'm logined on mysql server db1, and i want to SELECT some data from db1 tables and db2 server tables with one query. i've found out that i can use FEDERATED tables, but i'm looking for a simpler way, if it exists
I do not think there is a simpler way. I can't find one, at least.
I did a bit or research on this. (Was interested.) The manual entry for the SELECT query mentions nothing that would allow it to connect to a database on a second server, and I can't see anything else that would allow it either. (Nothing in the JOINS, nothing in the CREATE TABLE syntax, just nothing...)
Except for the FEDERATED engine, that is. This is exactly why it was crated; to allow this sort of thing.
I suppose you could try to set your OS up to create a symbolic link to the data files of a second server and have the first server set up a table that targets that, but that's hardly an improvement over the FEDERATED engine.
In any case, if there are other alternatives, I doubt they are simpler than the FEDERATED engine. I would at least give it a try.
Related
I am trying to setup a MySQL database that takes data from 3 other MySQL databases. The data that would be copied would be a query that standardizes the data format. The method would need to either be run daily as a script or synced in real time, either method would be fine for this project.
For example:
The query from source DB:
SELECT order_id, rate, quantity
WHERE date_order_placed = CUR_DATE()
FROM orders
Then I want to take the results of that query to be inserted into a destination DB.
The databases are on separate hosts.
I have tried creating scripts that run CSV and SQL exports/imports without success. I have also tried using Python pymysql library but seemed overkill. I'm pretty lost haha.
Thanks :)
Plan A:
Connect to source. SELECT ... INTO OUTFILE.
Connect to destination. LOAD DATA INFILE from the output above.
Plan B (both MySQL):
Set up Replication from the source (as a Master) and the destination (as a Slave)
Plan C (3 MySQL servers):
Multi-source replication to allow gathering data from two sources into a single, combined, destination.
I think MariaDB 10.0 is when they introduced multi-source repl. Caution: MariaDB's GTIDs are different than MySQL's. But I think there is a way to make the replication you seek to work. (It may be as simple as turning off GTIDs??)
Plan D (as mentioned):
Some ETL software.
Please ponder which Plan you would like to pursue, then ask for help in focusing on one. Meanwhile, your question is too broad.
"A FEDERATED table does not support indexes in the usual sense; because access to the table data is handled remotely, it is actually the remote table that makes use of indexes. This means that, for a query that cannot use any indexes and so requires a full table scan, the server fetches all rows from the remote table and filters them locally. This occurs regardless of any WHERE or LIMIT used with this SELECT statement; these clauses are applied locally to the returned rows.
Queries that fail to use indexes can thus cause poor performance and network overload. In addition, since returned rows must be stored in memory, such a query can also lead to the local server swapping, or even hanging."
16.8.3 FEDERATED Storage Engine Notes and Tips
Can anybody explain me on examples what is means?
What is "query that cannot use any indexes"?
This means that i get full data from remote server in any case or not?
The documentation means to say that if you run a query against a federated table, it generates another query that it runs against the remote base table. If the query that runs on the remote server cannot make use of an index, this forces a table-scan on the remote server, and therefore all the rows of that table are copied across the network.
You might think that the query should filter rows on the remote server before sending them back, but it seems it does not do that. It can filter rows on the remote server only if the filtering can be done on the remote side using an index.
There are very few cases where MySQL's federated storage engine is a good idea to use. I avoid it.
Is it possible to replicate a database to a second database in the same server?
I want to replicate a database that is used for an application, and create a copy that will be used for webservice testing puposes, like creating fake orders, fake data, etc... and it would be very nice to get updates from the main db, like product data updates...
I think i could use the binlog-do-db (or something similar) in mysql config and use the server as master and slave, but i played with that config before and had problems. In my current replications i replicate the entire mysql server, so it works.
Also, i dont want to replicate table1 to table1 and instead, table1 to table2. I dont know if thats allowed.
Is this the best approach or i'm trying to do something wrong/not possible? What would you recommend?
You might be tempted to try to set up a single mysqld instance as master and slave with replicate-same-server-id; this won't work, as the server-id must be unique between every other ID in use by any other replication master or slave.
See this Percona article on binlog-do-db et al. You could achieve this with running another mysqld instance on the same node, and configure on the slave replicate-rewrite-db to apply statements to a different database name. I do not see anything about rewriting table names in replication options, though.
Alternatively, depending on the size of the database you are looking to duplicate, you may mysqldump and import to another database.
I wonder if there is any easy way to keep the scheme consistent in two different MySQL clusters. Apart from classic replication, I would like to have a special "replication" which would reproduce all DDL queries (CREATE, ALTER, DROP, ...) on another cluster (namely the master of that cluster).
I don't need the actual data to be replicated.
Has anyone ever done or tried anything like this?
You can filter replication in MySQL based upon the database in which a query was executed. That doesn't prevent you making changes in other databases, however! So you can do;
USE ddl_repl_db;
ALTER TABLE other_db.foo ADD COLUMN <etc>
This relies on you configuring your servers correctly. I haven't set up MySQL replication for a while, but IIRC you can both filter what you send out from the master for replication and what you accept on the slave.
Old but still high in search.
So, on you DDL replica set all tables engine to BLACKHOLE
I have an VFP based application with a directory full of DBFs. I use ODBC in .NET to connect and perform transactions on this database. I want to mirror this data to mySQL running on my webhost.
Notes:
This will be a one-way mirror only. VFP to mySQL
Only inserts and updates must be supported. Deletes don't matter
Not all tables are required. In fact, I would prefer to use a defined SELECT statement to only mirror psuedo-views of the necessary data
I do not have the luxury of a "timemodified" stamp on any VFP records.
I don't have a ton of data records (maybe a few thousand total) nor do I have a ton of concurrent users on the mySQL side, want to be as efficient as possible though.
Proposed Strategy for Inserts (doesn't seem that bad...):
Build temp table in mySQL, insert all primary keys of the VFP table/view I want to mirror
Run "SELECT primaryKey from tempTable not in (SELECT primaryKey from mirroredTable)" on mySQL side to identify missing records
Generate and run the necessary INSERT sql for those records
Blow away the temp table
Proposed Strategy for Updates (seems really heavyweight, probably breaks open queries on mySQL dropped table):
Build temp table in mySQL and insert ALL records from VFP table/view I want to mirror
Drop existing mySQL table
Alter tempTable name to new table name
These are just the first strategies that come to mind, I'm sure there are more effective ways of doing it (especially the update side).
I'm looking for some alternate strategies here. Any brilliant ideas?
It sounds like you're going for something small, but you might try glancing at some replication design patterns. Microsoft has documented some data replication patterns here and that is a good starting point. My suggestion is to check out the simple Move Copy of Data pattern.
Are your VFP tables in a VFP database (DBC)? If so, you should be able to use triggers on that database to set up the information about what data needs to updated in MySQL.