HI everybody,
i have a question
if i have a computer run sqlite , and i want to make sqlite sync Mysql server in the external network.
If the data in the sqlite have been (changed/ modified), how can I sync my MYSQL DB and sqlite so that the data in sqlite will be (changed/ modified) when the data in MYSQL DB is changed/ modified?
thanks all.
You can try greplicator
geplicator is a real-time solution
designed to replicate data from MySQL
database to any other relational
database, such as Oracle, Microsoft
SQL Server, IBM DB2 UDB and MySQL.
I have a similar problem of 2-way syncing between an Android sqlite DB and a central MySql DB. The problem really comes when inserts are made, because it admits the possibility of replicated primary keys.
I did build a system a while back that used hardware and time-based GUID primary keys instead of simple, monotonically increasing primary keys. The theory is that if new records are inserted either on the handset using sqlite, or in the central server using mysql, the keys will never conflict because they are globally unique.
Unfortunately, this means you have get in the primary key management business instead of just taking the automatic one that comes with both DB engines. But in theory this approach allows data from many different handsets to be melded easily in the central database and redistributed back out to all the handsets. I am considering this approach for my Android problem although I'm still hoping to find a solution already crafted out there.
Related
I want to query data from 2 different database server using mysql. Is there a way to do that without having to create Federated database as Google Cloud Platform does not support Federated Engine.
Thanks!
In addition to #MontyPython's excellent response, there is a third, albeit a bit cumbersome, way to do this if by any chance you cannot use Federated Engine and you also cannot manage your databases replication.
Use an ETL tool to do the work
Back in the day, I faced a very similar problem: I had to join data from two separate database servers, neither of which I had any administrative access to. I ended up setting up Pentaho's ETL suite of tools to Extract data from both databases, Transform if (basically having Pentaho do a lot of work with both datasets) and Loading it on my very own local database engine where I ended up with exactly the merged and processed data I needed.
Be advised, this IS a lot of work (you have to "teach" your ETL tool what you need and depending on what tool you use, it may involve quite some coding) but once you're done, you can schedule the work to happen automatically at regular intervals so you always have your local processed/merged data readily accesible.
FWIW, I used Pentaho's community edition so free as in beer
You can achieve this in two ways, one you have already mentioned:
1. Use Federated Engine
You can see how it is done here - Join tables from two different server. This is a MySQL specific answer.
2. Set up Multi-source Replication on another server and query that server
You can easily set up Multi-source Replication using Replication channels
Check out their official documentation here - https://dev.mysql.com/doc/refman/8.0/en/replication-multi-source-tutorials.html
If you have an older version of MySQL where Replication channels are not available, you may use one of the many third-party replicators like Tungsten Replicator.
P.S. - There is no such thing in MySQL as a FDW in PostgreSQL. Joins across servers are easily possible in other database management systems but not in MySQL.
pre: my question is not how to generate this tables.
I just wonder, if there is a deeper reason that MySql does not offer a statement like this
CREATE TABLE my_fed_db.foobartab clone
ENGINE=FEDERATED
CONNECTION='my_remote[/localname]';
so all fields are the same definition (name, type, collation), the name of the table is the same and also DEFAULT CHARSET is the same.
Did I just not find the syntax, is it something with security? Or is it just not implemented?
I just scripted this, but I wonder if I am the only one that could use that, or - are there hidden problems with a full clone of definition, I just will run into later.
It's just not implemented.
I was told by a former engineering manager for MySQL that FEDERATED was developed specifically for one use case - to support high INSERT rates, like for an application that is logging high-traffic activity at a rate that a single MySQL instance cannot sustain.
The implementation was done in MySQL 5.0, around 2005. But it was labeled a "proof of concept" implementation (i.e. experimental), and it has not been developed much since then.
Basically it was created for a specific customer of MySQL, a telecom company as I was told.
I have never found a good use for the FEDERATED storage engine in its current implementation, that couldn't be handled better by sharding (coding your application to connect directly to multiple database server instances) or using a specialized proxy solution like ProxySQL.
Other solutions favored these days for high-rate INSERT workloads are the Elastic Stack, or time-series databases like InfluxDB, or data streaming platforms like Apache Kafka.
I have a table in MySQL that I need to join with a couple of tables in a different server. The catch is that these other tables are in Informix.
I could make it work by selecting the content of a MySQL table and creating a temp table in Informix with the selected data, but I think in this case it would be too costly.
Is there an optimal way to join MySQL tables with Informix tables?
I faced a similar problem a number of years ago while developing a Rails app that needed to draw data from both an Informix and a MySQL database. What I ended up doing was using of an ORM library that could connect to both databases, thereby abstracting away the fact that the data was coming from two different databases. Not sure if this will end up as a better technique than your proposed temp table solution. A quick google search also brought up this, which might be promising.
This can sometimes be solved in the database management system with a technique called federation. The idea is that you create virtual tables in one of the two systems that makes queries to the other database system on demand.
For both MySQL and MariaDB there is the FederatedX storage engine that unfortunately only works with other MySQL/MariaDB systems. This is a fork of the older, but as far as I know unmaintained, Federated storage engine.
Some might also consider migrating to MariaDB where you can use the CONNECT storage engine which contains an ODBC client.
What I ended up doing is manually (that is, from the php app) keeping in sync the mysql tables with their equivalents in informix, so I didn't need to change older code. This a temporary solution, given that the older system, which is using informix, is going to be replaced.
I am working with a client who is syncing between SQL Server and MySQL containing the exact same schema and data. We want to centralize that data into one database. Other then performance and maintainability issues, what else is bad about the original design?
You can create a linked server instance in SQL Server, with the MySQL instance.
Despite being completely proprietary, one of the nice connectivity features offered in SQL Server is the ability to query other servers through a Linked Server. Essentially, a linked server is a method of directly querying another RDBMS; this often happens through the use of an ODBC driver installed on the server.
Refer This article : step-by-step process SQL Server Linked Server to MySQL.
Providing you grant the MySQL user you connect on behalf of proper permissions, you can write to the MySQL instance accouding to you. So you can update stored procedures to do an additional step to insert records into MySQL.
Much easier solution is to use commercial application - Omega Sync from Spectral Core
Omega Sync can compare and synchronize both database schema and table data. You can even synchronize data of heterogeneous databases (for example, compare your local SQL Server database with a MySQL replica on your web site - and synchronize all the differences in just a few minutes).
on the otherhand I think you've already mentioned what possible problems you may encounter when synchronizing 2 db at the same time aside from this two I think it would be the resources. since there are different RDBMS working for the application they would also have a separate resources for each, like when I update a particular record of a user it still needs to check on which resource does it really exist, but I love to hear more from other people out there this is really an interesting topic to discuss. ;)
Is it possible create/configure MySQL for functionality like SQL Server's Linked Server?
If yes, would you please tell me how? I'm using MySQL 5.5.
MySQL's FEDERATED engine provides functionality similar to SQL Server's Linked Server (and Oracle's dblink) functionality, but doesn't support connecting to vendors other than MySQL. It's not clear from the question if you need the functionality to connect to vendors other than MySQL.
You might want to look into MySQL Proxy. This doesn't match the architecture of Linked Servers/dblink, but you can probably solve a similar set of problems that you would use Linked Servers/dblink to solve.
I am the developer of the MySQL Data Controller. Unfortunately, since we had lack of requests we have stopped development on it. The plugin was 100% functional with MySQL connecting to Oracle, MSSQL or MySQL.
Base on some requests, we had added back a blog and video about it :
http://www.acentera.com/mysql-datacontroller/
Regards,
Francis L.
Unfortunately you cannot link an entire MySQL database to another MySQL database like you can with MS SQL. However, you can link individual tables. A federated table is a local table you create that points to a table on another server.
You can run queries and stored procedures just like any other table. The two tables must have the same structure, except the Federated table uses a different database engine: Federated. If you make ANY changes to the structure of the remote table, you should re-create the local federated table.
The process is actually quite easy, here is an example: https://docs.oracle.com/cd/E17952_01/mysql-5.0-en/federated-use.html
In my experience, the time needed to create and implement this process is minimal, even when compared to linked servers. It should take you less than 30 minutes to get your first federated table working, after that its a 5 min process. Last item, when naming your federated table, I give it the same name as the remote table with a "federated_" in front, like federated_customer.
Also, store your federated table definitions as separate stored procedures so you can reuse them anytime you need to create the federated table again, AND so that other developers can see how you generated the federated table.
The MySQL Data Controller is a follow-on to the federated engine that allows connection to different database types, such as Microsoft SQL Server or Oracle. I am not sure how development is going, yet.
See: http://en.wikipedia.org/wiki/MySQL_DataController
or: https://launchpad.net/datacontroller