I'm developing a application that needs two diferent databases, this is because one of this databases is per client and the other one is a generic database.
I'm thinking in make a MySQL NDB Cluster and i need to know if it's possible to uses some Federated tables in the Cluster or all must use ndbcluster engine.
If this is not posible, how can i make joins with tables that are in other host using ndbcluster?
Please forget about, why i need this schema (one databse per client and one generic database) because i spent a lot of time thinking which should be the best schema for my application i choosed this one.
Thanks in for your help!!!
MySQL Cluster uses a full version of mysqld (slightly modified), which includes all storage engines included on a standalone version. So the question for your answer is YES, you can have some tables in FEDERATED storage engine, or any other storage engine.
However, only tables with storage engine=ndbcluster will be replicated to all the api nodes connected to the cluster.
The federated approach with a cluster of two api nodes could work, but keep in mind that only those tables with the same storage engine can have referential integrity (FK) between them.
Which version of MySQL Cluster are you using? It is recommended using always the latest GA release (now 7.4.12)
Regards
Related
In our project we are planning to introduce MySQL NDB cluster to have 99% uptime for our multiple applications dependent on MySQL.
So MySQL is being deployed in two machines. In both the machines Data Node, Management Server and SQL node is deployed and configured to form a cluster as shown in the below snippet.
Based on my understanding replication of data will be done for the data stored in Data Nodes. But can we restrict the replication only to a set of tables or database?
Reason for this query is, there are two applications that are dependent on MySQL, where only one application needs this replication and the other doesn't need this feature because it should connect to a standalone instance of MySQL to store it's local data which shouldn't be replicated as it would cause problem to the application running in another machine.
Please share your thoughts on this.
Though we have deployed MySQL NDB Cluster, the mysqld still supports INNODB storage type. So in order to achieve the above requested need, we created tables explicitly with Storage Engine by mentioning in CREATE TABLE statement like below. This overrode the storage configuration mentioned in my.cnf.
CREATE TABLE IF NOT EXISTS `CDS` (
`CD_ID` bigint(20) NOT NULL,
PRIMARY KEY (`CD_ID`)
) ENGINE=innodb
We have the requirement to synchronize the data in a table across multiple MySQL databases. One of the databases would be the source, and all others need to have the synchronized data for one of the tables.
We have multiple databases used in microservice architecture, and they all need to have a local copy of a specific table in their database, and not the entire database itself, hence read replica or multi-AZ configuration is not the solution.
Database: MySQL hosted on AWS RDS
Is there any managed service by AWS or another vendor that can be used to accomplish this? Or do we have to write a custom script to do that?
It's a simple MySQL replication. But you have to replicate the whole database. So create a MySQL database in AWS and enable Multi-AZ replication and activate the automatic snapshots.
The A-Z Replication is synchronous. When you use a "Read replica" it's asynchronous. So if you have very important data you should enable Multi-AZ replication.
https://aws.amazon.com/rds/details/multi-az/?nc1=h_ls
I think you have to either isolate the table in one database and replicate that database or write a custom script.
If I were writing a custom script I would look at the binlog functionality. Here are some helpful links:
https://dev.mysql.com/doc/refman/5.7/en/mysqlbinlog.html
https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_LogAccess.Concepts.MySQL.html (at the bottom).
I am designing a data replication solution across timezones and have run into the issue where I can only run and old version of MySQL (5.6) in one location, whereas the other two have MariaDB 10.2.
Now, I have read the information about Replication Compatibility over at MariaDB. Clearly I can't use MariaDB as a master and MySQL as a slave.
Intermediate solution based on Bash scripts
Yet, I want to use my EU server as a master and that is running MariaDB. So I'm now contemplating a way around the limitation. So far I have come up with an intermediate data storage solution in the overseas server, where data is shuffled periodically using my own Bash data migration scripts over a low bandwidth link.
MariaDB is required in the primary location
I have to use MariaDB in my primary location because I'm using the ColumnStore database there. That is unconditional as part of the application design.
Does this situation ring a bell?
Do you have experience with similar situations and would you mind sharing some inspiration as for how you did it?
My best solution so far is with Bash scripts that are cronned, where MySQL data is dumped (mysqldump) and transferred over a low bandwidth link, then merged with the master (ColumnStore storage engine). I'm looking at a T+1 data lag between my primary location and the secondary location that is running MySQL.
Any high level design thoughts or shared experience is highly appreciated.
Best regards
I have two tables named user and user_posts.These two tables are in different amazone rds instance. I wants to join these two tables. Is it possible to write mysql join using laravel framework?
Thanks in advance!
This isn't possible with RDS.
MySQL has a FEDERATED storage engine that allows one server to access tables on another server, but it is disabled in RDS for MySQL.
The Federated Storage Engine is currently not supported by Amazon RDS for MySQL.
— http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_MySQL.html
Clarificarion:
When using the FEDERATED storage engine to allow tables with data on one server to appear to exist on another server -- thus allowing the data to be queried from a server that does not actually have a persistent copy of the data -- only the server that does not have the actual data needs to actually support the FEDERATED engine.
This means that while an RDS instance can't access the data from another server using FEDERATED, a non-RDS MySQL server can access data on an RDS instance using a FEDERATED table configured on the non-RDS server, with the table configured to retrieve data from RDS.
This is because -- from the perspective of the server with the actual data -- the connection from the server using FEDERATED looks like an ordinary client connection. The data is retrieved using normal queries, so FEDERATED support does not need to be available on that side of the link.
This means that a non-RDS server running MySQL can access the data on one or more RDS servers, using FEDERATED tables.
I use this routinely for generating reports that join tables on two (and in one case, three) different RDS instances.
FEDERATED tables do have limitations -- they appear to the server that is fetching the remote data as being very similar to MyISAM tables, in the sense that they do not support transacrions and any query that would result in a full table scan will actually fetch the entire remote table for each query, which can obvioisly get out of hand... so they have to be used with diligence and discretion.
This may not be useful for the scenario described, since it would require the addition of a third server, but it is a solution that is not completely ruled out when RDS makes up a portion of the database infrastructure. The information above is also true when using RDS/MariaDB and RDS/Aurora for MySQL.
Currently we have a website running on a Centos 6.5 webserver with Direct Admin. APC was configured in the past and is working nicely, but some reading suggested to also implement memcached to cache some static tables (like for instance menu's).
As of MySQL 5.6, the innoDB tables are compatible with a mysql memcache deamon, so I started off following this guide: http://dev.mysql.com/doc/refman/5.6/en/innodb-memcached-installing.html.
The config script is ran and the deamon is installed.
However the Drupal memcache plugin does not see memcache is running. That module is obviously checking for the php memcached deamon, while my deamon is already running in mysql.
Since both extensions are called memcached.so, we strongly have the feeling this is the same thing. Are there actually two different things and does Drupal not support the InnoDB memcached deamon, do I need both (php extension to access the mysql extension?), or was this supposed to be working and did we something wrong?
Update
The status report showed something like "not running", but one of my colleagues has installed the PHP PECL extension now and it seems to be working. But then I still don't understand what the MySQL innodb deamon plugin does. Is it not needed, or does it improve database access even more then the PHP extension would?
The Memcached interface to InnoDB is a feature of MySQL to support the memcached protocol, yet with InnoDB as the back-end storage. It seems like lot of people have been confused by what this means, so I'll try to explain.
Whereas a real memcached daemon stores data in memory, MySQL stores data persistently in an InnoDB table. PHP applications can read and write data using the memcached extension, as if they're using a standard memcached in-memory store. However, they are really reading and writing rows from the InnoDB table.
This is somewhat slower than standard memcached, because it has the overhead of writing to disk is greater than accessing memory. But it's somewhat faster than using SQL to read and write those rows, because it skips the complexity of the SQL parser and query optimizer.
That's really the new feature in MySQL: to bypass SQL, and give access directly to the InnoDB storage engine through a simple, but familiar interface. They chose memcached on the theory that many developers would be familiar with it and have tools and language support for it already.
The InnoDB memcached interface is similar to the earlier experimental plugin called HandlerSocket developed in 2010. http://yoshinorimatsunobu.blogspot.com/2010/10/using-mysql-as-nosql-story-for.html
Here's a Percona blog that shows tests of the relative speeds of InnoDB memcached versus SQL queries: http://www.percona.com/blog/2013/03/29/mysql-5-6-innodb-memcached-plugin-as-a-caching-layer/
Re your question in comments:
You might be misunderstanding. The data is never in Memcached. There is no automatic synchronization between MySQL and Memcached. The only thing is that MySQL is mimicking the API and protocol of Memcached. There's no reason that they did this, except to make the API familiar to developers.
When you use the "Memcached API for InnoDB" you're connecting your application to a port listened to by the mysqld daemon process. Your requests on this connections read and write rows directly in the InnoDB storage engine. There's no Memcached instance in between.
The InnoDB Memcached Plugin is a feature in MySQL >= 5.6 that runs a Memcached daemon in the existing mysqld process supporting the Memcached API listening on a different port (11211 by default). Because it runs in the same process space as InnoDB, you get low-latency access to data stored in InnoDB tables and through existing, widely-available Memcached clients.
This has a few interesting use-cases:
Transparent support for adding InnoDB as a high-performance persistence layer to existing memcached clients/applications
Improved performance over SQL queries for interacting with InnoDB tables directly (especially for inserting new key/value pairs), since the simple Memcached protocol bypasses the overhead of SQL parsing and query-plan optimization
High-performance 'MySQL + caching layer' architecture, where the memcached daemon fetches data from from the underlying InnoDB table and serves cached data directly from local memory
Case #3 is particularly noteworthy: Although the Memcached plugin is configured by default to read/write directly to the underlying InnoDB table (innodb_only cache policy), it can be configured to use its own, separate local memory cache just like a standalone Memcached instance would, either without using InnoDB storage at all (cache-only), or using InnoDB as a backing store (caching). (Note that the currently-accepted answer is incorrect on this point.)
Refer to the architecture diagram from the documentation (note the 'local cache (optional)' component, which is used by the cache-only or caching cache policies):
(source: mysql.com)
In terms of setup/installation, if you're using the Memcached plugin for anything other than a standalone key-value cache (case #1), you will need to create a mapping from Memcached keys/values to your InnoDB tables/columns by writing a row to the special innodb_memcache.containers table. See Creating a New Table and Column Mapping for details.
A bit of a side-track here, but nowadays you're better of using the memcache_storage module. The module page has plenty of good pointers how to use the module and with what other modules it integrates nicely to have a better and faster caching for your site.