My organization is considering moving our current SQL Server RDS instance to an AWS Aurora instance. Our motivation is solely to cut down on costs. I have run some successful tests using the MySQL Workbench Database Migration tool to move the SQL Server db to an Aurora instance. The database is about 4GB, has about 100 tables, about a dozen views and stored procedures. I am already using a MySQL copy of the database for development on a local machine, so all SQL syntax differences are already handled.
Are there any serious downsides to this migration project? Anything we should consider before making the switch?
This is a really series step. You should consider a few key things while migrating:
Performance: for simple request Aurora (improved MySQL) can be faster than MS SQL, but:
a) MS SQL has smarter query analyzer and in order to take good Aurora (MySQL) performance you need to understand how it works very well. SQL Server can “excuse” a lot of developer things.
b) For such big databases SQL Server in Enterprise edition has a few great features: partitioning, data compressions, online rebuild indexes, always on. But Enterprise edition can be expensive enough. That’s true.
Development: MySQL (Aurora) syntax is really poor comparing to SQL Server. MySQL doesn’t support many many things like HierarchyID, recurrent CTE, included column in indexes, index filtering, change tracking, XML, JSON and so on. It even limites systax for nested queries a lot. Many thing you need to implement yourself. Also, SQL Server has more professional development tools like SQL Server Management Tool, SQL Profiler, and Tuning Adviser and so on.
Different implementations: Some things in Aurora work differently. For instance, unique index in Aurora allows having many null values but SQL Server doesn't allow this. And so on.
Priсe: For good performance for now you need to rent at least large Aurora instance which costs about $210 p/m + replica = about $420 - not so cheap. Amazon Aurora Pricing
So, I’d recommend calculating all pros and cons while migrating because you can reduce resource cost but spend additional time and affords (so money) on development and maintenance.
Related
I want to query data from 2 different database server using mysql. Is there a way to do that without having to create Federated database as Google Cloud Platform does not support Federated Engine.
Thanks!
In addition to #MontyPython's excellent response, there is a third, albeit a bit cumbersome, way to do this if by any chance you cannot use Federated Engine and you also cannot manage your databases replication.
Use an ETL tool to do the work
Back in the day, I faced a very similar problem: I had to join data from two separate database servers, neither of which I had any administrative access to. I ended up setting up Pentaho's ETL suite of tools to Extract data from both databases, Transform if (basically having Pentaho do a lot of work with both datasets) and Loading it on my very own local database engine where I ended up with exactly the merged and processed data I needed.
Be advised, this IS a lot of work (you have to "teach" your ETL tool what you need and depending on what tool you use, it may involve quite some coding) but once you're done, you can schedule the work to happen automatically at regular intervals so you always have your local processed/merged data readily accesible.
FWIW, I used Pentaho's community edition so free as in beer
You can achieve this in two ways, one you have already mentioned:
1. Use Federated Engine
You can see how it is done here - Join tables from two different server. This is a MySQL specific answer.
2. Set up Multi-source Replication on another server and query that server
You can easily set up Multi-source Replication using Replication channels
Check out their official documentation here - https://dev.mysql.com/doc/refman/8.0/en/replication-multi-source-tutorials.html
If you have an older version of MySQL where Replication channels are not available, you may use one of the many third-party replicators like Tungsten Replicator.
P.S. - There is no such thing in MySQL as a FDW in PostgreSQL. Joins across servers are easily possible in other database management systems but not in MySQL.
I am confused about the database I should choose, since the application I am doing will store and handle large amounts of data, I think SQL Server will be better than MySQL. Any recommendations?
Both platforms SQL Server and MySQL can handle small and large software projects, so you should anticipate similar performance from both, provided the database designer and programmer are familiar with the right way to optimize queries and code.
Structured query language (SQL) is the language of relational databases. Several database platforms use SQL, but a slight variation on it—each tend to have a slightly different syntax. Microsoft SQL and MySQL are two of the most common database platforms on the web. Once you go with one, it can be very difficult to switch to the other. That’s because the database platform you choose will end up being the core of your dynamic content moving forward. It stores, secures, and retrieves all the data for your applications.
It’s an important decision to make, and will likely hinge on a few things. If you’re having a hard time deciding which one is right for your project, here’s a look at some similarities and differences between the two, SQL and MySQL.
THE BASICS
Whether you want to store, retrieve or edit your data—the way dynamic websites and applications perform nearly every request a user makes—SQL is the language of choice for relational databases. On the surface, both Microsoft and MySQL look similar:
They both give you the ability to host several databases on one
server.
They use tables to store data.
They have primary and foreign key constraints.
They use indexes to sort data and speed up performance, and they both support desktop and web applications.
SQL Server is slightly older than MySQL. Microsoft SQL Server was introduced in 1989 and MySQL was introduced in 1995 as an open-source project. Since both of them have been in production for years, they both have a firm foothold in the market. MySQL runs on either Windows or Linux, typically as a part of a LAMP environment. SQL Server runs on Windows, and is usually a part of a Windows environment.
Both platforms handle small and large software projects, so you should anticipate similar performance from both, provided the database designer and programmer are familiar with the right way to optimize queries and code.
Microsoft SQL Server vs. MySQL: Similarities
Both Microsoft and MySQL are relational database platforms, so they have several similarities. Most developers specialize in either one or the other, because although they appear similar, the way they work in the underlying architecture is very different. Here are some similarities, which make it somewhat easy for a database developer to work on both platforms efficiently, even if they specialize in only one.
Scalability: Both platforms allow you to scale as your business grows. You can use both for small projects, however should these projects take off to an enterprise-level, they can still support millions of transactions a day.
High-performance: A database is your application’s backbone. It stores your all of data, so you need a database that can return data in less than a second. Both platforms can handle this type of high-performance speed.
Tables: Both platforms use the standard relational database table model to store data in rows and columns.
Keys: Both platforms use primary and foreign keys to establish relationships between tables.
Syntax: Syntax between the two database platforms are similar, although there are some minor differences across different CRUD (create, read, update, delete) statements.
Web-based popularity: Aside from Oracle, Microsoft SQL Server and MySQL are the most common databases used for web applications. When you sign up for hosting, you typically get a choice between MySQL databases or SQL Server.
Drivers: You can find connection drivers for almost any popular language on the web, so you can easily connect to both platforms without writing complex code.
MICROSOFT SQL SERVER VS. MYSQL: DIFFERENCES
While the two platforms are similar in the interface and basic relational database standards, they are two very different programs and operate differently. Most of the differences are in the way they operate in the background, and these differences are not seen by the average user. As long as the database performs well, it can be used with your project. However, it’s still important to know these differences because they’ll play a huge role in your developer’s choice platform.
Native compatibility: You can use either database with both Windows and Linux projects, but MySQL works natively with PHP and MSSQL is mainly used with .NET. It makes integration simpler if you stick with MySQL for PHP and MSSQL for Windows projects.
MyISAM and InnoDB: Both of these engines are configurations for MySQL and allow the developer to perform very different design and programming. With MSSQL, you create a database and don’t specify different engines.
Cost: SQL Server is generally expensive to run, because you need licenses for the server running the software. MySQL is free and open-source, but you’ll pay for support if you need it.
LINQ: With MSSQL, you can set up your entity framework classes in .NET and get started with LINQ queries. With MySQL and .NET, you need to download third-party provider tools.
IDE tools: Both platforms have IDE tools, but you need the right tool with the right server. MSSQL uses Management Studio and MySQL has Enterprise Manager. These tools let you connect to the server and manage settings and configurations for security, architecture, and table design.
WHICH ONE SHOULD YOU USE?
The database you use usually depends on the hosting environment you choose. Linux hosting providers usually offer MySQL. Since MySQL is open-source and free, you can have as many databases as you need. You can have 10 of them to support 10 different projects if need be.
Because SQL Server costs money for licenses, Windows hosts will give you one MSSQL database and you must pay for additional ones. Overall, this makes SQL Server more costly than MySQL. However, SQL Server works natively with .NET applications, so it’s the choice for software that runs on a Windows server or desktop. The development tools are free, but the production environment is not free.
The best way to determine the right platform is to first post your project in the marketplace and discuss your requirements with a few developers. You will get different opinions and preferences based on the developer’s area of expertise, but most Windows developers work with MSSQL and Linux developers work with MySQL. You should decide which environment you want to target, and then you can get a clearer idea of which platform is right for you.
Source
sql-vs-mysql-which-relational-database-is-right-for-you
Both Mysql and mssql can handle big data. but mysql is better in this ways
security: if You choose Mssql u'll host on windows server which is considered less secure.
NB: SQL Server 2017 available on Linux. but most hosting companys haven't updated it
Cost: hosting Companies will give little free database space on Mssql compared to mysql.
example in godaddy MSSQL 1 database x 200MB(economyPlan) 2 x 200MB(deluxe) Unlimited x 200MB(ultimate).
MySQL databases 10 x 1 GB(economy) 25 x 1 GB(deluxe) Unlimited x 1 GB(ultimate).
If your not sure, then you probably don't have alot of experience. I would suggest you start with MySQL assuming you have Linux available. It is much more forgiving and allows you to quickly re-start the database if you mess something up. SQL Server is almost impossible to stop and restart without rebooting the windows server, which is a long process and could bring your whole system down for 10 minutes or more. A Mysql restart in linux takes 2 seconds or less.
Sql Server is MUCH more expensive but it has many more tools and features.
If you are looking to use this as a backend to get more development work, SQL server DB's generally get more money, and it is used more in corporate world. Microsoft has extensive tools and addins with SQL server, much more than my sql.
It is interesting to note that SQL Server can connect to MySQl to allow you to write queries in SQL server that directly query Mysql (this is called a linked server) , but the reverse is not possible. So in the future, you could split up your databases. AS a general rule of thumb, if you have website traffic coming in AND backend processing, create multiple databases, even if they are on the same server, so you can move one of them off to another server when your the database gets busy.
I need to improve a PHP-MySQL web application, which only uses MySQL for REPL operations (and some search functions). 99% of the applications that I worked with never used advanced MySQL features, like replication, cross-table constraints, locking etc.
To my understanding I should instead use SQLite.
Are there any practical benefits if I do this?
Will I see a significant (>100ms) speed boost?
Should I expect problems with tables with more than 1,000,000 rows?
There is no catch-all answer to that, but there is a main point to consider: A very good rule of thumb is, that the higher your degree of concurrency is, the more you'll profit from MySQL and vice versa.
This means that in a scenario where database requests never ever are concurrent, you might see a speedup by using SQlite, though I doubt it would be in the 100ms order of magnitude.
The reason behind this is (very roughly):
In a database server environment, such as MySQL, PostgreSQL, MS SQL, Oracle and friends, a dedicated process (or a group of processes) exclusively touch the database files - the important part being dedicated. This means, that concurrency issues can be resolved in-process.
In a file-based database, such as SQlite, MS Access (Jet Engine) and friends, multiple processes will touch the DB files without knowing of each other - this implies that concurrency issues have to be resolved by writing them to the DB or helper file(s). This is typically much slower and less robust. In exchange for that, the overhead of communication between the database client (the web app) and the database server (which is in-process) is nonexistent.
Edit
After comment I want to make it more clear, that I am talking of concurrent writes, not concurrent reads. Concurrent reads of an unchanging dataset is not a hard problem - it doesn't need any locking at all.
The principal advantage of SQLite is that it is a file-based relational database that uses SQL as its query language. Being file-based tremendously simplifies deployment, making it very good for the case where an application needs a little database but must be run in an environment where having a database server would be problematic. (For example, many browsers use SQLite to manage their cookie stores; using a database server for that problem would be verging on the insane in many ways.)
The principal advantage of MySQL (with a sane table type) is that it is a database server that uses SQL as its query language. Being server-based allows for many features that a file-based system can't handle simply (such as replication) but does make things quite a bit more complex to deploy.
Whether the benefits of the additional complexity of a database server (e.g., MySQL) outweigh the costs (relative to a file-based database engine like SQLite) depends on a great many factors, notably including how many installations are expected and who is expected to perform those installations.
I am working with a client who is syncing between SQL Server and MySQL containing the exact same schema and data. We want to centralize that data into one database. Other then performance and maintainability issues, what else is bad about the original design?
You can create a linked server instance in SQL Server, with the MySQL instance.
Despite being completely proprietary, one of the nice connectivity features offered in SQL Server is the ability to query other servers through a Linked Server. Essentially, a linked server is a method of directly querying another RDBMS; this often happens through the use of an ODBC driver installed on the server.
Refer This article : step-by-step process SQL Server Linked Server to MySQL.
Providing you grant the MySQL user you connect on behalf of proper permissions, you can write to the MySQL instance accouding to you. So you can update stored procedures to do an additional step to insert records into MySQL.
Much easier solution is to use commercial application - Omega Sync from Spectral Core
Omega Sync can compare and synchronize both database schema and table data. You can even synchronize data of heterogeneous databases (for example, compare your local SQL Server database with a MySQL replica on your web site - and synchronize all the differences in just a few minutes).
on the otherhand I think you've already mentioned what possible problems you may encounter when synchronizing 2 db at the same time aside from this two I think it would be the resources. since there are different RDBMS working for the application they would also have a separate resources for each, like when I update a particular record of a user it still needs to check on which resource does it really exist, but I love to hear more from other people out there this is really an interesting topic to discuss. ;)
I've been trying to specialize into the MS SQL server technologies, but my job has required me to understand more MySQL recently. Has any MS SQL DBA or developer had to do this, and if so, what were their biggest challenges? At the moment, for me:
Lack of decent IDE
No easy ETL tools like ssis
As an IDE for designing MySQL DBs I'd suggest MySQL Workbench.
I think one of the biggest problems migrating from MSSql to MySQL is saying good bye to transactions and hello to autocommit.
Here's a little comparison between MySQL 5.0 and MSSql 2005 (a little outdated...)
and here's what MySQl says...
The biggest difference between the two technologies in my opinion is that the database engines that are under the hood are quite fundamentally different.
This has implications, particularly for enterprise class systems, as the performance tuning techniques and best practice design principals that you use to optimise and design a SQL Server platform will not necessarily translate to MySQL.
If you already have a solid understand of the SQL Server database engine then I would look to develop this level of understanding with MySQL, starting off by looking at the different database drivers that can be used to drive the MySQL Engine such as INODB and MyIsam etc.
The fundamentals of relational database management of course remain the same, i.e. you are working with tables, indexes and keys etc. so you are not starting from the absolute beginning here. If you are familiar with using T-SQL to navigate your database structures and tables then porting this to a MySQL command prompt is quite a smooth transition.
You may find GUI tools such as Navicat to be to your liking if you frequently use SQL Server Management Studio (SSMS) to administer your databases.
To my knowledge, there is no integrated ETL platform such as SSIS to compliment MySQL, as it focuses solely on the core database engine. Microsoft SQL Server is considered an Enterprise database suite and so incorporates additional components that complement the database engine, such as Reporting Services, Analysis Services, SSIS and Notification Services etc.
I hope this helps you on your quest to work with a new technology. Learning something new is a great problem to have.
Cheers, John
The last I checked there were no UNIQUEIDENTIFIER data type in MySql. That could become a hard part in migration since lots of applications are written based on vast use of Guids.
I'm not following of the development of MySql, could be that this datatype is already there.