To extent my understanding on MySQL.
1) Is going for 64bit help? Do I go with installation or stand alone?
2) If I am going to use 64-bit in MySQL Community Service, will it affect the MySQL Workbench which only available in 32-bit? If yes, in what sense?
3) Does the size of the file affect the speed of MySQL when doing calculation, analysis, and graph? I get a lot of big trouble using Excel in big size.
Please guide.
Note: I am working on excel database (which I found out on this website that it is not a good way to do it) with size of 1.34GB 63 files and growing. I would like to change and continue working in MySQL. I need the database for calculation, analysis and graph.
You are confusing a few concepts. But let me first answer your questions.
1/ 64-bit is more powerful than 32-bit, if your server supports 64-bit. So yes, it should "help".
2/ You can use a 64-bit server with a 32-bit client without any problems. They just communicate with eachother using MySQL-statements, bitsize makes no difference there.
3/ The size of the file (table/row/...) does matter. A lot of small tables can outperform one big table in certain cases, especially with multicore/raid/...
Related
At our company we are trying to migrate data from an old Local SQL Server database to a RDS MySql database using SSIS. The original database is roughly 4GB in size and we are required to do the migration without taking down the production servers. The dev team reports that the migration runs fine with data being transferred, but after several hours (roughly 8 hours, but it's not exact. Sometimes it's less sometimes it's more) the connection abruptly closes. We have tried everything we can possibly think of on our side but we don't know what else could be going wrong. Based on their tests and ours, we think it could be the instance is closing the connection after being open for too long. Does anyone know what could be causing this?.
We need another alternative tool to make the migration and the target databases remains fully operational during the process?
I recommend you try the MySQL workbench 6.3 that oracle has out which has a piece precisely designed for your purpose. It is under GNU license so they have a community version which is free. There is also Data Loader which has a free trial version. The standard version is only $99. You can use logical export and convert it, so there will be no down time. GoldenGate would be perfect, but it is crazy expensive. I know people who have used Kettle to do what you are doing. Kettle is open source but you will have to write transforms so it will be a bit more tedious. With SqlServer you can clone the database, and then use the cloned version to do whatever you need to do to get it converted to MySql, bring it down, whatever, while the original stays up.
Cheers
Why cannot a 4GB database be brought down for a bit? And why would a 4GB database take 8 hours using SSIS ? I commonly move terabytes around in less time than that. That is in an Oracle shop, but still...
I have an application that I will soon be doing a significant rewrite of that stores a significant quantity of transient blobs in a database. The application will insert and delete a large number of blobs (up to 5mb in size each) during the course of each day. Currently the application uses a version of PostgreSQL that is very old (7.3.x). With this version of PostgreSQL we had to routinely run the external vacuum process to keep the database size under control, additionally this process required the application to be shutdown to function correctly.
We were looking at either upgrading to the newest PostgreSQL or migrating to another database. Specifically we were interested in moving to MySQL. I was wondering if anyone here was familiar with the blob handling support of the newest versions of these servers, and provide any suggestions on which one might perform best for an application that will constantly be inserting and removing blobs. Other feature differences between the two servers aren't an issue for us.
I did some research and found plenty of feature comparisons between MySQL and PostgreSQL, but nothing that really addressed this issue. I'm hoping someone here might have some experience with this aspect of one or both database systems.
Thanks
Postgres 7.x indeed was a major PITA when it comes to vacuuming. 9.0 is a lot better in this area. The autovacuum daemon can be configured on a per-table level since I think 8.3 and for the described scenario you would probably make it very aggressive for that table (or tables if more than one is involved).
I don't think it matters whether you delete rows with BLOBs (i.e bytea) column or not. Especially because the blobs are stored out-of-line anyway (you might need to configure the auto-vaccuum daemon for the so called TOAST table as well, but I'm not sure)
The question is rather how many rows (in percent of the total rows) you delete/update in the table, rather than how big each blob is.
As much as I like PostgreSQL I do have to admit that the whole vacuum topic (even though getting easier and easier which each release) is still one of its weakest points (and the source of a lot of trouble).
I can't say anything about MySQL as I have never used in a production environment. In contrast to you the other features (beside blobs) are important enough for me to stay away from MySQL - and if it's only for the license.
I'm building a web application right now for my company and for whatever reason, can't get mySQL working on my Mac w/ my Ruby install (OSX 10.5). SQLite works fine though, so would it be a problem to use SQLite for now so I can get to work on this and then just change up my database.yml file to point to a mySQL database when I deploy (assuming I rerun migrations and such)?
Also, what are the benefits/drawbacks of using mySQL over SQLite in a RoR application? I've always used mySQL by default in the past, but never learned SQL directly (always through ActiveRecord) and never thought too much about the difference.
Benefits of MySQL/PostrgreSQL/etc
Pros
Stronger data typing, which means cleaner data
Ability to store more data
Scale better to larger data sets
Spatial support (think GPS)
Full Text Search (FTS)
Cons
Stronger data typing means data will be validated, bad data will cause errors
Not a good candidate (if even possible) for devices with limited resources (iPhone, Blackberry, iPad, etc)
I would pick PostgreSQL v8.4+ over MySQL given the choice. MySQL's features lag behind the rest of the major SQL database alternatives.
THe biggest performance issue you may run into is table locks. SQLite unfortunately does not have row level locking. So if your app is going to run multiple processes / threads (as with multiple web users) its likely some threads will not be able to perform an SQL op. For this reason i would go with MySQL - or perhaps Postgresql.
Should be no problems, as MySQL should have a superset of SQLite capabilities, and as #Sean pointed out, performance should only increase. Just try to make sure you're not using anything too SQLite specific (I'm mainly a SQL Server and Oracle guy, so don't know what that would be, if anything). Remember, the "S" in SQL stands for Structured, not Standard ;)
Paul.
SQLite is perfect for a desktop or smartphone application ("embedded" usage). However, if you plan to build a web-application, you are highly encouraged to make use of a non-embedded DMS like MySQL. The benefits are countless, such as 3rd party design and analysis apps, performance etc ...
Currently have approximately 2000 simultaneouse connections. We average approximately 425 reads and writes per second. We have a read to write ration of 3:1. All of our tables are myisam. Can we expect better or worse performance when we go from mysql 4.1.22 to 5.0?
There's no way for anyone here to tell you without the schema, queries and test data.
Why not setup a dev environment on 5.0 and testing it out?
The main concern should be that the 5.0 Information Schemas, are a HUGE vulnerability and can be used to very easily gain access to the SQL server from remote locations simply by printing off the schema using injection will let an unwanted viewer, view all of the tables and capitalize off the knowledge to get passwords using the same schema for its columns.
The MySQL source tree includes a set of benchmark tests written as Perl scripts. See The MySQL Benchmark Suite for some information. You can download the source distribution for MySQL 5.0.91 at the archives.
Source distribution of MySQL 4.1 doesn't seem to be easily available anymore. You might have to check it old sources from LaunchPad unless you can find a copy of an old source distribution elsewhere on the internet.
However, the comparison that these benchmarks show is only of general interest. It may be irrelevant to how your application performs. For instance, your usage of the database may not take advantage of some performance improvements in MySQL 5.0, but it may run into some regressions in MySQL 5.0 that were necessary.
The only way to get an answer that is relevant to your application is to try the new software with a test instance of your application, using a sample of data that is a realistic model of the type and volume of data your application typically deals with. As #BenS says, no one on a site like StackOverflow can give an answer specific to your application.
You say in a comment that you're very concerned about performance, but if you don't have an instance of your application and database that you can run tests on, you aren't doing the work necessary to satisfy this concern.
I would strongly suggest moving straight to 5.1.45 with Innodb Support. Percona provides an excellent version with XtraDB that provides a number of performance related improvements. Moving off of your MyISAM tables and onto Innodb will provide a huge performance increase in almost all cases. If you are going to burn the QA/Testing time to move, do a full move now, not a half-way step.
We had an applicationg running using MySql. We found MySql was not suitable for our app after we found that it didnt support some of the GIS capability that PostGIS has (note: mysql only supports minimum-bounding rectangle GIS search).
So we changed our DB to PostgreSQL. We then found out that Postgresql 8.2 running on Windows is so much slower compared to Mysql 5.1. By slower, I mean at roughly 4-5 times slower.
Why is this? Is there something in the configuration that we need to change?
I found some comments from other websites such as this:
UPDATE: We found that the cause of the slowness is due to the BLOB that we are inserting into the DB. We need to be able to insert BLOB at a sustained rate of 10-15 MB/s. We are using libpq's lo_read and lo_write for each BLOB we are inserting/reading. Is that the best way? Has anyone used Pgsql for inserting large BLOB at a high rate before?
EDIT: I heard that PgSql just recently got ported to Windows. Could this be one of the reasons?
There are cases where PostgreSQL on Windows pays an additional overhead compared to other solutions, due to tradeoffs made when we ported it.
For example, PostgreSQL uses a process per connection, MySQL uses a thread. On Unix, this is usually not a noticeable performance difference, but on Windows creating new processes is very expensive (due to the lack of the fork() system call). For this reason, using persistent connections or a connection pooler is much more important on Windows when using PostgreSQL.
Another issue I've seen is that early PostgreSQL on Windows will by default make sure that it's writes are going through the write cache - even if it's battery backed. AFAIK, MySQL does not do this, and it will greatly affect write performance. Now, this is actually required if you have a non-safe hardware, such as a cheap drive. But if you have a battery-backed write cache, you want to change this to regular fsync. Modern versions of PostgreSQL (certainly 8.3) will default to open_datasync instead, which should remove this difference.
You also mention nothing about how you have tuned the configuration of the database. By default, the configuration file shipped with PostgreSQL is very conservative. If you haven't changed anything there, you definitely need to take a look at it. There is some tuning advice available on the PostgreSQL wiki.
To give any more details, you will have to provide a lot more details about exactly what runs slow, and how you have tuned your database. I'd suggest an email to the pgsql-general mailinglist.
While the Windows port of PostgreSQL is relatively recent, my understanding is that it performs about as well as the other versions. But it's definitely a port; almost all developers work primarily or exclusively on Unix/Linux/BSD.
You really shouldn't be running 8.2 on Windows. In my opinion, 8.3 was the first Windows release that was truly production-ready; 8.4 is better yet. 8.2 is rather out of date anyway, and you'll reap several benefits if you can manage to upgrade.
Another thing to consider is tuning. PostgreSQL requires more tuning than MySQL to achieve optimal performance. You may want to consider posting to one of the mailing lists for help with more than basic tweaking.
PostgreSQL is already slower than MySQL up to a certain point (it is actually faster when you have a ridiculously large database). Just FYI, this isn't causing your problem but keep that in mind.