Coming for using Sql Server where there are file-groups, i was wondering if there is (i'm sure there is) something similar in MySQL. After all the database cant be limited to just one hard drive( if using windows that is). I've tried to search but its hard to find the something that you don't know the name of!.
By default a MySQL Server instance stores all data files under a single data directory.
There are ways to use symbolic links to spread files over multiple drives if you're clever.
Or you can use Partitioning to store your database over multiple directories. But I personally think the implementation of Partitioning prior to MySQL 5.5 is too awkward to be useful.
Re your comment about using a SAN: Here's an article: When would you use a SAN with MySQL?
Related
I just moved my Laravel project to aws Elastic Beanstalk, after moving I moved my local databse to aws RDS using Mysql Workbench. All worked fine. But mysql workbench converted all tablenames and column names to lower case. ( I used CamelCase for tables and column in my local mysql and coding). Now since linux is case sensitive its shwoing me tons of errors which says table names dont found etc. It is just because I used camelcase in my codig.
Now its just totally blowing my mind away, I am already late on launching date and this now this issue..ughh. Do I have to change all the tablenames, column names etc in my coding to lower case? That would be totally inefficient and ot would require a lot of time obviously. Is there anyway around to solve this issue? Do I have to use other tool like DBeaver or something? Please help. I just need to upload my mysqlDatabase to aws RDS in same camelCase that I have in local environment. Thanks Again.
OK So I have found a solution. You dont have to change all the occurrences of tables and column names in your laravel code. You just have to change tableNames to all lowercase in following files.
controllers
views (if you used queries to access data in your views)
models
We are finally moving from Excel and .csv files to databases. Currently, most of my Tableau files are connected to large .csv files (.twbx).
Is there any performance differences between PostgreSQL and MySQL in Tableau? Which would you choose if you were starting from scratch?
Right now, I am using pandas to join files together and creating a new .csv file based on the join.(Example, I take a 10mil row file and drop duplicates and create a primary key, then I join it with the same key on a 5mil row file, then I export the new 'Consolidated' file to .csv and connect Tableau to it. Sometimes the joins are complicated involving dates or times and several columns).
I assume I can create a view in a database and then connect to that view rather than creating a separate file, correct? Each of my files could instead be a separate table which should save space and allow me to query dates rather than reading the whole file into memory with pandas.
Some of the people using the RDMS would be completely new to databases in general (dashboards here are just Excel files, no normalization, formulas in the raw data sheet, etc.. it's a mess) so hopefully either choice has some good documentation to lesson the learning curve (inserting new data and selecting data mainly, not the actual database design).
Both will work fine with Tableau. In fact, Tableau's internal data engine is based on Postgres.
Between the two, I think Postgres is more suitable for a central data warehouse. MySQL doesn’t allow certain SQL methods such as Common Table Expressions and Window Functions.
Also, if you’re already using Pandas, Postgres has a built-in Python extension called PL/Python.
However, if you’re looking to store a small amount of data and get to it really fast without using advanced SQL, MySQL would be a fine choice but Postgres will give you a few more options moving forward.
As stated, either database will work and Tableau is basically agnostic to the type of database that you use. Check out https://www.tableau.com/products/techspecs for a full list of all native (inbuilt & optimized) connections that Tableau Server and Desktop offer. But, if your database isn't on that list you can always connect over ODBC.
Personally, I prefer postgres over mysql (I find it really easy to use psycopg2 to write to postgres from python), but mileage will vary.
The old versions of our product allowed to capture the current state of the system in a single archive file, which also contains the MySQL database files - lots of <XXX.frm, XXX.myd, XXX.myi> triples .
Now we have the next generation of the product, which does not do anything stupid like capturing the database files, but it must know to read the archives produced by the old versions.
Our product is a commercial closed source product, but it is not very expensive. We had to stop using MySQL, because of the second reason (Oracle has changed the MySQL licensing) and we cannot use MariaDB, because of the first one (their licensing freaked the s*t out of the company lawer).
So, my question is there another way to read these MySQL database files? A commercial light weight solution is fine - after all, we are talking about read-only exploration of the database files. Free/Open Source alternatives are welcome too, as long as they do not mean that the code using them must be Open Source too.
Thanks.
EDIT
Besides the issue whether I can or cannot continue using the old version of MySql to read the old MySql database files, the question remains how can I read them? I mean, MySql is no longer our database, so even if I can bundle with the old MySql implementation, do I have to install the full blown database engine to just read the files? I'd rather avoid that.
If you want to go thru tables structure it would be enough to read the following links.
MySQL internals (all), File Format, MyISAM
If it is not enough and you database size less then 10G you can use Ms SQL Server Express (which is free with DB less than 10G. Page to compare different versions of Ms SQL Server is here). Search for the way to convert MySQL files to Ms SQL Server. Here is the first link a got from Bing: link1 (I suppose not all of them need MySQL server)
If it is not suitable. You can try another MySQL forks like: XtraDB, OurDelta, Drizzle, PBX and so on.
Hope you will find something useful.
We have found a solution. Unfortunately, it involves MySQL, so there are potential licensing issues. Here it is - http://dev.mysql.com/doc/refman/5.0/en/libmysqld.html
All it takes is download the MySql source code and help yourself with:
libmysqld.dll
libmysqld.lib
header files from the include folder
Then it is possible to read the files using the embedded MySQL database engine inside libmysqld.dll.
Is there a way to use a MYSQL database without the database management system.. Like use tables offline without installing the db management system on the machine..
If there is can you please point me in the right direction?
Thank you!
As far as I know, there is no way to do this.
However, there is a portable DBMS SQLIte. It comes in different ways and can be used on other platform with different programming languages.
After reading your comment, I'm almost sure, this is what you need.
It's not that fast as MySQL I guess, but it works.
You can use The embedded MySQL Server Library to access MySQL data files without running the MySQL server.
You can setup a database to work on your localhost. This will be offline unless you setup the front-end stuff to let the internet interact with it.
What exactly do you mean "without the database management system"? You always need a way of interacting with it, even if it is offline. (Otherwise how can it work for you?)
The server side piece of the application, mysql-server, is needed at a minumum to run mysql. This server application comes with all the tools built-in to manage the instance. I doubt you can prevent installation of this.
If you've actually opened the table files in a hex or text editor, you'll see that you will definitely need the mysql application installed to make any sense of them to use them. Sure the records are all there in plain text (.myd files for myisam, the ibdata1 file for innodb tables), but it would be a complete time-waster devising a custom app to parse or update the file structure, as well as trying to tie in table structure contained in the related files for each table.
I am going to be writing to a MySQLite database file, using Perl's DBD:SQLite module, and I wondering if it is possible for this file to be read by any distribution of MySQL? Is there a better way to create a simple MySQL database (using Perl)?
If it means anything, I'm only going to be using the database to store key-value pairs based on unique ID numbers for the keys. I tried BerkeleyDB but there is little support for it on Perl and I could not get it to work correctly in the past on certain versions of Windows.
Edit: I am aware that BerkeleyDB is a better way to do this, but when I was writing scripts for it, most of the methods have TODO, and I've had mysterious bugs on Windows Server 2003 using the same airtight code that ran for 2 weeks straight on my Win7 machine at home.
MySQL and SQLite are completely separate RDBMS systems. There is no such thing as MySQLite. To the best of my knowledge, MySQL cannot read SQLite databases.
If all you really want is a key-value store, perhaps look at Redis: http://code.google.com/p/redis/
I use Perl's DBI module which I can use to read databases using either MySQL or SQLite. All you need is the correct driver. In fact, if you write your program correctly, the backend database (either SQLite or MySql) is irrelevant. Your program will work with either one.
However, you can't use a SQLite database and then treat it as a MySQL database. They're two different creatures. Your program can be database agnostic, but once you chose a database, you can't switch back and forth. It'd be like opening an Oracle database as a MySQL database.
See This posting on Perl Monks for more info.
BerkeleyDB is well supported by perl. You have a choice between the older DB_File and the more fully featured BerkeleyDB module.
But there are tons of choices. If you don't want to have to run a separate server process, use DBI and DBD::SQLite or BerkeleyDB or any of the AnyDBM_File modules. For a simple server-based key-value store, there's redis or the older memcached.