Can I store data files (e.g. txt files) to the MySql server? If I can, how to store them?
You can use LOAD DATA INFILE to read the contents of a file and store it in a table in the database in a structured format.
This can be considerably faster than reading and parsing the file on the client and then using multiple INSERT statements.
Example:
LOAD DATA INFILE 'data.txt' INTO TABLE db2.my_table;
Yes you can, but you would probably be better off storing them on the file system and storing a path to the file in the DB.
There are a few SO Posts that discuss this:
Storing Images in DB - Yea or Nay?
Storing a file in a database as opposed to the file system?
Sure you can.
I would suggest reading the data from your files and then saving it in your database, i would say in a text field.
You can use something like this to get the file's content:
$file = file_get_contents('./yourfile.txt');
Then insert it
Insert into myTable VALUES (mytextfile = $file)
How much text are we talking about here? If there's not that much text, you can store just the content of the file in the database.
You can also store the file itself as a BLOB. http://dev.mysql.com/doc/refman/5.5/en/blob.html
If you'll be dealing with many files, you'll probably be better off storing the files on your sever with the file path in the database.
Related
I have a database of entries consisting of a 'name', 'id' and a 'description', but currently the 'description' field is set to the file path of a .txt file that actually contains the description content. Each .txt file's name is each row's 'id', plus the .txt extension and they all reside in the same directory.
Can I load and replace each 'description' field with the content from the relevant text file (using MySQL)?
You can't write a MySQL query directly that will read the description values from your file system. That would require the MySQL server to be able to read raw text from files in your file system. You Can't Do That™.
You certainly can write a program in your favorite host language (php, java, PERL, you name it) to read the rows from your database, and update your description rows.
You could maybe contrive to issue a LOAD DATA INFILE command to read each text file. But the text files would have to be very carefully formatted to resemble CSV or TSV files.
Purely using mysql this would be a difficult, if not impossible exercise because mysql does not really offer any means to open files.
The only way to open an external text file from mysql is to use LOAD DATA INFILE command, which imports the text file into a mysql table. What you can do is to write a stored procedure in mysql that:
Create a temporary table with a description field large enough to accommodate all descriptions.
Reads all id and description field contents into a cursor from your base table.
Loop through the cursor and use load data infile to load the given text file's data into your temporary table. This is where things can go wrong. The account under which mysql daemon / service runs needs to have access to the directories and fiels where the description files are stored. You also need to be able to parametrise the load data infile command to read the full contents of the text file into a single field, so you need to set the field and line terminated by parameters to such values that cannot be found in any of the description files. But, even for this you need to use a native UDF (user defined function) that can execute command line programs because running load data infile directly from stored procedures is not allowed.
See Using LOAD DATA INFILE with Stored Procedure Workaround-MySQL for full description how to this.
Issue an update statement using the id from the cursor to update the description field in your base table from the temporary table.
Delete the record from your temp table.
Go to 3.
It may be a lot easier to achieve this from an external programming language, that has better file manipulation functions and can update each record in your base table accordingly.
I did some research on how to import XML data into MySQL possibly with the Workbench.
However, I was unable to find any easy tutorial how to do that. I have 6 XML files and all contain data, no schema.
From what I understood, the process consists of 2 parts:
1.Making the table (this is the part which is unclear to me) - is there a way to make the table from only XML data file?
2.Importing the data to the MySQL table. I think I understand this one, it could be done by executing this query:
LOAD XML LOCAL INFILE '/pathtofile/file.xml'
INTO TABLE my_tablename(personal_number, firstname, ...);
I've done it before where I read the XML files into a MySQL database where the data type was set to BLOB.
I haven't seen any data type that can store a file in SQL. Is there something like that? What I'm particularly talking about is that I want to insert into my table a source code. What is the best method to do it? It can be either stored in my database as a nicely formatted text, or better (what I actually want) to store it as a single file. Please note that I'm using MySQL.
It is best not to store a file in your SQL database but to store a path to the file in the server or any other UNC path that your application can retrieve by itself and do with it what ever is unnecessary.
see this: https://softwareengineering.stackexchange.com/questions/150669/is-it-a-bad-practice-to-store-large-files-10-mb-in-a-database
and this:
Better way to store large files in a MySQL database?
and if you still want to store the file on the DB.. here is an example:
http://mirificampress.com/permalink/saving_a_file_into_mysql
If you can serialized the file you can store it as binary and then deserialize when needed
http://dev.mysql.com/doc/refman/5.0/en/binary-varbinary.html
You can also use a BLOB (http://dev.mysql.com/doc/refman/5.0/en/blob.html) which has some differences. Normally I just store the file in the filesystem and a pointer in the DB, which makes serving it back via something like HTTP a bit easier and doesn't bloat up the Database.
Storing the file in a table only makes sense if you need to do searches in that code. In other cases, you should only store a file's URL.
If you want to store a text file, use the TEXT datatype. Since it is a source code, you may consider using the ASCII character set to save space - but be aware that this will cause character set conversions during your queries, and this affects performances. Also, if it is ASCII you can use REGEXP for searches (that operator doesnt work with multi-byte charsets).
To load the file, if the file is on the same server as MySQL, you can use the FILE() function within an INSERT.
I have to upload a crores of data into mysql table. The data is in the form of .csv format. I was tried with load infile method. but, it is also taking very long time. Is there any other way to upload a data ?
You can try the Load Data statement of MYSQL -
http://dev.mysql.com/doc/refman/5.0/en/load-data.html
The LOAD DATA INFILE statement reads rows from a text file into a table at a very high speed. The file name must be given as a literal string.
There is a MySQL utility called mysqlimport http://dev.mysql.com/doc/refman/5.0/en/mysqlimport.html
I suggest to try that.
EDIT Actually it seems to be the same as LOAD DATA INFILE
You can find some usefull tuning tips here:
Speed of INSERT Statements
Bulk Data Loading for MyISAM Tables
Bulk Data Loading for InnoDB Tables
Hope this helps
I'm using myisam_ftdump -c to dump the occurrences of words in my fulltext column. What's the simplest way to insert that information into a new MySQL table?
Thanks for any help, it's appreciated.
Dump the results > to a file and use a LOAD DATA INFILE query to import the contents back into your new table.
Note:
For security reasons, when reading text files located on the server, the files must either reside in the database directory or be readable by all. Also, to use LOAD DATA INFILE on server files, you must have the FILE privilege.