Is it possible to search a folder and grab a file name to convert into a mysql table? Can it be done in a query or stored procedures?
I'm new to mysql and so far I can read my data and create a table which is manually done, but don't know how to create a table with its file name automatically.
LOAD DATA
LOCAL INFILE 'D:/test.csv'
INTO TABLE test
Unfortunately it's not possible for LOAD DATA to create the table for you. The syntax for the function goes as ...
LOAD DATA [LOW_PRIORITY | CONCURRENT] [LOCAL] INFILE 'file_name'
[REPLACE | IGNORE]
**INTO TABLE tbl_name**
The table name has to be specified, which means you have to create the table first.
Why is it so?
There are so many different data types that are supported in a mysql table that the importer is unable to judge for itself which data type to use.
Fro example if the first line in the CSV was
hello, 1
The importer might choose CHAR(5) and INT but the next line might be
Colombo, Galle
This line cannot be imported into CHAR(5) or INT
You have tagged your question with mysql-workbench, even though you don't ask about this tool. But let me tell that MySQL Workbench indeed can be of help here. In the SQL IDE which opens when you click on a connection tile on the homescreen you can use the context menu in the sidebar to open the table data import wizard. This will help you to import CSV (or JSON) data from a file and will also create the target table for you (if you don't pick an existing one).
Related
The Sakila database comes with a schema.sql file, a data.sql file and a sakila.mwb file. To load the Sakila dataset in Workbench I first load the schema, then the data, and then open the .mwb file.
Is this the only way you can query in Workbench? Do you always have to import a schema, import data and then open a .mwb file?
For example, I want to be able to go on data.gov, download an XLS, HTML or CSV file (not that I really know how to work with all of these, but ultimately I'd like to) and create a database out of them. Do I have to do my own exploration on their format, create my own schema/model (do these mean the same thing?) and then somehow tell Workbench how to fill in each table in my schema with entries form the CSV/CLS/HTML file?
You can do something like:
1- Create table
CREATE TABLE test(id INT, value1 VARCHAR(255), value2 DECIMAL(8,4));
2- Maybe you have text or other file that have format like :
1 tareq 1.2
2 sarah 2.3
3 Hany 4.5
3- Use the load data command, to import that file into MySql table:
LOAD DATA LOCAL INFILE 'E:/data.txt'
INTO TABLE test COLUMNS TERMINATED BY '\t';
Note that here, values in data file should be delimited by a TAB, and new line after each row.
I have a database of entries consisting of a 'name', 'id' and a 'description', but currently the 'description' field is set to the file path of a .txt file that actually contains the description content. Each .txt file's name is each row's 'id', plus the .txt extension and they all reside in the same directory.
Can I load and replace each 'description' field with the content from the relevant text file (using MySQL)?
You can't write a MySQL query directly that will read the description values from your file system. That would require the MySQL server to be able to read raw text from files in your file system. You Can't Do That™.
You certainly can write a program in your favorite host language (php, java, PERL, you name it) to read the rows from your database, and update your description rows.
You could maybe contrive to issue a LOAD DATA INFILE command to read each text file. But the text files would have to be very carefully formatted to resemble CSV or TSV files.
Purely using mysql this would be a difficult, if not impossible exercise because mysql does not really offer any means to open files.
The only way to open an external text file from mysql is to use LOAD DATA INFILE command, which imports the text file into a mysql table. What you can do is to write a stored procedure in mysql that:
Create a temporary table with a description field large enough to accommodate all descriptions.
Reads all id and description field contents into a cursor from your base table.
Loop through the cursor and use load data infile to load the given text file's data into your temporary table. This is where things can go wrong. The account under which mysql daemon / service runs needs to have access to the directories and fiels where the description files are stored. You also need to be able to parametrise the load data infile command to read the full contents of the text file into a single field, so you need to set the field and line terminated by parameters to such values that cannot be found in any of the description files. But, even for this you need to use a native UDF (user defined function) that can execute command line programs because running load data infile directly from stored procedures is not allowed.
See Using LOAD DATA INFILE with Stored Procedure Workaround-MySQL for full description how to this.
Issue an update statement using the id from the cursor to update the description field in your base table from the temporary table.
Delete the record from your temp table.
Go to 3.
It may be a lot easier to achieve this from an external programming language, that has better file manipulation functions and can update each record in your base table accordingly.
I have tables that are on different mysql instances. I want to export some data as csv from a mysql instance, and perform a left join on a table with the exported csv data. How can I achieve this?
Quite surprisingly that is possible with MySQL, there are several steps that you need to go through.
First create a template table using CSV engine and desired table layout. This is the table into which you will import your CSV file. Use CREATE TABLE yourcsvtable (field1 INT NOT NULL, field2 INT NOT NULL) ENGINE=CSV for example. Please note that NULL values are not supported by CSV engine.
Perform you SELECT to extract the CSV file. E.g. SELECT * FROM anothertable INTO OUTFILE 'temp.csv' FIELDS TERMINATED BY ',';
Copy temp.csv into your target MySQL data directory as yourcsvtable.CSV. Location and exact name of this file depends on your MySQL setup. You cannot perform the SELECT in step 2 directly into this file as it is already open - you need to handle this in your script.
Use FLUSH TABLE yourcsvtable; to reload/import the CSV table.
Now you can execute your query against the CSV file as expected.
Depending on your data you need to ensure that the data is correctly enclosed by quotation marks or escaped - this needs to be taken into account in step 2.
CSV file can be created by MySQL on some another server or by some other application as long as it is well-formed.
If you export it as CSV, it's no longer SQL, it's just plain row data. Suggest you export as SQL, and import into the second database.
I know this is a really basic question but I am struggling on my first import of data from an xml file. I have created the table "Regions" which has just two columns - ID and Name. The xml file contains the same column names.
In order to bulk import the data, I am using the following SQL command:
LOAD XML LOCAL INFILE 'C:\Users\Dell\Desktop\regions.xml'
INTO TABLE Regions (ID, Name)
but I am getting the error #1148 - The used command is not allowed with this MySQL version
Now having researched the internet, to allow this command requires a change in one of the command files but my service provider doesn't allow me access to it. Is there an alternative way to write the SQL code and do exactly the same thing as the code above which is basically just import the data from an xml file?
Many thanks
Since LOAD DATA INFILE isn't enabled for you, it appears you have only one more option and that's to create a set of INSERT statements for each row. If you converted your XML file to CSV using Excel, that's an easy step. Assuming you have a rows of data like this
A | B
-----|-------------------------
1 | Region 1
2 | Region 2
I would create a formula like this in column C
=CONCATENATE("INSERT INTO Regions(ID,Name) VALUES(",A1,",'",B1,"');")
This will result in INSERT INTO Regions(ID,Name) VALUES(1,'Region 1'); for your first row. File this down to the last row of your spreadsheet. Select all the insert statements and copy them into a Query text box inside PHPMyAdmin and you should be able to insert your values.
I've used this method many times when I needed to import data into a database.
As the title says: I've got a bunch of tab-separated text files containing data.
I know that if I use 'CREATE TABLE' statements to set up all the tables manually, I can then import them into the waiting tables, using 'load data' or 'mysqlimport'.
But is there any way in MySQL to create tables automatically based on the tab files? Seems like there ought to be. (I know that MySQL might have to guess the data type of each column, but you could specify that in the first row of the tab files.)
No, there isn't. You need to CREATE a TABLE first in any case.
Automatically creating tables and guessing field types is not part of the DBMS's job. That is a task best left to an external tool or application (That then creates the necessary CREATE statements).
If your willing to type the data types in the first row, why not type a proper CREATE TABLE statement.
Then you can export the excel data as a txt file and use
LOAD DATA INFILE 'path/file.txt' INTO TABLE your_table;