Importing CSV to MySQL table returns an error #1148 - mysql

I am trying to import with DirectAdmin, when I selected CSV without using LOAD DATA - I got the error "Invalid field count in CSV input on line 1."
When I tried with LOAD DATA I got the following error: "#1148 - The used command is not allowed with this MySQL version."
The CSV was created in MS Access from MS Access database.
Here are the first 2 rows:
"product_id","vendor_id"," product_parent_id","product_sku","product_s_desc ","product_desc","product_thumb_image ","product_full_image","product_publish","product_weight","product_weight_uom ","product_length ","product_width","product_height ","product_lwh_uom ","disp_order","price","sale","product_url ","product_in_stock","product_available_date","product_availability ","product_special ","product_discount_id ","ship_code_id ","cdate ","mdate ","product_name ","product_sales ","attribute ","custom_attribute ","product_tax_id ","product_unit ","product_packaging ","child_options ","quantity_options ","child_option_ids ","product_order_levels "
41,2,0,1,,,"resized/Krug-Rose-Champagne-lg.jpg","Krug-Rose-Champagne-lg.jpg","Y","750.0000","grams","4.0000","4.0000",14,,14,3516,0,,,1296518400,,"N",0,"NULL ",1296574622,1297953843,"קרוג רוזה",0,,,2,"piece ",65537,"N,N,N,N,N,Y,20%,10%, ","none,0,0,1 ",,"0,0 "


From mysql command line pass the following parameter:
mysql -u username -p dbname --local-infile
Instead of using: load data infile, use: load data local infile and it should perform the import.
By default, mysql does not enable load data local as per the the security concerns defined here:
http://dev.mysql.com/doc/refman/5.0/en/load-data-local.html
If LOAD DATA LOCAL is disabled, either in the server or the client, a
client that attempts to issue such a statement receives the following
error message:
ERROR 1148: The used command is not allowed with this MySQL version

Related

Restore db from sql file

I am tring import data from sql file using command line like this:
mysys-12: mysql -u root -p my_db_t < my_db_t_2022_10_12.sql
but I got:
mysql: [ERROR] unknown variable 'sql-mode=STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_ENGINE_SUBSTITUTIO
How can I import data? Should I add any params?

It seems that the content in the sql file conflicts with the constraint of sql_mode. There are two methods. The first method is to modify the content of the file to meet the constraint of sql_mode. The second method is to set SQL _ mode ='' in mysql. Then import the data and change it back.

Problem solved, I removed sql-mode from client configuration, file: etc/mysql/conf.d/mysql.cnf
Server configuration - No change

Windows and Linux file path issues using python SQL load data infile

I am working on a mysql (8) db which is too big for the 2TB linux partition size so I have moved the mysql instance onto a 16TB nvme raid under windows 10. All my other code is running on Debian 10 on a virtualbox instance and I have mapped a perm drive between the Debian VM and the nvme raid array.
I can open the database from debian and read and write as normal, so the odbc connector is working fine.
The issue here is the fact I am loading very large json log files into one table and doing it a row at a time was taking hours, so I opted to create a csv file for each log file and use LOAD DATA INFILE as part of the SQL statement.
Trouble is, when I execute the sql statement I get the 'file not found' issue, even though, looking at debug code, the file path is correct and it actually exists.
An excert from my python 3 code is:
p = f"/media/sf_unpack/{filename}"
try:
with open(p, 'r', buffering=1024 * 1024) as csvfile:
print(csvfile.read())
SQL = f"LOAD DATA INFILE '{p}' INTO TABLE xxx fields terminated by ',' LINES TERMINATED BY '\n' (field,field,.....etc) ;"
try:
mycursor.execute(SQL)
connection_object.commit()
except Exception as ex:
displayerror(ex)
This code will open the file correctly and show a value for p of /media/sf_unpack/filename.csv (which is correct).
When we get to the mycusror.execute(SQL) is raises an exception and says the directory or filename cannot be found. Interestingly, and I am sure this is the issue, the dubugger tells me the file that cannot be found is a windows version D:\media/sf_unpack/filename.csv - which looks as if it has something to do with the virtualbox mapping.
i.e. p =
I have tried to use the Path method from pathlib i.e p = Path(f"D:\mysql\unpack\{filename}" but that makes no difference.
I know I am doing something stupid but I am not sure what it is.
Any help would be gratefully recieved

Mitaka nova-manage api_db sync 'error: No sql_connection parameter is established'

I am trying to set up a Mitaka OpenStack cloud. But when I try to execute:
# /usr/bin/nova-manage --debug api_db sync
And the I get the error message:
error: No sql_connection parameter is established
Yet I am able to access the nova database via mysql command line, using the values I am using for my I have in the /etc/nova/nova.conf:
[database]
connection=mysql://nova:nova#svl-os:3306/nova

In the Mitaka release they added a new DB schema, nova_api. So I needed to add ...
[api_database]
connection=mysql://nova_api_db_user:password#mydbhost:3306/nova_api
... to my /etc/nova/nova.conf file.

DatabaseError: 1 (HY000): Can't create/write to file '2015-04-06 20:48:33.418000'.csv (Errcode: 13 - Permission denied)

I am designing an application in Python and trying to write to a CSV file, but I am getting this error:
DatabaseError: 1 (HY000): Can't create/write to file '2015-04-06 20:48:33.418000'.csv (Errcode: 13 - Permission denied)
The Code:
def generate_report(self):
conn=mysql.connector.connect(user='root',password='',host='localhost',database='mydatabase')
exe2 = conn.cursor()
exe2.execute("""SELECT tbl_site.Site_name, State_Code, Country_Code,Street_Address, instrum_start_date, instrum_end_date, Comment INTO OUTFILE %s FIELDS TERMINATED BY '|' OPTIONALLY ENCLOSED BY '"' ESCAPED BY '\\\\' LINES TERMINATED BY '\\n'FROM tbl_site JOIN tbl_site_monit_invent ON site_id = tbl_Site_site_id """, (str(datetime.datetime.now()),))
I can run this code without any errors on a Mac, but I need it to work on Windows.
How can I resolve this error?

Simple really. A colon character is not a valid character in a filename on Windows. It's not allowed.
Reference: https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247%28v=vs.85%29.aspx
The colon character is in the list of "reserved characters", along with several others. (NOTE: One use of the colon character is as a separator for an Alternate Data Stream on NTFS. Ref: http://blogs.technet.com/b/askcore/archive/2013/03/24/alternate-data-streams-in-ntfs.aspx
Followup
The question has been significantly edited since my previous answer was provided. Some notes:
I'm not very familiar with running MySQL on Windows OS. Most of my work with MySQL server is on Linux.
The SELECT ... INTO OUTFILE statement will cause the MySQL server to attempt to write a file on the server host.
The MySQL user (the user logged in to MySQL) must have the FILE privilege in order to use the SELECT ... INTO OUTFILE statement.
Also, the OS account that is running MySQL server must have OS permissions to write a file to the specified directory, and the file to be written must not already exist. Also, the filename must conform to the naming rules for filenames on OS filesystem.
Ref: https://dev.mysql.com/doc/refman/5.5/en/select-into.html
For debugging this type of issue, I strongly recommend you echo out the actual SQL text that is going to be sent to the MySQL server. And then take that SQL text and run it from a different client, like the mysql command line client.
For debugging a privileges issues, you can use a much simpler statement. Test writing a file to a directory that is known to exist, that is known the mysql server has permissions to write files to, and with a filename that does not exist and that conforms to the rules for the OS and filesystem.
For example, on a normal Linux box, we could test with something like this:
mysql> SELECT 'bar' AS foo INTO OUTFILE '/tmp/mysql_foo.csv'
Before we run that, we can easily verify that the /tmp directory exists, that it is writable by the OS account that is running the mysql server, and that the filename conforms to the rules for the filesystem, and that the filename doesn't exist, e.g.
$ su - mysql
$ ls -l /tmp/mysql_foo.csv
$ echo "foo" >/tmp/mysql_foo.csv
$ cat /tmp/mysql_foo.csv
$ rm /tmp/mysql_foo.csv
$ ls -l /tmp/mysql_foo.csv
Once we get over that hurdle, we can move on to testing writing a file to a different directory, a file with a more more complex filename. Once we get that plumbing working, we can work on getting actual data, into a usable csv format.
The original question seems to indicate that the MySQL server is running on Windows OS, and it seems to indicate that the filename attempting to be written contains semicolon characters. Windows does not allow semicolon as part a filename.

It was simply permission error.

SOURCE error 2?

When i try to source an sql file i get the error:
mysql> source C:/Users/tom/Documents/insert.sql
ERROR:
Failed to open file 'C:/Users/tom/Documents/insert.sql', error: 2
I have checked the file path, which looks fine to me. I have also tried \. C:/Users/etc
I am trying to source the sql file which holds insert statements for particular tables. All the statements in the file work when entered manually. What else could i be doing wrong?
Have tried using both backslash and forward slash when using this command

Probably a problem of access right on the file (the file is being accessed by the mysqld server process, not yourself). Try placing the file into the data folder of MySQL, then import it from this location. The location of data folder depends on your distribution and on your own configuration.
Alternatively, feed the SQL script directly to your mysql client's stdin:
mysql [all relevant options] your_database < C:\path\to\your\script.sql

I am using Ubuntu 14.04 version.
I too faced below error 2.
mysql> SOURCE home/loc/Downloads/AllTables.sql;
Failed to open file 'home/loc/Downloads/AllTables.sql', error: 2
Solution :
mysql> SOURCE /home/loc/Downloads/AllTables.sql;
Just added a '/' in front of home
Hope this helps some one.

Have you checked if the file exits? I have had this problem before.
This:
this:
and this works: