I am having a baffling problem with a simple RDS read replica, where when we do a LOAD DATA LOCAL INFILE on the master, it kills the read replica reliably with :
Error 'Access denied for user ''#'' (using password: NO)' on query. Default database: 'testdb'. Query: 'LOAD DATA INFILE '/rdsdbdata/tmp/SQL_LOAD-5ce65f0d-1a6a-11e3-af1b-12313c014074-352397698-1549.data' IGNORE INTO TABLE `test` FIELDS TERMINATED BY '\t' ENCLOSED BY '' ESCAPED BY '\\' LINES TERMINATED BY '\n' (`name`, `age`)'
To replicate the situation, I had a simple master and create a table called test:
CREATE TABLE `test` (
`name` varchar(25) NOT NULL,
`age` int(11) DEFAULT NULL,
PRIMARY KEY (`name`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1
It was replicated to the read replica successfully. Then i setup my input file :
# cat test.csv
steve,24
bob,34
courtney,12
ben,28
aidan,15
Then imported it with (Ignore the warnings, i didnt get my separator right but data went in) :
mysqlimport --local testdb test.csv
testdb.test: Records: 5 Deleted: 0 Skipped: 0 Warnings: 5
And that's all it takes to kill the read replica. I'm connecting to mysql as the user you create when you make the RDS instance, so it should have all permissions. But regardless, the read replica would have its own replication user. I've tried making the read-replicate NOT read-only, and this didn't seem to work.
mysql> status;
--------------
mysql Ver 14.14 Distrib 5.5.27, for FreeBSD9.0 (amd64) using 5.2
Connection id: 13481
Current database: testdb
Current user: testuser#ip-10-50-99-59.eu-west-1.compute.internal
SSL: Not in use
Current pager: more
Using outfile: ''
Using delimiter: ;
Server version: 5.6.13-log MySQL Community Server (GPL)
Protocol version: 10
Connection: saturn.xxxxxxxxxxx.eu-west-1.rds.amazonaws.com via TCP/IP
Server characterset: latin1
Db characterset: latin1
Client characterset: latin1
Conn. characterset: latin1
TCP port: 3306
Uptime: 8 hours 52 min 23 sec
Threads: 3 Questions: 775195 Slow queries: 0 Opens: 51017 Flush tables: 1 Open tables: 2000 Queries per second avg: 24.268
Would love if anyone could help
thanks
Steve
The reason this breaks is essentially because the read replica is replaying the binary logs and when it gets to your LOAD DATA LOCAL INFILE statement, it doesn't have that file.
RDS Creates a brand new read replica from an existing database by using a snapshot, not by replaying the bin logs.
So the solution is: after any LOAD DATA LOCAL INFILE is executed against the master you will have to delete your read replica and re-create it.
According to AWS technical support, this is a known issue when importing a CSV via LOAD DATA INFILE.
For me, issuing the following command caused the read replica to skip the error and sync successfully:
CALL mysql.rds_skip_repl_error;
There was no need to delete and re-create the read replica.
Update: This issue resolved itself when I upgraded the read replica to MySQL 5.6.
Related
I am using Amazon RDS MySql for data storage. I am able to load data by specifying the column values, however, when I try to load the data from my local machine to MySql, it fails with
Error: 2068.
LOAD DATA LOCAL INFILE '/Users/priya/Documents/Atlas/data/The_Atlas_0820.csv' INTO TABLE schedule
FIELDS TERMINATED by ','
ENCLOSED by '"'
LINES TERMINATED by '\n' IGNORE 1 LINES (Carrier1, FlightNo1, DupCar1,DupCar2, DupCar3,
DupCar4,DupCar5, DupCar6,DupCar7, DupCar8,DepAirport, ArrAirport);
Error Code: 2068. LOAD DATA LOCAL INFILE file request rejected due to restrictions on access.
I also checked SHOW GLOBAL VARIABLES LIKE 'local_infile'; and it is set to "ON".
If i use Terminal to open a connection with MySql and execute the command there, it works ok.
$ ./mysql -h <hostname> -P 3306 --local_infile=1 -u <username> -p
But how to make it work in MySql Workbench.
It seems that there is a bug in MySQL Workbench for LOAD DATA LOCAL INFILE, check if the workaround in this link works for you:
MySQL Workbench 8.0 restricts usage of LOAD DATA LOCAL INFILE
or there is another solution that seems to work here:
Workbench 8.0.12 no longer allows LOAD DATA LOCAL INFILE
Adding what worked for me here, I was able to find a working solution through the links #Zacca provided, but I had to dig through solutions that didn't work, and I thought I'd repost here in case those links die at some point.
First, if I ran SHOW GLOBAL VARIABLES LIKE 'local_infile'; I can see that INFILE is enabled. However, I was still getting "ERROR CODE 2068"
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| local_infile | ON |
+---------------+-------+
I was able to resolve this by editing the connection string in MySQL workbench by navigating to Home, then:
Edit Connection > Advanced > Copy OPT_LOCAL_INFILE=1 into Others box on a new line
Then restart the connection. After that, LOAD LOCAL INFILE runs without error.
Also, as a side note, before I found this solution, I found I could run the LOAD LOCAL INFILE command from R without issue, which helped me isolate the client as the issue, as opposed to a security/permissions related problem.
The solution that worked for me is from the workarounds shared in the link:
Workbench 8.0.12 no longer allows LOAD DATA LOCAL INFILE as shared by #Zacca.
Steps for Mac:
Create a my.cnf file with the following statements at the path: /etc.
I created my.cnf file using root user.
[client]
port = 3306
[mysqld]
port = 3306
secure_file_priv=''
local-infile = 1
Set the my.cnf file as the default configuration file in MySql Workbench.
Click the Wrench icon next to Instance.
Under configuration file, enter the path to my.cnf file: /etc/my.cnf.
Restart the MySQL server workbench.
Try the following statements in MySQL Workbench:
SHOW VARIABLES LIKE "local_infile"; //Should be ON
SHOW VARIABLES LIKE "secure_file_priv"; //Should have no values (not NULL but blank)
Load the data.
LOAD DATA LOCAL INFILE '<path>/file.csv' INTO TABLE <tablename>
FIELDS TERMINATED by ','
ENCLOSED by '"'
LINES TERMINATED by '\n' IGNORE 1 LINES;
Using LOCAL keyword, loading is successful.
However, without LOCAL keyword, I get access error.
Error Code: 1045. Access denied for user 'admin'#'%' (using password: YES)
After beating my head around this for an hour, I switched back to MySQL Workbench 6.3.
Bliss.
I cant seem to use community server(or workbench for that matter) to use the Load local data infile command for csv. it always says cant use this command its not allowed on this version.
Ps I tried setting the local_infile global variable as 1 in the client cmd prompt but it still gives the same issue.Also i cant seem to give the parameters directly at start since the client opens on its own.(I dont have simple mysql command on windows only community server)
Why you want to use load local data infile command in Workbench? It already provides the option for table data import wizard.
Also, for the particular error, I tried following code and it worked for me:
(Before connecting to MySQL database type following)
mysql -u username -p --local-infile databasename
Enter Password:
Now, try using the database and importing the table.
MySQL> use databasename;
MySQL> load data local infile 'filename/filelocation.csv' into table tablename
-> fields terminated by ','
-> enclosed by '\r\n'
-> ignore 1 lines
-> (col1, col2, col3...);
I have created a table in MySQL and have the SQL script handy to create all the tables ans master data. However, i tried all possible way but it is not letting me to do so.
From the command prompt, i connected to mysql by passing this
>mysql -localhost -root -p;
mysql> Use MYDB
mysql> source d:\script\test.sql
it is giving me a message which i don't understand, please help.
mysql Ver 14.14 Distrib 5.6.20, for Win64 (x86_64)
Connection id: 22
Current database: oyeohdb
Current user: dms#localhost
SSL: Not in use
Using delimiter: ;
Server version: 5.6.20 MySQL Community Server (GPL)
Protocol version: 10
Connection: localhost via TCP/IP
Server characterset: latin1
Db characterset: latin1
Client characterset: cp850
Conn. characterset: cp850
TCP port: 3306
Uptime: 1 hour 13 min 34 sec
Threads: 1 Questions: 99 Slow queries: 0 Opens: 68 Flush tables: 1 Open tab
les: 61 Queries per second avg: 0.022
--------------
Outfile disabled.
ERROR:
Failed to open file 'D:criptest.sql', error: 2
mysql> SOURCE D:/script/test.sql;
ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that
corresponds to your MySQL server version for the right syntax to use near 'TYPE=
MyISAM' at line 10
mysql>
Backslash is MySQL's escape character in strings. You should either double it or use forward slash:
mysql> source d:\\script\\test.sql
mysql> d:/script/test.sql
I have had a similar issue and it was solved by running
mysql_upgrade.exe
Located at:
...bin\mysql\mysql5.6.17\bin
I hope that this issue works out for you. Cheers!
I am trying to understand Data Warehousing using the book 'Dimensional Data Warehousing with MySQL'. In Chapter 2 I need to insert the data to a table in MySQL.
I am trying this code
TRUNCATE customer_stg;
LOAD DATA INFILE 'customer2.csv'
INTO TABLE customer_stg
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"'
LINES TERMINATED BY '\r\n'
IGNORE 1 LINES(
customer_number,
customer_name,
customer_street_address,
customer_zip_code,
customer_city,
customer_state
);
I am using lampp on Ubuntu 12.04.2. The file 'customer2.csv' has been copied to the folder '/opt/lampp/var/mysql/dw'.
In MySQL I am using an user 'dwid' with password as 'password'.
The file permissions of the csv file is set to
-rwxr-xr-x 1 mysql mysql 469 Oct 31 2005 customer2.csv
but still I am unable to load the data in the table. The loading is possible from the user 'root'.
From the MySQL 5.5 manual page:
LOCAL works only if your server and your client both have been
configured to permit it. For example, if mysqld was started with
--local-infile=0, LOCAL does not work. See Section 6.1.6, “Security Issues with LOAD DATA LOCAL”.
You should set the option:
local-infile=1
into your [mysql] entry of my.cnf file or call mysql client with the --local-infile option:
mysql --local-infile -uroot -pyourpwd yourdbname
You have to be sure that the same parameter is defined into your [mysqld] section too to enable the "local infile" feature server side.
It's a security restriction. I learned this from some other question in stackoverflow, I couldn't find the link, I had the answer saved. Hope this will help.
I am very new to SQL and in fact just started yesterday. The book I am following is asking me to run sql statements from a text file they provided. However, I am getting an error message and I dont understand why. Here is the commands I entered along with the error message:
mysql> use bank;
Database changed
mysql> source c:\sql\LearningSQLExample.sql;
ERROR 2006 (HY000): MySQL server has gone away
No connection. Trying to reconnect...
Connection id: 9
Current database: bank
--------------
mysql Ver 14.14 Distrib 5.6.12, for Win64 (x86_64)
Connection id: 9
Current database: bank
Current user: lrngsql#localhost
SSL: Not in use
Using delimiter: ;
Server version: 5.6.12 MySQL Community Server (GPL)
Protocol version: 10
Connection: localhost via TCP/IP
Server characterset: utf8
Db characterset: utf8
Client characterset: cp850
Conn. characterset: cp850
TCP port: 3306
Uptime: 1 day 4 hours 9 min 17 sec
Threads: 1 Questions: 6474 Slow queries: 1 Opens: 111 F
tables: 79 Queries per second avg: 0.063
--------------
ERROR:
Unknown command '\L'.
ERROR:
Failed to open file 'c:ql\LearningSQLExample.sql', error: 2
Thanks for your help.
You need to escape the backslash. Type source c:\\sql\\LearningSQLExample.sql;
Just an additional solution when for newer version of mySQL command line
Rename the LearningSQLExample.sql file to something that have underscore in it.
Something like s_s.sql then at the command line try
source c:/temp/s_s.sql;