how to do LOAD DATA LOCAL INFILE in laravel 5.4 - laravel-5.4

i need to convert the following query in laravel 5.4
$loadDataToTempTableSql = "LOAD DATA LOCAL INFILE '".$filename."' INTO TABLE ABC FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '\"' LINES TERMINATED BY '\r' IGNORE 1 LINES";
$loadDataToTempTableRes = mysqli_query($link,$loadDataToTempTableSql);
$loadedData = mysqli_affected_rows($link);
what i did
Step1:
DB::select($load_data_to_temp_table_sql);
which is throwing exception:
SQLSTATE[HY000]: General error: 2014 Cannot execute queries while
other unbuffered queries are active. Consider using
PDOStatement::fetchAll(). Alternatively, if your code is only ever
going to run against mysql, you may enable query buffering by setting
the PDO::MYSQL_ATTR_USE_BUFFERED_QUERY attribute. (SQL: LOAD DATA
LOCAL INFILE
'/Library/WebServer/Documents/public/abc
copy.csv' INTO TABLE ABC FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"' LINES TERMINATED BY '
' IGNORE 1 LINES)
Step2:
$pdo = DB::connection()->getPdo();
$pdo->exec($load_data_to_temp_table_sql);
again exception:
PDO::exec(): LOAD DATA LOCAL INFILE forbidden
Please guide, what to do?

I have faced same kind of problems and after googling it has been solved now. You should add the following to your mysql/my.cnf file:
[Server]
local_infile=true
After that add 'options' => array(PDO::MYSQL_ATTR_LOCAL_INFILE => true) into your project's config/database.php file.
Now restart your MySQL from cmd/terminal. It will be fine to run the LOAD DATA LOCAL INFILE. If you still face a problem, you can visit https://tenerant.com/blog/using-load-data-local-infile-in-a-laravel-migration/.

i didn't work out with DB. i worked out with PDO and then i had to make a addition to MYSQL_ATTR_LOCAL_INFILE to the database.php file in config folder
'options' => [PDO::MYSQL_ATTR_LOCAL_INFILE => true],

Here is step by step your solution:
Goto/Open config file config/database.php
On connections => mysql array add following lines to enable local in file.
// config/database.php
...
...
'mysql' => [
...
...
'driver' => 'mysql',
'options' => extension_loaded('pdo_mysql') ? array_filter([
PDO::MYSQL_ATTR_SSL_CA => env('MYSQL_ATTR_SSL_CA'),
PDO::MYSQL_ATTR_LOCAL_INFILE => true, // this must be true
]) : [],
],
Restart server and clear configs and cache:
php artisan config:clear
php artisan cache:clear
php artisan serve
That it, now your are go to.

if your using ubuntu or virtual box server with homestead please fullow this:
in your root homestead(with the vagrant file) folder:
vagrant ssh
edit the file:
sudo nano /etc/apparmor.d/usr.sbin.mysqld
find:
Allow data files dir access
and add this line :
/home/Code/NameOfyourProject/public/[name of your upload folder]/* rw
save
then:
sudo service apparmor restart

Related

Laravel fails to connect to MariaDB with SSL enabled

I'm trying to connect InvoiceNinja to my MariaDB database using SSL.
The client server (InvoiceNinja) can connect to the database server via command line without an issue:
mysql -u Username -h Hostname -p
When I use the InvoiceNinja GUI via /setup, I am able to test the database connection okay, but after submitting the page, I end up in a setup loop.
Enabling the debug to true, outputs this:
[2022-07-23 22:34:41] production.INFO: account table not found
[2022-07-23 22:35:25] production.INFO: The command "mysql --user="${:LARAVEL_LOAD_USER}" --password="${:LARAVEL_LOAD_PASSWORD}" --host="${:LARAVEL_LOAD_HOST}" --port="${:LARAVEL_LOAD_PORT}" --database="${:LARAVEL_LOAD_DATABASE}" < "${:LARAVEL_LOAD_PATH}"" failed.
Exit Code: 1(General error)
Working directory: /var/www/ninja/public
Output:
================
Error Output:
================
ERROR 2026 (HY000): SSL connection error: Permission denied
I edited the config/database.php file to include the following under 'mysql':
'options' => [
PDO::MYSQL_ATTR_SSL_VERIFY_SERVER_CERT => true,
PDO::MYSQL_ATTR_SSL_KEY => '/var/www/ninja/db-certs/ninja.key',
PDO::MYSQL_ATTR_SSL_CERT => '/var/www/ninja/db-certs/ninja.pem',
PDO::MYSQL_ATTR_SSL_CA => '/var/www/ninja/db-certs/ca.pem',
],
I've also ran:
php artisan optimize
After removing the entire ninja directory and downloading a fresh file, everything worked. Its possible that previous attempts started to load data and could not be overwritten. I'm not 100% sure, but everything I had configured was correct.
Hope this helps someone.

Error Code: 2068 while loading csv file into MySql

I am using Amazon RDS MySql for data storage. I am able to load data by specifying the column values, however, when I try to load the data from my local machine to MySql, it fails with
Error: 2068.
LOAD DATA LOCAL INFILE '/Users/priya/Documents/Atlas/data/The_Atlas_0820.csv' INTO TABLE schedule
FIELDS TERMINATED by ','
ENCLOSED by '"'
LINES TERMINATED by '\n' IGNORE 1 LINES (Carrier1, FlightNo1, DupCar1,DupCar2, DupCar3,
DupCar4,DupCar5, DupCar6,DupCar7, DupCar8,DepAirport, ArrAirport);
Error Code: 2068. LOAD DATA LOCAL INFILE file request rejected due to restrictions on access.
I also checked SHOW GLOBAL VARIABLES LIKE 'local_infile'; and it is set to "ON".
If i use Terminal to open a connection with MySql and execute the command there, it works ok.
$ ./mysql -h <hostname> -P 3306 --local_infile=1 -u <username> -p
But how to make it work in MySql Workbench.
It seems that there is a bug in MySQL Workbench for LOAD DATA LOCAL INFILE, check if the workaround in this link works for you:
MySQL Workbench 8.0 restricts usage of LOAD DATA LOCAL INFILE
or there is another solution that seems to work here:
Workbench 8.0.12 no longer allows LOAD DATA LOCAL INFILE
Adding what worked for me here, I was able to find a working solution through the links #Zacca provided, but I had to dig through solutions that didn't work, and I thought I'd repost here in case those links die at some point.
First, if I ran SHOW GLOBAL VARIABLES LIKE 'local_infile'; I can see that INFILE is enabled. However, I was still getting "ERROR CODE 2068"
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| local_infile | ON |
+---------------+-------+
I was able to resolve this by editing the connection string in MySQL workbench by navigating to Home, then:
Edit Connection > Advanced > Copy OPT_LOCAL_INFILE=1 into Others box on a new line
Then restart the connection. After that, LOAD LOCAL INFILE runs without error.
Also, as a side note, before I found this solution, I found I could run the LOAD LOCAL INFILE command from R without issue, which helped me isolate the client as the issue, as opposed to a security/permissions related problem.
The solution that worked for me is from the workarounds shared in the link:
Workbench 8.0.12 no longer allows LOAD DATA LOCAL INFILE as shared by #Zacca.
Steps for Mac:
Create a my.cnf file with the following statements at the path: /etc.
I created my.cnf file using root user.
[client]
port = 3306
[mysqld]
port = 3306
secure_file_priv=''
local-infile = 1
Set the my.cnf file as the default configuration file in MySql Workbench.
Click the Wrench icon next to Instance.
Under configuration file, enter the path to my.cnf file: /etc/my.cnf.
Restart the MySQL server workbench.
Try the following statements in MySQL Workbench:
SHOW VARIABLES LIKE "local_infile"; //Should be ON
SHOW VARIABLES LIKE "secure_file_priv"; //Should have no values (not NULL but blank)
Load the data.
LOAD DATA LOCAL INFILE '<path>/file.csv' INTO TABLE <tablename>
FIELDS TERMINATED by ','
ENCLOSED by '"'
LINES TERMINATED by '\n' IGNORE 1 LINES;
Using LOCAL keyword, loading is successful.
However, without LOCAL keyword, I get access error.
Error Code: 1045. Access denied for user 'admin'#'%' (using password: YES)
After beating my head around this for an hour, I switched back to MySQL Workbench 6.3.
Bliss.

MySQL 8: Enable LOAD DATA LOCAL INFILE

I am using Mac with mysql workbench 8.0.19
when i run this
LOAD DATA LOCAL INFILE '~/Users/raianazaman/Desktop/DataBase/Sample.txt' INTO TABLE employee
FIELDS TERMINATED BY ','
ENCLOSED BY ''
LINES TERMINATED BY '\n'
IGNORE 0 LINES;
i get this error:
Error Code: 3948. Loading local data is disabled; this must be enabled on both the client and server sides
I have run this:
SHOW GLOBAL VARIABLES LIKE 'local_infile';
and it shows: 'local_infile','ON'
I have run this: SHOW GLOBAL VARIABLES LIKE 'local_infile'; and it shows: 'local_infile','ON'
I think you can run mysql --local-infile=1 -u root -p and try to load file again. It works for me.

Can not import big CSV to MySQL 5.6 even using --local-infile=1

I have a CSV file with about 600MB size. I wanted to import it using SSH into MySQL 5.6, and as I read, I can use LOAD command in MySQL 5.7
Due to it, I used --local-infile=1 when connecting to MySQL on my server like this:
mysql -u mydb -p --local-infile=1;
And after that, I ran my code like this:
LOAD DATA LOCAL INFILE 'myfile.csv'
INTO TABLE mytable
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\'
(col1,col2,col3);
But still I am getting this error whereas I read everywhere I can solve the problem using --local-infile=1
ERROR 1148 (42000): The used command is not allowed with this MySQL version
Do you know what is the matter?
It only works if the Server allows this:
see: https://dev.mysql.com/doc/refman/5.6/en/load-data.html
The LOCAL keyword affects expected location of the file and error
handling, as described later. LOCAL works only if your server and your
client both have been configured to permit it. For example, if mysqld
was started with --local-infile=0, LOCAL does not work. See Section
6.1.6, “Security Issues with LOAD DATA LOCAL”.
Change your Php settings execution time to unlimited and import using
//Process the score on csv
$location = $_FILES['fileField']['tmp_name'];
$file = fopen($location, "r");
while (($csvfile = fgetcsv($file, 10000, ",")) !== FALSE) {
$firstClm = $csvfile[0];
$secondClm = $csvfile[1];
$thirdClm = $csvfile[2];
//Insert to database
}

MySQL: Enable LOAD DATA LOCAL INFILE

I'm running Mysql 5.5 on Ubuntu 12 LTS. How should I enable LOAD DATA LOCAL INFILE in my.cnf?
I've tried adding local-infile in my config at various places but I'm still getting the "The used command is not allowed with this MySQL version"
From the MySQL 5.5 manual page:
LOCAL works only if your server and your client both have been
configured to permit it. For example, if mysqld was started with
--local-infile=0, LOCAL does not work. See Section 6.1.6, “Security Issues with LOAD DATA LOCAL”.
You should set the option:
local-infile=1
into your [mysql] entry of my.cnf file or call mysql client with the --local-infile option:
mysql --local-infile -uroot -pyourpwd yourdbname
You have to be sure that the same parameter is defined into your [mysqld] section too to enable the "local infile" feature server side.
It's a security restriction.
The my.cnf file you should edit is the /etc/mysql/my.cnf file. Just:
sudo nano /etc/mysql/my.cnf
Then add:
[mysqld]
local-infile
[mysql]
local-infile
The headers [mysqld] and [mysql] are already given, just locate them in the file and add local-infile underneath each of them.
It works for me on MySQL 5.5 on Ubuntu 12.04 LTS.
I solved this problem on MySQL 8.0.11 with the mysql terminal command:
SET GLOBAL local_infile = true;
I mean I logged in first with the usual:
mysql -u user -p*
After that you can see the status with the command:
SHOW GLOBAL VARIABLES LIKE 'local_infile';
It should be ON. I will not be writing about security issued with loading local files into database here.
Replace the driver php5-mysql by the native driver
On debian
apt-get install php5-mysqlnd
in case your flavor of mysql on ubuntu does NOT under any circumstances work and you still get the 1148 error, you can run the load data infile command via command line
open a terminal window
run mysql -u YOURUSERNAME -p --local-infile YOURDBNAME
you will be requested to insert mysqluser password
you will be running MySQLMonitor and your command prompt will be mysql>
run your load data infile command (dont forget to end with a semicolon ; )
like this:
load data local infile '/home/tony/Desktop/2013Mini.csv' into table Reading_Table FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY '\n';
See below image...
I've added --local-infile=1 to normal mysql command mysql -u root -p
So total line would be :
mysql --local-infile=1 -u root -p
Also, for other readers, if you are trying to do this in Django AND your server allows local_infile (you can check by typing SHOW VARIABLES via a mysql client) then you can add this to your settings.py file (since python MySQLdb doesn't by default read the .my.cnf file):
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'mydb',
'USER': 'myname',
'PASSWORD': 'mypass',
'HOST': 'myserver',
'PORT': '3306',
'OPTIONS' : {
'local_infile':1,
},
}
}
Add local_infile in both mysql and mysqld section.
[mysql]
local_infile=1
...
[mysqld]
local_infile=1
...
Tested in MySQL 8.x both in Windows and Linux.
I know this is not exactly what the OP is asking, but as this thread is quite old and none of the solutions proposed here worked for me, I decided to share this.
If someone is having trouble enabling local_infile in the version 8 of MySql, this command here did the trick for me:
SET PERSIST local_infile = 1;
It persists the configuration on the "mysqld-auto.cnf" config file and then the change will be remembered after service or server restart.
You have to take care how you establish your mysqli connection. Full credit for this solution goes to Jorge Albarenque, source
In order to fix it I had to:
Add local-infile=1 to the [mysqld] and [mysql] sections of my.cnf (as explained in the comments above)
Use mysqli_real_connect function (PHP documentation).
The catch is that with that function you can explicitly enable the support for LOAD DATA LOCAL INFILE. For example (procedural style):
$link = mysqli_init();
mysqli_options($link, MYSQLI_OPT_LOCAL_INFILE, true);
mysqli_real_connect($link, $host, $username, $password, $database);
or object oriented
$mysqli = mysqli_init();
$mysqli->options(MYSQLI_OPT_LOCAL_INFILE, true);
$mysqli->real_connect($host, $username, $password, $database);
if your csv file located same with db, you need to remove LOCAL in LOAD DATA INFILE,
or you will get the error
The used command is not allowed with this MySQL version
Another way is to use the mysqlimport client program.
You invoke it as follows:
mysqlimport -uTheUsername -pThePassword --local yourDatabaseName tableName.txt
This generates a LOAD DATA statement which loads tableName.txt into the tableName table.
Keep in mind the following:
mysqlimport determines the table name from the file you provide; using all text from the start of the file name up to the first period as the table name. So, if you wish to load several files to the same table you could distinguish them like tableName.1.txt, tableName.2.txt,..., etc, for example.
This went a little weird for me, from one day to the next one the script that have been working since days just stop working. There wasn´t a newer version of mysql or any kind of upgrade but I was getting the same error, so I give a last try to the CSV file and notice that the end of lines were using \n instead of the expected ( per my script ) \r\n so I save it with the right EOL and run the script again without any trouble.
I think is kind of odd for mysql to tell me The used command is not allowed with this MySQL version since the reason was completely different.
My working command looks like this:
LOAD DATA LOCAL INFILE 'file-name' IGNORE INTO TABLE table-name CHARACTER SET latin1 FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '\"' LINES TERMINATED BY '\r\n' IGNORE 1 LINES.
I used below method, which doesn't require any change in config, tested on mysql-5.5.51-winx64 and 5.5.50-MariaDB:
put 'load data...' in .sql file (ex: LoadTableName.sql)
LOAD DATA INFILE 'D:\\Work\\TableRecords.csv' INTO TABLE tbl1 FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY '\r\n' IGNORE 1 LINES (col1,col2);
then:
mysql -uroot -pStr0ngP#ss -Ddatabasename -e "source D:\Work\LoadTableName.sql"
In case if Mysql 5.7 you can use "show global variables like "local_infile" ;" which will give the local infile status ,You can turn it on using "set global local_infile=ON ; ".
I am using xampp v3.2.4 and mysql server 8.0.20.
I added local-infile=1 to [mysql] and [mysqld] in the file "my.ini". The file is located at "C:\xampp\mysql\bin\my.ini".
Then I inserted the data from csv file using the following code LOAD DATA INFILE .... It is important to move LOCAL. Otherwise it won't work.
Thanks for all suggestions above. A combination finally worked out for me.
Ok, something odd is happening here. To make this work, do NOT need to make any configuration changes in /etc/mysql/my.cnf . All you need to do is to restart the current mysql service in terminal:
sudo service mysql restart
Then if I want to "recreate" the bug, I simply restart the apache service:
sudo service apache2 restart
Which can then be fixed again by entering the following command:
sudo service mysql restart
So, it appears that the apache2 is doing something to not allow this feature
when it starts up (which is then reversed/corrected if restart the mysql service).
Valid in Debian based distributions.
service mysqld restart
service httpd restart
Valid in RedHat based distributions
For those of you looking for answers to make LOAD DATA LOCAL INFILE work like me, this might probably work. Well it worked for me, so here it goes. Install percona as your mysql server and client by following the steps from the link. A password will be prompted for during the installation, so provide one that you'll remember and use it later. One the installation is done, reboot your system and test if the server is up and running by going to the terminal and typing mysql -u root -p and then the password. Try running the command LOAD DATA LOCAL INFILE now.. Hope it works :)
BTW I was working on Rails 2.3 with Ruby 1.9.3 on Ubuntu 12.04.
All: Evidently this is working as designed. Please see new ref man dated 2019-7-23, Section 6.1.6, Security Issues with LOAD DATA LOCAL.
Please note that for newer MySQL servers, like version 8.0, this setting is not a boolean value, but ON|OFF as you can read here in the docs: https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_local_infile
If you set it to 1 MySQL will not recognize it and will warn this error on your logfiles:
# /var/log/mysql/mysql.err
2022-04-08T14:45:06.532132Z 0 [Warning] [MY-000076] [Server] option 'local_infile': boolean value '1;' was not recognized. Set to OFF.