Import CSV MySQL Workbench Mac Catalina Very Slow - mysql

Question

I have a Macbook pro 2019 touch,i5, 8M,HD 256 GB used 75GB, OSX 10.15.2. Installed AMPPS 3.9 and MSQL Workbench 8.0
I'm importing to Localhost, a mysql table in csv 1.9 GB 15 million register, after 12 hours only 3 Million have been imported.
Already probe XAMPP VM and in the mariaDB console failed to do LOAD DATA INFILE. It disconnects from the database.
I'm going to process the table with PHP. Any help or recommendation so that these processes become faster ?
I use Sequel pro to read remote databases, very fast but no longer works for localhost.

Answer 1

I had a similar problem in importing a csv file (approx 5 mil records) into a mysql table. I managed to do that using nodejs as it is able to open and read a file line by line (without loading the entire file into ram). The script reads 100 lines, then makes an INSERT then reads another 100 lines and so on. It processed the entire file in about 9 minutes. It uses the readline module. The main part of the script looks something like this:
const Fs = require('fs');
const Mysql = require('mysql2');
const Readline = require('readline');
const fileStream = Fs.createReadStream('/path/to/file');
var dbConnection = Mysql.createConnection({
host : "yourHost",
user : "yourUser",
password : "yourPassword",
database : "yourDatabase"
});
const rl = Readline.createInterface({
input: fileStream,
crlfDelay: Infinity
});
async function run() {
var lineElements = [];
for await (const line of rl) {
// Each line in will be successively available here as `line`.
lineElements = line.split(",");
// now lineElements is an array containing all the values of the current line
// here you can queue up multiple lines, to make a bigger insert
// or just insert line by line
await dbConnection.query('INSERT INTO ..........');
}
}
run();
The script above inserts on line per query. Fell free to modify it, if you want a query to insert 100 lines or any other value.
As a side note: because my file was "trusted" I did not used prepared statements, as I think that the simple query is faster. I do not know if the speed gain was significant, as I did not do any tests.