Load data infile ignore 1 lines skip 2 lines - mysql

I want to import a CSV file to my MySQL table using Load data infile, but sometimes it skips 2 lines.
My SQL:
LOAD DATA LOW_PRIORITY LOCAL INFILE 'C:\\csv\\member.csv'
REPLACE INTO TABLE `member`
CHARACTER SET utf8mb4 FIELDS
TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"'
ESCAPED BY '"'
LINES TERMINATED BY '\r\n'
IGNORE 1 LINES (`UserName`, `RegDate`, `Address`, `Remark`);
Data:
"UserName","RegDate","Address","Remark"
"Peter",07-AUG-2001 18:17,"10th Floor, xxx's PlazaI","Remark xxx" xxx"
"Mary",07-NOV-2001 15:17,,"The ""BIG"" boss"
"Paul",07-DEC-2001 15:17,"10th Floor, xxx's PlazaI",""

Related

LINES TERMINATED BY '\n' not working when store mysl result in txt file

When I try to store mysql result in txt file lines terminated is not working
select * from users INTO OUTFILE '/var/lib/mysql-
files/users.txt'
FIELDS TERMINATED BY '|'
ENCLOSED BY '"'
LINES TERMINATED BY '\n';---> not working
the result are all in one line
"1"|"John#test.com"|"20"|"John"|"test"/"2"|"omar#test.com"|"21"|"Omar"|"test2""3"|"Mytest#test.com"|"30"|"Mytest"|"test3"

How to convert string to date time format Mysql

I have this file csv:
20170117-00:00:6 3087A6B282A46C 124.191.134.139 2746 256 63628
20170117-00:00:53 330391C9F58B5A 120.18.97.172 2746 257 43077
20170117-00:02:27 3303CD14BD40C8 110.147.131.78 2746 256 63628
20170117-00:02:00 3304B3CD448E4F 189.35.218.62 2746 255 43074
and I am trying to upload it using load data local infile but I obtain only Null value in the time column. Here there is my code (Time is the name of the table column)
LOAD DATA LOCAL INFILE 'path.log'
INTO TABLE Clicks
CHARACTER SET latin1
FIELDS TERMINATED BY '\t'
ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 1 ROWS
SET Time= STR_TO_DATE(#Time,'%Y%m%d-%H:%i:%s')
;

This is the syntax you should be using:
LOAD DATA LOCAL INFILE 'path.log'
INTO TABLE Clicks
CHARACTER SET latin1
FIELDS TERMINATED BY '\t'
ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 1 ROWS
(#Time, column2, column3, column4, column5) -- read time data into #Time
SET column1 = STR_TO_DATE(#Time, '%Y%m%d-%H:%i:%s') -- then set column1
I think the #Time variable was never being set to anything for each record being read, hence you were getting NULL in the first column.

Try This
LOAD DATA LOCAL INFILE 'path.log'
INTO TABLE Clicks
CHARACTER SET latin1
FIELDS TERMINATED BY '\t'
ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 1 ROWS
SET TIME= CONCAT(SUBSTRING(#Time,1,4),'-',SUBSTRING(#Time,5,2),'-',SUBSTRING(#Time,7,2),' ',REPLACE(#Time,CONCAT(SUBSTRING_INDEX(#Time, '-',1),'-'),''))

Backslash throws unexpected beginning of statement error

So I have this SQL Script for bulk loading my data:
INSERT INTO DeliveryMethod (deliveryMethod)
VALUES ('Bicycle');
INSERT INTO DeliveryMethod (deliveryMethod)
VALUES ('Car');
INSERT INTO DeliveryMethod (deliveryMethod)
VALUES ('Van');
INSERT INTO DeliveryMethod (deliveryMethod)
VALUES ('None');
SET foreign_key_checks = 0;
# Load data into categories
LOAD DATA LOCAL INFILE
'C:/Users/ryank/OneDrive/Documents/GitHub/DBMS-CW/R4/CategoryData.txt'
REPLACE INTO TABLE CourierDB.Category
FIELDS TERMINATED BY ','
LINES STARTING BY '(' TERMINATED BY ')\r'
(name)
SET categoryID = NULL; # Should trigger auto increment (or \r)
# Load data into package
LOAD DATA LOCAL INFILE
'C:/Users/ryank/OneDrive/Documents/GitHub/DBMS-CW/R4/PackageData.txt'
REPLACE INTO TABLE CourierDB.Package
FIELDS TERMINATED BY ','
LINES STARTING BY '(' TERMINATED BY ')\r'
(price, itemName, category)
SET packageID = NULL; # Should trigger auto increment (or \r)
# Load data into address
LOAD DATA LOCAL INFILE
'C:/Users/ryank/OneDrive/Documents/GitHub/DBMS-CW/R4/AddressData.txt'
REPLACE INTO TABLE CourierDB.Address
FIELDS TERMINATED BY ','
LINES STARTING BY '(' TERMINATED BY ')\r'
(buildingName, streetName, county, postcode)
SET addressID = NULL;
# Load data into packages
LOAD DATA LOCAL INFILE
'C:/Users/ryank/OneDrive/Documents/GitHub/DBMS-CW/R4/PackagesData.txt'
REPLACE INTO TABLE CourierDB.Packages
FIELDS TERMINATED BY ','
LINES STARTING BY '(' TERMINATED BY ')\r'
(packagesID, package);
# Load data into branch
LOAD DATA LOCAL INFILE
'C:/Users/ryank/OneDrive/Documents/GitHub/DBMS-CW/R4/BranchData.txt'
REPLACE INTO TABLE CourierDB.Branch
FIELDS TERMINATED BY ','
LINES STARTING BY '(' TERMINATED BY ')\r'
(branchName, address, headOfficeID, managerID, deliveryMethods)
SET branchID = NULL;
# Load data into consignment
LOAD DATA LOCAL INFILE
'C:/Users/ryank/OneDrive/Documents/GitHub/DBMS-CW/R4/ConsignmentData.txt'
REPLACE INTO TABLE CourierDB.Consignment
FIELDS TERMINATED BY ','
LINES STARTING BY '(' TERMINATED BY ')\r'
(dispatchDate, consignmentType, branch, deliveryAddressID, returnAddressID, packages)
SET trackingID = NULL;
# Load data into DeliveryMethods
LOAD DATA LOCAL INFILE
'C:/Users/ryank/OneDrive/Documents/GitHub/DBMS-CW/R4/DeliveryMethodsData.txt'
REPLACE INTO TABLE CourierDB.DeliveryMethods
FIELDS TERMINATED BY ','
LINES STARTING BY '(' TERMINATED BY ')\r'
(deliveryMethodID, deliveryMethod);
# Load data into Employee
LOAD DATA LOCAL INFILE
'C:/Users/ryank/OneDrive/Documents/GitHub/DBMS-CW/R4/EmployeeData.txt'
REPLACE INTO TABLE CourierDB.Employee
FIELDS TERMINATED BY ','
LINES STARTING BY '(' TERMINATED BY ')\r'
(NIN, firstName, lastName, dateOfBirth, emailAddress, mobileNo, salary, branchID, supervisorID, address)
SET staffNo = NULL;
# Load data into CustomerConsignments
LOAD DATA LOCAL INFILE
'C:/Users/ryank/OneDrive/Documents/GitHub/DBMS-CW/R4/CustomerConsignmentsData.txt'
REPLACE INTO TABLE CourierDB.CustomerConsignments
FIELDS TERMINATED BY ','
LINES STARTING BY '(' TERMINATED BY ')\r'
(consignmentsID, consignment);
# Load data into Customer
LOAD DATA LOCAL INFILE
'C:/Users/ryank/OneDrive/Documents/GitHub/DBMS-CW/R4/CustomerData.txt'
REPLACE INTO TABLE CourierDB.Customer
FIELDS TERMINATED BY ','
LINES STARTING BY '(' TERMINATED BY ')\r'
(firstName, lastName, dateOfBirth, emailAddress, mobileNo, customerBranchID, address, consignments)
SET customerID = NULL;
SET foreign_key_checks = 1;
However with this said... I'm getting this error here for some reason when i put it into my relational schema:
Static analysis:
3 errors were found during analysis.
Unexpected character. (near "\" at position 0) Unexpected beginning of
statement. (near "\" at position 0) Unexpected beginning of statement.
(near "r" at position 1) SQL query:
\r LOAD DATA LOCAL INFILE
'C:/Users/ryank/OneDrive/Documents/GitHub/DBMS-CW/R4/PackageData.txt'
REPLACE INTO TABLE CourierDB.Package FIELDS TERMINATED BY ',' LINES
STARTING BY '(' TERMINATED BY ')\r' (price, itemName, category) SET
packageID = NULL
MySQL said: Documentation
1064 - You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use
near '\r LOAD DATA LOCAL INFILE
'C:/Users/ryank/OneDrive/Documents/GitHub/DBMS-CW/R4' at line 1
Any help would be greatly appreciated! Thanks

MYSQL command string being cut

i have this code on my program
strSQL = "load data local infile 'C:/csv.csv' replace into table student_records fields terminated by ',' enclosed by '"' lines terminated by '\r\n'"
but the string it considers is only "load data local infile 'C:/csv.csv' replace into table student_records fields terminated by ',' enclosed by '"
and this is considered a comment' lines terminated by '\r\n'"
the string is being terminated by the double quote while it is still part of my mysql command.
can you give me another way around on how to do this?
i'm trying to automate the updating of data in my table.

Try escaping double quotes with "\" character:
strSQL = "load data local infile 'C:/csv.csv' replace into table student_records fields terminated by ',' enclosed by '\"' lines terminated by '\r\n'"

load data infile with unknown system variable error

I´ve been trying to use load data infile procedure to load data from a csv file to mysql, althougt I'ven been unable to due it because of a unknown variable error in the set statement.
LOAD DATA LOCAL INFILE 'D:\\Bets\\BD\\tables\\match.csv'
INTO TABLE `bets`.`match`
FIELDS TERMINATED BY ';' OPTIONALLY ENCLOSED BY '"' ESCAPED BY '"'
LINES TERMINATED BY '\r\n'
(`idMatch`, `idChampionship`, #date, `homeTeam`, `awayTeam`, `homeTeamGoals`, `awayTeamGoals`, `matchType` );
SET matchDate = STR_TO_DATE(#date, 'YYYY-MM-DD')
The date field to import is in format 2011-08-07.
when running the query the error given is: SQL Error (1193): Unknown system variable 'matchDate'.
any help would be pappreciated... thanks!

When SET matchDate it should be like this:
SET matchDate = STR_TO_DATE(#date, '%Y-%m-%d');
instead of SET matchDate = STR_TO_DATE(#date, 'YYYY-mm-dd');
Example of MySQL STR_TO_DATE function you can read here.

I've stumbled upon the same issue. The SET line must be part of the LOAD DATA INFILE ... statement. In your code, the column expression is terminated by a semicolon, which executes the SET command after the LOAD DATA INFILE finishes. The user variables created in the statement are not existing any more after the statement completes. Correct statement would be as follows,
LOAD DATA LOCAL INFILE 'D:\\Bets\\BD\\tables\\match.csv'
INTO TABLE `bets`.`match`
FIELDS TERMINATED BY ';' OPTIONALLY ENCLOSED BY '"' ESCAPED BY '"'
LINES TERMINATED BY '\r\n'
(`idMatch`, `idChampionship`, #date, `homeTeam`,
`awayTeam`,`homeTeamGoals`, `awayTeamGoals`, `matchType`)
SET matchDate = STR_TO_DATE(#date, 'YYYY-MM-DD');