Load multiple records from multiple files into mysql table - mysql

Question

I'm trying to load different data, from different files, into multiple columns in MySQL. I'm not a big database guy, so maybe I have my data structured wrong. :)
Here's how I have it set up:
DATABASE: mydb
TABLE: aixserver1
COLUMNS: os, hostname, num_users, num_groups, pkg_epoch
shown from mysql:
+---------------+-----------+------+-----+-------------------+----------------+
| Field | Type | Null | Key | Default | Extra |
+---------------+-----------+------+-----+-------------------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| cur_timestamp | timestamp | NO | | CURRENT_TIMESTAMP | |
| pkg_epoch | int(11) | NO | | NULL | |
| os | char(5) | YES | | NULL | |
| hostname | char(40) | YES | | NULL | |
| num_users | int(10) | YES | | NULL | |
| num_groups | int(10) | YES | | NULL | |
+---------------+-----------+------+-----+-------------------+----------------+
So basically I want to populate pkg_epoch, os, hostname, num_users and num_groups into the database. The data I want to load is inside 5 flat files on the server. I'm using ruby to load the data.
My question is how do I load all these values from those files into my table at once. If I do my inserts one at a time, then the other records become NULL. I.E, I load data into just the hostname column, and all the other columns become NULL for that row.
What am I missing? :)

Answer 1

You can do this a couple ways but the trick is to use a variable placeholder. Here is an example if you used the database's LOAD DATA function:
LOAD DATA INFILE '/PATH/TO/FILE' IGNORE INTO TABLE tableName FIELDS TERMINATED BY '\t' LINES
TERMINATED BY '\r' (#skip, #skip, #skip, login_name, pwd, #skip, #skip, #skip, #skip, #skip, first_name, last_name);
You see I just set a variable #skip or #anything for the fields I don't want to include in the database and name the columns that I do want.
I can get you halfway there with this but am uncertain best approach if you build your own loader with Ruby. I would suggest you retrieve the file and let MySQL import using LOAD DATA as it'll be very performant and you can use trick above.