Null rows in output of query - postgresql-8.4

Question

System:
Windows XP professional
Postgresql 8.4 running on locally stored database (port 5432)
I am using the following code to import a decent size file into a table in my database:
-- Create headers and column variable types
CREATE TABLE special_raw_emissions
(
ORG_ID int ,
COUNTY text ,
MID_UP_STREAM text ,
SITE_ID int ,
PRF_ID int ,
RN text ,
ACCOUNT text ,
CERTIFYING_COMPANY_ORGANIZATION text ,
LEASE_NAME text ,
LEASE_NUMBER text ,
SOURCE text ,
FACILITY_ID int ,
PROFILE text ,
LATITUDE float ,
LONGITUDE float ,
OIL_bbl_yr float ,
CASINGHEAD_GAS_scf_yr float ,
GAS_WELL_GAS_scf_yr float ,
CONDENSATE_bbl_yr float ,
PRODUCED_WATER_bbl_yr float ,
TOTAL_VOC_EMISSION_tpy_EXTRACTED_FROM_SE_TAB float ,
CONTROL_PRESENT boolean ,
CONTROL_TYPE text ,
CONTROL_TYPE_IF_OTHER_DESCRIBE text ,
NOX_CONTROL_EFFICIENCY_PCNT float ,
VOC_CONTROL_EFFICIENCY_PCNT float ,
VENTED_VOLUME_scf_yr float ,
BLOWDOWN_EVENTS int ,
OPERATING_HOURS_hrs_yr float ,
FUEL_CONSUMPTION_MMscf_yr float ,
PILOT_GAS_USED_MMscf_yr float ,
WASTE_GAS_COMBUSTED_MMscf_yr float ,
GAS_TREATED_MMscf_yr float ,
AVERAGE_DAILY_PRODUCTION_RATE_MMscf_day float ,
THROUGHPUT_bbl_yr float ,
SEPARATOR_PRESSURE_psig float ,
SEPARATOR_TEMPERATURE_deg_F float ,
GAS_GRAVITY float ,
MAXIMUM_DAILY_PRODUCTION_bbl_day text ,
SOURCE_ANNUAL_THROUGHPUT_bbl_yr float ,
ANNUAL_THROUGHPUT_bbl_yr float ,
MAXIMUM_DAILY_PRODUCTION_RATE__bbl_day float ,
SERIAL_NUMBER text ,
MAKE text ,
MODEL text ,
FUEL_TYPE text ,
MAXIMUM_DESIGN_CAPACITY text ,
BURN_TYPE text ,
CYCLE text ,
ENGINE_RATING text ,
ASSIST_TYPE text ,
AUTOMATIC_AIR_TO_FUEL_RATIO_CONTROLLER boolean ,
DESTRUCTION_EFFICIENCY text ,
SUBJECT_TO_MACT boolean ,
IF_YES_INDICATE_MAJOR_OR_AREA_SOURCE text ,
SOURCE_TYPE text ,
IF_CONDENSER_WHAT_IS_EFFICIENCY text ,
LIQUID_TYPE text ,
IS_HARC_51C_ACCEPTED_METHOD text ,
WOULD_YOU_LIKE_TO_USE_HARC text ,
SINGLE_OR_MULTIPLE_TANKS text ,
NUMBER_OF_TANKS int ,
CONFIGURATION_TYPE text ,
WORKING_AND_BREATHING_EMISS_CALC_METHOD text ,
FLASH_EMISS_CAL_METHOD text ,
FLASH_IF_OTHER_PLEASE_DESCRIBE text ,
IS_MONITORING_PROGRAM_VOLUNTARY int ,
AIR_ACTUATED_PNEUMATIC_VALVES_GAS int ,
AIR_ACTUATED_PNEUMATIC_VALVES_LIGHT_OIL int ,
CONNECTORS_GAS int ,
CONNECTORS_LIGHT_OIL int ,
FLANGES_GAS int ,
FLANGES_LIGHT_OIL int ,
GAS_ACTUATED_PNEUMATIC_VALVES_GAS int ,
GAS_ACTUATED_PNEUMATIC_VALVES_LIGHT_OIL int ,
IS_COMPLETION_OPTIONAL text ,
NONACTUATED_VALVES_GAS int ,
NONACTUATED_VALVES_LIGHT_OIL int ,
OPEN_ENDED_LINES_GAS int ,
OPEN_ENDED_LINES_LIGHT_OIL int ,
OTHER_GAS int ,
OTHER_LIGHT_OIL int ,
PUMP_SEALS_GAS int ,
PUMP_SEALS_LIGHT_OIL int ,
TOTAL_COMPONENTS int ,
TOTAL_PUMPS_AND_COMPRESSOR_SEALS text ,
TOTAL_UNCONTROLLED_RELIEF_VALVES text ,
GAS_ACTUATED_PNEUMATIC_VALVES_HEAVY_OIL int ,
AIR_ACTUATED_PNEUMATIC_VALVES_HEAVY_OIL int ,
NON_ACTUATED_VALVES_HEAVY_OIL int ,
PUMP_SEALS_HEAVY_OIL int ,
CONNECTORS_HEAVY_OIL int ,
FLANGES_HEAVY_OIL int ,
OPEN_ENDED_LINES_HEAVY_OIL int ,
OTHER_HEAVY_OIL int ,
GAS_ACTUATED_PNEUMATIC_VALVES_WATER_SLASH_OIL text ,
AIR_ACTUATED_PNEUMATIC_VALVES_WATER_SLASH_OIL int ,
NON_ACTUATED_VALVES_WATER_SLASH_OIL int ,
PUMP_SEALS_WATER_SLASH_OIL int ,
CONNECTORS_WATER_SLASH_OIL int ,
FLANGES_WATER_SLASH_OIL int ,
OPEN_ENDED_LINES_WATER_SLASH_OIL int ,
OTHER_WATER_SLASH_OIL text ,
VOC_Gas_Mole_Percent float ,
BENZENE_Gas_Mole_Percent float ,
ETHYBENZENE_Gas_Mole_Percent float ,
n_HEXANE_Gas_Mole_Percent float ,
TOLUENE_Gas_Mole_Percent float ,
XYLENE_S_Gas_Mole_Percent float ,
HAPs_Gas_Mole_Percent float ,
VOC_Liquid_Mole_Percent float ,
BENZENE_Liquid_Mole_Percent float ,
ETHYBENZENE_Liquid_Mole_Percent float ,
n_HEXANE_Liquid_Mole_Percent float ,
TOLUENE_Liquid_Mole_Percent float ,
XYLENE_S_Liquid_Mole_Percent float ,
HAPs_Liquid_Mole_Percent float ,
VOC_Control_Factor_PERC float ,
CH4_Emission_Factor_tonne_Btu float,
Engine_LF float ,
CO2_M1 float ,
CO2_M2 float ,
CH4_M1 float ,
CH4_M2 float ,
Source_Class text ,
Site_class text);
-- Import data into database, note that the delimiter is '~'.
COPY special_raw_emissions
FROM 'C:/PostgreSQL/special results/batch.csv'
WITH DELIMITER AS '~'
CSV;
I was running into some strange error, so I did a QA check and queried this table to see if the data imported correctly, shown below is the query:
\o 'c:/postgresql/special_raw_emissions.csv'
select * from special_raw_emissions;
\o
My query returns all the data that was imported, but randomly there are 'null rows' added. Shown below is an example of 'null row'.
Data input:
155 Wise Midstream 8250 1
155 Wise Midstream 8250 1
4 Wise Upstream 7220 1
4 Wise Upstream 7220 1
95 Wise Midstream 7742 1
95 Wise Midstream 7742 1
7 Clay Upstream 1990 7
7 Cooke Upstream 1414 7
Data with null rows (the example shown below suggests a pattern, this is not the case in the larger output file)
7 Clay Upstream 1990 7
7 Cooke Upstream 1414 7
7 Cooke Upstream 1415 7
7 Cooke Upstream 1416 7
7 Cooke Upstream 3355 7
7 Cooke Upstream 3356 7
7 Cooke Upstream 1418 7
7 Cooke Upstream 3357 7
7 Cooke Upstream 1419 7
7 Cooke Upstream 7489 7
Like I said previously, these null rows are causing my queries to miss certain data and I am losing information.
Any help or guidance is greatly appreciated!

Answer 1

The problem was solved in two steps.
Open the raw data in excel, save the data with appropriate delimiter ('~' in my case) and close the file.
re-import the data into the database.
My speculation is that the raw data, which was created with another psql query, was somehow corrupted or had a line ending character missing. Re-saving in excel fixed the issue and allowed the import to work properly.
I still feel as though the problem is unsolved, I simply found a workaround.