Exported csv SAS table cannot be imported - csv

I exported my SAS table in the form of a csv file into a different folder for me to use with a different program using this code that worked:
PROC EXPORT data=CA_ISO_policyBYpolicy_&thestate.
outfile="&whichfolder.CA_ISO_policyBYpolicy_&thestate..csv"
dbms=dlm replace;
delimiter=",";
run;
Using a different program in a different folder I am trying to import the data via this code:
LIBNAME Home "/sasdata/sasperm2/act_cfr/fr/SJR/AmFam_vs_ISO_Compare/" ;
%let Filepath = /sasdata/sasperm2/act_cfr/fr/SJR/AmFam_vs_ISO_Compare/;
%sdwlogin;
RUN;
%let thestate = OR;
%let policyyr = 2012;
/*---- ISO_Compare ----*/
data Work.CA_ISO_policyBYpolicy_&thestate.;
length Policy $10.;
infile "&Filepath/CA_ISO_policyBYpolicy_&thestate..csv" DELIMITER=',' TERMSTR=CRLF LRECL=2500 FIRSTOBS=2 MISSOVER DSD;
input Policy;
run;
The program runs but I am getting no data. I shortened the variable list to make the code easier to read. When I manually copy and re-paste the data into a different csv file and re-name it the same "CA_ISO_policyBYpolicy_OR.csv" then it works in my program. My initial reason to incorporate this code was to get rid of the manual process... so if anybody has any hints I would be very thankful.

As Joe suggested in the comments, unless you need the csv files for another reason, it would be better to create a SAS data library for this.
Another way to do this would be to proc import it pretty much the same way you used proc export:
proc import datafile="&Filepath./CA_ISO_policyBYpolicy_&thestate..csv"
out=Work.CA_ISO_policyBYpolicy_&thestate. dbms=dlm replace;
delimiter=",";
getnames=yes; *this will create variable names from your first line;
*The opposite of what proc export did;
run;
Other thing I can think of is:
%let Filepath = /sasdata/sasperm2/act_cfr/fr/SJR/AmFam_vs_ISO_Compare/;
Might be causing problems because of the forward slash. Try it like:
%let Filepath = %str(/sasdata/sasperm2/act_cfr/fr/SJR/AmFam_vs_ISO_Compare/);
Also, does the sasdata directory actually exist right on the root directory or is it a subdirectory of the current directory where your sas program is located? If it's the current directory you need to lose the initial forward slash (or put a . in front of it):
%let Filepath = %str(./sasdata/sasperm2/act_cfr/fr/SJR/AmFam_vs_ISO_Compare/);

Related

How to import a CSV file with delimiter as ";" and decimal separator as "," into SAS?

I`ve got (and will receive in the future) many CSV files that use the semicolon as delimiter and the comma as decimal separator.
So far I could not find out how to import these files into SAS using proc import -- or in any other automated fashion without the need for messing around with the variable names manually.
Create some sample data:
%let filename = %sysfunc(pathname(work))\sap.csv;
data _null_;
file "&filename";
put 'a;b';
put '12345,11;67890,66';
run;
The import code:
proc import out = sap01
datafile= "&filename"
dbms = dlm;
delimiter = ";";
GETNAMES = YES;
run;
After the import a value for the variable "AMOUNT" such as 350,58 (which corresponds to 350.58 in the US format) would look like 35,058 (meaning thirtyfivethousand...) in SAS (and after re-export to the German EXCEL it would look like 35.058,00).
A simple but dirty workaround would be the following:
data sap02; set sap01;
AMOUNT = AMOUNT/100;
format AMOUNT best15.2;
run;
I wonder if there is a simple way to define the decimal separator for the CVS-import (similar to the specification of the delimiter). ..or any other "cleaner" solution compared to my workaround.
Many thanks in advance!
You technically should use dbms=dlm not dbms=csv, though it does figure things out. CSV means "Comma separated values", while DLM means "delimited", which is correct here.
I don't think there's a direct way to make SAS read in with the comma via PROC IMPORT. You need to tell SAS to use the NUMXw.d informat when reading in the data, and I don't see a way to force that setting in SAS. (There's an option for output with a comma, NLDECSEPARATOR, but I don't think that works here.)
Your best bet is either to write data step code yourself, or to run the PROC IMPORT, go to the log, and copy/paste the read in code into your program; then for each of the read-in records add :NUMX10. or whatever the appropriate maximum width of the field is. It will end up looking something like this:
data want;
infile "whatever.txt" dlm=';' lrecl=32767 missover;
input
firstnumvar :NUMX10.
secondnumvar :NUMX10.
thirdnumvar :NUMX10.
fourthnumvar :NUMX10.
charvar :$15.
charvar2 :$15.
;
run;
It will also generate lots of informat and format code; you can alternately convert the informats to NUMX10. instead of BEST. instead of adding the informat to the read-in. You can also just remove the informats, unless you have date fields.
data want;
infile "whatever.txt" dlm=';' lrecl=32767 missover;
informat firstnumvar secondnumvar thirdnumvar fourthnumvar NUMX10.;
informat charvar $15.;
format firstnumvar secondnumvar thirdnumvar fourthnumvar BEST12.;
format charvar $15.;
input
firstnumvar
secondnumvar
thirdnumvar
fourthnumvar
charvar $
;
run;
Your best bet is either to write data step code yourself, or to run
the PROC IMPORT, go to the log, and copy/paste the read in code into
your program
This has a drawback. If there is a change in the stucture of the csv file, for example a changed column order, then one has to change the code in the SAS programm.
So it is safer to change the input, substituting in the numeric fields the comma with dot and passing SAS the modified input.
The first idea was to use a perl program for this, and then use in SAS a filename with a pipe to read the modified input.
Unfortunately there is a SAS restriction in the proc import: The IMPORT procedure does not support device types or access methods for the FILENAME statement except for DISK.
So one has to create a workfile on disk with the adjusted input.
I used the CVS_PP package to read the csv file.
testdata.csv contains the csv data to read.
substitute_commasep.perl is the name of the perl program
perl code:
# use lib "/........"; # specifiy, if Text::CSV_PP is locally installed. Otherwise error message: Can't locate Text/CSV_PP.pm in ....;
use Text::CSV_PP;
use strict;
my $csv = Text::CSV_PP->new({ binary => 1
,sep_char => ';'
}) or die "Error creating CSV object: ".Text::CSV_PP->error_diag ();
open my $fhi, "<", "$ARGV[0]" or die "Error reading CSV file: $!";
while ( my $colref = $csv->getline( $fhi) ) {
foreach (#$colref) { # analyze each column value
s/,/\./ if /^\s*[\d,]*\s*$/; # substitute, if the field contains only numbers and ,
}
$csv->print(\*STDOUT, $colref);
print "\n";
}
$csv->eof or $csv->error_diag();
close $fhi;
SAS code:
filename readcsv pipe "perl substitute_commasep.perl testdata.csv";
filename dummy "dummy.csv";
data _null_;
infile readcsv;
file dummy;
input;
put _infile_;
run;
proc import datafile=dummy
out=data1
dbms=dlm
replace;
delimiter=';';
getnames=yes;
guessingrows=32767;
run;

SAS Mainframe to read CSV column names automatically

Is there any way to read read CSV column names automatically in Mainframe environment as PROC IMPORT is not supported in Mainframe? Tried the below code it is working in PC SAS but not in Mainframe SAS.
FILENAME FILEIN "ABC.CUST.FILE" DISP=SHR RECFM=V;
DATA VARNAMES;
INFILE FILEIN DELIMITER=',' DSD OBS=1 LRECL=32000;
INPUT VARNAME $ ##;
RUN;
Thanks in advance.
You can use PROC IMPORT on PC and copy the generated datastep code to mainframe.

SAS Proc Import and wrong format

I am trying to import a csv file through the proc import function. I am using the following syntax:
PROC IMPORT OUT= WORK.claims
DATAFILE= 'C:\Folder\File.csv'
DBMS=csv REPLACE;
GETNAMES=YES;
GUESSINGROWS=125;
RUN;
One of my variable is a character string of the following form: 15/04/2014AB280929D:01ABCDE. Thus it begins by a date, then 9 characters, a column and 7 characters.
The problem is that SAS detects this variable as a date and put a ddmmyy10 format on it. Then, when SAS tries to read the whole file I get errors on every line telling me that I have invalid data for this variable.
How can I fix this ?
Try using the File Import wizard from the menu instead. You'll get the same result initially, but you can then press F4 which will recall the last code submitted (in this case the code that import wizard runs in the background).
You can then modify the informats and formats to suit your needs, then rerun the code.

How to merge multiple csv files into 1 SAS file

I just started using SAS 3 days ago and I need to merge ~50 csv files into 1 SAS dataset.
The 50 csv files have multiple variables with only 1 variable in common i.e. "region_id"
I've used SAS enterprise guide drag and drop functionalities to do this but it was too manual and took me half a day to upload and merge 47 csv files into 1 SAS file.
I was wondering whether anyone has a more intelligent way of doing this using base SAS?
Any advice and tips appreciated!
Thank you!
Example filenames:
2011Census_B01_AUST_short
2011Census_B02A_AUST_short
2011Census_B02B_AUST_short
2011Census_B03_AUST_short
.
.
2011Census_xx_AUST_short
I have more than 50 csv files to upload and merge.
The number and type of variables in the csv file varies in each csv file. However, all csv files have 1 common variable = "region_id"
Example variables:
region_id, Tot_P_M, Tot_P_F, Tot_P_P, Age_0_4_yr_F etc...
First, we'll need an automated way to import. The below simple macro takes the location of the file and the name of the file as inputs, and outputs a dataset to the work directory. (I'd use the concatenate function in Excel to create the SAS code 50 times). Also, we are sorting it to make the merge easier later.
%macro importcsv(location=,filename=);
proc import datafile="&location./&filename..csv"
out=&filename.
dbms=csv
replace;
getnames=yes;
run;
proc sort data= &filename.; by region_id; run;
%mend;
%importcsv(location = C:/Desktop,filename = 2011Census_B01_AUST_short)
.
.
.
Then simply merge all of the data together again. I added ellipses simply because I didn't want to right out 50 times.
data merged;
merge dataseta datasetb datasetc ... datasetax;
by region_id;
run;
Hope this helps.

Can SAS convert CSV files into Binary Format?

The output we need to produce is a standard delimited file but instead of ascii content we need binary. Is this possible using SAS?
Is there a specific Binary Format you need? Or just something non-ascii? If you're using proc export, you're probably limited to whatever formats are available. However, you can always create the csv manually.
If anything will do, you could simply zip the csv file.
Running on a *nix system, for example, you'd use something like:
filename outfile pipe "gzip -c > myfile.csv.gz";
Then create the csv manually:
data _null_;
set mydata;
file outfile;
put var1 "," var2 "," var3;
run;
If this is PC/Windows SAS, I'm not as familiar, but you'll probably need to install a command-line zip utility.
This link from SAS suggests using winzip, which has a freely downloadable version. Otherwise, the code is similar.
http://support.sas.com/kb/26/011.html
You can actually make a CSV file as a SAS catalog entry; CSV is a valid SAS Catalog entry type.
Here's an example:
filename of catalog "sasuser.test.class.csv";
proc export data=sashelp.class
outfile=of
dbms=dlm;
delimiter=',';
run;
filename of clear;
This little piece of code exports SASHELP.CLASS to a SAS Catalog entry of entry type CSV.
This way you get a binary format you can move between SAS installations on different platforms with PROC CPORT/CIMPORT, not having to worry if the used binary package format is available to your SAS session, since it's an internal SAS format.
Are you saying you have binary data that you want to output to csv?
If so, I don't think there is necessarily a defined standard for how this should be handled.
I suggest trying it (proc export comes to mind) and seeing if the results match your expectations.
Using SAS, output a .csv file; Open it in Excel and Save As whichever format your client wants. You can automate this process with a little bit of scripting in ### as well. (Substitute ### with your favorite scripting language.)