I am using MYSQL. Basically I am trying to generate a dataset based on my own defined date range. Can someone share how I am able to do a looping like in SAS? example:
%macro date_loop(start,end);
/*converts the dates to SAS dates*/
%let start=%sysfunc(inputn(&start,anydtdte9.));
%let end=%sysfunc(inputn(&end,anydtdte9.));
/*determines the number of months between the two dates*/
%let dif=%sysfunc(intck(month,&start,&end));
%do i=0 %to &dif;
/*advances the date i months from the start date and applys the DATE9. format*/
%let date=%sysfunc(putn(%sysfunc(intnx(month,&start,&i,b)),date9.));
%put &date;
%end;
%mend;
%date_loop(2017-01-31,2019-01-31)
Part of my MySQL code: The st_date is self defined.
SET #st_date = '2019-01-31'
DATE_FORMAT(#st_date,'%d/%m/%Y') as POSITION_DATE,
Related
I need to write a macro program to generate a list for moving average calculation where I will need some lines to be constructed like this:
var1_ma_past_1=mean(var1, lag1(var1), lag2(var1), lag3(var1), lag4(var1), lag5(var1));
var1_ma_past_2=mean(lag1(var1), lag2(var1), lag3(var1), lag4(var1), lag5(var1), lag6(var1));
var1_ma_past_3=mean(lag2(var1), lag3(var1), lag4(var1), lag5(var1), lag6(var1), lag7(var1));
[...]
var2_ma_past_1=mean(var2, lag1(var2), lag2(var2), lag3(var2), lag4(var2), lag5(var2));
my sample program is
%macro test ;
%do i = 1 %to 5;
%let ln&i = ;
%do j = 1 %to 5;
%let dml = %str(,);
%let pos = %str(lag&i(var&j));
%let ln&j = %sysfunc(catx(&dml, &&ln&j, &pos));
%end;
%end;
/* example output */
%put &ln1;
%mend test;
%test
&j start and end values are planned to be replaced with parameters though.
the output is desired for &ln1
lag1(var1),lag2(var1),lag3(var1),lag4(var1),lag5(var1)
but for &ln2 &ln3 etc it's not (lag1(varn) is missing)
lag2(var4),lag3(var4),lag4(var4),lag5(var4)
lag2(var3),lag3(var3),lag4(var3),lag5(var3)
Besides, I got flooding log output saying ERROR: Required operator not found in expression: which is because of the parentheses inside of cats(), which is inside of %sysfunc(), an example macro to replicate this is
%macro test2;
%let x=meow;
%put %sysfunc(cats(x,lag()));
%mend;
%test2
I tried to mask the parentheses with %str, %superq, %bquote but none worked.
I'd like to learn
the reason for incorrect output for &ln2, &ln3 and so on
the reason for ERROR: Required operator not found in expression: and how to fix it (or a workaround, or even to suppress the error if it's not critical)
Thanks in advance.
There is no need to use CAT...() functions in macro code.
In macro code to concatenate values you just expand them next to each other.
Also it looks like your logic is mixing up the I and J counters.
%macro test ;
%do i = 1 %to 5;
%let list = ;
%let dlm = ;
%do j = 1 %to 5;
%let list = &list.&dlm.lag&j(var&i) ;
%let dlm = ,;
%end;
%put &=i &=list;
%end;
%mend test;
%test
Results:
I=1 LIST=lag1(var1),lag2(var1),lag3(var1),lag4(var1),lag5(var1)
I=2 LIST=lag1(var2),lag2(var2),lag3(var2),lag4(var2),lag5(var2)
I=3 LIST=lag1(var3),lag2(var3),lag3(var3),lag4(var3),lag5(var3)
I=4 LIST=lag1(var4),lag2(var4),lag3(var4),lag4(var4),lag5(var4)
I=5 LIST=lag1(var5),lag2(var5),lag3(var5),lag4(var5),lag5(var5)
For your actual problem you might want to create a macro that only returns the comma delimited list as the result of the macro call.
%macro lags(varname,first,last);
%local lag dlm;
%do lag= &first %to &last ;
%if (&lag > 0) %then %*;&dlm.lag&lag(&varname);
%else %*;&dlm.&varname;
%let dlm=,;
%end;
%mend lags;
%put var1_ma_past_1=mean(%lags(var1,0,5));
%put var1_ma_past_2=mean(%lags(var1,1,6));
%put var1_ma_past_3=mean(%lags(var1,2,7));
%put var2_ma_past_1=mean(%lags(var2,0,5));
Why you are getting those error messages:
The %sysfunc() macro function needs to try to figure out whether each argument is character or numeric for a function like CATX() that can operate on either type of input. That is why the () in the argument values is confusing it since it looks like you are trying to pass a numeric expression.
18 %put %sysfunc(catx(|,a(b),b));
ERROR: Required operator not found in expression: a(b)
a(b)|B
19 %put %sysfunc(catx(|,(1+2),b));
3|B
You could force in quotes around the values and then remove them later (if your values don't actually contain quotes).
%let left=A(b);
%let right=b;
%let intermediate=%sysfunc(catx(|,"&left","&right"));
%let want=%sysfunc(compress(&intermediate,%str(%"));
%put &=want;
I'm having an issue reading a CSV file into a SAS dataset without bringing each field with my import. I don't want every field imported, but that's the only way I can seem to get this to work. The issue is I cannot get SAS to read my data correctly, even if it's reading the columns correctly... I think part of the issue is that I have data above my actual column headers that I don't want to read in.
My data is laid out like so
somevalue somevalue somevalue...
var1 var2 var3 var4
abc abc abc abc
Where I want to exclude somevalue, only read in select var's and their corresponding data.
Below is a sample file where I've scrambled all the values in my fields. I only want to keep COLUMN H(8), AT(46) and BE(57)
Here's some code I've tried so far...
This was SAS generated from a PROC IMPORT. My PROC IMPORT worked fine to read in every field value, so I just deleted the fields that I didn't want, but I don't get the output I expect. The values corresponding to the fields does not match.
A) PROC IMPORT
DATAFILE="C:\Users\dip1\Desktop\TU_&YYMM._FIN.csv"
OUT=TU_&YYMM._FIN
DBMS=csv REPLACE;
GETNAMES=NO;
DATAROW=3;
RUN;
generated this in the SAS log (I cut out the other fields I didn't want)
B) DATA TU_&YYMM._FIN_TEST;
infile 'C:\Users\fip1\Desktop\TU_1701_FIN.csv' delimiter = ',' DSD lrecl=32767
firstobs=3 ;
informat VAR8 16. ;
informat VAR46 $1. ;
informat VAR57 $22. ;
format VAR8 16. ;
format VAR46 $1. ;
format VAR57 $22. ;
input
VAR8
VAR46 $
VAR57 $;
run;
I've also tried this below... I believe I'm just missing something..
C) DATA TU_TEST;
INFILE "C:\Users\fip1\Desktop\TU_&yymm._fin.csv" DLM = "," TRUNCOVER FIRSTOBS = 3;
LABEL ACCOUNT_NUMBER = "ACCOUNT NUMBER";
LENGTH ACCOUNT_NUMBER $16.
E $1.
REJECTSUBCATEGORY $22.;
INPUT ACCOUNT_NUMBER
E
REJECTSUBCATEGORY;
RUN;
As well as trying to have SAS point to the columns I want to read in, modifying the above to:
D) DATA TU_TEST;
INFILE "C:\Users\fip1\Desktop\TU_&yymm._fin.csv" DLM = "," TRUNCOVER FIRSTOBS = 3;
LABEL ACCOUNT_NUMBER = "ACCOUNT NUMBER";
LENGTH ACCOUNT_NUMBER $16.
E $1.
REJECTSUBCATEGORY $22.;
INPUT #8 ACCOUNT_NUMBER
#46 E
#57 REJECTSUBCATEGORY;
RUN;
None of which work. Again, I can do this successfully if I bring in all of the fields with either A) or B), given that B) includes all the fields, but I can't get C) or D) to work, and I want to keep the code to a minimum if I can. I'm sure I'm missing something, but I've never had time to tinker with it so I've just been doing it the "long" way..
Here's a snippet of what the data looks like
A(1) B(2) C(3) D(4) E(5) F(6) G(7)
ABCDEFGHIJ ABCDMCARD 202020 4578917 12345674 457894A (blank)
CRA INTERNALID SUBCODE RKEY SEGT FNM FILEDATE
CREDITBUR 2ABH123 AB2CHE123 A28O5176688 J2 Name 8974561
With a delimited file you need to read all of the fields (or at least all of the fields up to the last one you want to keep) even if you do not want to keep all of those fields. For the ones you want to skip you can just read them into a dummy variable that you drop. Or even one of the variables you want to keep that you will overwrite by reading from a later column.
Also don't model your DATA step after the code generated by PROC IMPORT. You can make much cleaner code yourself. For example there is no need for any FORMAT or INFORMAT statements for the three variables you listed. Although if VAR8 really needs 16 digits you might want to attach a format to it so that SAS doesn't use BEST12. format.
data tu_&yymm._fin_test;
infile 'C:\Users\fip1\Desktop\TU_1701_FIN.csv'
dlm=',' dsd lrecl=32767 truncover firstobs=3
;
length var8 8 var46 $1 var57 $22 ;
length dummy $1 ;
input 7*dummy var8 37*dummy var46 10*dummy var57 ;
drop dummy ;
format var8 16. ;
run;
You can replace the VARxx variable names with more meaningful ones if you want (or add a RENAME statement). Using the position numbers here just makes it clearer in this code that the INPUT statement is reading the 57 columns from the input data.
New to this, so apologies. I have a file in SAS that I need to export as a CSV and I need to add double quotes to all fields. How can I accomplish this? Thanks in advance.
There are many ways to create a CSV file from SAS. Using proc export won't wrap every field in double-quotes, so the easiest one that will do this is the %ds2csv() macro. This assumes you have SAS 9.2 or later. The documentation for it can be found here:
http://support.sas.com/documentation/cdl/en/lrdict/64316/HTML/default/viewer.htm#a002683390.htm
An example of running it:
%ds2csv(data=sashelp.retail, runmode=b, csvfile=c:\class.csv);
Produces:
"Retail sales in millions of $","DATE","YEAR","MONTH","DAY"
"$220","80Q1","1980","1","1"
"$257","80Q2","1980","4","1"
"$258","80Q3","1980","7","1"
Here is another one using data step
data _null_;
file 'data_want.csv' dsd dlm = ',';
set data_have;
if _n_ = 1 then put #1 "variables name to input"; /* Add variable names */
put (_all_) (~);
run;
Here is a way to get the same result with headers included
proc contents data=sashelp.class out=vars(keep=name varnum);run;
proc sql noprint;
select quote(strip(name)) into :vars separated by '|' from vars
order by varnum;
quit;
data _null_;
file 'test.txt' dsd dlm='|';
set sashelp.class;
if _n_ = 1 then put %sysfunc(quote(&vars.));
put (_all_) (~);
run;
I know it's an old question, but my 2 pence...
The macro below does not quote numeric values that are formatted to a character value, but it's easy to implement.
%macro csvQuoteChars(lib=,dset=,outPath=);
proc contents data=&lib..&dset out=c(keep=name type) noprint;
run;
%let dsid = %sysfunc(open(c));
%let nobs = %sysfunc(attrn(&dsid,nlobs));
data _null_;
set &lib..&dset;
file "&outPath\&lib..&dset..csv" dsd;
%do i=1 %to &nobs;
%let rc = %sysfunc(fetchobs(&dsid,&i));
%let vName = %sysfunc(getvarc(&dsid,1));
%let vType = %sysfunc(getvarn(&dsid,2));
%put &=vName &=vType;
%if &vType = 2 %then %do;
put (&vName) (~) #;
%end;
%else %if &vType = 1 %then %do;
put &vName #;
%end;
%if &i = &nobs %then %do;
put;
%end;
%end;
run;
%let dsid = %sysfunc(close(&dsid));
%mend csvQuoteChars;
%csvQuoteChars(lib=sashelp,dset=class,outPath=d:\temp);
I have many csv files with many variable column headers , up to 2000 variable column headers for some files.
I'm trying to do an import but at one point , the headers are truncated in a 'random' manner and the rest of the data are ignored therefore not imported. I'm putting random between quotes because it may not be random although I don't know the reason if it is not random. But let me give you more insight .
The headers are truncated randomly , some after the 977th variables , some others after the 1401th variable.
The headers are like this BAL_RT,ET-CAP,EXT_EA16,IVOL-NSA,AT;BAL_RT,ET-CAP,EXT_EA16,IVOL-NSA,AT;BAL_RT,ET-CAP,EXT_EA16,IVOL-NSA,AT
This the part of the import log
642130 VAR1439
642131 VAR1440
642132 VAR1441
642133 VAR1442 $
642134 VAR1443 $
642135 VAR1444 $
As you can see , some headers are seen as numeric although all the headers are alphanumeric as they are blending a mixture of character and numeric.
Please find my code for the import below
%macro lec ;
options macrogen symbolgen;
%let nfic=37 ;
%do i=1 %to &nfic ;
PROC IMPORT OUT= fic&i
DATAFILE= "C:\cygwin\home\appEuro\pot\fic&i..csv"
DBMS=DLM REPLACE;
DELIMITER='3B'x;
guessingrows=500 ;
GETNAMES=no;
DATAROW=1;
RUN;
data dico&i ; set fic&i (drop=var1) ;
if _n_ eq 1 ;
index=0 ;
array v var2-var1000 ;
do over v ;
if v ne "" then index=index+1 ;
end ;
run ;
data dico&i ; set dico&i ;
call symput("nvar&i",trim(left(index))) ;
run ;
%put &&nvar&i ;
%end ;
%mend ;
%lec ;
The code is doing an import and also creating a dictionnary with the headers as some of them are long (e.g more than 34 characters)
I'm not sure if these elements are related however, I would welcome any insights you will be able to give me.
Best.
You need to not use PROC IMPORT, as I mentioned in a previous comment. You need to construct your dictionary from a data step read in, because if you have 2000 columns times 34 or more long variable names, you will have more than 32767 record length.
An approach like this is necessary.
data headers;
infile "whatever" dlm=';' lrecl=99999 truncover; *or perhaps longer even, if that is needed - look at log;
length name $50; *or longer if 50 is not your real absolute maximum;
do until length(_infile_)=0;
input name $ #;
output;
end;
stop; *only want to read the first line!;
run;
Now you have your variable names. Now, you can read the file in with GETNAMES=NO; in proc import (you'll have to discard the first line), and then you can use that dictionary to generate rename statements (you will have lots of VARxxxx, but in a predictable order).
I have the following csv files and I need to read them into SAS datasets.
Also, I need to assign column names.
In addition, I need the column to be numberic but some columns have both number and character values.
folder aa: abc1.csv, abc2.csv, abc3.csv,......
folder bb: abc1.csv, abc2.csv, abc3.csv,......
folder cc: abc1.csv, abc2.csv, abc3.csv,......
You could do it in following way also.
Keep all your files in one folder.
Name all of them in a csv file with only one column.
Import the file (csv) with file names into SAS.
Create a macro to keep their name with "into" clause.
proc sql;
select name_list into :name separated by '*' from work.name;
%let count2 = &sqlobs;
quit;
Create a macro like below.
%macro yy;
%do i = 1 %to &count2;
%let j = %scan(&name,&i,*);
proc import out = &j datafile="folderwhereallcsvfilesarekept\&j..csv"
dbms=csv replace;
getnames = yes;
run;
%end;
%mend;
This isn't a complete answer, but it will get you started. You'll have to add an outer loop to go through the different directories you want to get files from.
/*List the files in a directory for use in a data step. This is written for Windows. Other operating systems will be slightly different. */
filename fnames pipe 'dir c:\temp\* /b';
/* Create a data set with one observation for each file name */
data fnames;
infile fnames pad missover;
input #1 filename $255.;
n=_n_;
run;
/* Store the number of files in a macro variable "num" */
proc sql noprint; select count(filename) into :num; quit;
/* Create a macro to iterate over the filenames, read them in, and append to a data set. */
%macro doit;
%do i=1 %to #
proc sql noprint;
select filename into :filename from fnames where n=&i;
quit;
data ds;
infile &filename;
input ...list of variable names...;
...other statements...;
run;
proc append data=ds base=final; run;
%end;
%mend;
%doit;