Efficiently reading csv file in MQL4? - csv

Question

I placed "AAPL.csv" into the MetaTrader terminal folder's Files subfolder (MQL4/Files) to be accessible by the EA. The structure of this csv is as follows:
Date,Open,High,Low,Close,Adj Close,Volume
1980-12-12,0.1283479928970337,0.1289059966802597,0.1283479928970337,0.1283479928970337,0.10092189908027649,469033600
1980-12-15,0.12221000343561172,0.12221000343561172,0.12165199965238571,0.12165199965238571,0.09565676748752594,175884800
I want to read this, as well as many other similar csv files. All files have different lengths. My question is, what is the best practice when reading variable-length files? For now, I managed to read the content of my file by creating a 2-dimensional array:
string s[7][1000000];
although this is poor programming (what if the file only has 500 rows?) and it can still fail if I encounter a csv that is longer (what if file has 1000001 rows?). I tried using a dynamic array:
string s[7][];
but it returns '[' - invalid index value error. Yet another idea I had to use the FileSize() function and allocate just-the-necessary amount of memory to the 2-dimensional array. However,
int handle=FileOpen(FileName,FILE_CSV|FILE_READ,",");
if(handle>0)
{
int size = FileSize(handle);
...
yielded a size that equals the product of column numbers and row numbers. I was hoping to obtain a row_count and col_count and use them to define s:
string s[col_count][row_count];
My full working code:
extern string FileName = "AAPL.csv";
int init()
{
int row=0,col=0;
string s[7][1000000];
ResetLastError();
int handle=FileOpen(FileName,FILE_CSV|FILE_READ,",");
if(handle>0)
{
while(True)
{
string temp = FileReadString(handle);
if(FileIsEnding(handle)) break; //FileIsEnding = End of File
s[col][row]=temp;
if(FileIsLineEnding(handle)) //FileIsLineEnding = End of Line
{
col = 0; //reset col = 0 for the next row
row++; //next row
}
else
{
col++; //next col of the same row
}
}
FileClose(handle);
}
else
{
Comment("File "+FileName+" not found, the last error is ", GetLastError());
}
return(0);
}
int start()
{
return(0);
}

Answer 1

You should use the first dimension of the array for your rows and the second dimension for your columns. You would therefore have an array statement of string s[1][7];
You can then resize your array as you loop through reading your csv file as follows:
int handle=FileOpen(FileName,FILE_CSV|FILE_READ,",");
if(handle==INVALID_HANDLE) Print("Error opening file ",GetLastError());
else
{
int row=0;
while(!FileIsEnding(handle))
{
if(row>0) ArrayResize(s,row+1);
//carry out array reading here
//eg s[row][0]=FileReadString(handle);
//eg s[row][1]=FileReadString(handle);
//etc
row++;
}
FileClose(handle);
}
You can not resize the 2nd dimension of a multi dimensional array.