I am currently using SuperCSV (v 2.4.0) for exporting data.
I am stuck in with the below problem.
※My problem:
- My CSV header has 9 columns
-There is a line which missing a comma dilimiter (,) between two fields, so that line become a 8-columns line
- The messing line is occur randomly, not exact the same line for every exporting.
※Here my code:
① CSV Writing function
// obmitting
stmt = con.prepareStatement(sql);
rs = stmt.executeQuery();
// obmitting
// ............
fos = new FileOutputStream(folderPath + "/" + FILE_NAME);
osw = new OutputStreamWriter(fos, "EUC-JP");
beanWriter = new CsvBeanWriter(osw, csvType);
beanWriter.writeHeader(this.CSV_HEADER);
while(rs.next()) {
AttachmentsBean bean = new AttachmentsBean();
bean.setCompDispId(pMng.getCompDispId());
bean.setFileId(rs.getString(1));
bean.setDocumentId(rs.getString(2));
bean.setFileName(rs.getString(3));
bean.setDiscription(rs.getString(4));
bean.setFileSize(rs.getString(5));
bean.setFilePath(rs.getString(6));
bean.setMimetype(rs.getString(7));
bean.setFileOrder(rs.getString(8));
progressCount++;
//出力
beanWriter.write(bean, this.CSV_HEADER);
}
//CSV出力終了
IOUtils.closeQuietly(beanWriter);
IOUtils.closeQuietly(osw);
IOUtils.closeQuietly(fos);
//文字変換
CommonUtil.convertFile(folderPath, folderPath +"/"+ FILE_NAME);
② ConvertFile helper function
this function is convert special character from EUC to UTF-8
※My question is, is my problem caused by superCSV bug ? can anyone help to figure out the cause ?
because i have read from the below discussion, maybe not only i meet the mess up csv problem, but some one did too.
Messed up CSV leads to Exception
Related
I currently have a flat file with around 1million rows.
I need to add a text string to the end of each row in the file.
I've been trying to adapt the following code but not having any success :-
public void Main()
{
// TODO: Add your code here
var lines = System.IO.File.ReadAllLines(#"E:\SSISSource\Source\Source.txt");
foreach (string item in lines)
{
var str = item.Replace("\n", "~20221214\n");
var subitems = str.Split('\n');
foreach (var subitem in subitems)
{
// write the data back to the file
}
}
Dts.TaskResult = (int)ScriptResults.Success;
}
I can't seem to get the code to recognise the carriage return "\n" & am not sure howto write the row back to the file to replace the existing rather than add a new row. Or is the above code sending me down a rabbit hole & there is an easier method ??
Many thanks for any pointers &/or assistance.
Read all lines is likely getting rid of the \n in each record. So your replace won't work.
Simply append your string and use #billinKC's solution otherwise.
BONUS:
I think DateTime.Now.ToString("yyyyMMdd"); is what you are trying to append to each line
Thanks #billinKC & #KeithL
KeithL you were correct in that the \n was stripped off. So I used a slightly amended version of #billinKC's code to get what I wanted :-
string origFile = #"E:\SSISSource\Source\Sourcetxt";
string fixedFile = #"E:\SSISSource\Source\Source.fixed.txt";
// Make a blank file
System.IO.File.WriteAllText(fixedFile, "");
var lines = System.IO.File.ReadAllLines(#"E:\SSISSource\Source\Source.txt");
foreach (string item in lines)
{
var str = item + "~20221214\n";
System.IO.File.AppendAllText(fixedFile, str);
}
As an aside KeithL - thanks for the DateTime code however the text that I am appending is obtained from a header row in the source file which is being read into a variable in an earlier step.
I read your code as
For each line in the file, replace the existing newline character with ~20221214 newline
At that point, the value of str is what you need, just write that! Instead, you split based on the new line which gets you an array of values which could be fine but why do the extra operations?
string origFile = #"E:\SSISSource\Source\Sourcetxt";
string fixedFile = #"E:\SSISSource\Source\Source.fixed.txt";
// Make a blank file
System.IO.File.WriteAllText(fixedFile, "");
var lines = System.IO.File.ReadAllLines(#"E:\SSISSource\Source\Source.txt");
foreach (string item in lines)
{
var str = item.Replace("\n", "~20221214\n");
System.IO.File.AppendAllText(fixedFile, str);
}
Something like this ought to be what you're looking for.
As the title says, I have been trying to write data that the user enters into a CEdit control to a file.
The system is a handheld terminal running Windows CE, in which my test application is running, and I try to enter test data (Japanese characters in Romaji, Hiragana, Katakana and Kanji mixed along with normal English alphanumeric data) that initially is displayed in a CListCtrl. The characters display properly on the handheld display screen in my test application UI.
Finally, I try to read back the data from the List control and write it to a text CSV file. The data I get on reading back from the control is correct, but on writing it to the CSV, things mess up and my CSV file is unreadable and shows strange symbols and nonsense alphanumeric garbage.
I searched about this, and I ended up with a similar question on stackOverflow:
UTF-8, CString and CFile? (C++, MFC)
I tried some of their suggestions and finally ended up with a proper UTF-8 CSV file.
The write-to-csv-file code goes like this:
CStdioFile cCsvFile = CStdioFile();
cCsvFile.Open(cFileName, CFile::modeCreate|CFile::modeWrite);
char BOM[3]={0xEF, 0xBB, 0xBF}; // Utf-8 BOM
cCsvFile.Write(BOM,3); // Write the BOM first
for(int i = 0; i < M_cDataList.GetItemCount(); i++)
{
CString cDataStr = _T("\"") + M_cDataList.GetItemText(i, 0) + _T("\",");
cDataStr += _T("\"") + M_cDataList.GetItemText(i, 1) + _T("\",");
cDataStr += _T("\"") + M_cDataList.GetItemText(i, 2) + _T("\"\r\n");
CT2CA outputString(cDataStr, CP_UTF8);
cCsvFile.Write(outputString, ::strlen(outputString));
}
cCsvFile.Close();
So far it is OK.
Now, for my use case, I would like to change things a bit such that the CSV file is encoded as Shift-JIS, not UTF-8.
For Shift-JIS, what BOM do I use, and what changes should I make to the above code?
Thank you for any suggestions and help.
Codepage for Shift-JIS is apparently 932. Use WideCharToMultiByte and MultiByteToWideChar for conversion. For example:
CStringW source = L"日本語ABC平仮名ABCひらがなABC片仮名ABCカタカナABC漢字ABC①";
CStringA destination = CW2A(source, 932);
CStringW convertBack = CA2W(destination, 932);
//Testing:
ASSERT(source == convertBack);
AfxMessageBox(convertBack);
As far as I can tell there is no BOM for Shift-JIS. Perhaps you just want to work with UTF16. For example:
CStdioFile file;
file.Open(L"utf16.txt", CFile::modeCreate | CFile::modeWrite| CFile::typeUnicode);
BYTE bom[2] = { 0xFF, 0xFE };
file.Write(bom, 2);
CString str = L"日本語";
file.WriteString(str);
file.Close();
ps, according to this page there are some problems between codepage 932 and Shift-JIS, although I couldn't duplicate any errors.
I'm writing a trivia game app in C# that writes data to a binary file, then reads the data from the file into six labels. The six labels are as follows:
lblQuestion // This is where the question text goes.
lblPoints // This is where the question points goes.
lblAnswerA // This is where multiple choice answer A goes.
lblAnswerB // This is where multiple choice answer B goes.
lblAnswerC // This is where multiple choice answer C goes.
lblAnswerD // This is where multiple choice answer D goes.
Here is the code for writing to the binary file:
{
bw.Write(Question);
bw.Write(Points);
bw.Write(AnswerA);
bw.Write(AnswerB);
bw.Write(AnswerC);
bw.Write(AnswerD);
}
Now for the code to read data from the file into the corresponding labels:
{
FileStream fs = File.OpenRead(ofd.FileName);
BinaryReader br = new BinaryReader(fs);
lblQuestion.Text = br.ReadString();
lblPoints.Text = br.ReadInt32() + " points";
lblAnswerA.Text = br.ReadString();
lblAnswerB.Text = br.ReadString();
lblAnswerC.Text = br.ReadString();
lblAnswerD.Text = br.ReadString();
}
The Question string reads correctly into lblQuestion.
The Points value reads correctly into lblPoints.
AnswerA, AnswerB, and AnswerC DO NOT read into lblAnswerA, lblAnswerB and lblAnswerC respectively.
lblAnswerD, however, gets the string meant for lblAnswerA.
Looking at the code for reading data into the labels, is there something missing, some sort of incremental value that needs to be inserted into the code in order to get the strings to the correct labels?
I've accidentally came across this weird buggy phenomenon when writing code with Nodejs.
I get input from a form, which are comprised of a series of ID's,
delimited with New Line(hit Enter)
To handle them, I use the split function to split them into an array:
var array = orig_input.split('\n');
For the debugging purpose,I print them.
console.log('orig_input = ' + input);
console.log('array = ' + array);
Well, here comes the first weird output:
orig_input = 1111
2222
,2222
Look! the supposed output string 'array = 1111' is swallowed !
My first thought is array[0] has some delete characters.
So I change the code to do some tests, like:
console.log('orig_input = ' + input);
console.log('***********************************' + array[0] + array[1]);
Wow, yet another weird output:
orig_input = 1111
2222
2222*******************************1111
It seems there is some weird wrap or something.
And finally, after some boring testing and debugging, it turns out that
I used the wrong split character, it should be '\r\n', instead of '\n'
We know that the former is the MS-style of new line, and the latter is the *nix-style of new line.
So question is
Is this a bug of console.log function in **Nodejs** ?
Anyone would explain this?
I've read through the Highcharts how-to, checked the demo galleries, searched google, read the X amount of exact similar threads here on stackoverflow yet I cannot get it to work.
I'm logging data in a csv file in the form of date,value.
Here's what the date looks like
1355417598678,22.25
1355417620144,22.25
1355417625616,22.312
1355417630851,22.375
1355417633906,22.437
1355417637134,22.437
1355417641239,22.5
1355417641775,22.562
1355417662373,22.125
1355417704368,21.625
And this is how far I've managed to get the code:
http://jsfiddle.net/whz7P/
This renders a chart, but with no series or data at all. I think I'm fudging things up while formatting the data so it can be interpreted in highcharts.
Anyone able to give a helping hand?
So, you have the following data structure, right ?
1355417598678,22.25
1355417620144,22.25
1355417625616,22.312
1355417630851,22.375
1355417633906,22.437
1355417637134,22.437
1355417641239,22.5
1355417641775,22.562
1355417662373,22.125
1355417704368,21.625
Then you split it into an array of lines, so each array item is a line.
Then for each line you do the following.
var items = line.split(';'); // wrong, use ','
But there ins't ; into the line, you should split using ,.
The result will be a multidimencional array which each item is an array with the following structure. It will be stored in a var named data.
"1355417598678","22.25" // date in utc, value
This is the expected data for each serie, so you can pass it directly to your serie.
var serie = {
data: data,
name: 'serie1' // chose a name
}
The result will be a working chart.
So everything can be resumed to the following.
var lines = data.split('\n');
lines = lines.map(function(line) {
var data = line.split(',');
data[1] = parseFloat(data[1]);
return data;
});
var series = {
data: lines,
name: 'serie1'
};
options.series.push(series);
Looking at your line.split part:
$.get('data.csv', function(data) {
// Split the lines
var lines = data.split('\n');
$.each(lines, function(lineNo, line) {
var items = line.split(';');
It looks like you are trying to split on a semi-colon (;) instead of a comma (,) which is what is in your sample CSV data.
You need to put
$(document).ready(function() {
in the 1st line, and
});
in the last line of the javascript to make this work.
Could you upload your csv file? Is it identical to what you wrote in your original post? I ran into the same problem, and it turns out there are errors in the data file.