Related
I am having issues with loading a CSV into a StringGrid. Occasionally, it runs out of memory, but also it seems to have blank columns after each value. I've not really read from a CSV as opposed to output to one, so I took a stock example online and modified it for my needs.
This is what I've currently got:
procedure x.LoadCSVtoGrid(ACSVFile : String; AStringGrid: TStringGrid)
var
LRowIndex, LColIndex: Integer;
LStrLine: string;
LFile: TStringList;
begin
AStringGrid.RowCount := 0;
AStringGrid.ColCount := 0;
if not FileExists(ACSVFile) then
exit;
LFile := TStringList.Create;
try
LFile.LoadFromFile(ACSVFile);
if LFile.Count = 0 then
exit;
AStringGrid.ColCount := Max(AStringGrid.ColCount, WordCount(LFile[0], [',', '"'], '"'));
AStringGrid.RowCount := LFile.Count;
for LRowIndex := 0 to LFile.Count - 1 do
begin
LStrLine := LFile[LRowIndex];
LColIndex := 0;
while LStrLine <> '' do
begin
if Pos('"', LStrLine) = 1 then
begin
Delete(LStrLine, 1, 1);
AStringGrid.Cells[LColIndex, LRowIndex] := Copy(LStrLine, 1, Pos('"', LStrLine) - 1);
Delete(LStrLine, 1, Pos('"', LStrLine));
end
else
begin
AStringGrid.Cells[LColIndex, LRowIndex] := Copy(LStrLine, 1, Pos(',', LStrLine) - 1);
Delete(LStrLine, 1, Pos(',', LStrLine));
end;
Inc(LColIndex);
end;
end;
finally
LFile.Free;
end;
For smaller CSV files, it does fine. I think it's reading up to 250-300 lines before. Some of the files it has to deal with now are 500+.
To be honest, I don't do much handling of the data of the CSV until it's been imported into the StringGrid, but once it's in the StringGrid, it's validated. I've got to make sure that commas within speech marks, ie "text, here", are ignored, as it's part of the value. Again, this appears to handle the reading fine.
Another issue I think I might run into is AStringGrid.RowCount := LFile.Count;, as some of the CSV files have blank lines. If there is a way to deal with this, I am happy to take suggestions.
There are a few versions of CSV files it should be able to read, ie the calculation of column counts and such. Code for WordCount:
function x.WordCount(const S: string; const WordDelims: TSysCharSet; const QuoteChar: Char) : Integer;
var
LInWord: Boolean;
LQuoteOpen: Boolean;
i: Integer;
begin
Result := 0;
LInWord := False;
LQuoteOpen := False;
for i := 1 to Length(S) do
begin
if S[i] in WordDelims then
begin
if not LInWord or LQuoteOpen then
LInWord := False
else
begin
LInWord := True;
Inc(Result);
end;
end
else
begin
if S[i] = QuoteChar then
LQuoteOpen := not LQuoteOpen;
LInWord := True;
end;
end;
if LInWord and (not LQuoteOpen) then
Inc(Result);
I've tried multiple files, for the most part this issue only happens with larger CSV files with more content. I've tried various versions of CSV-to-StringGrid procedures to see if there is something innately wrong with the example I took above. The example works, but only on smaller files.
Let me know if you need more information.
Memory issue
First you create a TStringList and then load it with data
LFile := TStringList.Create;
LFile.LoadFromFile(ACSVFile);
Because you load the whole file into the string list, you need that much of memory, plus equally much to hold the data in the TStringGrid.
Reduce memory requirement by reading the file in chunks of, say, 1000 lines at the time, which you then can throw away after they are moved to the string grid.
OTOH, your "Out of memory" problem might also be caused by the errors in your code. I experienced an "Out of memory" error with my very small test file when run with your unaltered code.
Issues with code
In my tests I used a simple file with a few records and a quoted field in different locations. The file content is:
one,two,"text, including comma",four,five
one,two,three,four,five
"text, including comma",two,three,four,five
one,two,three,four,"text, including comma"
You determine required number of columns in the TStringGrid, by calling the WordCount() function, to which you pass the first string from the string list.
WordCount(const S: string; const WordDelims: TSysCharSet; const QuoteChar: Char) : Integer;
When I pass in the first test string,
'one,two,three,four,five',
WordCount returns correctly 5
Then, control returns to LoadCSVtoGrid(), and after assigning AStringGrid.ColCount and RowCount the for LRowIndex loop starts to fill the grid with data for the current row. Pay attention to the second part, after else:
AStringGrid.Cells[LColIndex, LRowIndex] := Copy(LStrLine, 1, Pos(',', LStrLine) - 1);
Delete(LStrLine, 1, Pos(',', LStrLine));
The Delete() deletes from beginning of LStrLine to Pos(',', LStrLine). This works ok for items "one,", "two,", "three," and "four,", but not for "five" as there is no comma after the last item.
This is the major flaw in the code as it never deletes the last item. Instead, since the loop runs while LString <> '' it just continues incrementing LColIndex
On my machine it stops after a couple of minutes with an out-of-memory error.
Here is my take on WordCount (renamed WordCountNew) function:
function TForm50.WordCountNew(const s: string; const Delimiter: Char;
const QuoteChar: Char): Integer;
var
InWord, InQuote: boolean;
i: integer;
begin
if s = '' then // Just in case we are fed an empty string
Exit(0);
Result := 1; // Init, at least one data item
InWord := False; // Init
InQuote:= False; // Init
for i := 1 to Length(s) do
begin
if s[i] = QuoteChar then // The field is quoted
InQuote := not InQuote; // make note about it
if s[i] = Delimiter then // Delimiter found
begin
if not InQuote then // ... but only count it,
inc(Result); // if not within a quote
end;
end;
end;
Then the LoadCSVtoGrid procedure:
procedure TForm50.LoadCSVtoGrid(ACSVFile: String; AStringGrid: TStringGrid);
var
LRowIndex, LColIndex: Integer;
LStrLine: string;
LFile: TStringList;
CommaPos: integer; // added
begin
AStringGrid.RowCount := 0;
AStringGrid.ColCount := 0;
if not FileExists(ACSVFile) then
exit;
LFile := TStringList.Create;
try
LFile.LoadFromFile(ACSVFile);
if LFile.Count = 0 then
exit;
// When determining column count we should ONLY count the field separator, comma.
// A quote character is not an indication of a new column / field.
// Therefore we remove the array of chars, `[',', '"']` and replace with `','`
// AStringGrid.ColCount := Max(AStringGrid.ColCount, WordCount(LFile[0], [',', '"'], '"'));
AStringGrid.ColCount := Max(AStringGrid.ColCount, WordCountNew(LFile[0], ',', '"'));
AStringGrid.RowCount := LFile.Count;
for LRowIndex := 0 to LFile.Count - 1 do
begin
LStrLine := LFile[LRowIndex];
LColIndex := 0;
while LStrLine <> '' do
begin
if Pos('"', LStrLine) = 1 then
begin
Delete(LStrLine, 1, 1);
AStringGrid.Cells[LColIndex, LRowIndex] := Copy(LStrLine, 1, Pos('"', LStrLine) - 1);
AStringGrid.UpdateControlState;
Delete(LStrLine, 1, Pos('"', LStrLine));
Delete(LStrLine, 1, Pos(',', LStrLine));
end
else
begin
CommaPos := Pos(',', LStrLine);
if CommaPos = 0 then CommaPos := Length(LStrLine)+1;
AStringGrid.Cells[LColIndex, LRowIndex] := Copy(LStrLine, 1, CommaPos-1); //Pos(',', LStrLine) - 1);
AStringGrid.UpdateControlState;
Delete(LStrLine, 1, CommaPos); // Pos(',', LStrLine));
end;
Inc(LColIndex);
end;
end;
finally
LFile.Free;
end;
end;
I added the CommaPos variable, to make it easier to artificially simulate a comma at the end of the string.
With these changes the test file is properly read into the grid.
I have a Delphi XE2 application with a TEmbeddedWB that I use to simulate user actions. The application navigates to a URL, populates the relevant form fields with data and submits the data. The problem is that there is an <input type=file /> field which accepts files that are uploaded.
Having done a lot of reading on the matter I understand there is a security issue doing this programmatically but also found someone making a suggestion that the files could be ‘dragged’ from the clipboard and ‘dropped’ in place. I have since been successful in loading the relevant files (jpeg images) into the clipboard (thanks to CCR.Clipboard) and drop them onto my EmbeddedWB. However, as you are most likely aware, dropping an image on a TWebBrowser resorts to the image being displayed.
My issue is that the web page I’m accessing has a specific DIV element that accepts files to be dropped. Although I have successfully obtained the coordinates of that DIV as an IHTMLElement and even moved the mouse cursor into position (for visual confirmation), dropping an image there still opens it for display instead of uploading it. It’s as though the drop area doesn’t detect the drop, only the web browser does.
Any guidance on this matter will be greatly appreciated. Following is the relevant code.
Methods:
type
TElementsArray = array of IHTMLElement;
...
function TSiteRobot.FindElementByTagAttributeValue(const Document: IHTMLDocument2; TagName, Attribute, AttributeValue: String; out Info: String): IHTMLElement;
var i: integer;
HTMLElem: IHTMLElement;
ElementCount: integer;
OleElem: OleVariant;
ElementsArray: TElementsArray;
begin
Result := nil; //initialise
ElementsArray := GetElementsByTagName(Document, TagName);
if Length(ElementsArray) = 0 then
begin
Info := 'No elements with "'+TagName+'" tag found.';
Exit
end;
Info := 'No element found for tag "'+TagName+'" and attribute "'+Attribute+'" with Value "'+AttributeValue+'"';
for i := Low(ElementsArray) to High(ElementsArray) do
begin
HTMLElem := ElementsArray[i];
try
OleElem := HTMLElem.getAttribute(Attribute,0);
if (not varIsClear(OleElem)) and (OleElem <> null) then
begin
if (String(OleElem) = AttributeValue) then
begin
if HTMLElem <> nil then Result := HTMLElem;
Break;
end;
end;
except raise; end;
end;
end;
function TSiteRobot.GetElementScreenPos(WebBrowser: TEmbeddedWB; HTMLElement: IHTMLElement): TPoint;
var WinRect: TRect;
elTop, elLeft: integer;
HTMLElem2: IHTMLElement2;
begin
HTMLElement.scrollIntoView(True);
Application.ProcessMessages; //let the coordinates get updated since the page moved
GetWindowRect(WebBrowser.Handle, WinRect);
HTMLElem2 := (HTMLElement as IHTMLElement2);
elLeft := HTMLElem2.getBoundingClientRect.left + WinRect.Left;
elTop := HTMLElem2.getBoundingClientRect.top + WinRect.Top;
Result := Point(elLeft, elTop);
end;
procedure TfrmMain.DropFilesAtPoint(Area: TPoint; Wnd: HWND);
var DropTarget: IDropTarget;
DataObj: IDataObject;
DropFiles: PDropFiles;
StgMed: TSTGMEDIUM;
FormatEtc: TFORMATETC;
EnumFormatEtc: IEnumFORMATETC;
dwEffect: integer;
begin
DropTarget := IDropTarget(GetProp(Wnd, 'OleDropTargetInterface'));
OleGetClipboard(dataObj);
DataObj.EnumFormatEtc(DATADIR_GET, EnumFormatEtc);
while (EnumFormatEtc.Next(1, FormatEtc, nil) <> S_FALSE) do
begin
if (FormatEtc.cfFormat = CF_HDROP) and (DataObj.QueryGetData(FormatEtc) = S_OK) then
begin
DataObj.GetData(FormatEtc, StgMed);
DropFiles := GlobalLock(StgMed.hGlobal);
dwEffect := DROPEFFECT_COPY;
DropTarget.Drop(DataObj, Integer(DropFiles), Area, dwEffect); // This is where the image opens in the web browser
GlobalFree(StgMed.hGlobal);
ReleaseStgMedium(StgMed);
end;
end; //while
DataObj._Release;
end;
Calling Code:
var HTMLElem: IHTMLElement;
dndArea: TPoint;
…
HTMLElem := SiteRobot.FindElementByTagAttributeValue(Document, 'SPAN', 'id', 'dndArea', Info);
dndArea := SiteRobot.GetElementScreenPos(WebBrowser, HTMLElem);
dndArea.X := dndArea.X+24; //go ‘deeper’ into the drop area
dndArea.Y := dndArea.Y+24;
SetCursorPos(dndArea.X, dndArea.Y); //cursor moves onto the correct spot in the website every time
(HTMLElem as IHTMLElement2).focus;
DropFilesAtPoint(dndArea, webBrowser.Handle);
I have come to a solution regarding this problem. Rather than using the clipboard, I piggy-backed on Melander’s drag-and-drop PIDLDemo. Adding a TListView component to the form and giving it the ability to drag-and-drop files to the shell does the trick. Using Windows' MOUSE_EVENT I am able to (programmatically) drag the files from the TListView and drop them onto the TEmbeddedWB at the correct location. Presto! The files are accepted and uploaded to the website.
The calling code now looks as follows:
function TfrmMain.GetMickey(val: TPoint): TPoint;
begin
{
http://delphi.xcjc.net/viewthread.php?tid=43193
Mouse Coordinates given are in "Mickeys", where their are 65535 "Mickeys"
to a screen's width.
}
Result.X := Round(val.X * (65535 / Screen.Width));
Result.Y := Round(val.Y * (65535 / Screen.Height));
end;
procedure TfrmMain.DropFilesAtPoint(const Area: TPoint; Wnd: HWND);
var Rect: TRect;
DropPoint,
ListViewPoint,
ListViewItemPoint: TPoint;
begin
GetWindowRect(ListView1.Handle, Rect);
ListViewItemPoint := ListView1.Items.Item[0].GetPosition;
ListViewPoint := Point(Rect.Left + ListViewItemPoint.X+10,
Rect.Top + ListViewItemPoint.Y+10);
ListView1.SelectAll; //ensures all files are dragged together
SetCursorPos(ListViewPoint.X, ListViewPoint.Y);
ListViewPoint := GetMickey(ListViewPoint);
MOUSE_EVENT(MOUSEEVENTF_LEFTDOWN,
ListViewPoint.X, ListViewPoint.Y, 0, 0); //left mouse button down
Sleep(500);
DropPoint := ClientToScreen(Area);
DropPoint := GetMickey(DropPoint);
MOUSE_EVENT(MOUSEEVENTF_ABSOLUTE or MOUSEEVENTF_MOVE or
MOUSEEVENTF_LEFTDOWN or MOUSEEVENTF_LEFTUP,
DropPoint.X, DropPoint.Y, 0, 0); //move and drop
Application.ProcessMessages;
end;
I found a solution to extract the contents of a zip file by creating a DLL using Ole. I put my own touch on this function, but for some reason, the compiler complains that the function's result is never used...
library unzipper;
{
title : UnZip for InnoSetup
version : 1.0
author : Daniel P. Stasinski
email : daniel#genericinbox.com
begin : Fri Nov 22 17:31:33 MST 2013
license : None
}
uses
Windows,
SysUtils,
ComObj;
const
SHCONTCH_NOPROGRESSBOX = 4;
SHCONTCH_AUTORENAME = 8;
SHCONTCH_RESPONDYESTOALL = 16;
SHCONTF_INCLUDEHIDDEN = 128;
SHCONTF_FOLDERS = 32;
SHCONTF_NONFOLDERS = 64;
UNZIP_SUCCESS = 0;
UNZIP_FAIL = -1;
function UnzipFile(ZipFile, TargetFolder: WideString): Integer; stdcall;
var
shellobj: variant;
ZipFileV, SrcFile: variant;
TargetFolderV, DestFolder: variant;
shellfldritems: variant;
begin
Result:= UNZIP_FAIL;
try
shellobj := CreateOleObject('Shell.Application');
ZipFileV := string(ZipFile);
TargetFolderV := string(TargetFolder);
SrcFile := shellobj.NameSpace(ZipFileV);
DestFolder := shellobj.NameSpace(TargetFolderV);
shellfldritems := SrcFile.Items;
DestFolder.CopyHere(shellfldritems, SHCONTCH_NOPROGRESSBOX or SHCONTCH_RESPONDYESTOALL);
Result:= UNZIP_SUCCESS;
except
on e: exception do begin
Result:= GetLastError;
end;
end;
end;
exports
UnzipFile;
begin
end.
It gives me the message...
[DCC Hint] Unzipper.dpr(35): H2077 Value assigned to 'UnzipFile' never used
This is coming from the first line of code in the function, which I'm initializing to a constant of -1 - which is my own error code if the entire function fails. I don't believe the compiler should be complaining about this, but I could be wrong. I always exterminate all compiler hints and warnings, but in this case, the compiler is more of a complainer.
Is this a fluke in the compiler, or is something wrong in the code?
The compiler is correct, and there's something wrong in the code. :-)
The function will either return UNZIP_SUCCESS if it works, or the result of GetLastError if an exception is raised. Therefore, the first assignment to Result is unnecessary - there is no path of execution that would cause UNZIP_FAIL to be returned.
If you remove the first line result assignment, there is no execution path that leaves result unassigned. Therefore, UNZIP_FAIL value will never be returned.
I need to go through a HTML string and replace characters with 0 (zero), except tags, spaces and line breaks. I created this code bellow, but it is too slow. Please, can someone help me to make it faster (optimize)?
procedure TForm1.btn1Click(Sender: TObject);
var
Txt: String;
Idx: Integer;
Tag: Boolean;
begin
Tag := False;
Txt := mem1.Text;
For Idx := 0 to Length(Txt) - 1 Do
Begin
If (Txt[Idx] = '<') Then
Tag := True Else
If (Txt[Idx] = '>') Then
Begin
Tag := False;
Continue;
end;
If Tag Then Continue;
If (not (Txt[Idx] in [#10, #13, #32])) Then
Txt[Idx] := '0';
end;
mem2.Text := Txt;
end;
The HTML text will never have "<" or ">" outside tags (in the middle of text), so I do not need to worry about this.
Thank you!
That looks pretty straightforward. It's hard to be sure without profiling the code against the data you're using, (which is always a good idea; if you need to optimize Delphi code, try running it through Sampling Profiler first to get an idea where you're actually spending all your time,) but if I had to make an educated guess, I'd guess that your bottleneck is in this line:
Txt[Idx] := '0';
As part of the compiler's guarantee of safe copy-on-write semantics for the string type, every write to an individual element (character) of a string involves a hidden call to the UniqueString routine. This makes sure that you're not changing a string that something else, somewhere else, holds a reference to.
In this particular case, that's not necessary, because you got the string fresh in the start of this routine and you know it's unique. There's a way around it, if you're careful.
CLEAR AND UNAMBIGUOUS WARNING: Do not do what I'm about to explain without making sure you have a unique string first! The easiest way to accomplish this is to call UniqueString manually. Also, do not do anything during the loop that could assign this string to any other variable. While we're doing this, it's not being treated as a normal string. Failure to heed this warning can cause data corruption.
OK, now that that's been explained, you can use a pointer to access the characters of the string directly, and get around the compiler's safeguards, like so:
procedure TForm1.btn1Click(Sender: TObject);
var
Txt: String;
Idx: Integer;
Tag: Boolean;
current: PChar; //pointer to a character
begin
Tag := False;
Txt := mem1.Text;
UniqueString(txt); //very important
if length(txt) = 0 then
Exit; //If you don't check this, the next line will raise an AV on a blank string
current := #txt[1];
dec(current); //you need to start before element 1, but the compiler won't let you
//assign to element 0
For Idx := 0 to Length(Txt) - 1 Do
Begin
inc(current); //put this at the top of the loop, to handle Continue cases correctly
If (current^ = '<') Then
Tag := True Else
If (current^ = '>') Then
Begin
Tag := False;
Continue;
end;
If Tag Then Continue;
If (not (current^ in [#10, #13, #32])) Then
current^ := '0';
end;
mem2.Text := Txt;
end;
This changes the metaphor. Instead of indexing into the string as an array, we're treating it like a tape, with the pointer as the head, moving forward one character at a time, scanning from beginning to end, and changing the character under it when appropriate. No redundant calls to UniqueString, and no repeatedly calculating offsets, which means this can be a lot faster.
Be very careful when using pointers like this. The compiler's safety checks are there for a good reason, and using pointers steps outside of them. But sometimes, they can really help speed things up in your code. And again, profile before trying anything like this. Make sure that you know what's slowing things down, instead of just thinking you know. If it turns out to be something else that's running slow, don't do this; find a solution to the real problem instead.
Edit: Looks like I was wrong - UniqueString is not the problem. The actual bottleneck seems to be accessing the string by character. Given that my entire answer was irrelevent, I've completely replaced it.
If you use a PChar to avoid recalculating the string offset, while still updating the string via Txt[Idx], the method is much faster (5 seconds down to 0.5 seconds in my test of 1000 runs).
Here's my version:
procedure TForm1.btn1Click(Sender: TObject);
var
Idx: Integer;
Tag: Boolean;
p : PChar;
Txt : string;
begin
Tag := False;
Txt := Mem1.Text;
p := PChar(txt);
Dec(p);
For Idx := 0 to Length(Txt) - 1 Do
Begin
Inc(p);
If (not Tag and (p^ = '<')) Then begin
Tag := True;
Continue;
end
Else If (Tag and (p^ = '>')) Then
Begin
Tag := False;
Continue;
end;
If Tag Then Continue;
If (not (p^ in [#10, #13, #32])) Then begin
Txt[Idx] := '0';
end;
end;
mem2.Text := Txt;
end;
I did some profiling and came up with this solution.
A test for > #32 instead of [#10,#13,#32] gains some speed (thanks #DavidHeffernan).
A better logic in the loop also gives a bit extra speed.
Accessing the string exclusively with the help of a PChar is more effective.
procedure TransformHTML( var Txt : String);
var
IterCnt : Integer;
PTxt : PChar;
tag : Boolean;
begin
PTxt := PChar(Txt);
Dec(PTxt);
tag := false;
for IterCnt := 0 to Length(Txt)-1 do
begin
Inc(PTxt);
if (PTxt^ = '<') then
tag := true
else
if (PTxt^ = '>') then
tag := false
else
if (not tag) and (PTxt^ > #32) then
PTxt^ := '0';
end;
end;
This solution is about 30% more effective than Mason's solution and 2.5 times more effective than Blorgbeard's.
I'm using Delphi and WebBrowser componenet to navigate a html page . the page have a Combobox . is there any way to call the OnChange event ?
The ComboBox is like this :
<select name="comboname" onchange="Some Javascript codes">
Also , i have used this code :
function TFrmMain.SetComboboxValue(WB: TEmbeddedWB;
SelectName, ItemName: string): Boolean;
var
iForms, iFormItems, iSelectItems: Word;
FormItem: OleVariant;
begin
Result := false;
for iForms := 0 to WB.OleObject.Document.Forms.length - 1 do
begin
FormItem := WB.OleObject.Document.Forms.item(iForms);
for iFormItems := 0 to FormItem.length - 1 do
begin
if (FormItem.item(iFormItems). type = 'select-one') and SameText
(FormItem.item(iFormItems).Name, SelectName) then
begin
for iSelectItems := 0 to FormItem.item(iFormItems).Options.length - 1 do
begin
if SameText(FormItem.item(iFormItems).Options.item(iSelectItems)
.Text, ItemName) then
begin
FormItem.item(iFormItems).SelectedIndex := iSelectItems;
Result := true;
Break;
end;
end;
end;
end;
end;
end;
But it change the value only.
to execute the onchange event you can use the execScript method
check this sample
uses
MSHTML;
var
Doc: IHTMLDocument2;
HTMLWindow: IHTMLWindow2;
begin
Doc := WebBrowser1.Document as IHTMLDocument2;
if not Assigned(Doc) then
Exit;
HTMLWindow := Doc.parentWindow;
if not Assigned(HTMLWindow) then
Exit;
HTMLWindow.execScript('yourfunctioname()', 'JavaScript');
end;
for more info check this excellent article
How to call JavaScript functions in a TWebBrowser from Delphi
Inspired by the response. NET have been using the structures below:
FrameSet Document Elements Item Name Value Change ;
EWB.OleObject.Document.Frames.Item('mainFrame').Document.Forms.Item('invoiceForm').Elements.Item('inputname').Value:= '123456';
or
FrameSet Document Elements Items Lenth;
EWB.OleObject.Document.Forms.Item('invoiceForm').Elements.Length;