I have this code in one modules:
PROCEDURE Get (File: IN Ada.Text_IO.File_Type; Item : OUT Rational) IS
N: Integer;
D: Integer;
Dummy: Character;
BEGIN -- Get
LOOP
BEGIN
Ada.Integer_Text_IO.Get(File => File, Item => N);
Ada.Text_IO.Get (File => File, Item => Dummy);
Ada.Integer_Text_IO.Get(File => File, Item => D);
Item := N/D;
if Dummy /= '/' then
........;
end if;
EXIT;
EXCEPTION
when other =>
Ada.Text_IO.Put_Line(" here is exception ");
END;
END LOOP;
END Get;
What is differences with this second code.
The main of my question is if I don't put raise in body of statement of exception what is happen?
PROCEDURE Get (File: IN Ada.Text_IO.File_Type; Item : OUT Rational) IS
N: Integer;
D: Integer;
Dummy: Character;
BEGIN -- Get
LOOP
BEGIN
Ada.Integer_Text_IO.Get(File => File, Item => N);
Ada.Text_IO.Get (File => File, Item => Dummy);
Ada.Integer_Text_IO.Get(File => File, Item => D);
Item := N/D;
if Dummy /= '/' then
........;
end if;
EXIT;
EXCEPTION
when other =>
Ada.Text_IO.Put_Line(" here is exception ");
**raise;**
END;
END LOOP;
END Get;
The main of my question is if I don't put raise in body of statement of exception what is happen???
Thank you very much.
The only difference between both code modules is that the exception (if any is raised during execution of Get) is reraised, i.e. the exception is propagated to the caller of Get.
Whether this is the desired behaviour depends on your needs, i.e. does the caller of Get need to know that an exception occurred?
In your example several kinds of exceptions may occur, e.g.
not reading the correct/expected input (the file to read from does not start with a number)
trying to read from a file that has not been opened
reading D as 0 (thus resulting in a division by 0)
All of these are handled in the same manner by printing "here is exception". The first implementation of Get then silently returns control to the caller (who will not know that anything strange happened). The second implementation, however, will inform the caller by reraising the exception.
For more information see the Ada LRM §11.3 (Raise Statements).
I'm having serious problems with mysql using pthreads. The error I get after ending my program:
"Error in my_thread_global_end(): 1 threads didn't exit"
I called mysql_library_init in main before starting any threads. For the sake of it, I just started 1 thread. After the thread is closed (using pthread_join), I call mysql_library_end in main. In the pthread itself I call mysql_init. For some reason this seems incorrect cause I get the error. I use MySQL 5.6 and link with libmysqlclient.a.
The mysql manual is extremely unclear and contradictory, so I hope someone with a logical mind can explain this to me:
"In a nonmulti-threaded environment, mysql_init invokes mysql_library_init automatically as necessary. However, mysql_library_init is not thread-safe in a multi-threaded environment, and thus neither is mysql_init. Before calling mysql_init, either call mysql_library_init prior to spawning any threads, or use a mutex to protect the mysql_library_init call. This should be done prior to any other client library call."
First line: So mysql_init ONLY invokes mysql_library_init in a NONmulti-threaded environment "when needed" (when is it needed anyway in a NONmulti-threaded environment?) and so I can conlcude from this that mysql_init() thinks it is NOT needed in a multi-threaded environment? I guess not, so fine, I call mysql_library_init in my main... Then I read everywhere I should also call mysql_init within the thread after that. I want each thread to have his own connection, so fine, I also do that so each thread have their own MYSQL struct. But the manual sais mysql_init is not thread safe... Uhm, ok... So just with 1 thread, I still have the problem...
main -> mysql_library_init
main -> create 1 pthread
pthread -> mysql_init
pthread -> mysql_real_connect
pthread -> mysql_close
....
I press Ctrl C after a few seconds (mysql was closed by now in the thread) so the cleaning up starts:
main -> pthread_cancel
main -> pthread_join
main -> mysql_library_end
RESULT: Error in my_thread_global_end: 1 threads didn't exit
........
int main( void )
{
if ( mysql_library_init( 0, NULL, NULL ) != 0 ) { ... }
if ( mysql_thread_safe() ) { ... } // This goes fine
sem_init( &queue.totalStored, 0, 0 );
pthread_mutex_init( &mutex_bees, NULL );
pthread_create( &workerbees[tid], &attr, BeeWork, ( void * ) tid );
pthread_attr_destroy( &attr );
while ( recv_signal == 0 )
{
errno = 0;
sock_c = accept( sock_s, NULL, NULL );
if ( ( sock_c == -1 ) && ( errno == EINTR ) )
{
// do stuff
if ( recv_signal == SIGHUP ) { /* do stuff*/ }
} else { /* do stuff */ }
}
// CLEANUP
close( sock_s );
RC = pthread_cancel( workerbees[tid] );
if ( RC != 0 ) { Log( L_ERR, "Unsuccessful pthread_cancel()" ); }
// WAIT FOR THREADS TO FINISH WORK AND EXIT
RC = pthread_join( workerbees[tid], &res );
if ( RC != 0 ) { Log( L_ERR, "Error: Unsuccessful pthread_join()" ); }
if ( res == PTHREAD_CANCELED )
{ /* print debug stuff */ }
else { /* print debug stuff */ }
mysql_library_end();
sem_destroy( &queue.totalStored );
exit( 0 );
}
void *BeeWork( void *t )
{
// DISABLE SIGNALS THAT main() ALREADY LISTENS TO
sigemptyset( &sigset );
sigaddset( &sigset, SIGINT );
sigaddset( &sigset, SIGTERM );
sigaddset( &sigset, SIGQUIT );
sigaddset( &sigset, SIGHUP );
pthread_sigmask( SIG_BLOCK, &sigset, NULL );
MYSQL *conn;
conn = mysql_init( NULL );
if ( ! mysql_real_connect( conn, server, prefs.mysql_user, prefs.mysql_pass, prefs.mysql_db, 0, prefs.mysql_sock, 0 ) ) { /* error */ }
mysql_close( conn );
// Do stuff
...
pthread_exit( ( void * ) t );
}
I guess I can answer my own question, I found out my pthread cleanup handler was not executed (installed with pthread_cleanup_push) and the end of the code with pthread_exit was called sooner than main could cancel the thread. I did a pthread_cleanup_pop( 0 ) and changed it to pthread_cleanup_pop( 1 ) so the cleanup handler also got executed when the thread exits sooner than main could cancel. In this cleanup handler, now mysql_thread_end actually got a chance to run and it fixed the problem.
Addition: Apparently, the MySQL client library is not designed to use connection handles (MYSQL*) across threads.
Do not pass a MYSQL* from one thread to another.
Do not use std::async or simlilar in conjunction with MySQL functions. (Code may or may not be executed in a separate thread.)
I'm writing some functions in Delphi using Assembly. So I want to put it in a .pas file called Strings.pas. To use in uses of a new Delphi software. What do I need to write, to make it a valid library?
My function is like this:
function Strlen(texto : string) : integer;
begin
asm
mov esi, texto
xor ecx,ecx
cld
#here:
inc ecx
lodsb
cmp al,0
jne #here
dec ecx
mov Result,ecx
end;
end;
That counts the numbers of chars in the string. How can I make it in a lib Strings.pas to call with uses Strings; in my form?
A .pas file is a unit, not a library. A .pas file needs to have unit, interface, and implementation statements, eg:
Strings.pas:
unit Strings;
interface
function Strlen(texto : string) : integer;
implementation
function Strlen(texto : string) : integer;
asm
// your assembly code...
// See Note below...
end;
end.
Then you can add the .pas file to your other projects and use the Strings unit as needed. It will be compiled directly into each executable. You don't need to make a separate library out of it. But if you want to, you can. Create a separate Library (DLL) or Package (BPL) project, add your .pas file to it, and compile it into an executable file that you can then reference in your other projects.
In the case of a DLL library, you will not be able to use the Strings unit directly. You will have to export your function(s) from the library (and string is not a safe data type to pass over a DLL boundary between modules), eg:
Mylib.dpr:
library Mylib;
uses
Strings;
exports
Strings.Strlen;
begin
end.
And then you can have your other projects declare the function(s) using external clause(s) that reference the DLL file, eg:
function Strlen(texto : PChar) : integer; external 'Mylib.dll';
In this case, you can make a wrapper .pas file that declares the functions to import, add that unit to your other projects and use it as needed, eg:
StringsLib.pas:
unit StringsLib;
interface
function Strlen(texto : PChar) : integer;
implementation
function Strlen; external 'Mylib.dll';
end.
In the case of a Package, you can use the Strings units directly. Simply add a reference to the package's .bpi in your other project's Requires list in the Project Manager, and then use the unit as needed. In this case, string is safe to pass around.
Note: in the assembly code you showed, for the function to not cause an access violation, you need to save and restore the ESI register. See the section on Register saving conventions in the Delphi documentation.
The correct asm version may be:
unit MyStrings; // do not overlap Strings.pas unit
interface
function StringLen(const texto : string) : integer;
implementation
function StringLen(const texto : string) : integer;
asm
test eax,eax
jz #done
mov eax,dword ptr [eax-4]
#done:
end;
end.
Note that:
I used MyStrings as unit name, since it is a very bad idea to overlap the official RTL unit names, like Strings.pas;
I wrote (const texto: string) instead of (texto: string), to avoid a reference count change at calling;
Delphi string type already has its length stored as integer just before the character memory buffer;
In Delphi asm calling conventions, the input parameters are set in eax edx ecx registers, and the integer result of a function is the eax register - see this reference article - for Win32 only;
I tested for texto to be nil (eax=0), which stands for a void '' string;
This would work only under Win32 - asm code under Win64 would be diverse;
Built-in length() function would be faster than an asm sub-function, since it is inlined in new versions of Delphi;
Be aware of potential name collisions: there is already a well known StrLen() function, which expects a PChar as input parameter - so I renamed your function as StringLen().
Since you want to learn asm, here are some reference implementation of this function.
A fast PChar oriented version may be :
function StrLen(S: PAnsiChar): integer;
asm
test eax,eax
mov edx,eax
jz #0
xor eax,eax
#s: cmp byte ptr [eax+edx+0],0; je #0
cmp byte ptr [eax+edx+1],0; je #1
cmp byte ptr [eax+edx+2],0; je #2
cmp byte ptr [eax+edx+3],0; je #3
add eax,4
jmp #s
#1: inc eax
#0: ret
#2: add eax,2; ret
#3: add eax,3
end;
A more optimized version:
function StrLen(S: PAnsiChar): integer;
// pure x86 function (if SSE2 not available) - faster than SysUtils' version
asm
test eax,eax
jz ##z
cmp byte ptr [eax+0],0; je ##0
cmp byte ptr [eax+1],0; je ##1
cmp byte ptr [eax+2],0; je ##2
cmp byte ptr [eax+3],0; je ##3
push eax
and eax,-4 { DWORD Align Reads }
##Loop:
add eax,4
mov edx,[eax] { 4 Chars per Loop }
lea ecx,[edx-$01010101]
not edx
and edx,ecx
and edx,$80808080 { Set Byte to $80 at each #0 Position }
jz ##Loop { Loop until any #0 Found }
##SetResult:
pop ecx
bsf edx,edx { Find First #0 Position }
shr edx,3 { Byte Offset of First #0 }
add eax,edx { Address of First #0 }
sub eax,ecx { Returns Length }
##z: ret
##0: xor eax,eax; ret
##1: mov eax,1; ret
##2: mov eax,2; ret
##3: mov eax,3
end;
An SSE2 optimized version:
function StrLen(S: PAnsiChar): integer;
asm // from GPL strlen32.asm by Agner Fog - www.agner.org/optimize
or eax,eax
mov ecx,eax // copy pointer
jz #null // returns 0 if S=nil
push eax // save start address
pxor xmm0,xmm0 // set to zero
and ecx,0FH // lower 4 bits indicate misalignment
and eax,-10H // align pointer by 16
movdqa xmm1,[eax] // read from nearest preceding boundary
pcmpeqb xmm1,xmm0 // compare 16 bytes with zero
pmovmskb edx,xmm1 // get one bit for each byte result
shr edx,cl // shift out false bits
shl edx,cl // shift back again
bsf edx,edx // find first 1-bit
jnz #A200 // found
// Main loop, search 16 bytes at a time
#A100: add eax,10H // increment pointer by 16
movdqa xmm1,[eax] // read 16 bytes aligned
pcmpeqb xmm1,xmm0 // compare 16 bytes with zero
pmovmskb edx,xmm1 // get one bit for each byte result
bsf edx,edx // find first 1-bit
// (moving the bsf out of the loop and using test here would be faster
// for long strings on old processors, but we are assuming that most
// strings are short, and newer processors have higher priority)
jz #A100 // loop if not found
#A200: // Zero-byte found. Compute string length
pop ecx // restore start address
sub eax,ecx // subtract start address
add eax,edx // add byte index
#null:
end;
Or even a SSE4.2 optimized version:
function StrLen(S: PAnsiChar): integer;
asm // warning: may read up to 15 bytes beyond the string itself
or eax,eax
mov edx,eax // copy pointer
jz #null // returns 0 if S=nil
xor eax,eax
pxor xmm0,xmm0
{$ifdef HASAESNI}
pcmpistri xmm0,dqword [edx],EQUAL_EACH // comparison result in ecx
{$else}
db $66,$0F,$3A,$63,$02,EQUAL_EACH
{$endif}
jnz #loop
mov eax,ecx
#null: ret
#loop: add eax,16
{$ifdef HASAESNI}
pcmpistri xmm0,dqword [edx+eax],EQUAL_EACH // comparison result in ecx
{$else}
db $66,$0F,$3A,$63,$04,$10,EQUAL_EACH
{$endif}
jnz #loop
#ok: add eax,ecx
end;
You will find all those functions, including Win64 versions, in our very optimized SynCommons.pas unit, which is shared by almost all our Open Source projects.
The my two solutions to get the length of two types of string,
as for says Peter Cordes are not both useful.
Only the "PAnsiCharLen()" could be an alternative solution,
but not as fast as it is StrLen() (optimized) of Amaud Bouchez,
that it is about 3 times faster than mine.
10/14/2017 (mm/dd/yyy): Added one new function (Clean_Str).
However, for now, I propose three small corrections to both
of them (two suggested by Peter Cordes: 1) use MovZX instead of Mov && And;
2) Use SetZ/SetE instead LAHF/ShL, use XOr EAX,EAX instead XOr AL,AL);
in the future I could define the functions in assembly (now they are defined in Pascal):
unit MyStr;
{ Some strings' function }
interface
Function PAnsiCharLen(S:PAnsiChar):Integer;
{ Get the length of the PAnsiChar^ string. }
Function ShortStrLen(S:ShortString):Integer;
{ Get the length of the ShortString^ string. }
Procedure Clean_Str(Str:ShortString;Max_Len:Integer);
{ This function can be used to clear the unused space of a short string
without modifying his useful content (for example, if you save a
short-string field in a file, at parity of content the file may be
different, because the unused space is not initialized).
Clears a String Str_Ptr ^: String [], which has
Max_Len = SizeOf (String []) - 1 characters, placing # 0
all characters beyond the position of Str_Ptr ^ [Str_Ptr ^ [0]] }
implementation
Function PAnsiCharLen(S:PAnsiChar):Integer;
{ EAX EDX ECX are 1°, 2° AND 3° PARAMETERs.
Can freely modify the EAX, ECX, AND EDX REGISTERs. }
Asm
ClD {Clear string direction flag}
Push EDI {Save EDI's reg. into the STACK}
Mov EDI,S {Load S into EDI's reg.}
XOr EAX,EAX {Set AL's reg. with null terminator}
Mov ECX,-1 {Set ECX's reg. with maximum length of the string}
RepNE ScaSB {Search null and decrease ECX's reg.}
SetE AL {AL is set with FZero}
Add EAX,ECX {EAX= maximum_length_of_the_string - real_length_of_the_string}
Not EAX {EAX= real_length_of_the_string}
Pop EDI {Restore EDI's reg. from the STACK}
End;
Function ShortStrLen(S:ShortString):Integer; Assembler;
{ EAX EDX ECX are 1°, 2° AND 3° PARAMETERs.
Can freely modify the EAX, ECX, AND EDX REGISTERs. }
Asm
MovZX EAX,Byte Ptr [EAX] {Load the length of S^ into EAX's reg. (function's result)}
End;
Procedure Clean_Str(Str:ShortString;Max_Len:Integer); Assembler;
(* EAX EDX ECX are 1°, 2° AND 3° PARAMETERs.
Can freely modify the EAX, ECX, AND EDX REGISTERs. *)
Asm
ClD {Clear string direction flag}
Push EDI {Save EDI's reg. into the STACK}
Mov EDI,Str {Load input string pointer into EDI's reg.}
Mov ECX,Max_Len {Load allocated string length into ECX's reg.}
MovZX EDX,Byte Ptr [EDI] {Load real string length into EDX's reg.}
StC {Process the address of unused space of Str; ...}
AdC EDI,EDX {... skip first byte and useful Str space}
Cmp EDX,ECX {If EDX>ECX ...}
CMovGE EDX,ECX {... set EDX with ECX}
Sub ECX,EDX {ECX contains the size of unused space of Str}
XOr EAX,EAX {Clear accumulator}
Rep StoSB {Fill with 0 the unused space of Str}
Pop EDI {Restore EDI's reg. from the STACK}
End;
end.
Old (incomplete) answer:
"Some new string's functions, not presents in Delphi library, could be these:"
Type Whole=Set Of Char;
Procedure AsmKeepField (PStrIn,PStrOut:Pointer;FieldPos:Byte;
All:Boolean);
{ Given "field" as a sequence of characters that does not contain spaces
or tabs (# 32, # 9), it takes FieldPos (1..N) field
to PStrIn ^ (STRING) and copies it to PStrOut ^ (STRING).
If All = TRUE, it also takes all subsequent fields }
Function AsmUpCComp (PStr1,PStr2:Pointer):Boolean;
{ Compare a string PStr1 ^ (STRING) with a string PStr2 ^ (STRING),
considering the PStr1 alphabetic characters ^ always SHIFT }
Function UpCaseStrComp (Str1,Str2:String;Mode:Boolean):ShortInt;
{ Returns: -1 if Str1 < Str2.
0 is Str1 = Str2.
1 is Str1 > Str2.
MODE = FALSE means "case sensitive comparison" (the letters are
consider them as they are).
MODE = TRUE means that the comparison is done by considering
both strings as if they were all uppercase }
Function KeepLS (Str:String;CntX:Byte):String;
{ RETURN THE PART OF STR THAT INCLUDES THE FIRST CHARACTER
OF STR AND ALL THE FOLLOW UP TO THE POSITION CntX (0 to N-1) INCLUDED }
Function KeepRS (Str:String;CntX,CsMode:Byte):String;
{ RETURN THE PART OF STR STARTING TO POSITION CntX + 1 (0 to N-1)
UP TO END OF STR.
IF CsMode = 0 (INSERT MODE), IF CsMode = 1 (OVERWRITE-MODE):
IN THIS CASE, THE CHARACTER TO CntX + 1 POSITION IS NOT INCLUDED }
Function GetSubStr (Str:String;
Pos,Qnt:Byte;CH:Char):String;
{ RETURN Qnt STR CHARACTERS FROM POSITION Pos (1 to N) OF STR;
IF EFFECTIVE LENGTH IS LESS THAN Qnt, WILL ADDED CHARACTER = CH }
Function Keep_Right_Path_Str_W(PathName:String;FieldWidth:Byte;
FormatWithSpaces:Boolean):String;
{ RESIZE A STRING OF A FILE PATH, FROM PathName;
THE NEW STRING WILL HAVE A MAXIMUM LENGTH OF FieldWidth CHARACTERS.
REPLACE EXCEDENT CHARACTERS WITH 3 POINTS,
INSERTED AFTER DRIVE AND ROOT.
REPLACE SOME DIRECTORY WITH 3 POINTS,
ONLY WHEN IT IS NECESSARY, POSSIBLE FROM SECOND.
FORMAT RETURN WITH SPACE ONLY IF FormatWithSpaces = TRUE }
Function KeepBarStr (Percentage,Qnt:Byte;
Ch1,Ch2,Ch3:Char):String;
{ THIS IS A FUNCTION WICH MAKES A STRING WICH CONTAINS A REPRESENTATION OF STATE
OF ADVANCEMENT OF A PROCESS; IT RETURNS A CHARACTERS' SEQUENCE, CONSTITUTED BY "<Ch1>"
(LENGTH = Percentage / 100 * Qnt), WITH AN APPROXIMATION OF THE LAST CHARACTER TO
"<Ch2>" (IF "Percentage / 100 * Qnt" HAS HIS FRACTIONAL'S PART GREATER THAN 0.5),
FOLLOWED BY AN OTHER CHARACTERS' SEQUENCE, CONSTITUTED BY "<Ch3>" (LENGTH = (100 -
Percentage) / 100 * Qnt). }
Function Str2ChWhole (Str:String;Var StrIndex:Byte;
Var ChSet:Whole;
Mode:Boolean):Boolean;
{ CONVERT A PART OF Str, POINTED BY StrIndex, IN A ChSet CHARACTER SET;
IF Mode = TRUE, "StrIn" SHOULD CONTAIN ASCII CODES
OF CORRESPONDING CHARACTERS EXPRESSED IN DECIMAL SIZE;
OTHERWISE IT SHOULD CONTAIN CORRESPONDING CHARACTER SYMBOLS }
Function ChWhole2Str (ChSet:Whole;Mode:Boolean):String;
{ CONVERT A SET OF CHARACTERS IN A CORRESPONDING STRING;
IF Mode = TRUE ELEMENTS OF ChSet WILL BE CONVERTED IN ASCII CODES
EXPRESSED IN DECIMAL SIZE; OTHERWISE THE CORRESPONDING SYMBOLS
WILL BE RETURNED }
Function ConverteFSize (FSize:LongInt;
Var SizeStr:TSizeStr):Integer;
{ MAKES THE CONVERSION OF THE DIMENSION OF A FILE IN A TEXT,
LARGE TO MAXIMUM 5 CHARACTERS, AND RETURN THE COLOR OF THIS STRING }
Function UpCasePos (SubStr,Str:String):Byte;
{ Like the Pos () system function, but not "case sensitive" }
Notice:
Original post title
Why multithreaded JSON parser from DWScript does not scale with number of threads?
was changed because this problem is not related to processing JSON data with DWScript.
The problem is in default memory manager in Delphi XE2 to XE7 ( tested were XE2 and trial XE7 ), but problem appeared first in such type of application.
I have multithreaded Win32/Win64 vcl application which process JSON data in Delphi XE2.
Each thread parses JSON data using TdwsJSONValue.ParseString(sJSON) from DWScript, reads values using DWScript methods and stores result as records.
For testing purposes I process same JSON data in each thread.
Single thead run takes N seconds within thread to process data. Increasing number of threads to M lineary (approx. M * N) increases time within single thread necessary to process same data.
In result there is no speed improvment. Other parts of this applications ( JSON data delivery, storing results in target environment ) - scale as expected.
What could be a reason ? Any ideas appreciated.
Supplemental information:
Tested on Win7/32 and Win7/64, Win8/64 from 2-core to 12-core (w/w-out HT) systems
DWScript was choosen as fastest available (tested a bunch, among them: Superobject, build-in Delphi). SO behaves similar as JSON unit from DWS.
Below is complete console app illustrating the problem. To run it we need sample json data available here: https://www.dropbox.com/s/4iuv87ytpcdugk6/json1.zip?dl=0 This file contains data json1.dat for first thread. For threads up to 16 just copy json1.dat to json2.dat...json16.dat.
Program and data shoule be in the same folder. To run: convert.exe N, where N is number of threads.
Program writes time of execution in msecs to stout - spent in thread, time of parsing data and time of releasing (Destroy) TdwsJSONValue object.
Statement _dwsjvData.Destroy; does not scale.
program Convert;
{$APPTYPE CONSOLE}
{$R *.res}
uses
System.SysUtils,
System.Diagnostics,
System.Classes,
dwsJSON in 'dwsJSON.pas',
dwsStrings in 'dwsStrings.pas',
dwsUtils in 'dwsUtils.pas',
dwsXPlatform in 'dwsXPlatform.pas';
type
TWorkerThread = class (TThread)
private
_iUid: Integer;
_swWatch: TStopwatch;
_lRunning: Boolean;
_sFileJSonData: String;
_fJsonData: TextFile;
protected
constructor Create (AUid: Integer);
procedure Execute; override;
published
property Running: Boolean read _lRunning;
end;
TConverter = class (TObject)
private
_swWatch0, _swWatch1, _swWatch2: TStopwatch;
_dwsjvData: TdwsJSONValue;
protected
constructor Create;
destructor Destroy; override;
function Calculate (AUid: Integer; AJSonData: String; var AParse, ADestroy: Integer): Integer;
end;
const
MAX_THREADS = 16;
var
iHowMany: Integer;
athWorker: array [1..MAX_THREADS] of Pointer;
aiElapsed: array [1..MAX_THREADS] of Integer;
aiElapsedParse: array [1..MAX_THREADS] of Integer;
aiElapsedDestroy: array [1..MAX_THREADS] of Integer;
aiFares: array [1..MAX_THREADS] of Integer;
swWatchT, swWatchP: TStopwatch;
constructor TWorkerThread.Create (AUid: Integer);
begin
inherited Create (True);
_iUid := AUid;
_swWatch := TStopwatch.Create;
_sFileJSonData := ExtractFilePath (ParamStr (0)) + 'json' + Trim (IntToStr (_iUid)) + '.dat';
_lRunning := False;
Suspended := False;
end;
procedure TWorkerThread.Execute;
var
j: Integer;
sLine: String;
slLines: TStringList;
oS: TConverter;
begin
_lRunning := True;
oS := TConverter.Create;
slLines := TStringList.Create;
System.AssignFile (_fJsonData, _sFileJSonData);
System.Reset (_fJsonData);
j := 0;
repeat
System.Readln (_fJsonData, sLine);
slLines.Add (sLine);
Inc (j);
until (j = 50);
// until (System.Eof (_fJsonData));
System.Close (_fJsonData);
Sleep (1000);
_swWatch.Reset;
_swWatch.Start;
aiFares [_iUid] := 0;
aiElapsedParse [_iUid] := 0;
aiElapsedDestroy [_iUid] := 0;
for j := 1 to slLines.Count do
aiFares [_iUid] := aiFares [_iUid] + oS.Calculate (_iUid, slLines.Strings [j - 1], aiElapsedParse [_iUid], aiElapsedDestroy [_iUid]);
_swWatch.Stop;
slLines.Free;
os.Destroy;
aiElapsed [_iUid] := _swWatch.ElapsedMilliseconds;
_lRunning := False;
end;
constructor TConverter.Create;
begin
inherited Create;
_swWatch0 := TStopwatch.Create;
_swWatch1 := TStopwatch.Create;
_swWatch2 := TStopwatch.Create;
end;
destructor TConverter.Destroy;
begin
inherited;
end;
function TConverter.Calculate (AUid: Integer; AJSonData: String; var AParse, ADestroy: Integer): Integer;
var
jFare, jTotalFares, iElapsedParse, iElapsedDestroy, iElapsedTotal: Integer;
begin
_swWatch0.Reset;
_swWatch0.Start;
_swWatch1.Reset;
_swWatch1.Start;
_dwsjvData := TdwsJSONValue.ParseString (AJSonData);
_swWatch1.Stop;
iElapsedParse := _swWatch1.ElapsedMilliseconds;
if (_dwsjvData.ValueType = jvtArray) then
begin
_swWatch2.Reset;
_swWatch2.Start;
jTotalFares := _dwsjvData.ElementCount;
for jFare := 0 to (jTotalFares - 1) do
if (_dwsjvData.Elements [jFare].ValueType = jvtObject) then
begin
_swWatch1.Reset;
_swWatch1.Start;
_swWatch1.Stop;
end;
end;
_swWatch1.Reset;
_swWatch1.Start;
_dwsjvData.Destroy;
_swWatch1.Stop;
iElapsedDestroy := _swWatch1.ElapsedMilliseconds;
_swWatch0.Stop;
iElapsedTotal := _swWatch0.ElapsedMilliseconds;
Inc (AParse, iElapsedParse);
Inc (ADestroy, iElapsedDestroy);
result := jTotalFares;
end;
procedure MultithreadStart;
var
j: Integer;
begin
for j := 1 to iHowMany do
if (athWorker [j] = nil) then
begin
athWorker [j] := TWorkerThread.Create (j);
TWorkerThread (athWorker [j]).FreeOnTerminate := False;
TWorkerThread (athWorker [j]).Priority := tpNormal;
end;
end;
procedure MultithreadStop;
var
j: Integer;
begin
for j := 1 to MAX_THREADS do
if (athWorker [j] <> nil) then
begin
TWorkerThread (athWorker [j]).Terminate;
TWorkerThread (athWorker [j]).WaitFor;
TWorkerThread (athWorker [j]).Free;
athWorker [j] := nil;
end;
end;
procedure Prologue;
var
j: Integer;
begin
iHowMany := StrToInt (ParamStr (1));
for j := 1 to MAX_THREADS do
athWorker [j] := nil;
swWatchT := TStopwatch.Create;
swWatchT.Reset;
swWatchP := TStopwatch.Create;
swWatchP.Reset;
end;
procedure RunConvert;
function __IsRunning: Boolean;
var
j: Integer;
begin
result := False;
for j := 1 to MAX_THREADS do
result := result or ((athWorker [j] <> nil) and TWorkerThread (athWorker [j]).Running);
end;
begin
swWatchT.Start;
MultithreadStart;
Sleep (1000);
while (__isRunning) do
Sleep (500);
MultithreadStop;
swWatchT.Stop;
Writeln (#13#10, 'Total time:', swWatchT.ElapsedMilliseconds);
end;
procedure Epilogue;
var
j: Integer;
begin
for j := 1 to iHowMany do
Writeln ( #13#10, 'Thread # ', j, ' tot.time:', aiElapsed [j], ' fares:', aiFares [j], ' tot.parse:', aiElapsedParse [j], ' tot.destroy:', aiElapsedDestroy [j]);
Readln;
end;
begin
try
Prologue;
RunConvert;
Epilogue;
except
on E: Exception do
Writeln (E.ClassName, ': ', E.Message);
end;
end.
Have you tried my scaleable memory manager? Because Delphi (with fastmm internally) does not scale well with strings and other memory related stuff:
https://scalemm.googlecode.com/files/ScaleMM_v2_4_1.zip
And you could also try both profiler modes of my profiler to see which part is the bottleneck:
https://code.google.com/p/asmprofiler/
I did a (re)test of the FastCode MM Challenge, and the results were not that good for TBB (also out of memory exception in block downsize test).
In short: ScaleMM2 and Google TCmalloc are the fastest in this complex test, Fastmm and ScaleMM2 use the least memory.
Average Speed Performance: (Scaled so that the winner = 100%)
XE6 : 70,4
TCmalloc : 89,1
ScaleMem2 : 100,0
TBBMem : 77,8
Average Memory Performance: (Scaled so that the winner = 100%)
XE6 : 100,0
TCmalloc : 29,6
ScaleMem2 : 75,6
TBBMem : 38,4
FastCode Challenge: https://code.google.com/p/scalemm/source/browse/#svn%2Ftrunk%2FChallenge
TBB 4.3: https://www.threadingbuildingblocks.org/download
The solution is exchange default Delphi XE2 or XE7 memory manager with Intel® Threading Building Blocks memory manager. In example application it scales ca. lineary with number of threads up to 16 when app is 64 bits.
update: with assumption that number of threads running is less than number of cores
This was tested on machines from 2cores/4ht to 12cores/24ht running KVM virtualized Windows 7 with 124GB RAM
Interesting thing is virtualizing Win 7. memory allocation and deallocation is from 2 x faster as in native Win 7.
Conclusion: if you do a lot of memory allocation / deallocation operations of 10kB-10MB blocks in threads of multithreaded ( more than 4-8 threads) application - use only memory manager from Intel.
#André: thanks for tip pointing me to right direction!
Here is unit with TBB memory manager taken for tests, it has to appear as 1st on unit list in main project file .dpr
unit TBBMem;
interface
function ScalableGetMem (ASize: NativeInt): Pointer; cdecl; external 'tbbmalloc' name 'scalable_malloc';
procedure ScalableFreeMem (APtr: Pointer); cdecl; external 'tbbmalloc' name 'scalable_free';
function ScalableReAlloc (APtr: Pointer; Size: NativeInt): Pointer; cdecl; external 'tbbmalloc' name 'scalable_realloc';
implementation
Function TBBGetMem (ASize: Integer): Pointer;
begin
result := ScalableGetMem (ASize);
end;
Function TBBFreeMem (APtr: Pointer): Integer;
begin
ScalableFreeMem (APtr);
result := 0;
end;
Function TBBReAllocMem (APtr: Pointer; ASize: Integer): Pointer;
begin
result := ScalableRealloc (APtr, ASize);
end;
const
TBBMemoryManager: TMemoryManager = ( GetMem: TBBGetmem;
FreeMem: TBBFreeMem;
ReAllocMem: TBBReAllocMem; );
var
oldMemoryManager: TMemoryManager;
initialization
GetMemoryManager (oldMemoryManager);
SetMemoryManager (TBBMemoryManager);
finalization
SetMemoryManager (oldMemoryManager);
end.
I'm trying to write a function that receives a number (which I pushed earlier), and prints it. How can I do it?
What I have so far:
org 100h
push 10
call print_num
print_num:
push bp
mov bp, sp
mov ax, [bp+2*2]
mov bx, cs
mov es, bx
mov dx, string
mov di, dx
stosw
mov ah, 09h
int 21h
pop bp
ret
string:
What you're placing at the address of string is a numerical value, not the string representation of that value.
The value 12 and the string "12" are two separate things. Seen as a 16-bit hexadecimal value, 12 would be 0x000C while "12" would be 0x3231 (0x32 == '2', 0x31 == '1').
You need to convert the numerical value into its string representation and then print the resulting string.Rather than just pasting a finished solution I'll show a simple way of how this could be done in C, which should be enough for you to base an 8086 implementation on:
char string[8], *stringptr;
short num = 123;
string[7] = '$'; // DOS string terminator
// The string will be filled up backwards
stringptr = string + 6;
while (stringptr >= string) {
*stringptr = '0' + (num % 10); // '3' on the first iteration, '2' on the second, etc
num /= 10; // 123 => 12 => 1 => 0
if (num == 0) break;
stringptr--;
}