Storing MD5 in MySQL - mysql

Instead of storing a MD5 hash in a 32-byte field, I will like to store it in a 16-byte binary field. Mysql field "TEMP_MD5" is defined as Binary(16).
The MySQL CREATE TABLE with a sample row insert is:
CREATE TABLE `mytable` (
`TEMP_MD5` binary(16) DEFAULT NULL,
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
INSERT INTO mytable (TEMP_MD5) VALUES UNHEX("202cb962ac59075b964b07152d234b70") );
The sample code:
Let's say after the 16-byte binary field has been stored in the MySQL field TEMP_MD5, how do I compare this 16-byte field in Delphi code after I retrieve the value?
Is it possible to skip MySQL HEX/UNHEX internal functions, and just use Delphi code to compare the 16-byte binary field (32-byte string) in MySQL?
For example :
FDQuery1.Open( 'SELECT TEMP_MD5 from mytable;' );
if THashMD5.GetHashBytes('123') = fDQuery1.FieldByName('TEMP_MD5').VALUE then
SHOWMESSAGE('MATCHED!');
However, it seems that the values for FieldByName('TEMP_MD5').value never matched the THashMD5.GetHashString('123') value
and another way of comparing by using SELECT statement also failed
FDQuery1.Open( 'SELECT TEMP_MD5 mytable ' +
'WHERE (TEMP_MD5=:myvalue)',
[THashMD5.GetHashBytes('123')] );
above also failed to give FDQuery1.RecordCount = 1.
Basically I'm trying to compare the 16-byte Binary I stored in MySQL to a value, let's say '123' in code to see if both matches.
I'm using Delphi 10.2 moving to 10.4 next year.

Here is an example of code showing how to write an MD5 into your database and how to read it back and compare with a given MD5 hash:
Inserting data:
procedure TForm1.InsertDataButtonClick(Sender: TObject);
var
MD5 : TArray<Byte>;
begin
MD5 := THashMD5.GetHashBytes('123');
FDConnection1.Connected := TRUE;
FDQuery1.SQL.Text := 'INSERT INTO mytable (TEMP_MD5) VALUES(:MD5)';
FDQuery1.ParamByName('MD5').SetBlobRawData(Length(MD5), PByte(MD5));
FDQuery1.ExecSQL;
Memo1.Lines.Add('Rows affected = ' + FDQuery1.RowsAffected.ToString);
end;
Reading data back and comparing with given hash:
procedure TForm1.ReadDataButtonClick(Sender: TObject);
var
MD5 : TArray<Byte>;
MD5_123 : TArray<Byte>;
FieldMD5 : TField;
RecCnt : Integer;
begin
MD5_123 := THashMD5.GetHashBytes('123');
FDConnection1.Connected := TRUE;
// First version: get all records
// FDQuery1.SQL.Text := 'SELECT TEMP_MD5 FROM mytable';
// Second version: Get only records where TEMP_MD5 is hash('123').
FDQuery1.SQL.Text := 'SELECT TEMP_MD5 FROM mytable WHERE TEMP_MD5 = :MD5';
FDQuery1.ParamByName('MD5').SetBlobRawData(Length(MD5_123), PByte(MD5_123));
// Execute the query
FDQuery1.Open;
RecCnt := 0;
while not FDQuery1.Eof do begin
Inc(RecCnt);
FieldMD5 := FDQuery1.FieldByName('TEMP_MD5');
SetLength(MD5, FieldMD5.DataSize);
FieldMD5.GetData(MD5);
if (Length(MD5) = Length(MD5_123)) and
(CompareMem(PByte(MD5), PByte(MD5_123), Length(MD5))) then
Memo1.Lines.Add(RecCnt.ToString + ') MD5(123) = ' + MD5ToStr(MD5))
else
Memo1.Lines.Add(RecCnt.ToString + ') ' + MD5ToStr(MD5));
FDQuery1.Next;
end;
end;
As you can see reading the code, I compare the MD5 from database with given MD5 by comparing the memory containing the values (arrays of bytes).
Utility function:
function MD5ToStr(MD5 : TArray<Byte>) : String;
var
B : Byte;
begin
Result := '';
for B in MD5 do
Result := Result + B.ToHexString(2);
end;

Related

Insert the records of a dbf file into a table in MS Access database using Delphi

I am trying to insert the records of a dbf file into a table in a MS Access database that I have already created.
The dbf file's name is tab1.dbf and it has three columns: cl1, cl2, cl3.
The MS Access database name is db1 and it has one table tb2 with three columns: cl1, cl2, cl3.
I have connected Delphi to the MS Access database using ADOConnection1.
To insert the dbf file's records, I have to click in a Button1 with OpenDialog1
The code I use is this :
procedure TForm1.Button1Click(Sender: TObject);
var importdir,ipo : string;
begin
if form1.OpenDialog1.Execute then
begin
importdir:= extractfiledir(form1.OpenDialog1.FileName);
ipo:= form1.OpenDialog1.FileName ;
end;
form1.Edit1.Text:= importdir;
ADOConnection1.Execute('insert into tab2 SELECT * FROM [ database = '+ipo+' ].tab1' );
end;
but when i execut the form1 i have this error message :
name of the file is incorrect
can you help me guys ?
Here after is one simple solution. It simple because it assume the Access database structure is the same as the dBASE structure. You'll get started with this example that you'll adapt to your needs.
procedure TDbfToAccessWithAdoForm.DbfToAccessButtonClick(Sender: TObject);
var
Fld : Integer;
FldValue : Variant;
InsertSQL : String;
begin
ADOConnectionAccess.Connected := TRUE;
ADOConnectionDbf.Connected := TRUE;
ADOQueryDbf.SQL.Text := 'Select * from Clients';
ADOQueryDbf.Open;
// Build the parametrized INSERT statement
InsertSQL := 'insert into Clients(';
for Fld := 0 to ADOQueryDbf.FieldCount - 1 do
InsertSQL := InsertSQL + ADOQueryDbf.Fields[Fld].FieldName + ',';
// Remove extra coma
Delete(InsertSQL, Length(InsertSQL), 1);
InsertSQL := InsertSQL + ') values (';
for Fld := 0 to ADOQueryDbf.FieldCount - 1 do
InsertSQL := InsertSQL + ':' + ADOQueryDbf.Fields[Fld].FieldName + ',';
// Remove extra coma
Delete(InsertSQL, Length(InsertSQL), 1);
InsertSQL := InsertSQL + ')';
while not ADOQueryDbf.Eof do begin
ADOQueryAccess.SQL.Text := InsertSQL;
for Fld := 0 to ADOQueryDbf.FieldCount - 1 do begin
FldValue := ADOQueryDbf.Fields[Fld].Value;
// Here you can do whatever conversion is required
if FldValue = Null then begin
if ADOQueryDbf.FieldDefList[Fld].DataType = ftDateTime then
FldValue := 0 // My Access table doesn't like empty datetime
else
FldValue := ' '; // My Access table doesn't like empty string
end;
ADOQueryAccess.Parameters.ParamByName(ADOQueryDbf.Fields[Fld].FieldName).Value := FldValue;
end;
ADOQueryAccess.ExecSQL;
ADOQueryDbf.Next;
end;
ADOQueryDbf.Close;
ADOQueryAccess.Close;
end;
You should add error checking and try/finally or try/except. I let you do it as you do usually.

FireDAC, Array DML, SQL Server, and IDENTITY_INSERT

I'm trying to export data to SQL Server using Firedac Array DML feature. In the destination table, there is a IDENTITY column, and I need to put an explicit value to it.
But my query fails with the following error message:
SQL state: 23000. Native code: 544. Message: [Microsoft][SQL Server
Native Client 11.0][SQL Server]Cannot insert explicit value for
identity column in table 'dic_cities' when IDENTITY_INSERT is set to
OFF.
The destination table is defined as follows:
CREATE TABLE dbo.dic_cities (
id int IDENTITY(1,1) PRIMARY KEY,
city_name varchar(50) NOT NULL
)
My code:
MyFDQuery.Connection.ExecSQL('SET IDENTITY_INSERT dbo.dic_cities ON');
MyFDQuery.SQL.Text := 'INSERT INTO dbo.dic_cities (id, city_name) VALUES (:id, :city_name)';
//Populating parameters and preparing the query
{...}
//Execute Array DML query with batch size of 100
MyFDQuery.Execute(100, 0);
//Finally, set IDENTITY_INSERT off for the destination table
MyFDQuery.Connection.ExecSQL('SET IDENTITY_INSERT dbo.dic_cities OFF');
I must to note, that everything works well when I use a regular TFDQuery with parameters (i.e. when not using Array DML feature). But it fails for Array DML.
I also used Array DML for several other DBMS for years in a way like the code above, with success.
So, how to use the Array DML feature to insert explicit values to SQL Server IDENTITY column?
Update#2: See https://social.msdn.microsoft.com/Forums/sqlserver/en-US/e652377d-0607-45ca-b4a0-274361bff85a/how-to-set-identityinsert-in-dynamic-sql?forum=transactsql
I haven't fully digested it yet, but the OP's problem seems very similar.
I've tried constructing the SQL to use EXEC
FDQuery1.SQL.Text := 'EXEC (' + ''''+ sSetIIOn + ';' + sInsert + ';' + sSetIIOff + '''' + ')';
to avoid sp_executesql being used, but unfortunately FD then cannot parse the SQL properly so it produces an "argument out of range" error when setting up the parameters.
Update: Curiouser and curiouser...
The following code executes without error on SS2014 and inserts the expected 10 rows:
const
sEmptyTable = 'delete from dbo.identtest';
sSetIIOn = 'set identity_insert dbo.identtest ON';
sSetIIOff = 'set identity_insert dbo.identtest OFF';
sSelect = 'select * from dbo.identtest';
sInsert = 'insert dbo.identtest (ID, Name) values(%d, %s)';
procedure TForm2.TestIdentityInsert;
var
i : Integer;
S : String;
begin
FDQuery1.ExecSql(sEmptyTable);
FDQuery1.ExecSql(sSetIIOn);
for i := 1 to 10 do begin
S := Format(sInsert, [i, '''Name' + IntToStr(i) + '''']);
FDQuery1.ExecSQL(S);
end;
FDQuery1.ExecSql(sSetIIOff);
FDQuery1.Sql.Text := sSelect;
FDQuery1.Open;
end;
procedure TForm2.Button1Click(Sender: TObject);
begin
TestIdentityInsert;
end;
However, replacing the for loop by
FDQuery1.SQL.Text := sSetIIOn + ';' + sInsert + ';' + sSetIIOff;
FDQuery1.Params.ArraySize := Rows;
for i := 0 to Rows - 1 do begin
FDQuery1.Params[0].AsIntegers[i] := i;
FDQuery1.Params[1].AsStrings[i] := 'Name' + IntToStr(i);
end;
produces the exception you quote. I've verified using SSMS Profiler that the SQL sent to the server seems to be correct (and not f.i. being mangled by the MDac layer as sometimes happens):
exec sp_executesql N'set identity_insert dbo.identtest ON;insert dbo.identtest values(#P1, #P2);set identity_insert dbo.identtest OFF',N'#P1 int,#P2 nvarchar(4000)',0,N'Name0' [etc, repeated 9 times]
so the question seems to be why doesn't using sp_executesql respect the Identity_Insert setting and is there another way that does?

delphi mysql query result

for read data from database server im using mysql query like this:
FDQuery1.SQL.Text := 'select * from `table` WHERE dcid=3;';
FDQuery1.Open;
memo1.Lines.Add( FDQuery1.FieldByName('value').AsString );
but when i have to use a loop for read data from mysql
i need a array to read and store data on it
i know in php its like this
$arr = array();
while($obj = $q->fetch_object() ){
$arr[] = $obj;
}
but how can i do this syntax in Delphi ?
You don't need to do FDQuery1.First as that is implicit in the FDQuery1.Open. However if you want to know how many records you have it's important to do a FDQuery1.Last; before using FDQuery1.RecordCount to get the true record count otherwise you might get strange results.
All you need to do get your data into a memo is this
FDQuery1.SQL.Text := 'select * from table WHERE dcid=3;';
memo1.Lines.clear;
FDQuery1.Open;
While not FDQuery1.Eof do
begin
memo1.Lines.Add( FDQuery1.FieldByName('value').AsString );
FDQuery1.next;
end;
end;
FDQuery1.Close;
although a better solution with minimal exception handling is
FDQuery1.SQL.Text := 'select * from table WHERE dcid=3;';
memo1.Lines.clear;
try
try
begin
FDQuery1.Open;
While not FDQuery1.Eof do
begin
memo1.Lines.Add( FDQuery1.FieldByName('value').AsString );
FDQuery1.next;
end;
end;
end;
except
on E : Exception do
begin
showmessage ('Exception class name = '+E.ClassName+ slinebreak
+ 'Exception message = '+E.Message);
end //on E
end; //try-except
finally
FDQuery1.Close;
end; //try-finally
You mention an array. This is not needed if you only want to put the data in a memo but if you did want to put the data in an array (a dynamic array of variants as you don't know at design time how many record elements you need, how many field elements you need or what type each field is) then you would use the following code.
(Note this is deliberately not optimised code as I was trying to make the process clear)
Const
FirstRecordIndex = 0;
FirstFieldIndex = 0;
Var
DataArray : Variant;
TheRecordCount, TheFieldCount,
RecordNumber, FieldNumber : integer;
Data : variant;
begin
FDQuery1.SQL.Text := 'select * from table WHERE dcid=3;';
FDQuery1.Open;
FDQuery1.Last; //to get correct recordcount
TheRecordCount := FDQuery1.RecordCount;
TheFieldCount := FDQuery1.FieldCount;
FDQuery1.First; //go back to the beginning of the dataset
//set the dimensions of the 2D array of variants to hold data
DataArray := VarArrayCreate([FirstRecordIndex, TheRecordCount, FirstFieldIndex, TheFieldCount], varVariant ); //element can be of any type
//fill it
RecordNumber := -1; //initialise record indexe to just before the start
While not FDQuery1.Eof do
begin
inc(RecordNumber); //point to next record element in the array
for FieldNumber := FirstFieldIndex to TheFieldCount -1 do //load all the fields of this record
begin
Data := FDQuery1.Fields[FieldNumber].asVariant; //get the data
DataArray[RecordNumber, FieldNumber] := Data; //put into array
end;
FDQuery1.next; //get next record
end; //while
end;
FDQuery1.Close;
end;
To get the data back again use
For RecordNumber := FirstRecordIndex to TheRecordCount -1 do
For FieldNumber := FirstFieldIndex to TheFieldCount -1 do
begin
Data := DataArray[RecordNumber, FieldNumber] ;
//do something with the data ie put into a memo
end;
You can use this, i hope that help u:
FDQuery1.SQL.Text := 'select * from `table` WHERE dcid=3;';
FDQuery1.Open;
if FDQuery1.Active and (FDQuery1.RecordCount > 0) then
begin
FDQuery1.First;
While not FDQuery1.Eof do
begin
memo1.Lines.Add( FDQuery1.FieldByName('value').AsString );
FDQuery1.next;
end;
end;
you can do this with array of array consept
for this first you must create your new type
type
myArray = array [1..10] of integer;
myData= array of myArray;
and then you must define your variable
var
Records: myData;
i:integer;
on,now we have aarray of array to save database records on it so
begin
FDQuery1.SQL.Text := 'select * from table WHERE type=3;';
FDQuery1.Open;
FDQuery1.First;
SetLength(Records ,FDQuery1.recordcount);//or maybe you need a do a select count(*) query first
i:=0;
While not FDQuery1.Eof do
begin
Records[i][1]:= FDQuery1.FieldByName('value1').AsString;
Records[i][2]:= FDQuery1.FieldByName('value2').AsString;
Records[i][3]:= FDQuery1.FieldByName('value3').AsString;
Records[i][4]:= FDQuery1.FieldByName('value4').AsString;
Records[i][5]:= FDQuery1.FieldByName('value5').AsString;
Records[i][6]:= FDQuery1.FieldByName('value6').AsString;
i:=i+1;
FDQuery1.next;
end;
FDQuery1.Close;
end;
now all of your selected database records are in your local variable
you can go over the row and make new queries with FDQuery1...

Extracting very long string from JSON to CLOB

I'm trying to extract a very long string into clob from json_object_t and got some weird database behaviour (12.2c) with json_object_t.get_clob(key) method.
There is a sample code than does following:
DECLARE
l_data CLOB := '{"text": "very long string about 1M chars"}';
l_json json_object_t;
l_text CLOB := EMPTY_CLOB();
BEGIN
l_json := json_object_t.parse(l_data);
l_text := l_json.get_clob('text');
dbms_output.put_line('got ' || dbms_lob.getlength(l_text) || ' chars');
END;
When string length in a 'text' key is less than 32k chars, get_clob method works just fine and shows appropriate result, but with longer strings it produces an empty clob with zero length, just like get_string, but without 'character string buffer too small' exception.
I've tried to get same data via json_table query, but it cannot extract data to clob column at all, only varchar/number is allowed.
Is that a bug or am I doing something wrong? Is there any other ways to extract long strings from JSON keys?
I work with Oracle Database JSON Store group and would be happy to assist you with this issue you're facing. Could you try the alternate get_Clob procedure instead of this function and tell us what the behavior is?
Signature:
MEMBER PROCEDURE get_Clob(key VARCHAR2, c IN OUT CLOB)
Please try this:
DECLARE
content_json CLOB := '{"value":"';
content_json_end CLOB := '"}';
content_tmp CLOB := 'ab';
l_json json_object_t;
l_text CLOB := EMPTY_CLOB();
tmp clob;
BEGIN
-- 13 gives 16K
-- 14 gives 32K
FOR count IN 1 .. 14
loop
dbms_lob.append(content_tmp, content_tmp); -- a bad append for now
END loop;
dbms_lob.append(content_json, content_tmp);
dbms_lob.append(content_json, content_json_end);
l_json := json_object_t.parse(content_json);
l_json.get_clob('value', l_text); -- !!! TRY THIS PROC get_Clob
--l_text := l_json.get_clob('value');
dbms_output.put_line('Lob size in Kb: ');
dbms_output.put_line(dbms_lob.getLength(l_text) / 1024);
END;
/
Looking forward to your findings..
This works as well. Instead using the the get_clob method, use c:
DECLARE
CURSOR crsrJSON IS
SELECT
json_object( 'employee_id' VALUE employee_id,
'first_name' VALUE first_name,
'last_name' VALUE last_name,
'email' VALUE email,
'phone_number' VALUE phone_number,
'hire_date' VALUE to_char(hire_date,'MM/DD/YYYY'),
'job_id' VALUE job_id,
'salary' VALUE nvl(salary,0),
'commission_pct' VALUE nvl(commission_pct,0),
'manager_id' VALUE NVL(manager_id,0),
'department_id' VALUE NVL(department_id,0),
'department_name' VALUE (select department_name from departments x where x.department_id = hr.department_id),
'job_title' VALUE (select job_title from jobs x where x.job_id = hr.job_id)) emp_data
FROM
employees hr;
js_array JSON_ARRAY_T := new JSON_ARRAY_T;
json_obj JSON_OBJECT_T := JSON_OBJECT_T();
json_clob CLOB := EMPTY_CLOB();
BEGIN
FOR data_rec IN crsrJSON LOOP
js_array.append(JSON_ELEMENT_T.parse(data_rec.emp_data));
END LOOP;
json_obj.put('data',js_array);
IF json_obj.has('data') THEN
json_clob := json_obj.to_clob;
DBMS_OUTPUT.PUT_LINE(json_clob);
ELSE
DBMS_OUTPUT.PUT_LINE('Nope');
END IF;
END;
with data as
( select
xmlelement(e,regexp_replace('{"name":"'||colname||'"}', '[[:cntrl:]]', ''),',') col1
from tblname
)
select
rtrim(replace(replace(replace(xmlagg(col1).getclobval(),'&'||'quot;','"'),'<E>',''),'</E>',''),',')
as very_long_json
from data;

MySQL prepared statements result metadata

I have selective procedure
CREATE PROCEDURE sp_d_test()
NOT DETERMINISTIC
CONTAINS SQL
SQL SECURITY DEFINER
COMMENT ''
BEGIN
SELECT * FROM my_table;
END;
and I want to get metadata results (field names) using prepared statements
procedure TForm1.btnLowAPIClick(Sender: TObject);
var
LConn: Pointer;
LSQL: AnsiString;
LStmt: PMYSQL_STMT;
LCnt: Integer;
LFld: PMYSQL_FIELD;
LRes: PMYSQL_RES;
begin
LConn := MySQLInit;
mysql_real_connect(LConn, 'localhost', 'root', '', 'test', 0, nil, CLIENT_MULTI_RESULTS);
MySQLQuery(LConn, 'SET character_set_results=ucs2');
LSQL := 'CALL sp_d_test()';
LStmt := mysql_stmt_init(LConn);
if mysql_stmt_prepare(LStmt, PAnsiChar(LSQL), Length(LSQL)) <> 0 then
MySQLStmtError(LStmt);
if mysql_stmt_execute(LStmt) <> 0 then
MySQLStmtError(LStmt);
LCnt := mysql_stmt_field_count(LStmt); // Returns corrected number
LRes := mysql_stmt_result_metadata(LStmt);
LFld := mysql_fetch_field(LRes);
while LFld <> nil do begin
LStr := UCS2BytesToUTF16(PByte(LFld^.name), LFld^.name_length);
LFld := mysql_fetch_field(LRes);
end;
end;
The fields LFld^.name, LFld^.name_length contains garbage. The field LFld^.type contains corrected type
If I do
MySQLQuery(LConn, 'SET character_set_results=utf8');
then the field LFld^.name contains corrected value, but LFld^.name_length still contains garbage.
For simple select
LSQL := 'SELECT * FROM my_table';
The structure is filled correctly and for UCS2 and for UTF8. With one BUT. The documentation says
char * name
The name of the field, as a null-terminated string. If the field was given an alias with an AS clause, the value of name is the alias. For a procedure parameter, the parameter name.
For UCS2 the field name is NOT null-terminated, but name_length contains real size in bytes
Server version 5.1.41