I'm struggling to reverse engineer a section of data associated with a CRC-16 checksum.
I know the polynom used to calculate the original checksums is 0x8408 but nothing else, I don't know initial value (if any), final XOR value (if any), if the input or the result is reflected...
It seems like there is a known CRC-16 generator using thing polynom, CRC-16-CCITT but despite everything I've tried I just can't understand how the original checksum is being calculated.
Here is the data I've got with their respective checksums. I also included a byte that is between the data and the checksum, it's incremental and I'm not whether it's calculated or not. (see the last two lines, data is almost the same, increment is not the same and yet the checksum are identical)
| DATA |Inc|CRC|
|----------------------------------------------------------|---|---|
00 00 00 00 00 00 01 ef f7 fe ef ff fd ef fb fa fd a2 aa 21 01 f4 e0
00 00 00 00 00 00 01 ef f7 fd ef ff fd fe fb fa fd a2 aa 21 02 f4 d1
00 00 00 00 00 00 01 f7 fe fd fd ff fd df ff fb fd a2 aa 21 03 f4 cd
00 00 00 00 00 00 01 f7 fe fe fd ff f7 ef ff fa fd a2 aa 21 04 f4 c2
00 00 00 00 00 00 01 ef f7 fe ef ff fe ef fb fa fd a2 aa 21 05 f4 db
00 00 00 00 00 00 01 ef f7 fe ef ff fd ef fb fa fd a2 aa 21 06 f4 db
The last byte of each line appears to be 0xF3 + the negative sum of all but the last byte (including the 0xF4). This code works for the 5 examples:
typedef unsigned char uint8_t;
static uint8_t data0[] =
{0x00,0x00,0x00,0x00,0x00,0x00,0x01,0xef,0xf7,0xfe,0xef,
0xff,0xfd,0xef,0xfb,0xfa,0xfd,0xa2,0xaa,0x21,0x01,0xf4,0xe0};
static uint8_t data1[] =
{0x00,0x00,0x00,0x00,0x00,0x00,0x01,0xef,0xf7,0xfd,0xef,
0xff,0xfd,0xfe,0xfb,0xfa,0xfd,0xa2,0xaa,0x21,0x02,0xf4,0xd1};
static uint8_t data2[] =
{0x00,0x00,0x00,0x00,0x00,0x00,0x01,0xf7,0xfe,0xfd,0xfd,
0xff,0xfd,0xdf,0xff,0xfb,0xfd,0xa2,0xaa,0x21,0x03,0xf4,0xcd};
static uint8_t data3[] =
{0x00,0x00,0x00,0x00,0x00,0x00,0x01,0xf7,0xfe,0xfe,0xfd,
0xff,0xf7,0xef,0xff,0xfa,0xfd,0xa2,0xaa,0x21,0x04,0xf4,0xc2};
static uint8_t data4[] =
{0x00,0x00,0x00,0x00,0x00,0x00,0x01,0xef,0xf7,0xfe,0xef,0xff,
0xfe,0xef,0xfb,0xfa,0xfd,0xa2,0xaa,0x21,0x05,0xf4,0xdb};
static uint8_t data5[] =
{0x00,0x00,0x00,0x00,0x00,0x00,0x01,0xef,0xf7,0xfe,0xef,0xff,
0xfd,0xef,0xfb,0xfa,0xfd,0xa2,0xaa,0x21,0x06,0xf4,0xdb};
int main()
{
size_t i;
uint8_t c = 0xf3;
uint8_t s;
s = c;
for(i = 0; i < sizeof(data0)-1; i++)
s -= data0[i];
if (data0[i] != s)
printf("mismatch\n");
s = c;
for (i = 0; i < sizeof(data1) - 1; i++)
s -= data1[i];
if (data1[i] != s)
printf("mismatch\n");
s = c;
for (i = 0; i < sizeof(data2) - 1; i++)
s -= data2[i];
if (data2[i] != s)
printf("mismatch\n");
s = c;
for (i = 0; i < sizeof(data3) - 1; i++)
s -= data3[i];
if (data3[i] != s)
printf("mismatch\n");
s = c;
for (i = 0; i < sizeof(data2) - 1; i++)
s -= data4[i];
if (data4[i] != s)
printf("mismatch\n");
return 0;
}
S1=[20 32 44 56 68 80 92 104 116 128 140 152 164 176 188 200];
P=[16.82 26.93 37.01 47.1 57.21 67.32 77.41 87.5 97.54 107.7 117.8 127.9 138 148 158.2 168.3];
X = [0.119 0.191 0.262 0.334 0.405 0.477 0.548 0.620 0.691 0.763 0.835 0.906 0.978 1.049 1.120 1.192];
S = [2.3734 3.6058 5.0256 6.6854 8.6413 10.978 13.897 17.396 21.971 28.040 36.475 49.065 69.736 110.20 224.69 2779.1];
objective=#(x)((1250*x(3)*S(a)-(S(a)+x(2))*(P(a)+x(1)))/(1250*(S(a)+x(2))*(P(a)+x(1)))-x(5))^2+((x(2)*(P(a)^2+x(1)*P(a)))/(1250*x(4)*X(a)*x(3)-P(a)^2-x(1)*P(a))-S(a))^2+(74000/3*((X(a)*x(3)*S(a))/S1(a)*(S(a)+x(2)))-P(a))^2
%x0 = [Kp Ks mu.m Yp mu.d]
x0=[7.347705469 14.88611028 1.19747242 16.65696429 6.01E-03];
x=fminunc(objective,x0);
disp(x)
The code above is used for optimisizing the objective function, so that all the unknown values of the parameters can be found. As you may have seen, the objective function consists of 4 variables (S1, S, P, X), each having 16 data entities. My question is: how to create an objective function, so that all the data entities are utilised?
The final objective function has to be the sum of the objective function (shown above) with a=1:16. Any ideas?
Make the following changes to your code:
Replace, e.g. all S(a) variables with S to use the whole vector. Do the same for each of your four variables.
Convert all 'scalar' operations in your objective function to 'elementwise' ones, i.e. replace ^, * and / with .^, .* and ./. This produces 16 values, one for each index from 1 to 16 (i.e. what was previously referred to by a).
wrap the resulting expression into a sum() function to sum the 16 results into a final value
Use your optimiser as normal.
Resulting code:
S1 = [20 32 44 56 68 80 92 104 116 128 140 152 164 176 188 200];
P = [16.82 26.93 37.01 47.1 57.21 67.32 77.41 87.5 97.54 107.7 117.8 127.9 138 148 158.2 168.3];
X = [0.119 0.191 0.262 0.334 0.405 0.477 0.548 0.620 0.691 0.763 0.835 0.906 0.978 1.049 1.120 1.192];
S = [2.3734 3.6058 5.0256 6.6854 8.6413 10.978 13.897 17.396 21.971 28.040 36.475 49.065 69.736 110.20 224.69 2779.1];
objective = #(x) sum( ((1250.*x(3).*S-(S+x(2)).*(P+x(1)))./(1250.*(S+x(2)).*(P+x(1)))-x(5)).^2+((x(2).*(P.^2+x(1).*P))./(1250.*x(4).*X.*x(3)-P.^2-x(1).*P)-S).^2+(74000./3.*((X.*x(3).*S)./S1.*(S+x(2)))-P).^2 );
%x0 = [Kp Ks mu.m Yp mu.d]
x0 = [7.347705469 14.88611028 1.19747242 16.65696429 6.01E-03];
x = fminunc(objective,x0);
disp(x)
Note that you can make this code a lot clearer to read for humans; I just made the "direct" changes that illustrate conversion from your scalar expression to the desired vectorised one.
I'm trying to write to a hex file using PB12.5, I'm able to write to it without any issues but through testing noticed I will need to send a null value (00) to the file at certain points.
I know if I assign null to a string, it will null out the entire string so I tried using a Blob where I can insert a null value when needed (BlobEdit(blb_data, ll_pos, CharA(0)) )
But BlobEdit() automatically inserts a null value in between each position, I don't want this as it's causing issues as I'm trying to update the hex file. I just need to add my CharA(lb_byte) to each consecutive position in the Blob.
Is there any way around this or is PB just unable to do this? Below is the code:
ll_test = 1
ll_pos = 1
ll_length = Len(ls_output)
Do While ll_pos <= (ll_length)
ls_data = Mid(ls_output, ll_pos, 2)
lb_byte = Event ue_get_decimal_value_of_hex(ls_data)
ll_test = BlobEdit(blb_data, ll_test, CharA(lb_byte), EncodingANSI!)
ll_pos = ll_pos + 2
Loop
Hex file appears as follows:
16 35 2D D8 08 45 29 18 35 27 76 25 30 55 66 85 44 66 57 A4 67 99
After Blob update:
16 00 48 00 5D 00 C3 92 00 08 00 48 00 51 00 E2
I hope to help you:
//////////////////////////////////////////////////////////////////////////
// Function: f_longtohex
// Description: LONG to HEXADECIMAL
// Ambito: public
// Argumentos: as_number //Variable long to convert to hexadecimal
// as_digitos //Number of digits to return
// Return: String
// Example:
// f_longtohex(198 , 2) --> 'C6'
// f_longtohex(198 , 4) --> '00C6'
//////////////////////////////////////////////////////////////////////////
long ll_temp0, ll_temp1
char lc_ret
if isnull(as_digitos) then as_digitos = 2
IF as_digitos > 0 THEN
ll_temp0 = abs(as_number / (16 ^ (as_digitos - 1)))
ll_temp1 = ll_temp0 * (16 ^ (as_digitos - 1))
IF ll_temp0 > 9 THEN
lc_ret = char(ll_temp0 + 55)
ELSE
lc_ret = char(ll_temp0 + 48)
END IF
RETURN lc_ret + f_longtohex(as_number - ll_temp1 , as_digitos - 1)
END IF
RETURN ''
I'm using node-mysql2 and storing string-encoded binary data in MySQL. The column has a type of binary(16). The data stored in MySQL is being corrupted, for example:
NodeJS output: 47 23 43 14 ed 86 14 dc 12 f3 b8 6c dc 31 fb fa
MySQL HEX() : 47 23 43 14 c3 ad c2 86 14 c3 9c 12 c3 b3 c2 b8
Code:
var binaryString = randomBinaryString();
db.sqlQuery("INSERT INTO test(binaryString) VALUES(" + db.escape(binaryString) + ")");
console.log(new Buffer(binaryString, 'binary').toString('hex'));
function randomBinaryString(){
var str = "";
for(var i = 0; i < 16; i++){
str += String.fromCharCode(Math.floor(Math.random()*256));
}
return str;
}
How should a string that actually encodes binary data (each character being a byte) be stored in MySQL using node-mysql2?
You're not inserting a raw binary string but a UTF-8 encoded string. In UTF-8, codepoints after 127 are encoded using multiple bytes. The fifth byte ed (237) is encoded using 2 bytes.
Buffer(String.fromCharCode(0xed)) produces : <Buffer c3 ad>
Sending the random string as a Buffer with binary encoding should fix the problem. node-mysql will convert the buffer to a hex string when inserting.
db.query('INSERT INTO test VALUES (?)', [ Buffer(randomString(), 'binary') ]);
Also, the crypto module has a randomBytes method that generates n random bytes.
db.query('INSERT INTO test VALUES (?)', [crypto.randomBytes(16)]);
I'm trying to connect to the Safecom TA-810 (badge/registration system) to automate the process of calculating how long employee's have worked each day. Currently this is done by:
Pulling the data into the official application
Printing a list of all 'registrations'
Manually entering the values from the printed lists into our HR application
This is a job that can take multiple hours which we'd like to see automated. So far the official tech support has been disappointing and refused to share any details.
Using wireshark I have been capturing the UDP transmissions and have pretty much succeeded in understanding how the protocol is built up. I'm only having issues with what i suppose is a CRC field. I don't know how it is calculated (CRC type and parameters) and using which fields ...
This is how a message header looks like:
D0 07 71 BC BE 3B 00 00
D0 07 - Message type
71 BC - This i believe is the CRC
BE 3B - Some kind of session identifier. Stays the same for every message after the initial message (initial message has '00 00' as value)
00 00 - Message number. '01 00', '02 00', '03 00'
Some examples:
Header only examples
E8 03 17 FC 00 00 00 00 -> initial request (#0, no session nr)
D0 07 71 BC BE 3B 00 00 -> Initial response (#0, device sends a session nr)
4C 04 EF BF BE 3B 06 00 -> Message #6, still using the same session # as the initial response
Larger example, which has data
0B 00 07 E1 BE 3B 01 00 7E 45 78 74 65 6E 64 46 6D 74
I've also been trying to figure this out by reading the disassembled code from the original application. The screenshot below happens before the socket.sendto and seems to be related.
Any help will be extremely appreciated.
EDIT: Made some success with debugging the application using ollydbg. The CRC appears in register (reversed) EDX at the selected line in the following screenshot.
Take a look at CRC RevEng. If you can correctly identify the data that the CRC is operating on and the location of the CRC, you should be able to determine the CRC parameters. If it is a CRC.
I've managed to create a php script that does the CRC calculation by debugging the application using OllyDbg.
The CRC is calculated by Adding up every 2 bytes (every short). if the result is larger than a short, the 'most significant short' is added to the 'least significant short' until the result fits in a short. Finally, the CRC (short) is inverted.
I'll add my php script for completeness:
<?php
function CompareHash($telegram)
{
$telegram = str_replace(" ", "", $telegram);
$telegram_crc = substr($telegram, 4, 4);
$telegram = str_replace($telegram_crc, "0000", $telegram);
echo "Telegram: ", $telegram, ', Crc: ', $telegram_crc, ' (', hexdec($telegram_crc), ')<br />';
$crc = 0;
$i = 0;
while ($i < strlen($telegram))
{
$short = substr($telegram, $i, 4);
if (strlen($short) < 4) $short = $short . '00';
$crc += hexdec($short);
$i += 4;
}
echo "Crc: ", $crc, ', inverse: ', ~$crc;
// Region "truncate CRC to Int16"
while($crc > hexdec('FFFF'))
{
$short = $crc & hexdec ('FFFF');
// Region "unsigned shift right by 16 bits"
$crc = $crc >> 16;
$crc = $crc & hexdec ('FFFF');
// End region
$crc = $short + $crc;
}
// End region
// Region "invert Int16"
$crc = ~$crc;
$crc = $crc & hexdec ('FFFF');
// End region
echo ', shifted ', $crc;
if (hexdec($telegram_crc) == $crc)
{
echo "<br />MATCH!!! <br />";
}
else
{
echo "<br />failed .... <br />";
}
}
$s1_full = "E8 03 17 FC 00 00 00 00";
$s2_full = "D0 07 71 BC BE 3B 00 00";
$s3_full = "D0 07 4E D4 E1 23 00 00";
$s4_full = "D0 07 35 32 BE 3B 07 00 7E 44 65 76 69 63 65 4E 61 6D 65 3D 54 41 38 31 30 00";
$s5_full = "0B 00 39 6C BE 3B 05 00 7E 52 46 43 61 72 64 4F 6E";
CompareHash($s1_full);
CompareHash($s2_full);
CompareHash($s3_full);
CompareHash($s4_full);
CompareHash($s5_full);
?>
Thanks for the feedback!