How to convert my binary (hex) data to latitude and longitude? - binary

I have some binary data stream which passes geo location coordinates - latitude and longitude. I need to find the method they are encoded.
4adac812 = 74°26.2851' = 74.438085
2b6059f9 = 43°0.2763' = 43.004605
4adaee12 = 74°26.3003' = 74.438338
2a3c8df9 = 42°56.3177' = 42.938628
4ae86d11 = 74°40.1463' = 74.669105
2afd0efb = 42°59.6263' = 42.993772
1st value is hex value. 2nd & 3rd are values that I get in output (not sure which one is used in conversion).
I've found that first byte represents integer part of value (0x4a = 74). But I cannot find how decimal part is encoded.
I would really appreciate any help!
Thanks.
--
Upd: This stream comes from some "chinese" gps server software through tcp protocol. I have no sources or documentation for clent software. I suppose it was written in VC++6 and uses some standard implementations.
--
Upd: Here is packets I get:
Hex data:
41 00 00 00 13 bd b2 2c
4a e8 6d 11 2a 3c 8d f9
f6 0c ee 13
Log data in client soft:
[Lng] 74°40.1463', direction:1
[Lat] 42°56.3177', direction:1
[Head] direction:1006, speed:3318, AVA:1
[Time] 2011-02-25 19:52:19
Result data in client (UI):
74.669105
42.938628
Head 100 // floor(1006/10)
Speed 61.1 // floor(3318/54.3)
41 00 00 00 b1 bc b2 2c
4a da ee 12 2b 60 59 f9
00 00 bc 11
[Lng] 74°26.3003', direction:1
[Lat] 43°0.2763', direction:1
[Head] direction:444, speed:0, AVA:1
[Time] 2011-02-25 19:50:49
74.438338
43.004605
00 00 00 00 21 bd b2 2c
4a da c8 12 aa fd 0e fb
0d 0b e1 1d
[Lng] 74°26.2851', direction:1
[Lat] 42°59.6263', direction:1
[Head] direction:3553, speed:2829, AVA:1
[Time] 2011-02-25 19:52:33
74.438085
42.993772
I don't know what first 4 bytes mean.
I found the lower 7 bits of 5th byte represent the number of sec. (maybe 5-8 bits are time?)
Byte 9 represent integer of Lat.
Byte 13 is integer of Lng.
Bytes 17-18 reversed (word byte) is speed.
Bytes 19-20 reversed is ava(?) & direction (4 + 12 bits). (btw, somebody knows what ava is?)
And one note. In 3rd packet 13th byte you can see only lower 7 bits are used. I guess 1st bit doesnt mean smth (I removed it in the beginning, sorry if I'm wrong).

I have reordered your data so that we first have 3 longitures and then 3 latitudes:
74.438085, 74.438338, 74.669105, 43.004605, 42.938628, 42.993772
This is the best fit of the hexadecimals i can come up with is:
74.437368, 74.439881, 74.668392, 42.993224, 42.961388, 42.982391
The differences are: -0.000717, 0.001543, -0.000713, -0.011381, 0.022760, -0.011381
The program that generates these values from the complete Hex'es (4 not 3 bytes) is:
int main(int argc, char** argv) {
int a[] = { 0x4adac812, 0x4adaee12, 0x4ae86d11, 0x2b6059f9, 0x2a3c8df9, 0x2afd0efb };
int i = 0;
while(i<3) {
double b = (double)a[i] / (2<<(3*8)) * 8.668993 -250.0197;
printf("%f\n",b);
i++;
}
while(i<6) {
double b = (double)a[i] / (2<<(3*8)) * 0.05586007 +41.78172;
printf("%f\n",b);
i++;
}
printf("press key");
getch();
}

Brainstorming here.
If we look at the lower 6 bits of the second byte (data[1]&0x3f) we get the "minutes" value for most of the examples.
0xda & 0x3f = 0x1a = 26; // ok
0x60 & 0x3f = 0; // ok
0xe8 & 0x3f = 0x28 = 40; // ok
0x3c & 0x3f = 0x3c = 60; // should be 56
0xfd & 0x3f = 0x3d = 61; // should be 59
Perhaps this is the right direction?

I have tried your new data packets:
74+40.1463/60
74+26.3003/60
74+26.2851/60
42+56.3177/60
43+0.2763/60
42+59.6263/60
74.66910, 74.43834, 74.43809, 42.93863, 43.00460, 42.99377
My program gives:
74.668392, 74.439881, 74.437368, 42.961388, 42.993224, 39.407346
The differences are:
-0.000708, 0.001541, -0.000722, 0.022758, -0.011376, -3.586424
I re-used the 4 constants i derived from your first packet as those are probably stored in your client somewhere. The slight differences might be the result of some randomization the client does to prevent you from getting the exact value or reverse-engineering their protocol.

Related

Find used CRC-16 algorithm

I'm struggling to reverse engineer a section of data associated with a CRC-16 checksum.
I know the polynom used to calculate the original checksums is 0x8408 but nothing else, I don't know initial value (if any), final XOR value (if any), if the input or the result is reflected...
It seems like there is a known CRC-16 generator using thing polynom, CRC-16-CCITT but despite everything I've tried I just can't understand how the original checksum is being calculated.
Here is the data I've got with their respective checksums. I also included a byte that is between the data and the checksum, it's incremental and I'm not whether it's calculated or not. (see the last two lines, data is almost the same, increment is not the same and yet the checksum are identical)
| DATA |Inc|CRC|
|----------------------------------------------------------|---|---|
00 00 00 00 00 00 01 ef f7 fe ef ff fd ef fb fa fd a2 aa 21 01 f4 e0
00 00 00 00 00 00 01 ef f7 fd ef ff fd fe fb fa fd a2 aa 21 02 f4 d1
00 00 00 00 00 00 01 f7 fe fd fd ff fd df ff fb fd a2 aa 21 03 f4 cd
00 00 00 00 00 00 01 f7 fe fe fd ff f7 ef ff fa fd a2 aa 21 04 f4 c2
00 00 00 00 00 00 01 ef f7 fe ef ff fe ef fb fa fd a2 aa 21 05 f4 db
00 00 00 00 00 00 01 ef f7 fe ef ff fd ef fb fa fd a2 aa 21 06 f4 db
The last byte of each line appears to be 0xF3 + the negative sum of all but the last byte (including the 0xF4). This code works for the 5 examples:
typedef unsigned char uint8_t;
static uint8_t data0[] =
{0x00,0x00,0x00,0x00,0x00,0x00,0x01,0xef,0xf7,0xfe,0xef,
0xff,0xfd,0xef,0xfb,0xfa,0xfd,0xa2,0xaa,0x21,0x01,0xf4,0xe0};
static uint8_t data1[] =
{0x00,0x00,0x00,0x00,0x00,0x00,0x01,0xef,0xf7,0xfd,0xef,
0xff,0xfd,0xfe,0xfb,0xfa,0xfd,0xa2,0xaa,0x21,0x02,0xf4,0xd1};
static uint8_t data2[] =
{0x00,0x00,0x00,0x00,0x00,0x00,0x01,0xf7,0xfe,0xfd,0xfd,
0xff,0xfd,0xdf,0xff,0xfb,0xfd,0xa2,0xaa,0x21,0x03,0xf4,0xcd};
static uint8_t data3[] =
{0x00,0x00,0x00,0x00,0x00,0x00,0x01,0xf7,0xfe,0xfe,0xfd,
0xff,0xf7,0xef,0xff,0xfa,0xfd,0xa2,0xaa,0x21,0x04,0xf4,0xc2};
static uint8_t data4[] =
{0x00,0x00,0x00,0x00,0x00,0x00,0x01,0xef,0xf7,0xfe,0xef,0xff,
0xfe,0xef,0xfb,0xfa,0xfd,0xa2,0xaa,0x21,0x05,0xf4,0xdb};
static uint8_t data5[] =
{0x00,0x00,0x00,0x00,0x00,0x00,0x01,0xef,0xf7,0xfe,0xef,0xff,
0xfd,0xef,0xfb,0xfa,0xfd,0xa2,0xaa,0x21,0x06,0xf4,0xdb};
int main()
{
size_t i;
uint8_t c = 0xf3;
uint8_t s;
s = c;
for(i = 0; i < sizeof(data0)-1; i++)
s -= data0[i];
if (data0[i] != s)
printf("mismatch\n");
s = c;
for (i = 0; i < sizeof(data1) - 1; i++)
s -= data1[i];
if (data1[i] != s)
printf("mismatch\n");
s = c;
for (i = 0; i < sizeof(data2) - 1; i++)
s -= data2[i];
if (data2[i] != s)
printf("mismatch\n");
s = c;
for (i = 0; i < sizeof(data3) - 1; i++)
s -= data3[i];
if (data3[i] != s)
printf("mismatch\n");
s = c;
for (i = 0; i < sizeof(data2) - 1; i++)
s -= data4[i];
if (data4[i] != s)
printf("mismatch\n");
return 0;
}

Writing multivariate objective function, with matrix data entities

S1=[20 32 44 56 68 80 92 104 116 128 140 152 164 176 188 200];
P=[16.82 26.93 37.01 47.1 57.21 67.32 77.41 87.5 97.54 107.7 117.8 127.9 138 148 158.2 168.3];
X = [0.119 0.191 0.262 0.334 0.405 0.477 0.548 0.620 0.691 0.763 0.835 0.906 0.978 1.049 1.120 1.192];
S = [2.3734 3.6058 5.0256 6.6854 8.6413 10.978 13.897 17.396 21.971 28.040 36.475 49.065 69.736 110.20 224.69 2779.1];
objective=#(x)((1250*x(3)*S(a)-(S(a)+x(2))*(P(a)+x(1)))/(1250*(S(a)+x(2))*(P(a)+x(1)))-x(5))^2+((x(2)*(P(a)^2+x(1)*P(a)))/(1250*x(4)*X(a)*x(3)-P(a)^2-x(1)*P(a))-S(a))^2+(74000/3*((X(a)*x(3)*S(a))/S1(a)*(S(a)+x(2)))-P(a))^2
%x0 = [Kp Ks mu.m Yp mu.d]
x0=[7.347705469 14.88611028 1.19747242 16.65696429 6.01E-03];
x=fminunc(objective,x0);
disp(x)
The code above is used for optimisizing the objective function, so that all the unknown values of the parameters can be found. As you may have seen, the objective function consists of 4 variables (S1, S, P, X), each having 16 data entities. My question is: how to create an objective function, so that all the data entities are utilised?
The final objective function has to be the sum of the objective function (shown above) with a=1:16. Any ideas?
Make the following changes to your code:
Replace, e.g. all S(a) variables with S to use the whole vector. Do the same for each of your four variables.
Convert all 'scalar' operations in your objective function to 'elementwise' ones, i.e. replace ^, * and / with .^, .* and ./. This produces 16 values, one for each index from 1 to 16 (i.e. what was previously referred to by a).
wrap the resulting expression into a sum() function to sum the 16 results into a final value
Use your optimiser as normal.
Resulting code:
S1 = [20 32 44 56 68 80 92 104 116 128 140 152 164 176 188 200];
P = [16.82 26.93 37.01 47.1 57.21 67.32 77.41 87.5 97.54 107.7 117.8 127.9 138 148 158.2 168.3];
X = [0.119 0.191 0.262 0.334 0.405 0.477 0.548 0.620 0.691 0.763 0.835 0.906 0.978 1.049 1.120 1.192];
S = [2.3734 3.6058 5.0256 6.6854 8.6413 10.978 13.897 17.396 21.971 28.040 36.475 49.065 69.736 110.20 224.69 2779.1];
objective = #(x) sum( ((1250.*x(3).*S-(S+x(2)).*(P+x(1)))./(1250.*(S+x(2)).*(P+x(1)))-x(5)).^2+((x(2).*(P.^2+x(1).*P))./(1250.*x(4).*X.*x(3)-P.^2-x(1).*P)-S).^2+(74000./3.*((X.*x(3).*S)./S1.*(S+x(2)))-P).^2 );
%x0 = [Kp Ks mu.m Yp mu.d]
x0 = [7.347705469 14.88611028 1.19747242 16.65696429 6.01E-03];
x = fminunc(objective,x0);
disp(x)
Note that you can make this code a lot clearer to read for humans; I just made the "direct" changes that illustrate conversion from your scalar expression to the desired vectorised one.

Trying to write to a Hex file(.bin) using PB12.5 using BlobEdit (Blob)

I'm trying to write to a hex file using PB12.5, I'm able to write to it without any issues but through testing noticed I will need to send a null value (00) to the file at certain points.
I know if I assign null to a string, it will null out the entire string so I tried using a Blob where I can insert a null value when needed (BlobEdit(blb_data, ll_pos, CharA(0)) )
But BlobEdit() automatically inserts a null value in between each position, I don't want this as it's causing issues as I'm trying to update the hex file. I just need to add my CharA(lb_byte) to each consecutive position in the Blob.
Is there any way around this or is PB just unable to do this? Below is the code:
ll_test = 1
ll_pos = 1
ll_length = Len(ls_output)
Do While ll_pos <= (ll_length)
ls_data = Mid(ls_output, ll_pos, 2)
lb_byte = Event ue_get_decimal_value_of_hex(ls_data)
ll_test = BlobEdit(blb_data, ll_test, CharA(lb_byte), EncodingANSI!)
ll_pos = ll_pos + 2
Loop
Hex file appears as follows:
16 35 2D D8 08 45 29 18 35 27 76 25 30 55 66 85 44 66 57 A4 67 99
After Blob update:
16 00 48 00 5D 00 C3 92 00 08 00 48 00 51 00 E2
I hope to help you:
//////////////////////////////////////////////////////////////////////////
// Function: f_longtohex
// Description: LONG to HEXADECIMAL
// Ambito: public
// Argumentos: as_number //Variable long to convert to hexadecimal
// as_digitos //Number of digits to return
// Return: String
// Example:
// f_longtohex(198 , 2) --> 'C6'
// f_longtohex(198 , 4) --> '00C6'
//////////////////////////////////////////////////////////////////////////
long ll_temp0, ll_temp1
char lc_ret
if isnull(as_digitos) then as_digitos = 2
IF as_digitos > 0 THEN
ll_temp0 = abs(as_number / (16 ^ (as_digitos - 1)))
ll_temp1 = ll_temp0 * (16 ^ (as_digitos - 1))
IF ll_temp0 > 9 THEN
lc_ret = char(ll_temp0 + 55)
ELSE
lc_ret = char(ll_temp0 + 48)
END IF
RETURN lc_ret + f_longtohex(as_number - ll_temp1 , as_digitos - 1)
END IF
RETURN ''

Binary data stored in MySQL is getting corrupted with Node

I'm using node-mysql2 and storing string-encoded binary data in MySQL. The column has a type of binary(16). The data stored in MySQL is being corrupted, for example:
NodeJS output: 47 23 43 14 ed 86 14 dc 12 f3 b8 6c dc 31 fb fa
MySQL HEX() : 47 23 43 14 c3 ad c2 86 14 c3 9c 12 c3 b3 c2 b8
Code:
var binaryString = randomBinaryString();
db.sqlQuery("INSERT INTO test(binaryString) VALUES(" + db.escape(binaryString) + ")");
console.log(new Buffer(binaryString, 'binary').toString('hex'));
function randomBinaryString(){
var str = "";
for(var i = 0; i < 16; i++){
str += String.fromCharCode(Math.floor(Math.random()*256));
}
return str;
}
How should a string that actually encodes binary data (each character being a byte) be stored in MySQL using node-mysql2?
You're not inserting a raw binary string but a UTF-8 encoded string. In UTF-8, codepoints after 127 are encoded using multiple bytes. The fifth byte ed (237) is encoded using 2 bytes.
Buffer(String.fromCharCode(0xed)) produces : <Buffer c3 ad>
Sending the random string as a Buffer with binary encoding should fix the problem. node-mysql will convert the buffer to a hex string when inserting.
db.query('INSERT INTO test VALUES (?)', [ Buffer(randomString(), 'binary') ]);
Also, the crypto module has a randomBytes method that generates n random bytes.
db.query('INSERT INTO test VALUES (?)', [crypto.randomBytes(16)]);

Need help reverse engineering a CRC16

I'm trying to connect to the Safecom TA-810 (badge/registration system) to automate the process of calculating how long employee's have worked each day. Currently this is done by:
Pulling the data into the official application
Printing a list of all 'registrations'
Manually entering the values from the printed lists into our HR application
This is a job that can take multiple hours which we'd like to see automated. So far the official tech support has been disappointing and refused to share any details.
Using wireshark I have been capturing the UDP transmissions and have pretty much succeeded in understanding how the protocol is built up. I'm only having issues with what i suppose is a CRC field. I don't know how it is calculated (CRC type and parameters) and using which fields ...
This is how a message header looks like:
D0 07 71 BC BE 3B 00 00
D0 07 - Message type
71 BC - This i believe is the CRC
BE 3B - Some kind of session identifier. Stays the same for every message after the initial message (initial message has '00 00' as value)
00 00 - Message number. '01 00', '02 00', '03 00'
Some examples:
Header only examples
E8 03 17 FC 00 00 00 00 -> initial request (#0, no session nr)
D0 07 71 BC BE 3B 00 00 -> Initial response (#0, device sends a session nr)
4C 04 EF BF BE 3B 06 00 -> Message #6, still using the same session # as the initial response
Larger example, which has data
0B 00 07 E1 BE 3B 01 00 7E 45 78 74 65 6E 64 46 6D 74
I've also been trying to figure this out by reading the disassembled code from the original application. The screenshot below happens before the socket.sendto and seems to be related.
Any help will be extremely appreciated.
EDIT: Made some success with debugging the application using ollydbg. The CRC appears in register (reversed) EDX at the selected line in the following screenshot.
Take a look at CRC RevEng. If you can correctly identify the data that the CRC is operating on and the location of the CRC, you should be able to determine the CRC parameters. If it is a CRC.
I've managed to create a php script that does the CRC calculation by debugging the application using OllyDbg.
The CRC is calculated by Adding up every 2 bytes (every short). if the result is larger than a short, the 'most significant short' is added to the 'least significant short' until the result fits in a short. Finally, the CRC (short) is inverted.
I'll add my php script for completeness:
<?php
function CompareHash($telegram)
{
$telegram = str_replace(" ", "", $telegram);
$telegram_crc = substr($telegram, 4, 4);
$telegram = str_replace($telegram_crc, "0000", $telegram);
echo "Telegram: ", $telegram, ', Crc: ', $telegram_crc, ' (', hexdec($telegram_crc), ')<br />';
$crc = 0;
$i = 0;
while ($i < strlen($telegram))
{
$short = substr($telegram, $i, 4);
if (strlen($short) < 4) $short = $short . '00';
$crc += hexdec($short);
$i += 4;
}
echo "Crc: ", $crc, ', inverse: ', ~$crc;
// Region "truncate CRC to Int16"
while($crc > hexdec('FFFF'))
{
$short = $crc & hexdec ('FFFF');
// Region "unsigned shift right by 16 bits"
$crc = $crc >> 16;
$crc = $crc & hexdec ('FFFF');
// End region
$crc = $short + $crc;
}
// End region
// Region "invert Int16"
$crc = ~$crc;
$crc = $crc & hexdec ('FFFF');
// End region
echo ', shifted ', $crc;
if (hexdec($telegram_crc) == $crc)
{
echo "<br />MATCH!!! <br />";
}
else
{
echo "<br />failed .... <br />";
}
}
$s1_full = "E8 03 17 FC 00 00 00 00";
$s2_full = "D0 07 71 BC BE 3B 00 00";
$s3_full = "D0 07 4E D4 E1 23 00 00";
$s4_full = "D0 07 35 32 BE 3B 07 00 7E 44 65 76 69 63 65 4E 61 6D 65 3D 54 41 38 31 30 00";
$s5_full = "0B 00 39 6C BE 3B 05 00 7E 52 46 43 61 72 64 4F 6E";
CompareHash($s1_full);
CompareHash($s2_full);
CompareHash($s3_full);
CompareHash($s4_full);
CompareHash($s5_full);
?>
Thanks for the feedback!