I have a plaintext file that I wish to convert to something I can extract.
00000000 52 61 72 21 1a 07 01 00 f3 e1 82 eb 0b 01 05 07 |Rar!............|
00000010 00 06 01 01 80 80 80 00 3b fd 42 9f 51 02 03 31 |........;.B.Q..1|
00000020 a0 02 06 82 03 80 83 02 20 15 d4 6e 5b 46 b6 57 |........ ..n[F.W|
00000030 80 03 01 09 69 6e 73 74 72 2e 74 78 74 30 01 00 |....instr.txt0..|
00000040 03 0f 44 a5 ce af b3 09 b9 96 44 22 f4 99 ef 04 |..D.......D"....|
This is part of the file which made me believe it is a rar file. I tried using xxd with the -r option to no avail.
I tried the solution from here but it also didn't work.
Any ideas?
To solve my own question, and for future reference.
Use vim's visual block select to copy just the hex values into 'justhexvalues.txt'.
Then use xxd:
xxd -r -p justhexvalues.txt answer.rar
That was it.
I tried to process payment with MasterCard / MIR.
I do read data from VISA successfully after send PDOL, but MS does not requires PDOL.
1st step in transaction:
Select 2PAY.SYS
[SEND] : 00 A4 04 00 0E 32 50 41 59 2E 53 59 53 2E 44 44 46 30 31 00
[READ] : 6F 23 84 0E 32 50 41 59 2E 53 59 53 2E 44 44 46 30 31 A5 11
BF 0C 0E 61 0C 4F 07 A0 00 00 00 04 10 10 87 01 01 90 00
2nd step:
[SEND] : 00 A4 04 00 07 A0 00 00 00 04 10 10 00
F
[READ] : 6F 44 84 07 A0 00 00 00 04 10 10 A5 39 50 0A 4D 41 53 54 45 52 43 41 52 44 5F 2D 04 72 75 65 6E 87 01 01 9F 11 01 01 9F 12 0A 4D 41 53 54 45 52 43 41 52 44 BF 0C 0F 9F 4D 02 0B 0A 9F 6E 07 06 43 00 00 30 30 00 90 00
Card does not requres PDOL
3rd step:
[SEND] : 80 A8 00 00 02 83 00 00
[READ] : 77 16 82 02 19 80 94 10 08 01 01 00 10 01 01 01 18 01 02 00 20 01 02 00 90 00
Recieved Application File Locator (AFL)
Step 4:
Read all available data.
[SEND] : 00 B2 01 0C 00
and other sectors
I red all sectors. But there are not tags requires for payment: 9F26 - Application Cryptogram, 9F37 - Unpredictable Number, 9F36 - Transatcion Counter.
To get this tags I could make command Generate AC with CDOL, but how to generate CDOL?
Card says about CDOL1 and CDOL2. And CDOL1 requires tags that card generate itself.
Card answers, contains CDOL1 and CDOL2:
70 81 A0 57 13 55 45 46 77 77 25 42 79 D2 01 12 01 58 11 10 00 00 79 0F 5A 08 55 45 46 77 77 25 42 79 5F 24 03 20 11 30 5F 25 03 17 11 01 5F 28 02 06 43 5F 34 01 01 8C 21 9F 02 06 9F 03 06 9F 1A 02 95 05 5F 2A 02 9A 03 9C 01 9F 37 04 9F 35 01 9F 45 02 9F 4C 08 9F 34 03 8D 0C 91 0A 8A 02 95 05 9F 37 04 9F 4C 08 8E 0E 00 00 00 00 00 00 00 00 42 03 1E 03 1F 03 9F 07 02 3D 00 9F 08 02 00 02 9F 0D 05 B4 50 84 00 00 9F 0E 05 00 00 00 00 00 9F 0F 05 B4 70 84 80 00 9F 42 02 06 43 9F 4A 01 82 90 00
How generate offline limit transaction using paypass? How to do Generate AC?
Thank you!
CDOL1 and CDOL2 splits like below. Tag and its lengths. You are supposed to provide the data alone in the same order and size as you would for PDOL. In the below case, apart from two towards the bottom, rest all are available in the terminal ready to use.
CDOL1
9F02 06 //transaction amount
9F03 06 //other amount, cashback
9F1A 02 //termial country
95 05 //TVR terminal has arrived after terminal risk management
5F2A 02 //currency code
9A 03 //Transaction date
9C 01 //transaction type
9F37 04 //unpredictable number
9F35 01 //terminal type
9F45 02 //data Authentication code from Transaction Related Data ODA
9F4C 08 //icc dynamic number from Transaction Related Data ODA
9F34 03 //cvm results
CDOL2
91 0A //issuer authentication data
8A 02 //ARC
95 05 //TVR
9F37 04 //unpredictable number
9F4C 08 //icc dynamic number
9F26 and 9F36 will be returned by the card in response to your GEN AC.
9F37 is generated by the terminal.
My program was able to craft and send raw transactions to geth v1.3.3 before, but after I upgrade to geth v1.4.0, calling sendRawTransaction over RPC always returns invalid sender error.
Is transaction serialization (i.e. RLP) changed somehow from v1.3.3 to v1.4.0? Here is a dump of by raw transaction that triggers an invalid user error:
0x0000: F8 CA 80 85 0B A4 3B 74 00 83 01 5F 90 94 08 BE ......;t..._....
0x0010: 24 CD 8D CF 73 F8 FA 5D B4 2B 85 5B 43 70 BD 5C $...s..].+.[Cp.\
0x0020: 44 8B 80 B8 64 B0 70 B9 BA 00 00 00 00 00 00 00 D...d.p.........
0x0030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0040: 00 00 00 00 00 00 00 00 01 87 44 2E B8 96 6A 07 ..........D...j.
0x0050: 0C 31 C1 E8 AE A3 60 F5 35 32 47 81 13 34 31 D4 .1....`.52G..41.
0x0060: 4B FA 0A 0B 1B 9F 13 C6 F5 00 00 00 00 00 00 00 K...............
0x0070: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0080: 00 00 00 00 00 00 00 00 00 1B A0 DE A4 6B 8C E8 .............k..
0x0090: 72 5A 31 49 92 EC 6B 6F C6 89 8C BB D7 A4 B9 8A rZ1I..ko........
0x00A0: 10 D2 F7 9E CE 6B D5 0F C5 19 E9 A0 8F 74 57 C2 .....k.......tW.
0x00B0: 1C DA CB 7D 7A 2B 46 58 98 53 31 C3 4B CF 50 1F ...}z+FX.S1.K.P.
0x00C0: 17 CE 16 80 95 30 38 9B 98 3C 5B B8 .....08..<[.
A more machine readable version of my transaction is:
F8CA80850BA43B740083015F909408BE24CD8DCF73F8FA5DB42B855B4370BD5C448B80B864B070B9BA000000000000000000000000000000000000000000000000000000000000000187442EB8966A070C31C1E8AEA360F535324781133431D44BFA0A0B1B9F13C6F500000000000000000000000000000000000000000000000000000000000000001BA0DEA46B8CE8725A314992EC6B6FC6898CBBD7A4B98A10D2F79ECE6BD50FC519E9A08F7457C21CDACB7D7A2B4658985331C34BCF501F17CE16809530389B983C5BB8
Log from geth gives
I0504 20:22:27.392581 9768 types.go:106] Generated response: *shared.ErrorResponse &{%!s(float64=1) 2.0 %!s(*shared.ErrorObject=&{-32603 Invalid sender})}
I0504 20:22:27.392886 9768 http.go:157] Sending payload: {
"id": 1,
"jsonrpc": "2.0",
"error": {
"code": -32603,
"message": "Invalid sender"
}
}
I believe the JSON RPC stuff changed in geth v1.4.0. I can't tell why this is happening without seeing the full sendRawTransaction you are calling but check out the docs: https://github.com/ethereum/wiki/wiki/JSON-RPC#eth_sendtransaction
Also, this should be migrated to https://ethereum.stackexchange.com/
I got H264 RTP packet from RTSP stream. So I want to detect whether the frame is an I-frame or not.
Below is the first packet I got from the first time I open the stream. So I believe that it is an I-frame. Here are the first 160 bytes:
packet:
00 00 00 01 67 4D 00 1F : 95 A8 14 01 6E 40 00 00
00 01 68 EE 3C 80 00 00 : 00 01 06 E5 01 33 80 00
00 00 01 65 B8 00 00 08 : 52 90 9F F6 BE D6 C6 9C
3D F6 D4 2F 49 FB F7 13 : F2 A9 C7 27 2D A4 75 59
6C DB FF 35 27 A4 C7 B6 : E7 69 A2 E0 FB 0E FF 2D
0E E0 6F 25 43 78 BF B9 : 69 22 1B 24 E3 CA 60 56
44 16 6C 15 44 DA 55 29 : C2 39 24 86 CE D6 75 BB
E0 0C F4 F4 EC C5 76 E4 : 7B 59 B9 40 2D B3 ED 19
E4 1D 94 B7 54 9B B3 D0 : 8F 24 58 CD 3C F3 FA E0
D4 7D 88 70 0E 49 79 12 : B2 14 92 BA B6 9C 3A F7
8D 13 78 6B 4C CD C0 CC : C8 39 6A AC BE 3D AA 00
9A DB D2 68 70 5F C4 20 : B7 5C FC 45 93 DB 00 12
9F 87 5A 66 2C B2 B8 E7 : 63 C4 87 0B A4 AA 2E 6D
AB 42 3F 02 C2 A6 F9 41 : E5 FE 80 64 49 14 38 3D
52 4B F6 B2 E7 53 DD 3E : F6 BB A8 EB 13 23 BB 71
B1 C9 90 06 92 3E 5F 15 : F2 C0 39 43 EA 24 5A 86
AE 11 27 D4 C5 4B 5C CD : 6C 90 2B 44 80 18 76 95
6E 16 DF 5D 86 49 25 5A : B6 66 23 E6 40 D4 25 6B
CE A2 4C EE 13 DD 7B 88 : FF A0 64 EC 33 44 B1 DC
B7 0B 89 5B 8F 85 68 3C : 65 3E 55 0F 41 4B 32 C9
C8 56 78 1A 15 14 8C C7 : F5 17 40 D4 EC BC 5B 62
8A 24 66 6A C3 7E 3B DB : 44 A8 EC D8 EE 37 E0 DE
.. .. .. .. .. .. .. .. : .. .. .. .. .. .. .. ..
Then I used the below piece of code to determine the frame:
public static bool isH264iFrame(byte[] paket)
{
int RTPHeaderBytes = 0;
int fragment_type = paket[RTPHeaderBytes + 0] & 0x1F;
int nal_type = paket[RTPHeaderBytes + 1] & 0x1F;
int start_bit = paket[RTPHeaderBytes + 1] & 0x80;
if (((fragment_type == 28 || fragment_type == 29) && nal_type == 5 && start_bit == 128) || fragment_type == 5)
{
return true;
}
return false;
}
My problem is that I cannot know the exact value of RTPHeaderByte. In this case my packets always start with "00 00 00 01".
You will have to parse the payload. see the SO answer Possible Locations for Sequence/Picture Parameter Set(s) for H.264 Stream. For IDR, all VCL NALUs will be type 5. As for B/P you will need to parse out the exp-golmb encoded data to find the slice type.
Actually, this looks wrong:
int fragment_type = paket[RTPHeaderBytes + 0] & 0x1F;
int nal_type = paket[RTPHeaderBytes + 1] & 0x1F;
int start_bit = paket[RTPHeaderBytes + 1] & 0x80;
You have the NAL type first, then other things and bit 7 of the NAL type byte is always 0.
The fact is that you can simply search for two or three zeroes followed by a 1 and that's your marker for a NAL. The NAL follows just after that. It is not currently clear to me what's the difference between 2 and 3 zeroes.
So in your example, you have the following NALs:
00 00 00 01 67 4D 00 1F : 95 A8 14 01 6E 40 00 00
^^ ^^ ^^ ^^ ^^ ^^ ^^
00 01 68 EE 3C 80 00 00 : 00 01 06 E5 01 33 80 00
^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^
00 00 01 65 B8 00 00 08 : 52 90 9F F6 BE D6 C6 9C
^^ ^^ ^^ ^^
3D F6 D4 2F 49 FB F7 13 : F2 A9 C7 27 2D A4 75 59
.. .. .. .. .. .. .. .. : .. .. .. .. .. .. .. ..
So that means you have 0x67, 0x68, 0x06, 0x65, which as per the link given by szatmary, you have (i.e. we do type = (byte & 0x1F)):
7 Sequence parameter set non-VCL
8 Picture parameter set non-VCL
6 Supplemental enhancement information (SEI) non-VCL
5 Coded slice of an IDR picture VCL
The 5 means you have an I-Frame.
Looking at one of my files, the next group of NALs uses 0x41 or 0x01, which is a Coded slice of a non-IDR picture (i.e. B-Frame). Once in a while, I see a 5 instead of the 1 (i.e. an I-Frame). By default, the x264 generates a new I-Frame every 250 or so frames. You can change that parameter.
So your code detect whether this set of NALs represent an I-Frame or another frame needs to search for all the NALs within the frame and find the 1 (B-Frame) or 5 (I-Frame).
in.open("source-file.h264");
while(in)
{
char marker[4];
in.read(marker, 3);
for(;;)
{
in.read(marker + 3, 1);
if(marker[0] == 0
&& marker[1] == 0
&& marker[2] == 1)
{
// found one! (short one)
break;
}
if(marker[0] == 0
&& marker[1] == 0
&& marker[2] == 0
&& marker[3] == 1)
{
// found one! (long one)
break;
}
}
in.read(marker, 1);
type = marker[0] & 0x1F;
if(type == 1)
{
return B_FRAME;
}
if(type == 5)
{
return I_FRAME;
}
}
return NOT_FOUND;
WARNING: This code is going to be slow unless you have a good side buffer in your in file. This is C++ code. If you already have the data in a buffer, you should replace the in file with a pointer or an index within your buffer and that will definitely be very fast.
Note: The H.264 format makes sure to insert a 3 if it happens to have a 0x00 0x00 0x00 or a 0x00 0x00 0x01 sequence. That is, either one would look like this instead: 0x00 0x00 0x03 0x00 and 0x00 0x00 0x03 0x01. You can try to compress pure black frames, and you'll see many of those 0x03 appearing in the NAL picture data.
Posted previously the question with zero responses, so I am replacing it with further clarification and details from my research results based on this MySQL manual entry.
In the past, I've worked with alternate collations that allowed us to specify alternate default sorting. Mysql allows for this down to the column level, but I don't understand something to get it working.
Our customers use a standard set of one character codes in almost all references to any master table, and presenting these codes in the order they need is always very cumbersome and difficult using functions and routines in PHP.
SELECT * FROM myTable order by my_code
NORMAL, default sorting would return this: DESIRED, default sorting should return this:
my_code | Description my_code | Description
1 | Grade 1 P | Pre-Kindergarten
2 | Grade 2 K | Kindergarten
3 | Grade 3 1 | Grade 1
A | Adult 2 | Grade 2
K | Kindergarten 3 | Grade 3
P | Pre-Kindergarten A | Adult
The steps to accomplish this are described in the Docs at 10.4.3. , and examples are shown in the Docs at 10.1.78.
In the steps, it shows this table, and how the weights are specified. This, I think is where I get lost. I've altered the weights as shown below, putting "P" (x50P) and "K" (x4B) before "0" (x30), but all it accomplishes is changing the sorting so that "1" (x31) appears between "P" and "K", all other sorting appears to remain unchanged.
<collation name="latin1_test_ci">
<map>
00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F
10 11 12 13 14 15 16 17 18 19 1A 1B 1C 1D 1E 1F
20 21 22 23 24 25 26 27 28 29 2A 2B 2C 2D 2E 2F
50 4B 30 31 32 33 34 35 36 37 38 39 41 43 55 3A
3B 3C 3D 3E 3F 40 42 44 45 46 47 48 49 4A 4C 4D
4E 4F 51 52 53 54 56 57 58 59 5A 5B 5C 5D 5E 5F
60 41 42 43 44 45 46 47 48 49 4A 4B 4C 4D 4E 4F
50 51 52 53 54 55 56 57 58 59 5A 7B 7C 7D 7E 7F
80 81 82 83 84 85 86 87 88 89 8A 8B 8C 8D 8E 8F
90 91 92 93 94 95 96 97 98 99 9A 9B 9C 9D 9E 9F
A0 A1 A2 A3 A4 A5 A6 A7 A8 A9 AA AB AC AD AE AF
B0 B1 B2 B3 B4 B5 B6 B7 B8 B9 BA BB BC BD BE BF
41 41 41 41 5C 5B 5C 43 45 45 45 45 49 49 49 49
44 4E 4F 4F 4F 4F 5D D7 D8 55 55 55 59 59 DE DF
41 41 41 41 5C 5B 5C 43 45 45 45 45 49 49 49 49
44 4E 4F 4F 4F 4F 5D F7 D8 55 55 55 59 59 DE FF
</map>
</collation>
Sort results WITH the Alternate Collation above
Hex |my_code | Description
32 | 2 |Grade 2
33 | 3 |Grade 3
41 | A |Adult Ed
4B | K |Kindergarten
31 | 1 |Grade 1
50 | P |Pre-K
I realize you said you wanted to change the collation, but this requires no ORDER BY and is worth considering. You can convert these to an ENUM type and they will sort in the order they appear in the ENUM.
CREATE TABLE myTable (
my_code ENUM('P', 'K', '1', '2', '3', 'A'),
...
)
Using numbers in ENUMs is strongly discouraged, so you'll have to be careful. The main issue is that numbers can be treated as an index or a value in the ENUM. So it's behavior depends on it's type, leading to unexpected results.
This table are weights table. If you want that P i less than K, then put 00 weight to P and 01 weight to K. To put a weight you should assign a value in 'letter position': for P position 50. Sample to put P as first order letter:
<collation name="latin1_test_ci">
<map>
FF FF FF FF FF FF FF 07 08 09 0A 0B 0C 0D 0E 0F
10 11 12 13 14 15 16 17 18 19 1A 1B 1C 1D 1E 1F
20 21 22 23 24 25 26 27 28 29 2A 2B 2C 2D 2E 2F
30 31 32 33 34 35 36 37 38 39 3A 3B 3C 3D 3E 3F
40 41 42 43 44 45 46 47 48 49 4A 4B 4C 4D 4E 4F
50 51 52 53 54 55 56 57 58 59 5A 5B 5C 5D 5E 5F
60 41 42 43 44 45 46 47 48 49 4A 4B 4C 4D 4E 4F
00 51 52 53 54 55 56 57 58 59 5A 7B 7C 7D 7E 7F <-- first weight
80 81 82 83 84 85 86 87 88 89 8A 8B 8C 8D 8E 8F
90 91 92 93 94 95 96 97 98 99 9A 9B 9C 9D 9E 9F
A0 A1 A2 A3 A4 A5 A6 A7 A8 A9 AA AB AC AD AE AF
B0 B1 B2 B3 B4 B5 B6 B7 B8 B9 BA BB BC BD BE BF
41 41 41 41 5B 5D 5B 43 45 45 45 45 49 49 49 49
44 4E 4F 4F 4F 4F 5C D7 5C 55 55 55 59 59 DE DF
41 41 41 41 5B 5D 5B 43 45 45 45 45 49 49 49 49
44 4E 4F 4F 4F 4F 5C F7 5C 55 55 55 59 59 DE FF
</map>
</collation>
Edit: Adding test and compete table.
The complete table for should be:
<collation name="latin1_test_ci">
<map>
00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F
10 11 12 13 14 15 16 17 18 19 1A 1B 1C 1D 1E 1F
20 21 22 23 24 25 26 27 28 29 2A 2B 2C 2D 2E 2F
30 02 03 04 34 35 36 37 38 39 3A 3B 3C 3D 3E 3F
40 05 42 43 44 45 46 47 48 49 4A 01 4C 4D 4E 4F
00 51 52 53 54 55 56 57 58 59 5A 5B 5C 5D 5E 5F
60 41 42 43 44 45 46 47 48 49 4A 4B 4C 4D 4E 4F
50 51 52 53 54 55 56 57 58 59 5A 7B 7C 7D 7E 7F
80 81 82 83 84 85 86 87 88 89 8A 8B 8C 8D 8E 8F
90 91 92 93 94 95 96 97 98 99 9A 9B 9C 9D 9E 9F
A0 A1 A2 A3 A4 A5 A6 A7 A8 A9 AA AB AC AD AE AF
B0 B1 B2 B3 B4 B5 B6 B7 B8 B9 BA BB BC BD BE BF
41 41 41 41 5B 5D 5B 43 45 45 45 45 49 49 49 49
44 4E 4F 4F 4F 4F 5C D7 5C 55 55 55 59 59 DE DF
41 41 41 41 5B 5D 5B 43 45 45 45 45 49 49 49 49
44 4E 4F 4F 4F 4F 5C F7 5C 55 55 55 59 59 DE FF
</map>
</collation>
* Testing: *
mysql> create table b (a varchar(1) collate latin1_test_ci );
mysql> insert into b values
-> ( 'P' ),
-> ('K'),
-> ('A'),
-> ('1'),
-> ('2'),
-> ('3');
mysql> SHOW COLLATION LIKE 'latin1_test_ci';
+----------------+---------+----+---------+----------+---------+
| Collation | Charset | Id | Default | Compiled | Sortlen |
+----------------+---------+----+---------+----------+---------+
| latin1_test_ci | latin1 | 56 | | | 0 |
+----------------+---------+----+---------+----------+---------+
1 row in set (0.00 sec)
mysql> select * from b order by a;
+------+
| a |
+------+
| P |
| K |
| 1 |
| 2 |
| 3 |
| A |
+------+
6 rows in set (0.00 sec)
I don't think you need a custom collation to accomplish your goals.
To order the result set:
ORDER BY FIELD(my_code, 'P','K','1','2','3','4','5','6',
'7','8','9','10','11','12','A')
To limit the result set:
WHERE my_code IN('K','1','2','3','4','5')
If you'll be writing this sort of functionality into a lot of queries, a helper function might be a good idea:
DELIMITER $$
CREATE FUNCTION `f_position`(in_char CHAR(1)) RETURNS INTEGER
BEGIN
RETURN FIELD(in_char, 'P','K','1','2','3','4','5','6',
'7','8','9','10','11','12','A');
END$$
DELIMITER ;
Just make sure that all possible codes are referenced in the function, and are placed in the order that you want.
With a helper function like that, you can write queries like so:
WHERE f_position(grade) BETWEEN f_position('K') AND f_position('5')
ORDER BY f_position(grade)
The only downside to using a helper function to limit result sets like that (as opposed to the WHERE grade IN(...)) is that the function call would prevent any indexes on the column "grade" from being used.