Why do I get invalid num_units_in_tick and time_scale while reading VUI? - h.264

I'm reading Video usability information and it is correct for all 29.97fps files, but for a 25fps file I'm getting wrong values.
For the num_units_in_tick I get 771 and for the time_scale I get 3892314307.
The standard makes no difference regarding the fps:
if( timing_info_present_flag )
{
num_units_in_tick u (32)
time_scale u (32)
fixed_frame_rate_flag u (1)
}
Why am I able to read correct values from one file, but not from the other?
Edit:
NALU:
00 00 00 01 67 64 00 28 AD 84 3F FF C2 1F FF E1
0F FF F0 87 FF F8 43 FF FC 21 FF FE 10 FF FF FF
FF FF FF FF FF 08 7F FF FF FF FF FF FF FF 2C C5
01 E0 11 3F 78 0A 10 10 10 1F 00 00 03 03 E8 00
00 C3 50 94

The problem is in the emulation byte. If we take a look at the last 10 bytes, we have:
00 00 03 03 E8 00 00 C3 50 94
The first 03 is an emulation prevention byte and it should be skipped. Then we have num_units_in_tick = 1000 and time_scale = 50000.

Related

Reverse CRC16 calculation

I'm trying to understand how is calculated the CRC at the end of a radio packet.
Here are a few examples:
11 00 01 0D 30 10 05 1F 11 ED 7E 01 00 01 B9 33
11 00 01 0D 30 10 05 1F 11 ED 7E 01 00 00 B9 32
11 00 01 1D 30 10 05 1F 11 ED 7E 01 00 00 EA CC
11 00 01 2D 30 10 05 1F 11 ED 7E 01 00 00 1E CE
The 4th byte is a sequence number. All other bytes are constant. The last 2 bytes definitely look like a CRC16, as these are the only ones changing when the sequence byte increases. The last 2 bytes are not related to the time, as I can reproduce that exact same sequence anytime.
Here are a few more examples, from the same device but with a different command:
16 00 01 60 20 10 05 1F 11 ED 7E 01 02 00 04 00 02 00 65 32 CC
16 00 01 CB 20 10 31 53 11 ED 7E 01 42 00 04 00 02 00 65 B4 B9
This time again, the last 2 bytes look like a CRC16.
I've tried many CRC calculations, using online calculators like crccalc.com.
I've also used the RevEng tool, but got no results.
I can't figure out the method of calculation, so I must be missing something.
Any help to determine the calculation would be welcome.
Thanks!
It is the CRC-16/XMODEM, computed on your examples with the first three bytes and the last two bytes before the CRC removed, and then, oddly, that CRC exclusive-or'ed with the two bytes that precede it (those that were excluded from the CRC calculation). The resulting 16-bit value is appended in big-endian order.

'Invalid sender' error after upgrading to geth 1.4.0

My program was able to craft and send raw transactions to geth v1.3.3 before, but after I upgrade to geth v1.4.0, calling sendRawTransaction over RPC always returns invalid sender error.
Is transaction serialization (i.e. RLP) changed somehow from v1.3.3 to v1.4.0? Here is a dump of by raw transaction that triggers an invalid user error:
0x0000: F8 CA 80 85 0B A4 3B 74 00 83 01 5F 90 94 08 BE ......;t..._....
0x0010: 24 CD 8D CF 73 F8 FA 5D B4 2B 85 5B 43 70 BD 5C $...s..].+.[Cp.\
0x0020: 44 8B 80 B8 64 B0 70 B9 BA 00 00 00 00 00 00 00 D...d.p.........
0x0030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0040: 00 00 00 00 00 00 00 00 01 87 44 2E B8 96 6A 07 ..........D...j.
0x0050: 0C 31 C1 E8 AE A3 60 F5 35 32 47 81 13 34 31 D4 .1....`.52G..41.
0x0060: 4B FA 0A 0B 1B 9F 13 C6 F5 00 00 00 00 00 00 00 K...............
0x0070: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0080: 00 00 00 00 00 00 00 00 00 1B A0 DE A4 6B 8C E8 .............k..
0x0090: 72 5A 31 49 92 EC 6B 6F C6 89 8C BB D7 A4 B9 8A rZ1I..ko........
0x00A0: 10 D2 F7 9E CE 6B D5 0F C5 19 E9 A0 8F 74 57 C2 .....k.......tW.
0x00B0: 1C DA CB 7D 7A 2B 46 58 98 53 31 C3 4B CF 50 1F ...}z+FX.S1.K.P.
0x00C0: 17 CE 16 80 95 30 38 9B 98 3C 5B B8 .....08..<[.
A more machine readable version of my transaction is:
F8CA80850BA43B740083015F909408BE24CD8DCF73F8FA5DB42B855B4370BD5C448B80B864B070B9BA000000000000000000000000000000000000000000000000000000000000000187442EB8966A070C31C1E8AEA360F535324781133431D44BFA0A0B1B9F13C6F500000000000000000000000000000000000000000000000000000000000000001BA0DEA46B8CE8725A314992EC6B6FC6898CBBD7A4B98A10D2F79ECE6BD50FC519E9A08F7457C21CDACB7D7A2B4658985331C34BCF501F17CE16809530389B983C5BB8
Log from geth gives
I0504 20:22:27.392581 9768 types.go:106] Generated response: *shared.ErrorResponse &{%!s(float64=1) 2.0 %!s(*shared.ErrorObject=&{-32603 Invalid sender})}
I0504 20:22:27.392886 9768 http.go:157] Sending payload: {
"id": 1,
"jsonrpc": "2.0",
"error": {
"code": -32603,
"message": "Invalid sender"
}
}
I believe the JSON RPC stuff changed in geth v1.4.0. I can't tell why this is happening without seeing the full sendRawTransaction you are calling but check out the docs: https://github.com/ethereum/wiki/wiki/JSON-RPC#eth_sendtransaction
Also, this should be migrated to https://ethereum.stackexchange.com/

pep8 binary to decimal

I got stuck on this assignment:
Write a program in object code that will convert a 3-bit binary number to a decimal number. For example, if input is 101, output should be 5; if input is 011, output should be 3.
This is my code so far:
49 00 50 - Input char 1
49 00 51 - Input char 2
49 00 53 - Input char 3
c0 00 00 - 0 memory
d1 00 50 - Load char 1
80 00 48 - subtract 48 cose 0 is 48 in dec
1c - shift to left
1c - shift to left again cose 3rd position in binary is x4
e1 00 60 - store int 1
d1 00 51 - load char 2
80 00 48 - subtract 48 cose 0 is 48 in dec
1c - shift to left cose 2nd position in binary is x2
e1 00 62 - store int 2
d1 00 53 - load char 3
80 00 48 - subtract 48 cose 0 is 48 in dec
71 00 60 - add int 1
71 00 62 - add int 2
f1 00 64 - store char
51 00 64 - output character
00
zz
Can someone help me and guide me how to do this, keep in mind that I just started with pep8 2 weeks ago.
Thank you
04 00 06 00 00 00 49 00 03 D1 00 03 80 00 30 1C
1C E1 00 04 49 00 03 D1 00 03 80 00 30 1C 71 00
04 E1 00 04 49 00 03 D1 00 03 80 00 30 71 00 04
E1 00 04 39 00 04 00 zz
This code is written for only 3 bits binary values

How to detect I/P/B frame from H264 RTP packet

I got H264 RTP packet from RTSP stream. So I want to detect whether the frame is an I-frame or not.
Below is the first packet I got from the first time I open the stream. So I believe that it is an I-frame. Here are the first 160 bytes:
packet:
00 00 00 01 67 4D 00 1F : 95 A8 14 01 6E 40 00 00
00 01 68 EE 3C 80 00 00 : 00 01 06 E5 01 33 80 00
00 00 01 65 B8 00 00 08 : 52 90 9F F6 BE D6 C6 9C
3D F6 D4 2F 49 FB F7 13 : F2 A9 C7 27 2D A4 75 59
6C DB FF 35 27 A4 C7 B6 : E7 69 A2 E0 FB 0E FF 2D
0E E0 6F 25 43 78 BF B9 : 69 22 1B 24 E3 CA 60 56
44 16 6C 15 44 DA 55 29 : C2 39 24 86 CE D6 75 BB
E0 0C F4 F4 EC C5 76 E4 : 7B 59 B9 40 2D B3 ED 19
E4 1D 94 B7 54 9B B3 D0 : 8F 24 58 CD 3C F3 FA E0
D4 7D 88 70 0E 49 79 12 : B2 14 92 BA B6 9C 3A F7
8D 13 78 6B 4C CD C0 CC : C8 39 6A AC BE 3D AA 00
9A DB D2 68 70 5F C4 20 : B7 5C FC 45 93 DB 00 12
9F 87 5A 66 2C B2 B8 E7 : 63 C4 87 0B A4 AA 2E 6D
AB 42 3F 02 C2 A6 F9 41 : E5 FE 80 64 49 14 38 3D
52 4B F6 B2 E7 53 DD 3E : F6 BB A8 EB 13 23 BB 71
B1 C9 90 06 92 3E 5F 15 : F2 C0 39 43 EA 24 5A 86
AE 11 27 D4 C5 4B 5C CD : 6C 90 2B 44 80 18 76 95
6E 16 DF 5D 86 49 25 5A : B6 66 23 E6 40 D4 25 6B
CE A2 4C EE 13 DD 7B 88 : FF A0 64 EC 33 44 B1 DC
B7 0B 89 5B 8F 85 68 3C : 65 3E 55 0F 41 4B 32 C9
C8 56 78 1A 15 14 8C C7 : F5 17 40 D4 EC BC 5B 62
8A 24 66 6A C3 7E 3B DB : 44 A8 EC D8 EE 37 E0 DE
.. .. .. .. .. .. .. .. : .. .. .. .. .. .. .. ..
Then I used the below piece of code to determine the frame:
public static bool isH264iFrame(byte[] paket)
{
int RTPHeaderBytes = 0;
int fragment_type = paket[RTPHeaderBytes + 0] & 0x1F;
int nal_type = paket[RTPHeaderBytes + 1] & 0x1F;
int start_bit = paket[RTPHeaderBytes + 1] & 0x80;
if (((fragment_type == 28 || fragment_type == 29) && nal_type == 5 && start_bit == 128) || fragment_type == 5)
{
return true;
}
return false;
}
My problem is that I cannot know the exact value of RTPHeaderByte. In this case my packets always start with "00 00 00 01".
You will have to parse the payload. see the SO answer Possible Locations for Sequence/Picture Parameter Set(s) for H.264 Stream. For IDR, all VCL NALUs will be type 5. As for B/P you will need to parse out the exp-golmb encoded data to find the slice type.
Actually, this looks wrong:
int fragment_type = paket[RTPHeaderBytes + 0] & 0x1F;
int nal_type = paket[RTPHeaderBytes + 1] & 0x1F;
int start_bit = paket[RTPHeaderBytes + 1] & 0x80;
You have the NAL type first, then other things and bit 7 of the NAL type byte is always 0.
The fact is that you can simply search for two or three zeroes followed by a 1 and that's your marker for a NAL. The NAL follows just after that. It is not currently clear to me what's the difference between 2 and 3 zeroes.
So in your example, you have the following NALs:
00 00 00 01 67 4D 00 1F : 95 A8 14 01 6E 40 00 00
^^ ^^ ^^ ^^ ^^ ^^ ^^
00 01 68 EE 3C 80 00 00 : 00 01 06 E5 01 33 80 00
^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^
00 00 01 65 B8 00 00 08 : 52 90 9F F6 BE D6 C6 9C
^^ ^^ ^^ ^^
3D F6 D4 2F 49 FB F7 13 : F2 A9 C7 27 2D A4 75 59
.. .. .. .. .. .. .. .. : .. .. .. .. .. .. .. ..
So that means you have 0x67, 0x68, 0x06, 0x65, which as per the link given by szatmary, you have (i.e. we do type = (byte & 0x1F)):
7 Sequence parameter set non-VCL
8 Picture parameter set non-VCL
6 Supplemental enhancement information (SEI) non-VCL
5 Coded slice of an IDR picture VCL
The 5 means you have an I-Frame.
Looking at one of my files, the next group of NALs uses 0x41 or 0x01, which is a Coded slice of a non-IDR picture (i.e. B-Frame). Once in a while, I see a 5 instead of the 1 (i.e. an I-Frame). By default, the x264 generates a new I-Frame every 250 or so frames. You can change that parameter.
So your code detect whether this set of NALs represent an I-Frame or another frame needs to search for all the NALs within the frame and find the 1 (B-Frame) or 5 (I-Frame).
in.open("source-file.h264");
while(in)
{
char marker[4];
in.read(marker, 3);
for(;;)
{
in.read(marker + 3, 1);
if(marker[0] == 0
&& marker[1] == 0
&& marker[2] == 1)
{
// found one! (short one)
break;
}
if(marker[0] == 0
&& marker[1] == 0
&& marker[2] == 0
&& marker[3] == 1)
{
// found one! (long one)
break;
}
}
in.read(marker, 1);
type = marker[0] & 0x1F;
if(type == 1)
{
return B_FRAME;
}
if(type == 5)
{
return I_FRAME;
}
}
return NOT_FOUND;
WARNING: This code is going to be slow unless you have a good side buffer in your in file. This is C++ code. If you already have the data in a buffer, you should replace the in file with a pointer or an index within your buffer and that will definitely be very fast.
Note: The H.264 format makes sure to insert a 3 if it happens to have a 0x00 0x00 0x00 or a 0x00 0x00 0x01 sequence. That is, either one would look like this instead: 0x00 0x00 0x03 0x00 and 0x00 0x00 0x03 0x01. You can try to compress pure black frames, and you'll see many of those 0x03 appearing in the NAL picture data.

open a jpeg image from binary format

I have a binary file that I want to open as (or to convert to) jpeg image.
The file is only supposed by me to be a jpeg within a binary format, therefore I include a part of it so that You can tell me if I'm wrong ('cause unfortunately I never saw one of those before):
0000000000 0E 03 13 01 00 10 00 00 EC B0 00 1E 00 01 00 00 [................]
0000000016 00 CA 00 00 00 5C 00 0F 00 01 00 00 01 26 00 00 [.....\.......&..]
0000000032 00 00 01 2F 00 01 00 00 01 26 00 00 83 B2 00 6A [.../.....&.....j]
0000000048 00 01 00 00 84 D8 00 00 00 04 01 2C 00 01 00 00 [...........,....]
0000000064 84 DC 00 00 00 14 01 32 00 01 00 00 84 F0 00 00 [.......2........]
0000000080 00 08 07 AD 00 02 00 00 84 F8 00 00 00 19 00 0F [................]
0000000096 00 02 00 00 85 11 00 00 00 00 01 2F 00 02 00 00 [.........../....]
0000000112 85 11 00 00 0D D3 01 2C 00 02 00 00 92 E4 00 00 [.......,........]
0000000128 00 14 01 32 00 02 00 00 92 F8 00 00 00 08 07 AD [...2............]
0000000144 00 03 00 00 93 00 00 00 00 19 00 0F 00 03 00 00 [................]
0000000160 93 19 00 00 00 00 01 2F 00 03 00 00 93 19 00 00 [......./........]
0000000176 59 7B 01 2C 00 03 00 00 EC 94 00 00 00 14 01 32 [Y{.,...........2]
0000000192 00 03 00 00 EC A8 00 00 00 08 00 00 00 04 00 00 [................]
0000000208 00 02 00 00 00 00 4E 43 53 41 20 48 44 46 20 56 [......NCSA HDF V]
0000000224 65 72 73 69 6F 6E 20 34 2E 32 20 52 65 6C 65 61 [ersion 4.2 Relea]
0000000240 73 65 20 30 2C 20 44 65 63 65 6D 62 65 72 20 32 [se 0, December 2]
0000000256 2C 20 32 30 30 33 00 00 00 00 00 00 00 00 00 00 [, 2003..........]
0000000272 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 [................]
0000000288 00 00 00 00 00 00 FF D8 FF E0 00 10 4A 46 49 46 [............JFIF]
0000000304 00 01 01 00 00 01 00 01 00 00 FF DB 00 43 00 10 [.............C..]
0000000320 0B 0C 0E 0C 0A 10 0E 0D 0E 12 11 10 13 18 28 1A [..............(.]
0000000336 18 16 16 18 31 23 25 1D 28 3A 33 3D 3C 39 33 38 [....1#%.(:3=<938]
0000000352 37 40 48 5C 4E 40 44 57 45 37 38 50 6D 51 57 5F [7#H\N#DWE78PmQW_]
0000000368 62 67 68 67 3E 4D 71 79 70 64 78 5C 65 67 63 FF [bghg>Mqypdx\egc.]
0000000384 DB 00 43 01 11 12 12 18 15 18 2F 1A 1A 2F 63 42 [..C......./../cB]
0000000400 38 42 63 63 63 63 63 63 63 63 63 63 63 63 63 63 [8Bcccccccccccccc]
0000000416 63 63 63 63 63 63 63 63 63 63 63 63 63 63 63 63 [cccccccccccccccc]
0000000432 63 63 63 63 63 63 63 63 63 63 63 63 63 63 63 63 [cccccccccccccccc]
0000000448 63 63 63 63 FF C0 00 11 08 01 A0 01 C0 03 01 22 [cccc..........."]
0000000464 00 02 11 01 03 11 01 FF C4 00 1F 00 00 01 05 01 [................]
0000000480 01 01 01 01 01 00 00 00 00 00 00 00 00 01 02 03 [................]
0000000496 04 05 06 07 08 09 0A 0B FF C4 00 B5 10 00 02 01 [................]
How can I convert it to jpeg to view its content (that is supposed to be an image)?
Is this conversion necessary or would You reccomend a software that can open it as I like?
I hope someone could help me, thanks in advance.
ANSWER
(As I can't answer my own question right now, I'm doing it here...)
At the end I came up with a solution thanks to the input information of unwind.
I downloaded HEX editor (http://www.hhdsoftware.com/free-hex-editor) to edit my binary file. Then I searched for the string where the 0xff 0xd8 was (in may case in line 0000000288). This is supposed to be the beginning of a JPEG file. Then I deleted everything that came before of that (also the six pair of zeros within the same line). Then I saved m edits and tried again to open it with an image processing program (in my case, I'm usin ENVI), and....IT WORKS! Now the binary file is red as an image file!
The problem now is that I have plenty of those files (302), and I need to edit all of them. Moreover, they each contain more than one jpeg, so I need to modify each one ore times. Guess I need to improve my programming knowledge...
Well, according to this page JPEG files begin with the byte pair 0xff 0xd8, so you could search forwards for that sequence, and throw away the data before it.
In your file, it happens on the line starting 0000000288.