h264: Add user unregistered SEI message - h.264

I'm trying to write a NAL unit with a SEI user data message. My code looks like:
typedef struct {
unsigned char type_code;
unsigned char countryCode;
unsigned char countryCodeExtension[2];
unsigned char user_identifier[4];
unsigned char payloadBytes[20];
unsigned char marker_bits;
} userdata_reg_t35;
unsigned char begin[5] = {0x00, 0x00, 0x00, 0x01, 0x06};
unsigned char end[3] = {0x00, sizeof(userdata_reg_t35), 0x80};
userdata_reg_t35 m_userSEIData;
m_userSEIData.countryCode = 0xB5;
m_userSEIData.countryCodeExtension[0] = 0x31;
m_userSEIData.countryCodeExtension[1] = 0x00;
m_userSEIData.user_identifier[0] = 0x34;
m_userSEIData.user_identifier[1] = 0x39;
m_userSEIData.user_identifier[2] = 0x41;
m_userSEIData.user_identifier[3] = 0x47;
m_userSEIData.type_code = 0x03;
m_userSEIData.marker_bits = 0xFF;
sprintf((char*)m_userSEIData.payloadBytes, "%s", "My Payload");
memcpy(target, begin, 5);
memcpy(target + 5, &m_userSEIData, sizeof(userdata_reg_t35));
memcpy(target + 5 + sizeof(userdata_reg_t35), end, 3);
When I playback the file in mplayer or vlc, I receive errors:
[h264 # 0x7f5860c20720] SEI type 3 truncated at 216
What am I doing wrong?
** EDIT **
I have modified the code after reading http://git.videolan.org/?p=x264.git;a=blob;f=encoder/set.c#l563
static const uint8_t uuid[16] = {0xdc, 0x45, 0xe9, 0xbd, 0xe6, 0xd9, 0x48, 0xb7,
0x96, 0x2c, 0xd8, 0x20, 0xd9, 0x23, 0xee, 0xef};
unsigned char begin[7] = {0x00, 0x00, 0x00, 0x01, 0x06, 0x05, 16 + 20};
char szPayload[20];
memset(szPayload, 0, 20);
sprintf(szPayload, "%s", "My payload");
memcpy(target, begin, 7);
memcpy(target + 7, uuid, 16);
memcpy(target + 7 + 16, szPayload, 20);
but I'm still getting the libav error: https://ffmpeg.org/doxygen/2.6/h264__sei_8c_source.html, line #306. What am I still doing wrong?

This does not look good and it does not map well on the H.264 Annex D spec.
You are adding SEI NAL type, then you are expected to add payloadType, payloadSize values - you don't have them. Then you stated you want unregistered SEI message (type 5) and your content more looks like payload type 4 (apparently you are adding captions). So you need to include that and exclude type_code, then it would look about right.
That is, your unneeded type_code 3 goes in place of expected value of 5/4 and then there is no length. VLC stumbles on exactly this...
See H.264 D.1 SEI payload syntax for details.
UPDATE. Your updated code is incorrect [also] for another reason. You are doing Annex B byte stream with start codes and in the same time you are including 20 zero bytes of payload, which under normal conditions should be updated with emulation prevention bytes.
To compare what you get with what x264 produces, simply open x264 output using binary editor and check the SEI NALs, then compare to yours.

Related

How to set precision for json_real while doing json_dump using jansson library

If a double value of 8.13 is passed into json_real and dump the json i see that its printing 8.1300000000000008, Is there any way to get 8.13 or 8.13000000000000000 exactly in C?
double test = 8.13;
json_t* msgtest = json_object();
json_object_set_new(msgtest, "test", json_real(test));
char* msgStr;
msgStr = json_dumps(msgtest, 0);
You can use JSON_REAL_PRECISION, introduced in jansson 2.7.
Output all real numbers with at most n digits of precision. The valid
range for n is between 0 and 31 (inclusive), and other values result
in an undefined behavior.
By default, the precision is 17, to correctly and losslessly encode
all IEEE 754 double precision floating point numbers.
Consider the following program:
#include <jansson.h>
int main() {
double test = 8.13;
json_t* msgtest = json_object();
json_object_set_new(msgtest, "test", json_real(test));
printf("%s\n", json_dumps(msgtest, JSON_REAL_PRECISION(3)));
}
It prints
{"test": 8.13}

How do I get nbit types from fread?

I have a file that is a concatenation of K, 17-bit, little endian, unsigned integers. In Matlab I am able to use fread(fd, K, 'bit17', 'ieee-le'). How do I read 17 bits off of a file descriptor in octave?
You can read the file byte by byte then use bitget to get the binary representation of data and then convert the binary representation to decimal numbers.
nbits = 17;
fd = fopen("myfile","rb");
bytes = fread(fd,Inf,"uint8=>uint8");
n = numel(bytes);
bits = false(8, n);
for k = 1:8
bits(k,:)=bitget(bytes,k);
end
count = floor(n * 8/nbits);
val = 2.^(0:nbits-1) * reshape(bits(1:count*nbits),nbits,[]);

Encode an image as a base64 string in MATLAB

Currently, I have an image stored as an MxNx3 uint8 array in MATLAB. However, I need to embed it in an HTML document, and I can't include the image separately.
Instead, I've decided to try and encode the image as a base64 string. However, I can't seem to find a way to encode the image as a string without having to first save the image to disk. I tried looking into writebmp and the like, but I can't seem to get it to work.
I'd really rather not write the image to a file, just to read it back using fread. The computer I'm using has very low Disk I/O, so that will take way too long.
Any help would be appreciated!
Edit:
I looked here, but that errors in R2018b due to "no method found". When I linearize the image, the returned string is incorrect
From an image matrix to HTML
1 Convert the image to the bytes of a BMP
function [header] = writeBMP(IM)
header = uint8([66;77;118;5;0;0;0;0;0;0;54;0;0;0;40;0;0;0;21;0;0;0;21;0;0;0;1;0;24;0;0;0;0;0;64;5;0;0;0;0;0;0;0;0;0;0;0;0;0;0;0;0;0;0]);
IMr = IM(:,:,1);
IMg = IM(:,:,2);
IMb = IM(:,:,3);clear IM;
IM(:,:,1)=IMb';
IM(:,:,2)=IMg';
IM(:,:,3)=IMr';
IM(:,:,:)=IM(:,end:-1:1,:);
[i,j,~]=size(IM);
header(19:22) = typecast(int32(i),'uint8'); %width
header(23:26) = typecast(int32(j),'uint8'); %height
IM = permute(IM,[3,1,2]);
IM = reshape(IM,[i*3,j]);
W = double(i)*3;
W = ceil(W/4)*4;
IM(3*i+1:W,:)=0; %padd zeros
IM = IM(:); %linear
header(35:38) = typecast(uint32(length(IM)),'uint8'); %datasize
header = [header;IM];
header(3:6) = typecast(uint32(length(header)),'uint8'); %filesize
end
You can also look into ...\toolbox\matlab\imagesci\private\writebmp.m for a more detailed example.
2 Encode the bytes to base64 characters
This is best done in a mex-file.
Save this code as encodeB64.c and run mex encodeB64.c
/*==========================================================
* encodeB64.c - converts a byte vector to base64
*
* The calling syntax is:
*
* [B] = encodeB64(B)
*
* input: - B : vector of uint8
*
* output: - B : vector of base64 char
*
* This is a MEX-file for MATLAB.
*
*========================================================*/
#include "mex.h"
/* The computational routine */
void Convert(unsigned char *in, unsigned char *out,unsigned long Nin, unsigned long Nout)
{
int temp;
static unsigned char alphabet[64] = {65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,48,49,50,51,52,53,54,55,56,57,43,47};
for (int i=0;i<(Nin-2);i+=3){
temp = in[i+2] | (int)in[i+1]<<8 | (int)in[i]<<16;
for (int j=0;j<4;j++){
out[3+(i/3)*4-j] = alphabet[(temp >> (j*6)) & 0x3f];
}
}
if (Nin%3==1){
temp = (int)in[Nin-1]<<16;
out[Nout-1] = 61;
out[Nout-2] = 61;
out[Nout-3] = alphabet[(temp >> 12) & 0x3f];
out[Nout-4] = alphabet[(temp >> 18) & 0x3f];
}
if (Nin%3==2){
temp = in[Nin-1]<<8 | (int)in[Nin-2]<<16;
out[Nout-1] = 61;
out[Nout-2] = alphabet[(temp >> 6) & 0x3f];
out[Nout-3] = alphabet[(temp >> 12) & 0x3f];
out[Nout-4] = alphabet[(temp >> 18) & 0x3f];
}
}
/* The gateway function */
void mexFunction( int nlhs, mxArray *plhs[],int nrhs, const mxArray *prhs[])
{
unsigned char *InputV; /* input vector 1*/
unsigned char *OutputV; /* output vector 1*/
unsigned long Nin;
unsigned long Nout;
/* check for proper number of arguments */
if(nrhs!=1) {
mexErrMsgIdAndTxt("MyToolbox:arrayProduct:nrhs","One inputs required.");
}
if(nlhs!=1) {
mexErrMsgIdAndTxt("MyToolbox:arrayProduct:nlhs","One output required.");
}
/* make sure the first input argument is scalar integer*/
if( !mxIsClass(prhs[0],"uint8") || mxGetNumberOfElements(prhs[0]) == 1 || mxGetN(prhs[0]) != 1) {
mexErrMsgIdAndTxt("MyToolbox:arrayProduct:notRowInteger","Input one must be uint8 column vector.");
}
/* get the value of the scalar input */
InputV = mxGetPr(prhs[0]);
Nin = mxGetM(prhs[0]); /*number of input bytes */
Nout = 4*((Nin+2)/3);
/* create the output matrix */
plhs[0] = mxCreateNumericMatrix((mwSize)Nout,1,mxUINT8_CLASS,mxREAL);
/* get a pointer to the real data in the output matrix */
OutputV = (unsigned char *) mxGetData(plhs[0]);
/* call the computational routine */
Convert(InputV,OutputV,Nin,Nout);
}
To test it you can run
T = randi(255,[2^28,1],'uint8'); %250MB random data
tic;Res=encodeB64(T);t=toc %convert
(length(T)/2^20) / t %read in MB/s
(length(Res)/2^20) / t %write in MB/s
My result:
read: 467 MB/s write: 623 MB/s
3 Put it all together and test
file = 'test.html';
fid = fopen(file,'wt');
fwrite(fid,sprintf('<html>\n<header> </header>\n<body>\n'));
fwrite(fid,sprintf('<p>%s</p>\n','Show the Matlab demo image street1.jpg'));
IM = imread('street1.jpg');figure(1);clf;image(IM);
B = writeBMP(IM);
str = encodeB64(B);
fwrite(fid,sprintf('<img src="data:image/bmp;base64,%s"/>\n',str));
fwrite(fid,sprintf('</body>\n</html>'));
fclose(fid);
this should generate a 1,229,008 byte HTML file with an image encoded.

Misaligned address in CUDA

Can anyone tell me whats wrong with the following code inside a CUDA kernel:
__constant__ unsigned char MT[256] = {
0xde, 0x6f, 0x6f, 0xb1, 0xde, 0x6f, 0x6f, 0xb1, 0x91, 0xc5, 0xc5, 0x54, 0x91, 0xc5, 0xc5, 0x54,....};
typedef unsinged int U32;
__global__ void Kernel (unsigned int *PT, unsigned int *CT, unsigned int *rk)
{
long int i;
__shared__ unsigned char sh_MT[256];
for (i = 0; i < 64; i += 4)
((U32*)sh_MT)[threadIdx.x + i] = ((U32*)MT)[threadIdx.x + i];
__shared__ unsigned int sh_rkey[4];
__shared__ unsigned int sh_state_pl[4];
__shared__ unsigned int sh_state_ct[4];
sh_state_pl[threadIdx.x] = PT[threadIdx.x];
sh_rkey[threadIdx.x] = rk[threadIdx.x];
__syncthreads();
sh_state_ct[threadIdx.x] = ((U32*)sh_MT)[sh_state_pl[threadIdx.x]]^\
((U32*)(sh_MT+3))[((sh_state_pl[(1 + threadIdx.x) % 4] >> 8) & 0xff)] ^ \
((U32*)(sh_MT+2))[((sh_state_pl[(2 + threadIdx.x) % 4] >> 16) & 0xff)] ^\
((U32*)(sh_MT+1))[((sh_state_pl[(3 + threadIdx.x) % 4] >> 24) & 0xff )];
CT[threadIdx.x] = sh_state_ct[threadIdx.x];
}
At This line of code ,
((U32*)(sh_MT+3))......
The CUDA debugger gives me the error message :
misaligned address
How can I fix this error?
I am using CUDA 7 in MVSC and i use 1 Block and 4 threads for executing the Kernel Function as follow:
__device__ unsigned int *state;
__device__ unsigned int *key;
__device__ unsigned int *ct;
.
.
main()
{
cudaMalloc((void**)&state, 16);
cudaMalloc((void**)&ct, 16);
cudaMalloc((void**)&key, 16);
//cudamemcpy(copy some values to => state , ct, key);
Kernel << <1, 4 >> >(state, ct, key);
}
Remember please, I can't change my "MT Table" type.
Thanks in advance for any advice or answer .
What the error message means is that the pointer is not aligned to the boundary required by the processor.
From the CUDA Programming Guide, section 5.3.2:
Global memory instructions support reading or writing words of size equal to 1, 2, 4, 8, or 16 bytes. Any access (via a variable or a pointer) to data residing in global memory compiles to a single global memory instruction if and only if the size of the data type is 1, 2, 4, 8, or 16 bytes and the data is naturally aligned (i.e., its address is a multiple of that size).
This is what the debugger is trying to tell you: Basically, you shouldn't dereference a pointer pointing to a 32-bit value from an address not aligned at a 32-bit boundary.
You can do (U32*)(sh_MT) and (U32*)(sh_MT+4) just fine, but not (U32*)(sh_MT+3) or such.
You probably have to read the bytes separately and join them together.

Why we need to write u8NAL first to .h264 file before writing stream data

I am reading data from server in h264 format, it write successfully in a .h264 file when we include UINT8 u8NAL[4] = {0x00, 0x00, 0x00, 0x01};.
but when we comment //UINT8 u8NAL[4] = {0x00, 0x00, 0x00, 0x01}; .h264 file store data but we can not see data on Vlc or Qt.
UINT8 u8NAL[4] = {0x00, 0x00, 0x00, 0x01};
char name1[100], name2[100];
_snprintf(name1, 99, "D:\\sizeH264_%p.txt", threadArgs->clientHandle);
_snprintf(name2, 99, "D:\\dataH264_%p.h264", threadArgs->clientHandle);
fp = fopen(name1, "w");
fpFrames = fopen(name2, "wb");
fwrite(u8NAL, 4, 1, fpFrames);
fwrite(threadArgs->tmpStreamParams->spsData, threadArgs->tmpStreamParams->spsDataSize, 1, fpFrames);
0 0 0 1 its a header part of a h264 frame, so in short player can know where I have to start and where to stop, that's way we have to give this hex data to every starting point of frame.
And ya its only require when you store data, otherwise you can give direct frame to player.