I am beginner of video codec. not an video codec expert
I just want to know base on the same criteria, Comparing H254 encoding/decoding which is more efficiency.
Thanks
Decoding is more efficient. To be useful, decoding must run in real time, where encoding does not (except in videophone / conferencing applications).
How much more efficient? An encoder can generate motion vectors. The more compute power used on generating those motion vectors, the more accurate they are. And, the more accurate they are, the more bandwidth is available for the difference frames, so the quality goes up.
So, the kind of encoding used to generate video for streaming or distribution on DVD or BD discs can run many times slower than real time on server farms. But decoding for that kind of program is useless unless it runs in real time.
Even in the case of real-time encoding it takes more power (actual milliwatts, compute cycles, etc) than decoding.
It's true of H.264, H.265, VP8, VP9, and other video codecs.
Can anyone please tell me how to check how many hardware decoder instances (OMX.qcom.video.decoder.avc)can be created in my android phone (i.e. Google Pixel 2) for decoding H.264 video stream?
How to check this configuration?
There is a file on your phone: /etc/media_codecs.xml wich lists all available codecs and it's 'Quirks', 'Limits' and 'Features'. As of android 6 i think there is a 'Limit' called concurrent-instances. Every Codec should have that value.
E.g. <Limit name="concurrent-instances" max="16" />
This still doesn't guarantee that you can have 16 instances of a specific codec running at the same time as it is also dependent on other factors like bitrate, resolution in conjunction with HW resources. See it as more of an upper most limit of Codecs.
I've seen devices where you could only have a single FHD instance decoder operating at the same time while the concurrent-instances was set to 16. So it is still highly device depended.
Although I am quite familiar with H.264 encoding I came down to a point where I need advice from more experienced people. I'm performing hardware accelerated H.264 encoding using Intel Quick Sync and NVIDIA NVENC in a unified pipeline. The issue that troubles me is bitstream output buffer size. Intel Quick Sync provides a way to query the maximum bitstream size from the encoder, while NVDIA NVENC does not have such a feature (or at least I haven't found it, pointers are welcome). In their tutorial they state that:
NVIDIA recommends setting the VBV buffer size equal to single frame size. This is very
helpful in low latency applications, where network bandwidth is a concern. Single frame
VBV allows users to enable capped frame size encoding. In single frame VBV, VBV
buffer size must be set to maximum frame size which is equal to channel bitrate divided
by frame rate. With this setting, every frame can be sent to client immediately upon
encoding and the decoder can also decode without any buffering.
For example, if you have a channel bitrate of B bits/sec and you are encoding at N fps,
the following settings are recommended to enable single frame VBV buffer size.
uint32_t maxFrameSize = B/N;
NV_ENC_RC_PARAMS::vbvBufferSize= maxFrameSize;
NV_ENC_RC_PARAMS::vbvInitialDelay= maxFrameSize;
NV_ENC_RC_PARAMS::maxBitRate= NV_ENC_CONFIG::vbvBufferSize *N; // where N is the encoding frame rate.
NV_ENC_RC_PARAMS::averageBitRate= NV_ENC_RC_PARAMS::vbvBufferSize *N; // where N is the encoding frame rate.
NV_ENC_RC_PARAMS::rateControlMode= NV_ENC_PARAMS_RC_TWOPASS_CBR;
I am allocating a bitstream buffer pool for quite many encoding sessions so having an overhead of unused memory for each buffer by calculating the size from network bandwidth (in my case it is not the bottleneck) will cause ineffective memory usage.
So the general question is - is there any way how to determine the bitstream size for H.264 assuming there is no frame buffering and each frame should generate NAL units? Can I assume that it will never be larger than the input NV12 buffer (which seems unreliable since there may be many NAL units like SPS/PPS/AUD/SEI for the first frame and I am not sure if the size of those plus the same of IDR frame is not greater than the NV12 buffer size)? Does the standard have any pointers on this? Or is it totally encoder dependent?
Is it possible to do completely lossless encoding in h264? By lossless, I mean that if I feed it a series of frames and encode them, and then if I extract all the frames from the encoded video, I will get the exact same frames as in the input, pixel by pixel, frame by frame. Is that actually possible?
Take this example:
I generate a bunch of frames, then I encode the image sequence to an uncompressed AVI (with something like virtualdub), I then apply lossless h264 (the help files claim that setting --qp 0 makes lossless compression, but I am not sure if that means that there is no loss at any point of the process or that just the quantization is lossless). I can then extract the frames from the resulting h264 video with something like mplayer.
I tried with Handbrake first, but it turns out it doesn't support lossless encoding. I tried x264 but it crashes. It may be because my source AVI file is in RGB colorspace instead of YV12. I don't know how to feed a series of YV12 bitmaps and in what format to x264 anyway, so I cannot even try.
In summary what I want to know if that is there a way to go from
Series of lossless bitmaps (in any colorspace) -> some transformation -> h264 encode -> h264 decode -> some transformation -> the original series of lossless bitmaps
If there a way to achieve this?
EDIT: There is a VERY valid point about lossless H264 not making too much sense. I am well aware that there is no way I could tell (with just my eyes) the difference between and uncompressed clip and another compressed at a high rate in H264, but I don't think it is not without uses. For example, it may be useful for storing video for editing without taking huge amounts of space and not losing quality and spending too much encoding time every time the file is saved.
UPDATE 2: Now x264 doesn't crash. I can use as sources either avisynth or lossless yv12 lagarith (to avoid the colorspace compression warning). Howerver, even with --qp 0 and a rgb or yv12 source I still get some differences, minimal but present. This is troubling, because all the information I have found on lossless predictive coding (--qp 0) claims that the whole encoding should be lossless, but I am unable to verifiy this.
I am going to add a late answer to this one after spending all day trying to figure out how to get YUV 4:4:4 pixels into x264. While x264 does accept raw 4:2:0 pixels in a file, it is really quite difficult getting 4:4:4 pixels passed in. With recent versions of ffmpeg, the following works for completely lossless encoding and extraction to verify the encoding.
First, write your raw yuv 4:4:4 pixels to a file in a planar format. The planes are a set of Y bytes, then the U and V bytes where U and V use 128 as the zero value. Now, invoke ffmpeg and pass in the size of the raw YUV frames as use the "yuv444p" pixel format twice, like so:
ffmpeg -y -s 480x480 -pix_fmt yuv444p -i Tree480.yuv \
-c:v libx264 -pix_fmt yuv444p -profile:v high444 -crf 0 \
-preset:v slow \
Tree480_lossless.m4v
Once the encoding to h264 and wrapping as a Quicktime file is done, one can extract the exact same bytes like so:
ffmpeg -y -i Tree480_lossless.m4v -vcodec rawvideo -pix_fmt yuv444p \
Tree480_m4v_decoded.yuv
Finally, verify the two binary files with diff:
$ diff -s Tree480.yuv Tree480_m4v_decoded.yuv
Files Tree480.yuv and Tree480_m4v_decoded.yuv are identical
Just keep in mind that you need to write the YUV bytes to a file yourself, do not let ffmpeg do any conversion of the YUV values!
If x264 does lossless encoding but doesn't like your input format, then your best bet is to use ffmpeg to deal with the input file. Try starting with something like
ffmpeg -i input.avi -f yuv4mpegpipe -pix_fmt yuv420p -y /dev/stdout \
| x264 $OPTIONS -o output.264 /dev/stdin
and adding options from there. YUV4MPEG is a lossless uncompressed format suitable for piping between different video tools; ffmpeg knows how to write it and x264 knows how to read it.
FFmpeg has a "lossless" mode for x264, see FFmpeg and x264 Encoding Guide
ยง Lossless H.264
in essence it's -qp 0
I don't know your requirements for compression and decompression, but a general purpose archiver (like 7-zip with LZMA2) should be able to compress about as small or, in some cases, even significantly smaller than a lossless video codec. And it is much simpler and safer than a whole video processing chain. The downside is the much slower speed, and that you have to extract before seeing it. But for images, I think you should try it.
There is also lossless image formats, like .png.
For encoding lossless RGB with x264, you should use the command line version of x264 (you can't trust GUIs in this edge case, they will probably mess-up) r2020 or newer, with something like that:
x264 --qp 0 --preset fast --input-csp rgb --output-csp rgb --colormatrix GBR --output "the_lossless_output.mkv" "someinput.avs"
Any losses/differences between the input and output should be from some colour space conversion (either before encoding, or at playback), wrong settings or some header/meta-data that was lost. x264 don't supports RGBA, but RGB is ok. YUV 4:4:4 compression is more efficient, but you will lose some data in colour space conversion as your input is RGB. YV12/i420 is much smaller, and by far the most common colour space in video, but you have less chroma resolution.
More information on x264 settings:
http://mewiki.project357.com/wiki/X264_Settings
Also, avoid lagarith. It uses x87 floating point... and there are better alternatives.
http://codecs.multimedia.cx/?p=303
http://mod16.org/hurfdurf/?p=142
EDIT:
I don't know why I was donwvoted. Please leave a comment when you do that.
I agree that sometimes the loss in data is acceptable, but it's not simply a matter of how it looks immediately after compression.
Even a visually imperceptible loss of color data can degrade footage such that color correction, greenscreen keying, tracking, and other post tasks become more difficult or impossible, which add expense to a production.
It really depends when and how you compress in the pipeline, but ultimately it makes sense to archive the original quality, as storage is usually far less expensive than reshooting.
To generate lossless H.264 with HandBrake GUI, set Video Codec: H.264, Constant Quality, RF: 0, H.264 Profile: auto. Though this file is not supported natively by Apple, it can be re-encoded as near-lossless for playback.
HandBrake GUI's Activity Window:
H.264 Profile: auto; Encoding at constant RF 0.000000...profile High 4:4:4 Predictive, level 3.0, 4:2:0 8-bit
H.264 Profile: high; Encoding at constant RF 0.000000...lossless requires high444 profile, disabling...profile High, level 3.0
If you can't get lossless compression using a h.264 encoder and decoder,
perhaps you could look into two alternatives:
(1) Rather than passing all the data in h.264 format, some people are experimenting with transmitting some of the data with a residual "side channel":
(h.264 file) -> h264 decode -> some transformation -> a lossy approximation of the original series of bitmaps
(compressed residual file) --> decoder -> a series of lossless residual bitmaps
For each pixel in each bitmap, approximate_pixel + residual_pixel = a pixel bit-for-bit equal to the original pixel.
(2) Use Dirac video compression format in "lossless" mode.
Use FFmpeg with PowerShell. Type ffmpeg -h encoder=libx264rgb.
You can see Supported pixel formats: bgr0 bgr24 rgb24
When you encoding RGB to YUV or vice versa, you always loose quality.
But if you use -pix_fmt yuv444p -profile:v high444p your losses are least.
But if you use libx264rgb of ffmpeg encoder libx264rgb with format of pixel rgb24 you don't have any loss of quality.
A lot of application (for example Davinci Resolve) cannot read rgb 24 format of pixel.
I reccomend you to use:
ffmpeg -i ["your sequence of rgb image.png"] -c:v libx264rgb -video_size [your size] -framerate [your fps] -r [your fps] -qp 0 -pix_fmt rgb24 -profile:v high444 -preset veryslow -level 6.2 "your_video.mov"
Unfortunately, I don't know how to create sequence. But it is possible in FFmpeg.
I'm playing with Air and trying to get the PaperVision3D engine to display some 3D models.
But its really slow since Air seemingly does not support GPU acceleration. Any way to enable GPU rendering in air apps? browser apps can use "wmode=gpu" in the HTML settings...
I think you'll need to wait for molehill, http://labs.adobe.com/technologies/flash/molehill/
the current hardware acceleration is pretty limited.
http://help.adobe.com/en_US/ActionScript/3.0_ProgrammingAS3_Flex/WS2E9C7F3B-6A7C-4c5d-8ADD-5B23446FBEEB.html
there is some info at the bottom of the page there
Direct and GPU-compositing window
modes
Flash Player 10 introduces two window
modes, direct and GPU compositing,
which you can enable through the
publish settings in the Flash
authoring tool. These modes are not
supported in AIR. To take advantage of
these modes, you must enable hardware
acceleration for Flash Player.
Direct mode uses the fastest, most
direct path to push graphics to the
screen, which is advantageous for
video playback.
GPU Compositing uses the graphics
processing unit on the video card to
accelerate compositing. Video
compositing is the process of layering
multiple images to create a single
video image. When compositing is
accelerated with the GPU it can
improve the performance of YUV
conversion, color correction, rotation
or scaling, and blending. YUV
conversion refers to the color
conversion of composite analog
signals, which are used for
transmission, to the RGB (red, green,
blue) color model that video cameras
and displays use. Using the GPU to
accelerate compositing reduces the
memory and computational demands that
are otherwise placed on the CPU. It
also results in smoother playback for
standard-definition video.
Be cautious in implementing these
window modes. Using GPU compositing
can be expensive for memory and CPU
resources. If some operations (such as
blend modes, filtering, clipping or
masking) cannot be carried out in the
GPU, they are done by the software.
Adobe recommends limiting yourself to
one SWF file per HTML page when using
these modes and you should not enable
these modes for banners. The Flash
Test Movie facility does not use
hardware acceleration but you can use
it through the Publish Preview option.
Setting a frame rate in your SWF file
that is higher than 60, the maximum
screen refresh rate, is useless.
Setting the frame rate from 50 through
55 allows for dropped frames, which
can occur for various reasons from
time to time.
Using direct mode requires Microsoft
DirectX 9 with VRAM 128 MB on Windows
and OpenGL for Apple Macintosh, Mac OS
X v10.2 or higher. GPU compositing
requires Microsoft DirectX 9 and Pixel
Shader 2.0 support on Windows with 128
MB of VRAM. On Mac OS X and Linux, GPU
compositing requires OpenGL 1.5 and
several OpenGL extensions (framebuffer
object, multitexture, shader objects,
shading language, fragment shader).
You can activate direct and gpu
acceleration modes on a per-SWF basis
through the Flash Publish Settings
dialog box, using the Hardware
Acceleration menu on the Flash tab. If
you choose None, the window mode
reverts to default, transparent, or
opaque, as specified by the Window
Mode setting on the HTML tab.
Also Mike Chambers has some info somewhere about GPU acceleration (look for notes on pewpew), but I think this is all meant for mobile only.