I have a MF topology that captures video and audio and encodes it into H264, then writes it to an MPEG4 sink. However, the problem is that my H264 encoder (Intel QuickSync H264 Encoder) does not define a value for MF_MT_MPEG_SEQUENCE_HEADER in its output types. Thus, when I set the video media type of my MPEG4 sink, no sequence header is defined and the sink cannot correctly finalize, as mentioned in the MPEG4 Sink documentation:
https://msdn.microsoft.com/en-us/library/windows/desktop/dd757763(v=vs.85).aspx
After searching around, I learned that I need to get the SPS & PPS values for the MF_MT_MPEG_SEQUENCE_HEADER attribute. I am not sure about how to get these. My application is designed only for Windows 7, but in Windows 8 it seems like you can just set the MF_MPEG4SINK_SPSPPS_PASSTHROUGH attribute to have the sink grab the SPS & PPS from the input samples (see the above link). I have no interest in individual frame samples other than to obtain this value, and currently my application code is not looking at individual H264 samples.
What is an easy way to obtain the SPS & PPS values from a MF H264 stream on Windows 7?
I could explain exactly how to do it. But I believe the how will be confusing if you don't understand the why. I have another post that explains these concepts in pretty good detail, and writing the code to accomplish this should be trivial after understanding the bitstream format.
You should pay specific attention to the AVCC section
Possible Locations for Sequence/Picture Parameter Set(s) for H.264 Stream
Related
I have an RFC 6184 stream that transmits SPS/PPS in its first packets.
After around 5 seconds there is another SPS/PPS that changes the resolution of the stream.
My decoder handles this as expected nicely. Though my first question would be if it is legal according to the ISO standard.
My second concern is that when the stream is dumped to an ISO MP4, if it would cause any problem. As far as I know the AVC Configuration Record can handle multiple parameter sets.
I would conciser It undefined behavior. If it works for you, that’s great, but it may not work forever or in all environments. MP4 can put SPS/PPS in with other frame data. So it’s at least possible to package. Again different players may or may not work correctly.
I'm in search of h265/H264 video to json in effective way. I understood binary data can be converted to JSON but using base64. I'm interested to know is there any way we can define entire 90 minutes H265 video in JSON and then transfer it seamlessly over internet? Making chunks of data based on timestamps and then define encoding details for each video chunk as json object and define correlation between those json objects in JSON file. A player which can read each chunk separately when streaming, only upon streaming entire first chunk(piece) allows user to load immediate chunk in SSL communication between client(browser) and streaming server. I might sound very naive but are there any options to do so?
I am working on existing available opensource HEVC encoders, Right now i got reference HEVC encoder from Fraunhofergroup and also a optimized X265 encoder of multicoreware. However i also found OpenHEVC module my question is that in their readme file they only demonstrate decoding of a HEVC stream. Does OpenHEVC has encoder module which can encode a yuv input file in to h265 stream.
Thanks alot in advance
If you have mean this OpenHEVC than no it doesn't have its own HEVC encoder. And it was said about this in first paragraph of README.md:
openHEVC is a fork of Libav with only the files needed to decode HEVC
content, it was created for research purposes. Most people will not
need to use this and should use the libav HEVC decoder available at
https://github.com/OpenHEVC/libav instead (see
https://libav.org/documentation.html for documentation).
No, OpenHEVC does not have Video encoding.
You can use x265 / FFMPEG, if you want to encode videos in HEVC.
(x265 is GPL licensed)
I am decoding a streaming video. It is sent as RTP packages.
Before sending the video the receiver gets an SDP file that, among other things, has the sprop parameter sets.
However, the decoding works even if i remove that parameter. That's why I'm presuming the sprop parameter sets are also present in the H264 packages (in the RTP payload).
So, we can have sprop parameter sets on two places, which is considered the prioritized one?
There is no priority. The sprop contains an SPS/PPS. Each SPS/PPS has an id, when needed the NALS in the stream just indicate the specific SPS/PPS it needs. It is also legal for an h.264 encoder to just repeat the same SPS/PPS in the stream for protocols that do not have a method of transmitting out of band data, and require the ability to just a stream in progress (Like over the air TV)
I was wondering if there's a way to get streaming audio data (mp3, flac, arbitrary chunks of data in any encoding) from a server, process it and start playing after first chunk?
Background: All of our data is stored in compressed chunks, with a LAMP environment serving the data. de-compression and re-assembling is done client-side with
xhr-downloads, indexeddb, filehandle and/or chromefs. All currently
available audio/video functionality requires the audio to be
downloaded completely (otherwise decodeAudioData fails) or requires an
URL to the source without giving me a chance to process incoming data
client-side.
I am looking for a solution to squeeze my processing into the browser "inbuild" streaming/caching/decoding functionality (e.g. audio/video tag). I don't want to pre-process anything server-side, I don't want flash/java-applets and I'd like to avoid aligning data client-side (e.g. process mp3)
Question: Would it be possible to dynamically "grow" a storage that a bloburl points to? In other words: Create a filehandle / fileentry, generate a blobURL, feed it into an audio tag and grow the file with more data ?
Any other ideas?
Michaela
Added: Well, after another day of fruitless attempts, I must confirm that there are two problems in dealing with streamed/chunked mp3|ogg data:
1) decodeAudioData ist just too picky about what's fed into it. Even if I pre-align ogg-audio (splitting at "OggS" boundaries) I am unable to get the second chunk decoded.
2) Even IF I would be able to get the chunks decoded, how would I proceed playing them without setting timers, start positions or other head-banging detours? Maybe the webAudioAPI developers should take a look at aurora/mp3 ?
Added: Sorry to be bitching. But my newest experiments with recording audio from the microphone are not very promising either. 400K of WAV for a few seconds of recording? I have taken a few minutes to write about my experiences with webAudioAPI and added a few suggestions - from a coders perspective: http://blog.michaelamerz.com/wordpress/a-coders-perspective-on-the-webaudioapi/
Checkout https://github.com/brion/ogv.js. Brion's project chunk-loads an .ogv video and outputs the raw data back to screen through WebAudio API and Canvas, playing in the original FPS/timing of the file itself.
There is a StreamFile object in the codebase that handles the chunked load, buffering and readout of the data, as well as an example of how it is being assembled for playback through WebAudio.
I actually emailed Brion directly for a little help and he got back to me within an hour. It wan't built for exactly your use case, but the elements are there and I highly recommend Brion who is very knowledgeable on file formats, encoding and playback.
You cannot use <audio> tag for this. However, what you can use
Web Audio API - allows you dynamically construct audio stream in JavaScript
WebRTC - might need the streamed data pre-processing on the server-side, not sure
Buffers are recyclable, so you can discard already played audio.
How you load your data (XHR, WebSockets, chuncked file downloads) really doesn't matter as long as you can stick the raw data to a JavaScript buffer.
Please note that there is no universal audio format the browsers can decode and your mileage with MP3 may vary. AAC (mpeg-4 audio) is more supported and it has best web and mobile coverage. You can also decode AAC using pure JavaScript in Firefox: http://jster.net/library/aac-js - you can also decode MP3 in pure JavaScript: http://audiocogs.org/codecs/mp3/
Note that localStorage supports only 5 MB data per origin without additional dialog boxes, so this severely limits storing audio on the client-side.