I am developing h265 parser and give parsed data to LIVE555 stack, but not able to find NAL, SPS, PPS, VPS from data.
I have taken reference from the following link:
H264 Parser
So my question is: How to find VPS, SPS, PPS, NAL from h265 data?
Related
I am attempting to implement a streaming audio solution for the web. My requirements are these:
Relatively low latency (no more than 2 seconds).
Streaming in a compressed format (Ogg Vorbis/MP3) to save on bandwidth.
The stream is generated on the fly and is unique for each client.
To clarify the last point, my case does not fit the usual pattern of having a stream being generated somewhere and then broadcast to the clients using something like Shoutcast. The stream is dynamic and will adapt based on client input which I handle separately using regular http requests to the same server.
Initially I looked at streaming Vorbis/MP3 as http chunks for use with the html5 audio tag, but after some more research I found a lot of people who say that the audio tag has pretty high latency which disqualifies it for this project.
I also looked into Emscripten which would allow me to play audio using SDL2, but the prospect of decoding Vorbis and MP3 in the browser is not too appealing.
I am looking to implement the server in C++ (probably using the asynchronous facilities of boost.asio), and to have as small a codebase as possible for playback in the browser (the more the browser does implicitly the better). Can anyone recommend a solution?
P.S. I have no problem implementing streaming protocol support from scratch in C++ if there are no ready to use libraries that fit the bill.
You should look into Media Source Extension.
Introduction: http://en.wikipedia.org/wiki/Media_Source_Extensions
Specification: https://w3c.github.io/media-source/
I'd like to stream a user's webcam (from the browser) to a server and I need the server to be able to manipulate the stream (run some C algorithms on that video stream) and send the user back information.
I have heavily looked at WebRTC and MediaCapture and read the examples here : https://bitbucket.org/webrtc/codelab/overview .
However this is made for peer-to-peer video chat. From what I have understood, the MediaStream from getUserMedia is transmitted via a RTCPeerConnection (with addStream) ; what I'd like to know is : can I use this, but process the video stream on the server ?
Thanks in advance for your help
Here is the solution I have designed.
I post here for people seeking the same kind of information :-)
Front End side
I use the WebRTC API : get webcam stream with getUserMedia, open RTCPeerConnection (and RTCDataChannel for downside information).
The stream is DTLS encrypted (mandatory), multimedia streams use RTP and RTCP. The video is VP8 encoded and the audio in Opus encoded.
Back End side
On the backend, this is the complex part.
The best (yet) alternative I could find is the Janus Gateway. It takes cares of a lot of stuff, like DTLS handshake, RTP / RTCP demuxing, etc. Basically, it fires a event each time a RTP packet is transmitted. (RTP packets are typically the size of the MTU, so there is not a 1:1 mapping between video frames and RTP packets).
I then built a GStreamer (version 1.0) to depacketize the RTP packets, decode the VP8, ensure video scaling and colorspace / format conversion to issue a BGR matrix (compatible with OpenCV). There is an AppSrc component at the beginning of the pipeline and a AppSink at the end.
What's left to do
I have to take extra measures to ensure good scalability (threads, memory leaks, etc) and find a clean and efficient way of using the C++ library I have inside this program.
Hope this helps !
How do I achieve streaming of audio and video data and pass it on the network. I gone through a good article Here, But did not get in depth. I want to have chat application in HTML5
There are mainly below question
How to stream the audio and video data
How to pass to particular IP address.
Get that data and pass to video and audio control
If you want to serve a stream, you need a server doing so, by either downloading and installing, or coding on your own.
Streams only work in one direction, there is no responding or "retrieve back". Streaming is almost the same as downloading, with slight differences, depending on the service and use case.
Most streams are downstreams, but there are also upstreams. Did you hear about BufferStreams in PHP, Java, whatever? It's basically the same: data -> direction -> cursor.
Streams work over many protocols, even via different network layers, for example:
network/subnet broadcast, peer 2 peer, HTTP, DLNA, even FTP streams, ...
The basic nature of a stream is nothing more than data beeing sent to an audience.
You need to decide:
which protocol do you want to use for streaming
which server software
which media / source / live or with selectable start/end
which clients
The most popular HTTP streaming server is Shoutcast by Nullsoft (Winamp).
There is also DLNA which afaik is not HTTP based.
To provide more information, you need to be more specific regarding your basic requirements and decisions.
I am trying to learn enough about H.264, RTP, RTSP and encapsulation file formats to develop a video recording application.
Specifically, what should I read to understand the problem?
I want to be able to answer the following questions:
Can I save H.264 packets or NALs (Per RFC 6184) to a file?
Can I save the individual payloads as files?
Can I join the RTP payloads simply by concatenating them?
What transformation is needed to save
several seconds of H.264 video in an MP4 container.
What must be done
to later join these MP4 files, or arbitrarily split them, or serve
them as a new RTSP presentation?
I want to be able to answer these questions on a fairly low level so I can implement software that does some of the processes (capture RTP streams, rebroadcast joined MP4s).
Background
The goal is to record video from a network camera onto disk. The camera has an RTSP server that provides an H.264 encoded stream which it sends via RTP to a player. I have successfully played the stream using VLC, but would like to customize the process.
The "raw" video stream is a sequence of NAL units, per H.264 specification. Neither on RTSP, nor on MP4 file you have this stream "as is".
On RTSP connection you typically receive NAL units fragmented, and you need to depacketize them (no you cannot simply concatenate):
RTP H.264 Packet Depacketizer
How to process raw UDP packets so that they can be decoded by a decoder filter in a directshow source filter
MP4 file is a container formatted file, and has its own structure (boxes). So you cannot simply stream NALs into such file and you have to do what is called multiplexing.
How do I create an mp4 file from a collection of H.264 frames and audio frames?
just install rtmpdump along with rtmpsrv and rtmpsuck...
this will do all the work
in one terminal open rtmpsrv and in other open rtmpdump -r "RTMP URL"
this will save the stream in mystream.flv
As far as i know there is no audio streaming available in html5. Even with the audio tag.
That is, you always have to provide a file instead of passing an audio stream of some sort.
So, we know that most commonly used formats are ogg and mp3(not free). Also wav can be used but due to its size not commonly used.
My question is can I fake a file as if it was a stream, say create the wav file (with the riff header) and specify the PCM format details(freq,channel,blah blah) and pass that as the first few bytes and then send the PCM stream over the wire(actualy audio chunks).
The first issue I see if that RIFF header in the wav files require the chunk sizes which is the length of the file. WELL WE DONT HAVE A LENGTH SINCE IT IS AN AUDIO STREAM.
Any ideas.
Yes, absolutely.
The client doesn't need to know or care that the media it is playing is created live, or being loaded from disk.
You may have challenges depending on the codec used. WAV present the problems you have mentioned with the header, but it should be possible. With MP3, you can generally just send data at any point, and the decoder will sync to the frames on its own.