I have a video being streamed over WebRTC. I wish to extract the raw video data -- as delivered by the server (byte for byte)
The reason for this is I want to read the PTS timestamps from the video in order to determine how long as live stream has been playing
I can utilize the MediaRecorder API and record the MediaStream I'm getting back from the RTCPeerConnection, however doing this re-encodes the video. When I do this I get back a WEBM file that, once parsed, has PTS values starting at 0
Is there a way to retrieve the raw data being delivered over the RTCPeerConnection without using a MediaRecorder?
I have been using twilio services to recording calls.
I want to perform range request on the recording url.
I am using the plain html audio tag.
It is working with the format .wav
https://api.twilio.com/2010-04-01/Accounts/ACXXXXXXXXXXXXXX/Recordings/REXXXXXXXXXXXXXXXXXX.wav. I am getting response like this with header accept-ranges: bytes
but when I tried with .mp3 it does not work https://api.twilio.com/2010-04-01/Accounts/ACXXXXXXXXXXXXXX/Recordings/REXXXXXXXXXXXXXXXXXX.mp3
Twilio developer evangelist here.
I'm afraid we don't support range requests on mp3 files as the files are converted on the fly.
I would like to know the container format of the following stream:
rtsp://8.15.251.47:1935/rtplive/FairfaxVideo3595
According to ffprobe, the container format is RTSP (format_long_name = RTSP input).
I also looked through the debug messages in VLC but I did not find any information on the stream's container format. What I DID find was that the codec was H264 and that VLC was using live555 to decode the stream. The media files live555 can support according to their website (http://www.live555.com/mediaServer/) makes me think that the above stream is an H264 elementary stream and is not in a container format. Am I correct?
Also, if the stream indeed does not have a container format, is it ok to say the container format is RTP (not RTSP as ffprobe says) because that's the protocol used to send the media data?
Thanks!
RTSP is more of a handshake done with the server, while RTP is the actual stream coming in once the handshake is done and you start streaming. RTSP URLs usually start with RTSP://... and the sequence of requests goes roughly something like
RTSP DESCRIBE, RTSP SETUP, RTSP PLAY, TEARDOWN
The response from the server to DESCRIBE will contain the information you need to know about the encoding of the file (H264, JPEG, etc.) while PLAY will cause the server to start sending the RTP stream. I suggest looking up RTSP SDP (session description protocol) for how to extract this information.
In case of streams, you are most likely correct, since the protocol used for streaming is usually RTP, and it tends to go hand in hand with RTSP (however I'm unsure whether or not we can apply the term container in the context of streaming)
I'm little confused about sending audio files from my client app (max. 10 sec of audio) into my server. The question is - which of the following options is the best?
From the client side save audio file into byte array and then convert it into Base64. After that send it in json request. On the server side, handle request in a Rest Api, decode Base64 and save it on the server. I was also wondering about making hash function of the audio file from the client side and send it also with the response and from the api compare these two hashes for integrity purposes (Missing packets or something).
Send an audio file as multipart form-data in json response and handle it in my Rest Api.
Simply saving file using FTP into my server.
Which option is the best? Or do you have any ideas?
When an image is uploaded from the client's machine to the client (browser), it requires FileReader() API in html, thereafter a base64 encoded url (say) of the image is sent in chunks to the server, where it needs to be re-assembled. All of this is taken care by the developer.
However, when an image is sent from the server to the client, only the directory path of the image inside the server machine suffices, no chunking and encoding is required.
My questions are:
1. Does the server send the image in chunks to the html file. If it does not, how does sending full images not bottle server's network? What would happen in case of large video files?
2. In what form of binary data is the image sent to the client - base64url / ArrayBuffer / binarystring / text / etc.
3. If the server does send the image in chunks, who is doing the chunking and the re-assembly on the client thereafter?
Thanks.
HTML isn't really important here. What you care about are the transport protocols used - HTTP and TCP, most likely.
HTTP isn't chunked by default, though there are advanced headers that do allow that - those are mostly used to support seeking in large files (e.g. PDF, video). Technically, this isn't really chunking - it's just the infrastructure for allowing for partial downloads (i.e. "Give me data from byte 1024 to byte 2048.").
TCP is a stream-based protocol. From programmer point of view, that's all there is to it - no chunking. Technically, though, it will process your input data and send it as distinct packets that are reassembled in-order on the other side. This is a matter of practicality - it allows you to manage data latency, streaming, packet retransmission etc. TCP handles the specifics during connection negotiation - flow control, window sizing, congestion control...
That's not the end of it, though. All the other layers add their own bits - their own ways to package the payload and split it as necessary, their own ways to handle routing and filtering, their own ways to handle congestion...
Finally, just like HTTP natively supports downloads, it supports uploading data as well. Just send an HTTP request (usually POST or PUT) and write data in a format the server understands - anything from text through base-64 to raw binary data. The limiting factor in your case isn't the server, the browser or HTTP - it's JavaScript. The basic mechanism is still the same - a request followed by a response.
Now, to address your questions:
Server doesn't send images to the HTML file. HTML only contains an URL of the image[1], and when the browser sees an URL in the img tag, it will initiate a new, separate request just for the image data. It isn't fundamentally different from downloading a file from a link. As for the transfer itself, it follows pretty much exactly the same way as the original HTML document - HTTP headers, then the payload.
Usually, raw binary data. HTTP is a text-based protocol, but it's payload can be arbitrary. There's little reason to use Base-64 to transfer image data (though in the past, there have been HTTP and FTP servers that didn't support binary at all, so you had to use something like Base-64).
The HTTP server doesn't care (with the exception of "partial downloads" mentioned above). The underlying network protocols handle this.
[1] Nowadays, there's methods to embed images directly in the HTML text, but it's of varying practicality depending on the image size, caching requirements etc.