can't re-stream using FFMPEG to MP4 HTML5 video - html

I have been struggling to get live streaming working from FFMPEG for many hours so raising the white flag and asking for help here.
My scenario is I have an IP security camera that I can successfully connect via RTSP (h.264) and save the video as file segments, and they play back fine either through a standalone app like VLC or via a node.js web server app that sends the 'video/mp4' & keep-alive header and streams the mp4 files previously saved by FFMPEG to a HTML5 video client.
However I want to take the same RTSP stream and re-stream it live to a HTML5 client. I know the HTML5 client bits and the FFMPEG remuxing to MP4 work as the MP4 recording/streaming works.
I have tried the following:
1) Set the output as a HTTP string. I don't think FFMPEG supports this as I get 'input/output error' and the FFMPEG documentation talks about another app called FFSERVER which isn't supported on Windows
ffmpeg -i rtsp://admin:12345#192.168.1.234:554 -vcodec copy -f mp4 -movflags frag_keyframe+empty_moov http://127.0.0.1:8888
2) As ffmpeg runs as a spawn in node.js, I have tried piping the STDOUT to a node http server, using the same header I use for the recording playback stream. I can view this stream in VLC which is a good sign but I can't get the HTML client to recognize the stream and it shows blank, or occasionally a static image of the stream.
var liveServer = http.createServer(liveStream);
var liveStream = function (req, resp) { // handle each client request by instantiating a new FFMPEG instance
resp.writeHead(200, {"Content-Type": "video/mp4", "Connection": "keep-alive"});
var xffmpeg = child_process.spawn("ffmpeg", [
"-i", "rtsp://admin:12345#192.168.1.234:554" , "-vcodec", "copy", "-f", "mp4", "-movflags", "frag_keyframe+empty_moov", "-" // output to stdout
], {detached: false});
xffmpeg.stdout.pipe(resp);
xffmpeg.on("exit", function (code) {
console.log("Xffmpeg terminated with code " + code);
});
xffmpeg.on("error", function (e) {
console.log("Xsystem error: " + e);
});
xffmpeg.stdout.on("data",function(data) {
console.log('Xdata rcv ' + data);
});
xffmpeg.stderr.on("data", function (data) {
console.log("XFFMPEG -> " + data);
}
}
I have tried both IE11 and Chrome HTML5 clients.
I suspect there is something not quite right with the format of the stream being sent which stops the HTML5 video client but not enough to stop VLC. The irritating thing is that the code above works just fine for playing back MP4 streams that have been recorded.
Any ideas how to get live re-streaming via FFMPEG working? Thanks.

Just curious have you solved it ? I'm doing basically the same thing except for it being screen cast.... I've switched to using webm format which plays perfectly...however the video will "lag" behind by an additional few seconds more... Which is bad for me, but might work for you

Related

Get the raw audio in getUserMedia success callback

I'm trying to get the raw audio in getUserMedia() success callback and post it to the server.
The success callback receives the LocalMediaStream object.
var onSuccess = function(s) {
var m=s.getAudioTracks(s);
//m[0] contains MediaStreamTrack object for audio
//get the raw audio and do the stuff
}
But there is no attribute or method to get the raw audio from channels in MediaStreamTrack.
How can we access the raw audio into this callback which is called on success of getUserMedia()?
I found the Recorder.js library-- https://github.com/mattdiamond/Recorderjs
But it is recording blank audio in Chrome: Version 26.0.1410.64 m.
It works fine on Chrome: Version 29.0.1507.2 canary SyzyASan.
I think there is issue of Web Audio API used by recorder.js
I'm looking for the solution without Web Audio API, that should work at least on chrome's official build.
Check out the MediaStreamAudioSourceNode. You can create one of those (via the AudioContext's createMediaStreamSource method) and then connect the output to RecorderJS or a plain old ScriptProcessorNode (which is what RecorderJS is built on).
Edit: Just realized you're asking if you can access the raw audio samples without the Web Audio API. As far as I know, I don't think that's possible.

org.osmf.media.MediaPlayer Throw Error: The specified capability is not currently supported

I have a back end sound.php which can return .m4a sound file from web server and I can make a web request with id to the sound.php to return specify .m4a file. i.e. sound.php?id=1234
I am now trying to use org.osmf.media.MediaPlayer and AudioElement and URLResource
var mediaPlayer:MediaPlayer = new MediaPlayer();
var ae:AudioElement = new AudioElement(new URLResource("http://xxx.com/sound.php?id=12"));
mediaPlayer.media = ae;
mediaPlayer.play();
and it throw error of The specified capability is not currently supported .I have tested the link via browser which is return a .m4a file sucessfully .
I dont understand if it is claiming the requesting method or the returned file , would somebody has any idea? Thanks
Try setting the MediaPlayer.autoPlay to true or you should wait until the media is loaded, which is signaled through the mediaPlayerStateChange event, with state READY.
[UPDATE]
As stated in the NetStream - Adobe ActionScript® 3 (AS3 ) API Reference page and also in the Supported codecs | Flash Player page:
Flash Player 9 Update 3 plays files derived from the standard MPEG-4
container format that contain H.264 video and/or HE-AAC audio, such as
F4V, MP4, M4A, MOV, MP4V, 3GP, and 3G2. One thing to note is that
protected MP4 files, such as those downloaded from iTunes or digitally
encrypted by FairPlay, are not supported.
It seems you will have to try the NetStream approach.

Using local file for Web Audio API in Javascript

I'm trying to get sound working on my iPhone game using the Web Audio API. The problem is that this app is entirely client side. I want to store my mp3s in a local folder (and without being user input driven) so I can't use XMLHttpRequest to read the data. I was looking into using FileSystem but Safari doesn't support it.
Is there any alternative?
Edit: Thanks for the below responses. Unfortunately the Audio API is horribly slow for games. I had this working and the latency just makes the user experience unacceptable. To clarify, what I need is sounething like -
var request = new XMLHttpRequest();
request.open('GET', 'file:///./../sounds/beep-1.mp3', true);
request.responseType = 'arraybuffer';
request.onload = function() {
context.decodeAudioData(request.response, function(buffer) {
dogBarkingBuffer = buffer;
}, onError);
}
request.send();
But this gives me the errors -
XMLHttpRequest cannot load file:///sounds/beep-1.mp3. Cross origin requests are only supported for HTTP.
Uncaught Error: NETWORK_ERR: XMLHttpRequest Exception 101
I understand the security risks with reading local files but surely within your own domain should be ok?
I had the same problem and I found this very simple solution.
audio_file.onchange = function(){
var files = this.files;
var file = URL.createObjectURL(files[0]);
audio_player.src = file;
audio_player.play();
};
<input id="audio_file" type="file" accept="audio/*" />
<audio id="audio_player" />
You can test here:
http://jsfiddle.net/Tv8Cm/
Ok, it's taken me two days of prototyping different solutions and I've finally figured out how I can do this without storing my resources on a server. There's a few blogs that detail this but I couldn't find the full solution in one place so I'm adding it here. This may be considered a bit hacky by seasoned programmers but it's the only way I can see this working, so if anyone has a more elegent solution I'd love to hear it.
The solution was to store my sound files as a Base64 encoded string. The sound files are relatively small (less than 30kb) so I'm hoping performance won't be too much of an issue. Note that I put 'xxx' in front of some of the hyperlinks as my n00b status means I can't post more than two links.
Step 1: create Base 64 sound font
First I need to convert my mp3 to a Base64 encoded string and store it as JSON. I found a website that does this conversion for me here - xxxhttp://www.mobilefish.com/services/base64/base64.php
You may need to remove return characters using a text editor but for anyone that needs an example I found some piano tones here - xxxhttps://raw.github.com/mudcube/MIDI.js/master/soundfont/acoustic_grand_piano-mp3.js
Note that in order to work with my example you're need to remove the header part data:audio/mpeg;base64,
Step 2: decode sound font to ArrayBuffer
You could implement this yourself but I found an API that does this perfectly (why re-invent the wheel, right?) - https://github.com/danguer/blog-examples/blob/master/js/base64-binary.js
Resource taken from - here
Step 3: Adding the rest of the code
Fairly straightforward
var cNote = acoustic_grand_piano.C2;
var byteArray = Base64Binary.decodeArrayBuffer(cNote);
var context = new webkitAudioContext();
context.decodeAudioData(byteArray, function(buffer) {
var source = context.createBufferSource(); // creates a sound source
source.buffer = buffer;
source.connect(context.destination); // connect the source to the context's destination (the speakers)
source.noteOn(0);
}, function(err) { console.log("err(decodeAudioData): "+err); });
And that's it! I have this working through my desktop version of Chrome and also running on mobile Safari (iOS 6 only of course as Web Audio is not supported in older versions). It takes a couple of seconds to load on mobile Safari (Vs less than 1 second on desktop Chrome) but this might be due to the fact that it spends time downloading the sound fonts. It might also be the fact that iOS prevents any sound playing until a user interaction event has occured. I need to do more work looking at how it performs.
Hope this saves someone else the grief I went through.
Because ios apps are sandboxed, the web view (basically safari wrapped in phonegap) allows you to store your mp3 file locally. I.e, there is no "cross domain" security issue.
This is as of ios6 as previous ios versions didn't support web audio api
Use HTML5 Audio tag for playing audio file in browser.
Ajax request works with http protocol so when you try to get audio file using file://, browser mark this request as cross domain request. Set following code in request header -
header('Access-Control-Allow-Origin: *');

How to receive a rtp, rtcp or udp, from a stream of gstreamer, on video HTML5?

I'm trying to get a video stream RTP/RTCP using HTML5, the stream was generated by gstreamer. I used examples of gstreamer, so I can pass through RTP ports:5000, and RTCP:5001, and can receive streams using gstreamer. But using HTML5 could not receive them. So I tried to read a bit about HTML5 and saw that it can receive theora/ogg, webm/vp8, mp4/avc, and protocols may be, HTTP, RTP, RTCP, UDP, and others, but I could not use RTP, RTCP or UDP, HTTP only managed to receive. But I had a very satisfactory result using the VLC plugin for Mozilla Firefox, using the UDP protocol. I wonder if anyone has any tips, I do not want to use source files as src="/tmp/test.avi" needs to be a video stream that can be udp, RTP, RTCP. Thank you!
If you don't need to stream at low fps, you can use GStreamer to transcode your stream in MJPEG and stream it in TCP, and then use VLC to get this TCP stream and stream it to HTTP. It works very well (0.5 sec of latency), but if you decrease the fps (1 fps) VLC introduces a latency of around 11 sec.
Here are some test commands that should work out of the box, using the GStreamer videotestsrc :
GStreamer :
gst-launch -v videotestsrc horizontal-speed=1 ! deinterlace ! videorate ! videoscale ! video/x-raw-yuv, framerate=15/1, width=256,
height=144 ! jpegenc quality=20 ! multipartmux
boundary="--videoboundary" ! tcpserversink host=localhost port=3000
VLC :
vlc -vvv -I rc tcp://localhost:3000 --sout
'#standard{access=http{mime=multipart/x-mixed-replace;boundary=--7b3cc56e5f51db803f790dad720ed50a},mux=mpjpeg,dst=localhost:8081}'
then open a browser to http://localhost:8081 (or create an HTML page with an img tag whose "src" attribute is http://localhost:8081)

Expression Encoder 4 live stream consumed by HTML 5 <video>

I'm trying to serve up a live stream (ie. completely buffered in memory, cannot access the past) and am having trouble with Expression Encoder 4.
Ideally, I'd like to just stream a bare H.264 byte stream to the client consumed by:
<video id="mainVideoWindow">
<source src='http://localhost/path/to/my/stream.mp4' type='video/mp4' />
</video>
I figured I could stream it to the client just like any other byte stream over HTTP. However, I'm having trouble figuring out the appropriate code required to do (first day with Expression Encoder, not sure how to go about getting the raw byte stream) so nor do I know if it would work in the first place.
An alternate was to use IIS Live Streaming server:
var source = job.AddDeviceSource(device, null);
job.ActivateSource(source);
job.ApplyPreset(LivePresets.VC1IISSmoothStreaming720pWidescreen);
var format = new PushBroadcastPublishFormat();
format.PublishingPoint = new Uri("http://localhost/test.isml");
job.PublishFormats.Add(format);
job.StartEncoding();
// Let's listen for a keypress or error message to know when to stop encoding
while (Console.ReadKey(true).Key != ConsoleKey.X) ;
// Stop our encoding
Console.WriteLine("Encoding stopped.");
job.StopEncoding();
However, I'm having trouble getting the client side markup to want to display the video on Chrome and I haven't seen anything to indicate that it'd work on Chrome (though http://learn.iis.net/page.aspx/854/apple-http-live-streaming-with-iis-media-services indicates how it would work with an iOS device).
Anyone have any insights?
You are trying to consume (with your sencond example) a Smooth Streaming feed (HTTP-Adaptive Streaming by Microsoft) through HTML5, which is not supported.
This could work on iOS devices if you enable the Apple HTTP Live Streaming to transmux the fragments into MPEG-2 Transport Stream. This will also generate an Apple HTTP Live Streaming manifest which than can be called though the video tag.
...I saw that you have the IIS link. The Apple HTTP Live Streaming needs to be enabled on the IIS Server (IIS Media Services). This will work for iOS devices. Quicktime will get into play...