I have a fully working red5 (1.0 version) server that relay live streams from webcams.
However, when I send metadata to red5 , I can see (with a sniffing program) those metadata info going to red5 server, But other clients playing the webcam stream never get those metadata just like red5 doesn't relay them.
here is the sending brodcaster code:
if (event.info.code == "NetStream.Publish.Start")
sendMetadata();
and the sendMetadata function :
function sendMetadata()
{
var metaData:Object = new Object();
metaData.duration = 0;
metaData.filesize = 0;
metaData.framerate = fpsx;
metaData.width = large;
metaData.height = haute;
metaData.stereo = 0;
metaData.videocodecid = 7;
metaData.videodatarate = "800";
ns.send("#setDataFrame", "onMetaData", metaData);
}
now on playing clients reading the incoming stream , I just added this function to listen for metadatas when opening the stream :
function onMetaData(infoObject:Object):void
{
var key:String;
for (key in infoObject)
{
ExternalInterface.call("alerter", "" +key + ": " + infoObject[key]);
}
}
this function is never called , meaning incoming stream does not contain metadata.
please help me tuning red5 so all subscribing clients to the webcam stream can read the metadatas send once at start by the publisher
Thanks,
Related
Goal: I am trying to use VLC as a local server to expand the video capabilities of an app created with Adobe AIR, Flex and Actionscript. I am using VLC to stream to stdoutand reading that output from within my app.
VLC Streaming capabilities
VLC Flash Video
Stream VLC to Website with asf and Flash
Status: I am able to launch VLC as a background process and control it through its remote control interface (more detail). I can load, transcode and stream a local video file. The example app below is a barebones testbed demonstrating this.
Issue: I am getting data in to my app but it is not rendering as video. I don't know if it is a problem with my VLC commands or with writing to/reading from stdout. This technique of reading from stdout in AIR works (with ffmpeg for example).
One of the various transcoding commands I have tried:
-I rc // remote control interface
-vvv // verbose debuging
--sout // transcode, stream to stdout
"#transcode{vcodec=FLV1}:std{access=file,mux=ffmpeg{mux=flv},dst=-}"
This results in data coming into to my app but for some reason it is not rendering as video when using appendBytes with the NetStream instance.
If instead I write the data to an .flv file, a valid file is created – so the broken part seems to be writing it to stdout. One thing I have noticed: I am not getting metadata through the stdout`method. If I play the file created with the command below, I do see metadata.
// writing to a file
var output:File = File.desktopDirectory.resolvePath("stream.flv");
var outputPath:String = output.nativePath;
"#transcode{vcodec=FLV1}:std{access=file,mux=ffmpeg{mux=flv},dst=" + outputPath + "}");
Hoping someone sees where I am going wrong here.
Update 1: Just to add some more detail (!) – I took a look at the .flv file that is generated to examine the metadata. It appears at the head of the file as shown below. I have the correct onMetaData handler set up and see a trace of this data if I play the file from disk. I do not see this trace when reading from stdout and NetStream is in Data Generation mode. Is it possible that it isn't getting sent to stdout for some reason? I've tried generating my own header and appending that before the stream starts – I may not have the header format correct.
Update 2: So in my AIR app I was able to crudely parse the incoming stdout stream coming from VLC. I wanted to see if the FLV header data was being sent – and it appears that it is. I don't know if it is in the correct format, etc. but as I mention above, if I write to an .flv file instead of stdout, a valid .flv file is created.
Completely at a loss now – have tried everything I could think of and followed up every web link I could find on the issues involved. Alas – so close and it would have been so cool to leverage VLC from within AIR. 🙁
Update 3: Per VC ONE's suggestion, I have used his/her example code to check the incoming bytes for correct data. I get a massive string (1000's of chars) but these are the first ones:
What I get:
464C560105000000090000000012000111000000000000000200
46 4C 56 01 05 00 00 00 09 00 00 00 00 // check outs
What it should be:
46 4C 56 01 05 00 00 00 09 00 00 00 00
Note: In order to get this to work in AIR, you need to define the app profile as "extendedDesktop"
<?xml version="1.0" encoding="utf-8"?>
<s:WindowedApplication xmlns:fx="http://ns.adobe.com/mxml/2009"
xmlns:s="library://ns.adobe.com/flex/spark"
xmlns:mx="library://ns.adobe.com/flex/mx"
width="1024" height="768"
showStatusBar="false"
applicationComplete="onApplicationCompleteHandler(event)">
<fx:Script>
<![CDATA[
import mx.events.FlexEvent;
public var dataIn:Number = 0;
public var dataTotal:Number = 0;
private var processExe:File;
private var processArgs:Vector.<String>;
private var process:NativeProcess;
private var nc:NetConnection;
private var ns:NetStream;
private var vid:Video;
private var videoPath:String; // video to be streamed
protected function onApplicationCompleteHandler(event:FlexEvent):void {
var testFile:File = File.desktopDirectory.resolvePath("test.mp4");
if (testFile.exists){
videoPath = testFile.nativePath;
}
setUpNetStream();
createNativeProcess();
startNativeProcess();
}
protected function setUpNetStream():void {
nc = new NetConnection();
nc.addEventListener(AsyncErrorEvent.ASYNC_ERROR, errorHandler);
nc.addEventListener(NetStatusEvent.NET_STATUS, connStatusHandler);
nc.connect(null);
ns = new NetStream(nc);
ns.addEventListener(AsyncErrorEvent.ASYNC_ERROR, errorHandler);
ns.addEventListener(NetStatusEvent.NET_STATUS, streamStatusHandler);
var client:Object = new Object();
client.onMetaData = onMetaDataHandler;
ns.client = client;
vid = new Video(640,480);
vid.x= 100;
vid.y = 200;
this.stage.addChild(vid);
vid.attachNetStream(ns);
ns.play(null);
}
private function createNativeProcess():void {
if(NativeProcess.isSupported) {
// This is for OSX;
var pathToVLC:String = "utils/OSX/VLC.app/Contents/MacOS/VLC";
processExe = File.applicationDirectory.resolvePath(pathToVLC);
if (processExe.exists){
process = new NativeProcess();
process.addEventListener(ProgressEvent.STANDARD_OUTPUT_DATA, onOutputData);
process.addEventListener(ProgressEvent.STANDARD_ERROR_DATA, onErrorData);
process.addEventListener(ProgressEvent.PROGRESS, onOutputData);
process.addEventListener(ProgressEvent.SOCKET_DATA, onOutputData);
process.addEventListener(IOErrorEvent.STANDARD_OUTPUT_IO_ERROR, onIOError);
process.addEventListener(IOErrorEvent.STANDARD_ERROR_IO_ERROR, onIOError);
} else {
trace("process not found");
}
} else {
trace("Native Process not supported");
}
}
private function startNativeProcess():void {
processArgs = new Vector.<String>();
processArgs.push("-I rc");
processArgs.push("-vvv"); // verbose debug output
processArgs.push("--sout");
// -------TO WRITE TO A FILE ----------
// file to playback from
//var output:File = File.desktopDirectory.resolvePath("stream.flv");
//var outputPath:String = output.nativePath;
//processArgs.push("#transcode{vcodec=FLV1}:std{access=file,mux=ffmpeg{mux=flv},dst=" + outputPath + "}");
processArgs.push("#transcode{vcodec=FLV1,acodec=mp3}:gather:std{access=file,mux=flv,dst=-}");
processArgs.push("--sout-keep");
// ------VARIATIONS-------
//processArgs.push("#transcode{vcodec=FLV1,acodec=mp3}:std{access=file,mux=flv,dst=-}");
//processArgs.push("#transcode{vcodec=h264,vb=512,acodec=mp3,ab=128,samplerate=44100}:std{mux=ffmpeg{mux=flv},access=file,dst=-}");
var nativeProcessStartupInfo:NativeProcessStartupInfo = new NativeProcessStartupInfo();
nativeProcessStartupInfo.executable = processExe;
nativeProcessStartupInfo.arguments = processArgs;
process.start(nativeProcessStartupInfo);
// add video to playlist and play
process.standardInput.writeUTFBytes("add " + videoPath + " \n" );
process.standardInput.writeUTFBytes("play" + "\n" );
}
public function onOutputData(event:ProgressEvent):void {
if (process && process.running){
if (process.standardOutput.bytesAvailable){
var videoStream:ByteArray = new ByteArray();
process.standardOutput.readBytes(videoStream,0, process.standardOutput.bytesAvailable);
dataIn = videoStream.length;
dataTotal+= dataIn;
report.text = String("Current Bytes: " + dataIn + "\t Total Bytes: "+ dataTotal);
if (videoStream.length){
ns.appendBytes(videoStream);
}
//trace(ns.info);
}
}
}
private function errorHandler(e:AsyncErrorEvent):void {
trace('ERROR: ' + e.text);
}
private function connStatusHandler(e:NetStatusEvent):void {
trace('CONN_STATUS: ' + e.info.code);
switch(e.info.code){
case "NetConnection.Connect.Success":
//onFinishSetup();
break;
}
}
private function streamStatusHandler(e:NetStatusEvent):void {
trace('STREAM_STATUS: ' + e.info.code);
}
private function streamMetadataHandler(info:Object):void {
for (var key:String in info) {
trace("STREAM_METADATA: " + key + "=" + info[key]);
}
}
public function onErrorData(event:ProgressEvent):void {
if (process && process.running){
trace(process.standardError.readUTFBytes(process.standardError.bytesAvailable));
}
}
public function onIOError(event:IOErrorEvent):void {
trace(event.toString());
}
private function onMetaDataHandler(metadata:Object):void {
trace("### Begin Metadata listing : FLV Entries ### " );
for (var entry:* in metadata)
{
var value:Object = metadata[ entry ];
trace(" > " + entry + " : " + value);
}
trace("### End of Metadata listing for this FLV ### " );
}
]]>
</fx:Script>
<s:Label id="report" x="25" y="25" fontSize="18" />
</s:WindowedApplication>
In your other Question's comments you asked for my thoughts :
I noticed in your code you're running VLC process under OSX environment.
On Windows PC be aware that -I rc does not later respond to standardInput commands sent. I'm a Windows user so cannot help with that part.
Tried using --no-rc-fake-tty or even --rc-fake-tty, VLC still did not respond to stdout on PC.
You want to do playback & seeking within VLC but watch result in AS3 (like a projection screen), right? but I'm not even sure VLC will give you back FLV tags starting from your selected time stamps etc (by seeking you are accessing an FLV tag of a specific timestamp & the related a/v data)...
Other FFmpeg/Mencoder powered players like MPlayer I tested only send back "status" text data into stdout during playback (so cannot be fed to NetStream decoder for display).
I was able to crudely parse the incoming stdout stream coming from
VLC. I wanted to see if the FLV header data was being sent – and it
appears that it is. I don't know if it is in the correct format, etc.
Check the bytes: (a valid FLV header begins with 46 4C 56 01 05 00 00 00 09 00 00 00 00)
Just update your Question with a copy-paste of the "bytes check" result from the below function. Then easier to tell you if it's playable or maybe you need some alternative.
1) Setup some public (or private) vars...
Make a public var temp_String : String = "";
Make a public var videoStream:ByteArray = new ByteArray();
2) Replace your function onOutputData with below code...
public function onOutputData(event:ProgressEvent):void
{
if (process && process.running)
{
if (process.standardOutput.bytesAvailable)
{
//# make a private/public bytearray outside of this function
//var videoStream:ByteArray = new ByteArray();
process.standardOutput.readBytes(videoStream, videoStream.length, process.standardOutput.bytesAvailable);
dataIn = process.standardOutput.bytesAvailable;
dataTotal += dataIn;
//report.text = String("Current Bytes: " + dataIn + "\t Total Bytes: "+ dataTotal);
if (videoStream.length >= 1000 )
{
//ns.appendBytes(videoStream);
temp_String = bytes_toString(videoStream);
trace("bytes checking : " + "\n");
trace( temp_String ); //see hex of FLV bytes
//# temporary pausing of progress events
process.removeEventListener(ProgressEvent.STANDARD_OUTPUT_DATA, onOutputData);
}
//trace(ns.info);
}
}
}
Supporting function bytes_toString code :
public function bytes_toString ( ba:ByteArray ) : String
{
var str_Hex:String = ""; var len:uint = ba.length;
ba.position = 0;
for (var i:uint = 0; i < len; i++)
{
var n:String=ba.readUnsignedByte().toString(16);
if(n.length<2) //padding
{ n="0"+n; } str_Hex += n ;
}
return str_Hex.toUpperCase();
}
Some other notes :
Each firing of progress events only captures 32kb / 64kb packets of incoming stdout bytes at a time.
You make your videoStream:ByteArray = new ByteArray(); outside of the progressEvent so that each event firing does not make a new byteArray (which discards the old data that may be needed later for a full FLV tag).
Don't write each packet to 0 position since that will overwrite existing data. Add to the end-of existing by using videoStream.length as new writing position.
process.standardOutput.readBytes(videoStream, videoStream.length, process.standardOutput.bytesAvailable);
Also if (videoStream.length){ ns.appendBytes(videoStream); } is kinda dangerous. Any incomplete data (of header, frame or whatever) will jam the NetStream decoder if you append too soon. It will not restart unless you reset everything and begin again (re-append bytes of full FLV header, full frame tag, etc).
A couple of things I would try, some of which you might have thought of already:
Try without debugging turned on, in case that is fouling up your stdout stream.
Try a different video format (not FLV) in case it's a format-specific issue. For example, you might try mpeg4
Try piping your stdout stream to something else, like ffplay to see if the problem is the stream or your receiving app's assumptions about the stream.
I'm trying to play an infinite stream coming from the fetch API using Chrome 51. (a webcam audio stream as Microsoft PCM, 16 bit, mono 11025 Hz)
The code works almost OK with mp3s files, except some glitches, but it does not work at all with wav files for some reason i get "DOMException: Unable to decode audio data"
The code is adapted from this answer Choppy/inaudible playback with chunked audio through Web Audio API
Any idea if its possible to make it work with WAV streams ?
function play(url) {
var context = new (window.AudioContext || window.webkitAudioContext)();
var audioStack = [];
var nextTime = 0;
fetch(url).then(function(response) {
var reader = response.body.getReader();
function read() {
return reader.read().then(({ value, done })=> {
context.decodeAudioData(value.buffer, function(buffer) {
audioStack.push(buffer);
if (audioStack.length) {
scheduleBuffers();
}
}, function(err) {
console.log("err(decodeAudioData): "+err);
});
if (done) {
console.log('done');
return;
}
read()
});
}
read();
})
function scheduleBuffers() {
while ( audioStack.length) {
var buffer = audioStack.shift();
var source = context.createBufferSource();
source.buffer = buffer;
source.connect(context.destination);
if (nextTime == 0)
nextTime = context.currentTime + 0.01; /// add 50ms latency to work well across systems - tune this if you like
source.start(nextTime);
nextTime += source.buffer.duration; // Make the next buffer wait the length of the last buffer before being played
};
}
}
Just use play('/path/to/mp3') to test the code. (the server needs to have CORS enabled, or be on the same domain your run script from)
AudioContext.decodeAudioData just isn't designed to decode partial files; it's intended for "short" (but complete) files. Due to the chunking design of MP3, it sometimes works on MP3 streams, but wouldn't on WAV files. You'll need to implement your own decoder in this case.
Making the wav stream sound correctly implies to add WAV headers to the chunks as Raymond suggested, plus some webaudio magic and paquet ordering checks;
Some cool guys helped me to setup that module to handle just that and it works beautifully on Chrome : https://github.com/revolunet/webaudio-wav-stream-player
Now works on Firefox 57+ with some config flags on : https://developer.mozilla.org/en-US/docs/Web/API/ReadableStream/getReader#Browser_compatibility
We loaded video as bytes array, created InMemoryRandomAccessStream over this array and tried to MediaElement.SetSource. In UI we have message on MediaElement - Invalid Source. We tried to save this stream to file and read new stream from this file - works perfectly. Both stream are the identical (we check it using SequenceEqual).
What is the problem?
Part of our code:
var stream = await LoadStream();
mediaElement.SetSource(stream , #"video/mp4");
...
public async Task<IRandomAccessStream> LoadStream()
{
...
var writeStream = part.ParentFile.AccessStream.AsStreamForWrite();
foreach (var filePart in part.ParentFile.Parts)
{
writeStream.Write(filePart.Bytes, 0, filePart.Bytes.Length);
}
writeStream.Seek(0, SeekOrigin.Begin);
return part.ParentFile.AccessStream;
}
P.S - the mime-type is correct for sure
Thanks!
I brought this up in my last post but since it was off topic from the original question I'm posting it separately. I'm having trouble with getting my transmitted audio to play back through Web Audio the same way it would sound in a media player. I have tried 2 different transmission protocols, binaryjs and socketio, and neither make a difference when trying to play through Web Audio. To rule out the transportation of the audio data being the issue I created an example that sends the data back to the server after it's received from the client and dumps the return to stdout. Piping that into VLC results in a listening experience that you would expect to hear.
To hear the results when playing through vlc, which sounds the way it should, run the example at https://github.com/grkblood13/web-audio-stream/tree/master/vlc using the following command:
$ node webaudio_vlc_svr.js | vlc -
For whatever reason though when I try to play this same audio data through Web Audio it fails miserably. The results are random noises with large gaps of silence in between.
What is wrong with the following code that is making the playback sound so bad?
window.AudioContext = window.AudioContext || window.webkitAudioContext;
var context = new AudioContext();
var delayTime = 0;
var init = 0;
var audioStack = [];
client.on('stream', function(stream, meta){
stream.on('data', function(data) {
context.decodeAudioData(data, function(buffer) {
audioStack.push(buffer);
if (audioStack.length > 10 && init == 0) { init++; playBuffer(); }
}, function(err) {
console.log("err(decodeAudioData): "+err);
});
});
});
function playBuffer() {
var buffer = audioStack.shift();
setTimeout( function() {
var source = context.createBufferSource();
source.buffer = buffer;
source.connect(context.destination);
source.start(context.currentTime);
delayTime=source.buffer.duration*1000; // Make the next buffer wait the length of the last buffer before being played
playBuffer();
}, delayTime);
}
Full source: https://github.com/grkblood13/web-audio-stream/tree/master/binaryjs
You really can't just call source.start(audioContext.currentTime) like that.
setTimeout() has a long and imprecise latency - other main-thread stuff can be going on, so your setTimeout() calls can be delayed by milliseconds, even tens of milliseconds (by garbage collection, JS execution, layout...) Your code is trying to immediately play audio - which needs to be started within about 0.02ms accuracy to not glitch - on a timer that has tens of milliseconds of imprecision.
The whole point of the web audio system is that the audio scheduler works in a separate high-priority thread, and you can pre-schedule audio (starts, stops, and audioparam changes) at very high accuracy. You should rewrite your system to:
1) track when the first block was scheduled in audiocontext time - and DON'T schedule the first block immediately, give some latency so your network can hopefully keep up.
2) schedule each successive block received in the future based on its "next block" timing.
e.g. (note I haven't tested this code, this is off the top of my head):
window.AudioContext = window.AudioContext || window.webkitAudioContext;
var context = new AudioContext();
var delayTime = 0;
var init = 0;
var audioStack = [];
var nextTime = 0;
client.on('stream', function(stream, meta){
stream.on('data', function(data) {
context.decodeAudioData(data, function(buffer) {
audioStack.push(buffer);
if ((init!=0) || (audioStack.length > 10)) { // make sure we put at least 10 chunks in the buffer before starting
init++;
scheduleBuffers();
}
}, function(err) {
console.log("err(decodeAudioData): "+err);
});
});
});
function scheduleBuffers() {
while ( audioStack.length) {
var buffer = audioStack.shift();
var source = context.createBufferSource();
source.buffer = buffer;
source.connect(context.destination);
if (nextTime == 0)
nextTime = context.currentTime + 0.05; /// add 50ms latency to work well across systems - tune this if you like
source.start(nextTime);
nextTime+=source.buffer.duration; // Make the next buffer wait the length of the last buffer before being played
};
}
I want to cache the response of a urlRequest for offline usage in Adobe Air. When I compile for flash player the cache works and I get the response even when I disconnect the network, but when I compile for Adobe Air I get error. PS: useCache and cacheResponse dose not work!
stage.addEventListener(MouseEvent.CLICK , callReq)
var loader:URLLoader = new URLLoader()
function callReq(e:Event):void
{
//URLRequestDefaults.manageCookies = true;
//URLRequestDefaults.useCache = true;
var r:String = "http://onecom.no/presentation_json.php?what=get_slides&slide_id[]=2540"
var urlRequest:URLRequest = new URLRequest(r)
// urlRequest.cacheResponse = true
// urlRequest.useCache = true
urlRequest.url = r
loader.addEventListener(Event.COMPLETE , Comp)
loader.load(request)
}
function Comp(e:Event):void
{
trace( e.target.data)
}
Air can not use the browsers cache. I had to build my own cache class that saves all URLRequest responses to a Shared Object and loads them if there is no internet connection
save them to variables and store them in the local SQLlite database on your device/computer?...
Using local SQLlite on device