VLC syntax to transcode and stream to stdout? - actionscript-3

Question

Goal: I am trying to use VLC as a local server to expand the video capabilities of an app created with Adobe AIR, Flex and Actionscript. I am using VLC to stream to stdoutand reading that output from within my app.
VLC Streaming capabilities
VLC Flash Video
Stream VLC to Website with asf and Flash
Status: I am able to launch VLC as a background process and control it through its remote control interface (more detail). I can load, transcode and stream a local video file. The example app below is a barebones testbed demonstrating this.
Issue: I am getting data in to my app but it is not rendering as video. I don't know if it is a problem with my VLC commands or with writing to/reading from stdout. This technique of reading from stdout in AIR works (with ffmpeg for example).
One of the various transcoding commands I have tried:
-I rc // remote control interface
-vvv // verbose debuging
--sout // transcode, stream to stdout
"#transcode{vcodec=FLV1}:std{access=file,mux=ffmpeg{mux=flv},dst=-}"
This results in data coming into to my app but for some reason it is not rendering as video when using appendBytes with the NetStream instance.
If instead I write the data to an .flv file, a valid file is created – so the broken part seems to be writing it to stdout. One thing I have noticed: I am not getting metadata through the stdout`method. If I play the file created with the command below, I do see metadata.
// writing to a file
var output:File = File.desktopDirectory.resolvePath("stream.flv");
var outputPath:String = output.nativePath;
"#transcode{vcodec=FLV1}:std{access=file,mux=ffmpeg{mux=flv},dst=" + outputPath + "}");
Hoping someone sees where I am going wrong here.
Update 1: Just to add some more detail (!) – I took a look at the .flv file that is generated to examine the metadata. It appears at the head of the file as shown below. I have the correct onMetaData handler set up and see a trace of this data if I play the file from disk. I do not see this trace when reading from stdout and NetStream is in Data Generation mode. Is it possible that it isn't getting sent to stdout for some reason? I've tried generating my own header and appending that before the stream starts – I may not have the header format correct.
Update 2: So in my AIR app I was able to crudely parse the incoming stdout stream coming from VLC. I wanted to see if the FLV header data was being sent – and it appears that it is. I don't know if it is in the correct format, etc. but as I mention above, if I write to an .flv file instead of stdout, a valid .flv file is created.
Completely at a loss now – have tried everything I could think of and followed up every web link I could find on the issues involved. Alas – so close and it would have been so cool to leverage VLC from within AIR. 🙁
Update 3: Per VC ONE's suggestion, I have used his/her example code to check the incoming bytes for correct data. I get a massive string (1000's of chars) but these are the first ones:
What I get:
464C560105000000090000000012000111000000000000000200
46 4C 56 01 05 00 00 00 09 00 00 00 00 // check outs
What it should be:
46 4C 56 01 05 00 00 00 09 00 00 00 00
Note: In order to get this to work in AIR, you need to define the app profile as "extendedDesktop"
<?xml version="1.0" encoding="utf-8"?>
<s:WindowedApplication xmlns:fx="http://ns.adobe.com/mxml/2009"
xmlns:s="library://ns.adobe.com/flex/spark"
xmlns:mx="library://ns.adobe.com/flex/mx"
width="1024" height="768"
showStatusBar="false"
applicationComplete="onApplicationCompleteHandler(event)">
<fx:Script>
<![CDATA[
import mx.events.FlexEvent;
public var dataIn:Number = 0;
public var dataTotal:Number = 0;
private var processExe:File;
private var processArgs:Vector.<String>;
private var process:NativeProcess;
private var nc:NetConnection;
private var ns:NetStream;
private var vid:Video;
private var videoPath:String; // video to be streamed
protected function onApplicationCompleteHandler(event:FlexEvent):void {
var testFile:File = File.desktopDirectory.resolvePath("test.mp4");
if (testFile.exists){
videoPath = testFile.nativePath;
}
setUpNetStream();
createNativeProcess();
startNativeProcess();
}
protected function setUpNetStream():void {
nc = new NetConnection();
nc.addEventListener(AsyncErrorEvent.ASYNC_ERROR, errorHandler);
nc.addEventListener(NetStatusEvent.NET_STATUS, connStatusHandler);
nc.connect(null);
ns = new NetStream(nc);
ns.addEventListener(AsyncErrorEvent.ASYNC_ERROR, errorHandler);
ns.addEventListener(NetStatusEvent.NET_STATUS, streamStatusHandler);
var client:Object = new Object();
client.onMetaData = onMetaDataHandler;
ns.client = client;
vid = new Video(640,480);
vid.x= 100;
vid.y = 200;
this.stage.addChild(vid);
vid.attachNetStream(ns);
ns.play(null);
}
private function createNativeProcess():void {
if(NativeProcess.isSupported) {
// This is for OSX;
var pathToVLC:String = "utils/OSX/VLC.app/Contents/MacOS/VLC";
processExe = File.applicationDirectory.resolvePath(pathToVLC);
if (processExe.exists){
process = new NativeProcess();
process.addEventListener(ProgressEvent.STANDARD_OUTPUT_DATA, onOutputData);
process.addEventListener(ProgressEvent.STANDARD_ERROR_DATA, onErrorData);
process.addEventListener(ProgressEvent.PROGRESS, onOutputData);
process.addEventListener(ProgressEvent.SOCKET_DATA, onOutputData);
process.addEventListener(IOErrorEvent.STANDARD_OUTPUT_IO_ERROR, onIOError);
process.addEventListener(IOErrorEvent.STANDARD_ERROR_IO_ERROR, onIOError);
} else {
trace("process not found");
}
} else {
trace("Native Process not supported");
}
}
private function startNativeProcess():void {
processArgs = new Vector.<String>();
processArgs.push("-I rc");
processArgs.push("-vvv"); // verbose debug output
processArgs.push("--sout");
// -------TO WRITE TO A FILE ----------
// file to playback from
//var output:File = File.desktopDirectory.resolvePath("stream.flv");
//var outputPath:String = output.nativePath;
//processArgs.push("#transcode{vcodec=FLV1}:std{access=file,mux=ffmpeg{mux=flv},dst=" + outputPath + "}");
processArgs.push("#transcode{vcodec=FLV1,acodec=mp3}:gather:std{access=file,mux=flv,dst=-}");
processArgs.push("--sout-keep");
// ------VARIATIONS-------
//processArgs.push("#transcode{vcodec=FLV1,acodec=mp3}:std{access=file,mux=flv,dst=-}");
//processArgs.push("#transcode{vcodec=h264,vb=512,acodec=mp3,ab=128,samplerate=44100}:std{mux=ffmpeg{mux=flv},access=file,dst=-}");
var nativeProcessStartupInfo:NativeProcessStartupInfo = new NativeProcessStartupInfo();
nativeProcessStartupInfo.executable = processExe;
nativeProcessStartupInfo.arguments = processArgs;
process.start(nativeProcessStartupInfo);
// add video to playlist and play
process.standardInput.writeUTFBytes("add " + videoPath + " \n" );
process.standardInput.writeUTFBytes("play" + "\n" );
}
public function onOutputData(event:ProgressEvent):void {
if (process && process.running){
if (process.standardOutput.bytesAvailable){
var videoStream:ByteArray = new ByteArray();
process.standardOutput.readBytes(videoStream,0, process.standardOutput.bytesAvailable);
dataIn = videoStream.length;
dataTotal+= dataIn;
report.text = String("Current Bytes: " + dataIn + "\t Total Bytes: "+ dataTotal);
if (videoStream.length){
ns.appendBytes(videoStream);
}
//trace(ns.info);
}
}
}
private function errorHandler(e:AsyncErrorEvent):void {
trace('ERROR: ' + e.text);
}
private function connStatusHandler(e:NetStatusEvent):void {
trace('CONN_STATUS: ' + e.info.code);
switch(e.info.code){
case "NetConnection.Connect.Success":
//onFinishSetup();
break;
}
}
private function streamStatusHandler(e:NetStatusEvent):void {
trace('STREAM_STATUS: ' + e.info.code);
}
private function streamMetadataHandler(info:Object):void {
for (var key:String in info) {
trace("STREAM_METADATA: " + key + "=" + info[key]);
}
}
public function onErrorData(event:ProgressEvent):void {
if (process && process.running){
trace(process.standardError.readUTFBytes(process.standardError.bytesAvailable));
}
}
public function onIOError(event:IOErrorEvent):void {
trace(event.toString());
}
private function onMetaDataHandler(metadata:Object):void {
trace("### Begin Metadata listing : FLV Entries ### " );
for (var entry:* in metadata)
{
var value:Object = metadata[ entry ];
trace(" > " + entry + " : " + value);
}
trace("### End of Metadata listing for this FLV ### " );
}
]]>
</fx:Script>
<s:Label id="report" x="25" y="25" fontSize="18" />
</s:WindowedApplication>

Answer 1

In your other Question's comments you asked for my thoughts :
I noticed in your code you're running VLC process under OSX environment.
On Windows PC be aware that -I rc does not later respond to standardInput commands sent. I'm a Windows user so cannot help with that part.
Tried using --no-rc-fake-tty or even --rc-fake-tty, VLC still did not respond to stdout on PC.
You want to do playback & seeking within VLC but watch result in AS3 (like a projection screen), right? but I'm not even sure VLC will give you back FLV tags starting from your selected time stamps etc (by seeking you are accessing an FLV tag of a specific timestamp & the related a/v data)...
Other FFmpeg/Mencoder powered players like MPlayer I tested only send back "status" text data into stdout during playback (so cannot be fed to NetStream decoder for display).
I was able to crudely parse the incoming stdout stream coming from
VLC. I wanted to see if the FLV header data was being sent – and it
appears that it is. I don't know if it is in the correct format, etc.
Check the bytes: (a valid FLV header begins with 46 4C 56 01 05 00 00 00 09 00 00 00 00)
Just update your Question with a copy-paste of the "bytes check" result from the below function. Then easier to tell you if it's playable or maybe you need some alternative.
1) Setup some public (or private) vars...
Make a public var temp_String : String = "";
Make a public var videoStream:ByteArray = new ByteArray();
2) Replace your function onOutputData with below code...
public function onOutputData(event:ProgressEvent):void
{
if (process && process.running)
{
if (process.standardOutput.bytesAvailable)
{
//# make a private/public bytearray outside of this function
//var videoStream:ByteArray = new ByteArray();
process.standardOutput.readBytes(videoStream, videoStream.length, process.standardOutput.bytesAvailable);
dataIn = process.standardOutput.bytesAvailable;
dataTotal += dataIn;
//report.text = String("Current Bytes: " + dataIn + "\t Total Bytes: "+ dataTotal);
if (videoStream.length >= 1000 )
{
//ns.appendBytes(videoStream);
temp_String = bytes_toString(videoStream);
trace("bytes checking : " + "\n");
trace( temp_String ); //see hex of FLV bytes
//# temporary pausing of progress events
process.removeEventListener(ProgressEvent.STANDARD_OUTPUT_DATA, onOutputData);
}
//trace(ns.info);
}
}
}
Supporting function bytes_toString code :
public function bytes_toString ( ba:ByteArray ) : String
{
var str_Hex:String = ""; var len:uint = ba.length;
ba.position = 0;
for (var i:uint = 0; i < len; i++)
{
var n:String=ba.readUnsignedByte().toString(16);
if(n.length<2) //padding
{ n="0"+n; } str_Hex += n ;
}
return str_Hex.toUpperCase();
}
Some other notes :
Each firing of progress events only captures 32kb / 64kb packets of incoming stdout bytes at a time.
You make your videoStream:ByteArray = new ByteArray(); outside of the progressEvent so that each event firing does not make a new byteArray (which discards the old data that may be needed later for a full FLV tag).
Don't write each packet to 0 position since that will overwrite existing data. Add to the end-of existing by using videoStream.length as new writing position.
process.standardOutput.readBytes(videoStream, videoStream.length, process.standardOutput.bytesAvailable);
Also if (videoStream.length){ ns.appendBytes(videoStream); } is kinda dangerous. Any incomplete data (of header, frame or whatever) will jam the NetStream decoder if you append too soon. It will not restart unless you reset everything and begin again (re-append bytes of full FLV header, full frame tag, etc).

Answer 2

A couple of things I would try, some of which you might have thought of already:
Try without debugging turned on, in case that is fouling up your stdout stream.
Try a different video format (not FLV) in case it's a format-specific issue. For example, you might try mpeg4
Try piping your stdout stream to something else, like ffplay to see if the problem is the stream or your receiving app's assumptions about the stream.