I'm interested in learning how to decode and playback audio in ActionScript 3. I understand how to write bytes to a Sound object using the SAMPLE_DATA event, so that's not really a problem. What I want to understand is how I could implement alternate audio formats for native playback inside of Flash Player.
I guess what I'm asking is: how do I take something in X format and "convert/decode" it to WAV format and write the bytes to a Sound object, playing back the audio? I'm interested in writing a decoder for FLAC audio and possibly OGG audio, as these seem to be some of the most widely used open source audio formats.
Can anyone give me any advice on this?
If you want to write a decoder, the first thing you should probably look at is the spec for the format you want to decode.
The ogg/vorbis spec can be found here: http://xiph.org/vorbis/doc/Vorbis_I_spec.html.
Also, it could be of help to take a look (or maybe port) some other open source library that already does this (I'm not aware of any written in Actionscript), such as this, in Java: http://www.jcraft.com/jorbis/ (I don't know this library, I've just found it googling "ogg vorbis open source".
At any rate, you'll have to put some work to get it working and I don't mean this to discourage you, but I'm not sure Actionscript is fast enough for real time audio decoding.
You can try, but you're not going to have much grunt left to do other stuff. Prior to Flash 10, I wrote an article detailing a hack to feed PCM data into sound output in Flash. Someone got in touch because they had written an AS3 Ogg decoder, but... even after fully optimzing the code, it was found that AVM2 is really not that much up to the job. Basically, it's rather slow and decoding OGG is quite processor intensive. I can't see that things will have changed that much in the years since, because CPUs have become "wider" and not really that much faster. ActionScript is single threaded, so you can't offload to another core.
Probably worth checking out this... maybe performance has improved.
EDIT: Having said all that, as Juan has said, don't be discouraged by this answer. I suspect the computational demands of FLAC decoding are probably considerably less than OGG, and if DSP gets you excited, taking the time to figure all this stuff out is 100% worth it, even if the Flash route (possibly) leads to disappointment. Personally I think that the MediaStreamSource for Silverlight looks really promising,but haven't really dabbled that much.
Related
I have developed a website where the user can upload videos.
Everything worked fine until I discovered that if the video codec is not supported, it will not be displayed by the browser.
I realized that this same video that is not displayed by my site, is displayed perfectly by youtube.
is there a way to support all video formats in html 5?
If it does not exist, is there any way I can convert the video to another format with javascript or java on the backend?
Every help is welcome!!!
Current HTML5 implementations do not provide any way to get to your goal, even if they did, it would be very OS and browser specific. What i do to get this done is to install a local application that "live encodes" the stream locally and streams the output to the html5 video element.
In fact, there is no way at all on this world to support "all video formats" in one shot. The best you can currently do is to use ffmpeg on the backend to transcode.
Simply install ffmpeg on the backend and from your backend java, you just use the Runtime.getRuntime().exec method to call an ffmpeg commandline, like this
ffmpeg -i "%yourvideo%" "%youroutput".mp4"
It is a totally different topic to get a ffmpeg commandline done that is compatible to a lot of input formats, but using the above command you might hit already lots of formats.
Edit: please be aware of the consequences of "transcoding" on your server. It uses huge amounts of CPU and GPU usage is extremely complex. Due to this fact, you should not expect any really native way in java to do the same job as it would cost even a lot more CPU than the compiled C and assembler code of ffmpeg uses. Even if you find a native way how to use it natively in java, it will take weeks or months of R&D to get the same job done than a simple commandline exec could do for you.
Edit2: it might be a way for you to go through online encoding services like encoding.com and similar. But those do cost lots of money compared to running ffmpeg locally.
Is there any library in Action Script that be able to convert any video format to .FLV?
I've been looking for it with no success. I thought that as3 had functions for that purposes but not found.
I want to give the possibility that in my site, users can upload any video, so I need to convert it to a standard and compress it to a fixed resolution.
Thanks in advance.
There's a FLV encoder AS3 library. As far as I remember it requires image bytes and audio bytes for a frame. But you will have to get image data and sound data somewhere. You could either
play the video in flash and grab 'screenshots'
decode it and get actual data
In the first case you'll end with a mp4/flv -> flv converter which I guess is not what you need. In the second case you will have to decode videos somehow. So you either will have to implement decoding algorythms in AS3 or use Alchemy and existing C/C++ solutions.
Adobe is messing up with Alchemy at the moment, so I guess it is not an option anyway.
But everyone in their mind uses a server-side script to convert videos. There are tons of articles in the web. This will be the simplest, fastest and least painful solution.
I have a quick question relating to Actionscript 3 / AIR. I have looked through the site, but may be searching for the wrong thing or terms.
Is it possible to get an application wide audio stream from an AIR application?
I am trying to create a local audio file containing everything (all channels) played in an AIR app.
Does anybody have any ideas or suggestions?
Kind regards,
Phil
Update:
I have tried using the SoundMixer.computeSpectrum() function, but it returns a bytearray (size 2480) full of zeros...
It looks like it isn't reading the audio data correctly... I am playing a swf loaded using a SwfLoader and it is playing the sound out...
Odd... Anybody got an idea..?
There is SoundMixer.computeSpectrum, which will give you the global waveform or frequency data for the app. Unfortunately, it is fairly unreliable, and if you are trying to piece together the entire accurate audio stream, it is probably insufficient. But it might be enough to suit your needs if you are making, say, a visualizer or something else that only wants to know the current state of the audio stream.
Have you looked at SoundMixer Class? It has a computeSpectrum method that can get all playing sounds into a ByteArray, if I'm not totally misinterpreting the docs there.
I'm sick of waiting hours for Flash to publish. .NET / VisualStudio projects are WAAAAY faster - is that only compiling the classes that have changed?
Update: Does the Flash IDE re-encode all your sounds and images every time you publish? Can't it cache them somewhere?
In Flash CS4, disabling Warnings Mode speeds up code compilation by about 30% (still quite slow). Strict Mode also, but I'm not sure to what extent.
As for library assets, the optimal speed is achieved by setting JPEGs to "Use imported JPEG data" and PNGs/GIFs as "Loosless PNG/GIF".
As for sounds, I'm not 100% sure, but I think that ADPCM/mono-to-stereo/5kHz/2bit was the fastest.
The [Embed] tag might also help (it doesn't do any convertion) but it includes quite a bit of Flex code.
And as a tip, in Flash CS4 you can batch select assets in the library and apply settings, so you can easily set the optimal speed settings while on development and then change them for deploy.
Uncheck "Warnings Mode" in File > Publish Settings > Actionscript 3.0 Settings. This makes a huge difference. In a recent project I reduced my compile time from 1 min 20 sec down to 7 secs! Yeah, I was astonished too.
For me it turned out that I removed the character filter on my text fields. This seems innocuous, but in fact it has to embed every fracking glyph it can which resulted in a huge swf file that took forever to compile! Anyhow, definitely check that - I think a lot of people could easily overlook this.
You could try to split up your application to separate .swf files and then load them from the main movie.
Also it might be worth to look at your environment. Maybe your machine is running out of RAM, you are using an network drive or your hard disk is encrypted.
Also if the application is getting that big, is Flash really the right technology?
The compiler is just plain slow in the Flash IDE. I can understand the slowness if you have a lot of images in the library that have to be re-exported, but the compiler is inexplicably slow when only compiling code as well.
For example, I have a super basic Away3d scene that requires five full seconds to compile (no library assets). The exact same code compiles in Flex in less than a second. I don't know what's different between the two compilers, but doing any project with a heavy code base in the Flash IDE is just plain painful. As the project grows you'll have to wait longer and longer.
If you're doing a 3D project or something that requires a lot of code compilation, I recommend doing an Actionscript-only project in Flex/Flash Builder. Maybe CS5 will have a better compiler...
You might want to check your character embedding -- If you accidentally click Embed all characters it could embed 1000s of characters, it would significantly slow down your publishing speed
It all depends on your project. I know of some animators that use Flash for publishing broadcast content and that usually takes a long time to publish. There's no getting around it for them. Otherwise, if it takes hours to publish an SWF that will be viewed on the web, then you're probably not going about things the right way.
Additionally, a few obvious culprits that immediately increase the time that it takes to compile a SWF are embedded video and embedded sound. The more items the IDE must compress for output (this includes images too), the longer it will take to compile.
You can't make the Flash IDE cache embedded media (but images won't be reencoded if you import PNGs/GIFs and set the compression to lossless in the library). But if it's a web project you should be loading media from external files anyway, so you don't have to preload all data before anything shows up on the user's screen. You can either load (or even stream) the media files directly or embed them in a separate SWF and load from there, but I don't recommend the latter as it adds unnecessary complexity.
When compiling is too slow, I usually strip down the library - read load images (png, jpg) and sound via URLRequest.
Usually the thing that really slows down the compiler is importing illustrator (ai) files.
disable warnings in the Publish Settings > Flash > as3
If you're even willing to change languages you could use Haxe. It compiles to SWF and is incredibly fast at compiling. It's probably not worth it to change to a different language at the point where you have hour long compile times, but look into it when you are starting your next large project.
I'm wondering about MP3 decoding/encoding, and I was hoping to pull this off in Flash using AS3
I'm sure it'll be a right pain...
I have no idea where to start, can anyone offer any pointers? reference material?
----much later---
Thank you all very much for your input... It seems I have a long road ahead of me yet!
You could also theoretically do this as a PixelBender filter, and should get significantly better performance than using a pure ActionScript 3 implementation.
More info on PixelBender here:
http://labs.adobe.com/wiki/index.php/Pixel_Bender_Toolkit
mike chambers
mesh#adobe.com
this would help
http://labs.adobe.com/technologies/alchemy/
See LAME MP3 Encoder. You can checkout their source code and their link page. Mpeg.org should have documents too.
I've got a project converting WAV files (actually Asterisk voice mails) into MP3's. If I remember correctly there are some oddities about Lame's license, so I've downloaded and compiled first LAME, then SOX by hand.
I have a web process written in PHP to actually convert the files from WAV to MP3 on the web server's local file system (actually PHP is just supervising the command-line sox tool via exec()). Then I attach all the metadata the MP3 needs using the PEAR Mp3_Id package.
Then I move the newly constructed MP3 file into a folder Apache is sharing, and point the outstanding SoundManager2 flash-based MP3 player at it.
For small transactions this works very well -- converting a minute or two voice mail does not add any appreciable lag to actually rendering and returning the rest of the page. As I get more users on a single server, it will probably eventually become necessary to write a cron job or something to do the conversion before the user actually asks for the file the first time.
It's going to be VERY slow doing this in AS3. You really need a C/C++ implementation if you care at all about how long it will take.
Andre Michelle and the Hobnox guys pulled off something similar with their Hobnox AudioTool, they ported a Java Vorbis encoder to AS3. They supposedly ended up with encoding taking twice the time of the audio duration.
Don't know what your use case is, but in the Hobnox tool apparently audio is created at the client side, encoded as Vorbis, sent to the server, converted to mp3 and stored in the users library.