I have multiple audio files that must be played in sync. I have read that Web Audio API is the best solution for this. But, I can't find any document that shows how to achieve this.
Almost all articles I have read do this to start playback.
//Let's say I have AudioBufferSourceNode connected to two buffers
var source1, source2;
source1.start(0);
source2.start(0);
Shouldn't this cause source2 to start playing slightly later than source1?
Also, what makes the sources stay in sync? I can not find any mention in any documentation that assures that sources are played in sync.
Thanks.
There is a single clock for the audio context, and the buffer playback is on that clock - so yes, they will stay in sync.
Even calling start(0); start(0); as above will be perfectly synchronized, because start() is setting up a scheduling request on the audio thread, and the actual scheduling of both of those will happen together. "now" is actually slightly in the future (the audio system latency).
You can schedule them slightly in the future.
var source1, source2;
var when = context.currentTime + 0.01;
source1.start(when);
source2.start(when);
That'll schedule both sounds to play exactly 10ms from the moment you define when. It's quick enough that it'll be perceived as immediate, but gives a bit of breathing room for the overhead of actually calling start on the first source node.
There are better ways to do this if you have a ton of nodes, but for simple situations, this should be fine.
Related
I'm writing an application which pulls up to several dozen images from a server using Loader objects. It works fine in all browsers except Firefox, where I'm finding that, with over 6 or so connections, some simply never load - and I cease to get progress events (and can detect no errors/error events)
I extended the Loader class so that it will kill and reopen the transfer if it takes longer than ten seconds, but this temporary hack has created a new problem, in that when there are quite a few connections open, many of them will load 90-odd percent of the image, get killed for exceeding the time limit, open again, load 90-odd percent etc...until the traffic is low enough for it to actually complete. This means I'm getting transfers of many times the amount of data that is actually being requested!
It doesn't happen in any other browser (I was anticipating IE errors, so for Firefox to be the anomaly was unexpected!), I can write a class to manage Loaders, but wondered if anyone else had seen this problem?
Thanks in advance for any help!
Maybe try to limit number of concurrent connections.
Instead of loading all assets at once (then FP or browser manages the connections) try to build a queue.
Building a simple queue is fairly easy - just create an array of URLs and shift or pop a value every time loader has finished loading previous asset.
You might use an existing loader manager like LoaderMax or BulkLoader - they allow to create a queue, limit number of connections and are fairly robust. LoaderMax is my favourite.
I am creating an interactive flash application where I need to use many sound files and images. I am currently loading resources (both images and sound files) as I need them. When I run it on my computer everything works fine and I see no delay. Running it on an offsite computer shows pauses and delays. I know this is due to the time it takes to load the resources.
I want to know what the best practice for loading resources is. Should I preload everything in the beginning (which will create a considerable pause)? Is it better to have a short pause multiple times throughout when a new resource is needed? Or is there a better solution (I hope this is the case)?
I have been programming in AS3 for about 3 months so I am relatively new. I'm sorry if this question is really noob, but I do need to figure out what to do about the delays.
It depends on what kind of application it is:
If you expect a user to go through your whole application anyway (like a game for example) you should do preload the resources. If your application is more like a media player or an image gallery where most users will only see a few resources, you should load resources on demand.
It also depends on how large the resources are. You shouldn't let users wait to long or create to much traffic. On the other hand it may be easier to do a single preload than loading each tiny resources in a single request. Maybe you coulddefine larger sets to in a single step to avoid this. You may even combine both methods: preload some common resources and load some other stuff in the background.
I agree with #kapep that you should pre-load commonly used resources at the beginning. If there are large resources that the user may not need you could load those on demand.
If this test is sequential you could load "test question 2" material right after "test question 1" starts. You can load asynchronously so the application is still interactive (user can interact with test question 1) while more material is being loaded (test question 2 resources).
For preloading the resources I strongly recommend using Greensock library. Actually by using LoaderMax you can queue the content to be loaded and then attach the loaded content to the containers (movieclips, sprites etc.)
Here is a sample code for loading multiple resources:
var queue:LoaderMax = new LoaderMaxVars()
.maxConnections(1)
.onProgress(onContentProgress)
.onIOError(onIOErrorHandler)
.onComplete(onContentLoaded));
queue.append(new SWFLoader(contentTobeLoaded,new SWFLoaderVars().name(loaderName)));
....
private function onEmployeeContentLoaded(event:LoaderEvent):void {
var loadedContent:Array = (event.target as LoaderMax).content;
....
I'm writing a recorder for website with flash using the flash.events.SampleDataEvent from the Microphone. But there is one strange thing:
At the beginning the SampleEvent occurs approximately every second. That's really slow. But after waiting a while in front of the browser and starting it again, it's very fast.
So 2 questions:
Is there a way to influence the time between the events
Why is this happening?
Thanks in advance
That is kind of strange. There's no way to tell the event when to dispatch. It just gets triggered when the microphone has sound data in the buffer. For some reason, it doesn't seem like your microphone is recording very much data at first. Try adjusting Microphone.gain and also Microphone.rate. Higher gain will amplify what you are recording to hopefully trigger the event faster and increasing the rate will give you more samples per event.
I'm writing an app that requires an audio stream to be recording while a backing track is played. I have this working, but there is an inconsistent gap in between playback and record starting.
I don't know if I can do anything to make the sync perfect every time, so I've been trying to track what time each stream starts so I can calculate the delay and trim it server-side. This also has proved to be a challenge as no events seem to be sent when a connection starts (as far as I know). I've tried using various properties like the streams' buffer sizes, etc.
I'm thinking now that as my recorded audio is only mono, I may be able to put some kind of 'control signal' on the second stereo track which I could use to determine exactly when a sound starts recording (or stick the whole backing track in that channel so I can sync them that way). This leaves me with the new problem of properly injecting this sound into the NetStream.
If anyone has any idea whether or not any of these ideas will work, how to execute them, or some alternatives, that would be extremely helpful! Been working on this issue for awhile
The only thing that comes to mind is to try and use metadata, flash media streams support metadata and the onMetaData callback. I assume you're using flash media server for the audio coming in and to record the audio going out. If you use the send method while your streaming the audio back to the server, you can put the listening audio track's playhead timestamp in it, so when you get the 2 streams back to the server you can mux them together properly. You can also try encoding the audio that is streamed to the client with metadata and try and use onMetaData to sync them up. I'm not sure how to do this, but a second approach is to try and combine the 2 streams together as the audio goes back so that you don't need to mux them later, or attach it to a blank video stream with 2 audio tracks...
If you're to inject something into the NetStream... As complex as SOUND... I guess here it would be better to go with Socket instead. You'll be directly reading bytes. It's possible there's a compression on the NetStream, so the data sent is not raw sound data - some class for decompressing the codec there would be needed. When you finally get the raw sound data, add the input in there, using Socket.readUnsignedByte() or Socket.readFloat(), and write back the modified data using Socket.writeByte(), or Socket.writeFloat().
This is the alternative with injecting the back into the audio.
For syncing, it is actually quite simple. Even though the data might not be sent instantly, one thing still stays the same - time. So, when user's audio is finished, just mix it without anything else to the back track - the time should stay the same.
IF the user has slow internet DOWNLOAD, so that his backtrack has unwanted breaks - check in the SWF if the data is buffered enough to add the next sound buffer (usually 4096 bytes if I remember correctly). If yes, continue streaming user's audio.
If not, do NOT stream, and start as soon as the data catches back on.
In my experience NetStream is one of the most inaccurate and dirty features of Flash (NetStream:play2 ?!!), which btw is quite ironic seeing how Flash's primary use is probably video playback.
Trying to sync it with anything else in a reliable way is very hard... events and statuses are not very straight forward, and there are multiple issues that can spoil your syncing.
Luckily however, netStream.time will tell you quite accurately the current playhead position, so you can eventually use that to determine starting time, delays, dropped frames, etc... Notice that determining the actual starting time is a bit tricky though. When you start loading a netStream, the time value is zero, but when it shows the first frame and is waiting for the buffer to fill (not playing yet) the time value is something like 0.027 (depends on the video), so you need to very carefully monitor this value to accurately determine events.
An alternative to using NetStream is embedding the video in a SWF file, which should make synchronization much easier (specially if you use frequent keyframes on encoding). But you will lose quality/filesize ratio (If I remember correctly you can only use FLV, not h264).
no events seem to be sent when a connection starts
sure there does.. NetStatusEvent.NET_STATUS fires for a multitude of reasons for NetConnections and Netstreams, you just have to add a listener and process the contents of NET_STATUS.info
the as3 reference docs here and you're looking for NET_STATUS.info
I am a student of Computer Science and have learned many of the basic concepts of what is going on "under the hood" while a computer program is running. But recently I realized that I do not understand how software events work efficiently.
In hardware, this is easy: instead of the processor "busy waiting" to see if something happened, the component sends an interrupt request.
But how does this work in, for example, a mouse-over event? My guess is as follows: if the mouse sends a signal ("moved"), the operating system calculates its new position p, then checks what program is being drawn on the screen, tells that program position p, then the program itself checks what object is at p, checks if any event handlers are associated with said object and finally fires them.
That sounds terribly inefficient to me, since a tiny mouse movement equates to a lot of cpu context switches (which I learned are relatively expensive). And then there are dozens of background applications that may want to do stuff of their own as well.
Where is my intuition failing me? I realize that even "slow" 500MHz processors do 500 million operations per second, but still it seems too much work for such a simple event.
Thanks in advance!
Think of events like network packets, since they're usually handled by similar mechanisms. Now think, your mouse sends a couple of hundred packets a second, maximum, and they're around 6 bytes each. That's nothing compared to the bandwidth capabilities of modern machines.
In fact, you could make a responsive GUI where every mouse motion literally sent a network packet (86 bytes including headers) on hardware built around 20 years ago: X11, the fundamental GUI mechanism for Linux and most other Unixes, can do exactly that, and frequently was used that way in the late 80s and early 90s. When I first used a GUI, that's what it was, and while it wasn't great by current standards, given that it was running on 20 MHz machines, it really was usable.
My understanding is as follows:
Every application/window has an event loop which is filled by the OS-interrupts.
The mouse move will come in there.
All windows have a separate queue/process by my knowledge (in windows since 3.1)
Every window has controls.
The window will bubble up this events to the controls.
The control will determine if the event is for him.
So its not necessary to "compute" which item is drawn under the mouse cursor.
The window, and then the control will determine if the event is for them.
By what criteria do you determine that it's too much? It's as much work as it needs to be. Mouse events happen in the millisecond range. The work required to get it to the handler code is probably measured in microseconds. It's just not an issue.
You're pretty much right - though mouse events occur at a fixed rate(e.g. an USB mouse on linux gives you events 125 times a second by default - which really is not a lot),, and the OS or application might further merge mouse events that's close in time or position before sending it off to be handled