As far as I've read in the manuals, when you have to play a sound, you need a Sound object, and make a temporary SoundChannel object that will control the actual playback. I want to know if there is a memory-efficient way of managing those SoundChannel objects. So far it seems that these are of "fire-and-forget" type of objects, and the only way to make them semi-persistent is make the call to Sound.play() with a really great number of plays. But this approach will not work for one-time sounds, like an arrow shot or a button click, for example. And if I call SoundChannel.stop() I can as well discard the object as there is no means to make it resume playing. Is there any solution to not to spawn SoundChannel objects like crazy, and to be able to handle both one-time sound plays and infinite-time sounds aka background music?
SoundChannel is indeed meant to be throw-away, and this kind of heap usage comes with the territory of using a language like ActionScript.
You shouldn't worry about GC usage from sounds - premature optimization is evil! The best you can do is just reuse your Sound object instead of creating a new one each play. There shouldn't be much of a GC issue if you are playing a reasonable number of sounds per frame, say, in a game. SoundChannels are lightweight and reference a single copy of the audio data, so they aren't such a big deal. There will probably be much heavier allocations to worry about, such as game objects or bitmaps.
You could avoid using SoundChannel by dynamically mixing the audio using SampleDataEvent, but this will certainly have the opposite effect and be much more processor-intensive, not to mention more difficult to code.
If you are really worried about the GC, you could use System.pauseForGCIfCollectionImminent method to hint the GC to run during a non-intrusive time, such as during a transition in a game.
Related
I'm trying to make a program that can convert ORG files into WAV files directly. The ORG format is similar to MIDI, in the sense that it is a list of "instructions" about when and how to play specific instruments, and a program plays these instruments for it to create the song.
However, as I said, I want to generate a WAV directly, instead of just playing the ORG. So, in a sense, I want to "play" the sounds into a WAV. I do know the WAV format and have created some files from raw PCM samples, but this isn't as simple.
The sounds generated by the ORG come from a bunch of files containing WAV samples I have. They're mono, 8-bit samples should be played at 22050Hz. They're all under a second long, and the largest aren't more than 11KB. I would assume that to play them all after each other, I would simply put the samples into the WAV one after the other. It isn't that simple though, as the ORG can have up to 16 different instruments playing at once, and each note of each instrument also has a pan (i.e. a balance, allowing stereo sound). What's more, each ORG has its own tempo (i.e. milliseconds between each point a sound can be played), and some sounds may be longer than this tempo, which means that two sounds on the same instrument can overlap. For instance, a note plays on an instrument, 90 milliseconds later the same note plays on the same instrument, but the first not hasn't finished, hence the first note plays into the second.
I just thought to explain all of that to be sure the situation is clear. In any case, I'd basically like to know how I would go about converting or "playing" an ORG (or if you like, a MIDI (since they're essentially the same)) into a WAV. As I mentioned each note does have a pan/balance, so the WAV would also need to be stereo.
If it matters at all, I'll be doing this in ActionScript 3.0 in FlashDevelop. I don't need any code (as that would be asking someone to do the work for me), but I just want to know how I would go about doing this correctly. An algorithm or two may be handy as well.
First let me say AS3 is not the best language to do these kind of things. Super collider would be a better and easier choice.
But if you want to do it in AS3 here's a general approach. I haven't tested any of it, this is pure theory.
First, put all your sounds into an array, and then find a way of matching the notes from your midi file to a position in the array.
I don't know the format of midi in depth, but I know the smallest value is a tick, and the length of a tick depends on the BPM. Here's the formula to calculate a midi tick: Midi Ticks to Actual PlayBack Seconds !!! ( Midi Music)
Let's say your tick is 2ms in length. So now you have a base value. You can fill a Vector (like an Array but faster) with what happens at every tick. If nothing happens at a particular tick, then insert a null value.
Now the big problem is reading that Vector. It's a problem because the Timer class does not work at small values like 2ms. But what you can do is check the ellapsed time in ms since the app started using getTimer(). You can have some loop that will check the ellapsed time, and whenever you have 2ms more, you read the next index in the Vector. If there are notes on that index, you play the sounds. If not you wait for the next tick.
The problem with this, is that if a loop goes on for more than 15 seconds (I'm not sure of that value) Flash will think the program is not responding and will kill it. So you have to take care of that too, ending the loop and opening a new one before Flash kills your program.
Ok, so now you have sounds playing. You can record the sounds that flash is making (wavs, mp3, mic) with a library called Standing Wave 3.
https://github.com/maxl0rd/standingwave3
This is very theoretical... and I'm quite sure depending on the number of sounds you want to play you can freeze your program... but I hope it will help to get you going.
I'm new to Actionscript. There's probably a better way to do this, and if there is, I'm all ears.
What I'm trying to do is have a background layer run for, say 150 seconds in a loop. Then have another layer (we'll call it Layer 1) with a single object on it loop for 50 seconds. Is there a way to have Layer 1 loop 3 times inside of that 150 seconds that the background layer is looping?
Here's the reason I want Layer 1 to be shorter:
When a certain combination is entered (for example, A1), an item will pop out of and in front of the object on Layer 1.
I haven't written any code for it yet, but my hopeful plan is to have the background layer run continuously then have different scene sections on Layer 1 for each of the items coming out of the object on Layer 1. That way when A1 is entered, Layer1 can goToAndPlay(51) without messing up the background layer.
If it helps you understand it at all, it's a vending machine project. My group's vending machine is the TARDIS. The TARDIS is flying through space while you're entering what you want out of the vending machine and stuff is popping out of it.
If I understand correctly, the background is a MovieClip that loops within its own timeline. When Flash plays through a timeline, the timing is dependent on the performance of the computer and how complex the animation is. You can add an audio track set to 'streaming' to lock the timing down, which will then drop frames if the CPU is overloaded. I have used a silent sound set to loop infinitely and play mode 'streaming' to do this when there is no audio to be used.
Instead of using timeline animations I would recommend using TweenMax http://www.greensock.com/tweenmax/ as it allows tween chaining, that is creating a chain of sequential and parallel tweens. When you use a tween you define the timing in seconds and can use values like 1.25 seconds. It will be accurate to the timing you define. You can also run methods on complete, use easing and all sorts of goodies. If you get comfortable using this you will be able to create much more complex interactions in your Flash projects and also be able to change animations and timing much easier than messing with the timeline.
In fact when hiring Flash developers we always screen candidates by asking if they prefer to do animations on the timeline or programmatically. Although Flash is on its way out, still good to learn as the ideas will apply to javascript and whatever new technology comes about.
I'm currently programming a little game which involved that a dynamically created music is playing, and on a specific tone, a function is called to update the game.
I have no problem with dynamically creating music, either using SampleDataEvent.SAMPLE_DATA or the wonderful standingwave2 lib, but I can't find out how to synchronize sound with code.
I know the "sync" note play every X ms (let's say 500), so I've tried to start a timer which ticks every 500ms right after starting the sound, but it gets eventually out of sync. I'm not sure if the timer isn't good enough to follow the path
I know there's a way to put music on Adobe IDE Frames, play sound as "stream" and then put some code on each frame so I can know where it's called, but I can't dynamically create music that way.
So, does anyone knows a way to synchronize my function call with the sound I'm creating ?
I think this depends on when and how does the music generation takes place. If you are generating that music prior to running the game, then you can yield time offset list in that music when the particular tone is generated into that music, then you make a sorted array out of those values, then when the music is actively started, you take flash.utils.getTimer() value and store it as your base time. After this, each frame you check if current getTimer() value is greater than current array position, and if so, the function you want is called, and you advance one position in the array, to be ready for the next pre-set occurrence of your desired tone.
If, on the other hand, you generate music on the fly, a couple of frames length each, then you have to lock getTimer() value at the start of the game and (supposedly) music generation, so that each pair of values you put into sampleData are exactly 1/44100 second of music played. You then count those pairs (on the fly, of course) until it'll be time to insert your desired tone into the generated music, then you'll have an offset from sound start. Convert it to milliseconds, then check each frame if current getTimer() minus stored tick count is greater or equal to discovered offset, and if true, call the function.
As I know sounds playing correlates with frames even if you add them dynamicaly. Try to use Event.ENTER_FRAME. If you know framerate (by default it's equal to 24 fps) and delay (X ms; when a "sync" note plays) then you can get a "sync" frame's index: index = fps * delay. Because for syncronization of a sound and frames important only a nominal fps, not real. Count frames in the Event.ENTER_FRAME handler. When you will achieve the "sync" frame then you can execute your code.
Why don't you just inform your mechanics code about what happens with music when you do render the music handling SampleDataEvent.SAMPLE_DATA? It should be pretty accurate, and you'll never be out of sync for more than one sound chunk (which is usually 2048-4096 float pairs as far as I remember, means 2048/44100 - 4096/44100 ~= 1/22 - 1/11 seconds). Also, I haven't checked this, but I believe, SAMPLE_DATA is fired right after the existing sound chunk started to play, so you can have the next one after it finishes ready, which would mean, if you write down the time of sound render start, then the actual sound will be played the exact that time later(if your system isn't overloaded), so you can calculate it very precisely, down to milliseconds.
You need to enapsulate that WAV generated music within a FLV stream (using only audio tags). Between the FLV audio tags insert FLV metadata tags, which you will receive through onMetaData just when that portion is playing. Since you are generating music on the fly, you might wanna use NetStream.appendBytes() instead of just passing WAV files to NetStream.
You need to familiarize yourself with how FLV works, how appendBytes works and how to create a FLV (which you write into appendBytes() as if you are writing it to a file) which contains WAV audio.
I'm writing an app that requires an audio stream to be recording while a backing track is played. I have this working, but there is an inconsistent gap in between playback and record starting.
I don't know if I can do anything to make the sync perfect every time, so I've been trying to track what time each stream starts so I can calculate the delay and trim it server-side. This also has proved to be a challenge as no events seem to be sent when a connection starts (as far as I know). I've tried using various properties like the streams' buffer sizes, etc.
I'm thinking now that as my recorded audio is only mono, I may be able to put some kind of 'control signal' on the second stereo track which I could use to determine exactly when a sound starts recording (or stick the whole backing track in that channel so I can sync them that way). This leaves me with the new problem of properly injecting this sound into the NetStream.
If anyone has any idea whether or not any of these ideas will work, how to execute them, or some alternatives, that would be extremely helpful! Been working on this issue for awhile
The only thing that comes to mind is to try and use metadata, flash media streams support metadata and the onMetaData callback. I assume you're using flash media server for the audio coming in and to record the audio going out. If you use the send method while your streaming the audio back to the server, you can put the listening audio track's playhead timestamp in it, so when you get the 2 streams back to the server you can mux them together properly. You can also try encoding the audio that is streamed to the client with metadata and try and use onMetaData to sync them up. I'm not sure how to do this, but a second approach is to try and combine the 2 streams together as the audio goes back so that you don't need to mux them later, or attach it to a blank video stream with 2 audio tracks...
If you're to inject something into the NetStream... As complex as SOUND... I guess here it would be better to go with Socket instead. You'll be directly reading bytes. It's possible there's a compression on the NetStream, so the data sent is not raw sound data - some class for decompressing the codec there would be needed. When you finally get the raw sound data, add the input in there, using Socket.readUnsignedByte() or Socket.readFloat(), and write back the modified data using Socket.writeByte(), or Socket.writeFloat().
This is the alternative with injecting the back into the audio.
For syncing, it is actually quite simple. Even though the data might not be sent instantly, one thing still stays the same - time. So, when user's audio is finished, just mix it without anything else to the back track - the time should stay the same.
IF the user has slow internet DOWNLOAD, so that his backtrack has unwanted breaks - check in the SWF if the data is buffered enough to add the next sound buffer (usually 4096 bytes if I remember correctly). If yes, continue streaming user's audio.
If not, do NOT stream, and start as soon as the data catches back on.
In my experience NetStream is one of the most inaccurate and dirty features of Flash (NetStream:play2 ?!!), which btw is quite ironic seeing how Flash's primary use is probably video playback.
Trying to sync it with anything else in a reliable way is very hard... events and statuses are not very straight forward, and there are multiple issues that can spoil your syncing.
Luckily however, netStream.time will tell you quite accurately the current playhead position, so you can eventually use that to determine starting time, delays, dropped frames, etc... Notice that determining the actual starting time is a bit tricky though. When you start loading a netStream, the time value is zero, but when it shows the first frame and is waiting for the buffer to fill (not playing yet) the time value is something like 0.027 (depends on the video), so you need to very carefully monitor this value to accurately determine events.
An alternative to using NetStream is embedding the video in a SWF file, which should make synchronization much easier (specially if you use frequent keyframes on encoding). But you will lose quality/filesize ratio (If I remember correctly you can only use FLV, not h264).
no events seem to be sent when a connection starts
sure there does.. NetStatusEvent.NET_STATUS fires for a multitude of reasons for NetConnections and Netstreams, you just have to add a listener and process the contents of NET_STATUS.info
the as3 reference docs here and you're looking for NET_STATUS.info
I'm pulling my hair out (once again), trying to find a way to read the activity level of audio of a NetStream, similar to how you can do it with a Microphone. I'd hate to have to let each client send it's activitylevel through SharedObjects or the like, which right seems to be the only way to actually get it to work.
Thanks so much in advance!
-Dave
In AS3 the Netstream Object has a property called .info. This holds the object NetStreamInfo. NetstreamInfo will give you all sorts of metrics. Among them is the property 'audioBytesPerSecond' which will give you an indication of the audio activity at a certain point in time. Requesting the NetStreamInfo for the incoming stream will provide you with the data from the client. Requesting the NetStreamInfo for the outgoing stream will provide you data from your own cam and mic activity. More detail on the NetStreamInfo object can be found here: http://help.adobe.com/en_US/AS3LCR/Flash_10.0/flash/net/NetStreamInfo.html
NetStreamInfo.audioBytesPerSecond is unreliable.
Being a per second average, it takes one extra second before you could detect the lack of sound.
You should instead use NetStreamInfo.audioByteCount. As from Adobe documentation:
Specifies the total number of audio bytes that have arrived in the queue, regardless of how many have been played or flushed. You can use this value to calculate the incoming audio data rate, using the metric of your choice, by creating a timer and calculating the difference in values in successive timer calls.
That's probably the only way to do it. NetStream is a pretty generic object by design. The best site to ask a question like this might be on FlashComGuru.com where a lot of NetStream/FMS guys hang out.
I've seen Flex examples that displays an eq bar.
the code uses two objects
SoundTransform and flash.media.SoundChannel
it functions by dispatching a custom event that has a property which is the SoundChannel object containing the EQ of the playing audio stream.
not sure exactly how the doe works cause it's bundled up in flex, or how to get from the NetStream to the audio based SoundChannel.
The example is in chapter 15 "Building your own components" of the book "Flex 3 component solutions" by jack herrington. published by friends of ed (the pink spine books).
hope that helps.