I'm pulling my hair out (once again), trying to find a way to read the activity level of audio of a NetStream, similar to how you can do it with a Microphone. I'd hate to have to let each client send it's activitylevel through SharedObjects or the like, which right seems to be the only way to actually get it to work.
Thanks so much in advance!
-Dave
In AS3 the Netstream Object has a property called .info. This holds the object NetStreamInfo. NetstreamInfo will give you all sorts of metrics. Among them is the property 'audioBytesPerSecond' which will give you an indication of the audio activity at a certain point in time. Requesting the NetStreamInfo for the incoming stream will provide you with the data from the client. Requesting the NetStreamInfo for the outgoing stream will provide you data from your own cam and mic activity. More detail on the NetStreamInfo object can be found here: http://help.adobe.com/en_US/AS3LCR/Flash_10.0/flash/net/NetStreamInfo.html
NetStreamInfo.audioBytesPerSecond is unreliable.
Being a per second average, it takes one extra second before you could detect the lack of sound.
You should instead use NetStreamInfo.audioByteCount. As from Adobe documentation:
Specifies the total number of audio bytes that have arrived in the queue, regardless of how many have been played or flushed. You can use this value to calculate the incoming audio data rate, using the metric of your choice, by creating a timer and calculating the difference in values in successive timer calls.
That's probably the only way to do it. NetStream is a pretty generic object by design. The best site to ask a question like this might be on FlashComGuru.com where a lot of NetStream/FMS guys hang out.
I've seen Flex examples that displays an eq bar.
the code uses two objects
SoundTransform and flash.media.SoundChannel
it functions by dispatching a custom event that has a property which is the SoundChannel object containing the EQ of the playing audio stream.
not sure exactly how the doe works cause it's bundled up in flex, or how to get from the NetStream to the audio based SoundChannel.
The example is in chapter 15 "Building your own components" of the book "Flex 3 component solutions" by jack herrington. published by friends of ed (the pink spine books).
hope that helps.
Related
I want to create about 20,000 "Person" objects in a worker, and then pass them to the main application.
I understand that these objects will be cloned between threads and that "Person" support AMF serialization.
But I worry about performance - will my game suffer any performance hit when receiving those thousands of objects? will the current frame get stuck? or does AIR take care of it, and it will be a smooth transaction?
Will be thankful for any help on this issue.
Thanks
Update:
The objects I am creating are sort of animation instructions, not BOTs o AI.
They are downloaded in binary format, and get serialized to AS3 objects.
My game must load them during active play, and today I pay a painful price of 4-5 frames getting stuck.
If Workers would download and create them, and if the main thread would receive them without getting stuck - I would call it a victory...
I want only 1 thread, which will create about 20,000 objects.
That background thread will send those objects to the main thread.
If the main thread won't get stuck when receiving/deserializing the objects - challenge achieved.
So my hope is that the deserialization on the main thread doesn't stick it.
Am I on the right path?
Thanks for all the answers!
Other questions didn't really have an answer. I'm making a singleplayer game that saves to a leaderboard, and I can't have the scores be 999999999999999999999999 from Cheat Engine. How do I secure my AS3 Flash Game so that Cheat Engine does nothing?
You can't. It's unavoidable. Abandon all hope. Your game is client-side and can be tampered with. The score is sent from the client which can be intercepted and changed before it is sent to the server. Anything you do to try encode the score will fail because your SWF can be decompiled and the algorithm reverse-engineered. Even if you put yourself through hell to obfuscate your SWF and the logic used to encode a decode a score, you will not prevail. All you can do is make it not worthwhile to cheat by maximising the difficulty of sending faux scores; make the criteria of a valid score strict and hard to determine e.g. a multiple of a given number minus x.
There are ways to make it very difficult, although as others have mentioned, it will never be completely secure.
Neopets, a popular site with flash games, combats this problem by sending extra information to the server. For example, in a game you might send to the server:
Time to complete level
Number of enemies killed
Number of items collected
Score
Then on the server, check if the values make sense. If they did not kill enough enemies, reject their score. If they completed the level too quickly, reject their score.
Bind your data to something dynamically changing like time. Because cheat engine does not have such option to trace time in data. Additionally if the trainer is programmed so professionally you can combine this method and other distraction methods together like multiplying the number into another dynamic data.
I'm trying to make a program that can convert ORG files into WAV files directly. The ORG format is similar to MIDI, in the sense that it is a list of "instructions" about when and how to play specific instruments, and a program plays these instruments for it to create the song.
However, as I said, I want to generate a WAV directly, instead of just playing the ORG. So, in a sense, I want to "play" the sounds into a WAV. I do know the WAV format and have created some files from raw PCM samples, but this isn't as simple.
The sounds generated by the ORG come from a bunch of files containing WAV samples I have. They're mono, 8-bit samples should be played at 22050Hz. They're all under a second long, and the largest aren't more than 11KB. I would assume that to play them all after each other, I would simply put the samples into the WAV one after the other. It isn't that simple though, as the ORG can have up to 16 different instruments playing at once, and each note of each instrument also has a pan (i.e. a balance, allowing stereo sound). What's more, each ORG has its own tempo (i.e. milliseconds between each point a sound can be played), and some sounds may be longer than this tempo, which means that two sounds on the same instrument can overlap. For instance, a note plays on an instrument, 90 milliseconds later the same note plays on the same instrument, but the first not hasn't finished, hence the first note plays into the second.
I just thought to explain all of that to be sure the situation is clear. In any case, I'd basically like to know how I would go about converting or "playing" an ORG (or if you like, a MIDI (since they're essentially the same)) into a WAV. As I mentioned each note does have a pan/balance, so the WAV would also need to be stereo.
If it matters at all, I'll be doing this in ActionScript 3.0 in FlashDevelop. I don't need any code (as that would be asking someone to do the work for me), but I just want to know how I would go about doing this correctly. An algorithm or two may be handy as well.
First let me say AS3 is not the best language to do these kind of things. Super collider would be a better and easier choice.
But if you want to do it in AS3 here's a general approach. I haven't tested any of it, this is pure theory.
First, put all your sounds into an array, and then find a way of matching the notes from your midi file to a position in the array.
I don't know the format of midi in depth, but I know the smallest value is a tick, and the length of a tick depends on the BPM. Here's the formula to calculate a midi tick: Midi Ticks to Actual PlayBack Seconds !!! ( Midi Music)
Let's say your tick is 2ms in length. So now you have a base value. You can fill a Vector (like an Array but faster) with what happens at every tick. If nothing happens at a particular tick, then insert a null value.
Now the big problem is reading that Vector. It's a problem because the Timer class does not work at small values like 2ms. But what you can do is check the ellapsed time in ms since the app started using getTimer(). You can have some loop that will check the ellapsed time, and whenever you have 2ms more, you read the next index in the Vector. If there are notes on that index, you play the sounds. If not you wait for the next tick.
The problem with this, is that if a loop goes on for more than 15 seconds (I'm not sure of that value) Flash will think the program is not responding and will kill it. So you have to take care of that too, ending the loop and opening a new one before Flash kills your program.
Ok, so now you have sounds playing. You can record the sounds that flash is making (wavs, mp3, mic) with a library called Standing Wave 3.
https://github.com/maxl0rd/standingwave3
This is very theoretical... and I'm quite sure depending on the number of sounds you want to play you can freeze your program... but I hope it will help to get you going.
I'm currently programming a little game which involved that a dynamically created music is playing, and on a specific tone, a function is called to update the game.
I have no problem with dynamically creating music, either using SampleDataEvent.SAMPLE_DATA or the wonderful standingwave2 lib, but I can't find out how to synchronize sound with code.
I know the "sync" note play every X ms (let's say 500), so I've tried to start a timer which ticks every 500ms right after starting the sound, but it gets eventually out of sync. I'm not sure if the timer isn't good enough to follow the path
I know there's a way to put music on Adobe IDE Frames, play sound as "stream" and then put some code on each frame so I can know where it's called, but I can't dynamically create music that way.
So, does anyone knows a way to synchronize my function call with the sound I'm creating ?
I think this depends on when and how does the music generation takes place. If you are generating that music prior to running the game, then you can yield time offset list in that music when the particular tone is generated into that music, then you make a sorted array out of those values, then when the music is actively started, you take flash.utils.getTimer() value and store it as your base time. After this, each frame you check if current getTimer() value is greater than current array position, and if so, the function you want is called, and you advance one position in the array, to be ready for the next pre-set occurrence of your desired tone.
If, on the other hand, you generate music on the fly, a couple of frames length each, then you have to lock getTimer() value at the start of the game and (supposedly) music generation, so that each pair of values you put into sampleData are exactly 1/44100 second of music played. You then count those pairs (on the fly, of course) until it'll be time to insert your desired tone into the generated music, then you'll have an offset from sound start. Convert it to milliseconds, then check each frame if current getTimer() minus stored tick count is greater or equal to discovered offset, and if true, call the function.
As I know sounds playing correlates with frames even if you add them dynamicaly. Try to use Event.ENTER_FRAME. If you know framerate (by default it's equal to 24 fps) and delay (X ms; when a "sync" note plays) then you can get a "sync" frame's index: index = fps * delay. Because for syncronization of a sound and frames important only a nominal fps, not real. Count frames in the Event.ENTER_FRAME handler. When you will achieve the "sync" frame then you can execute your code.
Why don't you just inform your mechanics code about what happens with music when you do render the music handling SampleDataEvent.SAMPLE_DATA? It should be pretty accurate, and you'll never be out of sync for more than one sound chunk (which is usually 2048-4096 float pairs as far as I remember, means 2048/44100 - 4096/44100 ~= 1/22 - 1/11 seconds). Also, I haven't checked this, but I believe, SAMPLE_DATA is fired right after the existing sound chunk started to play, so you can have the next one after it finishes ready, which would mean, if you write down the time of sound render start, then the actual sound will be played the exact that time later(if your system isn't overloaded), so you can calculate it very precisely, down to milliseconds.
You need to enapsulate that WAV generated music within a FLV stream (using only audio tags). Between the FLV audio tags insert FLV metadata tags, which you will receive through onMetaData just when that portion is playing. Since you are generating music on the fly, you might wanna use NetStream.appendBytes() instead of just passing WAV files to NetStream.
You need to familiarize yourself with how FLV works, how appendBytes works and how to create a FLV (which you write into appendBytes() as if you are writing it to a file) which contains WAV audio.
I'm writing an app that requires an audio stream to be recording while a backing track is played. I have this working, but there is an inconsistent gap in between playback and record starting.
I don't know if I can do anything to make the sync perfect every time, so I've been trying to track what time each stream starts so I can calculate the delay and trim it server-side. This also has proved to be a challenge as no events seem to be sent when a connection starts (as far as I know). I've tried using various properties like the streams' buffer sizes, etc.
I'm thinking now that as my recorded audio is only mono, I may be able to put some kind of 'control signal' on the second stereo track which I could use to determine exactly when a sound starts recording (or stick the whole backing track in that channel so I can sync them that way). This leaves me with the new problem of properly injecting this sound into the NetStream.
If anyone has any idea whether or not any of these ideas will work, how to execute them, or some alternatives, that would be extremely helpful! Been working on this issue for awhile
The only thing that comes to mind is to try and use metadata, flash media streams support metadata and the onMetaData callback. I assume you're using flash media server for the audio coming in and to record the audio going out. If you use the send method while your streaming the audio back to the server, you can put the listening audio track's playhead timestamp in it, so when you get the 2 streams back to the server you can mux them together properly. You can also try encoding the audio that is streamed to the client with metadata and try and use onMetaData to sync them up. I'm not sure how to do this, but a second approach is to try and combine the 2 streams together as the audio goes back so that you don't need to mux them later, or attach it to a blank video stream with 2 audio tracks...
If you're to inject something into the NetStream... As complex as SOUND... I guess here it would be better to go with Socket instead. You'll be directly reading bytes. It's possible there's a compression on the NetStream, so the data sent is not raw sound data - some class for decompressing the codec there would be needed. When you finally get the raw sound data, add the input in there, using Socket.readUnsignedByte() or Socket.readFloat(), and write back the modified data using Socket.writeByte(), or Socket.writeFloat().
This is the alternative with injecting the back into the audio.
For syncing, it is actually quite simple. Even though the data might not be sent instantly, one thing still stays the same - time. So, when user's audio is finished, just mix it without anything else to the back track - the time should stay the same.
IF the user has slow internet DOWNLOAD, so that his backtrack has unwanted breaks - check in the SWF if the data is buffered enough to add the next sound buffer (usually 4096 bytes if I remember correctly). If yes, continue streaming user's audio.
If not, do NOT stream, and start as soon as the data catches back on.
In my experience NetStream is one of the most inaccurate and dirty features of Flash (NetStream:play2 ?!!), which btw is quite ironic seeing how Flash's primary use is probably video playback.
Trying to sync it with anything else in a reliable way is very hard... events and statuses are not very straight forward, and there are multiple issues that can spoil your syncing.
Luckily however, netStream.time will tell you quite accurately the current playhead position, so you can eventually use that to determine starting time, delays, dropped frames, etc... Notice that determining the actual starting time is a bit tricky though. When you start loading a netStream, the time value is zero, but when it shows the first frame and is waiting for the buffer to fill (not playing yet) the time value is something like 0.027 (depends on the video), so you need to very carefully monitor this value to accurately determine events.
An alternative to using NetStream is embedding the video in a SWF file, which should make synchronization much easier (specially if you use frequent keyframes on encoding). But you will lose quality/filesize ratio (If I remember correctly you can only use FLV, not h264).
no events seem to be sent when a connection starts
sure there does.. NetStatusEvent.NET_STATUS fires for a multitude of reasons for NetConnections and Netstreams, you just have to add a listener and process the contents of NET_STATUS.info
the as3 reference docs here and you're looking for NET_STATUS.info