I'm trying to make a program that can convert ORG files into WAV files directly. The ORG format is similar to MIDI, in the sense that it is a list of "instructions" about when and how to play specific instruments, and a program plays these instruments for it to create the song.
However, as I said, I want to generate a WAV directly, instead of just playing the ORG. So, in a sense, I want to "play" the sounds into a WAV. I do know the WAV format and have created some files from raw PCM samples, but this isn't as simple.
The sounds generated by the ORG come from a bunch of files containing WAV samples I have. They're mono, 8-bit samples should be played at 22050Hz. They're all under a second long, and the largest aren't more than 11KB. I would assume that to play them all after each other, I would simply put the samples into the WAV one after the other. It isn't that simple though, as the ORG can have up to 16 different instruments playing at once, and each note of each instrument also has a pan (i.e. a balance, allowing stereo sound). What's more, each ORG has its own tempo (i.e. milliseconds between each point a sound can be played), and some sounds may be longer than this tempo, which means that two sounds on the same instrument can overlap. For instance, a note plays on an instrument, 90 milliseconds later the same note plays on the same instrument, but the first not hasn't finished, hence the first note plays into the second.
I just thought to explain all of that to be sure the situation is clear. In any case, I'd basically like to know how I would go about converting or "playing" an ORG (or if you like, a MIDI (since they're essentially the same)) into a WAV. As I mentioned each note does have a pan/balance, so the WAV would also need to be stereo.
If it matters at all, I'll be doing this in ActionScript 3.0 in FlashDevelop. I don't need any code (as that would be asking someone to do the work for me), but I just want to know how I would go about doing this correctly. An algorithm or two may be handy as well.
First let me say AS3 is not the best language to do these kind of things. Super collider would be a better and easier choice.
But if you want to do it in AS3 here's a general approach. I haven't tested any of it, this is pure theory.
First, put all your sounds into an array, and then find a way of matching the notes from your midi file to a position in the array.
I don't know the format of midi in depth, but I know the smallest value is a tick, and the length of a tick depends on the BPM. Here's the formula to calculate a midi tick: Midi Ticks to Actual PlayBack Seconds !!! ( Midi Music)
Let's say your tick is 2ms in length. So now you have a base value. You can fill a Vector (like an Array but faster) with what happens at every tick. If nothing happens at a particular tick, then insert a null value.
Now the big problem is reading that Vector. It's a problem because the Timer class does not work at small values like 2ms. But what you can do is check the ellapsed time in ms since the app started using getTimer(). You can have some loop that will check the ellapsed time, and whenever you have 2ms more, you read the next index in the Vector. If there are notes on that index, you play the sounds. If not you wait for the next tick.
The problem with this, is that if a loop goes on for more than 15 seconds (I'm not sure of that value) Flash will think the program is not responding and will kill it. So you have to take care of that too, ending the loop and opening a new one before Flash kills your program.
Ok, so now you have sounds playing. You can record the sounds that flash is making (wavs, mp3, mic) with a library called Standing Wave 3.
https://github.com/maxl0rd/standingwave3
This is very theoretical... and I'm quite sure depending on the number of sounds you want to play you can freeze your program... but I hope it will help to get you going.
Related
I'm making a game based on simple song creation, and I'm planning on having a feature where players can listen to the songs they've created with the game. the rhythm of the melody is controlled with a system of timers, but this will not work for the backing track presets I am planning on implementing, as each mp3 file in the backing tracks represents one bar instead of one note.
while it would be possible to use my timer system for playing the backing tracks, this would require several more audio files, much more coding and would push the project far behind schedule. therefore, i need to manipulate the playback speeds of the files I already have. I've commonly seen two examples of how to do this, here: http://2008.kelvinluck.com/2008/11/first-steps-with-flash-10-audio-programming/ and here: http://blog.andre-michelle.com/2009/pitch-mp3/
the problem with both of these is that they also alter the pitch as well. this is a problem for me, as I would very much like players to be able to alter the pitch and tempo of their songs separately. I think the code I need is similar to the examples above, but I'm having trouble understanding those since I haven't had much experience with bytearrays and such. I'd like to be able to understand the examples i included so that I can figure out what I need to do in order to get my game working the way it should, but help of any sort is appreciated. thank you =)
You can try https://github.com/also/soundtouch-as3
there is an early (alpha) demo at http://static.ryanberdeen.com/projects/soundtouch-as3/demo/player/stretch.swf
quality is "acceptable" if you are using 1.0x-1.5x factor (less than 1.0 gives very artificial distortion)
You could also try :
http://iq12.com/old_blog/2009/08/25/real-time-pitch-shifting/
test : Online demo
Their online demo didn't load a track for me (deleted MP3?), so I put a recompiled SWF on my server just for testing. It loads this MP3 audio clip, if you wanna compare results to original sound.
It aims to preserve audio length (time scale) whilst adjusting pitch (deeper when slow or higher when faster). You could possibly combine this with Kelvin Luck's Second Steps.... Thereby having example codes for speed and pitch.
It was inspired by some C# code ported from this C code as found at (with concept explanation) : http://blogs.zynaptiq.com/bernsee/pitch-shifting-using-the-ft/
Is it possible to create an automatically generated Rhythm game for Flash Action Script 3 ?
But not just randomly generated, generated from the notes of a song. Or is that something I have to do manually?
How would I go about doing either of these?
I am currently following this tutorial: http://www.flashgametuts.com/tutorials/as3/how-to-make-a-rhythm-game-in-as3-part-4/ so perhaps it can be made to fit around this? (Go to the final part and View Source to see the full thing)
Thanks!
Depending on what you mean by rhythm game, check out the computeSpectrum() function of the SoundMixer class: http://help.adobe.com/en_US/FlashPlatform/reference/actionscript/3/flash/media/SoundMixer.html#computeSpectrum()
There's an example of it working in the link, but basically what it does is take a snapshot of the current sound wave and puts normalised (-1 to 1) values in a ByteArray. What you do with those values is up to you - e.g. you might use them as a height field to generate terrain for example.
Repeat this every frame, and you get the gist
Welcome to SO!
First off, there is nothing already built in, to my knowledge. There may be something lurking around Google that someone else wrote, but you'd need to dig around for that (though I assume you already did.)
Generated from the notes of a song. Hmm, this will take some serious ingenuity and coding on your part. I'll point you in the right direction, but it is up to you to write the code. No one here will do it for you, but we'll happily help with specific problems in your code.
The crazy (yet potentially more fun) approach MAY BE to derive the data in a similar manner that an audio visualizer does...but I can't guarantee that will work. This would work best with MIDI-generated, single instrument songs. Here is a tutorial on visualizers.
A second approach may be to actually convert MIDI files directly. Again, I can't guarantee it will work, but it would theoretically be possible, seeing how MIDI files store data to begin with. Here's an answer on playing MIDI files, to get you started. Consider looking through their class.
However, the "easiest" approach would be to come up with some sort of system by which you store the note values for a song. You can manually enter the values in an array, or in a data file (such as XML) that you can load.
I put "easiest" in quotes because you'd have to account for a LOT of information - not just note values, but note duration, rhythm, and rests.
Anyway, those are just a few ideas to get you started. Good luck!
I'm currently programming a little game which involved that a dynamically created music is playing, and on a specific tone, a function is called to update the game.
I have no problem with dynamically creating music, either using SampleDataEvent.SAMPLE_DATA or the wonderful standingwave2 lib, but I can't find out how to synchronize sound with code.
I know the "sync" note play every X ms (let's say 500), so I've tried to start a timer which ticks every 500ms right after starting the sound, but it gets eventually out of sync. I'm not sure if the timer isn't good enough to follow the path
I know there's a way to put music on Adobe IDE Frames, play sound as "stream" and then put some code on each frame so I can know where it's called, but I can't dynamically create music that way.
So, does anyone knows a way to synchronize my function call with the sound I'm creating ?
I think this depends on when and how does the music generation takes place. If you are generating that music prior to running the game, then you can yield time offset list in that music when the particular tone is generated into that music, then you make a sorted array out of those values, then when the music is actively started, you take flash.utils.getTimer() value and store it as your base time. After this, each frame you check if current getTimer() value is greater than current array position, and if so, the function you want is called, and you advance one position in the array, to be ready for the next pre-set occurrence of your desired tone.
If, on the other hand, you generate music on the fly, a couple of frames length each, then you have to lock getTimer() value at the start of the game and (supposedly) music generation, so that each pair of values you put into sampleData are exactly 1/44100 second of music played. You then count those pairs (on the fly, of course) until it'll be time to insert your desired tone into the generated music, then you'll have an offset from sound start. Convert it to milliseconds, then check each frame if current getTimer() minus stored tick count is greater or equal to discovered offset, and if true, call the function.
As I know sounds playing correlates with frames even if you add them dynamicaly. Try to use Event.ENTER_FRAME. If you know framerate (by default it's equal to 24 fps) and delay (X ms; when a "sync" note plays) then you can get a "sync" frame's index: index = fps * delay. Because for syncronization of a sound and frames important only a nominal fps, not real. Count frames in the Event.ENTER_FRAME handler. When you will achieve the "sync" frame then you can execute your code.
Why don't you just inform your mechanics code about what happens with music when you do render the music handling SampleDataEvent.SAMPLE_DATA? It should be pretty accurate, and you'll never be out of sync for more than one sound chunk (which is usually 2048-4096 float pairs as far as I remember, means 2048/44100 - 4096/44100 ~= 1/22 - 1/11 seconds). Also, I haven't checked this, but I believe, SAMPLE_DATA is fired right after the existing sound chunk started to play, so you can have the next one after it finishes ready, which would mean, if you write down the time of sound render start, then the actual sound will be played the exact that time later(if your system isn't overloaded), so you can calculate it very precisely, down to milliseconds.
You need to enapsulate that WAV generated music within a FLV stream (using only audio tags). Between the FLV audio tags insert FLV metadata tags, which you will receive through onMetaData just when that portion is playing. Since you are generating music on the fly, you might wanna use NetStream.appendBytes() instead of just passing WAV files to NetStream.
You need to familiarize yourself with how FLV works, how appendBytes works and how to create a FLV (which you write into appendBytes() as if you are writing it to a file) which contains WAV audio.
I tried to look for free H.264 editors that could do such forementioned tasks, but i couldn't find any suitable results (probably due to my incorrect(?) search terms).
Basically, i have quite a few 20 to 40 second looping movie files (rendered with Adobe Premiere), and i would like to multiply an individual movie to about ten or twenty fold, and then save it again, preferably without the need to re-render (or re-encode?). Is this possible?
Hopefully i managed to make myself understood, thanks :)
After a whole bunch of searching and testing, i ended up with Avidemux and it did the task perfectly, this time around. I'm sure there would've been a way to automatize the appending / "multiplying" of the file, but with a tablet pen (main click set as double click) and specific folder it truly went by like a breeze.
I was also interested of Smart Cutter (i quite enjoyed the interface after getting used to it), but since it was trialware the choice was clear. Still, if i would have to do these kind of tasks more often, i might consider purchasing it.
I'm writing an app that requires an audio stream to be recording while a backing track is played. I have this working, but there is an inconsistent gap in between playback and record starting.
I don't know if I can do anything to make the sync perfect every time, so I've been trying to track what time each stream starts so I can calculate the delay and trim it server-side. This also has proved to be a challenge as no events seem to be sent when a connection starts (as far as I know). I've tried using various properties like the streams' buffer sizes, etc.
I'm thinking now that as my recorded audio is only mono, I may be able to put some kind of 'control signal' on the second stereo track which I could use to determine exactly when a sound starts recording (or stick the whole backing track in that channel so I can sync them that way). This leaves me with the new problem of properly injecting this sound into the NetStream.
If anyone has any idea whether or not any of these ideas will work, how to execute them, or some alternatives, that would be extremely helpful! Been working on this issue for awhile
The only thing that comes to mind is to try and use metadata, flash media streams support metadata and the onMetaData callback. I assume you're using flash media server for the audio coming in and to record the audio going out. If you use the send method while your streaming the audio back to the server, you can put the listening audio track's playhead timestamp in it, so when you get the 2 streams back to the server you can mux them together properly. You can also try encoding the audio that is streamed to the client with metadata and try and use onMetaData to sync them up. I'm not sure how to do this, but a second approach is to try and combine the 2 streams together as the audio goes back so that you don't need to mux them later, or attach it to a blank video stream with 2 audio tracks...
If you're to inject something into the NetStream... As complex as SOUND... I guess here it would be better to go with Socket instead. You'll be directly reading bytes. It's possible there's a compression on the NetStream, so the data sent is not raw sound data - some class for decompressing the codec there would be needed. When you finally get the raw sound data, add the input in there, using Socket.readUnsignedByte() or Socket.readFloat(), and write back the modified data using Socket.writeByte(), or Socket.writeFloat().
This is the alternative with injecting the back into the audio.
For syncing, it is actually quite simple. Even though the data might not be sent instantly, one thing still stays the same - time. So, when user's audio is finished, just mix it without anything else to the back track - the time should stay the same.
IF the user has slow internet DOWNLOAD, so that his backtrack has unwanted breaks - check in the SWF if the data is buffered enough to add the next sound buffer (usually 4096 bytes if I remember correctly). If yes, continue streaming user's audio.
If not, do NOT stream, and start as soon as the data catches back on.
In my experience NetStream is one of the most inaccurate and dirty features of Flash (NetStream:play2 ?!!), which btw is quite ironic seeing how Flash's primary use is probably video playback.
Trying to sync it with anything else in a reliable way is very hard... events and statuses are not very straight forward, and there are multiple issues that can spoil your syncing.
Luckily however, netStream.time will tell you quite accurately the current playhead position, so you can eventually use that to determine starting time, delays, dropped frames, etc... Notice that determining the actual starting time is a bit tricky though. When you start loading a netStream, the time value is zero, but when it shows the first frame and is waiting for the buffer to fill (not playing yet) the time value is something like 0.027 (depends on the video), so you need to very carefully monitor this value to accurately determine events.
An alternative to using NetStream is embedding the video in a SWF file, which should make synchronization much easier (specially if you use frequent keyframes on encoding). But you will lose quality/filesize ratio (If I remember correctly you can only use FLV, not h264).
no events seem to be sent when a connection starts
sure there does.. NetStatusEvent.NET_STATUS fires for a multitude of reasons for NetConnections and Netstreams, you just have to add a listener and process the contents of NET_STATUS.info
the as3 reference docs here and you're looking for NET_STATUS.info