The scenario: A guy walks along a route, through the crowd at a pool party.
The camera setup, is a customised rig with an array of GoPro's covering 360 degrees of rotation.
The end result needs to be a video that'll let the user
click & drag the video to change his viewpoint on the video. So for eg, he can turn the angle to behind him, and will see where the guy has walked from. Or he can look sideways as he walks. (Likely some up/down movement too)
pause playback
zoom in/out
So for eg, you spot a hot girl in the crowd. You'd pause, zoom in and then play the video watching her as the guy walks past her.
How could this be achieved with HTML5 (non-Flash) methods?
I don't even know what technologies would be required to achieve something like this, so I'm hoping that someone with a bit of experience in something similar could give me some pointers as to required
coding languages
server technologies
bandwidth considerations
etc
Thanks for your help!
(ps: this is a paid client job. so if you can do exactly this, lets talk about a quote?)
You'd be attempting something quite state-of-the-art.
The way I'd experiment with is to stream a video to the client and display it using WebGL, which the client can then manipulate without latency.
http://riaconnection.wordpress.com/2011/11/03/testing-live-video-streaming-to-webgl-and-html5-video-tag/
One way might be to stream 6 feeds - top, bottom, left, right, back front. These would be pre-processed so when displayed as a cube viewed from the center of the cube, the perspectives are corrected.
If the client can zoom in and out, then that means you'll need higher resolution stream. Six of them would mean very large bandwidth. You'll have to decide on a trade-off between bandwidth, quality and latency. If the client zooms in and changes pan / tilt, you could trade off latency and get better quality and bandwidth, but at the cost of higher server resource requirements.
There are plenty of video processing libraries for PHP, which would probably be my choice of server, but I'm biased.
Related
One pet peeve I have with a lot of older rock music records from the 60's and 70's
is they sometimes are slightly of of tune with the standard a440 reference pitch, so playing along on my guitar sounds bad. Right now I manually adjust the pitch of the track by ear in Audacity, but this often takes some time. I was hoping there was something out there that could detect the frequency of the first few notes of a music file, analyze that and see how far it is from any standard A440 note, and then you could use that number to determine how much you needed to make the track sharp or flat to match A440 tuning.
Is it possible to create an app that automatically retunes slightly out-of-tune music to be A440 with Javascript?
Yes. Check out this pitch detection demo by Chris Wilson.
There are some big caveats though, and they don't really have anything to do with JavaScript.
Voice Selection
What instruments do you care about? Obviously wideband stuff like percussion is atonal... but it's still going to turn up in your audio, and will have a pitch. How about the difference between a voice and wind instruments? A lot of vocalists will go off pitch intentionally. You will need to use bandpass fliters to get the range you care about, at a minimum.
Original Intonation
What about the instruments being in tune in the first place? "Another one bites the dust" comes to mind as a rock record from the 70s that starts off terribly out of tune... and maybe intentionally.
Pyscoacoustics and Harmonics
Sometimes we hear things that aren't really there. If I play a trombone down a scale, to the very bottom of its range, it will actually stop producing a fundamental... yet because I was playing a familiar scale downward, us humans will still perceive one. This might not throw off your detection by a lot, but it will somewhat, particularly on those atonal-but-not-really percussion instruments.
In summary, this is possible but it's going to be a lot of work. Please come back and post a link once you've done this!
For the past year I have been working on an isometric city builder. So far I have not used any framework apart from a loose PureMVC clone.
I have heard of Starling but only recently have I played with it.
From my research, the performance boost is fenomenal, but this forces me to manage my resource a lot tighter.
At the moment, I am exporting building animations one building at a time, in ~16 frames/pngs. These are cropped, resized and exported in Photoshop by a script and then imported in Flash, then exported as a swf, to be loaded / preloaded / postloaded on demand.
The frames are way too big to make a spritesheet with them, per building. I believe its called an atlas.
These pngs are then blited between lock() and unlock(). After the buildings + actors walking around are sorted, that is.
I am unsure if just using starling.Movieclip for the buildings, where instead of loading the pngs, I would build a MovieClip symbol with its frames. So bliting wouldn't even be necessary. Unless adding bliting on top of Starling would improve performance even more. That would allow fatter features such as particles effects, maybe some lighting.
Google isn't offering me a strait answer, thus I am asking here.
Google isn't offering a straight answer because there isn't such. It depends very very much on what you've done, how much knowledge you've got and what are your goals.
Using Starling gives benefits as well as drawbacks. Your idea of resources will change totally. If you really have enormous amount of assets, then putting them into GPU will be really slow process. You must start from there - learn what Starling does, how resources are managed with it and what you need to change in order to make the transition between the two.
If this is not that hard and time consuming task, you will have some performance optimization. BUT again it depends on your current code. Your current code is really important in this situation as if it's perfectly optimized your gain won't be that much (but commonly would still be).
If you need to switch between graphics regularly or you need to have dynamic assets (as images for example) you must keep in mind that uploading to GPU is the slowest part when talking about Starling and Stage3D.
So again, there is not a straight answer. You must think of GPU memory and limit, GPU upload time, as well as assets management. You also need to think of the way your code is built and if you are going to have any impact if you make the switch (if your code heavily depends on the MovieClip like structure, with all that frames and things) - it will be hard for you. One of the toughest things I fought with Stage3D was the UI implementation - there is almost only Feathers UI which will take you a few weeks to get along with.
On the other hand, Starling performs pretty well, especially on mobile devices. I was able to maintain a stable 45fps on a heavy UI app with a lot of dynamic loading content and multiple screens on an old iPhone 4S, which I find great. Latest mobile devices top at 60fps.
It's up to you to decide, but I'll advise you to have some experimental long-lasting project to test with, and then start applying this approach to your regular projects. I've done the dive to use it in a regular very tightened deadline project, and it was a nightmare. Everything worked out great, but I thought I would have a heart attack - the switch is not that easy :)
I would suggest using DMT for rasterizing your vector assets into Straling sprites at runtime, and it'll also keep your DisplayTree! meaning that you'll still have the parent/child relations that you had in your Flash Assets.
DMT will not duplicate assets, and will rasterize the vectors into texture atlases only one time (Cache is saved)
you can find it here: https://github.com/XTDStudios/DMT
I have a very simple concept of minigame that I want to be playable by up to four players at once. It will be played on an HTML5 canvas, and I already have it working in single-player with three CPU-controlled "players".
The game itself is simple, it just relies on timing and pushing your button as the arrow points towards it. In theory I could just have the game focus on the user's own button, and then just show how everyone did at the end, but what I'd really like to do is for the arrow to move faster the better people are doing - this will essentially allow the difficulty of the game to adjust itself.
So I'm thinking I need to use WebSockets. Only trouble is, I've never used them before and I can't find a single resource that isn't insanely confusing.
By my understanding, I just need the client to send a message to the server when the player pushes their button, and on receipt of that message the server can compute how accurate the player was, and adjust the speed accordingly, passing on that result to the other players.
I'm just worried about what effect lag could have on this setup. What if the arrow reaches the next player before their browser has received the message from the server informing them of how the last player did (and therefore what speed the arrow is going at)?
Am I just being too ambitious with this? Should I just stick with the idea of having four individual games whose total scores are tallied together?
I can see how this might not be a good enough question but I have just embarked on a journey to build the first decent Game Engine for HTML5 canvas that is cross browser and most of all fast. The only problem is I am very new to game design and don't know many tricks of the trade that will help me.
The game I am currently implementing for which the engine will be taken out of is a tile based 2D platformer with MANY tiles (around 3500). I'll start with some tips that I've thus far learnt.
Redraw Regions - only redraw areas that change
Avoid unnecessary function calls (Firefox does not like too many of them)
Use the DOM if you can
Chunk tiles together for quicker access
Other things I am looking for are things like Terrain Generation, Lighting in 2D, Maps, quick server communication. If this is too vague, I will try and close it. Just want to know game design better.
Links/resources would be good. Especially for physics or important maths.
Only draw stuff that's visible, that means only draw the tiles etc. that are currently on the screen. For tiles that's fairly easy, if you got lots of entities you may either want to use a sliding window to keep a list of screen local objects or use such a thing like a quadtree.
Since there's no easy/fast way to copy one canvas to another, redrawing regions is really complicated, since you can't keep a buffered state of (for example) the background if it hasn't changed. So keeping a list of "dirty rectangles" will be a computational overhead for sure.
The whole topic is very broad, even handling the FPS rate can be quite difficult, this question contains some good links and answers on that topic:
https://gamedev.stackexchange.com/questions/1589/fixed-time-step-vs-variable-time-step
You've also mentioned server communication, if you want to do some multiplayer you'll have to care about even more stuff, you can't trust the client, need to worry about bandwidth, synchronization issues, interpolation on the client etc.
I've done some rather simple 2D games in the past, most of them are not in JavaScript but they should give you some hints:
http://github.com/BonsaiDen/Norum
(Platformer engine demo in C, camera zones, moving platforms)
http://github.com/BonsaiDen/Tuff
(2D Platformer in Java, got never finished, powerups and some cool stuff)
http://github.com/BonsaiDen/NodeGame-Shooter
(2D multiplayer space shooter written in JS, using Node.js for the server and WebSockets for communication)
For some final words I'd say that you should start small, like for example just do a scrolling tile map first, then add a player, then rewrite the whole thing. You want write the perfect engine just from scratch it will take many iterations until you find out all the quirks and tricks.
If you want more precise answers you should open questions on the single components you run into troubles with.
I'm working on an application that needs to key out the background from an image taken by a webcam in front of a green screen. I figured this would be a very common task, but to my surprise i'm having trouble finding code samples for anything more advanced than a simple color-threshold and those do not quite cut it quality wise.
I've found a few pdf-papers, but I'm having trouble translating these rather high concepts into actual implementations, I'd much rather look at some code.
Focus here is on quality, having a second or more of processing time is not a problem.
I will be using actionscript 3 (and possibly pixel bender) to implement this, but I'll happily look at other languages aswell.
If you have any good samples doing this, the whole process or parts, please do post them!
If you have no high requirement for chromakey quality, maybe this(http://www.quasimondo.com/archives/000615.php) will be okay.
But for high quality video( i.e, 720*576 video ), it is not easy job.
I have spent a lot time to research high quality chromakey algorithm, finally I have figured out one algorithm(implemented with c/c++) for real time video,which can be used in none-linear video editing system as plugin or standalone application.
I put some demo static images to show at voicethread.com websites, everyone can comment with text, or voice there.
http://voicethread.com/share/801789/