I'm building a webcam recording app in CS5 and I'm having some seemingly random issues with the recorded flv. Currently I'm publishing a stream to Wowza Media Server using the standard _netstream.publish("movieName", "record") command. Everything regarding this works fine and I can play the file back but sometimes there's a 3 to 4 second pause at the beginning or end of the video. There will be a still frame and the time will sit at 0 and then snap to 4. I've explored bandwidth options and I've turned the resolution and quality down considerably and it doesn't seem to have any effect and the rest of the video will play back smoothly. Here are my current camera and mic settings.
_cam.setMode(160, 120, 30, false);
_cam.setQuality(0, 88);
_cam.setKeyFrameInterval(30);
_mic.rate = 11;
I'm also flushing the buffer before closing out the publish stream
_netstream.publish('null');
Could there be something going on with camera initialization/deactivation that causes the lag?
Any help would be greatly appreciated. Let me know if you need more details
I believe this has something to do with the way that the Flash plugin itself initializes and displays the camera.
If you set up a simple test to try setting and unsetting the video stream:
var cam:Camera = Camera.getCamera();
var webcam:Video = new Video(500, 375);
addChild(webcam);
var isPaused:Boolean = false;
function showWebcam():void {
if (!isPaused) {
cam = null;
} else {
cam = Camera.getCamera();
}
webcam.attachCamera(cam);
isPaused = !isPaused;
}
pausingButton.addEventListener(MouseEvent.CLICK, showWebcam);
You'll notice a definite pause as it switches between the two states.
From what I've seen, every time I call attachCamera() with a video object, there is a noticeable pause of the Flash Player itself (including all tweens, interactions, everything) when the method is called, even if the object I'm attaching is null.
Four seconds seems like an excessive lag, but I have noticed that the larger the input/video render and with smoothing = true set on the video object can affect the length of the delay.
As for a solution; I'm not sure if there is one achievable via pure Actionscript, since the delay appears to be down to how the Flash Player itself initializes and renders the live video object.
Related
The sound works for the first few minutes but after a while .play() doesn't play any sound. It seems like if there is a huge duration where no sound has played the sound stops working.
It works fine on Desktop, iOS and the generic android browser. I'm just running into this problem specifically on android devices using mobile chrome as the browser.
This turns out to be a Chrome bug that causes web audio to stop playing sounds if no sound has played for something like 30 seconds.
https://code.google.com/p/chromium/issues/detail?id=518863
The fix seems to be to watch
audioContext.currentTime
and when it gets stuck after 30 secs create a new audioContext.
The solution I ended up using is as follows:
Note I am using the phaser library - so this exact solution wont work for you - but it will give you the general idea
//This is run using a timer event every second
//this.game.time.events.loop(1000, this.checkAudioContext, this);
evil.AudioManager.prototype.checkAudioContext=function(){
//work out when the audio context has stopped
if(this.game.sound.context.currentTime-this.last_context_time===0){
//close out the existing context and create a new one
//you will also need new gain nodes if you are using them
this.game.sound.context.close();
this.game.sound.context=new AudioContext();
this.game.sound.masterGain= this.game.sound.context.createGain();
this.game.sound.masterGain.gain.volume=this.volume;
this.game.sound.masterGain.connect(this.game.sound.context.destination);
//now go through every sound and connect them to the new context
//creating gain nodes as we go.
for(var key in this.tracks){
var snd=this.tracks[key].snd;
snd.context=this.game.sound.context;
snd.masterGainNode = this.game.sound.masterGain;
snd.gainNode=this.game.sound.context.createGain();
snd.gainNode.gain.value = snd.volume * this.volume;
snd.gainNode.connect(snd.masterGainNode);
}
}else{
//update out time variable
this.last_context_time=ctx.currentTime;
}
}
What version of phaser and android do you have? For me it seems to play without issues for a 4-minute song i tried. Even if the screen goes of, it continues without any problem when it is resumed. A dirty solution will be to add an loop that plays a silent sound every 2 minutes for example just to "refresh" the sound manager if that solves your problem.
I am developing an HTML5 game and using Web Audio API for sounds. I am having an issue in which sounds start slowing down as game progress and game also starts feeling jerks which i guess is due to java-script GC doing memory cleanup.There are two types of sound i am playing in the game:
1) Background sound which continuously loop
2) Jump sound, hit sound etc occurs due to some event in the game which occurs very frequently. For example: firing multiple bullets from gun.
Not sure what i am doing wrong, please help. Please refer below code
function play(){
this.startTime = this.actx.currentTime;
this.soundNode = this.actx.createBufferSource();
this.soundNode.buffer = this.buffer;
this.soundNode.connect(this.volumeNode);
//If there's no reverb, bypass the convolverNode
if (this.reverb === false) {
this.volumeNode.connect(this.panNode);
}
//If there is reverb, connect the `convolverNode` and apply
//the impulse response
else {
this.volumeNode.connect(this.convolverNode);
this.convolverNode.connect(this.panNode);
this.convolverNode.buffer = this.reverbImpulse;
}
this.panNode.connect(this.actx.destination);
this.soundNode.loop = this.loop;
this.soundNode.playbackRate.value = this.playbackRate;
this.soundNode.start(
this.startTime,
this.startOffset % this.buffer.duration
);
this.playing = true;
}
There's nothing in your code that stands out as specifically memory intensive, apart from the use of the convolver (which can be really expensive and cause bad performance on lower-end devices). I'd try this though:
Try disabling your audio (don't run any of the audio code, don't just mute it). Do you still have the janks in the game visuals? If so, it's not your audio that's the culprit.
Try running your audio but always run it without the convolver. If the jank disappears, the convolver is your culprit. The only thing I could think of there is to try setting the convolver buffer only once and not every call to play().
Try running different profiles in Chrome Dev Tools (JS, Memory, Paints etc.) and try to figure out where the janks come from. https://developer.chrome.com/devtools/docs/cpu-profiling
Good luck!
A notable issue that's appearing as I'm building a simple audio streaming element in HTML5 is that the <audio> tag doesn't behave as one would expect in regards to playing and pausing a live audio stream.
I'm using the most basic HTML5 code for streaming the audio, an <audio> tag with controls, the source of which is a live stream.
Current outcome: When the stream is first played, it plays whatever is streaming as expected. When it's paused and played again, however, the audio resumes exactly where it left off when the stream was previously paused. The user is now listening to a delayed version of the stream. This occurrence isn't browser-specific.
Desired outcome: When the stream is paused, I want the stream to stop. When it is played again, I want it resume where the stream is currently at, not where it was when the user paused the stream.
Does anyone know of a way to make this audio stream resume properly after it's been paused?
Some failed attempts I've made to fix this issue:
Altering the currentTime of the audio element does nothing to streaming audio.
I've removed the audio element from the DOM when the user stops stream playback and added it back in when user resumes playback. The stream still continues where the user left off and worse yet downloads another copy of the stream behind the scenes.
I've added a random GET variable to the end of the stream URL every time the stream is played in an attempt to fool the browser into believing that it's playing a new stream. Playback still resumes where the user paused the stream.
Best way to stop a stream, and then start it again seems to be removing the source and then calling load:
var sourceElement = document.querySelector("source");
var originalSourceUrl = sourceElement.getAttribute("src");
var audioElement = document.querySelector("audio");
function pause() {
sourceElement.setAttribute("src", "");
audioElement.pause();
// settimeout, otherwise pause event is not raised normally
setTimeout(function () {
audioElement.load(); // This stops the stream from downloading
});
}
function play() {
if (!sourceElement.getAttribute("src")) {
sourceElement.setAttribute("src", originalSourceUrl);
audioElement.load(); // This restarts the stream download
}
audioElement.play();
}
Resetting the audio source and calling the load() method seems to be the simplest solution when you want to stop downloading from the stream.
Since it's a stream, the browser will stop downloading only when the user gets offline. Resetting is necessary to protect your users from burning through their cellular data or to avoid serving outdated content that the browser downloaded when they paused the audio.
Keep in mind though that when the source attribute is set to an empty string, like so audio.src = "", the audio source will instead be set to the page's hostname. If you use a random word, that word will be appended as a path.
So as seen below, setting audio.src ="", means that audio.src === "https://stacksnippets.net/js". Setting audio.src="meow" will make the source be audio.src === "https://stacksnippets.net/js/meow" instead. Thus the 3d paragraph is not visible.
const audio1 = document.getElementById('audio1');
const audio2 = document.getElementById('audio2');
document.getElementById('p1').innerHTML = `First audio source: ${audio1.src}`;
document.getElementById('p2').innerHTML = `Second audio source: ${audio2.src}`;
if (audio1.src === "") {
document.getElementById('p3').innerHTML = "You can see me because the audio source is set to an empty string";
}
<audio id="audio1" src=""></audio>
<audio id="audio2" src="meow"></audio>
<p id="p1"></p>
<p id="p2"></p>
<p id="p3"></p>
Be aware of that behavior if you do rely on the audio's source at a given moment. Using the about URI scheme seems to trick it into behaving in a more reliable way. So using "about:" or "about:about", "about:blank", etc. will work fine.
const resetAudioSource = "about:"
const audio = document.getElementById('audio');
audio.src = resetAudioSource;
document.getElementById('p1').innerHTML = `Audio source: -- "${audio.src}"`;
// Somewhere else in your code...
if (audio.src === resetAudioSource){
document.getElementById('p2').innerHTML = "You can see me because you reset the audio source."
}
<audio id="audio"></audio>
<p id="p1"></p>
<p id="p2"></p>
Resetting the audio.src and calling the .load() method will make the audio to try to load the new source. The above comes in handy if you want to show a spinner component while the audio is loading, but don't want to also show that component when you reset your audio source.
A working example can be found here: https://jsfiddle.net/v2xuczrq/
If the source is reset using a random word, then you might end up with the loader showing up when you also pause the audio, or until the onError event handler catches it. https://jsfiddle.net/jcwvue0s/
UPDATE: The strings "javascript:;" and "javascript:void(0)" can be used instead of the "about:" URI and this seems to work even better as it will also stop the console warnings caused by "about:".
Note: I'm leaving this answer for the sake of posterity, since it was the best solution I or anyone could come up with at the time for my issue. But I've since marked Ciantic's later idea as the best solution because it actually stops the stream downloading and playback like I originally wanted. Consider that solution instead of this one.
One solution I came up with while troubleshooting this issue was to ignore the play and pause functions on the audio element entirely and just set the volume property of the audio element to 0 when user wishes to stop playback and then set the volume property back to 1 when the user wishes to resume playback.
The JavaScript code for such a function would look much like this if you're using jQuery (also demonstrated in this fiddle):
/*
* Play/Stop Live Audio Streams
* "audioElement" should be a jQuery object
*/
function streamPlayStop(audioElement) {
if (audioElement[0].paused) {
audioElement[0].play();
} else if (!audioElement[0].volume) {
audioElement[0].volume = 1;
} else {
audioElement[0].volume = 0;
}
}
I should caution that even though this achieves the desired functionality for stopping and resuming live audio streams, it isn't ideal because the stream, when stopped, is actually still playing and being downloaded in the background, using up bandwidth in the process.
However, this solution doesn't necessarily take up more bandwidth than just using .play() and .pause() on a streaming audio element. Simply using the audio tag with streaming audio uses up a great deal of bandwidth anyway, because once streaming audio is played, it continues to download the contents of the stream in the background when it is paused.
It should be noted that this method won't work in iOS because of purposefully built-in limitations for iPhones and iPads:
On iOS devices, the audio level is always under the user’s physical control. The volume property is not settable in JavaScript. Reading the volume property always returns 1.
If you choose to use the workaround in this answer, you'll need to create a fallback for iOS devices that uses the play() and pause() functions normally, or your interface will be unable to pause the stream.
Tested #Ciantics code and it worked with some modifications, if you want to use multiple sources.
As the source is getting removed, the HTML audio player becomes inactive, so the source (URL) needs to be added directly after again to become active.
Also added an event listener at the end to connect the function when pausing:
var audioElement = document.querySelector("audio");
var sources = document.querySelector("audio").children;
var sourceList = [];
for(i=0;i<sources.length;i++){
sourceList[i] = sources[i].getAttribute("src");
}
function pause() {
for(i=0;i<sources.length;i++){
sources[i].setAttribute("src", "");
}
audioElement.pause();
// settimeout, otherwise pause event is not raised normally
setTimeout(function () {
audioElement.load(); // This stops the stream from downloading
});
for(i=0;i<sources.length;i++){
if (!sources[i].getAttribute("src")) {
sources[i].setAttribute("src", sourceList[i]);
audioElement.load(); // This restarts the stream download
}
}
}
audioElement.addEventListener("pause", pause);
What I need to do is easy: the purpose of this test is show down the speed of my webcam when the camera capture a white pixel so:
1/ I create a Camera
this.cam = Camera.getCamera();
this.velocidad = 24; // I set up the fps in 24
this.cam.setMode(ancho,alto,velocidad);
vid = new Video(640,480);
vid.width = ancho;
vid.height = alto;
vid.attachCamera(cam);
addChild(vid);
2/ So now, when the pixel is recognized I need to change the current speed of the camera to 12 in order to slow down the user speed
I've tried with this code but the camera is frozen and nothing change.. I don't know if I have to delete the current instance of the camera and set up again with the disire fps
cam.setMode(640,480,12);
See Camera.setMode() to request a different frame rate, but note that what's available will depend on the camera.
I think this is not possible with just a config setting, property or method.
The possible solution would be to capture the cam and store its frames as bitmaps in and play (or render) them in sequence.
If this is not a commercial project, you can use this: http://code.google.com/p/flvrecorder/ to record and than load the video to play as you want.
Edit:
Here are some more links, since I can't code something for you now:
Post-processing captured video in AS3, creating slow motion
playing slow motion, fast forward , rewind in a video player in flash video player
Also, you can search Google for "as3 video slow" and it will give you more reference material and some examples.
I'm on an AS3 project, playing a video (H264). I want, for some special reasons, to go to a certain position.
a) I try it with NetStream.seek(). There it only goes to keyframes. In my current setting, this means, i can find a position every 1 second. (for a better resolution, i'd have to encode the movie with as many keyframes as possible, aka every frame a keyframe)
this is definetly not my favourite way, because I don't want to reencode all the vids.
b) I try it with NetStream.step(). This should give me the opportunity to step slowly from frame to frame. But in the documentation it says:
This method is available only when data is streaming from Flash Media Server 3.5.3 or higher and when NetStream.inBufferSeek is true.
http://help.adobe.com/en_US/FlashPlatform/reference/actionscript/3/flash/net/NetStream.html#step()
Does this mean, it is not possible with Air for Desktop? When I try it, nothing works.
Any suggestions, how to solve this problem?
Greetings & Thank you!
Nicolas
Flash video can only be advanced by seconds unless you have Flash Media Server hosting your video. Technically, that means that you can have it working as intended in Air, however, the video would have to be streaming (silly adobe...).
You have two options:
1) Import the footage as a movieclip. The Flash IDE has a wizard for this, and if you're developing exclusively in non-FlashIDE environment, you can convert and export as an external asset such as an SWF or SWC. This would then be embedded or runtime loaded into your app giving you access to the per-frame steppable methods of MovieClip. This, however, does come with some audio syncing issues (iirc). Also, scrubbing backwards is not an MC's forté.
2) Write your own video object that loads an image sequence and displays each frame in order. You'd have to setup your own audio syncing abilities, but it might be the most direct solution apart from FLVComponent or NetStream.
I've noticed that flash player 9 scrubs nice and smooth but in players 10+ I get this no scrub problem.
My fix, was to limit frequency the calls to the seek function to <= 200ms. This fixed scrubbing but is much less smooth as player 9. Perhaps because of the "Flash video can only be advanced by seconds" limitation? I used a timer to tigger the function that calls seek() for the video.
private var scrubInterval:Timer = new Timer(200);
private function videoScrubberTouch():void {
_ns.pause();
var bounds:Rectangle = new Rectangle(0,0,340,0);
scrubInterval.addEventListener(TimerEvent.TIMER, scrubTimeline);
scrubInterval.start();
videoThumb.startDrag(false, bounds);
}
private function scrubTimeline(e:TimerEvent):void {
var amt:Number = Math.floor((videoThumb.x / 340) * duration);
trace("SCRUB duration: "+duration+" videoThumb.x: "+videoThumb.x+" amt "+amt);
_ns.seek(amt);
}
Please check this Demo link (or get the SWF file to test outside of browser via desktop Flash Player).
Note: Demo requires FLV with H.264 video codec and AAC or MP3 audio codec.
The source code for that is here: Github link
In the above demo there is (bytes-based) seeking and frame by frame stepping. The functions you want to study mainly are:
Append_SEEK ( position amount ) - This will got to the specified position in bytes and search for the nearest available keyframe.
get_frame_TAG - This will extract a tag holding one frame of data. Audio can be in frames too but lets assume you have video-only. That function is your opportunity to adjust timestamps. When it's run it will also append the tag (so each "get_frame_TAG" is also a "frame step").
For example : You have a 25fps video, you want the third-frame at 4 seconds into playback...
1000 milisecs / 25 fps = 40 units for each timestamp. So 4000 ms == 4 secs + add the 40 x 3rd frame == an expected timestamp of 4120.
So getting that frame means... First find a keyframe. Then step through each frame checking the timestamps that represent a frame you want. If it isnt then change it to the same as most recent keyframe timestamp (this forces Flash to fast-forward through the frames to keep things in sync as it assumes the frame [with smaller than expected timestamp] should have been played by that time). You can "hide" the video object during this process if you don't like the look of fast-forwarding.