I'm trying to get HTML5 audio sprites working on a Nexus 7 using the Android Chrome browser. To change the audio being played, I change the source of the audio player, then reload the audio player.
I then have to detect two 'timeupdate' events to make sure the audio is actually playing, because HTML5 audio on Android can be a bit unreliable.
It's one part of a large page so I can't post it all, but the setup boils down to basically:
<body>
<audio id="myAudioPlayer" src="audio_1.wav">
<script type=javascript>
function changeAudio() {
var audioplayer = document.getElementByID("myAudioPlayer");
audioplayer.currentSrc = getNextAudioClip();
audioplayer.load();
audioplayer.play();
audioplayer.addEventListener('timeupdate', firstTimeUpdate, false);
}
function firstTimeUpdate() {
this.removeEventListener('timeupdate', firstTimeUpdate, false);
this.addEventListener('timeupdate', secondTimeUpdate, false);
this.play();
}
function secondTimeUpdate() {
this.removeEventListener('timeupdate', secondTimeUpdate, false);
this.addEventListener('timeupdate', timeSliderUpdate, false);
}
</script>
</body>
So, this works fine the first few times. The audio is changed, 'timeupdate' fires to get us to firstTimeUpdate(), then the audio starts playing and 'timeupdate' fires again to get to secondTimeUpdate().
But the 6th or 7th time I change the audio, the second 'timeupdate' event doesn't fire, so secondTimeUpdate() is never reached, unless I rotate the Nexus 7 90 degrees and then rotate it back again, whereupon the audio starts playing, the second 'timeupdate' is fired, and everything begins to work again.
This continues until I change audio for the 12th or 13th time, where the same thing is observed. The getNextAudioClip() function is working fine (i.e. it's returning a valid audio file).
I'm stumped by this. Can anyone hazard a guess as to what could be going on here, or why rotating the Nexus 7 "fixes" it?
EDIT: Ok, so after having a bit more time to play with this, it seems that the issue is related to garbage collection. Rotating the Nexus 7 seems to trigger garbage collection on the previously-loaded audio sprites, which allows the new sprite to load and play. Changing to a different tab and then back again also does this. So if I could just get this audio GC'd automatically somehow, I could have a workaround.
Related
I'm using JW Player to live stream content onto a web page. The player is backed by an open-source library called cine.io.
My issue is that the player falls back to an HTML 5 video element for all mobile web, both on iPhone and Android. There are some differences between the flash solution of JW Player and HTML5 - notably that if a live stream starts, then stops, then restarts, the video element will not pick up the restarted stream.
This is a problem since streams often drop in and out - and the flash solution does pick up the restarted stream.
I tested a bunch of listener methods on the video and the only one that signalled that the stream had ended was a "time update" listener:
$video.on('timeupdate', function(){
//Do something
});
However none of my attempts to re-open the stream have been effective.
Is this even possible? Can anyone provide pointers?
Would an example like this work?
http://support.jwplayer.com/customer/portal/articles/1442607-example-a-custom-error-message
I have a video embedded on a website and I'm getting some odd behavior in Firefox.
Every time I change the playback position by clicking on the time slider, the pause event is fired. Similarly, every time the video ends the pause event is fired, even before the ended event.
I was first wondering if I had an error in some JS I had written (to make the video bigger on playback, and then shrink it on pause - code here). However, I tested with just this simple code and the behavior still exists -
$(document).ready(function(){
video = $('#intro-video');
video.on('pause', function(){
alert('pause');
});
});
This doesn't happen in other browsers, and it is very undesirable behavior for Firefox. Is there a way to prevent this from happening? Thanks,
Note
If anyone is interested, here is the link to Pastebin containing my code, but after testing with the above I'm pretty sure this is not the issue.
Update
I have found this thread which suggests the behavior described about is actually by design, but I find that hard to believe. The author says that IE, Opera and Safari get it wrong while Firefox gets it right, but surely that's not correct...
I have been working on monitoring events for HTML5 video/audio and from time to time I find something that you described: it is not consistent across browsers and given the same browser it can vary between versions. I would suggest you stick to the specs in this case:
ended event: it should fire a pause event right before the ended event (given you have no current media controller ie multiple sync media players). Here Firefox does it as expected.
seeking: it should not fire a pause event. There is no mention of that in the specs. Here Firefox seems to drop the ball. However if the video is paused in FF and you seek then the pause event is not fired.
Amongst IE 11, FF 28 and Chrome 34 I found that only Chrome does fit the specs to the letter for this question on both events.
You could consider asking Mozilla why is such a behavior implemented?
If you want to bypass this behavior (do not know how to call it) you need to build you own custom controls in JS. The code below illustrates that the pause event does not fire if you programmatically seek to a point in time of the video (the #seekBar div):
<video id="intro-video" width="640" height="360" controls>
<source src="../media/360p.mp4" type='video/mp4' />
</video>
<div id="seekBar">Seek +5sec</div>
<script type="text/javascript">
$(document).ready(function(){
video = $('#intro-video');
videoJ = $('#intro-video')[0];
video.on('pause', function(){
console.log('pause');
});
video.on('play', function(){
console.log('play');
});
video.on('ended', function(event){
console.log('ended');
});
video.on('seeking', function(){
console.log('seeking');
});
video.on('seeked', function(){
console.log('seeked');
});
$('#seekBar').on('click',function(){
var currentT = videoJ.currentTime;
videoJ.currentTime = currentT + 5;//on click seek forward + 5 sec
});
});
</script>
You can have a look here to start building your own controls.
Maybe a lib like videojs can fix it for you (not tested on this chain of events)
I was able to work around this issue on Firefox by adding a timeout and checking if the video is paused inside the timeout. Only when the video.paused returns true, I execute my code:
const showOverlay = video => video.closest('.jsvideo').classList.remove('is--playing');
$$('.js-video video', video => {
video.addEventListener('pause', () => {
if (video.readyState !== 4) {
return true;
}
// Firefox dispatches the pause event but
// doesn't set the video to paused
setTimeout(() => {
if (video.paused) {
showOverlay();
}
}, 250);
});
video.addEventListener('ended', showOverlay);
});
The $$ functions is just a helper function which queries the DOM.
I used a 250ms timeout but I think it can be significantly reduced (even to zero) because the check will be executed after the pause event is triggered.
The actual UX is left untouched because the timeout is unnoticeable and doesn't affect the speed of the UI updates.
Before I get flamed to death, I know this doesn't work currently due to Apple's concern over downloading an audio file automatically.
However, my question is: Has anyone found a cunning workaround yet? I just want to play a start up sound on the launch of a game and currently have to wait for the user to click somewhere before I can play the audio. One of you clever chaps must have got this working by now?
There is no chance to get autoplay working in mobile browsers. Android and iOS doesn't allow it and personally I think that is a feasible confinement! Imagine every second website you will open plays and ugly sound at start!
But you can make a little hack, so that the user will not remark that he currently started the audio for your application:
You WILL need an user interaction to start your audio. So, your app or game maybe has a startscreen or a welcome button which needs a click to get to mainmenu or start the game. Bind to an user event (the accepted events are: "click", "touchend", "doubleclick" and "keydown") and call the load() method for your <audio>.
Bind to the "canplaythrough" event of the <audio>. This event is triggered when your source is ready to play. Here you can now call play(), pause() or wait for other interactions. So the audio is ready but now you have full controll when to start or stop the sound.
I also advise you to use audio sprites on mobile clients. iOS (and Android?) internally implemented audio support through a Singleton. That means that you cannot, like in desktop browser, have 10 audio elements and play differents sound at once. You can only play one file!
So changing the source for different sounds takes to much time. With an audio sprite you can start your sprites when the user first interact with your website or game. Pause your sprite and when you need to play sound you have to set the currentTime to the beginning of the sprite and pause the sprite when currentTime of your file reaches the end of your sprite. There is an timeupdate Event where your can check the currentTime of your sprite.
If you are more interested I can prepare my javascript audio sprites player for you!!
Only solution I have seen so far is to create a shell app and put the web app inside a UIWebView.
http://flowz.com/2011/03/apple-disabled-audiovideo-playback-in-html5/
UIWebView.allowsInlineMediaPlayback = YES;
UIWebView.mediaPlaybackRequiresUserAction = NO;
I also would really like to know how to do this in a plain-old web app.
I believe I just solved this for my own app.
The steps,
Controller loads up,
Then.... in viewDidLoad
have your web view load the HTML : loadHTMLString:htmlFile baseURL:[self getBasePath]];
THEN... set mediaPlaybackRequiresUserAction = NO on the webview.
If you set it too soon, I think the load for the html resets it.
I have a bot chat app that has voice messages and I needed them to be autoplayed whenever needed ... so here is what worked for me in my angular app :
just attach a click eventlistener to document and call player.load(), after that whenever you set player.src = x; it will autoplay.
<audio id="voicePlayer" controls autoplay playsinline #voicePlayer></audio>
#ViewChild('voicePlayer', { read: ViewContainerRef }) voicePlayerRef: ViewContainerRef;
...
ngAfterContentInit(): void {
this.voicePlayer = this.voicePlayerRef.element.nativeElement;
document.addEventListener('click', this._onDocumentClick.bind(this));
}
_onDocumentClick() {
this.voicePlayer.load();
document.removeEventListener('click', this._onDocumentClick);
}
...
this.voicePlayer.src = 'x';
HowlerJS creates is a workaround. The user doesn't need to allow autoplay for the audio to be played automatically
Explanation from Docs on how this workaround is created :
howler.js is an audio library for the modern web. It defaults to Web Audio API and falls back to HTML5 Audio. This makes working with audio in JavaScript easy and reliable across all platforms.
I am porting a vehicle in-dash display unit app over to the browser. The big goal is to get it running completely within Mobile Safari.
It's an HTML5/JS music app that relies heavily on jQuery.load to move around between different "screens" by loading in page fragments.
The issue is that when a user selects a track to play, they are taken to the "now playing" screen, which should start playback of the track. The track is not playing though once this screen is reached, and the user instead needs to explicitly click play from this screen in order for audio playback to start. Once this has happened for the first time, autoplay works for the duration of the app.
I understand that Mobile Safari intentionally put blockers in place to prevent audio from autoplaying so that web apps are kept from consuming data unless in direct response to user input.
The thing is, my audio playback IS in direct response to user input...sort of. However, a bunch of things need to happen before my playback actually starts, and those things happen within the now playing page (calling API to get URL for track to be played, report user is playing track, get track metadata, yatta yatta yatta).
To try and get around this, I have the app preload a silent .1s mp3 file into the audio element on startup. Then in direct response to a click event to transition to now playing screen, I call .play() on the audio element.
I assumed that having a call to .play in direct response to user input would the subsequent call to .play() within the now playing page to work, since this is the behavior I had previously observed.
Only, it didn't seem to make any difference.
Any ideas on how I can adjust my flow in order for audio playback to start after loading the now playing screen?
EDIT:
Added some code snippets below
Vehicle.audio = {
init: function () {
audioElement.setAttribute("src", "/audioinit.mp3");
audioElement.play();
},
play: function (source) {
log("VEHICLE: playing audio from source " + source);
Vehicle.audio.stop();
audioElement.setAttribute("src", source);
audioElement.play();
},
In response to user selecting a track:
$(document).on("click", "a.play-track", function () {
Vehicle.audio.init();
/* my wrapper function for jQuery.load() */
replace_wrap("nowplaying.html");
});
Then on the now playing page, Vehicle.audio.play() is called.
The thing is, my audio playback IS in direct response to user input...sort of. However, a bunch of things need to happen before my playback actually starts, and those things happen within the now playing page (calling API to get URL for track to be played, report user is playing track, get track metadata, yatta yatta yatta).
Try adding a "touchstart" event listener to the entire "now playing page" which will then synchronously call .play() on the audio element:
$(document).one("touchstart", function () {
Vehicle.audio.init()
})
This way as soon as an iOS user touches anywhere on the now playing page, the audio should begin loading/playing.
AFAIK it is a restriction placed intentionally by iOS. User must interract with the device (touch/click event) before the script is allowed to start playback.
Check out this space shooter demo.
The HTML5 audio is perfect on Chrome 18 and Firefox 10. There is no lag in playing sounds and each sample plays perfectly. The last time I tried to play sounds using HTML5 audio and JavaScript I couldn't get a sound to play more than once.
What sorcery is Scirra doing to make this so perfect?
I'm the developer of Construct 2, so I hope I'm sufficiently qualified to answer your question :)
HTML5 audio is indeed a mess, so I've gone to considerable lengths to try and make it bulletproof in Construct 2. Here's an outline of what I've done:
Use the Web Audio API
HTML5 audio appears designed for streaming music, so a HTML5 Audio object is kind of a heavyweight object. Playing 10 sounds a second with it like Space Blaster does can easily seize up the browser. On the other hand, the Web Audio API is a high-performance audio engine with routing, effects, and lightweight sound playback. It's perfect for games. Audio buffers and audio playback are separated, so you can have one data buffer and efficiently play it many times simultaneously, whereas some browsers are so buggy if you play a HTML5 sound a few times it re-downloads it each time! Since it was actually designed for games and such, you can happily play back tonnes of sound for ages and it will still hum along nicely. It can also use HTML5 audio as a sound source, although I only use HTML5 audio for things the user has designated as music tracks (since that's where you'd prefer to have streaming - typically everything else in the Web Audio API is fully downloaded before playing).
The Web Audio API is supported in Chrome, has also made it in to iOS 6+ (although it's muted until you try to play some audio in a touch event), Firefox are working on support, and it should be coming soon to Chrome for Android. So on these platforms audio will be significantly more reliable.
More info on HTML5Rocks and the proposed spec - you'll have to use the spec as the documentation for now, there's not much else out there.
Other browsers: implement an audio recycling system
The Web Audio API isn't yet supported everywhere, notably IE, which means you still need to crowbar HTML5 audio in to something that might work for games for backwards compatibility. The way to do this is to recycle audio objects.
The player's laser in Space Blaster fires 10 times a second - and that's not including any other sound effects! As I mentioned earlier, Audio is kind of a heavyweight object, so if you're doing new Audio() 10+ times a second, lo and behold, the browser eventually dies and audio starts glitching up. However, you can drastically reduce the number of Audio objects created by recycling them.
Basically, for each sound effect, keep a cache of every Audio object you've created with that sound as a source. Then, when playing a new sound, search the cache for any sound effects which have finished playing (the ended property will be true). If you find one, rewind it back to the beginning (currentTime = 0) and play() again. Otherwise, create a new Audio() object in the cache.
Since the player's laser sound effect is short, instead of creating 600 Audio objects a minute, there will just be 3 or 4 that it keeps cycling round. Some browsers unfortunately will still download it 4 times (Safari did this last I checked!) or have high latency the first time each sound buffer is played, but eventually the browser catches up since the same buffers are always being reused. So basically sound might be a bit weird for a few moments, then it clears up. We also use the HTML5 app cache so next time you play everything loads from disk, so subsequent plays should perform well immediately.
That's basically it. It's still a little dodgy on the first play with HTML5 audio, but every time after that should be fairly solid providing the browser has a sane audio implementation. There are a number of ways to try to clone Audio objects, but I've found that rewinding existing Audios works best.
There's no SoundManager or any Flash/plugin-based fallbacks at all since we make a point of being pure HTML5.
We also support audio APIs provided by PhoneGap and appMobi for mobile, since HTML5 audio on mobile isn't even worth trying. That makes a total of four audio APIs our audio engine wraps, and yes, it does look like a frankenstein mess, but it works.
That's it. I suppose our competitors will read this, but who cares when there's SO rep to be had???!!!1111