I am porting a vehicle in-dash display unit app over to the browser. The big goal is to get it running completely within Mobile Safari.
It's an HTML5/JS music app that relies heavily on jQuery.load to move around between different "screens" by loading in page fragments.
The issue is that when a user selects a track to play, they are taken to the "now playing" screen, which should start playback of the track. The track is not playing though once this screen is reached, and the user instead needs to explicitly click play from this screen in order for audio playback to start. Once this has happened for the first time, autoplay works for the duration of the app.
I understand that Mobile Safari intentionally put blockers in place to prevent audio from autoplaying so that web apps are kept from consuming data unless in direct response to user input.
The thing is, my audio playback IS in direct response to user input...sort of. However, a bunch of things need to happen before my playback actually starts, and those things happen within the now playing page (calling API to get URL for track to be played, report user is playing track, get track metadata, yatta yatta yatta).
To try and get around this, I have the app preload a silent .1s mp3 file into the audio element on startup. Then in direct response to a click event to transition to now playing screen, I call .play() on the audio element.
I assumed that having a call to .play in direct response to user input would the subsequent call to .play() within the now playing page to work, since this is the behavior I had previously observed.
Only, it didn't seem to make any difference.
Any ideas on how I can adjust my flow in order for audio playback to start after loading the now playing screen?
EDIT:
Added some code snippets below
Vehicle.audio = {
init: function () {
audioElement.setAttribute("src", "/audioinit.mp3");
audioElement.play();
},
play: function (source) {
log("VEHICLE: playing audio from source " + source);
Vehicle.audio.stop();
audioElement.setAttribute("src", source);
audioElement.play();
},
In response to user selecting a track:
$(document).on("click", "a.play-track", function () {
Vehicle.audio.init();
/* my wrapper function for jQuery.load() */
replace_wrap("nowplaying.html");
});
Then on the now playing page, Vehicle.audio.play() is called.
The thing is, my audio playback IS in direct response to user input...sort of. However, a bunch of things need to happen before my playback actually starts, and those things happen within the now playing page (calling API to get URL for track to be played, report user is playing track, get track metadata, yatta yatta yatta).
Try adding a "touchstart" event listener to the entire "now playing page" which will then synchronously call .play() on the audio element:
$(document).one("touchstart", function () {
Vehicle.audio.init()
})
This way as soon as an iOS user touches anywhere on the now playing page, the audio should begin loading/playing.
AFAIK it is a restriction placed intentionally by iOS. User must interract with the device (touch/click event) before the script is allowed to start playback.
Related
Is it possible to play video online that's made of two or more video files?
Since my original post wasn't clear enough, here's expanded explanation and question.
My site is hosted on Linux/Apache/PHP server. I have video files in FLV/F4V format. I can also convert them to other available formats if necessary. All videos have same aspect ratio and other parameters.
What I want is to build (or use if exist) online video player that plays video composed of multiple video files concatenated together in real-time, i.e. when user clicks to see a video.
For example, visitor comes to my site and sees video titled "Welcome" available to play. When he/she clicks to play that video, I take video files "Opening.f4v", "Welcome.f4v" and "Ending.f4v" and join/merge/concatenate them one after another to create one continuous video on the fly.
Resulting video looks like one video, with no visual clues, lags or even smallest observable delay between video parts. Basically what is done is some form of on-the-fly editing or pre-editing, and user sees the result. This resulting video is not saved on the server, it's just composed and played that way real-time.
Also, if possible, user shouldn't be made to wait for this merging to be over before he/she sees resulting video, but to be able to get first part of the video playing immediately, while merging is done simultaneously.
Is this possible with flash/actionscript, ffmpeg, html5 or some other online technology? I don't need explanation how it's possible, just a nod that it's possible and some links to further investigate.
Also, if one option is to use flash, what are alternatives for making this work when site is visited from iphone/ipad?
An approach that will work on some browsers currently, and on most browsers going forwards is to use the HTML5 Video Media Source Extension mechanism.
This essentially allows you replace a static 'src' file for a video in your HTML5 page, with a dynamic src buffer which you can then fill any way you want using your own Javascript code.
So you can write code to pre-buffer the second video when you get towards the end of the first video and then immediately start adding packets from the second video to the src right after the last packet for the first video.
In very high level terms this looks like:
Your HTML to insert the video where you want it in your page:
.
.
.
<div>
<video id="yourVideo1" controls="" autoplay="" width="320" height="240"></video>
</div>
.
.
.
Your Javascript to provide the source for your video:
//Get the video element
var videoElement = document.getElementById('yourVideo1');
//Create a 'MediaSource' and associate it with this video
var mediaSource = new MediaSource();
video.src = window.URL.createObjectURL(mediaSource);
//Add a listener to the MediaSource object to check for
//the video been opened. In this function you can add your
//code to get the get your videos from the servers and add
//'chunks' to the media source buffer
mediaSource.addEventListener('sourceopen', function(e) {
//Set the format of the source video
var mediaSourceBuffer = mediaSource.addSourceBuffer('video/webm; codecs="vorbis,vp8"');
//Get your video from the web
while (not the end of your video playlist) {
...
//Stream video from server
...
//Add packets received to the media source bufer
mediaSourceBuffer.appendBuffer(receivedVideoPackets);
//If near end of video start fetching next video to
//avoid buffering delay
...
//If end of video go to next video in playlist
...
}
}, false);
Look at the HTML5 Rocks demo below to see this in action (for a slightly different usecase).
Given how tricky video manipulation is and the multitude of formats etc, it would be much easier for you if one of the established video players provided the functionality out of the box so I would still try their forums as mentioned in the comment, but at least you know it is technically possible.
The MSE spec is available here:
http://w3c.github.io/media-source/
And a good intro blog and demo is here (make sure your browser supports MSE - latest version of Chrome does):
http://updates.html5rocks.com/2011/11/Stream-video-using-the-MediaSource-API
You can find latest browser support here:
http://www.jwplayer.com/html5/mediasource/
Use two video elements, first visible and second display:none for buffering.
After first video playing, check currentPosition and currentDuration,
specify second video preloader prepare visible video player's buffer data.
First video element's SRC could be specified anytime for seamless playing.
I have web application(similar to Karaoke) where user can record his voice over instrumental.
After recording user plays back recording. Here I play instrumental in <audio> tag and voice using web audio api. To sync both audios on play/pause I calculate time like this
pausedAt = Date.now() - startedAt;
startedAt = Date.now() - pausedAt;
This works fine. Issue is when user uses slider on audio tag to move forward/backward. I am thinking of solution like this
Use ontimeupdate event, stop the voice and then use startAt(currentTime) where currentTime is of instrumental playing in audio tag.
Since there is no seekTo function in api, I have to stop and then start audio. Is there any better solution for this?
Second issue I face is seeking on audio tag is not smooth. If I arbitrarily clicks on progress bar sometimes it doesn't work. When I saw network tab in developer tool window I saw something like shown in image. It sends out some 600 requests and some 86 MB data downloaded whereas file size is less than 10 MB.
You really should use the Web Audio API to do this. will never give you precise synchronization, and it will rely on streaming to seek - which is going to result in extra downloading, as you saw. Just load the song via XHR and decodeAudioData, and provide your own playback controls.
I'm trying to get HTML5 audio sprites working on a Nexus 7 using the Android Chrome browser. To change the audio being played, I change the source of the audio player, then reload the audio player.
I then have to detect two 'timeupdate' events to make sure the audio is actually playing, because HTML5 audio on Android can be a bit unreliable.
It's one part of a large page so I can't post it all, but the setup boils down to basically:
<body>
<audio id="myAudioPlayer" src="audio_1.wav">
<script type=javascript>
function changeAudio() {
var audioplayer = document.getElementByID("myAudioPlayer");
audioplayer.currentSrc = getNextAudioClip();
audioplayer.load();
audioplayer.play();
audioplayer.addEventListener('timeupdate', firstTimeUpdate, false);
}
function firstTimeUpdate() {
this.removeEventListener('timeupdate', firstTimeUpdate, false);
this.addEventListener('timeupdate', secondTimeUpdate, false);
this.play();
}
function secondTimeUpdate() {
this.removeEventListener('timeupdate', secondTimeUpdate, false);
this.addEventListener('timeupdate', timeSliderUpdate, false);
}
</script>
</body>
So, this works fine the first few times. The audio is changed, 'timeupdate' fires to get us to firstTimeUpdate(), then the audio starts playing and 'timeupdate' fires again to get to secondTimeUpdate().
But the 6th or 7th time I change the audio, the second 'timeupdate' event doesn't fire, so secondTimeUpdate() is never reached, unless I rotate the Nexus 7 90 degrees and then rotate it back again, whereupon the audio starts playing, the second 'timeupdate' is fired, and everything begins to work again.
This continues until I change audio for the 12th or 13th time, where the same thing is observed. The getNextAudioClip() function is working fine (i.e. it's returning a valid audio file).
I'm stumped by this. Can anyone hazard a guess as to what could be going on here, or why rotating the Nexus 7 "fixes" it?
EDIT: Ok, so after having a bit more time to play with this, it seems that the issue is related to garbage collection. Rotating the Nexus 7 seems to trigger garbage collection on the previously-loaded audio sprites, which allows the new sprite to load and play. Changing to a different tab and then back again also does this. So if I could just get this audio GC'd automatically somehow, I could have a workaround.
Before I get flamed to death, I know this doesn't work currently due to Apple's concern over downloading an audio file automatically.
However, my question is: Has anyone found a cunning workaround yet? I just want to play a start up sound on the launch of a game and currently have to wait for the user to click somewhere before I can play the audio. One of you clever chaps must have got this working by now?
There is no chance to get autoplay working in mobile browsers. Android and iOS doesn't allow it and personally I think that is a feasible confinement! Imagine every second website you will open plays and ugly sound at start!
But you can make a little hack, so that the user will not remark that he currently started the audio for your application:
You WILL need an user interaction to start your audio. So, your app or game maybe has a startscreen or a welcome button which needs a click to get to mainmenu or start the game. Bind to an user event (the accepted events are: "click", "touchend", "doubleclick" and "keydown") and call the load() method for your <audio>.
Bind to the "canplaythrough" event of the <audio>. This event is triggered when your source is ready to play. Here you can now call play(), pause() or wait for other interactions. So the audio is ready but now you have full controll when to start or stop the sound.
I also advise you to use audio sprites on mobile clients. iOS (and Android?) internally implemented audio support through a Singleton. That means that you cannot, like in desktop browser, have 10 audio elements and play differents sound at once. You can only play one file!
So changing the source for different sounds takes to much time. With an audio sprite you can start your sprites when the user first interact with your website or game. Pause your sprite and when you need to play sound you have to set the currentTime to the beginning of the sprite and pause the sprite when currentTime of your file reaches the end of your sprite. There is an timeupdate Event where your can check the currentTime of your sprite.
If you are more interested I can prepare my javascript audio sprites player for you!!
Only solution I have seen so far is to create a shell app and put the web app inside a UIWebView.
http://flowz.com/2011/03/apple-disabled-audiovideo-playback-in-html5/
UIWebView.allowsInlineMediaPlayback = YES;
UIWebView.mediaPlaybackRequiresUserAction = NO;
I also would really like to know how to do this in a plain-old web app.
I believe I just solved this for my own app.
The steps,
Controller loads up,
Then.... in viewDidLoad
have your web view load the HTML : loadHTMLString:htmlFile baseURL:[self getBasePath]];
THEN... set mediaPlaybackRequiresUserAction = NO on the webview.
If you set it too soon, I think the load for the html resets it.
I have a bot chat app that has voice messages and I needed them to be autoplayed whenever needed ... so here is what worked for me in my angular app :
just attach a click eventlistener to document and call player.load(), after that whenever you set player.src = x; it will autoplay.
<audio id="voicePlayer" controls autoplay playsinline #voicePlayer></audio>
#ViewChild('voicePlayer', { read: ViewContainerRef }) voicePlayerRef: ViewContainerRef;
...
ngAfterContentInit(): void {
this.voicePlayer = this.voicePlayerRef.element.nativeElement;
document.addEventListener('click', this._onDocumentClick.bind(this));
}
_onDocumentClick() {
this.voicePlayer.load();
document.removeEventListener('click', this._onDocumentClick);
}
...
this.voicePlayer.src = 'x';
HowlerJS creates is a workaround. The user doesn't need to allow autoplay for the audio to be played automatically
Explanation from Docs on how this workaround is created :
howler.js is an audio library for the modern web. It defaults to Web Audio API and falls back to HTML5 Audio. This makes working with audio in JavaScript easy and reliable across all platforms.
I've made a custom video player in as3. Everything works, but once a video has loaded and played through completely, if the user replays the video it continuously fires the NetStream.Buffer.Flush event until the video stops playing again (by either ending or by the user hitting pause).
It doesn't seem to effect anything negatively, but I would rather not have an event continuously firing in the background like that.
I can't find any documentation of this happening online anywhere. Is this normal behavior for a NetStream that has fully loaded?
I believe I've fixed it, though I'm still not sure why it was repeatedly firing the flush event.
My custom videostream class was set to seek(0) once the video completed to rewind the current flv. It seems this was what was causing the issue. Instead of using seek(0), I just reset the source back to the original url, which reloads the video completely. Since the video is cached at that point, it reloads immediately so it's not an issue.