How can I stop and resume a live audio stream in HTML5 instead of just pausing it? - html

A notable issue that's appearing as I'm building a simple audio streaming element in HTML5 is that the <audio> tag doesn't behave as one would expect in regards to playing and pausing a live audio stream.
I'm using the most basic HTML5 code for streaming the audio, an <audio> tag with controls, the source of which is a live stream.
Current outcome: When the stream is first played, it plays whatever is streaming as expected. When it's paused and played again, however, the audio resumes exactly where it left off when the stream was previously paused. The user is now listening to a delayed version of the stream. This occurrence isn't browser-specific.
Desired outcome: When the stream is paused, I want the stream to stop. When it is played again, I want it resume where the stream is currently at, not where it was when the user paused the stream.
Does anyone know of a way to make this audio stream resume properly after it's been paused?
Some failed attempts I've made to fix this issue:
Altering the currentTime of the audio element does nothing to streaming audio.
I've removed the audio element from the DOM when the user stops stream playback and added it back in when user resumes playback. The stream still continues where the user left off and worse yet downloads another copy of the stream behind the scenes.
I've added a random GET variable to the end of the stream URL every time the stream is played in an attempt to fool the browser into believing that it's playing a new stream. Playback still resumes where the user paused the stream.

Best way to stop a stream, and then start it again seems to be removing the source and then calling load:
var sourceElement = document.querySelector("source");
var originalSourceUrl = sourceElement.getAttribute("src");
var audioElement = document.querySelector("audio");
function pause() {
sourceElement.setAttribute("src", "");
audioElement.pause();
// settimeout, otherwise pause event is not raised normally
setTimeout(function () {
audioElement.load(); // This stops the stream from downloading
});
}
function play() {
if (!sourceElement.getAttribute("src")) {
sourceElement.setAttribute("src", originalSourceUrl);
audioElement.load(); // This restarts the stream download
}
audioElement.play();
}

Resetting the audio source and calling the load() method seems to be the simplest solution when you want to stop downloading from the stream.
Since it's a stream, the browser will stop downloading only when the user gets offline. Resetting is necessary to protect your users from burning through their cellular data or to avoid serving outdated content that the browser downloaded when they paused the audio.
Keep in mind though that when the source attribute is set to an empty string, like so audio.src = "", the audio source will instead be set to the page's hostname. If you use a random word, that word will be appended as a path.
So as seen below, setting audio.src ="", means that audio.src === "https://stacksnippets.net/js". Setting audio.src="meow" will make the source be audio.src === "https://stacksnippets.net/js/meow" instead. Thus the 3d paragraph is not visible.
const audio1 = document.getElementById('audio1');
const audio2 = document.getElementById('audio2');
document.getElementById('p1').innerHTML = `First audio source: ${audio1.src}`;
document.getElementById('p2').innerHTML = `Second audio source: ${audio2.src}`;
if (audio1.src === "") {
document.getElementById('p3').innerHTML = "You can see me because the audio source is set to an empty string";
}
<audio id="audio1" src=""></audio>
<audio id="audio2" src="meow"></audio>
<p id="p1"></p>
<p id="p2"></p>
<p id="p3"></p>
Be aware of that behavior if you do rely on the audio's source at a given moment. Using the about URI scheme seems to trick it into behaving in a more reliable way. So using "about:" or "about:about", "about:blank", etc. will work fine.
const resetAudioSource = "about:"
const audio = document.getElementById('audio');
audio.src = resetAudioSource;
document.getElementById('p1').innerHTML = `Audio source: -- "${audio.src}"`;
// Somewhere else in your code...
if (audio.src === resetAudioSource){
document.getElementById('p2').innerHTML = "You can see me because you reset the audio source."
}
<audio id="audio"></audio>
<p id="p1"></p>
<p id="p2"></p>
Resetting the audio.src and calling the .load() method will make the audio to try to load the new source. The above comes in handy if you want to show a spinner component while the audio is loading, but don't want to also show that component when you reset your audio source.
A working example can be found here: https://jsfiddle.net/v2xuczrq/
If the source is reset using a random word, then you might end up with the loader showing up when you also pause the audio, or until the onError event handler catches it. https://jsfiddle.net/jcwvue0s/
UPDATE: The strings "javascript:;" and "javascript:void(0)" can be used instead of the "about:" URI and this seems to work even better as it will also stop the console warnings caused by "about:".

Note: I'm leaving this answer for the sake of posterity, since it was the best solution I or anyone could come up with at the time for my issue. But I've since marked Ciantic's later idea as the best solution because it actually stops the stream downloading and playback like I originally wanted. Consider that solution instead of this one.
One solution I came up with while troubleshooting this issue was to ignore the play and pause functions on the audio element entirely and just set the volume property of the audio element to 0 when user wishes to stop playback and then set the volume property back to 1 when the user wishes to resume playback.
The JavaScript code for such a function would look much like this if you're using jQuery (also demonstrated in this fiddle):
/*
* Play/Stop Live Audio Streams
* "audioElement" should be a jQuery object
*/
function streamPlayStop(audioElement) {
if (audioElement[0].paused) {
audioElement[0].play();
} else if (!audioElement[0].volume) {
audioElement[0].volume = 1;
} else {
audioElement[0].volume = 0;
}
}
I should caution that even though this achieves the desired functionality for stopping and resuming live audio streams, it isn't ideal because the stream, when stopped, is actually still playing and being downloaded in the background, using up bandwidth in the process.
However, this solution doesn't necessarily take up more bandwidth than just using .play() and .pause() on a streaming audio element. Simply using the audio tag with streaming audio uses up a great deal of bandwidth anyway, because once streaming audio is played, it continues to download the contents of the stream in the background when it is paused.
It should be noted that this method won't work in iOS because of purposefully built-in limitations for iPhones and iPads:
On iOS devices, the audio level is always under the user’s physical control. The volume property is not settable in JavaScript. Reading the volume property always returns 1.
If you choose to use the workaround in this answer, you'll need to create a fallback for iOS devices that uses the play() and pause() functions normally, or your interface will be unable to pause the stream.

Tested #Ciantics code and it worked with some modifications, if you want to use multiple sources.
As the source is getting removed, the HTML audio player becomes inactive, so the source (URL) needs to be added directly after again to become active.
Also added an event listener at the end to connect the function when pausing:
var audioElement = document.querySelector("audio");
var sources = document.querySelector("audio").children;
var sourceList = [];
for(i=0;i<sources.length;i++){
sourceList[i] = sources[i].getAttribute("src");
}
function pause() {
for(i=0;i<sources.length;i++){
sources[i].setAttribute("src", "");
}
audioElement.pause();
// settimeout, otherwise pause event is not raised normally
setTimeout(function () {
audioElement.load(); // This stops the stream from downloading
});
for(i=0;i<sources.length;i++){
if (!sources[i].getAttribute("src")) {
sources[i].setAttribute("src", sourceList[i]);
audioElement.load(); // This restarts the stream download
}
}
}
audioElement.addEventListener("pause", pause);

Related

getUserMedia mutes the sound while recording current tab

[Goal]: To record the sound of a tab and at the same time hear the sound while it's being recorded
[Means]: To use chrome.tabCapture because it's less invasive than chrome.desktopCapture or navigator.mediaDevices.getDisplayMedia. The former provides a better user experience as opposed to the latter which asks for tab selection every time and then intrusively shows the blue rectangle with a status bar and the "Stop Sharing" button
[Issue]: Tab sound is muted after getting the stream (recording not necessary, only calling getUserMedia) and can not be unmuted. Is this a bug? How can it be avoided?
[Technicals]:
Obtain the stream id in a foreground page (popup, options, or iframe pointing to an extension page, injected in the current DOM by a content script) because that's the only place where tabCapture is available
const tabId = ...get current tab id...
chrome.tabCapture.getMediaStreamId({ consumerTabId: tabId }, function(streamId) {
// send the streamId to the content script
})
Receive the streamId then use it to start the capture
const options = {
audio: {
mandatory: {
chromeMediaSource: 'tab',
chromeMediaSourceId: streamId
}
}
}
navigator.mediaDevices.getUserMedia(options).then((tabStream) => {
// at this point the sound of the tab becomes muted with no way to unmute it
});
Just invoking tabCapture will mute the tab. AudioContext source connecting to the default destination doesn't do anything for this stream and no error is thrown. If the callback promise from getUserMedia is empty, the sound of the tab comes back on its own in about 20 seconds or so. If MediaRecorder actually uses the stream to record the sound, the sound doesn't come back until recording is stopped. The sound of the tab is present in the recording, just muted while recording.
[Comparison 1]: Obtaining a MediaStream directly with chrome.tabCapture.capture instead of the above mechanism works as intended. The tab sound is muted, but can be reconnected to the speakers using AudioContext. The issue with this approach is that chrome.tabCapture.capture can't be started successfully in an injected iframe (always fails with the cryptic "Error starting tab capture"), so there's no way to capture the sound. It does work in a popup though, but not really an option given its volatile nature.
[Comparison 2]: chrome.desktopCapture works in the opposite way. The sound is always present in the tab while recording and can not be muted, not that this poses a problem.

Using the standard gamePad code, not finding sound files even though the path name is correct?

Using the standard gamePad code, not finding sound files even though the path name is correct?
I have definitely researched this question. For sure, I have found code on SO claiming to solve this dilemma, but this published code doesn't.
I am successfully finding the sound file using Preview under BBEdit's "Markup" Menu. But, the oh-oh surfaces when running my game on my commercial Server.
I am even successful when using keypress code is activated -- local on my Mac and on the Server.
The failure is when I am using the external Server to run my gamePad code to find the sound file when all my source code is loaded onto my Server. In this case, the sound does not play.
FILE HIERARCHY
games folder
Game_1 folder
game1.html
Game_Support folder
audio folder
js folder
HTML
<body onload="doBodyOnLoad()">
JS:
function doBodyOnLoad() {
$(document).ready(function() {
// ... //
}); // $(document).ready
} // doBodyOnload
function setupKeypresses() {
$(document).keydown(function(evt) {
let code = evt.keyCode || evt.which;
// for example:
if (code === "R")
{
movePaddleRight(); // this will call ouch() below
}
});
} // setupKeypresses
function PlaySound(id, src) {
let theSound = new Audio();
theSound.src = src;
theSound.play();
} // PlaySound
function ouch() {
updateScore(--thisScore);
// even the absolute path doesn't work ?
// var theSnd = "http://lovesongforever.com/games/Game_1/Game_1_Support/audio/explosion.mp3";
var theSnd = "Game_1_Support/audio/explosion.mp3";
PlaySound("audioPlaceHolder", theSnd);
// fade either the whole Board (okay), or just the Paddle (not! so much)
doFade("#gameBoard"); // doFade("#gameBoard > #gamePaddle") [not ready for Prime Time]
} // ouch
Thanks bunches for your patience with me!
This is because you're trying to autoplay an Audio element without user interaction to initiate the audio.
Firefox expresses a blocked play() call to JavaScript by rejecting the promise returned by HTMLMediaElement.play() with a NotAllowedError. All major browsers which block autoplay express a blocked play via this mechanism. In general, the advice for web authors when calling HTMLMediaElement.play(), is to not assume that calls to play() will always succeed, and to always handle the promise returned by play() being rejected.
https://hacks.mozilla.org/2019/02/firefox-66-to-block-automatically-playing-audible-video-and-audio/
I think it will work if you click or tap on the page first. Gamepad button presses may work for this too depending on the implementation.

HTML 5 video tag video and audio de-synchronisation

While having a video play through my local website, it's audio and video become de-synced after a while, like 40 minutes or so, also if I pause the video and then un pause it... I don't know if this is a problem with html 5, my browser, computer or what? But my audio is around 1 second ahead of the video, it's very noticeable... here's my code for the video in case it matters:
echo "<video class=\"videoContainer\" controls autoplay>
<source src=\"$movieUrl\" type=\"video/mp4\">
</video>";
I couldn't find any solution for this, in-fact... I couldn't find anyone with this same problem!
P.S Refreshing the page fixes the issue but I don't want to do that every time the video de-syncs... Also I don't have de-sync issues on YouTube etc...
There's currently no good API for synchronizing things with the timeline of a video, for instance captions or infoboxes. The spec has had "cue ranges" for this purpose earlier (which even earlier were "cue points"); it is expected that something similar will be added in the future, including support for declarative captions.
However, for now, you will have to either use a timer and read currentTime, or listen for timeupdate and read currentTime. timeupdate is fired at 15 to 250 ms intervals while the video is playing, unless the previous event handler for timeupdate is still running, in which case the browser should skip firing another event. Opera currently always fires it at 250 ms intervals while the video is playing, while Firefox currently fires it once per rendered frame. The idea is to allow the event to be fired at greater intervals if the system load increases, which could save battery life on a handheld device or keep things responsive in a heavy application. The bottom line is that you should not rely on the interval being the same over time or between browsers or devices.
Let's say you want to show a div element between the times 3s and 7s of the video; you could do it like this:
Hello world! var video = document.getElementsByTagName('video')[0]; var hello = document.getElementById('hello'); var hellostart = hello.getAttribute('data-starttime'); var helloend = hello.getAttribute('data-endtime'); video.ontimeupdate = function(e) { var hasHidden = hello.hasAttribute('hidden'); if (video.currentTime > hellostart && video.currentTime
The hidden attribute indicates that the element is not relevant and should be hidden. This is not supported in browsers yet, so you have to hide it with CSS:
*[hidden] { display:none }
The data-starttime and data-endtime attributes are custom data-* attributes that HTML5 allows to be placed on any element. It's great for including data that you want to read with script, instead of abusing the class or title atributes. HTML5 also has a convenience API for data-* attributes, but it's not supported in browsers yet, so we have to use getAttribute a little longer.
The above would look like this using a timer instead:
Hello world! var video = document.getElementsByTagName('video')[0]; var hello = document.getElementById('hello'); var hellostart = hello.getAttribute('data-starttime'); var helloend = hello.getAttribute('data-endtime'); setInterval(function() { var hasHidden = hello.hasAttribute('hidden'); if (video.currentTime > hellostart && video.currentTime
This will run every 100 ms. Whether you should use setInterval or timeupdate depends on what you're doing and whether you're ok with the interval changing. Note that the setInterval example above also runs when the video is not playing, which the timeupdate example doesn't. It's possible to clear the interval with clearInterval when the video stops playing and setting it again when it starts playing, though.
If you want to synchronize something with the time playback starts, or after a seek, you should listen for playing and seeked — not play or seeking. The former indicate when playback has actually started and a seek has finished, respectively, while the latter indicate that playback or seeking has just been requested, but could take some time before it actually occurs.

How to properly dispose of an HTML5 Video and close socket or connection

I am building a web page with a list of video records. Clicking on each video record opens a modal dialog on the same page with detail of the record and an HTML5 Video player.
A user can open one video, close it, and open another as many times as they want. However, on Chome specifically, after 3-5 videos, the browser starts hanging for upwards of two minutes while displaying a message "waiting for socket".
Doing some reading, I have narrowed it to Chrome's inability to open more than 6 connections to the same host.
I must be doing something wrong with how I dispose of the Media players. I believe the socket remains open to the media for some period even though the html for the player has been removed from the dom.
Using:
Bootstrap,
MediaElement.js,
HTML5 Video,
MVC,
Controller returning "Range Request" of FilePathResult
// Handling Bootstrap Modal Window Close Event
// Trigger player destroy
$("#xr-interaction-detail-modal").on("hidden.bs.modal", function () {
var player = xr.ui.mediaelement.xrPlayer();
if (player) {
player.pause();
player.remove();
}
});
I am going to go for my Self Learner badge, here, and answer my own question.
I did about 8 hours of research on this and came up with a great solution. Three things had to be done.
Set the HTML5 video src to something other than the original URL. This will trigger the player to close the open socket connection.
Set the HTML5 video src to a Base64 encoded data object. This will prevent the Error Code 4 issue MEDIA_ERR_SRC_NOT_SUPPORTED.
For issues in older versions of Firefox, also trigger the .load() event.
My Code:
// Handling Bootstrap Modal Window Close Event
// Trigger player destroy
$("#xr-interaction-detail-modal").on("hidden.bs.modal", function () {
var player = xr.ui.mediaelement.xrPlayer();
if (player) {
player.pause();
("video,audio").attr("src","data:video/mp4;base64,AAAAHG...MTAw");
player.load();
player.remove();
}
});
I came up with the idea to load the data object as the src. However, I'd like to thank kud on GitHub for the super small base64 video.
https://github.com/kud/blank-video
Added a line between pause() and remove():
// Handling Bootstrap Modal Window Close Event
// Trigger player destroy
$("#xr-interaction-detail-modal").on("hidden.bs.modal", function () {
var player = xr.ui.mediaelement.xrPlayer();
if (player) {
player.pause();
("video,audio").attr("src", "");
player.remove();
}
});

Pausing issue with flash webcam recording

I'm building a webcam recording app in CS5 and I'm having some seemingly random issues with the recorded flv. Currently I'm publishing a stream to Wowza Media Server using the standard _netstream.publish("movieName", "record") command. Everything regarding this works fine and I can play the file back but sometimes there's a 3 to 4 second pause at the beginning or end of the video. There will be a still frame and the time will sit at 0 and then snap to 4. I've explored bandwidth options and I've turned the resolution and quality down considerably and it doesn't seem to have any effect and the rest of the video will play back smoothly. Here are my current camera and mic settings.
_cam.setMode(160, 120, 30, false);
_cam.setQuality(0, 88);
_cam.setKeyFrameInterval(30);
_mic.rate = 11;
I'm also flushing the buffer before closing out the publish stream
_netstream.publish('null');
Could there be something going on with camera initialization/deactivation that causes the lag?
Any help would be greatly appreciated. Let me know if you need more details
I believe this has something to do with the way that the Flash plugin itself initializes and displays the camera.
If you set up a simple test to try setting and unsetting the video stream:
var cam:Camera = Camera.getCamera();
var webcam:Video = new Video(500, 375);
addChild(webcam);
var isPaused:Boolean = false;
function showWebcam():void {
if (!isPaused) {
cam = null;
} else {
cam = Camera.getCamera();
}
webcam.attachCamera(cam);
isPaused = !isPaused;
}
pausingButton.addEventListener(MouseEvent.CLICK, showWebcam);
You'll notice a definite pause as it switches between the two states.
From what I've seen, every time I call attachCamera() with a video object, there is a noticeable pause of the Flash Player itself (including all tweens, interactions, everything) when the method is called, even if the object I'm attaching is null.
Four seconds seems like an excessive lag, but I have noticed that the larger the input/video render and with smoothing = true set on the video object can affect the length of the delay.
As for a solution; I'm not sure if there is one achievable via pure Actionscript, since the delay appears to be down to how the Flash Player itself initializes and renders the live video object.