I am trying to record from the microphone using HTML5 in Chrome 29 Beta for Android (it has enabled web audio support in its beta 29). In the code below, ProcessAudio is the web audio filter function, in which I receive the input buffer from the microphone. I get the correct sample size. However, the audio pcm samples in mobile chrome are always zero. Does chrome disable the input to the audio filters in its mobile version? Has anybody got audio recording working using HTML5 in chrome mobile version?
The following code works fine in chrome (desktop version).
<!DOCTYPE HTML>
<html>
<head>
<script>
function GetMedia(obj, fnSuccess, fnFailure)
{
if(navigator.getUserMedia)
{
return navigator.getUserMedia(obj, fnSuccess, fnFailure);
}
else if(navigator.webkitGetUserMedia)
{
return navigator.webkitGetUserMedia(obj, fnSuccess, fnFailure);
}
else if(navigator.mozGetUserMedia)
{
return navigator.mozGetUserMedia(obj, fnSuccess, fnFailure);
}
else if(navigator.msGetUserMedia)
{
return navigator.msGetUserMedia(obj, fnSuccess, fnFailure);
}
alert("no audio capture");
}
var incrementer = 0;
function ProcessAudio(e)
{
var inputBuffer = e.inputBuffer.getChannelData(0);
var outputBuffer = e.outputBuffer.getChannelData(0);
outputBuffer.set(inputBuffer, 0);
document.getElementById("display").innerText =
incrementer + " " + inputBuffer[0];
incrementer++;
}
var context = null;
function Success(localMediaStream)
{
context = new window.webkitAudioContext();
var microphone = context.createMediaStreamSource(localMediaStream);
var node = context.createScriptProcessor(4096, 1, 1);
node.onaudioprocess = ProcessAudio;
microphone.connect(node);
node.connect(context.destination);
}
function Error(err)
{
alert("no audio support");
}
function load(e)
{
GetMedia({audio:true,video:false}, Success, Error);
}
</script>
</head>
<body onload="load(event)">
<div>Audio Player</div>
<div id="display"></div>
<video id="localvideo" autoplay="autoplay" style="opacity:1"></video>
</body>
</html>
We now have Web Audio input working in Chrome for Android Beta (31.0.1650.11).
There's an issue with inputBuffer in Chrome beta for Android right now - it always contains zeros (at least on those devices that I tested) as you mentioned - confirmed. That ain't much of help but for demo purposes the best I could achieve right now is playing "live" input stream via <audio> object. See here. Hit "record" and then start playing the audio object. The rest beyond that is TARFU in the current beta. Guess I'm waiting for G-folks to patch it asap. What I found out though, is that zingaya.com works fine (that is, records your audio and plays it back to you) on Chrome Beta for Android. Just try them out and they'll play back your stream. Apparently, this is made by leveraging WebRTC. In other words you evidently can record an audio stream that way, I haven't figured out how to do it just yet. Post it up if you come up with something. G-guys are working fast on the other hand - they've released a new beta version less than a week ago that at least can play audio fetched with XHR. The version prior to that couldn't do that even.
Related
I want to play stream from gstreamer in a web browser.
I played around a with RTP, WebRTC and SDP files but, while VLC was able to connect to stream by simple SDP, browsers were not. I later understood that WebRTC requires secure connection which only complicates things and is not needed for my purposes. I stumbled upon Media Source Extension (MSE) of html5, which seems that it could help, but I'm not able to find some comprehensive tutorial or appropriate specs on how to get gstreamer to stream correct data and later how to play them using MSE. I'm also not sure about latency with using MSE.
So is there a way to play stream from gstreamer in a browser?
Thanks.
Using node webrtc project, I was able to combine output from gstreamer with webrtc call. For gstreamer, there is a project which enables it's use with node gstreamer superficial. So basically, you need to run gstremaer process from node process, which can then control output from gstremaer. On every gstreamer frame there is a callback called which takes the frame and can send it to webrtc calls.
Then an webrtc calls needs to be implemented. There is required some signaling protocol for calls. One side of the call will be the server and another will be the client's browser, instead of two browsers. Then a video track will be created where frames from gstreamer superficial will be pushed.
const { RTCVideoSource } = require("wrtc").nonstandard;
const gstreamer = require("gstreamer-superficial");
const source = new RTCVideoSource();
// This is WebRTC video track which should be used with addTransceiver see below
const track = source.createTrack();
const frame = {
width: 1920,
height: 1080,
data: null
};
const pipeline = new gstreamer.Pipeline("v4l2src ! videorate ! video/x-raw,format=YUY2,width=1920,height=1080,framerate=25/1 ! videoconvert ! video/x-raw,format=I420 ! appsink name=sink");
const appsink = pipeline.findChild("sink");
const pull = function() {
appsink.pull(function(buf, caps) {
if (buf) {
frame.data = new Uint8Array(buf);
try {
source.onFrame(frame);
} catch (e) {}
pull();
} else if (!caps) {
console.log("PULL DROPPED");
setTimeout(pull, 500);
}
});
};
pipeline.play();
pull();
// Example:
const useTrack = SomeRTCPeerConnection => SomeRTCPeerConnection.addTransceiver(track, { direction: "sendonly" });
This's a simple Phaser audio example. It works well on my Android web browser. However, it's muted after porting to Android app by Phonegap cloud build.
I know how to play sound (and loop) in Phonegap app (How to loop a audio in phonegap?) but don't know how to apply it into the Phaser JS framework.
Here's the ported app. I can install and run it but without sound. Do I miss something or Phonegap Cloud Build does support the WebAudio in Phaser JS?
https://build.phonegap.com/apps/1783695/
My config.xml is:
<?xml version="1.0" encoding="UTF-8" ?>
<widget id="com.phaser.phasersound" version="1.0.0" xmlns="http://www.w3.org/ns/widgets" xmlns:gap="http://phonegap.com/ns/1.0">
<name>Phaser sound complete</name>
<description>
Phaser sound phonegap
</description>
<gap:plugin name="org.apache.cordova.media" />
<icon src="icon.png" />
<preference name="splash-screen-duration" value="1"/>
<!--
If you do not want any permissions to be added to your app, add the
following tag to your config.xml; you will still have the INTERNET
permission on your app, which PhoneGap requires.
-->
<preference name="permissions" value="none"/>
</widget>
The source code is: (I changed the local audio files from local to github links to run on code snippet)
var game = new Phaser.Game(600, 800, Phaser.AUTO, 'phaser-example', { preload: preload, create: create });
function preload() {
game.scale.scaleMode = Phaser.ScaleManager.SHOW_ALL;
//have the game centered horizontally
game.scale.pageAlignHorizontally = true;
game.scale.pageAlignVertically = true;
game.stage.backgroundColor = '#414040';
// I changed the local audio files from local to github links to run on code snippet
/*
game.load.audio('explosion', 'assets/audio/SoundEffects/explosion.mp3');
game.load.audio('sword', 'assets/audio/SoundEffects/sword.mp3');
game.load.audio('blaster', 'assets/audio/SoundEffects/blaster.mp3');
*/
game.load.audio('explosion', 'https://raw.githubusercontent.com/nguoianphu/phaser-sound-complete-phonegap/master/www/assets/audio/SoundEffects/explosion.mp3');
game.load.audio('sword', 'https://raw.githubusercontent.com/nguoianphu/phaser-sound-complete-phonegap/master/www/assets/audio/SoundEffects/sword.mp3');
game.load.audio('blaster', 'https://raw.githubusercontent.com/nguoianphu/phaser-sound-complete-phonegap/master/www/assets/audio/SoundEffects/blaster.mp3');
}
var explosion;
var sword;
var blaster;
var text;
var text1;
var text2;
var text3;
function create() {
var style = { font: "65px Arial", fill: "#52bace", align: "center" };
text = game.add.text(game.world.centerX, 100, "decoding", style);
text.anchor.set(0.5);
explosion = game.add.audio('explosion');
sword = game.add.audio('sword');
blaster = game.add.audio('blaster');
// Being mp3 files these take time to decode, so we can't play them instantly
// Using setDecodedCallback we can be notified when they're ALL ready for use.
// The audio files could decode in ANY order, we can never be sure which it'll be.
game.sound.setDecodedCallback([ explosion, sword, blaster ], start, this);
}
var keys;
function start() {
text.text = 'Press 1, 2 or 3';
var style = { font: "48px Arial", fill: "#cdba52", align: "center" };
text1 = game.add.text(game.world.centerX, 250, "Blaster: Stopped", style);
text1.anchor.set(0.5);
text2 = game.add.text(game.world.centerX, 350, "Explosion: Stopped", style);
text2.anchor.set(0.5);
text3 = game.add.text(game.world.centerX, 450, "Sword: Stopped", style);
text3.anchor.set(0.5);
explosion.onStop.add(soundStopped, this);
sword.onStop.add(soundStopped, this);
blaster.onStop.add(soundStopped, this);
keys = game.input.keyboard.addKeys({ blaster: Phaser.Keyboard.ONE, explosion: Phaser.Keyboard.TWO, sword: Phaser.Keyboard.THREE });
keys.blaster.onDown.add(playFx, this);
keys.explosion.onDown.add(playFx, this);
keys.sword.onDown.add(playFx, this);
// And for touch devices you can also press the top, middle or bottom of the screen
game.input.onDown.add(onTouch, this);
}
function onTouch(pointer) {
var b = game.height / 3;
if (pointer.y < b)
{
playFx(keys.blaster);
}
else if (pointer.y > b * 2)
{
playFx(keys.sword);
}
else
{
playFx(keys.explosion);
}
}
function playFx(key) {
switch (key.keyCode)
{
case Phaser.Keyboard.ONE:
text1.text = "Blaster: Playing";
blaster.play();
break;
case Phaser.Keyboard.TWO:
text2.text = "Explosion: Playing";
explosion.play();
break;
case Phaser.Keyboard.THREE:
text3.text = "Sword: Playing";
sword.play();
break;
}
}
function soundStopped(sound) {
if (sound === blaster)
{
text1.text = "Blaster: Complete";
}
else if (sound === explosion)
{
text2.text = "Explosion: Complete";
}
else if (sound === sword)
{
text3.text = "Sword: Complete";
}
}
<script src="https://cdnjs.cloudflare.com/ajax/libs/phaser/2.4.4/phaser.js"></script>
UPDATE 2015-12-01
Here is my completed source code. It has both .mp3 and .ogg sound files. You can play them on Android native browser (tested on 4.4.4 Samsung E5).
Source: https://github.com/nguoianphu/phaser-sound-complete-phonegap
Here is the ported app on Phonegap. It can display the screen but can't play sounds.
https://build.phonegap.com/apps/1783695/builds
You are trying to play the audio with the webview library. It is likely using the HTML5 API for audio or webaudio. If it is neither of these, then you need to ask the author.
Next, it is not best practice to use external source (http:). Your assests (javascript, css, audio files, etc) should live on the device. If you load files from the web, then the sound quality could be poor (or the audio may not play at all - see whitelist below). Load from the device.
Android 4.4.4 is Kitkat. The standard webview library was exchanged for the chromium version. This means your audio library might be confused about this or you need to give the library knowledge about this library. This also means your code may not work on devices before 4.4.4. (Mostly, because you cannot test it.)
The link you point to is likely using the core media plugin, even though they dont say so. In addition, the post is over 3 years old. Many thing have changed since them. NOTE: you have installed the media plugin in your config.xml. This is likely why your loop works.
You should start over. You've made many errors. In addition, to all that you have, You will need to implement the whitelist plugin (if you are going to import files, or talk to the network).
FIRST TRY this sample app - example plays on Android and iOS. You can download the Android version and test it. The iOS version requires I have your UUID compiled in.
There are 16 audio plugins you can choose from. I know a few do real time audio playback and have better control than the "core" plugin.
You should read:
Top Mistakes by Developers new to Cordova/Phonegap - read the bold sentences.
HOW TO apply the Cordova/Phonegap the whitelist system
HOWTO Core Plugins Setup
Phonegap--Generic-Boilerplate7 - just wrote this. It works.
Phonegap Demo Apps
Phonegap-Media-Test - source code for the example that plays on Android and iOS. You can download the Android version and test it.
UPDATE: 2015-12-01 - 2am Previously, I had forgotten to add a wild-card (*) to the CSP meta tag. I am now including this. This meta tag should be added to the header of the index.html file that is playing the audio.
NOTE YOUR APP IS NOW INSECURE. IT IS UP TO YOU TO SECURE YOUR APP.
<meta http-equiv="Content-Security-Policy"
content="default-src *;
style-src * 'self' 'unsafe-inline' 'unsafe-eval';
script-src * 'self' 'unsafe-inline' 'unsafe-eval';">
UPDATE: 2015-12-01 - 3pm
#Tuan, I've applied all the fixes as outlined in
HOW TO apply the Cordova/Phonegap the whitelist system
HOWTO Core Plugins Setup
Phonegap--Generic-Boilerplate7 - just wrote this. It works.
The audio is now working on my Android LG Leon/Android 5.1.1
Truthfully, I would never do this on my own, but your code had enough working that after I tested it on my firefox(v34) browser, I was fairly certain it would work.
UPDATE: 2016-04-15
The code has been removed. Ask in the comments, if you need code.
There should be enough code in place for you to work off of.
- Code
- Working Android App
- Phonegap Build Documentation
I am working with the HTML5 audio api to play sound. This works fine with regular mp3 files but when using a sound stream such as http://95.173.167.24:8009, it fails to play.
Here is the code i'm using:
if('webkitAudioContext' in window) {
var myAudioContext = new webkitAudioContext();
}
request = new XMLHttpRequest();
request.open('GET', 'http://95.173.167.24:8009', true);
request.responseType = 'arraybuffer';
request.addEventListener('load', bufferSound, false);
request.send();
function bufferSound(event) {
var request = event.target;
var source = myAudioContext.createBufferSource();
source.buffer = myAudioContext.createBuffer(request.response, false);
source.connect(myAudioContext.destination);
source.noteOn(0);
}
Can anyone point me in the right direction on this?
Any help is appreciated.
Thanks
The problem is likely that SHOUTcast is detecting your User-Agent string as a browser. It looks for any string with Mozilla in it, and says "Oh, that's a browser! Send them the admin panel."
You need to force the usage of the audio stream. Fortunately, this is easily done by adding a semicolon at the end of your URL:
http://95.173.167.24:8009/;
Note that the User-Agent string in your logs will be MPEG OVERRIDE.
This will work for most browsers. Some browsers may still not like the HTTP-like resopnses that come from SHOUTcast, but this will at least get you started.
i had a question
on some devices like iPad and Android Tables you cannt change Volume of Video Tags becuase Volume API dont supported on this devices. have yo a idea how i can detect if this isnt support?
The best I could come up with is this:
function volumeChangeSupported () {
var ua = navigator.userAgent.toLowerCase();
// got information from jplayer:
var noVolume = /ipad|iphone|ipod|android|blackberry|windows ce|windows phone|webos|playbook/.exec(ua);
if (noVolume) {
if (noVolume[0] === 'android' && /gecko/.test(ua)) {
// Firefox on android DOES support changing the volume:
return true;
}
else {
return false;
}
}
return true;
}
This doesn't really "detect" support for changing the volume. I got this information partly from jPlayer and partly from my own experience testing Firefox 19 on an old Android 3 tablet. Who knows if Firefox on an Android phone or a different Android version behaves differently.
But before this I tried to detect volume change support like this:
function volumeChangeSupported () {
var audio = new Audio();
audio.volume = 0.5;
return audio.volume === 0.5;
}
This yielded the correct result for iPhone Safari and Android Firefox, but not for other Android Browsers (the "Android Browser" and "Dolphin", which can't change the volume but had audio.volume === 0.5 to be true).
I have a video as a background to a web page, and I am trying to get it to loop. Here is the code:
<video autoplay='true' loop='true' muted='true'>
<source src='/admin/wallpapers/linked/4ebc66e899727777b400003c' type='video/mp4'></source>
</video>
Even though I have told the video to loop, it does not. I also tried to get it to loop with the onended attribute (as per this Mozilla support thread, I also tried that bit of jQuery). Nothing has worked so far. Is it an issue with Chrome, or my code?
Edit:
I checked the Network events and HEAD of a working copy (http://fhsclock-labs.heroku.com/no-violence) versus the application I'm trying to get working. The difference is the working copy is serving up the video from a static asset on Heroku (via Varnish, apparently), whilst mine is serving from GridFS (MongoDB).
The Network tab of Chrome's Inspector show that in my application, the video is requested three times. One time the Status is "pending", the second is "canceled", and the final one is 200 OK. The working copy only shows two requests, one's Status is pending and the other is 206 Partial Content. However, after the video plays once, that request changes to "Cancelled" and it makes another request for that video. In my application, that does not happen.
As for Type, in my application, two are "undefined" and the other "video/mp4" (which it is supposed to be). In the working app, all of the requests are "video/mp4".
In addition, I'm getting Resource interpreted as Other but transferred with MIME type undefined. warnings in the Console.
I'm not really quite sure where to begin on this. It's my belief that the issue is server-side, as serving the file as static assets works fine. It could be that the server isn't sending the correct content type. It could be an issue with GridFS. I do not know.
At any rate, the source is here. Any insight that you can offer is appreciated.
Ah, I just stumbled into this exact problem.
As it turns out, looping (or any sort of seeking, for that matter) in <video> elements on Chrome only works if the video file was served up by a server that understands partial content requests. i.e. the server needs to honor requests that contain a "Range" header with a 206 "Partial Content" response. This is even the case if the video is small enough to be fully buffered by chrome, and no more server-round trips are made: if your server didn't honor chrome's Range request the first time, the video will not be loopable or seekable.
So yes, an issue with GridFS, although arguably Chrome should be more forgiving.
Simplest workaround:
$('video').on('ended', function () {
this.load();
this.play();
});
The 'ended' event fires when the video reaches the end, video.load() resets the video to the beginning, and video.play() starts it playing immediately once loaded.
This works well with Amazon S3 where you don't have as much control over server responses, and also gets around Firefox issues related to video.currentTime not being settable if a video is missing its length metadata.
Similar javascript without jQuery:
document.getElementsByTagName('video')[0].onended = function () {
this.load();
this.play();
};
Looks like its been an issue in the past, there are at least two closed bugs on it, but both state that it was fixed:
http://code.google.com/p/chromium/issues/detail?id=39683
http://code.google.com/p/chromium/issues/detail?id=18846
Since Chrome and Safari both use webkit based browsers you might be able to use some of these work arounds:
http://blog.millermedeiros.com/2011/03/html5-video-issues-on-the-ipad-and-how-to-solve-them/
function restartVideo(){
vid.currentTime = 0.1; //setting to zero breaks iOS 3.2, the value won't update, values smaller than 0.1 was causing bug as well.
vid.play();
}
//loop video
vid.addEventListener('ended', restartVideo, false);
Just in case none of the answers above help you, make sure you don't have your inspector running with the Disable cache option checked. Since Chrome grabs the video from cache, it will basically work once. Just debugged this for 20 minutes before realizing this was the cause. For reference and so I know I am not the only one someone else's chromium bug report.
My situation:
I have the exact same problem, however changing the header of the response message alone didnt do. No loop, replay or seek. Also a pure stop doesnt work, but that could be my configuration.
Answer:
According to some sites (couldnt find them anymore) its also possible to trigger the load() method right after the video ends, and before the next one is supposed to start. That should reload the source causing a once again working video/audio element.
#john
Please note that your answers/links are normal bugs, and not focused on this problem. Using a server/webserver is what causes this problem. Whereas the bugs these links describe are of a different kind. Thats also why the answer isnt working.
I hope it helps, i am still looking for a solution.
For anyone coming on this page 9 years later and if all the above answers didn't work: I had this issue too and I thought the source of the issue was either my browsers or with the server.
I've later noticed that the other websites on internet which use looping videos they don't have issue with looping videos. To troubleshoot I have downloaded a random video from one of the sites and I visited and uploaded on my own server to delightedly find out it was working, so it seemed that the source of the issue was the video I was using.
Then I fixed my video with an online video converter website (don't want to publicize any in particular but the first ones from a quick google research do work) and alas, this solved the issue.
I'm not sure what the real reason of the issue was. I do assume there was a conversion or compression error of the original video that was handed me from my client.
I know this doesn't pertain exactly to the question asked, but if someone comes across this when having a similar issue, make sure that you have your sources in order properly.
I was loading an mp4 and a webm file and noticed that the video was not looping in Chrome. It was because the webm file was the first source listed so Chrome was loading the webm file and not the mp4.
Hope that helps someone else that comes across this issue.
<video autoplay loop>
<source src="/path-to-vid/video.mp4" type="video/mp4">
<source src="/path-to-vid/video.webm" type="video/webm">
</video>
it is super lame but dropbox use the right status code. So upload to dropbox and replace the www by dl.
Thus using a dropbox url the video play fine.
I had same issue and inevitably solved problem by streaming the content.
e.g this is the code with PHP laravel blade html code which is requesting to streaming route:
<video>
<source src="{{route('getVideoStream',$videoId)}}" type="video/mp4"/>
</video>
in the Controller I will stream video and return it as laravel stream function:
public function getVideoStream($videoId){
$path = $pathOfVideo;
$headers = [
'Content-Type' => 'video/mp2t',
'Content-Length' => File::size($path),
'Content-Disposition' => 'attachment; filename="start.mp4"'
];
$stream = new VideoStream($path);
return response()->stream(function () use ($stream) {
$stream->start();
});
}
and VideoStream Class is the streaming class I found from a GitHub gist:
class VideoStream
{
private $path = "";
private $stream = "";
private $buffer = 102400;
private $start = -1;
private $end = -1;
private $size = 0;
function __construct($filePath)
{
$this->path = $filePath;
}
/**
* Open stream
*/
private function open()
{
if (!($this->stream = fopen($this->path, 'rb'))) {
die('Could not open stream for reading');
}
}
/**
* Set proper header to serve the video content
*/
private function setHeader()
{
ob_get_clean();
header("Content-Type: video/mp4");
header("Cache-Control: max-age=2592000, public");
header("Expires: " . gmdate('D, d M Y H:i:s', time() + 2592000) . ' GMT');
header("Last-Modified: " . gmdate('D, d M Y H:i:s', #filemtime($this->path)) . ' GMT');
$this->start = 0;
$this->size = filesize($this->path);
$this->end = $this->size - 1;
header("Accept-Ranges: 0-" . $this->end);
if (isset($_SERVER['HTTP_RANGE'])) {
$c_start = $this->start;
$c_end = $this->end;
list(, $range) = explode('=', $_SERVER['HTTP_RANGE'], 2);
if (strpos($range, ',') !== false) {
header('HTTP/1.1 416 Requested Range Not Satisfiable');
header("Content-Range: bytes $this->start-$this->end/$this->size");
exit;
}
if ($range == '-') {
$c_start = $this->size - substr($range, 1);
} else {
$range = explode('-', $range);
$c_start = $range[0];
$c_end = (isset($range[1]) && is_numeric($range[1])) ? $range[1] : $c_end;
}
$c_end = ($c_end > $this->end) ? $this->end : $c_end;
if ($c_start > $c_end || $c_start > $this->size - 1 || $c_end >= $this->size) {
header('HTTP/1.1 416 Requested Range Not Satisfiable');
header("Content-Range: bytes $this->start-$this->end/$this->size");
exit;
}
$this->start = $c_start;
$this->end = $c_end;
$length = $this->end - $this->start + 1;
fseek($this->stream, $this->start);
header('HTTP/1.1 206 Partial Content');
header("Content-Length: " . $length);
header("Content-Range: bytes $this->start-$this->end/" . $this->size);
} else {
header("Content-Length: " . $this->size);
}
}
/**
* close curretly opened stream
*/
private function end()
{
fclose($this->stream);
exit;
}
/**
* perform the streaming of calculated range
*/
private function stream()
{
$i = $this->start;
set_time_limit(0);
while (!feof($this->stream) && $i <= $this->end) {
$bytesToRead = $this->buffer;
if (($i + $bytesToRead) > $this->end) {
$bytesToRead = $this->end - $i + 1;
}
$data = fread($this->stream, $bytesToRead);
echo $data;
flush();
$i += $bytesToRead;
}
}
/**
* Start streaming video content
*/
function start()
{
$this->open();
$this->setHeader();
$this->stream();
$this->end();
}
}