WebRTC: Connecting multiple listeners to one client, one at a time - html

I have been trying to get WebRTC to function with a broadcaster and multiple listeners but am stuck when it comes to transferal descriptions and candidates via signalling (with nodejs & socket.io).
I can get the process working between two browsers with a simple nodejs socket app which simply broadcasts the descriptions and candidates to other already connected clients, but when I attempt to store a description and connect with a newly opened browser, nothing happens.
What I basically need to understand is what do I need to provide to one browser, in order for it to begin communicating with another? The project I am working on requires the ability for listeners to join rooms, authenticate, and begin listening to whatever media is being sent.
Below is my client side code:
var audioContext = new webkitAudioContext()
var client = null
var configuration =
{
'iceServers':
[{
'url': 'stun:stun.example.org'
}]
}
$(function ()
{
window.RTCPeerConnection = window.RTCPeerConnection || window.webkitRTCPeerConnection || window.mozRTCPeerConnection
client = new RTCPeerConnection(configuration, { optional:[ { RtpDataChannels: true } ]})
client.onnegotiationneeded = function ()
{
console.log('Negotiation needed')
createOffer()
}
client.onicecandidate = function (event)
{
console.log('onicecandidate')
socket.emit('candidate', JSON.stringify({ 'candidate': event.candidate }))
}
client.onaddstream = function (event)
{
console.log('onaddstream')
$('#player').attr('src', URL.createObjectURL(event.stream))
player.play()
}
socket.on('candidate', function (event)
{
candidate(event)
})
socket.on('description', function (message)
{
if(!client) { return }
client.setRemoteDescription(new RTCSessionDescription(message.sdp), function () {
if (client.remoteDescription.type == 'offer')
client.createAnswer(function (description)
{
client.setLocalDescription(description, function ()
{
socket.emit('description', JSON.stringify({ 'sdp':client.localDescription }))
})
}, function (err)
{
console.log('error: ' + err)
})
}, function(err)
{
console.log('error: ' + err)
})
})
addStream()
})
function createOffer ()
{
if(!client) { return; }
client.createOffer(function (description)
{
console.log(description)
client.setLocalDescription(description, function ()
{
socket.emit('description', JSON.stringify({ 'sdp': client.localDescription }))
console.log('set local description')
})
})
}
function candidate (message)
{
if(message.candidate)
{
console.log('candidate')
client.addIceCandidate(new RTCIceCandidate(message.candidate))
}
}
function addStream ()
{
navigator.webkitGetUserMedia({audio: true, video: false}, function(stream)
{
client.addStream(stream)
})
}
And my signalling part of my server as it currently stands:
io.on 'connection', (socket) ->
socket.on 'description', (data) ->
parsed = JSON.parse data
socket.broadcast.emit 'description', parsed
socket.on 'candidate', (candidate) ->
parsed = JSON.parse candidate
socket.broadcast.emit 'candidate', parsed
I'd appreciate any insight into this. Thanks.

The "PeerConnection" as the name indicates can be used with only one other peer. You cannot cache the offer SDP generated by one PeerConnection instance to use it with more than one other peers.
In your case, you must create a PeerConnection for each browser that you want to send/receive audio and video from and then exchange the corresponding SDP offer and answers with those browsers via your signaling mechanism.
Please feel free to go through some of the links I have mentioned here to understand how WebRTC works.

Related

How do I use promises in a Chrome extension?

What I am trying to do is create a chrome extension that creates new, nested, bookmark folders, using promises.
The function to do this is chrome.bookmarks.create(). However I cannot just
loop this function, because chrome.bookmarks.create is asynchronous. I need to wait until the folder is created, and get its new ID, before going on to its children.
Promises seem to be the way to go. Unfortunately I cannot find a minimal working example using an asynchronous call with its own callback like chrome.bookmarks.create.
I have read some tutorials 1, 2, 3, 4. I have searched stackOverflow but all the questions do not seem to be about plain vanilla promises with the chrome extension library.
I do not want to use a plugin or library: no node.js or jquery or Q or whatever.
I have tried following the examples in the tutorials but many things do not make sense. For example, the tutorial states:
The promise constructor takes one argument—a callback with two
parameters: resolve and reject.
But then I see examples like this:
const wait = ms => new Promise(resolve => setTimeout(resolve, ms));
How this works is a mystery to me.
Also, how can you call resolve() when its never been defined? No example in the tutorials seem to match real life code. Another example is:
function isUserTooYoung(id) {
return openDatabase() // returns a promise
.then(function(col) {return find(col, {'id': id});})
How do I pass in col, or get any results!
So if anyone can give me a minimal working example of promises with an asynchronous function with its own callback, it would be greatly appreciated.
SO wants code, so here is my non-working attempt:
//loop through all
function createBookmarks(nodes, parentid){
var jlen = nodes.length;
var i;
var node;
for(var i = 0; i < nodes.length; i++){
var node = nodes[i];
createBookmark(node, parentid);
}
}
//singular create
function createBookmark(node, parentid){
var bookmark = {
parentId : parentid,
index : node['index'],
title : node['title'],
url : node['url']
}
var callback = function(result){
console.log("creation callback happened.");
return result.id; //pass ID to the callback, too
}
var promise = new Promise(function(resolve, reject) {
var newid = chrome.bookmarks.create(bookmark, callback)
if (newid){
console.log("Creating children with new id: " + newid);
resolve( createBookmarks(bookmark.children, newid));
}
});
}
//allnodes already exists
createBookmarks(allnodes[0],"0");
Just doesn't work. The result from the callback is always undefined, which it should be, and I do not see how a promise object changes anything. I am equally mystified when I try to use promise.then().
var newid = promise.then( //wait for a response?
function(result){
return chrome.bookmarks.create(bookmark, callback);
}
).catch(function(error){
console.log("error " + error);
});
if (node.children) createBookmarks(node.children, newid);
Again, newid is always undefined, because of course bookmarks.create() is asynchronous.
Thank you for any help you can offer.
Honestly, you should just use the web extension polyfill. Manually promisifying the chrome APIs is a waste of time and error prone.
If you're absolutely insistent, this is an example of how you'd promisify chrome.bookmarks.create. For other chrome.* APIs, you also have to reject the callback's error argument.
function createBookmark(bookmark) {
return new Promise(function(resolve, reject) {
try {
chrome.bookmarks.create(bookmark, function (result) {
if (chrome.runtime.lastError) reject(chrome.runtime.lastError)
else resolve(result)
})
} catch (error) {
reject(error)
}
})
}
createBookmark({})
.then(function (result) {
console.log(result)
}).catch(function (error) {
console.log(error)
})
To create multiple bookmarks, you could then:
function createBookmarks(bookmarks) {
return Promise.all(
bookmarks.map(function (bookmark) {
return createBookmark(bookmark)
})
)
}
createBookmarks([{}, {}, {}, {}])
.catch(function (error) {
console.log(error)
})
Take the advantage of the convention that the callback function always be the last argument, I use a simple helper function to promisify the chrome API:
function toPromise(api) {
return (...args) => {
return new Promise((resolve) => {
api(...args, resolve);
});
};
}
and use it like:
toPromise(chrome.bookmarks.create)(bookmark).then(...);
In my use case, it just works most of the time.

Redux saga: yield put not working inside nested callback

const { payload: {loginType, email, password, notification, self} } = action;
console.log("--TRY--");
Firebase.login(loginType, { email, password })
.catch(function(result) {
const message =
result && result.message ? result.message : 'Sorry Some error occurs';
notification('error', message);
self.setState({
confirmLoading: false
});
isError = true;
})
.then(function(result) {
if (isError) {
return;
}
if (!result || result.message) {
const message =
result && result.message
? result.message
: 'Sorry Some error occurs';
notification('error', message);
self.setState({
confirmLoading: false
});
} else {
self.setState({
visible: false,
confirmLoading: false
});
console.log("--RIGHT BEFORE I CHECK AUTH STATE--");
//the following does NOT fire
firebaseAuth().onAuthStateChanged(function*(user) {
console.log("THE GENERATOR RUNS");
if (user) {
console.log(user);
yield put({
type: actions.LOGIN_SUCCESS,
token: 'secret token',
profile: 'Profile'
});
yield put(push('/dashboard'));
}
else {
yield put({ type: actions.LOGIN_ERROR });
}
});
}
}); });
Hi. I'm currently working with redux saga for the first time. I've been trying to get yield put to fire in the callback of the firebaseAuth().onAuthStateChanged listener. The yield keyword won't work in a function that is not an ES6 generator, so I added an asterisk to the callback but now it won't execute at all. Would really appreciate any advice on the matter.
As you noticed, redux-saga effects can only be used within a generator function, and you cannot use a generator function as a regular function: calling a generator function only returns a special object.
The right way to approach this is to use an eventChannel: it lets you connect your saga to a source of events external to the redux ecosystem.
First create your eventChannel using the provided factory function: it hands you an emit function that you can use to emit events; then consume these events using the take effect.
import { eventChannel } from 'redux-saga';
import { cancelled, take } from 'redux-saga/effects';
// first create your eventChannel
const authEventsChannel = eventChannel( emit => {
const unsubscribe = firebaseAuth().onAuthStateChanged( user => {
emit({ user });
});
// return a function that can be used to unregister listeners when the saga is cancelled
return unsubscribe;
});
// then monitor those events in your saga
try {
while (true) {
const { user } = yield take (authEventsChannel);
// handle auth state
}
} finally {
// unregister listener if the saga was cancelled
if (yield cancelled()) authEventsChannel.close();
}

WebRTC video chat for Firefox-to-Chrome and Chrome-to-Firefox not working

I'm trying to implement a very simple video chat based on the WebRTC API.
Unfortunately my Code is just working from Chrome-to-Chrome and from Firefox-to-Firefox so far.
If I try it from Chrome-to-Firefox or from Firefox-to-Chrome I get the following error output:
Failed to set local offer sdp: Session error code: ERROR_CONTENT. Session error description: Failed to set local video description recv parameters..(anonymous function) # helloWebRtc.js:126***
Did I possibly missed something or do I need some flags in the Chrome or Firefox browser?
Do you have any idea? I would be grateful for any help I can get to solve this issue.
Thank you all in advance!
My helloWebRtc.js looks like this:
var localVideo = document.querySelector("#localVideo");
var remoteVideo = document.querySelector("#remoteVideo");
var SIGNAL_ROOM = "signal_room";
var CHAT_ROOM = "chat_room";
var serverConfig = {
"iceServers": [
{
"urls": "stun:stun.l.google.com:19302"
}
]
};
var optionalConfig = {
optional: [
{
RtpDataChannels: true
},
{
DtlsSrtpKeyAgreement: true
}
]
};
var rtcPeerConn,
localStream;
io = io.connect();
io.emit("ready", {"chat_room": CHAT_ROOM, "signal_room": SIGNAL_ROOM});
io.emit("signal", {
"room": SIGNAL_ROOM,
"type": "user_here",
"message": "new user joined the room"
});
io.on("rtcSignaling", function(data) {
if(!rtcPeerConn) {
startSignaling();
}
if(data.type !== "user_here" && data.message) {
var message = JSON.parse(data.message);
if(message.description) {
var remoteDesc = new RTCSessionDescription(message.description);
rtcPeerConn.setRemoteDescription(remoteDesc, function() {
// if we receive an offer we need to answer
if(rtcPeerConn.remoteDescription.type === "offer") {
rtcPeerConn.createAnswer(sendLocalDescription, function(error) {
console.error("error on creating answer", error);
});
}
}, function(error) {
console.error("error on set remote description", error);
});
} else if(message.candidate) {
var iceCandidate = new RTCIceCandidate(message.candidate);
rtcPeerConn.addIceCandidate(iceCandidate);
}
}
});
function startSignaling() {
rtcPeerConn = new RTCPeerConnection(serverConfig, optionalConfig);
//send any ice candidate to the other peer
rtcPeerConn.onicecandidate = function(event) {
if(event.candidate) {
io.emit("signal", {
"room": SIGNAL_ROOM,
"type": "candidate",
"message": JSON.stringify({
"candidate": event.candidate
})
});
}
};
rtcPeerConn.onnegotiationneeded = function() {
rtcPeerConn.createOffer(sendLocalDescription, function(error) {
console.error("error on creating offer", error);
});
};
// add the other peer's stream
rtcPeerConn.onaddstream = function(event) {
console.info("on add stream called");
remoteVideo.srcObject = event.stream;
};
// add local stream
navigator.mediaDevices.getUserMedia({
audio: true,
video: true
})
.then(function(stream) {
localVideo.srcObject = stream;
localStream = stream;
rtcPeerConn.addStream(localStream);
})
.catch(function(e) {
alert('getUserMedia() error: ' + e.name);
});
}
function sendLocalDescription(description) {
rtcPeerConn.setLocalDescription(
description,
function() {
io.emit("signal", {
"room": SIGNAL_ROOM,
"type": "description",
"message": JSON.stringify({
"description": rtcPeerConn.localDescription
})
});
},
function(error) {
console.error("error to set local desc", error);
}
);
}
My NodeJS server (using express.io) looks like the following:
var express = require('express.io');
var app = express();
var PORT = 8686;
app.http().io();
console.log('server started # localhost:8686');
// declaring folders to access i.e.g html files
app.use(express.static(__dirname + '/views'));
app.use('/scripts', express.static(__dirname + '/scripts'));
// root url i.e. "localhost:8686/"
app.get('/', function(req, res) {
res.sendFile('index.html');
});
/**
* Socket.IO Routes for signaling pruposes
*/
app.io.route('ready', function(req) {
req.io.join(req.data.chat_room);
req.io.join(req.data.signal_room);
app.io.room(req.data.chat_room).broadcast('announce', {
message: 'New client in the ' + req.data.chat_room + ' room.'
});
});
app.io.route('send', function(req) {
app.io.room(req.data.room).broadcast('message', {
message: req.data.message,
author: req.data.author
});
});
app.io.route('signal', function(req) {
// Note: req means just broadcasting without letting the sender also receive their own message
if(req.data.type === "description" || req.data.type === "candidate")
req.io.room(req.data.room).broadcast('rtcSignaling', {
type: req.data.type,
message: req.data.message
});
else
req.io.room(req.data.room).broadcast('rtcSignaling', {
type: req.data.type
});
});
app.listen(PORT);
You can compare the offer SDP generated by the chrome and firefox, there might be some difference which is not interoperable to other.
Edit to the old answer below: there are several bugs in interoperability between Chrome and Firefox. Someone from the webrtc team gave me the suggestion to keep the offerer to the same party. So if A creates an offer when setting up a stream to B, then B asks A to create a new offer, instead of creating one self, when setting up a stream to A.
See also here:
https://bugs.chromium.org/p/webrtc/issues/detail?id=5499#c15
I did note that if Firefox initiates a session, Chrome will kick the stream coming from Firefox out of the video element but you can create a new object URL on the stream and set it as the source.
Hope that helps.
Old message:
I am experiencing the same thing, so if you have an answer, I'm curious.
I do believe that there is a mismatch (bug) between FireFox and Chrome in setting up DTLS roles, see also:
https://bugs.chromium.org/p/webrtc/issues/detail?id=2782#c26
just check if you are setting DtlsSrtpKeyAgreement parameter to true while you create the peerconnection.
pc = new RTCPeerConnection(pc_config,{optional: [{RtpDataChannels: true},{
DtlsSrtpKeyAgreement: true}]});

DevExtreme datasource can't load Data Service data

I tried to accomplish the tutorial here, and when I used their data service, it worked just fine.
I modified the source to my data service (WCF Data Service v5.6, OData V2), and the list just shows the Loading sign and nothing happens.
The code should load any data type, it just has to be mapped accordingly. My service is availabe through the browser, I checked.
Here is the code:
DevExTestApp.home = function (params) {
var viewModel = {
dataSource: DevExpress.data.createDataSource({
load: function (loadOptions) {
if (loadOptions.refresh) {
try {
var deferred = new $.Deferred();
$.get("http://192.168.1.101/dataservice/dataservice.svc/People")
.done(function (result) {
var mapped = $.map(result, function (data) {
return {
name: data.Name
}
});
deferred.resolve(mapped);
});
}
catch (err) {
alert(err.message);
}
return deferred;
}
}
})
};
return viewModel;
}
What else should I set?
The try-catch block would not help is this case, because data loading is async. Instead, subscribe to the fail callback:
$.get(url)
.done(doneFunc)
.fail(failFunc);
Another common problem with accessing a web service from JavaScript is Same-Origin Policy. Your OData service have to support either CORS or JSONP. Refer to this discussion.

peerConnection.addIceCandidate giving error: invalid string

I am trying to implement a Voice-only WebRTC app. I am running it on Chrome Version 29.0.1547.0 dev. My app uses Socket.IO for the signaling mechanism.
peerConnection.addIceCandidate() is giving me this error: Uncaught SyntaxError: An invalid or illegal string was specified.
and separately, peerConnection.setRemoteDescription(); is giving me this error: Uncaught TypeMismatchError: The type of an object was incompatible with the expected type of the parameter associated to the object.
Here's my code:
SERVER (in CoffeeScript)
app = require("express")()
server = require("http").createServer(app).listen(3000)
io = require("socket.io").listen(server)
app.get "/", (req, res) -> res.sendfile("index.html")
app.get "/client.js", (req, res) -> res.sendfile("client.js")
io.sockets.on "connection", (socket) ->
socket.on "message", (data) ->
socket.broadcast.emit "message", data
CLIENT (in JavaScript)
var socket = io.connect("http://localhost:3000");
var pc = new webkitRTCPeerConnection({
"iceServers": [{"url": "stun:stun.l.google.com:19302"}]
});
navigator.getUserMedia = navigator.webkitGetUserMedia ||
navigator.mozGetUserMedia;
navigator.getUserMedia({audio: true}, function (stream) {
pc.addStream(stream);
}, function (error) { console.log(error); });
pc.onicecandidate = function (event) {
if (!event || !event.candidate) return;
socket.emit("message", {
type: "iceCandidate",
"candidate": event.candidate
});
};
pc.onaddstream = function(event) {
var audioElem = document.createElement("audio");
audioElem.src = webkitURL.createObjectURL(event.stream);
audioElem.autoplay = true;
document.appendChild(audioElem);
console.log("Got Remote Stream");
};
socket.on("message", function(data) {
if (data.type === "iceCandidate") {
console.log(data.candidate);
candidate = new RTCIceCandidate(data.candidate);
console.log(candidate);
pc.addIceCandidate(candidate);
} else if (data.type === "offer") {
pc.setRemoteDescription(data.description);
pc.createAnswer(function(description) {
pc.setLocalDescription(description);
socket.emit("message", {type: "answer", description: description});
});
} else if (data.type === "answer") {
pc.setRemoteDescription(data.description);
}
});
function offer() {
pc.createOffer( function (description) {
pc.setLocalDescription(description);
socket.emit("message", {type: "offer", "description": description});
});
};
The HTML just contains a button that calls offer().
I can confirm the ICECandidates and SessionDescriptions are transferring successfully from one client to the other.
What am I doing wrong? And how should I fix these and any other errors so that I can transfer audio from one client to the other?
PS: If you know about a good source documenting the WebRTC API (except the W3C documentation), please tell me about it!
Thanks!
For that error the point is, ICE Candidates must be added only after successfully setting remote description.
Note that just after creating Offer (by Offerer), ice candidates are produced immediately. So, if somehow the answerer receives those candidates, before setting remote description (which in theory would arrive before candidates), you get error.
The same is true for offerer. It must set remote description before adding any ice candidate.
I see that in your javascript code you are not guaranteeing that remote description is set before adding ice candidates.
First of all you can check just before pc.addIceCandidate(candidate); if pc's remoteDescription is set. If you see that it is null (or undefined), you can locally store received ice candidates to add them after setting remoteDescription (or wait in offerer to send them in proper time.)