so i switched to HTTPS, everything worked nicely exept the sockets.
If I try to visit website with the HTTP, the sockets connect, but if I try to connect with HTTPS I get:
The Error on Console:
Failed to load resource: net::ERR_CONNECTION_CLOSED
Failed to load resource: net::ERR_CONNECTION_CLOSED
Failed to load resource: net::ERR_CONNECTION_CLOSED
Failed to load resource: net::ERR_CONNECTION_CLOSED
Frontend:
function connect()
{
if (!SOCKET)
{
var hash = getCookie('hash');
if (hash == "") {
//$.notify('You must login!', 'success');
}
if (hash != "") {
$.notify('Connecting...', 'success');
}
SOCKET = io(':3001');
SOCKET.on('connect', function(msg) {
if (hash != "") {
//$.notify('Connected!', 'success');
}
SOCKET.emit('hash', {
hash: hash
});
$('#games tr').remove();
});
SOCKET.on('connect_error', function(msg) {
$.notify('Connection lost!', 'success');
});
SOCKET.on('message', function(msg) {
onMessage(msg);
});
SOCKET.on('disconnect', function(m) {
SOCKET.emit('disconnect', {
uhash: hash
});
});
}
else
{
console.log("Error: connection already exists.");
}
}
Node.js/Backend
var httpsOptions = {
cert: fs.readFileSync("/path/to/cert/cert.pem"),
ca: fs.readFileSync("/path/to/cert/chain.pem"),
key: fs.readFileSync("/path/to/cert/privkey.pem"),
}
var server = require('https').createServer(httpsOptions);
var io = require('socket.io').listen(server);
server.listen(3001);
Your Node application should only accept HTTP connections. Your Apache server should be responsible for the HTTPS. This simplifies your app, and allows Apache to load balance for you if you ever decide to do so.
So between your app and Apache should be HTTP, and then HTTPS between Apache and the client. There are many guides on how to do this, I suggest reading the official documentation
If you decide to go against this, and implement SSL at the web app level (not recommended), then we'll need more information.
Related
In chrome extension, I use native messaging to call the local application. But I found an issue that everytime I reload the extension, it seems like a new process is created for the application. By the documentation, the app will be ended if the port is disconnected or the page is closed. Does that mean reload extension won't close the background page? How can I solve this problem? Also, I cannot find my local application process in the chrome task manager.
// background.js
var port = null;
connectToNativeHost();
// Receive message from other js
chrome.runtime.onMessage.addListener(
function(request, sender, sendResponse) {
console.log("background recieved message from " + sender.url + JSON.stringify(request));
parseMessage(request);
}
);
//onNativeDisconnect
function onDisconnected()
{
console.log(chrome.runtime.lastError);
console.log('disconnected from native app.');
port = null;
}
// Receive message from native app
function onNativeMessage(message)
{
console.log('recieved message from native app: ' + JSON.stringify(message));
}
//connect to native host and get the communicatetion port
function connectToNativeHost()
{
var nativeHostName = 'com.group_project.time_tracker';
port = chrome.runtime.connectNative(nativeHostName);
port.onMessage.addListener(onNativeMessage);
port.onDisconnect.addListener(onDisconnected);
console.log("connected");
}
// Send message to native app
function sendMessage(message)
{
port.postMessage(message);
console.log('send messsage to native app: ' + JSON.stringify(message));
}
OBJECTIVE:
I'm building a FPV robot, I want to control it with a with a webbrowser over a local wi-fi connection.
I'm using a raspberry pi 3B+ with Raspbian Stretch. I built my own motor control and power regulator hat.
After lots of research testing, I decided to use node.JS as http server and socket.io to provide a low latency bidirectional communication with my robot. This stack achieve about 7ms of latency.
Picture of the robot
PROBLEM:
I need to stream low latency video from an USB camera attached to the RPI to the browser. My target is to achieve at least 640x480 resolution at 10FPS with 50ms of latency or better. I'm happy sacrificing visual fedelity to get a quicker response from my robot.
If possible I would like to stream in UDP to improve the reliability of the stream.
If possible I would like to stream a video that modern webbrowsers can natively decode. I'd like to use a H264 codec and the HTML5 video tag.
I can fall back to use a javascript player if there is no other option.
WHAT I TRIED:
I did an extensive research and tried many tools.
Amongst other, I tried VLC, mjpg streamer, gstreamer and raspivid. A few times I got to a stream the webbrowser could view, but at best I got a latency of 700ms at 320x240. Very very far from my target.
Currently I'm looking into WebRTC solutions.
QUESTION:
I'd like suggestions for NODE.JS packages or other solutions to provide a UDP H264 video stream that can be decoded by an HTML5 video tag with a target latency of 50ms.
Thanks
UPDATE:
Thanks for your answers! I'll keep updating this question and I'll post the solution once it works.
PUSH INDIVIDUAL FRAMES
I tried a different approach by pushing individual 200KB 640x480 jpg frame through websocket and I got a latency of about 190ms. I can probably do a lot better by reusing objects but I'm putting this attempt in hold for now.
UPDATE2:
While researching WebRTC I found a stack that looked easy enough.
Server side it uses V4L2 as driver, FFMPEG to transcode into an MPEG1 http stream with TS encapsulation locally, node js to flip the stream into a websocket.
Client side there is a javascript that decode the MPEG1 TS stream and paint a canvas object into the HTML page.
It achieves 640x480#20FPS with 240mS of latency.
Good enough for an MVP, but I'll keep working to get it down.
Code in the answer.
I adapted code from here and integrated it with an http server and socket.io controls:
https://github.com/phoboslab/jsmpeg
Server:
V4L2 -> FFMPEG (MPEG1 TS) -> NODE HTTP Server -> NODE Websocket broadcast
Client:
Websocket -> Javascript (Decode MPEG1 TS and paint to html canvas) -> Html Canvas
This stack achieve 640x480#20FPS with 240ms of latency. Still far from my target but good enough as MVP. The controls in both directions have a latency of 7ms, which is excellent.
This stack is held back by the transcoding and decoding stage, and the RPI gets really hot. The transport of raw data through websocket looks good, I'm going to profile the latency of each steps in the future.
Execution:
pi#MazeRunner:~ $ node node.js &
pi#MazeRunner:~ $ ffmpeg -f v4l2 -framerate 20 -video_size 640x480 -i /dev/video0 -f mpegts -codec:v mpeg1video -s 640x480 -b:v 600k -bf 0 http://localhost:8080/mystream
Server side NODE.JS
//operating system library. Used to get local IP address
var os = require("os");
//file system library. Used to load file stored inside back end server (https://nodejs.org/api/fs.html)
var fs = require("fs");
//http system library. Handles basic html requests
var http = require("http").createServer(http_handler);
//url library. Used to process html url requests
var url = require("url");
//Websocket
var io = require("socket.io")(http);
//Websocket used to stream video
var websocket = require("ws");
//-----------------------------------------------------------------------------------
// CONFIGURATION
//-----------------------------------------------------------------------------------
//Port the server will listen to
var server_port = 8080;
var websocket_stream_port = 8082;
//Path of the http and css files for the http server
var file_index_name = "index.html";
var file_css_name = "style.css";
var file_jsplayer_name = "jsmpeg.min.js";
//Http and css files loaded into memory for fast access
var file_index;
var file_css;
var file_jsplayer;
//Name of the local video stream
var stream_name = "mystream";
//-----------------------------------------------------------------------------------
// DETECT SERVER OWN IP
//-----------------------------------------------------------------------------------
//If just one interface, store the server IP Here
var server_ip;
//Get local IP address of the server
//https://stackoverflow.com/questions/3653065/get-local-ip-address-in-node-js
var ifaces = os.networkInterfaces();
Object.keys(ifaces).forEach
(
function (ifname)
{
var alias = 0;
ifaces[ifname].forEach
(
function (iface)
{
if ('IPv4' !== iface.family || iface.internal !== false)
{
// skip over internal (i.e. 127.0.0.1) and non-ipv4 addresses
return;
}
if (alias >= 1)
{
// this single interface has multiple ipv4 addresses
console.log('INFO: Server interface ' +alias +' - ' + ifname + ':' + alias, iface.address);
}
else
{
server_ip = iface.address;
// this interface has only one ipv4 adress
console.log('INFO: Server interface - ' +ifname, iface.address);
}
++alias;
}
);
}
);
//-----------------------------------------------------------------------------------
// HTTP SERVER
//-----------------------------------------------------------------------------------
// Fetch and serves local files to client
//Create http server and listen to the given port
http.listen
(
server_port,
function( )
{
console.log('INFO: ' +server_ip +' listening to html requests on port ' +server_port);
//Pre-load http, css and js files into memory to improve http request latency
file_index = load_file( file_index_name );
file_css = load_file( file_css_name );
file_jsplayer = load_file( file_jsplayer_name );
}
);
//-----------------------------------------------------------------------------------
// HTTP REQUESTS HANDLER
//-----------------------------------------------------------------------------------
// Answer to client http requests. Serve http, css and js files
function http_handler(req, res)
{
//If client asks for root
if (req.url == '/')
{
//Request main page
res.writeHead( 200, {"Content-Type": detect_content(file_index_name),"Content-Length":file_index.length} );
res.write(file_index);
res.end();
console.log("INFO: Serving file: " +req.url);
}
//If client asks for css file
else if (req.url == ("/" +file_css_name))
{
//Request main page
res.writeHead( 200, {"Content-Type": detect_content(file_css_name),"Content-Length" :file_css.length} );
res.write(file_css);
res.end();
console.log("INFO: Serving file: " +req.url);
}
//If client asks for css file
else if (req.url == ("/" +file_jsplayer_name))
{
//Request main page
res.writeHead( 200, {"Content-Type": detect_content(file_jsplayer_name),"Content-Length" :file_jsplayer.length} );
res.write(file_jsplayer);
res.end();
console.log("INFO: Serving file: " +req.url);
}
//Listening to the port the stream from ffmpeg will flow into
else if (req.url = "/mystream")
{
res.connection.setTimeout(0);
console.log( "Stream Connected: " +req.socket.remoteAddress + ":" +req.socket.remotePort );
req.on
(
"data",
function(data)
{
streaming_websocket.broadcast(data);
/*
if (req.socket.recording)
{
req.socket.recording.write(data);
}
*/
//console.log("broadcast: ", data.length);
}
);
req.on
(
"end",
function()
{
console.log("local stream has ended");
if (req.socket.recording)
{
req.socket.recording.close();
}
}
);
}
//If client asks for an unhandled path
else
{
res.end();
console.log("ERR: Invalid file request" +req.url);
}
}
//-----------------------------------------------------------------------------------
// WEBSOCKET SERVER: CONTROL/FEEDBACK REQUESTS
//-----------------------------------------------------------------------------------
// Handle websocket connection to the client
io.on
(
"connection",
function (socket)
{
console.log("connecting...");
socket.emit("welcome", { payload: "Server says hello" });
//Periodically send the current server time to the client in string form
setInterval
(
function()
{
socket.emit("server_time", { server_time: get_server_time() });
},
//Send every 333ms
333
);
socket.on
(
"myclick",
function (data)
{
timestamp_ms = get_timestamp_ms();
socket.emit("profile_ping", { timestamp: timestamp_ms });
console.log("button event: " +" client says: " +data.payload);
}
);
//"ArrowLeft"
socket.on
(
"keyboard",
function (data)
{
timestamp_ms = get_timestamp_ms();
socket.emit("profile_ping", { timestamp: timestamp_ms });
console.log("keyboard event: " +" client says: " +data.payload);
}
);
//profile packets from the client are answer that allows to compute roundway trip time
socket.on
(
"profile_pong",
function (data)
{
timestamp_ms_pong = get_timestamp_ms();
timestamp_ms_ping = data.timestamp;
console.log("Pong received. Round trip time[ms]: " +(timestamp_ms_pong -timestamp_ms_ping));
}
);
}
);
//-----------------------------------------------------------------------------------
// WEBSOCKET SERVER: STREAMING VIDEO
//-----------------------------------------------------------------------------------
// Websocket Server
var streaming_websocket = new websocket.Server({port: websocket_stream_port, perMessageDeflate: false});
streaming_websocket.connectionCount = 0;
streaming_websocket.on
(
"connection",
function(socket, upgradeReq)
{
streaming_websocket.connectionCount++;
console.log
(
'New websocket Connection: ',
(upgradeReq || socket.upgradeReq).socket.remoteAddress,
(upgradeReq || socket.upgradeReq).headers['user-agent'],
'('+streaming_websocket.connectionCount+" total)"
);
socket.on
(
'close',
function(code, message)
{
streaming_websocket.connectionCount--;
console.log('Disconnected websocket ('+streaming_websocket.connectionCount+' total)');
}
);
}
);
streaming_websocket.broadcast = function(data)
{
streaming_websocket.clients.forEach
(
function each(client)
{
if (client.readyState === websocket.OPEN)
{
client.send(data);
}
}
);
};
//-----------------------------------------------------------------------------------
// FUNCTIONS
//-----------------------------------------------------------------------------------
//-----------------------------------------------------------------------------------
// SERVER DATE&TIME
//-----------------------------------------------------------------------------------
// Get server time in string form
function get_server_time()
{
my_date = new Date();
return my_date.toUTCString();
}
//-----------------------------------------------------------------------------------
// TIMESTAMP
//-----------------------------------------------------------------------------------
// Profile performance in ms
function get_timestamp_ms()
{
my_date = new Date();
return 1000.0* my_date.getSeconds() +my_date.getMilliseconds()
}
//-----------------------------------------------------------------------------------
// FILE LOADER
//-----------------------------------------------------------------------------------
// Load files into memory for improved latency
function load_file( file_name )
{
var file_tmp;
var file_path = __dirname +"/" +file_name;
//HTML index file
try
{
file_tmp = fs.readFileSync( file_path );
}
catch (err)
{
console.log("ERR: " +err.code +" failed to load: " +file_path);
throw err;
}
console.log("INFO: " +file_path +" has been loaded into memory");
return file_tmp;
}
//-----------------------------------------------------------------------------------
// CONTENT TYPE DETECTOR
//-----------------------------------------------------------------------------------
// Return the right content type to give correct information to the client browser
function detect_content( file_name )
{
if (file_name.includes(".html"))
{
return "text/html";
}
else if (file_name.includes(".css"))
{
return "text/css";
}
else if (file_name.includes(".js"))
{
return "application/javascript";
}
else
{
throw "invalid extension";
}
}
Client Side html
<!DOCTYPE html>
<meta charset="utf-8"/>
<html>
<head>
<title>Maze Runner</title>
<link rel="stylesheet" href="style.css">
<script type="text/javascript" src="/socket.io/socket.io.js"></script>
<script type="text/javascript">
var host_ip = document.location.hostname;
console.log("connecting to host: ", host_ip);
//Get references to the html controls
textbox_input1 = window.document.getElementById("my_text_box")
//Connect to the server via websocket
var mysocket = io("http://" +host_ip +":8080");
//Long lived frame object
var last_frame;
//-----------------------------------------
// CONNESSION ACKNOWLEDGE
//-----------------------------------------
// Link is initiated by the client
// Server sends a welcome message when link is estabilished
// Server could send an auth token to keep track of individual clients and login data
mysocket.on
(
"welcome",
(message) =>
{
console.log("Server websocket connession acknoweldged... " +message.payload);
}
)
//-----------------------------------------
// SERVER->CLIENT CONTROLS
//-----------------------------------------
// Server can send an async message to dinamically update the page without reloading
// This is an example message with the server local date and time in string form
mysocket.on
(
"server_time",
(message) =>
{
fill_label( message.server_time );
console.log("Server sent his local time... " +message.server_time);
}
)
function fill_label( payload )
{
textbox_input1.value=payload;
}
//-----------------------------------------
// CLIENT->SERVER CONTROLS
//-----------------------------------------
// Controls inside the webpage can emit async events to the server
// In this example I have a push button and I catch keyboard strokes
//Handler for a pushbutton
function socket_button_handler()
{
mysocket.emit("myclick", { payload: "button was clicked" });
console.log("Button was clicked...");
}
//Listen for keystrokes
window.document.addEventListener
(
"keypress",
function onEvent(event)
{
//Inform the server that a key has been pressed
mysocket.emit("keyboard", { payload: event.key });
console.log("Key press...");
}
);
//-----------------------------------------
// PING-PONG
//-----------------------------------------
// Server sends ping messages with a timestamp
// Client answers with pongs to allow server to profile latency of the channel
//profile messages means the server wants to compute roundway trip
mysocket.on
(
"profile_ping",
(message) =>
{
//Answer back with the received timestamp so that server can compute roundway trip
mysocket.emit("profile_pong", { timestamp: message.timestamp });
console.log( "server wants a pong. server absolute timestamp[ms]: " +message.timestamp );
}
);
</script>
</head>
<body>
<h1>Html+Css Server +low latency Websocket server</h1>
<!-- button control with socket emitter as handler -->
<p> This button will emit a websocket event. The server will be informed in real time of the event. </p>
<button id="my_button" type="button" onclick="socket_button_handler()">Websocket Button!</button>
<!-- input text control -->
<p> This input can be filled through websockets directly by the server in real time </p>
<input id="my_text_box" type="text" value="" size="40">
<!-- canvas object, it's painted by the javascript video decoder -->
<p> This canvas is painted by the javascript player and shows the live stream.'</p>
<canvas id="video-canvas" width=640 height=480></canvas>
<!-- Javascript video decoder, take in a data stream from a websocket and paint on a canvas -->
<script type="text/javascript" src="jsmpeg.min.js"></script>
<script type="text/javascript">
var mycanvas = document.getElementById("video-canvas");
var url = "ws://" + host_ip +":8082/";
var player = new JSMpeg.Player(url, {canvas: mycanvas});
</script>
</body>
</html>
Javascript Player
You can get the javascript player I used from here:
https://github.com/phoboslab/jsmpeg/blob/master/jsmpeg.min.js
I’d like suggestions for NODE.JS packages or other solutions to provide a UDP H264 video stream that can be decoded by an HTML5 video tag with a target latency of 50ms.
That’s Almost certainly not possible in that configuration.
If you drop the video tag requirement, and use just straight WebRTC in the browser, you may be able to get down to about 150ms.
I'm trying to make my OpenShift Node.js app working, but the WS connection is not working. Client error is: connection refused.
Client side factory service:
var dataStream = $websocket(localStorageService.get('wsUrl'))
dataStream.onMessage(function(message) {
var call = JSON.parse(message.data)
if (fnMap[call.fn]) {
fnMap[call.fn](call.event, call.data)
}
})
dataStream.onError(function(err) {
console.log(err)
})
dataStream.onClose(function(event){
console.log('event: ' + JSON.stringify(event))
})
var fnMap = {
"broadcastResult": function(event, data) {
$rootScope.$broadcast(event, data)
}
}
var methods = {
callFn: function(paramJSON) {
dataStream.send(JSON.stringify(paramJSON));
}
}
return methods
I'm trying to connect on the following URL: ws://myapp-myname.rhcloud.com:8000
Could you please help?
Thank you in advance,
Csaba
First, you may want to make sure the websocket library, ws, is installed by declaring it in the dependencies section of your package.json and then doing another git push.
Otherwise, try the example provided in the openshift blog: https://blog.openshift.com/paas-websockets/
Specifically:
var websocket = new WebSocket("ws://myapp-myname.rhcloud.com:8000");
websocket.onopen = function(event) {
// The connection was opened
console.log(event)
};
websocket.onclose = function(event) {
// The connection was closed
console.log(event)
};
websocket.onmessage = function(event) {
// New message arrived
message = event.data
console.log(event)
};
websocket.onerror = function(event) {
// There was an error with your WebSocket
console.log(event)
};
The above at least might help point you toward a specific part of your code that's not working. Or, if the example isn't even working, it could be possibly something wrong with your cartridge, and you could try reaching out to RedHat's support.
I am trying to implement a Voice-only WebRTC app. I am running it on Chrome Version 29.0.1547.0 dev. My app uses Socket.IO for the signaling mechanism.
peerConnection.addIceCandidate() is giving me this error: Uncaught SyntaxError: An invalid or illegal string was specified.
and separately, peerConnection.setRemoteDescription(); is giving me this error: Uncaught TypeMismatchError: The type of an object was incompatible with the expected type of the parameter associated to the object.
Here's my code:
SERVER (in CoffeeScript)
app = require("express")()
server = require("http").createServer(app).listen(3000)
io = require("socket.io").listen(server)
app.get "/", (req, res) -> res.sendfile("index.html")
app.get "/client.js", (req, res) -> res.sendfile("client.js")
io.sockets.on "connection", (socket) ->
socket.on "message", (data) ->
socket.broadcast.emit "message", data
CLIENT (in JavaScript)
var socket = io.connect("http://localhost:3000");
var pc = new webkitRTCPeerConnection({
"iceServers": [{"url": "stun:stun.l.google.com:19302"}]
});
navigator.getUserMedia = navigator.webkitGetUserMedia ||
navigator.mozGetUserMedia;
navigator.getUserMedia({audio: true}, function (stream) {
pc.addStream(stream);
}, function (error) { console.log(error); });
pc.onicecandidate = function (event) {
if (!event || !event.candidate) return;
socket.emit("message", {
type: "iceCandidate",
"candidate": event.candidate
});
};
pc.onaddstream = function(event) {
var audioElem = document.createElement("audio");
audioElem.src = webkitURL.createObjectURL(event.stream);
audioElem.autoplay = true;
document.appendChild(audioElem);
console.log("Got Remote Stream");
};
socket.on("message", function(data) {
if (data.type === "iceCandidate") {
console.log(data.candidate);
candidate = new RTCIceCandidate(data.candidate);
console.log(candidate);
pc.addIceCandidate(candidate);
} else if (data.type === "offer") {
pc.setRemoteDescription(data.description);
pc.createAnswer(function(description) {
pc.setLocalDescription(description);
socket.emit("message", {type: "answer", description: description});
});
} else if (data.type === "answer") {
pc.setRemoteDescription(data.description);
}
});
function offer() {
pc.createOffer( function (description) {
pc.setLocalDescription(description);
socket.emit("message", {type: "offer", "description": description});
});
};
The HTML just contains a button that calls offer().
I can confirm the ICECandidates and SessionDescriptions are transferring successfully from one client to the other.
What am I doing wrong? And how should I fix these and any other errors so that I can transfer audio from one client to the other?
PS: If you know about a good source documenting the WebRTC API (except the W3C documentation), please tell me about it!
Thanks!
For that error the point is, ICE Candidates must be added only after successfully setting remote description.
Note that just after creating Offer (by Offerer), ice candidates are produced immediately. So, if somehow the answerer receives those candidates, before setting remote description (which in theory would arrive before candidates), you get error.
The same is true for offerer. It must set remote description before adding any ice candidate.
I see that in your javascript code you are not guaranteeing that remote description is set before adding ice candidates.
First of all you can check just before pc.addIceCandidate(candidate); if pc's remoteDescription is set. If you see that it is null (or undefined), you can locally store received ice candidates to add them after setting remoteDescription (or wait in offerer to send them in proper time.)
I try to use SecureSocket but SecureSocket.isSupported == false.
When I use simple Socket everything is ok.
Did anybody use SecureSocket?
here is my code:
Security.allowDomain('');
Security.allowInsecureDomain("");
Security.loadPolicyFile("xmlsocket://" + host + ':' + port + "/crossdomain.xml");
if(SecureSocket.isSupported)
{
c = new SecureSocket();
receiveBuffer = new ByteArray();
receiveBuffer.endian = Endian.LITTLE_ENDIAN;
c.addEventListener(Event.CLOSE, closeHandler);
c.addEventListener(IOErrorEvent.IO_ERROR, ioErrorHandler);
c.addEventListener(SecurityErrorEvent.SECURITY_ERROR, securityErrorHandler);
c.addEventListener(Event.CONNECT, connectHandler);
}
else
{
try
{
c = new SecureSocket();
}
catch(e:Error)
{
trace(e.toString());
}
}
Later I have error:
[SWF] /assets/flash/ssl/Main.swf - 63,146 bytes after decompression
Error: Request for resource at tlssocket://game9.lgr.su:8081 by requestor from https://game9.lgr.su/assets/flash/ssl/Main.swf has failed because the server cannot be reached.
* Security Sandbox Violation *
Connection to game9.lgr.su:8081 halted - not permitted from https://game9.lgr.su/assets/flash/ssl/Main.swf
The problem was on server side. Our server side on Node JS was listening for NOT SECURE SOCKET but was getting SECURE connection. So when server responded with NOT encrypted content Socket on client side just closed.
So make better server.