I'm trying to use server sent events so my webpage can have periodic time updates from my server. The issue is that my client is able to interact with my server, however, the response from my server is not making it to my client? Basically, when I open my html file with firefox I know my server gets the request and then it starts sending responses, but nothing shows up on my webpage... Not quite sure what's the issue. Help appreciated!
Here is my client code:
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8" />
</head>
<script>
function init(){
if(typeof(EventSource)!=="undefined"){
var source = new EventSource('localhost');
source.onmessage = function(e) {
document.body.innerHTML += e.data + '<br>';
};
}
else{
document.body.innerHTML = "Sorry, your browser does not support server-sent events...";
}
}
</script>
<body onload="init()">
</body>
</html>
Here is my server code (node.js):
var http = require('http');
var sys = require('sys');
var fs = require('fs');
http.createServer(function(req, res) {
if (req.headers.accept && req.headers.accept == 'text/event-stream') {
sendSSE(req, res);
}).listen(80, "127.0.0.1");
function sendSSE(req, res) {
res.writeHead(200, {
'Content-Type': 'text/event-stream',
'Cache-Control': 'no-cache',
'Connection': 'keep-alive'
});
var id = (new Date()).toLocaleTimeString();
// Sends a SSE every 5 seconds on a single connection.
setInterval(function() {
constructSSE(res, id, (new Date()).toLocaleTimeString());
}, 5000);
}
function constructSSE(res, id, data) {
res.write('id: ' + id + '\n');
res.write("data: " + data + '\n\n');
}
You need to add in
res.end();
somewhere and remove the 'setInterval'. It looks like what you're trying to do, however, is keep the connection alive, in which case you need to change your code significantly. Look into the 'net' module, which is more designed for a "constant on" interactive type of connection.
http://nodejs.org/api/net.html#net_net
The http module is designed for finite, exchange of data, type of requests. You're trying to make it do something it isn't designed to do.
/*jshint node:true */
var http = require('http');
http.createServer(function (req, res) {
'use strict';
console.log('GOOD request recieved');
res.write('hi there');
res.end();
console.log('GOOD end sent');
}).listen(8888);
http.createServer(function (req, res) {
'use strict';
console.log('BAD request received');
res.write('hi there');
console.log('BAD response wrote not ending');
}).listen(8889);
Consider the two servers I have above. If you ping them both with node client code, you will see data come in to both, and should see chunks as they are sent. However, if you try to ping 8889 with a browser, the webpage will never render, because the end event is never sent. Browsers depend on this to know all content has been received. If you're client code is working within a browswer, this may be effecting things. Attempt to use plain NodeJS client code against your server first, and make sure data is getting sent the way you expect it is. Then work on figuring out how the browser is upsetting things. My guess would be that the data gets received by the browser, but it never does anything with it, and sits and waits to distribute it for that 'end' event, the same way in which the 8889 server's webpage never renders... it believes there is more data to wait for.
Sample client code:
var options = {
hostname: '127.0.0.1',
method: 'GET'
};
options.port = 8888;
http.request(options, function (res) {
'use strict';
console.log('GOOD Pinged server');
res.on('data', function (chunk) {
console.log('GOOD data chunk:' + chunk);
});
res.on('end', function () {
console.log('GOOD end event recieved');
});
}).end();
options.port = 8889;
http.request(options, function (res) {
'use strict';
console.log('BAD Pinged server');
res.on('data', function (chunk) {
console.log('BAD data chunk:' + chunk);
});
res.on('end', function () {
console.log('BAD end event recieved');
});
}).end();
The http module in node works fine for Server Sent Events and most demo code for Node SSE uses it. But you do have to be careful about some gotchas.
I got bit by compression being turned on. The solution there is to add res.flush() after the last res.write() of each data message. Since that's an easy fix, I would try that first before rewriting using a different module.
Related
I have an express server running which is sending data with res.write()
Fetcher.js
function fetcher() {
console.log("called fetcher")
fetch('/units.html',
{
method: "POST",
body: JSON.stringify({
"NAME": document.getElementById("NAME").value,
}),
//headers: { "Content-Type": "application/json" }
}
)
.then(function(response) {
console.log("inside middle function")
return response.text();
})
.then(function(data) {
console.log(data)
});
}
index.js
app.post('/units.html', function(req, res, next) {
res.write("hello once")
res.write("hello twice");
res.end()
});
I want to do something every time I run res.write.
I see "hello oncehello twice" in browser console, which tells me that all the data is passed after res.end is executed.
Is there a way to execute console.log whenever res.write is executed?
Server => res.write("hello once")
Client => console.log("hello once")
Server => res.write("hello twice")
Client => console.log("hello twice")
Server => res.end()
Modern browsers (Chrome, Edge, Firefox) do allow you to get access to the incoming stream retrieved by a fetch() call. The MDN doc is here, but I found that quite confusing and had to use several other internet sources to figure out how to make it work.
Here's a simple example.
First, the server that writes some strings to a response separated by a delay when /stream is requested:
const express = require('express');
const app = express();
const path = require('path');
function delay(t, v) {
return new Promise(resolve => {
setTimeout(resolve, t, v);
});
}
app.get("/", (req, res) => {
res.sendFile(path.resolve("temp.html"));
});
// output separate pieces of data with a delay between them
app.get("/stream", async (req, res) => {
// must set content-type for this to work in Firefox
res.setHeader('Content-Type', 'text/plain');
res.write("hello1");
await delay(1000);
res.write("hello2");
await delay(1000);
res.write("hello3");
await delay(1000);
res.end("hello4");
});
app.listen(80);
Then, the "/" HTML page from this web server, that page then makes a request to /stream:
<!doctype html>
<html lang="en">
<head>
<meta charset="UTF-8">
<link rel="icon" href="data:,">
</head>
<body>
This is an empty page
<script>
function Decodeuint8arr(uint8array){
return new TextDecoder("utf-8").decode(uint8array);
}
async function run() {
const response = await fetch('/stream');
const reader = response.body.getReader();
while (true) {
const { value, done } = await reader.read();
if (done) break;
console.log(`Received: "${Decodeuint8arr(value)}"`);
}
console.log('Response fully received');
}
run().catch(err => {
console.log(err);
});
</script>
</body>
</html>
When I request this page in Chrome, I get this output:
Received: "hello1"
Received: "hello2"
Received: "hello3"
Received: "hello4"
Response fully received
So, you can get incoming data as it arrives on the stream. Keep in mind that there are no particular guarantees about the boundaries between packets that arrive. This happens to show each packet nice and neatly separate, but that is only because I'm sending a very small piece of data that nicely fits in one packet and I'm sending it over a very fast network. If the data was larger, it may not arrive in just one chunk.
So, for anything other than very simple uses, you will need at least some form of simple "protocol" to your data format so you can for sure discern where one packet ends and the next begins and have some code that parses for that. For example, you could delineate each packet with some special character (like a linefeed) and your reading code would not assume it has a complete packet until it gets a linefeed character.
I can't quite grasp the last step in this puzzle. Everything compiles, and it is "bug-free". This is my first foray into communications/full-stack and I have been stumped despite numerous excellent tutorials.
[WORKING] Arduino reads and interprets sensor data
[WORKING] index.js acquires data with serial communication over USB
[WORKING] index.js creates a WebSocket connection using nodejs
[WORKING] index.html performs the WebSocket handshake
[WORKING] index.html uses Plotly to create a real-time line graph
[WIP] index.html passes through the Arduino data in the Plotly function
Chopped down index.html:
<script src="server/plotly.min.js"></script>
<script>
//connection to the web socket server
const ws = new WebSocket("ws://localhost:5000");
let foo = 0.0;
//working
ws.addEventListener("open", () => {
console.log("We Are Connected");
ws.send("TestData");
});
//working
ws.addEventListener("message", e => {
console.log(e);
console.log("Data Recieved! Success.");
});
</script>
Rest of the file is just the graphing function which I would like to pass through Arduino data.
index.js
const WebSocket = require("ws");
const wss = new WebSocket.Server({ port: 5000 });
//create a serial port that allows serial connection from Arduino
let SerialPort = require("serialport");
let port = new SerialPort('COM4', { baudRate: 9600 });
let Readline = require("#serialport/parser-readline");
let parser = port.pipe(new Readline({ delimiter: '\n' }));
wss.on("connection", ws => {
//working
console.log("New Client Connection");
//this is what I need to passthrough my Plotly arg
parser.on("data", data => {
//event is firing but can't get client to grab this. Console logs data correctly.
console.log(RPM: ${data});
});
//working on both ends
ws.on("message", data => {
console.log("TEST")
ws.send(data);
});
//doesn't log?
port.on("open", () => {
console.log("Serial Port Open");
});
});
//working
console.log("The server is ON");
I'm looking for a strategy or method to grab the sensor data in my HTML file. Is it something simple conceptually I am missing? Thank you.
You established a websocket server, and it is works. If you want to send message to websocket, define a socket to the websocket server, get out the Serial part from websocket server, and run is standalone, and send data from this to the websocket.
Like this:
const http = require('http');
const WebSocketServer = require('websocket').server;
const server = http.createServer();
server.listen(5000);
const wsServer = new WebSocketServer({
httpServer: server
});
let SerialPort = require("serialport");
var serialPort = new SerialPort("COM5", {
baudRate: 9600,
parser: new SerialPort.parsers.Readline("\n")
});
var connection;
wsServer.on('request', function(request) {
connection = request.accept(null, request.origin);
connection.on('message', function(message) {
console.log('Received Message:', message.utf8Data);
connection.sendUTF('Hi this is WebSocket server!');
});
connection.on('close', function(reasonCode, description) {
console.log('Client has disconnected.');
});
});
serialPort.on('open',function(){
//connection.sendUTF('Hi this is WebSocket server!');
console.log('open');
serialPort.on('data', function(data){
readData = data.toString();
console.log("N<", readData);
if( typeof connection!="undefined")
connection.sendUTF( readData);
});
});
Pretty much purely for pedagogical purposes, I'm serving both my front and back end data out of my one node server. Right now, I'm at the point where I've received my client request successfully, created some data based on said request, am able to console log it, etc. Everything is fine up to that point. My issue is that in the event that my data is only an html file, which is being read with the fs library, it will not render on the page when I attempt to serve it out in my res.end() or res.write(). I can see it's exactly what I want and expect when I console log it, but it just doesn't render in the browser. Any help would be appreciated. I've got it set up to where I'm handling my requests in an "if/else" wherein I only have the two scenarios of "/" (home), in which case I serve the html file, and anything else because the server really only needs to handle those two events. Thanks in advance.
Edit. This is what I have so far:
function responseHandler(req, res) {
res.writeHead(200, {"Content-Type": "text/html"});
if (req.url.match("fav")) {
res.end("");
return;
}
else if (req.url.match("/endpoint")) {
var input = req.url.match(/endpoint\/(.*)/)[1];
var output = endpoint.toHTML(decodeURI(input));
res.end(data);
console.log(input, req.url)
}
else {
fs.readFile("index.html", "utf8", function(err, data) {
console.log("data:" + data);
var input = req.url.match(/endpoint\/(.*)/)[1];
var output = endpoint.toHTML(decodeURI(input));
});
}
res.end();
}
I can see the data in the console which, in the last case, is just my HTML file. It just won't render in the page.
How did you attempted to serve the html with res.end() and res.write() ?
I just made a small test here, and this works:
app.js
var http = require('http');
var fs = require('fs');
var html = fs.readFileSync('hello-world.html');
http.createServer(function (req, res) {
res.writeHead(200, {'Content-Type': 'text/html'});
res.end(html);
}).listen(8000);
hello-world.html
<h3>Hello World</h3>
Edit: To match with your code, try this:
function responseHandler(req, res) {
res.writeHead(200, {"Content-Type": "text/html"});
if (req.url.match("fav")) {
res.end("");
return;
} else if (req.url.match("/endpoint")) {
var input = req.url.match(/endpoint\/(.*)/)[1];
var output = endpoint.toHTML(decodeURI(input));
console.log(input, req.url);
// we have no data variable in this scope
res.end("");
// I added a return statement in each step
// Just to be clear that we don't want to go if any
// condition have fit, since we cannot call res.end()
// more than once
return;
} else {
fs.readFile("index.html", "utf8", function(err, data) {
// error handling
if (err) return res.end(err);
// now we have the data
console.log("data:" + data);
res.end(data);
});
return;
}
}
Serving html in asynchronous way works something like that;
var fs = require('fs');
var http = require('http');
http.createServer(function(req, res){
res.writeHead(200, {'Content-Type': 'text/html'});
fs.readFile('index.html', function(err, data){
if(err){
return console.log(err);
}
res.end(data);
});
}).listen(8080);
console.log('Server is running on Port: 8080');
I have a nodejs app with socket.io. To test this, save the below listing as app.js. Install node, then npm install socket.io and finally run on command prompt: node app.js
var http = require('http'),
fs = require('fs'),
// NEVER use a Sync function except at start-up!
index = fs.readFileSync(__dirname + '/index.html');
// Send index.html to all requests
var app = http.createServer(function(req, res) {
res.writeHead(200, {'Content-Type': 'text/html'});
res.end(index);
});
// Socket.io server listens to our app
var io = require('socket.io').listen(app);
// Send current time to all connected clients
function sendTime() {
io.sockets.emit('time', { time: new Date().toJSON() });
}
// Send current time every 10 secs
setInterval(sendTime, 5000);
// Emit welcome message on connection
io.sockets.on('connection', function(socket) {
socket.emit('welcome', { message: 'Welcome!' });
socket.on('i am client', console.log);
});
app.listen(3000);
This code sends data to the file index.html. After running the app.js, open this file in your browser.
<!doctype html>
<html>
<head>
<script src='http://code.jquery.com/jquery-1.7.2.min.js'></script>
<script src='http://localhost:3000/socket.io/socket.io.js'></script>
<script>
var socket = io.connect('//localhost:3000');
socket.on('welcome', function(data) {
$('#messages').html(data.message);
socket.emit('i am client', {data: 'foo!'});
});
socket.on('time', function(data) {
console.log(data);
$('#messages').html(data.time);
});
socket.on('error', function() { console.error(arguments) });
socket.on('message', function() { console.log(arguments) });
</script>
</head>
<body>
<p id='messages'></p>
</body>
</html>
The data sent right now is the current time and index.html works fine, updates the time every five seconds.
I want to modify the code so that, it reads my sensor data over TCP. My sensors are connected thru a data acquisition system and relays the sensor data over IP: 172.16.103.32 port:7700. (This is over LAN, so will not the accessible to you.)
How can this be implemented in nodejs?
Is SensorMonkey a viable alternative ? If so, any pointers on how to go about using it?
I have a decent hack that is working right now, for which I request the readers to comment on....
var net = require('net'),
http = require('http'),
port = 7700, // Datalogger port
host = '172.16.103.32', // Datalogger IP address
fs = require('fs'),
// NEVER use a Sync function except at start-up!
index = fs.readFileSync(__dirname + '/index.html');
// Send index.html to all requests
var app = http.createServer(function(req, res) {
res.writeHead(200, {'Content-Type': 'text/html'});
res.end(index);
});
// Socket.io server listens to our app
var io = require('socket.io').listen(app);
// Emit welcome message on connection
io.sockets.on('connection', function(socket) {
socket.emit('welcome', { message: 'Welcome!' });
socket.on('i am client', console.log);
});
//Create a TCP socket to read data from datalogger
var socket = net.createConnection(port, host);
socket.on('error', function(error) {
console.log("Error Connecting");
});
socket.on('connect', function(connect) {
console.log('connection established');
socket.setEncoding('ascii');
});
socket.on('data', function(data) {
console.log('DATA ' + socket.remoteAddress + ': ' + data);
io.sockets.emit('livedata', { livedata: data }); //This is where data is being sent to html file
});
socket.on('end', function() {
console.log('socket closing...');
});
app.listen(3000);
References:
Socket.io Website - www.socket.io - Its the buzzword now.
TCP Socket Programming
Nodejs "net" module
Simplest possible socket.io example.
I'm new to HTML5 and node.js. I'm trying to create a very basic client-server application. Here is the code.
Server side (node.js):
var net = require('net');
var server = net.createServer(function(c) {
console.log('client connected');
c.setEncoding('utf8');
c.on('end', function() {
console.log('client disconnected');
});
c.on('data', function(data) {
console.log(data);
c.write("Got it");
});
});
server.listen(9998);
Client side (websockets):
<!DOCTYPE html>
<html>
<head>
<script>
try {
var ws = new WebSocket('ws://127.0.0.1:9998');
ws.onopen = function() {
ws.send("Message to send");
alert("Message is sent...");
};
ws.onmessage = function (evt) {
var message = evt.data;
alert("Message is received: " + message);
};
ws.onclose = function() {
alert("Connection is closed...");
};
} catch (err) {
alert(err.message);
}
</script>
</head>
<body>
</body>
</html>
As far as I understand, the client should connect to the server, send "Message to send" and the server should reply with "Got it". Instead what the server receives is an http GET request for the client html page and none of the client callbacks are ever fired. What am I missing?
You are missing the very point of WebSocket, its TCP but not raw TCP, to establish a connection the client must send an HTTP request first, then the servers switches protocols to websockets, but remember websocket is not raw TCP, there are custom headers in packets and other.
To save your self some time, try https://github.com/LearnBoost/Socket.IO/ or other websocket libraries https://github.com/joyent/node/wiki/Modules.