I'm using express-session with sequelize and connect-session-sequelize within my Node.js App. By default express-session logs all SQL calls, which highlighted the issue I am trying to resolve. A call to a view template via res.render('index') initiates both a SELECT and UPDATE call as expected (below):
Executing (default): SELECT `sid`, `expires`, `data` FROM `session` AS `session` WHERE `session`.`sid` = 'T84YX7hvKWmihWPMzT0Cg_97Wao7kCF5';
Executing (default): UPDATE `session` SET `expires`=? WHERE `sid` = ?
However, adding linked header elements (style/script tags) or img tags within the template results in an additional call to both the SELECT and UPDATE query for every linked element. As a result, adding more content will result in an exponential number of database calls.
How can I prevent style,script and img calls from initiating the call within express?
app.js
const config = require('./config');
const http = require('http');
const express = require('express');
const session = require('express-session');
const Sequelize = require('sequelize');
const sequelize = new Sequelize(
config.dbDatabase,
config.dbUser,
config.dbPassword,
{
host : config.dbHost,
dialect : config.dbDialect,
define : {
timestamps : false,
underscored : true
}
}
);
const app = express();
const sequelizeStore = require('connect-session-sequelize')(session.Store);
const sessionStore = new sequelizeStore({
db: database.sequelize,
modelKey: 'session'
});
app.set('view engine', 'pug');
app.set('views', './views');
app.use(express.static('./public'));
app.get('/', function(req, res){
res.render('index'); // pug
}
const server = http.createServer(app);
server.listen(config.port, () => {
console.log(`App listening on port: ${config.port}`);
});
index.pug
doctype html
html
head
meta(charset='utf8')
title example
link(rel='stylesheet' href='/css/example.css')
script(src='/js/lib/angular/angular.min.js')
script(src='/js/lib/angular/angular-resource.min.js')
script(src='/js/example.min.js')
body
H1 Example Template
img(src= "/img/sample.jpg" alt="Sample image")
The issue is caused when app.use(express.static('./public')); is used after app.use(session({...});. Hope this helps anyone else encountering the same issue.
Related
I am working on rest api with node.js and express that connects to the client side I have found a tutorial here https://www.taniarascia.com/node-express-postgresql-heroku/ and everything works fine but it will not deploy to heroku and I do not know why because everything works fine when I run it on localhost. Why is this happening?
Here is my code
index.js
const express = require("express");
const bodyParser = require("body-parser");
const cors = require("cors");
const {pool} = require("./config");
const { get } = require("mongoose");
const app = express();
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({extended: true}));
app.use(cors());
const getBooks = (req, res) => {
pool.query('SELECT * FROM books', (err, results) => {
if(err){
throw err;
}
res.status(200).json(results.rows);
});
}
const addBook = (req, res) => {
const {author, title} = req.body;
pool.query(
'INSERT INTO books (author, title) VALUES ($1, $2)',
[author, title],
(err) => {
if(err){
throw err;
}
res.status(201).json({status: "success", message: "Book added."});
},
)
}
app.route("/books").get(getBooks).post(addBook);
app.listen(process.env.PORT || 8000, () => {
console.log(`Server listening`);
})
config.js
require("dotenv").config();
const {Pool} = require("pg");
const isProduction = process.env.NODE_ENV === "production";
const connectionString = `postgresql://${process.env.DB_USER}:${process.env.DB_PASSWORD}#${process.env.DB_HOST}:${process.env.DB_PORT}/${process.env.DB_DATABASE}`;
const pool = new Pool({
connectionString: isProduction ? process.env.DATABASE_URL : connectionString,
ssl: isProduction,
});
module.exports = {pool};
init.sql
CREATE TABLE books (
ID SERIAL PRIMARY KEY,
author VARCHAR(255) NOT NULL,
title VARCHAR(255) NOT NULL
);
INSERT INTO books (author, title)
VALUES ('J.K. Rowling', 'Harry Potter');
I'd have a look at this, if your app works fine locally but not on heroku it's probably a deployment issue:
https://devcenter.heroku.com/articles/getting-started-with-nodejs
Specifically, l think you probably need a Procfile which is like an extra file that Heroku needs to deploy. Assuming you got set up using the cli (https://devcenter.heroku.com/categories/command-line) you can get more detail on how your app is failing by getting the logs using 'heroku logs -t' inside the folder you deployed from.
I am currently using MySQL for the db instead of the popular mongodb, since that is the case there isn't much documentation out there as far as architecture and getting set up. This is my current structure
client
-- angular files
routes
-- index.js
views
-- 404 page
app.js
I don't understand how I can implement controllers or models into this structure. I'm currently grabbing data from the db or sending it with the routes..I'm not sure what the added layer of controllers would do. Maybe this is a dumb question but I would just like to have a clear baseline so that my project will scale well. I feel like there should be way more to this than what I currently have.
index.js
const express = require('express');
const mysql = require('mysql');
const router = express.Router();
const db = mysql.createConnection({
host : 'localhost',
user : 'root',
password : 'password',
database : 'db'
});
// Connect
db.connect((err) => {
if(err){
throw err;
}
console.log('MySql Connected...');
});
// Select Data
router.get('/getData', (req, res) => {
let sql = 'SELECT * FROM data';
let query = db.query(sql, (err, results) => {
if(err) throw err;
console.log(results);
res.send(results)
});
});
module.exports = router;
app.js
const express = require('express');
const mysql = require('mysql');
const bodyParser = require('body-parser');
const path = require('path');
const cors = require('cors');
const compression = require('compression');
const helmet = require('helmet')
const expressSanitizer = require('express-sanitizer');
const index = require('./routes/index');
const app = express();
const port = 3000;
var corsOptions = {
origin: 'http://localhost:8100',
optionsSuccessStatus: 200 // some legacy browsers (IE11, various SmartTVs) choke on 204
}
// var logger = (req, res, next) => {
// console.log('logging...')
// next();
// }
//added security
app.use(helmet())
// //set logger
// app.use(logger)
//cors options
app.use(cors(corsOptions))
//body parser middleware
app.use(bodyParser.json())
app.use(bodyParser.urlencoded({extended: false}))
// Mount express-sanitizer here
app.use(expressSanitizer()); // this line follows bodyParser() instantiations
//set static path
app.use(express.static(path.join(__dirname, 'client')));
// set our default template engine to "ejs"
// which prevents the need for using file extensions
app.set('view engine', 'ejs');
//gzip compression
app.use(compression())
//set views for error and 404 pages
app.set('views', path.join(__dirname, 'views'));
app.use('/', index);
app.use('/fp/trips', trips);
app.listen(port, () => {
console.log('server started on port 3000')
})
When working on Node apps I tend to favor a scheme where controllers are (almost) services -- I think it works really well for small applications.
This is an example:
index.js
let app = express()
let users = require('./services/users')
app.get('/users/:id', async function(req, res, next) => {
try {
res.json(users.getByid(req.params.id))
} catch() {
next(err)
}
})
app.listen(8080)
services/users.js
let db = require('./db')
async function getById(id) {
let conn = await db.connect()
let user = conn.query('SELECT * FROM user WHERE id = ?', [id])
if (!user) {
throw new Error("404")
}
return user
}
module.exports = {getById}
services/db.js
let realDb = require('some-open-source-library-to-interact-with-db')
realDb.initialize(process.env.DB_CREDENTIALS) // pseudo-code here
module.exports = realDb
This though, won't work well when you're building large, complex apps -- I think you will require more structure in that case.
PS: I wouldn't suggest to build a large, complex app ever -- split it into smaller ones where patterns like the one I presented work nicely.
You can use Sequelize as ORM (Object Relational Mapper) for your MySQL DB to make your code more readable and to allow you to create better structure of your app. It also has support for PostgreSQL, MySQL, MariaDB, SQLite, and MSSQL.
There are samples out there how to integrate Sequelize with Express. I'm not sure if I'm allowed to post a github repository here but here it is:
https://github.com/jpotts18/mean-stack-relational
PS. I don't own this repository but this might help you somehow.
I have a problem using Node.js. I am sending data via GET request to special script, which should read this data and write it in MySQL. Here is an example of GET request (very long):
http://demo.com/?&bssid=16~6D~57~7F~D8~AD&mac=60~01~94~2B~A8~85&rssi=0&hotspots_list=~~0_74~85~2A~39~77~48_-85~~~~1_78~8C~54~33~D8~3C_-91~~~
~2_64~6E~EA~0E~B3~26_-84~~~~3_76~85~2A~39~77~49_-87~~~~4_2C~33~7A~16~91~3B_-92~~~~5_54~BE~F7~6B~EE~43_-92~~~~6_98~DE~D0~AB~B8~64_-91~~~~7_14~DD~A9~1E
~F6~F8_-93~~~~8_64~6E~EA~0C~9A~3E_-84~~~~9_E8~DE~27~50~2A~12_-88~~~~10_84~C9~B2~0B~BA~C2_-81~~~~11_00~0F~94~BE~FC~44_-87~~~~12_56~BE~F7~6B~EE~44_-86~
~~~13_00~0E~8F~85~5F~73_-53~~~~14_0C~54~A5~A6~C1~1F_-87~~~~15_16~6D~57~7F~D8~AD_-78~~~~16_0E~54~A5~A6~C1~10_-92~~~~0_EC~1A~59~17~CF~5F_-91~~~~1_74~85
~2A~39~77~48_-92~~~~2_16~6D~57~7F~D8~AD_-77~~~~3_0C~54~A5~A6~C1~1F_-92~~~~4_0E~54~A5~A6~C1~10_-83~~~~5_84~C9~B2~0B~BA~C2_-82~~~~6_14~DD~A9~1E~F6~F8_-
92~~~~7_64~6E~EA~0C~9A~3E_-85~~~~8_E8~DE~27~50~2A~12_-93~~~~9_00~22~6B~56~2B~83_-92~~~~10_00~0E~8F~85~5F~73_-58~~~~11_00~0F~94~DB~DE~64_-91~~~~12_16~
DD~A9~1E~F6~F9_-92~~
I've done few experiments, and I found out, that it works fine in PHP, my code (only for output):
<?php
echo $_GET["bssid"]." ".$_GET["mac"]." ".$_GET["rssi"]." ".$_GET["hotspots_list"];
?>
But not in Node.js !This is my Node.js code:
var express = require("express");
var app = express();
var mysql = require('mysql');
var connection = mysql.createConnection({
host : 'host',
user : 'user',
password : 'pswd',
database : 'db'
});
var bssid, mac, rssi, hotspots_list;
const port = 3000;
const host = "0.0.0.0";
app.get('/', function (request, response) {
//Writing data to vars
bssid = request.query.bssid;
mac = request.query.mac;
rssi = request.query.rssi;
hotspots_list = request.query.hotspots_list;
//Sending request to MySQL server
connection.query(`INSERT INTO locations (mac, bssid_current, rssi_currnet, hotspots_list) VALUES ('${mac}','${bssid}','${rssi}','${hotspots_list}')`);
//Debugging line
console.log(`BSSID: ${bssid}`);
//Response
response.statusCode = 200;
response.setHeader('Content-Type', 'text/plain');
return response.send('Done');
});
app.listen(port, host, () => {
console.log(`Server running!`);
});
In your code, your route is /. And you are trying to access it via /server.js
Two solutions, either you add
app.get('/server.js', [...]) or you access it after removing server.js from your url.
I've set up a database on Heroku and I've created a table called users with 2 records, and now I'm trying to get the data into my Node application through express.
I've set up an connection like so in my app.js file:
// connect to the heroku database
var mysql = require('mysql');
var connection = mysql.createConnection({
host : 'us-cdbr-iron-***-**.cleardb.net',
user : 'bfe4***0ede74',
password : '6742****',
database : 'heroku_****ee0f0e9102'
});
I then have a routes folder with an index.js file:
var express = require('express');
var router = express.Router();
/* GET home page. */
router.get('/', function(req, res, next) {
// res.render('layout', { title: 'Movieseat' });
connection.connect();
connection.query('SELECT * FROM `users` WHERE `first_name` = "Kees"', function (error, results, fields) {
// error will be an Error if one occurred during the query
// results will contain the results of the query
// fields will contain information about the returned results fields (if any)
console.log(results);
});
connection.end();
});
module.exports = router;
In this index route I'm trying to serve the record of first_name Kees. But when I visit my host I get the following error:
connection is not defined
ReferenceError: connection is not defined
So it looks like connection has no reference in my route file, but in my WebStorm IDE when I ctrl + click on the connection I get my app.js file where I define my connection. So how do I reference connection in my route file?
Also when I uncomment the following line in my index.js route file:
res.render('layout', { title: 'Movieseat' });
I get the error:
Error: Can't set headers after they are sent. What would be the propper way to request data and render a jade template?
The second error is likely because somewhere you're calling res.send() or res.end() or res.render() already, you just don't realize it. Check your middleware and so on to make sure you're not doing so.
The first issue is because you're neither exporting the connection object from your connection file, nor requiring it in your router file. You always have to explicitly require a module in order to have reference to it.
// connect to the heroku database
var mysql = require('mysql');
var connection = mysql.createConnection({
host : 'us-cdbr-iron-***-**.cleardb.net',
user : 'bfe4***0ede74',
password : '6742****',
database : 'heroku_****ee0f0e9102'
});
module.exports = connection;
NOte that in that case, you will always have the same connection, which isn't great for scalability. Have a look at connection pooling and consider exporting a method that gets the connection rather than passing around a global object.
Then in your router:
var express = require('express');
var router = express.Router();
var connection = require('./path/to/your/connection.js');
/* GET home page. */
router.get('/', function(req, res, next) {
// res.render('layout', { title: 'Movieseat' });
connection.connect();
connection.query('SELECT * FROM `users` WHERE `first_name` = "Kees"', function (error, results, fields) {
// error will be an Error if one occurred during the query
// results will contain the results of the query
// fields will contain information about the returned results fields (if any)
console.log(results);
});
connection.end();
});
module.exports = router;
I'm trying to build an auth system and I have app.js
var express = require('express')
, MemoryStore = require('express').session.MemoryStore
, app = express();
app.use(express.cookieParser());
app.use(express.session({ secret: 'keyboard cat', store: new MemoryStore({ reapInterval: 60000 * 10 })}));
app.use(app.router);
and the route.index as
var express = require('express')
, mysql = require('mysql')
, crypto = require('crypto')
, app = module.exports = express();
app.get('/*',function(req,res){
var url = req.url.split('/');
if (url[1] == 'favicon.ico')
return;
if (!req.session.user) {
if (url.length == 4 && url[1] == 'login') {
var connection = mysql.createConnection({
host : 'localhost',
user : 'user',
password : 'pass',
});
var result = null;
connection.connect();
connection.query('use database');
var word = url[3];
var password = crypto.createHash('md5').update(word).digest("hex");
connection.query('SELECT id,level FROM users WHERE email = "'+url[2]+'" AND password = "'+password+'"', function(err, rows, fields) {
if (err) throw err;
for (i in rows) {
result = rows[i].level;
}
req.session.user = result;
});
connection.end();
}
}
console.log(req.session.user)
when I access http://mydomain.com/login/user/pass a first time it shows in the last console call but a second time access the cookie is clean
Why do you not just use Express's session handling? if you use the express command line tool as express --sessions it will create the project template with session support. From there you can copy the session lines into your current project. There more information in How do sessions work in Express.js with Node.js? (which this looks like it may be a duplicate of)
As for sanitizing your SQL, you seem to be using the library, which will santitize your inputs for your if you use parameterized queries (ie, ? placeholders).
Final thing, you are using Express wrong (no offence). Express's router will let you split alot of your routes (along with allowing you to configure the favicon. See Unable to Change Favicon with Express.js (second answer).
Using the '/*' route will just catch all GET requests, which greatly limits what the router can do for you.
(continued from comments; putting it here for code blocks)
Now that you have an app with session support, try these two routes:
app.get('/makesession', function (req, res) {
req.session.message = 'Hello world';
res.end('Created session with message : Hello world');
});
app.get('/getsession', function (req, res) {
if (typeof req.session.message == 'undefined') {
res.end('No session');
} else {
res.end('Session message: '+req.session.message);
}
});
If you navigate in your browser to /makesession, it will set a session message and notify you that it did. Now if you navigate to /getsession, it will send you back the session message if it exists, or else it will tell you that the session does not exist.
You need to save your cookie value in the response object:
res.cookie('session', 'user', result);
http://expressjs.com/api.html#res.cookie