Trying to send form-data in postman and sequelize return error:
value cannot be null
But when send raw request with json all ok. Trying body-parser and multer, but nothing working
This is my index.ts
import express from "express";
import fileUpload from "express-fileupload"
...
const app = express()
const PORT = process.env.PORT || 5100
app.use(cors())
app.use(express.json())
app.use('/api', router)
app.use(fileUpload({}))
app.use(errorHandler)
const start = async () => {
try {
await sequelize.authenticate()
await sequelize.sync()
console.log(chalk.cyanBright('Successful conection to data base'));
app.listen(PORT, () => { console.log(chalk.cyanBright(`Server has been started on port ${PORT}`)) })
}
catch (e) {
console.log(e);
}
}
start()
And this is my controller
export const DeviceController = {
async create(req: Request, res: Response, next:nextType) {
try {
const { brandId, typeId, name, price } = req.body
const img = req.files
let filename = 'uuid.v4()' + '.jpg'
img?.mv(path.resolve(__dirname, '..', 'static', filename))
const device = await Models.Device.create({ brandId, typeId, name, price, img: filename })
return res.json(device)
} catch (error: any) {
next(ApiError.badRequest(error.message))
console.log(error);
}
app.use(express.json())
You have body parsing middleware for JSON request bodies.
You don't have body parsing middleware for multipart/form-data request bodies. The documentation for body-parser lists a several middlewares you could use.
Trying body-parser
… which says it doesn't support that format
and multart
… that doesn't appear to exist. Do you mean multiparty? Or maybe multer?
We can't tell you what you did wrong without seeing your attempt.
Re edit:
You said:
const img = req.files
img?.mv(path.resolve(__dirname, '..', 'static', filename))
But the documentation says:
console.log(req.files.foo); // the uploaded file object
The files property contains all the files, indexed by the the names given to them in the multipart request.
You're trying to read that collection of files as if it were a single file.
I am using the restify framework to build a small app that copies an uploaded file from its temporary location to a permanent location and then inserts that new location into a MySQL database. However, when attempting to copy the file and then run the promisified query, the system throws a silent error not caught by the promise chain causing a 502 error on the web server end. A minimal working example is below. This example has been tested and does fail out of the gate.
If one of the steps in the process is removed (copying the file or storing the string in the database), the silent error disappears and API response is sent. However, both steps are needed for later file retrieval.
Main Restify File
const restify = require('restify');
const corsMiddleware = require('restify-cors-middleware');
const cookieParser = require('restify-cookies');
const DataBugsDbCredentials = require('./config/config').appdb;
const fs = require('fs');
const { host, port, name, user, pass } = DataBugsDbCredentials;
const database = new (require('./lib/database'))(host, port, name, user, pass);
const server = restify.createServer({
name: 'insect app'
});
// enable options response in restify (anger) -- this is so stupid!! (anger)
const cors = corsMiddleware({});
server.pre(cors.preflight);
server.use(cors.actual);
// set query and body parsing for access to this information on requests
server.use(restify.plugins.acceptParser(server.acceptable));
server.use(restify.plugins.queryParser({ mapParams: true }));
server.use(restify.plugins.bodyParser({ mapParams: true }));
server.use(cookieParser.parse);
server.post('/test', (req, res, next) => {
const { files } = req;
let temporaryFile = files['file'].path;
let permanentLocation = '/srv/www/domain.com/permanent_location';
// copy file
return fs.promises.copyFile(temporaryFile, permanentLocation)
// insert into database
.then(() => database.query(
`insert into Specimen (
CollectorId,
HumanReadableId,
FileLocation
) values (
1,
'AAA004',
${permanentLocation}
)`
))
.then(() => {
console.log('success!!!')
return res.send('success!')
})
.catch(error => {
console.error(error)
return res.send(error);
});
});
./lib/database.js
'use strict';
const mysql = require('mysql2');
class Database {
constructor(host, port, name, user, pass) {
this.connection = this.connect(host, port, name, user, pass);
this.query = this.query.bind(this);
}
/**
* Connects to a MySQL-compatible database, returning the connection object for later use
* #param {String} host The host of the database connection
* #param {Number} port The port for connecting to the database
* #param {String} name The name of the database to connect to
* #param {String} user The user name for the database
* #param {String} pass The password for the database user
* #return {Object} The database connection object
*/
connect(host, port, name, user, pass) {
let connection = mysql.createPool({
connectionLimit : 20,
host : host,
port : port,
user : user,
password : pass,
database : name,
// debug : true
});
connection.on('error', err => console.error(err));
return connection;
}
/**
* Promisifies database queries for easier handling
* #param {String} queryString String representing a database query
* #return {Promise} The results of the query
*/
query(queryString) {
// console.log('querying database');
return new Promise((resolve, reject) => {
// console.log('query promise before query, resolve', resolve);
// console.log('query promise before query, reject', reject);
// console.log('query string:', queryString)
this.connection.query(queryString, (error, results, fields) => {
console.log('query callback', queryString);
console.error('query error', error, queryString);
if (error) {
// console.error('query error', error);
reject(error);
} else {
// console.log('query results', results);
resolve(results);
}
});
});
}
}
module.exports = Database;
./testfile.js (used to quickly query the restify API)
'use strict';
const fs = require('fs');
const request = require('request');
let req = request.post({
url: 'https://api.databugs.net/test',
}, (error, res, addInsectBody) => {
if (error) {
console.error(error);
} else {
console.log('addInsectBody:', addInsectBody);
}
});
let form = req.form();
form.append('file', fs.createReadStream('butterfly.jpg'), {
filename: 'butterfly.jpg',
contentType: 'multipart/form-data'
});
If the request is made to the localhost, then an 'ECONNRESET' error is thrown as shown below:
Error: socket hang up
at connResetException (internal/errors.js:570:14)
at Socket.socketOnEnd (_http_client.js:440:23)
at Socket.emit (events.js:215:7)
at endReadableNT (_stream_readable.js:1183:12)
at processTicksAndRejections (internal/process/task_queues.js:80:21) {
code: 'ECONNRESET'
}
This error is only thrown if both the database and the file I/O are both present in the promise chain. Additionally, the error does not occur if the database request is made first with the file I/O occurring second; however, another rapid request to the server will immediately lead to the 'ECONNRESET' error.
I feel as though I should edit this answer, despite the solution revealing a rookie mistake, in the hopes that it may help someone else. I will keep the previous answer below for full transparency, but please not that it is incorrect.
Correct Answer
TL;DR
PM2 restarted the NodeJS service with each new file submitted to and saved by the API. The fix: tell PM2 to ignore the directory that stored the API's files. See this answer
Long Answer
While the OP did not mention it, my setup utilized PM2 as the NodeJS service manager for the application, and I had turned on the 'watch & reload' feature that restarted the service with each file change. Unfortunately, I had forgotten to instruct PM2 to ignore file changes in the child directory storing new files submitted through the API. As a result, each new file submitted into the API caused the service to reload. If more instructions remained to be executed after storing the file, they were terminated as PM2 restarted the service. The 502 gateway error was a simple result of the NodeJS service becoming temporarily unavailable during this time.
Changing the database transactions to occur first (as incorrectly described as a solution below) simply insured that the service restart occurred at the very end when no other instructions were pending.
Previous Incorrect Answer
The only solution that I have found thus far is to switch the file I/O and the database query so that the file I/O operation comes last. Additionally, changing the file I/O operation to rename rather than copy the file prevents rapidly successive API queries from throwing the same error (having a database query rapidly come after any file I/O operation that is not a rename seems to be the problem). Sadly, I do not have a reasonable explanation for the socket hang up in the OP, but below is the code from the OP modified to make it functional.
const restify = require('restify');
const corsMiddleware = require('restify-cors-middleware');
const cookieParser = require('restify-cookies');
const DataBugsDbCredentials = require('./config/config').appdb;
const fs = require('fs');
const { host, port, name, user, pass } = DataBugsDbCredentials;
const database = new (require('./lib/database'))(host, port, name, user, pass);
const server = restify.createServer({
name: 'insect app'
});
// enable options response in restify (anger) -- this is so stupid!! (anger)
const cors = corsMiddleware({});
server.pre(cors.preflight);
server.use(cors.actual);
// set query and body parsing for access to this information on requests
server.use(restify.plugins.acceptParser(server.acceptable));
server.use(restify.plugins.queryParser({ mapParams: true }));
server.use(restify.plugins.bodyParser({ mapParams: true }));
server.use(cookieParser.parse);
server.post('/test', (req, res, next) => {
const { files } = req;
let temporaryFile = files['file'].path;
let permanentLocation = '/srv/www/domain.com/permanent_location';
// copy file
// insert into database
return database.query(
`insert into Specimen (
CollectorId,
HumanReadableId,
FileLocation
) values (
1,
'AAA004',
${permanentLocation}
)`
)
.then(() => fs.promises.rename(temporaryFile, permanentLocation))
.then(() => {
console.log('success!!!')
return res.send('success!')
})
.catch(error => {
console.error(error)
return res.send(error);
});
});
You did not handle the database promise in then and catch -
Main Restify File
const restify = require('restify');
const corsMiddleware = require('restify-cors-middleware');
const cookieParser = require('restify-cookies');
const DataBugsDbCredentials = require('./config/config').appdb;
const fs = require('fs');
const { host, port, name, user, pass } = DataBugsDbCredentials;
const database = new (require('./lib/database'))(host, port, name, user, pass);
const server = restify.createServer({
name: 'insect app'
});
// enable options response in restify (anger) -- this is so stupid!! (anger)
const cors = corsMiddleware({});
server.pre(cors.preflight);
server.use(cors.actual);
// set query and body parsing for access to this information on requests
server.use(restify.plugins.acceptParser(server.acceptable));
server.use(restify.plugins.queryParser({ mapParams: true }));
server.use(restify.plugins.bodyParser({ mapParams: true }));
server.use(cookieParser.parse);
server.post('/test', (req, res, next) => {
const { files } = req;
let temporaryFile = files['file'].path;
let permanentLocation = '/srv/www/domain.com/permanent_location';
// copy file
return fs.promises.copyFile(temporaryFile, permanentLocation)
// insert into database
.then(() =>{
// Your database class instance query method returns promise
database.query(
`insert into Specimen (
CollectorId,
HumanReadableId,
FileLocation
) values (
1,
'AAA004',
${permanentLocation}
)`
).then(() => {
console.log('success!!!')
return res.send('success!')
})
.catch(error => {
console.error('Inner database promise error', error)
return res.send(error);
});
}).catch(error => {
console.error('Outer fs.copyfile promise error', error)
return res.send(error);
})
});
I want to call a Node-RED flow from IBM Cloud Functions.
const https = require('https');
function main(params) {
const path = "/" + params.route + "?" + params.query_params ;
const options = {
hostname: params.hostname,
path: path,
port: 443,
method: 'GET'
};
return new Promise((resolve, reject) => {
https.get(options, (resp) => {
resp.on('data', (d) => {
let s = d.toString();
obj = JSON.parse(s);
resolve({ "gw_result": obj })
});
});
})
}
In the Node-RED flow I'm using a HTTP request to get data from another server. For test purposes I used a GET request to google.com but have same results using another Node-RED endpoint.
As soon as I invoke the web action I get the error message "The action did not produce a valid response and exited unexpectedly". The output of the Node-RED flow appears some seconds later in the web action's log although the Node-RED flow works properly and promptly (I used debug Node-RED debug nodes to check this).
The https GET request to Node-RED works well when I replace the http request in Node-RED by something else, e.g. a Function node, even when I use a Delay node to delay the response for a second or so.
This code works, although google.com does not return an object, of course.
var rp = require('request-promise');
function main(params) {
var uri = params.hostname + params.route + params.query_params
return new Promise(function (resolve, reject) {
rp(uri)
.then(function (parsedBody) {
obj = JSON.parse(parsedBody);
resolve({ "gw_result": obj
});
})
.catch(function (err) {
resolve({ message: 'failed!!', error: err.toString() });
});
});
}
I am working on a MEAN stack app, and I'm trying to create a couple of API endpoints that the Angular client can $http.get, with simple JSON files populated with dummy data.
Here's the orders.json file I'm trying to test it with:
[
{
"order_status":"Shipped",
"order_qty":30
},
{
"order_status":"Shipped",
"order_qty":6
}
]
For example, the api route to $http.get:
apiRouter.get('/:fileName', queries.getStaticJSONFileForDevelopment);
But when I try to use express's sendFile method with a local .json file, like orders.json:
queries.js:
exports.getStaticJSONFile = function(req, res) {
var fileName = req.params.fileName;
console.log('path: ' + path.normalize(__dirname + '/' + fileName));
res.sendFile(path.normalize(__dirname + '/' + fileName), function(err) {
if (err) return res.send({ reason:error.toString() });
});
};
The console.log tells me I'm pointed at the correct path to the file, but Postman delivers this error:
TypeError: undefined is not a function
at Object.exports.getStaticJSONFile [as handle] (path/to/queries.js:260:7)
// queries.js:260:7 points to the 's' in 'sendFile' above
However, when I just send the json data by itself:
res.send([{"order_status":"Shipped","order_qty":30},{"order_status":"Shipped","order_qty":6}]);
...the endpoint renders the data as you would expect. Am I trying to get the sendFile method to do something it's not meant to do, or is there something I'm missing? Thanks very much for any advice you may have!
If You want to read json file and response with json so You can try this:
var jsonfile = require('jsonfile');
exports.getStaticJSONFile = function(req, res) {
var fileName = req.params.fileName;
var file = path.normalize(__dirname + '/' + fileName);
console.log('path: ' + file);
jsonfile.readFile(file, function(err, obj) {
if(err) {
res.json({status: 'error', reason: err.toString()});
return;
}
res.json(obj);
});
};
Using node.js I am attempting to load a .json file (file) and have it viewed on a web browser. So far I have been able to create a JSON object (obj) and viewed the JSON string of that object on the web browser. However, when I attempt to load the file I get the file link and not the file itself. Below is the code:
var jf = require('jsonfile');
var jsonfn = function(request, response) {
var file = '/home/ubuntu/public/json/foods.json';
var obj = {name: 'JP'};
jf.writeFile(file, obj, function(err) {
console.log(util.inspect(jf.readFileSync(file)));
})
response.json(obj);
};
Can anyone provide advice for me being able to load a JSON file and create a response to allow me to view the file in the web browser?
jf.readFile(file, function(err, obj) {
if (err) return response.send(500);
response.json(obj);
});
// or ...
require('fs').readFile(file, function(err, data) {
if (err) return response.send(500);
response.setHeader('Content-Type', 'application/json');
response.send(data);
});