New line in json array is getting converted to a comma | nodejs - json

I am relatively new to nodejs and running into an issue while parsing a Json post request.
Here is the JSON format of the post request:
{"parameters":{"issuerId":[96409],"source":["'XYZ'"]}}
And here is my code to read it.
function getSearchData(req, res, next) {
console.log("req is" + req.body);
try {
JSON.parse(reqJSON);
} catch (e) {
console.log(e);
}
}
This parsing works fine and I am able to parse it and do my further logic. However, if I change my format of post request(same request with additional new lines) it fails to parse as it adds additional commas in place of each new line in the request.
{
"parameters": {
"issuerId": [96409],
"source":["'XYZ'"]
}
}
Here's the output from the code with the second request.
req is{,"parameters":{"id":[96409],,"source":["'XYZ'"]}}
[SyntaxError: Unexpected token ,]
If you notice, an extra comma gets added at each new line, which was never in the request to begin with.
What am I doing wrong here?

You should never have to parse the JSON yourself, unless you're concatenating the request body stream yourself.
Hint 1: Do you use any framework like Express? Do you use body parser?
Hint 2: How do you create the JSON?
Hint 3: Do you use correct content-type?
Hint 4: How do you create req.body from the request stream?
You didn't include the entire code so it's impossible to give you a specific solution.
What am I doing wrong here?
Whatever you're doing wrong here, it's not included in the question.
However, if I change my format of post request(same request with additional new lines)
It would be useful if you included more details of how you do it.
I see two potential sources of that problem:
either the commas are introduced during the on the client side
or they are introduced during the request reading on the server side
You didn't show us any of those two parts - you didn't show the serializing code and the code that sends the data, and you didn't include the code that gets the data, possibly joins it from chunks and parses the JSON. But the problem is likely in one of those parts.
Update
Here is an example on how to do what you need using Express. You didn't answer whether you use any framework like Express or not, but I think that you should if you can't achieve that simple task without it, so here is a working example:
const express = require('express');
const bodyParser = require('body-parser');
const app = express();
function getSearchData(req, res, next) {
console.log('req body is', req.body);
console.log('req body JSON is', JSON.stringify(req.body));
res.json({ ok: true });
}
app.use(bodyParser.json());
app.use(getSearchData);
app.listen(3335, () => console.log('Listening on http://localhost:3335/'));
It shows how to correctly get a parsed JSON request body as an object req.body, how to print the data as a standard console.log representation and serialized again as JSON. See how it works and compare it to your own solution. This is all I can do because not having seen your entire solution I cannot tell you more than the hints that I've already given.

Related

express.json() block request reach latter middleware

When my program like this, the former middware can send string 'Hello world' to Postman.
app.use((req, res, next) => {
console.log(req.body);
res.send('Hello world');
next();
})
app.use(express.json());
})
But I don't know why when I put the test middleware after app.use(express.json()), it do not send anything. I use postman to send post request with raw data of type json.
You must declare that middleware express.json before than any route handlers, because that middleware is in charge of parsing the information of the request and put into the request body property, after that it automatically call next to continue the execution flow. If you put that middleware at the end of your request, after the parsing of the information next will be called as you don't have more middleware to manage the execution flow your server hangs.
OK, I just found that because my POSTMAN POST request use raw json with single quoted instead of double quoted. I changed it into double quoted so it worked. Thanks.

How can I send data (string) from my html to my server (node or express) and execute certain function with it?

I have a node.js server running and is executing what I want it to do, create an excel document with data fetched with Axios from an API, now, I want to allow a user to input a string on my HTML and then send that string to my web server, and perform an Axios request to the API that I am consuming. How can I do that? Should I use a POST request, a PUT request, or anything else?
Thanks!
You should be able to send data from your client to the server via the request body regardless of the request method whether GET/POST/PUT.
fetch call would look like this
// just a simple get request
fetch('/example-endpoint', {
method: 'POST',
body: {
data: 'String to be sent'
}
});
And on your node express server
app.post('/example-endpoint', (req, res) => {
const data = req.body.data;
// do stuff with your data here.
})
GET request method is used for getting data, while POST and PUT are used for creating and updating data, so use them as you need them.
POST request to your server, like uh https://myserver.com/api/${userString}
something like that

Superagent: Error: Parser is unable to parse the response

I'm using Superagent in my react app, and i'm making some call's to the IPFS api. Specifically, I am uploading files to my IPFS server. Now, everything works, When I upload one or multiple files the call goes through and the files show up in IPFS no problem.
A problem occurs when I upload multiple files though, the response seems to come back as plain text, instead of JSON, and superagent throws the error
client.js:399 Uncaught (in promise) Error: Parser is unable to parse the response
at Request.<anonymous> (client.js:399)
at Request.Emitter.emit (index.js:133)
at XMLHttpRequest.xhr.onreadystatechange (client.js:708)
So to be clear, when uploading a single file, I get a nice JSON response, but when I upload multiple files, the response is in plain text.
Can I force Superagent to give me the response back and parse it myself? Or can I set something when making the call so that it forces a json parse? Below is my superagent request function
add : acceptedFiles => {
const url = ipfs.getUrl("add")
const req = request.post(url)
acceptedFiles.forEach(file => req.attach(file.name, file))
req.then(res => {
return console.log(res);
})
}
I'm searching for this for a more elegant solution, but before I would have found it , I'd like to provide my own solution.
I think this problem caused by wrong responsive Content-Type set, but I've not confirmed this opinion yet.
However, you can try this:
req.catch(function (err) {
console.log(err.rawResponse)
})
At least, this solves my problem.
According to their docs you can specify custom parser that will take precedence over built-in parser:
You can set a custom parser (that takes precedence over built-in parsers) with the .buffer(true).parse(fn) method. If response buffering is not enabled (.buffer(false)) then the response event will be emitted without waiting for the body parser to finish, so response.body won't be available.
I tried and it worked well for me.
superagent.get('....')
.buffer(true)
.parse(({ text }) => JSON.parse(text))
.then(...)

View RawBody In SailsJS

I'd like to dump out, via sails.log.debug(), the raw POSTed data as seen by a controller function. I am dealing with JSON coming from a third-party that may be badly formatted and need to figure out where/how. I'd like to see the whole, raw dump.
create: function(req, res) {
sails.log.debug(???);
//var ticket = JSON.parse(req.param("webhook"));
return res.echoRequest(true);
}
You will need to use middleware to get the "RAW" body. You will want to grab that pre-sails
Check out answer to this question Node.js - get raw request body using Express
You could use:
var packet = req.params.all();
sails.log.debug(packet);
Hope that helps.

How to send data back to node from casperjs?

How can I send data back to node, from a process launched via execfile in nodeJS? preferably in a JSON format.
This is how my code looks like right now:
//index.js NodeJS
var execFile = require('child_process').execFile;
var express = require('express');
app.get('/', function(req, res) {
var lchecker = execFile('/usr/local/bin/casperjs', [myprogram, myargs]);
lchecker.stdout.on('data', function(data) {
var dataObject = JSON.parse(data); //This throws an error.
});
});
The casperjs script that I'm calling returns JSON-like strings, like this:
console.log("[{href: targetLink.href, anchor: targetLink.anchor, isLive: 0, isFollowed: null}]");
This is the error that I get
When I'm trying to parse the JSON-like string, I get an error that says:
19 Jun 16:46:43 - [nodemon] starting node index.js
undefined:1
[{href: targetLink.href, anchor: targetLink.anchor, isLive: 1, isFollow: 1}]
^
Unexpected token h
So my JSON is invalid, and sincerely, I'm sure that there's a better way to send data back to node from casperjs, but I don't know how.
I've been thinking about creating a new route in express, and then make casperjs visit that route and pass the information via GET, and then manipulate that information in node. Is this a good way to achieve this?
Even though I received good and viable answers, I ultimately ended up outputting everything to stdout in casperjs, to send it back to PHP through a JSON array.
so in casperjs I wrote something like:
console.log(JSON.stringify(targetLink))
And then in node, I could just access that through JSON.parse and manipulate the data in any way I want.
EDIT:
I've run into this situation more often than not, so alternatively you can make CasperJS POST the information to a web endpoint, it's sometimes cleaner, but it adds overhead if you are worried about security and you need to make sure that only authorized scrapers can post data to your endpoint.
I have used CasperJS on NodeJS by running CasperJs as a service.
Basically NodeJS through http.get() makes a request to CasperJS script which return a JSON object as response.
Here an example and more details about how a CasperJS script can start a web server:
CasperJS passing data back to PHP
You may probably prefer to use something like SpookyJS (https://github.com/WaterfallEngineering/SpookyJS) which offer the ability to use CasperJs inside a Node.js program.
I don't know if you will find the feature you want but it's probably cleaner anyway.