Post Multiple JSON Objects Simultaneously with Express and Postman - json

I've been researching this question to no-ends, but can't find the simple answer I'm looking for. Basically, I'd like to batch POST JSON objects in array.
I've got a a giant array of JSON objects.
[
{
"Name": "SEARCH Resource Center",
"Address": "2505 Fannin St, Houston, TX 77002",
"Phone": "(713) 739-7752",
"Hours": "Mon-Fri, 8am to 3pm",
"Category": "Drop-In Centers"
},
{
"Name": "Salvation Army Social Services - Young Adult Resource Center",
"Address": "2208 Main St, Houston, TX 77002",
"Phone": "(713) 658-9205",
"Hours": "Mon-Thurs, 11am to 3pm",
"Category": "Drop-In Centers"
},
...
]
I'm using an Express server that handles post requests looks like this:
app.post('/api/orgs', function(req, res) {
// Creates a new User based on the Mongoose schema and the post body
var newOrg = new Organization(req.body);
// New User is saved in the db.
newOrg.save(function(err){
if(err)
res.send(err);
// If no errors are found, it responds with a JSON of the new user
res.json(req.body);
});
});
These objects are then saved in MongoDB as individual records.
I'm using POSTMAN to send HTTP POSTs to my Express Server. As of now, I've been sending all of my JSON POSTS one at a time, because I can't figure out the best way to batch post all the sub-objects stored in the array as individual objects.
Any suggestions or best practices?

If you send you array as a key in your request body, something like this
You'll get it as req.body.my_restaurants. Then simply use this :
db.collection('restaurants').insertMany(req.body.my_restaurants, function(err, restaurants){
if(err) console.log(err);
else console.log("restaurants Added Successfully");
});
I'm assuming that restaurants is the name of your collection.

I've been playing with the previous solution I marked as correct. An even better approach than using the .insertMany() function is to use the .create() function.
The .insertMany() function skips the middleware associated with .save(), whereas the create() function uses the same process, but can also handle arrays.
So my modified express route looked like the below (where Organization is the name of my schema):
app.post('/api/orgs', function(req, res) {
// Array of JSON Objects
if (req.body.batch){
Organization.create(req.body.batch, function(err){
if(err)
res.send(err);
else
res.json(req.body);
});
}
// Single JSON Object
else {
var newOrg = new Organization(req.body);
// New User is saved in the db.
newOrg.save(function(err){
if(err)
res.send(err);
// If no errors are found, it responds with a JSON of the new user
else
res.json(req.body);
});
}
});
And I'm sending JSON objects that look like:
Hope this helps someone else down the line.

Related

Node-Express MySQL API Outputing JSON Array as String

Trying to setup an Express API server to grab some data for a portfolio site. I have set up the MySQL table with an JSON data type for my 'images' column. 'images' is supposed to have multiple image links for a gallery. However, the server outputs the images array as a string instead of an array of strings.
Javascript Code on the API Server
app.get("/getWorks", (req, res) => {
let sql = "select * from works";
db.query(sql, (err, result) => {
if (err) throw err;
console.log(result);
res.send(result);
});
});
The Results
[
{
"workID": 1,
"title": "example",
"images": "[\"https://SERVER_IP/images/example.png\", \"https://SERVER_IP/images/example.png\"]"
}
]
Workaround
I found out a work around to get the desired output adding this:
result = result.map((row) => ((row.images = JSON.parse(row.images)), row));
[
{
"workID": 1,
"title": "example",
"images": ["https://SERVER_IP/images/example.png", "https://SERVER_IP/images/example.png"]
}
]
How come the query is not outputting the data in a JSON array in the first place even though I specified that particular column to be JSON data type in the table?
I figured it out the problem. I was using the wrong MySQL node package. Need MySQL2 for the json formatting.
npm install mysql2
const mysql = require("mysql2");

NodeJS: How to send entity from mysql to rabbitmq

I'm trying to send an "entity" obtained from MySQL to RabbitMQ.
I'm able to make the connection to the database and return data. Example:
dbConnection.query("SELECT * FROM customer WHERE Id = ?", customerId, (err, rows, fields) => {
...
res.status(200).json(rows)
...
}
After this I am able to watch in Postman the "JSON result", so, I want to send this "JSON result" as an string to RabbitMQ.
I can send to RabbitMq a fake data object with no problem:
const fakeData = {
name: "Elon Musk",
company: "SpaceX",
};
channel.sendToQueue("message-queue", Buffer.from(JSON.stringify(fakeData)));
So, how must I convert the "rows" object returned from MySQL to send it to the queue?
Thank you in advance!
The solution to my problem is as follows:
rows.forEach(function (row) {
channel.sendToQueue("message-queue", Buffer.from(JSON.stringify(row)));
});

NodeJS gives MongoDB data with missing values and an unexpected $db field

I have a simple NodeJS app which is running a http server that is collecting data from a MongoDB instance and presenting the result as JSON:
db.collection(collectionName).findOne({ '_id': id }, function (err, result) {
if (err) {
reportError(err, res);
return;
} else {
outPut(result, res);
}
});
In the outPut function I'm calling JSON.stringify() on the 'result' variable and writing it in the response.
However much of the data is missing, and an empty $db object is included from somewhere. Here is a subset of the data:
"Kommun":1292,
"Lansdel":28,
"Delyta":[
{
"$id":"2",
"$db":""
},
{
"$ref":"691"
},
{
"$ref":"247"
}
Looking at the record using Studio 3T it seems that all the data I expect has been saved.
Why am I not getting all my data in the JSON object? Where is the $db coming from? What is it?
My guess is that you are using DBRefs. In order to include the referenced data from different collections, you must query those yourself. I cannot show you a code example without some more info on the data schema.

Using existing fields as objectId in mongodb to create REST api with node and express?

My JSON file is:
[
{'name': name,
'birthday': birthday,
'telephone': telephone,
'details': details},
....
]
My application will be able to accept name, birthday and telephone values, and these three values combined are going to be unique for each entity. I am hoping to retrieve an entity based on these three values, say http://localhost:8000/name?=mary&birthday?=19880902&telephone?=2234324567 will return the json object of Mary.
I am looking at the examples available on using mongodb and nodejs, most of them suggest creating a completely new _id field using new BSON.ObjectID(id) and do something similar to app.get('/users/:id', users.findById) to find the id. Is it possible not to create a separate id field?
Try using a callback with the (req, res) parameters. See here:
app.get('/', function(req, res) {
var findCursor = Users.find({
"name": req.query.name,
"birthday": req.query.birthday,
"telephone": req.query.telephone
});
cursor.each(function(err, doc) {
if (err) console.log("Oh no!");
if (doc != null) {
// Do stuff
res.send('GET request to user page?');
}
}
});

node.js read file in, parse to JSON and output

I'm very new to anything much code related, but I'm on a slow and sometimes painful learning curve.
I have a file containing some json which I read into node.js parse and push to a web socket. The script works fine, but I only ever get one json object returned.
devices.json: (Complete file) Not every element has the same data contents, and there is any number of element objects within a data source.
{
"element":"SENS01",
"data":{
"type":"SEN",
"descr":"T100"
},
"element":"SENS02",
"data":{
"type":"SEN",
"descr":"T088",
"uid":"X1A1AA",
"check_on":"2014/06/29"
},
"element":"SENS03",
"data":{
"type":"SEN",
"descr":"T000:",
"uid":"X1A1AB",
"check_on":"2014/06/29"
},
"element":"LED1",
"data":{
"type":"LED",
"state":"0"
}
}
The code which does the stuff is;
server.js:
var app = require('http').createServer(handler),
io = require('socket.io').listen(app),
fs = require('fs');
// creating the server ( localhost:8000 )
app.listen(8000);
// Server started - load page.
function handler(req, res) {
fs.readFile('/var/www/html/dashboard.html', function (err, data) {
if (err) {
console.log(err);
res.writeHead(500);
return res.end('Error loading web page');
}
res.writeHead(200);
res.end(data);
});
}
// creating a new websocket.
io.sockets.on('connection', function (socket) {
console.log();
// 1st READ of json state file.
fs.readFile('devices.json', 'utf8', function (err, data) {
if (err) throw err;
// Parse/check its valid json.
var dump = JSON.parse(data);
socket.volatile.emit('MAP.room1', dump);
});
});
When I connect to the socket the following is sent (as logged from the server console)
debug - websocket writing 5:::{"name":"MAP.room1","args":[{"element":"LED1","data":{"type":"LED","state":"0"}}]}
I never get any of the other objects, only this one. I've had a look round the net about how to iterate over objects, but it was all largely meaningless to me :(
What I am trying to achieve is when you connect to the web socket every object from the devices.json file is pushed out 1 object at a time. So once this is working I would expect to see;
debug - websocket writing 5:::{"name":"MAP.room1","args":[{"element":"LED1","data":{"type":"LED","state":"0"}}]}
debug - websocket writing 5:::{"name":"MAP.room1","args":[{"element":"SENS03","data":{"type":"SEN","descr":"T000:","uid":"X1A1AB","check_on":"2014/06/29"}}]} etc...
If I put a console.log(data) line in my server.js then I see the entire file as expected. Its only once its been parsed am I left with the 1 entry.
Can anyone please explain what's going on, and how I can best overcome this. It needs to be in a really simple way ideally using my own code/dataset as examples so I can understand 'what this means for me' A lot of the web examples and stuff I read tend to use different examples which just confuses me. I know the basics of declaring variables etc, and have an extremely limited experience with Ruby with a simple script to parse some push data received from an API but that's about it.
If you need any more context etc then please let me know, otherwise any help gratefully received.
I think your problem is that you're using the same keys in your JSON. When the parser reads in that JSON, it continuously overwrites previous values of element and data and since those are the only unique key names, those are the only two values you see.
If you modified your JSON so that the same key names are not used on the same "level," then you would see all of the data you are expecting. Here's an example that makes it easy to iterate through each element:
[
{
"element":"SENS01",
"data":{
"type":"SEN",
"descr":"T100"
}
},
{
"element":"SENS02",
"data":{
"type":"SEN",
"descr":"T088",
"uid":"X1A1AA",
"check_on":"2014/06/29"
}
},
{
"element":"SENS03",
"data":{
"type":"SEN",
"descr":"T000:",
"uid":"X1A1AB",
"check_on":"2014/06/29"
}
},
{
"element":"LED1",
"data":{
"type":"LED",
"state":"0"
}
}
]
Or if you can guarantee that the element values are always unique, then perhaps you could also do this:
{
"SENS01":{
"type":"SEN",
"descr":"T100"
},
"SENS02":{
"type":"SEN",
"descr":"T088",
"uid":"X1A1AA",
"check_on":"2014/06/29"
},
"SENS03":{
"type":"SEN",
"descr":"T000:",
"uid":"X1A1AB",
"check_on":"2014/06/29"
},
"LED1":{
"type":"LED",
"state":"0"
}
}
Ok so I found out my data was actually JS objects represented as below in a flat format with every object seperated with a linefeed.
{"SENS01":{"type":"SEN","descr":"T100"}
{"element":"LED1","data":{"type":"LED","state":"0"}
Using linereader (from npm) I was able to read the file in by doing;
lineReader.eachLine('simple.txt', function(line) {
var dump = JSON.parse(line);
socket.emit('MAP.room1', dump);
});
That then output the required data from the web socket.