I'm very new to anything much code related, but I'm on a slow and sometimes painful learning curve.
I have a file containing some json which I read into node.js parse and push to a web socket. The script works fine, but I only ever get one json object returned.
devices.json: (Complete file) Not every element has the same data contents, and there is any number of element objects within a data source.
{
"element":"SENS01",
"data":{
"type":"SEN",
"descr":"T100"
},
"element":"SENS02",
"data":{
"type":"SEN",
"descr":"T088",
"uid":"X1A1AA",
"check_on":"2014/06/29"
},
"element":"SENS03",
"data":{
"type":"SEN",
"descr":"T000:",
"uid":"X1A1AB",
"check_on":"2014/06/29"
},
"element":"LED1",
"data":{
"type":"LED",
"state":"0"
}
}
The code which does the stuff is;
server.js:
var app = require('http').createServer(handler),
io = require('socket.io').listen(app),
fs = require('fs');
// creating the server ( localhost:8000 )
app.listen(8000);
// Server started - load page.
function handler(req, res) {
fs.readFile('/var/www/html/dashboard.html', function (err, data) {
if (err) {
console.log(err);
res.writeHead(500);
return res.end('Error loading web page');
}
res.writeHead(200);
res.end(data);
});
}
// creating a new websocket.
io.sockets.on('connection', function (socket) {
console.log();
// 1st READ of json state file.
fs.readFile('devices.json', 'utf8', function (err, data) {
if (err) throw err;
// Parse/check its valid json.
var dump = JSON.parse(data);
socket.volatile.emit('MAP.room1', dump);
});
});
When I connect to the socket the following is sent (as logged from the server console)
debug - websocket writing 5:::{"name":"MAP.room1","args":[{"element":"LED1","data":{"type":"LED","state":"0"}}]}
I never get any of the other objects, only this one. I've had a look round the net about how to iterate over objects, but it was all largely meaningless to me :(
What I am trying to achieve is when you connect to the web socket every object from the devices.json file is pushed out 1 object at a time. So once this is working I would expect to see;
debug - websocket writing 5:::{"name":"MAP.room1","args":[{"element":"LED1","data":{"type":"LED","state":"0"}}]}
debug - websocket writing 5:::{"name":"MAP.room1","args":[{"element":"SENS03","data":{"type":"SEN","descr":"T000:","uid":"X1A1AB","check_on":"2014/06/29"}}]} etc...
If I put a console.log(data) line in my server.js then I see the entire file as expected. Its only once its been parsed am I left with the 1 entry.
Can anyone please explain what's going on, and how I can best overcome this. It needs to be in a really simple way ideally using my own code/dataset as examples so I can understand 'what this means for me' A lot of the web examples and stuff I read tend to use different examples which just confuses me. I know the basics of declaring variables etc, and have an extremely limited experience with Ruby with a simple script to parse some push data received from an API but that's about it.
If you need any more context etc then please let me know, otherwise any help gratefully received.
I think your problem is that you're using the same keys in your JSON. When the parser reads in that JSON, it continuously overwrites previous values of element and data and since those are the only unique key names, those are the only two values you see.
If you modified your JSON so that the same key names are not used on the same "level," then you would see all of the data you are expecting. Here's an example that makes it easy to iterate through each element:
[
{
"element":"SENS01",
"data":{
"type":"SEN",
"descr":"T100"
}
},
{
"element":"SENS02",
"data":{
"type":"SEN",
"descr":"T088",
"uid":"X1A1AA",
"check_on":"2014/06/29"
}
},
{
"element":"SENS03",
"data":{
"type":"SEN",
"descr":"T000:",
"uid":"X1A1AB",
"check_on":"2014/06/29"
}
},
{
"element":"LED1",
"data":{
"type":"LED",
"state":"0"
}
}
]
Or if you can guarantee that the element values are always unique, then perhaps you could also do this:
{
"SENS01":{
"type":"SEN",
"descr":"T100"
},
"SENS02":{
"type":"SEN",
"descr":"T088",
"uid":"X1A1AA",
"check_on":"2014/06/29"
},
"SENS03":{
"type":"SEN",
"descr":"T000:",
"uid":"X1A1AB",
"check_on":"2014/06/29"
},
"LED1":{
"type":"LED",
"state":"0"
}
}
Ok so I found out my data was actually JS objects represented as below in a flat format with every object seperated with a linefeed.
{"SENS01":{"type":"SEN","descr":"T100"}
{"element":"LED1","data":{"type":"LED","state":"0"}
Using linereader (from npm) I was able to read the file in by doing;
lineReader.eachLine('simple.txt', function(line) {
var dump = JSON.parse(line);
socket.emit('MAP.room1', dump);
});
That then output the required data from the web socket.
Related
So let's say I've got a massive JSON file, and the general structure is roughly like so:
{
"apples": { complex object },
"oranges": { complex object },
"grapes": { complex object }
}
Is there some way to specifically target an object to return while using express? As in, say, if someone made a simple get request to my server, it'd return specifically the given object(s). I know the syntax and concept is completely wrong in this instance but for lack of a better way to say it, something like...
let testData = 'testdata.json';
app.get('/thing', res => {
res.json(testData.oranges);
}
I know you can return the entire file, but that adds a good amount of loading time in this instance, and is impractical in this particular case.
Or, alternatively - would it be better to have node parse the JSON file and split it into an apples.json, oranges.json, etc files to use? Trying to understand A, the best practice for doing something like this, and B, the most effective way to translate this into a practical application for a medium sized project.
Any thoughts or advice along this line - even if it's a library recommendation - would be greatly appreciated.
It should work if you make a POST request caring the payload of the specific 'thing', and then returning an object based on that thing. Example:
let testData = {
"apples": { complex object },
"oranges": { complex object },
"grapes": { complex object }
};
app.post('/route', (req, res) => {
thing = req.body.thing;
res.json(testData[thing]);
}
This is a GET request for some data and essentially since the JSON file can be used as a key/value store to query for the desired response data.
Assuming the query parameter for specifying the desired key for the object to return is part then the following example would work:
const testData = require('./testdata.json');
app.get('/thing', (req, res) => res.json(testdata[req.query.part]);
Querying for /thing?part=apples would return testdata.apples in the response.
I have a simple NodeJS app which is running a http server that is collecting data from a MongoDB instance and presenting the result as JSON:
db.collection(collectionName).findOne({ '_id': id }, function (err, result) {
if (err) {
reportError(err, res);
return;
} else {
outPut(result, res);
}
});
In the outPut function I'm calling JSON.stringify() on the 'result' variable and writing it in the response.
However much of the data is missing, and an empty $db object is included from somewhere. Here is a subset of the data:
"Kommun":1292,
"Lansdel":28,
"Delyta":[
{
"$id":"2",
"$db":""
},
{
"$ref":"691"
},
{
"$ref":"247"
}
Looking at the record using Studio 3T it seems that all the data I expect has been saved.
Why am I not getting all my data in the JSON object? Where is the $db coming from? What is it?
My guess is that you are using DBRefs. In order to include the referenced data from different collections, you must query those yourself. I cannot show you a code example without some more info on the data schema.
I have a basic Angular web app which reads from a JSON file located on the same server as the app and parses through the JSON file in order to set certain values on objects which drive certain behavior in my app (applies css classes, etc.)
I am not able to find online and/or figure out myself how to set up the controller to read from the JSON file in a way that allows the file to be changed and Angular to dynamically reload the file once it has been changed without reloading the entire page. The JSON file is local on the server where the app is deployed, and I wanted to avoid standing up a web service just to serve a file that already exists on the same server the app is deployed.
Here is what I am doing now:
ngOnInit(): void {
// Make the HTTP request:
this.http.get('../assets/applicationLogs.json').subscribe(data => {
// Read the result field from the JSON response.
this.node_a_status= data.nodes[0].status;
this.node_b_status= data.nodes[1].status;
this.node_c_status= data.nodes[2].status;
});
}
And here is a what my JSON file looks like:
{
"nodes":[
{ "node":"Node A", "status":"processing", "errors":null },
{ "node":"Node B", "status":"processing", "errors":null },
{ "node":"Node C", "status":"inactive", "errors":null }
]
}
First, I know I will probably need to move this get logic out of ngOnInit(), but I am a little lost on how I should go about achieving the desired behavior I have described with typescript.
You're using an http request method on the file so "Poll it"... same way you would any other http JSON service. Here's a ready made poller for you to import: https://www.npmjs.com/package/rx-polling
Best thing you can do is create a service out of it and call it in ngOnInit method and use the response the same way you've shown.
import { Observable } from 'rxjs/Observable';
import 'rxjs/add/observable/dom/ajax';
import 'rxjs/add/operator/map';
import polling from 'rx-polling';
// Example of an Observable which requests some JSON data
const request$ = Observable.ajax({
url: '../assets/applicationLogs.json',
crossDomain: true
}).map(response => response.response || [])
.map(response => response.slice(0, 10)); // Take only first 10 comments
polling(request$, { interval: 5000 }).subscribe((comments) => {
console.log(comments);
}, (error) => {
// The Observable will throw if it's not able to recover after N attempts
// By default it will attempts 9 times with exponential delay between each other.
console.error(error);
});
I've been researching this question to no-ends, but can't find the simple answer I'm looking for. Basically, I'd like to batch POST JSON objects in array.
I've got a a giant array of JSON objects.
[
{
"Name": "SEARCH Resource Center",
"Address": "2505 Fannin St, Houston, TX 77002",
"Phone": "(713) 739-7752",
"Hours": "Mon-Fri, 8am to 3pm",
"Category": "Drop-In Centers"
},
{
"Name": "Salvation Army Social Services - Young Adult Resource Center",
"Address": "2208 Main St, Houston, TX 77002",
"Phone": "(713) 658-9205",
"Hours": "Mon-Thurs, 11am to 3pm",
"Category": "Drop-In Centers"
},
...
]
I'm using an Express server that handles post requests looks like this:
app.post('/api/orgs', function(req, res) {
// Creates a new User based on the Mongoose schema and the post body
var newOrg = new Organization(req.body);
// New User is saved in the db.
newOrg.save(function(err){
if(err)
res.send(err);
// If no errors are found, it responds with a JSON of the new user
res.json(req.body);
});
});
These objects are then saved in MongoDB as individual records.
I'm using POSTMAN to send HTTP POSTs to my Express Server. As of now, I've been sending all of my JSON POSTS one at a time, because I can't figure out the best way to batch post all the sub-objects stored in the array as individual objects.
Any suggestions or best practices?
If you send you array as a key in your request body, something like this
You'll get it as req.body.my_restaurants. Then simply use this :
db.collection('restaurants').insertMany(req.body.my_restaurants, function(err, restaurants){
if(err) console.log(err);
else console.log("restaurants Added Successfully");
});
I'm assuming that restaurants is the name of your collection.
I've been playing with the previous solution I marked as correct. An even better approach than using the .insertMany() function is to use the .create() function.
The .insertMany() function skips the middleware associated with .save(), whereas the create() function uses the same process, but can also handle arrays.
So my modified express route looked like the below (where Organization is the name of my schema):
app.post('/api/orgs', function(req, res) {
// Array of JSON Objects
if (req.body.batch){
Organization.create(req.body.batch, function(err){
if(err)
res.send(err);
else
res.json(req.body);
});
}
// Single JSON Object
else {
var newOrg = new Organization(req.body);
// New User is saved in the db.
newOrg.save(function(err){
if(err)
res.send(err);
// If no errors are found, it responds with a JSON of the new user
else
res.json(req.body);
});
}
});
And I'm sending JSON objects that look like:
Hope this helps someone else down the line.
I am creating a web application that allows a user to load data in JSON format. I am currently using the following function to read JSON files that I have saved on my local disk:
function retrieveJSON(url, callback)
{
// this is needed because FireFox tries to parse files as XML
$.ajaxSetup({ mimeType: "text/plain" });
// send out an AJAX request and return the result
$.getJSON(url, function(response) {
console.log("Data acquired successfully");
callback(response);
}).error(function(jqXHR, textStatus, errorThrown) {
console.log("Error...\n" + textStatus + "\n" + errorThrown);
});
}
This works perfectly for well-formed JSON data. However, for malformed data, the console log displays the following:
Error...
parsererror
SyntaxError: JSON.parse: unexpected character
This is almost entirely unhelpful because it does not tell me what the unexpected character is or what line number it can be found on. I could use a JSON validator to correct the file on my local disk, but this is not an option when the page is loading files from remote URLs on the web.
How can I obtain the location of any error? I would like to obtain the token if possible, but I need to obtain the line number at minimum. There is a project requirement to display an excerpt of the JSON code to the user and highlight the line where any error occurred.
I am currently using jQuery, but jQuery is not a project requirement, so if another API or JSON parser provides this functionality, I could use that instead.
Yeah, life with deadlines is never easy :).
This might help you out, after couple of hours googling around, I've found jsonlint on Git Hub. It looks promising, it includes a shell script that could be used on server side, and there is a browser JavaScript version of it that seems to be exactly what you were looking for.
Hope that this will help You.
i agree that life with deadlines is hard.
i'm incredibly happy that i don't have to live with deadlines, i'm my own boss.
so in search of a better solution to this problem, i came up with the following :
...
readConfig : function () {
jQuery.ajax({
type : 'GET',
url : 'config.json',
success : function (data, ts, xhr) {
var d = JSON.parse(data);
},
error : function (xhr, ajaxOptions, thrownError) {
if (typeof thrownError.message=='string') {
// ./config.json contains invalid data.
var
text = xhr.responseText,
pos = parseInt(thrownError.message.match(/position (\d+)/)[1]),
html = text.substr(0,pos)+'<span style="color:red;font-weight:bold;">__'+text.substr(pos,1)+'__</span>'+text.substr(pos+1, text.length-pos-1);
cm.install.displayErrorMsg('Could not read ./config.json :(<br/>'+thrownError+'<br/>'+html);
} else {
cm.install.displayErrorMsg('Error retrieving ./config.json<br/>HTTP error code : '+xhr.status);
};
}
});
},
...