I have a problem with getting some JSON fields in NodeJS.
Here is my JSON structure:
{
"header":
{
"file1":0,
"file2":1,
"subfiles":{
"subfile1":"true",
"subfile2":"true",
}
},
"response":
{
"number":678,
"start":0,
"docs":[
{
"id":"d3f3d",
"code":"l876s",
"country_name":"United States",
"city":"LA"
},
{
"id":"d2f2d",
"code":"2343g",
"country_name":"UK",
"city":"London"
}
]
}
}
How to access to specific fields (id, city or country_name) in structure like this?
I'm trying something in NodeJS, but I cannot get specific fields.
request('http://url.com', function (error, res, body) {
if (!error && res.statusCode == 200) {
console.log(body)
}
res.on('data', function(chunk){
body += chunk;
});
res.on('end', function(){
var urlResponse = JSON.parse(body);
console.log("Got a response: ", urlResponse.response.docs.city);
});
});
Have you tried something like this:
thatObject.response.docs.forEach((element) => { console.log( element.id ); });
?
property docks is an array, so you can use functions: forEach, or for, or whatever you wish
Try using using a data variable to accumulate your chunks and then data.toString() to force the buffered content to string, which will be parseable JSON.
Additionally, you need to access the specific member of the docs array.
Related
Background
Recently, I have been working with the Elasticsearch Node.js API to bulk-index a large JSON file. I have successfully parsed the JSON file. Now, I should be able to pass the index-ready array into the Elasticsearch bulk command. However, using console log, it appears as though the array shouldn't be causing any problems.
Bugged Code
The following code is supposed to take an API URL (with a JSON response) and parse it using the Node.js HTTP library. Then using the Elasticsearch Node.js API, it should bulk-index every entry in the JSON array into my Elasticsearch index.
var APIUrl = /* The url to the JSON file on the API providers server. */
var bulk = [];
/*
Used to ready JSON file for indexing
*/
var makebulk = function(ParsedJSONFile, callback) {
var JSONArray = path.to.array; /* The array was nested... */
var action = { index: { _index: 'my_index', _type: 'my_type' } };
for(const item of items) {
var doc = { "id": `${item.id}`, "name": `${item.name}` };
bulk.push(action, doc);
}
callback(bulk);
}
/*
Used to index the output of the makebulk function
*/
var indexall = function(madebulk, callback) {
client.bulk({
maxRetries: 5,
index: "my_index",
type: "my_type",
body: makebulk
}, function(err, resp) {
if (err) {
console.log(err);
} else {
callback(resp.items);
}
});
}
/*
Gets the specified URL, parses the JSON object,
extracts the needed data and indexes into the
specified Elasticsearch index
*/
http.get(APIUrl, function(res) {
var body = '';
res.on('data', function(chunk) {
body += chunk;
});
res.on('end', function() {
var APIURLResponse = JSON.parse(body);
makebulk(APIURLResponse, function(resp) {
console.log("Bulk content prepared");
indexall(resp, function(res) {
console.log(res);
});
console.log("Response: ", resp);
});
});
}).on('error', function(err) {
console.log("Got an error: ", err);
});
When I run node bulk_index.js on my web server, I receive the following error: TypeError: Bulk body should either be an Array of commands/string, or a String. However, this doesn't make any sense because the console.log(res) command (From the indexall function under http.get client request) outputs the following:
Bulk content prepared
Response: [ { index: { _index: 'my_index', _type: 'my_type', _id: '1' } },
{ id: '5', name: 'The Name' }, ... },
... 120690 more items ]
The above console output appears to show the array in the correct format.
Question
What does TypeError: Bulk body should either be an Array of commands/string, or a String indicate is wrong with the array I am passing into the client.bulk function?
Notes
My server is currently running Elasticsearch 6.2.4 and Java Development Kit version 10.0.1. Everything works as far as the Elaticsearch server and even my Elaticsearch mappings (I didn't provide the client.indices.putMapping code, however I can if it is needed). I have spent multiple hours reading over every scrap of documentation I could find regarding this TypeError. I couldn't find much in regards to the error being thrown, so I am not sure where else to look for information regarding this error.
Seems a typo in your code.
var indexall = function(**madebulk**, callback) {
client.bulk({
maxRetries: 5,
index: "my_index",
type: "my_type",
body: **makebulk**
Check the madebulk & makebulk.
I am creating a website that reads externally hosted json files and then uses node.js to populate the sites content.
Just to demonstrate what I'm after, this is a really simplified version of what I'm trying to do in node.js
var ids = [111, 222, 333];
ids.forEach(function(id){
var json = getJSONsomehow('http://www.website.com/'+id+'.json');
buildPageContent(json);
});
Is what I want to do possible?
(Marked as a duplicate of "How do I return the response from an asynchronous call?" see my comment below for my rebuttal)
You are trying to get it synchronously. What you should aim for instead, is not a function used like this:
var json = getJSONsomehow('http://www.website.com/'+id+'.json');
but more like this:
getJSONsomehow('http://www.website.com/'+id+'.json', function (err, json) {
if (err) {
// error
} else {
// your json can be used here
}
});
or like this:
getJSONsomehow('http://www.website.com/'+id+'.json')
.then(function (json) {
// you can use your json here
})
.catch(function (err) {
// error
});
You can use the request module to get your data with something like this:
var request = require('request');
var url = 'http://www.website.com/'+id+'.json';
request.get({url: url, json: true}, (err, res, data) => {
if (err) {
// handle error
} else if (res.statusCode === 200) {
// you can use data here - already parsed as json
} else {
// response other than 200 OK
}
});
For a working example see this answer.
For more info see: https://www.npmjs.com/package/request
I think problem is in async request. Function will return result before request finished.
AJAX_req.open( "GET", url, true );
Third parameter specified async request.
You should add handler and do all you want after request finished.
For example:
function AJAX_JSON_Req( url ) {
var AJAX_req = new XMLHttpRequest.XMLHttpRequest();
AJAX_req.open( "GET", url, true );
AJAX_req.setRequestHeader("Content-type", "application/json");
AJAX_req.onreadystatechange = function() {
if (AJAX_req.readyState == 4 && AJAX_req.status == 200) {
console.log(AJAX_req.responseText);
}
};
}
I'm trying to access JSON Object properties directly and log it, here is my function :
loadProcesses(filter?){
this._postService.getAllProcess(filter)
.subscribe(
res=> {
this.processListe = res;
// console.log(this.processListe.)
}
,null,
() =>{
console.log("get processes liste" + filter)
});
So this.processListe contain a JSON Object, and my JSON format is like this:
{"Person": {
"id": "A256",
"name": "GET",
"status": "active",
"description": "hardworking, openminded",
...
So it will contains exactly the same things, for example if i want to simply print the label on a console log how can i do it ??
Are you looking for something like this:
function parseObject(obj)
{
for(var key in obj)
{
console.log("key: " + key + ", value: " + obj[key])
if(obj[key] instanceof Object)
{
parseObject(obj[key]);
}
}
}
just call parseObject(res) in the subscribe method.
parse it and access the fields.
var obj = JSON.parse(filter);
obj.Person.id;
//etc
parse it in the .subscribe:
res => this.processListe = res.json();
a better solution is to declare your response with any :
loadProcesses(filter?){
this._postService.getAllProcess(filter)
.subscribe(
(res: any)=> {
this.processListe = res;
// console.log(this.processListe.)
}
,null,
() =>{
console.log("get processes liste" + filter)
});
this way you can access any attirbute in your response
I have a JSON file that contains multiple objects of the same structure that look like this:
{
"id": "123",
"type": "alpha"
}
{
"id": "321",
"type": "beta"
}
I'm using node.js to read the file.
fs.readFile(__dirname + "/filename.json", 'utf8', function(err, data) {
var content = JSON.parse(JSON.stringify(data));
If I do a console.log(content) things look good. I see the content of the json file. I'm trying to iterate over each object but I'm not sure how to do that. I've tried using
for(var doc in content)
but the doc isn't each object as I was expecting. How do I loop over the content to get each object in a json format so that I can parse it?
If content is an array, you can use
content.forEach(function (obj, index) { /* your code */ })
See documentation for Array.prototype.forEach()
if you need to just iterate, a forEach loop would work or a normal for loop :
for(var i = 0; i<content.length(); i++){
//perform whatever you need on the following object
var myobject = content[i];
}
Depend of the files, the two current answer (Osama and Daniel) assume you have a JSON Array:
[
{
"id": "123",
"type": "alpha"
},
{
"id": "456",
"type": "beta"
}
]
In which case, you can use any array iterator:
var async = require('async'),
content = require(__dirname + "/filename.json");
async.each(content, function (item, callback) {
//...
});
But in your case, it seems to not be JSON (no bracket to indicate array, and no comma to separate the objects), so in the case JSON.parse doesn t throw up any error, you'll need to isolate your objects first:
var fs = require('fs'),
async = require('async');
fs.readFile(__dirname + "/filename.notjson", 'utf8', function(err, data) {
var content = data.split('}');
async.map(content, function (item, callback) {
callback(null, JSON.parse(item));
}, function (err, content) {
console.log(content);
};
});
I have a very basic express server which simply offers model-data via mongoose for webapps. So now since in my Framework (ember) the JSON-API is the new default adapter for requesting model-data, Im wondering how to implement a response for my route in a way which respects the JSON API specification.
My current route in express looks like that.
router.get('/:id', function(req, res, next) {
postModel.find({ //Mongoose functions
_id: req.params.id
}, function(err, doc) {
res.json({ //returning the doc
data: doc //within 'doc', the 'type'-key must be added
});
});
});
I have to include a key for the "type" in each responded object so the responding object will looks like that:
{
data:{
type:"post",
id:"123",
title:"test"
}
}
Here is a better answer. My request handler in express:
router.get('/:id', function(req, res, next) {
Lookup.findById(req.params.id, function(err, result) {
if (err) {
res.send({
error: err
});
} else {
res.send(to_jsonapi(result, 'lookup'));
}
});
});
This calls a utility function which converts the mongoose result into a valid jsonapi result. You just need to call it with the result plus the 'type' value.
function to_jsonapi(result, type) {
datajson = [];
if (Array.isArray(result)) {
result.forEach(function(item) {
datajson.push({
"type": type,
"id": item._id,
"attributes": item
});
});
} else if (typeof result === "object") {
// Happens when there is only one item
datajson.push({
"type": type,
"id": result._id,
"attributes": result
});
} else {
datajson.push({
"type": type
});
}
return {
"data": datajson
};
}
This is not perfect yet but should get you down the road!
I am just running in to this same problem. There is an ember-mongoose npm package but I have not used it yet. The way they solve the problem is actually back in the ember-data side by extending the DS.RESTAdapter. There they overwrite the findQuery function.
findQuery: function(store, type, query) {
var data = { query: query };
return this.ajax(this.buildURL(type.typeKey), 'POST', { data: data });
}
This is not that elegant though since it requires a customization for every data type. You can see the example here.
Hope it helps.
Try using lean()
router.get('/:id', function(req, res, next) {
postModel
.find({_id: req.params.id})
.lean()
.then( docs => {
_.each( docs, doc => {
doc.type = "post";
});
res.json(data: docs);
});
});
If you are implementing jsonapi on server side because it will be then automatically parsed by your Ember.js Data, then I would suggest to look at different Serializers which Ember.js provides. It will solve the problem. Because it's the job of the client to parse the data in whatever way Backend provides, not that we have to make our backend compatible with clients :)