Express: How to implement the JSON API - json

I have a very basic express server which simply offers model-data via mongoose for webapps. So now since in my Framework (ember) the JSON-API is the new default adapter for requesting model-data, Im wondering how to implement a response for my route in a way which respects the JSON API specification.
My current route in express looks like that.
router.get('/:id', function(req, res, next) {
postModel.find({ //Mongoose functions
_id: req.params.id
}, function(err, doc) {
res.json({ //returning the doc
data: doc //within 'doc', the 'type'-key must be added
});
});
});
I have to include a key for the "type" in each responded object so the responding object will looks like that:
{
data:{
type:"post",
id:"123",
title:"test"
}
}

Here is a better answer. My request handler in express:
router.get('/:id', function(req, res, next) {
Lookup.findById(req.params.id, function(err, result) {
if (err) {
res.send({
error: err
});
} else {
res.send(to_jsonapi(result, 'lookup'));
}
});
});
This calls a utility function which converts the mongoose result into a valid jsonapi result. You just need to call it with the result plus the 'type' value.
function to_jsonapi(result, type) {
datajson = [];
if (Array.isArray(result)) {
result.forEach(function(item) {
datajson.push({
"type": type,
"id": item._id,
"attributes": item
});
});
} else if (typeof result === "object") {
// Happens when there is only one item
datajson.push({
"type": type,
"id": result._id,
"attributes": result
});
} else {
datajson.push({
"type": type
});
}
return {
"data": datajson
};
}
This is not perfect yet but should get you down the road!

I am just running in to this same problem. There is an ember-mongoose npm package but I have not used it yet. The way they solve the problem is actually back in the ember-data side by extending the DS.RESTAdapter. There they overwrite the findQuery function.
findQuery: function(store, type, query) {
var data = { query: query };
return this.ajax(this.buildURL(type.typeKey), 'POST', { data: data });
}
This is not that elegant though since it requires a customization for every data type. You can see the example here.
Hope it helps.

Try using lean()
router.get('/:id', function(req, res, next) {
postModel
.find({_id: req.params.id})
.lean()
.then( docs => {
_.each( docs, doc => {
doc.type = "post";
});
res.json(data: docs);
});
});

If you are implementing jsonapi on server side because it will be then automatically parsed by your Ember.js Data, then I would suggest to look at different Serializers which Ember.js provides. It will solve the problem. Because it's the job of the client to parse the data in whatever way Backend provides, not that we have to make our backend compatible with clients :)

Related

Post JSON in DynamoDB via Lambda

I have trouble storing a JSON file in my DynamoDB table with the help of my Lambda function and my API Gateway on AWS. I have the following piece of code which gets executed once I press a button on my HTML site:
$('#submit').on('click', function(){
var example = {"number":"121212"};
$.ajax({
type: 'POST',
url: API_URL,
data: JSON.stringify(example),
contentType: "application/json",
success: function(data){
location.reload();
}
});
return false;
});
When pressed the website reloads, hence I assume function has successfully executed. However my problem is that the data does not arrive in the correct format in the lambda function and hence does not execute properly. When checking in CloudWatch it is shown as { number: '121212' } instead of {"number":"121212"}. Any idea how I can make sure that the value 'arrives' has a valid JSON format in my Lambda function?
Here's my Lambda function:
exports.handler = function index(e, ctx, callback) {
var params = {
Item: { number: e.number },
TableName: 'collectionOfNumbers'
};
docCLient.put(params, function(err, data) {
if (err) {
callback(err, null);
} else {
callback(null, data);
}
});
}
If I'm reading this right, e.number is the value of the JSON parameter 'number' that you are passing in, e.g. '121212'. I'm making the assumption from the usage that docClient is putItem under the hood.
I think your Item param should look like:
Item: {"number": {N: e.number}}
See AWS Docs for info regarding PutItem https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_PutItem.html

Elasticsearch js Bulk Index TypeError

Background
Recently, I have been working with the Elasticsearch Node.js API to bulk-index a large JSON file. I have successfully parsed the JSON file. Now, I should be able to pass the index-ready array into the Elasticsearch bulk command. However, using console log, it appears as though the array shouldn't be causing any problems.
Bugged Code
The following code is supposed to take an API URL (with a JSON response) and parse it using the Node.js HTTP library. Then using the Elasticsearch Node.js API, it should bulk-index every entry in the JSON array into my Elasticsearch index.
var APIUrl = /* The url to the JSON file on the API providers server. */
var bulk = [];
/*
Used to ready JSON file for indexing
*/
var makebulk = function(ParsedJSONFile, callback) {
var JSONArray = path.to.array; /* The array was nested... */
var action = { index: { _index: 'my_index', _type: 'my_type' } };
for(const item of items) {
var doc = { "id": `${item.id}`, "name": `${item.name}` };
bulk.push(action, doc);
}
callback(bulk);
}
/*
Used to index the output of the makebulk function
*/
var indexall = function(madebulk, callback) {
client.bulk({
maxRetries: 5,
index: "my_index",
type: "my_type",
body: makebulk
}, function(err, resp) {
if (err) {
console.log(err);
} else {
callback(resp.items);
}
});
}
/*
Gets the specified URL, parses the JSON object,
extracts the needed data and indexes into the
specified Elasticsearch index
*/
http.get(APIUrl, function(res) {
var body = '';
res.on('data', function(chunk) {
body += chunk;
});
res.on('end', function() {
var APIURLResponse = JSON.parse(body);
makebulk(APIURLResponse, function(resp) {
console.log("Bulk content prepared");
indexall(resp, function(res) {
console.log(res);
});
console.log("Response: ", resp);
});
});
}).on('error', function(err) {
console.log("Got an error: ", err);
});
When I run node bulk_index.js on my web server, I receive the following error: TypeError: Bulk body should either be an Array of commands/string, or a String. However, this doesn't make any sense because the console.log(res) command (From the indexall function under http.get client request) outputs the following:
Bulk content prepared
Response: [ { index: { _index: 'my_index', _type: 'my_type', _id: '1' } },
{ id: '5', name: 'The Name' }, ... },
... 120690 more items ]
The above console output appears to show the array in the correct format.
Question
What does TypeError: Bulk body should either be an Array of commands/string, or a String indicate is wrong with the array I am passing into the client.bulk function?
Notes
My server is currently running Elasticsearch 6.2.4 and Java Development Kit version 10.0.1. Everything works as far as the Elaticsearch server and even my Elaticsearch mappings (I didn't provide the client.indices.putMapping code, however I can if it is needed). I have spent multiple hours reading over every scrap of documentation I could find regarding this TypeError. I couldn't find much in regards to the error being thrown, so I am not sure where else to look for information regarding this error.
Seems a typo in your code.
var indexall = function(**madebulk**, callback) {
client.bulk({
maxRetries: 5,
index: "my_index",
type: "my_type",
body: **makebulk**
Check the madebulk & makebulk.

How to dynamically read external json files in node.js?

I am creating a website that reads externally hosted json files and then uses node.js to populate the sites content.
Just to demonstrate what I'm after, this is a really simplified version of what I'm trying to do in node.js
var ids = [111, 222, 333];
ids.forEach(function(id){
var json = getJSONsomehow('http://www.website.com/'+id+'.json');
buildPageContent(json);
});
Is what I want to do possible?
(Marked as a duplicate of "How do I return the response from an asynchronous call?" see my comment below for my rebuttal)
You are trying to get it synchronously. What you should aim for instead, is not a function used like this:
var json = getJSONsomehow('http://www.website.com/'+id+'.json');
but more like this:
getJSONsomehow('http://www.website.com/'+id+'.json', function (err, json) {
if (err) {
// error
} else {
// your json can be used here
}
});
or like this:
getJSONsomehow('http://www.website.com/'+id+'.json')
.then(function (json) {
// you can use your json here
})
.catch(function (err) {
// error
});
You can use the request module to get your data with something like this:
var request = require('request');
var url = 'http://www.website.com/'+id+'.json';
request.get({url: url, json: true}, (err, res, data) => {
if (err) {
// handle error
} else if (res.statusCode === 200) {
// you can use data here - already parsed as json
} else {
// response other than 200 OK
}
});
For a working example see this answer.
For more info see: https://www.npmjs.com/package/request
I think problem is in async request. Function will return result before request finished.
AJAX_req.open( "GET", url, true );
Third parameter specified async request.
You should add handler and do all you want after request finished.
For example:
function AJAX_JSON_Req( url ) {
var AJAX_req = new XMLHttpRequest.XMLHttpRequest();
AJAX_req.open( "GET", url, true );
AJAX_req.setRequestHeader("Content-type", "application/json");
AJAX_req.onreadystatechange = function() {
if (AJAX_req.readyState == 4 && AJAX_req.status == 200) {
console.log(AJAX_req.responseText);
}
};
}

Get specific fields in JSON

I have a problem with getting some JSON fields in NodeJS.
Here is my JSON structure:
{
"header":
{
"file1":0,
"file2":1,
"subfiles":{
"subfile1":"true",
"subfile2":"true",
}
},
"response":
{
"number":678,
"start":0,
"docs":[
{
"id":"d3f3d",
"code":"l876s",
"country_name":"United States",
"city":"LA"
},
{
"id":"d2f2d",
"code":"2343g",
"country_name":"UK",
"city":"London"
}
]
}
}
How to access to specific fields (id, city or country_name) in structure like this?
I'm trying something in NodeJS, but I cannot get specific fields.
request('http://url.com', function (error, res, body) {
if (!error && res.statusCode == 200) {
console.log(body)
}
res.on('data', function(chunk){
body += chunk;
});
res.on('end', function(){
var urlResponse = JSON.parse(body);
console.log("Got a response: ", urlResponse.response.docs.city);
});
});
Have you tried something like this:
thatObject.response.docs.forEach((element) => { console.log( element.id ); });
?
property docks is an array, so you can use functions: forEach, or for, or whatever you wish
Try using using a data variable to accumulate your chunks and then data.toString() to force the buffered content to string, which will be parseable JSON.
Additionally, you need to access the specific member of the docs array.

Node Exports Function Returning Undefined

I have a an exports function I'm calling that should return a json array of draft results. In the route below in app.js, when I console.log draft_results, I get undefined
app.get('/draft-results', function(req, res) {
var draft_results = fantasy.getDraftResults(req, res);
console.log(util.inspect(draft_results, false, null));
//looks in views folder by default
res.render('draft-results', {
draft_results: draft_results
});
});
In my other file, this is the function that should be returning the json array. If i console.log draft, the data is there.
exports.getDraftResults = function(req, res, cb) {
oauth.get(
"http://fantasysports.yahooapis.com/fantasy/v2/league/" + conf.LEAGUE_ID + "/draftresults?format=json",
req.user.accessToken,
req.user.tokenSecret,
function(e, data, resp) {
if (e) console.error(e);
data = JSON.parse(data);
var draft = data.fantasy_content.league[1].draft_results;
res.json(draft);
}
);
};
I feel like I am returning the data incorrectly, and I can't seem to find any other good examples out there. Could someone please assist?
getDraftResults() is asynchronous. That means the results it generates occur sometime later. Thus, it cannot return its results directly from the function like you are trying to use.
It is unclear what you want to be doing here. Inside of getDraftResults() you are creating a JSON response back to the web request that started all this. That, in itself would be fine and will work as you have it (except the error handling is missing).
But, in your app.get() handler, you have completely different code that seems to thing that getDraftResults() is going to return a value (it has no return value at all) and then you will later use that return value.
So, if you just want getDraftResults to make a JSON response to the original web request, it's already doing that and you can remove the rest of what you have in the app.get() handler. If that's not really what you want to do and you want to use the response from getDraftResults() inside of the app.get() handler, then you will have to change the design of both functions and likely pass a callback to getDraftResults() so the callback can supply the asynchronous response and you can then continue the rest of the app.get() functionality in that callback.
If you're trying to do the latter, then here's a scaffolding (I don't know exactly what you're trying to accomplish so I can't be too detailed here):
app.get('/draft-results', function(req, res) {
fantasy.getDraftResults(req, function(err, draft_results) {
if (err) {
// send some sort of error response here
console.error(err);
return;
}
console.log(util.inspect(draft_results, false, null));
//looks in views folder by default
res.render('draft-results', {
draft_results: draft_results
});
});
});
exports.getDraftResults = function(req, cb) {
oauth.get(
"http://fantasysports.yahooapis.com/fantasy/v2/league/" + conf.LEAGUE_ID + "/draftresults?format=json",
req.user.accessToken,
req.user.tokenSecret,
function(e, data, resp) {
if (e) {
console.error(e);
cb(e);
return;
}
data = JSON.parse(data);
var draft = data.fantasy_content.league[1].draft_results;
// send results back to caller
cb(null, draft);
}
);
};