I am new to NodeJS and trying to figure out how to load some xml data from a REST service and convert it to JSON so I can afterwards load it into my view.
I'm trying to do so querying with Hapi API and loading it into an xml parser then convert it to JSON.
Doing the following seems loading correctly the xml object and when printing it it actually shows me some JSON. Does it mean that I don't need to convert to JSON anymore?
const server = new Hapi.Server();
...
server.route({
method: 'GET',
path: '/',
handler: function (request, reply) {
Request.get('http://ws.seloger.com/search.xml?&idtt=2&idtypebien=2,1&ci=750056&pxmax=400000&tri=initial&naturebien=1,2&surfacemin=65search.xml?',
function (error, response, body) {
if (error) {
throw error;
}
var parse = require('xml-parser');
var inspect = require('util').inspect;
var obj = parse(body);
console.log(inspect(obj, { colors: true, depth: 4 }));
Note that the JSON displayed is also not what I am looking for, ie. showing the details with attributes, children, etc.:
{ declaration: { attributes: { version: '1.0', encoding: 'UTF-8' } },
root:
{ name: 'recherche',
attributes: {},
children:
[ { name: 'resume',
attributes: {},
children: [],
content: '....' },
However looking for something more like this (might be just a different representation)
So I figured out that my issue seemed to have been related to the parser itself. Using this one I get what I need:
var parseString = require('xml2js').parseString;
parseString(body, function (err, jsonData) {
reply.view('index', { result: body });
});
Related
Background
Recently, I have been working with the Elasticsearch Node.js API to bulk-index a large JSON file. I have successfully parsed the JSON file. Now, I should be able to pass the index-ready array into the Elasticsearch bulk command. However, using console log, it appears as though the array shouldn't be causing any problems.
Bugged Code
The following code is supposed to take an API URL (with a JSON response) and parse it using the Node.js HTTP library. Then using the Elasticsearch Node.js API, it should bulk-index every entry in the JSON array into my Elasticsearch index.
var APIUrl = /* The url to the JSON file on the API providers server. */
var bulk = [];
/*
Used to ready JSON file for indexing
*/
var makebulk = function(ParsedJSONFile, callback) {
var JSONArray = path.to.array; /* The array was nested... */
var action = { index: { _index: 'my_index', _type: 'my_type' } };
for(const item of items) {
var doc = { "id": `${item.id}`, "name": `${item.name}` };
bulk.push(action, doc);
}
callback(bulk);
}
/*
Used to index the output of the makebulk function
*/
var indexall = function(madebulk, callback) {
client.bulk({
maxRetries: 5,
index: "my_index",
type: "my_type",
body: makebulk
}, function(err, resp) {
if (err) {
console.log(err);
} else {
callback(resp.items);
}
});
}
/*
Gets the specified URL, parses the JSON object,
extracts the needed data and indexes into the
specified Elasticsearch index
*/
http.get(APIUrl, function(res) {
var body = '';
res.on('data', function(chunk) {
body += chunk;
});
res.on('end', function() {
var APIURLResponse = JSON.parse(body);
makebulk(APIURLResponse, function(resp) {
console.log("Bulk content prepared");
indexall(resp, function(res) {
console.log(res);
});
console.log("Response: ", resp);
});
});
}).on('error', function(err) {
console.log("Got an error: ", err);
});
When I run node bulk_index.js on my web server, I receive the following error: TypeError: Bulk body should either be an Array of commands/string, or a String. However, this doesn't make any sense because the console.log(res) command (From the indexall function under http.get client request) outputs the following:
Bulk content prepared
Response: [ { index: { _index: 'my_index', _type: 'my_type', _id: '1' } },
{ id: '5', name: 'The Name' }, ... },
... 120690 more items ]
The above console output appears to show the array in the correct format.
Question
What does TypeError: Bulk body should either be an Array of commands/string, or a String indicate is wrong with the array I am passing into the client.bulk function?
Notes
My server is currently running Elasticsearch 6.2.4 and Java Development Kit version 10.0.1. Everything works as far as the Elaticsearch server and even my Elaticsearch mappings (I didn't provide the client.indices.putMapping code, however I can if it is needed). I have spent multiple hours reading over every scrap of documentation I could find regarding this TypeError. I couldn't find much in regards to the error being thrown, so I am not sure where else to look for information regarding this error.
Seems a typo in your code.
var indexall = function(**madebulk**, callback) {
client.bulk({
maxRetries: 5,
index: "my_index",
type: "my_type",
body: **makebulk**
Check the madebulk & makebulk.
I am using https://github.com/pablojim/highcharts-ng to build Highcharts in my angularjs app. As suggested on that site I am configuring my chart configs in my controller as follows:
$scope.chartConfig6 = {
options: {
chart: {
type: 'bar'
}
},
series: [{
data: [33, 55, 10, 18, 17]
}],
title: {
text: 'Agent FNOL Ranking'
},
loading: false
}
Now I have a json file: http://pastebin.com/SEu8dHkB if you do a search you can find the field called "highChart" on the json which contains the configurations for a highchart. Now I want to be able to use these configurations directly from the json to to my highcharts config either in the controller or as a directive. This is how I get the json file via http in my service:
.factory('Swoop',function($http,$filter,$q){
return {
all: function(){
var deferred = $q.defer();
$http.get("swoop.json").success(
function(data){
// angular.copy(data, studies);
deferred.resolve(data);
}
);
return deferred.promise;
}
}})
However, I am not sure how I could render this data(under the highChart object in the json) to the $scope.chartConfig6 like I did above. Can someone please show me how this could be done?
I'm using the JQuery UI autocomplete plugin (cached version) with JQuery-UI 1.11.1
Due to some server-side changes in the JSON I am using as source, I need to adapt my code.
Here is an example of my JSON:
[{
"name": "Varedo"
}, {
"name": "Varena"
}, {
"name": "Varenna"
}, {
"name": "Varese"
}]
produced by an URL with this style:
[url]/?name=vare
Since the GET variable is different from the default one ("term"), I already adapted my code for the custom request as suggested here:
$(function () {
var cache = {};
$("#searchTextField").autocomplete({
minLength: 3,
source: function (request, response) {
var term = request.term;
if (term in cache) {
response(cache[term]);
return;
}
$.getJSON("[url]", {
name: request.term
}, function (data, status, xhr) {
cache[term] = data;
response(data);
});
}
});
});
However I need to also adapt the code in order to use a custom JSON value (the default is "value" http://api.jqueryui.com/autocomplete/#option-source) which is in my case is "name" (as you can see from the JSON).
How can I do that?
At the moment this is what I get from the autocomplete:
So I guess I am somehow giving as response JS Objects and not strings.
Thanks in advance.
Currently you're saving the response as it is into your cache object, which is not valid format for jQuery UI autocomplete. You should convert the data into proper format digestable for autocomplete.
Either you should pass an array of strings, or an array of objects having label and value properties.
Since the response only contains name properties, you can convert it into an array of strings using jQuery map() method and save it in cache variable as follows:
$("#searchTextField").autocomplete({
minLength: 3,
source: function (request, response) {
var term = request.term;
if (term in cache) {
response(cache[term]);
return;
}
$.getJSON("[url]", {
name: request.term
}, function (data, status, xhr) {
cache[term] = $.map(data, function (obj) { // returns array of strings
return obj.name
});
// return the new array rather than original response
response(cache[term]);
});
}
});
I am having trouble getting JSON data to display in my grid. The html:
<div data-role="grid" data-bind="source: systems" data-columns='["SystemName", "SystemKey"]' ></div>
The mvvm view model:
var viewModel = kendo.observable({
isVisible: true,
systems: new kendo.data.DataSource({
schema: {
model: {
id: "system_id"
}
},
transport: {
read: {
url: "/api/HomeApi?method=Ref/Systems"
}
}
})
});
kendo.bind($("#systems"), viewModel);
The structure of the JSON Data:
- JSON
- Data
- Data
{}
SystemName=TIBCO
SystemKey=TIBCO
..... etc
Now, if the data were to come back in the following format, it would display fine:
[{ "SystemKey": "TIBCO", "SystemName": "TIBCO" }, { "SystemKey": "TIBCO", "SystemName": "TIBCO" }]
And yet, as stated, the actual data is in a more complex structure, rather than the simple structure above. So, do I have to parse this in some way, perhaps using schema.parse, to extract the SystemName and SystemKey, since it is down a few levels? Currently no data is displayed in the grid and I get "Uncaught TypeError: undefined is not a function " as an error.
The answer is yes, I do have to use schema.parse. This is what did it:
schema: {
parse: function (response) {
var systems = [];
for (var i = 0; i < response.Data.Data.length; i++) {
var system = {
SystemName: response.Data.Data[i].SystemName,
SystemKey: response.Data.Data[i].SystemKey
};
systems.push(system);
}
return systems;
}
}
I have a JsonStore configured like so:
var store = new Ext.data.JsonStore({
restful: true,
url: '/categories',
remoteSort: true,
idProperty: 'Id',
totalProperty: 'total',
root: 'results',
writer: new Ext.data.JsonWriter({
encode: false
}),
fields: [ 'Id', 'Name' ]
});
I grab some data from the server, then edit one of the records. When I tell the store to save, it sends this JSON back to the server:
{
"results":
{
"Name":"Trivial123",
"Id":2
}
}
The store is wrapping the JSON inside the results property (the root property configured on the store). However, the server expects this:
{
"Name":"Trivial123",
"Id":2
}
In other words, the serialized entity should be put directly in the response body, and not wrapped in a property. Does anyone know how I can configure the store to do this?
You need to override the data rendering function in the JsonWriter, like so:
var rootlessRenderFunction = function (params, baseParams, data) {
if (this.encode === true) {
Ext.apply(params, baseParams);
params = Ext.encode(data);
} else {
params.jsonData = data;
}
};
var myWriter = new Ext.data.JsonWriter({
encode: false,
writeAllFields: true
});
myWriter.render = rootlessRenderFunction;
var myStore = new Ext.data.JsonStore({
// ... various config values ...
writer: myWriter
});
This "rootlessRenderFunction" implementation is the same as the Ext JsonWriter's render code except it doesn't interpose a root in the request data.
This is a hack, of course.
I'm assuming you can't just not set the root value for the store for some reason? That's the way I normally do it.