DocumentDB: bulkImport Stored Proc - Getting 400 Error on Array/JSON issue - json

I'm simply trying to execute the standard example bulkImport sproc for documentDB API and I can't seem to pass it an array of objects. I always get 400 errors despite the documentation giving clear direction to send an array of objects
.. very frustrating.
Additional details: Even if I wrap the array in an object with the array under a property of 'items' and include it in my sproc it still errors out saying the same bad request, needs to be an object or JSON-serialized. When I try to do JSON.stringify(docs) before sending it fails to parse on the other side.
Bad Request: The document body must be an object or a string representing a JSON-serialized object.
bulkInsert.js:
https://github.com/Azure/azure-documentdb-js-server/blob/master/samples/stored-procedures/BulkImport.js
My Code (using documentdb-util for async):
execProc(docs, insertProc);
async function execProc(docs, insertProc){
let database = await dbUtil.database('test');
let collection = await dbUtil.collection(database, 'test');
let procInstance = await dbUtil.storedProcedure(collection, insertProc);
try{
let result = await dbUtil.executeStoredProcedure(procInstance, docs);
console.log(result);
} catch(e){
console.log(e.body)
}
}
Header
Object {Cache-Control: "no-cache", x-ms-version: "2017-11-15",
User-Agent: "win32/10.0.16299 Nodejs/v8.9.0 documentdb-nodejs-s…",
x-ms-date: "Mon, 11 Dec 2017 07:32:29 GMT",
Accept:"application/json"
authorization: myauth
Cache-Control:"no-cache"
Content-Type:"application/json"
User-Agent:"win32/10.0.16299 Nodejs/v8.9.0 documentdb-nodejs-sdk/1.14.1"
x-ms-date:"Mon, 11 Dec 2017 07:32:29 GMT"
x-ms-version:"2017-11-15"
Path
"/dbs/myDB/colls/myColl/sprocs/myBulkInsert"
Params
Array(3) [Object, Object, Object]
length:3
0:Object {id: "0001", type: "donut", name: "Cake", …}
1:Object {id: "0002", type: "donut", name: "Raised", …}
2:Object {id: "0003", type: "donut", name: "Old Fashioned", …}
[{
"id": "0001",
"type": "donut",
"name": "Cake",
"ppu": 0.55
},
{
"id": "0002",
"type": "donut",
"name": "Raised",
"ppu": 0.35
},
{
"id": "0003",
"type": "donut",
"name": "Old Fashioned",
"ppu": 0.25
}]

The "docs" must be an array of array of params, otherwise, the procedure executor will treat them as multiple params of the procedure, not a single-array-param.

the following code works when call storedProcedure to pass argument with array type.
JS:
var docs = [{'id':1},{'id':2}];
executeStoredProcedure(proc, [docs])
C#
var docs = new[] {new MyDoc{id=1, source="abc"}, new MyDoc{id=2, source="abc"}];
dynamic[] args = new dynamic[] {docs};
ExecuteStoredProcedureAsync<int>(
procLink,
new RequestOptions {PartitionKey = new PartitionKey("abc")},
args);
NOTE: you must ensure the 'docs' have the same partition key, and pass partion key in RequestionOptions

I had the same problem. I was able to get it to work by Stringify the Array and parse it in the stored procedure. I opened an issue on the github where that code originated as well. Below is what worked for me. Good luck.
---- Stringify Array
var testArr = []
for (var i = 0; i < 50; i++) {
testArr.push({
"id": "test" + i
})
}
var testArrStr = JSON.stringify(testArr)
//pass testArrStr to stored procedure and parse in stored procedure
---- Slightly altered original BulkImport
exports.storedProcedure = {
id: "bulkImportArray",
serverScript:function bulkImportArray(docs) {
var context = getContext();
var collection = context.getCollection();
var docsToCreate = JSON.parse(docs)
var count = 0;
var docsLength = docsToCreate.length;
if (docsLength == 0) {
getContext().getResponse().setBody(0);
}
var totals = ""
function insertDoc(){
var msg = " count=" + count+" docsLength=" +docsLength + " typeof docsToCreate[]=" + typeof docsToCreate+ " length =" + docsToCreate.length
if(typeof docsToCreate[count] != 'undefined' ) {
collection.createDocument(collection.getSelfLink(),
docsToCreate[count],
function (err, documentCreated) {
if (err){
// throw new Error('Error' + err.message);
getContext().getResponse().setBody(count + " : " + err);
}else{
if (count < docsLength -1) {
count++;
insertDoc();
getContext().getResponse().setBody(msg);
} else {
getContext().getResponse().setBody(msg);
}
}
});
}else{
getContext().getResponse().setBody(msg);
}
}
insertDoc()
}
}
If you want to test it in the portal Script Explorer I had to create an escaped string i.e.
var testArr = []
for(var i=200; i<250; i++){
testArr.push({"id":"test"+i})
}
var testArrStr = JSON.stringify(testArr)
console.log('"'+testArrStr.replace(/\"/g,'\\"') + '"')

Related

How to check if the response includes specified value and if so - end test in Postman

I'm learning Postman test scripts and I am stuck with one exercise. I need to check if the response includes specified value (one of the planet from array is called Tatooine). Body response:
"results": [
{
"name": "Tatooine",
"rotation_period": "23",
"orbital_period": "304",
"diameter": "10465",
{
"name": "Alderaan",
"rotation_period": "24",
"orbital_period": "364",
"diameter": "12500",
},
I created this script:
const jsonData = pm.response.json();
pm.test("Your test name", function () {
for (let i = 0; i <= jsonData.results.length; i++) {
pm.expect(jsonData.results[i].name).to.include("Tatooine")};
});
But I don't know how to get out of the loop and mark test as "passed" after finding searched value.
I assume you want to verify that at least there is a name Tatooine.
Step 1: Get all names
const jsonData = pm.response.json();
let names = _.map(jsonData.results, "name");
Step 2: Validate the array names contains Tatooine
pm.test("Your test name", function () {
pm.expect(names).to.include("Tatooine")
});

JSON.net parsing of dynamic JSON

I have JSON that looks like this:
{
"status": {
"code": 0,
"message": "OK"
},
"data": {
"_idtype": "cusip",
"_id": "00768Y883",
"api": {
"_name": "PortfolioBreakdownsRaw",
"PortfolioDate": "2015-10-12",
"GlobalBondSuperSectorLongSalePositionBreakdown": [
{
"Name": "Municipal",
"Value": "0.57842"
},
{
"Name": "Corporate",
"Value": "1.79649"
},
{
"Name": "Securitized",
"Value": "5.29493"
},
{
"Name": "Cash & Equivalents",
"Value": "166.20776"
}
],
"GlobalBondSuperSectorShortSalePositionBreakdown": [
{
"Name": "Government",
"Value": "0.90557"
}
]
}
}
}
I am able to get the api portion of the response easily:
var jObject = JObject.Parse(json);
var api = jObject["data"]["api"];
From here, I don't what if any arrays will be included in the response. The ultimate goal will be to create a parser that will be able to get the array names (GlobalBondSuperSectorShortSalePositionBreakdown) and as many rows of key-value pairs that it may contain, without first knowing the names such as (GlobalBondSuperSectorShortSalePositionBreakdown) beforehand.
I can't seem to find a good way to loop through the object, determine there are arrays at the api level and then iterate through those to get the values.
Any help would be appreciated.
Here's an example. In this code, the api variable holds a JObject, so we can iterate over its properties. From there, we look at the Type of each property value to see if it is an array or not. If it is, then we can iterate over that array to get the JObjects within it, and extract the Name and Value values that we expect to find there. Does this help?
var jObject = JObject.Parse(json);
var api = jObject["data"]["api"];
foreach (JProperty prop in api.Children<JProperty>())
{
JToken value = prop.Value;
if (value.Type == JTokenType.Array)
{
Console.WriteLine(prop.Name + ": ");
foreach (JObject jo in value.Children<JObject>())
{
Console.WriteLine(" " + jo["Name"] + ": " + jo["Value"]);
}
}
else
{
Console.WriteLine(prop.Name + ": " + value);
}
}
Output:
_name: PortfolioBreakdownsRaw
PortfolioDate: 2015-10-12
GlobalBondSuperSectorLongSalePositionBreakdown:
Municipal: 0.57842
Corporate: 1.79649
Securitized: 5.29493
Cash & Equivalents: 166.20776
GlobalBondSuperSectorShortSalePositionBreakdown:
Government: 0.90557
Fiddle: https://dotnetfiddle.net/XyoXQy
With Linq you can play pretty nice with Json.net:
Here is an easily readable version of the chunk of code that will create two dictionaries out of the JArray properties under the api element:
var api = jObject["data"]["api"];
var arrays = api.Cast<JProperty>().Where(o => o.Value.Type == JTokenType.Array).Select(token => token.Value).ToArray();
var dictionaries = new List<Dictionary<string, string>>();
foreach (var array in arrays)
{
var dictionary = array.ToDictionary(token => token["Name"].Value<string>(), token => token["Value"].Value<string>());
dictionaries.Add(dictionary);
}
alternative:
The same thing, but a shorter, more compact version :
var api = jObject["data"]["api"];
var dictionaries = api
.Cast<JProperty>()
.Where(o => o.Value.Type == JTokenType.Array)
.Select(token => token.Value)
.Select(array => array.ToDictionary(token => token["Name"].Value<string>(), token => token["Value"].Value<string>()));

json object properties

I am trying to iterate over this JSON:
{
"info":{
"version": "0.0.1",
"status": "prototype"
},
"config":{
"display_random": true,
"welcome_message": "Welcome to MagSee!",
"welcome_display_sec": 10
},
"config_form":{
"display_random":{
"label":"Display Random Image on Start",
"type": "Boolean",
"default": true
},
"welcome_message":{
"label": "Welcome Message",
"type": "TextInput",
"default": "Welcome to MagSee"
}
}
}
I read this from a file, then parse it and pass it into jade template:
router.get('/view_config', function(req, res){
var fs = require('fs');
var file = './config.json';
var json_data = null;
var buffer = fs.readFileSync(file, 'utf8');
json_data = JSON.parse(buffer);
if (json_data == null){
console.log('Null json_data. Does the file exist?');
//todo: need to return 500/null config data message instead
return;
}
res.render('admin_view_config',{'config': json_data.config, 'config_form': json_data.config_form});
});
then within Jade template I am trying to display the properties nicely:
h1='Config Form'
p
ul
each object in config_form
li=object
- console.dir(object)
ul
each value, key in object
li=key+": "+value
And the outcome is almost there but I am missing the actual names of the object and can't figure how to get it:
Config Form
[object Object]
label: Display Random Image on Start
type: Boolean
default: true
[object Object]
label: Welcome Message
type: TextInput
default: Welcome to MagSee
the console.dir(object) will only show it's portion within the {} and no name (such as "welcome_message") but I can't figure how to access it from within the config_form itself.
There is NO way of knowing which object it came from.
Although, you can modify your loops to achieve this. Like this,
- for(i=0; i<Object.keys(config_form).length; i++)
- var key = Object.keys(config_form)[i]
li=key
- console.dir(config_form[key])
ul
each val, index in config_form[key]
li= index + ': ' + val

Parsing doubly nested JSON object to MongoDB

Schema for my MongoDB model:
var resultsSchema = new mongoose.Schema({
start_date: String,
end_date: String,
matches:[{
id:Number,
match_date:String,
status:String,
timer:Number,
time:String,
hometeam_id:Number,
hometeam_name:String,
hometeam_score:Number,
awayteam_id:Number,
awayteam_name:String,
awayteam_score:Number,
ht_score:String,
ft_score:String,
et_score:String,
match_events:[{
id:Number,
type:String,
minute:Number,
team:String,
player_name:String,
player_id:Number,
result:String
}]
}]
});
Example of JSON data coming from the server:
"matches":
[
{
"match_id":"1234"
"match_date":"Aug 30"
...
...
"match_events":
[
{
"event_id":"234",
"event_minute":"38",
...,
...
},
{
"event_id":"2334",
"event_minute":"40",
...,
...
}
],
{
"match_id":"454222"
"match_date":"Aug 3"
...
...
"match_events":
[
{
"event_id":"234",
"event_minute":"38",
...,
...
},
....
My current implementation works for parsing just the matches (i.e the first array). But I can't seem to access the inner array properly.
async.waterfall([
function(callback) {
request.get('http://football-api.com/api/?Action=fixtures&APIKey=' + apiKey + '&comp_id=' + compId +
'&&from_date=' + lastWeek_string + '&&to_date=' + today_string, function(error, response, body) {
if (error) return next(error);
var parsedJSON = JSON.parse(body);
var matches = parsedJSON.matches;
var events = parsedJSON.matches.match_events;
var results = new Results({
start_date: lastWeek_string,
end_date: today_string,
matches:[]
});
_.each(matches, function(match) {
results.matches.push({
id: match.match_id,
match_date: match.match_formatted_date,
status:match.match_status,
timer:match.match_timer,
hometeam_id:match.match_localteam_id,
hometeam_name:match.match_localteam_name,
hometeam_score:match.match_localteam_score,
awayteam_id:match.match_visitorteam_id,
awayteam_name:match.match_visitorteam_name,
awayteam_score:match.match_visitorteam_score,
ht_score:match.match_ht_score,
ft_score:match.match_ft_score,
et_score:match.match_et_score,
match_events:[]
});
});
_.each(events, function(event) {
results.matches.match_events.push({
id:event.event_id,
type:event.event_type,
minute:event.event_minute,
team:event.event_team,
player_name:event.event_player,
player_id:event.event_player_id,
result:event.event_result
});
});
I understand that the second _.each loop should be iterating for every match, since very match has it's own events subarray. I'm just not sure how to structure this and have been struggling with it for a while.
I tried nesting that loop inside the _.each(matches, function(match) { loop but that didn't work.
Thank you.
Edit: How could I get this to work?
var results = new Results({
start_date: lastWeek_string,
end_date: today_string,
matches:[
match_events: []
]
});
Because then as #zangw says I could construct the match_events array first, append it to matches, and so on.

IBM Worklight Adapter parsing JSON

{
"isSuccessful": true,
"resultSet": [
{
"name": "pradeep",
"password": 123,
"timestamp": "2014-04-08T12:58:45.000Z"
},
{
"name": "dileep",
"password": 1234,
"timestamp": "2014-04-08T13:00:52.000Z"
}
]
}
This invocation result i have got by using Sql adapter so how to parse this invocation result and how can i display name,password,timestamp from this JSON object.Do i need to use HTTP Adapter.
If you want to get the length of results you should use result.invocationResult.resultSet.length which will give you the total number of results, where items is the response coming from the adapter and invocationResult contains the results and other paramaters, from which you will have to access the results for accessing only the particular output.To get value call
result.invocationResult.resultSet.name[position]
Like that call all the fields password,timestamp with position
in for loop
function handleSuccess(result) {
var invocationResult = result.invocationResult;
var isSuccessful = invocationResult.isSuccessful;
if (true == isSuccessful) {
var result = invocationResult.resultSet;
for ( var i = 0; i < result.length; i++) {
alert(result.name[i]);
alert(result.password[i]);
alert(result.timestamp[i]);
}
} else {
alert("error");
}