I am wondering if there is any difference between Riak.mapValuesJson() and JSON.parse(). I did not know about Riak.mapValuesJson() so in my map reduce functions in my Riak database i kept using JSON.parse() which worked I am not sure if there are extra optimizations or something that are in the built in riak version of if Riak.mapValuesJson() is an alias of JSON.parse()
It uses JSON.parse()
See: https://github.com/basho/riak_kv/blob/master/priv/mapred_builtins.js
mapValues: function(value, keyData, arg) {
if (value["not_found"]) {
return [value];
}
var data = value["values"][0]["data"];
if (Riak.getClassName(data) !== "Array") {
return [data];
}
else {
return data;
}},
mapValuesJson: function(value, keyData, arg) {
if (value["not_found"]) {
return [value];
}
var newValues = Riak.mapValues(value, keyData, arg);
return newValues.map(function(nv) { return JSON.parse(nv); });
}
Edit:
The first step is in the if statment to make sure there is a value to the object because riak will return an object even if there is no value, allowing you to set a value.
The next step is calling the parent object and calling that method using a call back to parse the now decoded json data, there should not be any major speed differance between the two it adds to if stements and initalizing an object, but it does make the calling of the map reduce easier, I would use this on smaller buckets and the JSON.pare() on larger buckets
Related
I'm learning how to use json, not used to Dart null safety & I don't like it so far. But I will have no choice but to adapt to it. I'm trying to parse my json object list into a list of objects. I was able to work out the basics in my main, but when I attempt to create an actual method in a class using the same structure I'm getting a null error. As far as I can tell I'm doing the exact same thing in both with addition of the loop for iterating the entire json list.
Note: I of course did try inserting the optional ? where it asks but the IDE will not allow this.
Can someone help with explaining what I'm doing wrong here?
Error for class method jsonToDatabaseSyncItem()
lib/services/remote_database_services.dart:52:43: Error: Property 'length' cannot be accessed on 'List<dynamic>?' because it is potentially null.
- 'List' is from 'dart:core'.
Try accessing using ?. instead.
final jsonListLength = jsonObjectList.length;
^^^^^^
lib/services/remote_database_services.dart:55:38: Error: Operator '[]' cannot be called on 'List<dynamic>?' because it is potentially null.
- 'List' is from 'dart:core'.
var jsonObject = jsonObjectList[index]['DatabaseSyncItem'];
Class method (class- RemoteDatabaseService)
// This method will get server database
Future<List<dynamic>?> getRemoteDatabase() async {
final events = QueryBuilder<ParseObject>(ParseObject('Event'));
final apiResponse = await events.query();
if (apiResponse.success && apiResponse.result != null) {
return apiResponse.results;
} else {
return [];
}
}
// Method to parse json result list back to objects
Future<List<dynamic>?> jsonToDatabaseSyncItem() async {
final remoteDatabaseList = await getRemoteDatabase();
final jsonObjectList = await Future.value(remoteDatabaseList); /// This method throws the above
final jsonListLength = jsonObjectList.length; /// error when run in main
for (var index = 0; index == jsonListLength; index++) {
var jsonObject = jsonObjectList[index]['DatabaseSyncItem'];
print(jsonObject);
}
}
Main file working code
Future<void> main(List<String> arguments) async {
final test = remoteDatabaseServices.getRemoteDatabase();
Future<List<dynamic>?> getList() {
return Future.value(test);
}
var list = await getList();
print(list?.length);
var jsonObject = list![0]['DatabaseSyncItem'];
print(jsonObject);
var toObject = DatabaseSyncItem.fromJson(jsonObject);
print(toObject);
}
Your problem with null-safety seems to be a missing understanding about the feature. I will recommend you to read through the documentation here and read all the chapters: https://dart.dev/null-safety
About your code, you should consider when something can be null and when you can check the null and handle that case. In your example, it seems like getRemoteDatabase() should just return an empty List if an error happens or no result is returned. So we don't need to have the return type of this method as Future<List<dynamic>?> if we rewrite the method a bit:
Future<List<dynamic>> getRemoteDatabase() async {
final events = QueryBuilder<ParseObject>(ParseObject('Event'));
final apiResponse = await events.query();
if (apiResponse.success) {
return apiResponse.results ?? <dynamic>[];
} else {
return <dynamic>[];
}
}
(the ?? operator will here test if apiResponse.results is null, if that is the case, we return <dynamic>[]. If not, we use the value of apiResponse.results).
Since this method is now guarantee to never return null, we can use that to simplify the next method. I have also rewritten it to use a for-each loop since we don't really need the index of each element.
Future<void> jsonToDatabaseSyncItem() async {
final jsonObjectList = await getRemoteDatabase();
for (final jsonObject in jsonObjectList) {
print(jsonObject['DatabaseSyncItem']);
}
}
I have also removed this line since it does nothing at all. If remoteDatabaseList is a Future you should just await on that instead of creating a new Future.
final jsonObjectList = await Future.value(remoteDatabaseList);
Also, the return type of jsonToDatabaseSyncItem() have been changed to Future<void> since we are never returning any value.
I am a little confused about your main but I think this is where it is most clear that you got a confused about Future and null-safety. I have tried to rewrite it so it is much cleaner but should still do the same:
Future<void> main(List<String> arguments) async {
final list = await getRemoteDatabase();
print(list.length);
final dynamic jsonObject = list[0]['DatabaseSyncItem'];
print(jsonObject);
final toObject = DatabaseSyncItem.fromJson(jsonObject);
print(toObject);
}
I have a function that will return either a string, if there is an error, or two objects, when there is no error. My function looks like this:
function logResults(json) {
const one = json[0]
const two = json[1]
const error = json[0].error
if (error) {
return 'error at logResults' // string type
}
return (one, two) // object type
}
My question is would it possible to destructure this function's return types? This line works if two objects are successfully returned: let [ one, two ] = logResults(json), but it won't work if a string is returned. If destructuring is not possible, what is the most efficient way to handle the different return types?
return either a string, if there is an error, or two objects, when there is no error
Uh, don't do that. For exactly the reasons you have demonstrated: the function becomes unusable. Just throw an error or return an array with the two objects.
function logResults(json) {
const [one, two] = json;
if (one.error) {
throw new Error('error at logResults');
}
return [one, two]; // or just `json`?
}
Now you can use destructuring after the call as you imagined.
I have a unique situation here which I am having trouble solving in an elegant fashion.
A user passes up an array of signals which they want to export data for. This array can be 1 -> Any_Number so first I go fetch the table names (each signal stores data in a separate table) based on the signals passed and store those in an object.
The next step is to iterate over that object (which contains the table names I need to query), execute the query per table and store the results in an object which will be passed to next chain in the Promise. I haven't seen any examples online of good ways to handle this but I know it's a fairly unique scenario.
My code prior to attempting to add support for arrays of signals was simply the following:
exports.getRawDataForExport = function(data) {
return new Promise(function(resolve, reject) {
var getTableName = function() {
return knex('monitored_parameter')
.where('device_id', data.device_id)
.andWhere('internal_name', data.param)
.first()
.then(function(row) {
if(row) {
var resp = {"table" : 'monitored_parameter_data_' + row.id, "param" : row.display_name};
return resp;
}
});
}
var getData = function(runningResult) {
return knexHistory(runningResult.table)
.select('data_value as value', 'unit', 'created')
.then(function(rows) {
runningResult.data = rows;
return runningResult;
});
}
var createFile = function(runningResult) {
var fields = ['value', 'unit', 'created'],
csvFileName = filePathExport + runningResult.param + '_export.csv',
zipFileName = filePathExport + runningResult.param + '_export.gz';
var csv = json2csv({data : runningResult.data, fields : fields, doubleQuotes : ''});
fs.writeFileSync(csvFileName, csv);
// create streams for gZipping
var input = fs.createReadStream(csvFileName);
var output = fs.createWriteStream(zipFileName);
// gZip
input.pipe(gzip).pipe(output);
return zipFileName;
}
getTableName()
.then(getData)
.then(createFile)
.then(function(zipFile) {
resolve(zipFile);
});
});
}
Obviously that works fine for a single table and I have gotten the getTableName() and createFile() methods updated to handle arrays of data so this question only pertains to the getData() method.
Cheers!
This kind of problem is far from unique and, approached the right way, is very simply solved.
Don't rewrite any of the three internal functions.
Just purge the explicit promise construction antipattern from .getRawDataForExport() such that it returns a naturally occurring promise and propagates asynchronous errors to the caller.
return getTableName()
.then(getData)
.then(createFile);
Now, .getRawDataForExport() is the basic building-block for your multiple "gets".
Then, a design choice; parallel versus sequential operations. Both are very well documented.
Parallel:
exports.getMultiple = function(arrayOfSignals) {
return Promise.all(arrayOfSignals.map(getRawDataForExport));
};
Sequential:
exports.getMultiple = function(arrayOfSignals) {
return arrayOfSignals.reduce(function(promise, signal) {
return promise.then(function() {
return getRawDataForExport(signal);
});
}, Promise.resolve());
};
In the first instance, for best potential performance, try parallel.
If the server chokes, or is likely ever to choke, on parallel operations, choose sequential.
I have a practice problem and I need to replicate JSON.stringify, without actually using JSON.stringify. I was having trouble getting to the return value when it's an object that your inputting into the function. For example, if you insert
var obj = {a:1, b:2, c:3};
JSON.stringify (obj); // returns "{"a":1,"b":2,"c":3}"
That being said, I was trying to use a for in loop to set the property to the string value of the object that's being passed through. I was doing this only for an object container, but the function should actually work with anything you pass through it and it would JSON.stringify it.
var stringifyJSON = function(obj) {
var newObj = {};
for (var prop in obj){
newObj ={
stringProp:obj[prop]
};
}
return newObj;
};
I think I have the array portion down. I'm pretty terrible with object traversing. The result here is 'Object {stringProp:3}'. This is a practice problem within a recursion problem set, so I think they don't want me to use a loop. I was just trying to make it a little simpler by looping it, and then I would try to replicate it through recursion.
Any help would be appreciated!
Thanks,
B
Since it is a practice problem I'll just try to point you in the right direction:
You will need a loop in the code, as you will be required to recursively loop over each property in the object.
As for the recursive part, first you need to define your 'base case'. What is the condition based on the input that should not result in a recursive call? I'll give you a big hint, it's when the argument is not an object.
The tricky part for you is determining how you are going to be appending the resulting string from the recursive calls.
Some basic template code to start you off:
function myStringify( obj ) {
if ( typeof obj !== 'object' ) {
// base case
return ""+obj;
}
var str = "";
for ( var prop in obj ) {
if ( obj.hasOwnProperty( prop ) ) {
// recursive calls and string formatting magic
}
}
return str;
}
I returned mongoose docs as json in this way:
UserModel.find({}, function (err, users) {
return res.end(JSON.stringify(users));
}
However, user.__proto__ was also returned. How can I return without it? I tried this but not worked:
UserModel.find({}, function (err, users) {
return res.end(users.toJSON()); // has no method 'toJSON'
}
You may also try mongoosejs's lean() :
UserModel.find().lean().exec(function (err, users) {
return res.end(JSON.stringify(users));
});
Late answer but you can also try this when defining your schema.
/**
* toJSON implementation
*/
schema.options.toJSON = {
transform: function(doc, ret, options) {
ret.id = ret._id;
delete ret._id;
delete ret.__v;
return ret;
}
};
Note that ret is the JSON'ed object, and it's not an instance of the mongoose model. You'll operate on it right on object hashes, without getters/setters.
And then:
Model
.findById(modelId)
.exec(function (dbErr, modelDoc){
if(dbErr) return handleErr(dbErr);
return res.send(modelDoc.toJSON(), 200);
});
Edit: Feb 2015
Because I didn't provide a solution to the missing toJSON (or toObject) method(s) I will explain the difference between my usage example and OP's usage example.
OP:
UserModel
.find({}) // will get all users
.exec(function(err, users) {
// supposing that we don't have an error
// and we had users in our collection,
// the users variable here is an array
// of mongoose instances;
// wrong usage (from OP's example)
// return res.end(users.toJSON()); // has no method toJSON
// correct usage
// to apply the toJSON transformation on instances, you have to
// iterate through the users array
var transformedUsers = users.map(function(user) {
return user.toJSON();
});
// finish the request
res.end(transformedUsers);
});
My Example:
UserModel
.findById(someId) // will get a single user
.exec(function(err, user) {
// handle the error, if any
if(err) return handleError(err);
if(null !== user) {
// user might be null if no user matched
// the given id (someId)
// the toJSON method is available here,
// since the user variable here is a
// mongoose model instance
return res.end(user.toJSON());
}
});
First of all, try toObject() instead of toJSON() maybe?
Secondly, you'll need to call it on the actual documents and not the array, so maybe try something more annoying like this:
var flatUsers = users.map(function() {
return user.toObject();
})
return res.end(JSON.stringify(flatUsers));
It's a guess, but I hope it helps
model.find({Branch:branch},function (err, docs){
if (err) res.send(err)
res.send(JSON.parse(JSON.stringify(docs)))
});
I found out I made a mistake. There's no need to call toObject() or toJSON() at all. The __proto__ in the question came from jquery, not mongoose. Here's my test:
UserModel.find({}, function (err, users) {
console.log(users.save); // { [Function] numAsyncPres: 0 }
var json = JSON.stringify(users);
users = users.map(function (user) {
return user.toObject();
}
console.log(user.save); // undefined
console.log(json == JSON.stringify(users)); // true
}
doc.toObject() removes doc.prototype from a doc. But it makes no difference in JSON.stringify(doc). And it's not needed in this case.
Maybe a bit astray to the answer, but if anyone who is looking to do the other way around, you can use Model.hydrate() (since mongoose v4) to convert a javascript object (JSON) to a mongoose document.
An useful case would be when you using Model.aggregate(...). Because it is actually returning plain JS object, so you may want to convert it into a mongoose document in order to get access to Model.method (e.g. your virtual property defined in the schema).
PS. I thought it should have a thread running like "Convert json to Mongoose docs", but actually not, and since I've found out the answer, so I think it is not good to do self-post-and-self-answer.
You can use res.json() to jsonify any object.
lean() will remove all the empty fields in the mongoose query.
UserModel.find().lean().exec(function (err, users) {
return res.json(users);
}
It worked for me:
Products.find({}).then(a => console.log(a.map(p => p.toJSON())))
also if you want use getters, you should add its option also (on defining schema):
new mongoose.Schema({...}, {toJSON: {getters: true}})
Try this options:
UserModel.find({}, function (err, users) {
//i got into errors using so i changed to res.send()
return res.send( JSON.parse(JSON.stringify(users)) );
//Or
//return JSON.parse(JSON.stringify(users));
}
Was kinda laughing at how cumbersome this was for a second, given that this must be extremely common.
Did not bother digging in the docs and hacked this together instead.
const data = await this.model.logs.find({ "case_id": { $regex: /./, $options: 'i' }})
let res = data.map(e=>e._doc)
res.forEach(element => {
//del unwanted data
delete element._id
delete element.__v
});
return res
First i get all docs which have any value at all for the case_id field(just get all docs in collection)
Then get the actual data from the mongoose document via array.map
Remove unwanted props on object by mutating i directly