AngularJS + Parse REST API - Paging through more than 1,000 results - json

Im using Parse REST API + AngularJS and Im trying to be able to get more than 1000 items per query. I try to develop a recursive function and concatenate each query until I get all the data. My problem is that I am not able to concatenate the JSON objects successfully. Here is what I have:
$scope.getAllItems = function(queryLimit, querySkip, query) {
$http({method : 'GET',
url : 'https://api.parse.com/1/classes/myClass',
headers: { 'X-Parse-Application-Id':'XXX','X-Parse-REST-API-Key':'YYY'},
params: {limit:queryLimit, skip:querySkip},
}).success(function(data, status) {
query.concat(data.results);
if(query.lenth == queryLimit) {
querySkip += queryLimit;
queryLimit += 100;
$scope.getAllItems(queryLimit, querySkip, query);
} else {
$scope.clients = query;
}
})
.error(function(data, status) {
alert("Error");
});
};
var myQuery = angular.toJson([]); //Am I creating an empty JSON Obj?
$scope.getAllItems(100,0, myQuery);
Is there any better solution to achieve this?

There may be better, more concise ideas available, but this is what I worked out for myself.
In my service ...
fetch : function(page, perpage) {
var query = // build the query
// the whole answer to your question might be this line:
query.limit(perpage).skip(page*perpage);
return query.find();
},
fetchCount : function() {
var query = // build the same query as above
return query.count();
},
In the controller...
$scope.page = 0; // the page we're on
$scope.perpage = 30; // objects per page
MyService.fetchCount().then(function(count) {
var pagesCount = Math.ceil(count / $scope.perpage);
$scope.pages = [];
// pages is just an array of ints to give the view page number buttons
for (var i=0; i<pagesCount; i++) { $scope.pages.push(i); }
fetch();
});
function fetch() {
return MyService.fetch($scope.page, $scope.perpage)).then(function(results) {
$scope.results = results;
});
}
// functions to do page navigation
$scope.nextPage = function() {
$scope.page += 1;
fetch();
};
$scope.prevPage = function() {
$scope.page -= 1;
fetch();
};
$scope.selectedPage = function(p) {
$scope.page = p;
fetch();
};
Then paging buttons and results in my view (bootstrap.css)...
<ul class="pagination">
<li ng-click="prevPage()" ng-class="(page==0)? 'disabled' : ''"><a>«</a></li>
<li ng-repeat="p in pages" ng-click="selectedPage(p)" ng-class="(page==$index)? 'active' : ''"><a>{{p+1}}</a></li>
<li ng-click="nextPage()" ng-class="(page>=pages.length-1)? 'disabled' : ''"><a>»</a></li>
</ul>
<ul><li ng-repeat="result in results"> ... </li></ul>

I fixed my recursive function and now its working. Here it is:
$scope.getAllItems = function(queryLimit, querySkip, query, first) {
$http({method : 'GET',
url : 'https://api.parse.com/1/classes/myClass',
headers: { 'X-Parse-Application-Id':'XXX','X-Parse-REST-API-Key':'YYY'},
params: {limit:queryLimit, skip:querySkip},
}).success(function(data, status) {
if(first) {
query = data.results;
first = !first;
if(query.length == queryLimit) {
querySkip += queryLimit;
$scope.getAllItems(queryLimit, querySkip, query, first);
} else {
$scope.clients = query;
}
} else {
var newQ = data.results;
for (var i = 0 ; i < newQ.length ; i++) {
query.push(newQ[i]);
}
if(query.length == queryLimit + querySkip) {
querySkip += queryLimit;
$scope.getAllItems(queryLimit, querySkip, query, first);
} else {
$scope.clients = query;
}
}
})
.error(function(data, status) {
alert("Error");
});
};
Simply pushed each element to my empty array, also I was mutating queryLimit instead of querySkip in order to iterate through all the elements.

Related

Transform Request to Autoquery friendly

We are working with a 3rd party grid (telerik kendo) that has paging/sorting/filtering built in. It will send the requests in a certain way when making the GET call and I'm trying to determine if there is a way to translate these requests to AutoQuery friendly requests.
Query string params
Sort Pattern:
sort[{0}][field] and sort[{0}][dir]
Filtering:
filter[filters][{0}][field]
filter[filters][{0}][operator]
filter[filters][{0}][value]
So this which is populated in the querystring:
filter[filters][0][field]
filter[filters][0][operator]
filter[filters][0][value]
would need to be translated to.
FieldName=1 // filter[filters][0][field]+filter[filters][0][operator]+filter[filters][0][value] in a nutshell (not exactly true)
Should I manipulate the querystring object in a plugin by removing the filters (or just adding the ones I need) ? Is there a better option here?
I'm not sure there is a clean way to do this on the kendo side either.
I will explain the two routes I'm going down, I hope to see a better answer.
First, I tried to modify the querystring in a request filter, but could not. I ended up having to run the autoqueries manually by getting the params and modifying them before calling AutoQuery.Execute. Something like this:
var requestparams = Request.ToAutoQueryParams();
var q = AutoQueryDb.CreateQuery(requestobject, requestparams);
AutoQueryDb.Execute(requestobject, q);
I wish there was a more global way to do this. The extension method just loops over all the querystring params and adds the ones that I need.
After doing the above work, I wasn't very happy with the result so I investigated doing it differently and ended up with the following:
Register the Kendo grid filter operations to their equivalent Service Stack auto query ones:
var aq = new AutoQueryFeature { MaxLimit = 100, EnableAutoQueryViewer=true };
aq.ImplicitConventions.Add("%neq", aq.ImplicitConventions["%NotEqualTo"]);
aq.ImplicitConventions.Add("%eq", "{Field} = {Value}");
Next, on the grid's read operation, we need to reformat the the querystring:
read: {
url: "/api/stuff?format=json&isGrid=true",
data: function (options) {
if (options.sort && options.sort.length > 0) {
options.OrderBy = (options.sort[0].dir == "desc" ? "-" : "") + options.sort[0].field;
}
if (options.filter && options.filter.filters.length > 0) {
for (var i = 0; i < options.filter.filters.length; i++) {
var f = options.filter.filters[i];
console.log(f);
options[f.field + f.operator] = f.value;
}
}
}
Now, the grid will send the operations in a Autoquery friendly manner.
I created an AutoQueryDataSource ts class that you may or may not find useful.
It's usage is along the lines of:
this.gridDataSource = AutoQueryKendoDataSource.getDefaultInstance<dtos.QueryDbSubclass, dtos.ListDefinition>('/api/autoQueryRoute', { orderByDesc: 'createdOn' });
export default class AutoQueryKendoDataSource<queryT extends dtos.QueryDb_1<T>, T> extends kendo.data.DataSource {
private constructor(options: kendo.data.DataSourceOptions = {}, public route?: string, public request?: queryT) {
super(options)
}
defer: ng.IDeferred<any>;
static exportToExcel(columns: kendo.ui.GridColumn[], dataSource: kendo.data.DataSource, filename: string) {
let rows = [{ cells: columns.map(d => { return { value: d.field }; }) }];
dataSource.fetch(function () {
var data = this.data();
for (var i = 0; i < data.length; i++) {
//push single row for every record
rows.push({
cells: _.map(columns, d => { return { value: data[i][d.field] } })
})
}
var workbook = new kendo.ooxml.Workbook({
sheets: [
{
columns: _.map(columns, d => { return { autoWidth: true } }),
// Title of the sheet
title: filename,
// Rows of the sheet
rows: rows
}
]
});
//save the file as Excel file with extension xlsx
kendo.saveAs({ dataURI: workbook.toDataURL(), fileName: filename });
})
}
static getDefaultInstance<queryT extends dtos.QueryDb_1<T>, T>(route: string, request: queryT, $q?: ng.IQService, model?: any) {
let sortInfo: {
orderBy?: string,
orderByDesc?: string,
skip?: number
} = {
};
let opts = {
transport: {
read: {
url: route,
dataType: 'json',
data: request
},
parameterMap: (data, type) => {
if (type == 'read') {
if (data.sort) {
data.sort.forEach((s: any) => {
if (s.field.indexOf('.') > -1) {
var arr = _.split(s.field, '.')
s.field = arr[arr.length - 1];
}
})
}//for autoquery to work, need only field names not entity names.
sortInfo = {
orderByDesc: _.join(_.map(_.filter(data.sort, (s: any) => s.dir == 'desc'), 'field'), ','),
orderBy: _.join(_.map(_.filter(data.sort, (s: any) => s.dir == 'asc'), 'field'), ','),
skip: 0
}
if (data.page)
sortInfo.skip = (data.page - 1) * data.pageSize,
_.extend(data, request);
//override sorting if done via grid
if (sortInfo.orderByDesc) {
(<any>data).orderByDesc = sortInfo.orderByDesc;
(<any>data).orderBy = null;
}
if (sortInfo.orderBy) {
(<any>data).orderBy = sortInfo.orderBy;
(<any>data).orderByDesc = null;
}
(<any>data).skip = sortInfo.skip;
return data;
}
return data;
},
},
requestStart: (e: kendo.data.DataSourceRequestStartEvent) => {
let ds = <AutoQueryKendoDataSource<queryT, T>>e.sender;
if ($q)
ds.defer = $q.defer();
},
requestEnd: (e: kendo.data.DataSourceRequestEndEvent) => {
new DatesToStringsService().convert(e.response);
let ds = <AutoQueryKendoDataSource<queryT, T>>e.sender;
if (ds.defer)
ds.defer.resolve();
},
schema: {
data: (response: dtos.QueryResponse<T>) => {
return response.results;
},
type: 'json',
total: 'total',
model: model
},
pageSize: request.take || 40,
page: 1,
serverPaging: true,
serverSorting: true
}
let ds = new AutoQueryKendoDataSource<queryT, T>(opts, route, request);
return ds;
}
}

Setting data in viewModel knockoutjs from html5 websocket

I am trying to create knockout.js component that is getting data from HTML5 Websocket. Websocket code is in separate script e.g. util.js. I am able to connect and get data from socket, but dont know how correctly to set corresponding property in component`s ViewModel.
Websocket - util.js:
var options = {
server: '127.0.0.1',
port: '12345'
};
var socket, loadedFlag;
var timeout = 2000;
var clearTimer = -1;
var data = {};
function handleErrors(sError, sURL, iLine)
{
return true;
};
function getSocketState()
{
return (socket != null) ? socket.readyState : 0;
}
function onMessage(e)
{
data=$.parseJSON(e.data);
// ???? Is it possible to have here something like
// ???? viewModel.getDataWS1(data);
}
function onError()
{
clearInterval(clearTimer);
socket.onclose = function () {
loadedFlag = false;
};
clearTimer = setInterval("connectWebSocket()", timeout);
}
function onClose()
{
loadedFlag = false;
clearInterval(clearTimer);
clearTimer = setInterval("connectWebSocket()", timeout);
}
function onOpen()
{
clearInterval(clearTimer);
console.log("open" + getSocketState());
}
function connectWebSocket()
{
if ("WebSocket" in window)
{
if (getSocketState() === 1)
{
socket.onopen = onOpen;
clearInterval(clearTimer);
console.log(getSocketState());
}
else
{
try
{
host = "ws://" + options.server + ":" + options.port;
socket = new WebSocket(host);
socket.onopen = onOpen;
socket.onmessage = function (e) {
onMessage(e);
};
socket.onerror = onError;
socket.onclose = onClose;
}
catch (exeption)
{
console.log(exeption);
}
}
}
}
Component (productDisplay.js) - creating so that is can be used on multiple pages:
define([
'jquery',
'app/models/productDisplayModel',
'knockout',
'mapping',
'socket'
],
function ($, model, ko, mapping) {
ko.components.register('product', {
viewModel: {require: 'app/models/productModel'},
template: {require: 'text!app/views/product.html'}
});
});
Product ViewModel (productModel.js) - where I struggle to set viewModel property to data from websocket:
var viewModel = {};
define(['knockout', 'mapping', 'jquery'], function (ko, mapping, $) {
function Product(name, rating) {
this.name = name;
this.userRating = ko.observable(rating || null);
}
function MyViewModel() {
this.products = ko.observableArray(); // Start empty
}
MyViewModel.prototype.getDataWS1 = function () {
//Websocket has not connected and returned data yet, so data object is empty
// ???? Is there anyway I can add something like promise so that the value is set once socket is connected?
this.products(data);
};
// apply binding on page load
$(document).ready(function () {
connectToServer1();
viewModel = new MyViewModel();
ko.applyBindings(viewModel);
viewModel.getDataWS1();
});
});
Thank you for any ideas.
You can update an observable when you get a message in the following manner:
util.js
function onMessage(e) {
var productData = $.parseJSON(e.data);
viewModel.addNewProduct(productData);
}
productModel.js
function Product(name, rating) {
this.name = name;
this.userRating = ko.observable(rating || null);
}
function MyViewModel() {
this.products = ko.observableArray(); // Start empty
}
MyViewModel.prototype.addNewProduct(product) {
var newProduct = new Product(product.name, product.rating);
this.products.push(newProduct);
}
Basically the idea is that when you get a message (in onMessage function), you will parse the data and call a function in your viewmodel to add the message data to the viewmodel properties (observables, observableArrays, etc.)

ionic: I have a scope variable empty from a stringified json

I'm trying to filter a JSON result from a SQLite query. The filter works when I use JSON directly, but it doesn't when I use the query from the service. Then, the $scope.arrayme just appears as empty.
Where is the error? Thank you!
This is the service:
getSubtipos: function() {
var query = "SELECT subtipos.idsubtipo, subtipos.tipos_idtipo, subtipos.nombre, subtipos.icon, subtipos.val FROM subtipos";
var arraySubtipos = [];
$cordovaSQLite.execute(db, query, []).then(function(res) {
if(res.rows.length > 0) {
for(var i = 0; i < res.rows.length; i++) {
arraySubtipos.push(res.rows.item(i));
}
} else {
console.log("No results found");
}
}, function (err) {
console.error("ERROR: " + err.message);
}).finally(function() {
arraySubtipos = JSON.stringify(arraySubtipos);
});
return arraySubtipos;
}
This is the controller:
.controller('MenuSubtiposCtrl', function($scope, $filter, miJson, $stateParams, $cordovaSQLite){
var arrayme = JSON.stringify(miJson.getSubtipos());
$scope.arrayme = $filter("filter")(JSON.parse(arrayme), {tipos_idtipo: $stateParams.foo});
})
And this is the state:
.state('app.menusubtipos', {
url: "/menusubtipos/:foo",
views: {
'menuContent': {
templateUrl: "templates/menuSubtipos.html",
controller: "MenuSubtiposCtrl"
}
}
})
There may be more problems than what I've immediately noticed, but I have have noticed that you're returning a variable within your getSubtipos function before it's set.
The $cordovaSQL.execute() function is an asyncronous function. As a result, you are returning arraySubtipos before it's set.
A better way to do this would be within getSubtipos to do the following:
var arraySubtipos = [];
return $q.when($cordovaSQLite.execute(db, query, [])
.then(function(res) {
if(res.rows.length > 0) {
for(var i = 0; i < res.rows.length; i++) {
arraySubtipos.push(res.rows.item(i));
}
} else {
console.log("No results found");
}
return JSON.stringify(arraySubtipos);
}));
// Then, within your controller, do the following:
.controller('MenuSubtiposCtrl', function($scope, $filter, miJson, $stateParams, $cordovaSQLite){
miJson.getSubtipos()
.then(function(arrayMe) {
// No need to stringify it again
$scope.arrayme = $filter("filter")(JSON.parse(arrayme), {tipos_idtipo: $stateParams.foo});
})
.catch(function(error) {
// Handle the error here
});
var arrayme = JSON.stringify(miJson.getSubtipos());
});
I'm also a little suspicious about your use of JSON.stringify and JSON.parse. It's likely that they're not needed, but without knowing the format of your data, I've left that as is.

Angularjs ajax request in Symfony2 and Doctrine json response with relationships

I am trying to work with Symfony2, Doctrine and Angujarjs. There is no problem with Symfony2 or Doctrine but I have issues using an ajax request with angularjs. Either it doesn't load anything and I did make a mistake while loading the json (json comes from Symfony and its working) or if it's working, but the json doesn't contain any of the relationship's data.
So, what's the correct way to
A: create a response for angularjs with relationship data (such as articles and categories)
B: load the requested json into an angularjs variable
Here is my controller.js
var app = angular.module("MyApp", []) .config(['$interpolateProvider', function ($interpolateProvider) {
$interpolateProvider.startSymbol('[[');
$interpolateProvider.endSymbol(']]');
}]);
app.filter('offset', function() {
return function(input, start) {
start = parseInt(start, 10);
return input.slice(start);
};
});
app.filter('htmlToPlaintext', function() {
return function(text) {
return String(text).replace(/<[^>]+>/gm, '');
};
});
app.controller("PaginationCtrl", function($scope, $http) {
$scope.articlesPerPage = 8;
$scope.currentPage = 0;
function htmlToPlaintext(text) {
return String(text).replace(/<[^>]+>/gm, '');
}
// this should load the json from '/admin/jsonallarticles' into the articles variable
$http.get('/admin/jsonallarticles').success(function(data) {
$scope.articles = data;
});
$scope.range = function() {
var rangeSize = 5;
var ret = [];
var start;
start = $scope.currentPage;
if ( start > $scope.pageCount()-rangeSize ) {
start = $scope.pageCount()-rangeSize+1;
}
for (var i=start; i<start+rangeSize; i++) {
ret.push(i);
}
return ret;
};
$scope.prevPage = function() {
if ($scope.currentPage > 0) {
$scope.currentPage--;
}
};
$scope.prevPageDisabled = function() {
return $scope.currentPage === 0 ? "disabled" : "";
};
$scope.pageCount = function() {
return Math.ceil($scope.articles.length/$scope.articlesPerPage)-1;
};
$scope.nextPage = function() {
if ($scope.currentPage < $scope.pageCount()) {
$scope.currentPage++;
}
};
$scope.nextPageDisabled = function() {
return $scope.currentPage === $scope.pageCount() ? "disabled" : "";
};
$scope.setPage = function(n) {
$scope.currentPage = n;
};
});
This is my symfony2 function
public function jsonallarticlesAction() {
$articles = $this->getDoctrine()
->getRepository('AcmeBlogBundle:Articles');
if ( !$articles ) {
throw $this->createNotFoundException(
'Keine Beiträge gefunden!');
}
$queryArticles = $articles->createQueryBuilder('a')
->where('a.status = :status')
->setParameter('status', 0)
->orderBy('a.createdDate', 'DESC')
->getQuery()
->getResult(\Doctrine\ORM\Query::HYDRATE_ARRAY);;
$articles = array_values($queryArticles);
$response = new Response();
$response->setContent(json_encode($articles));
$response->headers->set('Content-Type', 'application/json');
return $response;
}
EDITED CONTROLLER
I tried using the serializer which comes with Symfony
$encoders = array(new XmlEncoder(), new JsonEncoder());
$normalizers = array(new GetSetMethodNormalizer());
$serializer = new Serializer($normalizers, $encoders);
$articles = $this->getDoctrine()
->getRepository('AcmeBlogBundle:Articles')
->findAll();
if ( !$articles ) {
throw $this->createNotFoundException(
'Keine Artikel gefunden!');
}
$serializer->serialize($articles, 'json');
return new Response(json_encode($json));
But I receive an error:
A circular reference has been detected (configured limit: 1).
For work with Angular.js you must write Rest API. For this you can use https://github.com/FriendsOfSymfony/FOSRestBundle
And for serialize your entities with needed data use http://jmsyst.com/bundles/JMSSerializerBundle
It compatible with FOSRestBundle.
As example of use those bundles you can look one our project https://github.com/stfalcon-studio/lost-and-found
I ran into the same issue and it was due to the fact that my Entity was related back to the same entity from my second entity on a different field. I just simply created this function in my Entity:
public function removeRelationsThatCauseCircularError()
{
$this->companyEvents = NULL;
}
And run the function before going through the serializer.

kendo treeview with new data source

Ok, so I have this situation here:
A CSHTML view with a kendo tree in it:
#(Html.Kendo().TreeView()
.Name("treeview")
.DataTextField("Name")
.DataSource(d => d.Read(r => r.Action("WorkedHours", "TaskManager")))
.Events(e => e.Select("onSelect"))
)
to the right of that there is a kendo grid. and above the tree there is a (kendo) dropdown list to select a user.
this is the controller method called by the tree:
public JsonResult WorkedHours(uint? id)
{
DocObjectArray docObjects = null;
if (id == null)
{
// get root elements
var loggedInUserRef = OmanagerUtils.GetInstance().LoggedInUser;
if (loggedInUserRef != null && loggedInUserRef.GetObject() != null && loggedInUserRef.GetObject().SubObjects != null)
{
for (int i = 0; i < loggedInUserRef.GetObject().SubObjects.GetLength(); i++)
{
var item = loggedInUserRef.GetObject().SubObjects.GetAt(i);
if (item.ToString() == TaskManagerConstants.UserWorkHours)
{
docObjects = item.TreeSubObjects;
break;
}
}
}
}
else
{
// get sub objects of a root object
var rootObj = new DocObjectRef((int)id);
docObjects = rootObj.GetObject().TreeSubObjects;
}
var returnDocObjects = new List<OmanagerItem>();
for (int i = 0; i < docObjects.GetLength(); i++)
{
var item = docObjects.GetAt(i);
var hasChildren = true;
if (item.TreeSubObjects == null)
{
hasChildren = false;
}
else
{
if (item.TreeSubObjects.GetLength() == 0)
{
hasChildren = false;
}
}
var listItem = new OmanagerItem
{
hasChildren = hasChildren,
id = item.GetOID(),
Name = item.ToString()
};
returnDocObjects.Add(listItem);
}
return Json(returnDocObjects, JsonRequestBehavior.AllowGet);
}
now, the problem is that i have to be able to select a user from the dropdown list and refresh the tree with this new data.
$("#employee").kendoDropDownList({
change: function () {
var postdata = {
id:$("#employee").val()
}
$.ajax({
url: "TaskManager/WorkedHours",
cache: false,
type: "POST",
data: postdata,
success: function (data) {
$("#treeview").data("kendoTreeView").setDataSource(data);
},
});
}
});
the problem is what do i do with this data? because my attempt did not really work.
many thanks.
You can use OutputCache attribute on WorkedHours action:
[OutputCache(NoStore = true, Duration = 0, VaryByParam = "None")]
public JsonResult WorkedHours(uint? id)
{
// rest of method
}
It helped in my case :)
Maybe this little snippet is of any help to you.
Similar to your code in the change event of my dropdown I'm calling a function that will change the request data of my TreeView DataSource.
After changing it, it calls the read() handler of the datasource so it re-reads the data:
function loadTreeViewData() {
var employee = $('#employee').getKendoDropDownList().dataItem();
WorkedHoursDataSource.transport.options.read.data = {Employee_Id:employee.id};
WorkedHoursDataSource.read();
}