I am currently designing a Rest API and is a little stuck on performance matters for 2 of the use cases in the system:
List all campaigns (api/campaigns) - needs to return campaign data needed for listing and paging campaigns. Maybe return up to 1000 records and would take ages to retreive and return detailed data. The needed data can be returned in a single DB call.
Retrieve campaign item (api/campaigns/id) - need to return all data about the campaign and may take up to a second to run. Multiple DB calls is needed to get all campaign data for a single campaign.
My question is: Is it valid to return different json-responses to those 2 calls (if well documented) even if it regards the same resource? I am thinking that the list response is a sub set of the retreive-response. The reason for this is to make to save DB calls and bandwitdh + parsing.
Thanks in advance!
I think it's both fine and expected for /campaigns and /campaigns/{id} to return different information. I would suggest using query parameters to limit the amount of information you need to return. For instance, only return a URI to each player unless you see a ?expand=players query parameter, in which case you return detailed player information.
Related
I have two tables, one called Company and the other called User, each user is related to one company using ForeignKey. So I can use reverse relation in Django to get all users for specific company (e.g. company.users)
In my case, I'm building ListAPIView which return multiple companies, and I'd like to return latest created user. My problem is that I don't want to use prefetch_related or select_related so it will load all the users, as we might end up having thousands of users per each company! Also I don't want to load each latest user in a separate query so we end up having tens of queries per API request!
I've tried something like this:
users_qs = models.User.objects.filter(active=True).order_by('-created')
company_qs = models.Company.objects.prefetch_related(
Prefetch('users', queryset=users_qs[:1], to_attr='user')
).order_by('-created')
In this case, prefetch_related failed as we can't set limit on the Prefetch's queryset filter (it gives this error "Cannot filter a query once a slice has been taken.")
Any ideas?
I think you are providing an object instead of a queryset Prefetch('users', queryset=users_qs[:1], to_attr='user')
In my class based view I am handling a post request (which is an AJAX call).I am inserting some data in a database table and returning some json to the frontend.
def post(self,request,*args,**kwargs):
result_data = {}
doc = kwargs['doc']
doc_obj = Document.objects.get_document(doc)
doc_id = doc_obj.doc_id
reference_obj = Reference.objects.save_to_db(request,doc_id)
friendly_name = reference_obj.friendly_name
result_data['friendly_name'] = friendly_name
return HttpResponse(json.dumps(result_data),content_type='application/json')
My question is that is this the right way of handling a post request?My doubt arises here - I am writing some data into database and returning a json of properties of the same data I have written to the database.
Should I do it differently, ie first storing some data in database and returning just id of the row and again making GET request on the same URL?
I would say if it is an API, it is best to minimize the number of queries by returning the whole object the way you did. In case it is to serve a web user interface, it is simpler to return the id and get the rest of the data, if it needs to be displayed, from the client itself.
Either way is fine. The gain is negligible. I have also seen applications give a local positive feedback and then replace with a negative one if the request was unsuccessful. The technique can improve responsiveness, which is always appreciated.
Django-rest-framework returns the json of the inserted object, so i would think it reasonable to copy that behaviour.
I am trying to obtain a data set via json and url, using SODA's API. The issue is that the data set is greater then 50K, and I need to sort the data set using multiple keys. Sorting by multiple keys is not something that is permitted by SODA's API. The question is how could I get around that?
Example (This table is small, but for illustrative purposes I have included it):
https://data.medicare.gov/resource/apyc-v239.json?$Limit=1000&$Order=measure_id
but once I attempt to add state to the order the API errors out.
The data set above is only 3800 recs, however there are other datasets with over 250000 recs, which require the same approach - sorting, then paging through the results...
Any assistance would be greatly appreciated.
Try the new API endpoint for that dataset: http://dev.socrata.com/foundry/#/data.medicare.gov/apyc-v239
It'll allow you to sort by multiple columns and there's no maximum for $limit on the new endpoints, so you can do stuff like this:
https://data.medicare.gov/resource/q7p2-jxeh.json?$order=state,measure_name&$limit=100000000
When I query some view in Couchbase I get the response that has following structure:
{
"total_rows":100,
"rows":[...]
}
'total_rows' is very useful property that I can use for paging.
But lets say I select only a subset of view using 'start_key' and 'end_key' and of course I don't know how big this subset will be. 'total_rows' is still the same number (as I understand it's just total of whole view). Is there any easy way to know how many rows was selected in subset?
You can use the in-built reduce function _count to get the total count of your query.
Just add _count as reduce function for your view. After that, you will need to make two calls to couchbase:
In one call, you'll set the query param reduce=true (along with either group=true or group_level=n, depending upon how you're sending your key(s)). This will give you the total count of your filtered rows.
In the other call, you'll disable the reduce function with reduce=false because you now need the actual rows.
You can find more details about map and reduce at http://docs.couchbase.com/admin/admin/Views/views-writing.html
You can just use an array count/total/length in whatever language you are using.
For example in PHP:
$result = $cb->view("dev_beer", "beer_by_name", array('startkey' => 'O', 'endkey'=>'P'));
echo "total = >>".count($result["rows"])
If you're actually wanting to paginate your data then you should use limit and skip:
http://www.couchbase.com/docs/couchbase-manual-2.0/couchbase-views-writing-querying-pagination.html
If you have to paginate the view in the efficient way, you actually don't need to specify both start and the end.
Generally it is possible to use startkey/startkey_id and limit. In this case the limit will tell you that the page won't be bigger than known size.
Both cases are described in CouchDB book: http://guide.couchdb.org/draft/recipes.html#pagination
Here is how it works:
Request rows_per_page + 1 rows from the view
Display rows_per_page rows, store + 1 row as next_startkey and next_startkey_docid
As page information, keep startkey and next_startkey
Use the next_* values to create the next link, and use the others to create the previous link
i'm working in a symfony project and using sfPropelPager to show a paged list of elements.
The problem is that with a great amount of data to list (i.e. thousands of registers) it makes a query to the database for each page to show!!!! That means about 100 extra queries in my case, and that is unacceptable.
Showing some of my code: the function that returns the pager object
$pager = new sfPropelPager('MyTable',sfConfig::get('sfPropelPagerLines'));
$c = new Criteria();
$c->add('my_table_field',$value);
$c->addDescendingOrderByColumn('date');
$pager->setCriteria($c);
$pager->init();
return $pager;
So, please, if you know a way to get all the results with only one query, it would be a great solution for my problem. Otherwise i must implement that list with an ajax call for every page the user wants to see
Thank you very much for your time.
I'm not sure to get your problem but, anyway, avoid the use of Criteria. Try to make queries with the ModelCriteria API: http://www.propelorm.org/reference/model-criteria.html.
For each paginated page, a query to the database will be done, this is the standard behavior for all pagers I know. If it's related to related objects (assuming you want to display information from relations), you may want to create a query that links those objects before to paginate, that way you'll get one query per page for all your data to display.
Read this doc for instance: http://www.propelorm.org/documentation/03-basic-crud.html#query_termination_methods
At last i did'nt get a solution for the problem, i had to implement the list via AJAX call, calling to a function that returns the requested page, so at the load of the page, no query for this list is slowing the user experience.
Thank you anyway to help me :)