How to Retrieve Large URL Json Data Set? - json

I am trying to obtain a data set via json and url, using SODA's API. The issue is that the data set is greater then 50K, and I need to sort the data set using multiple keys. Sorting by multiple keys is not something that is permitted by SODA's API. The question is how could I get around that?
Example (This table is small, but for illustrative purposes I have included it):
https://data.medicare.gov/resource/apyc-v239.json?$Limit=1000&$Order=measure_id
but once I attempt to add state to the order the API errors out.
The data set above is only 3800 recs, however there are other datasets with over 250000 recs, which require the same approach - sorting, then paging through the results...
Any assistance would be greatly appreciated.

Try the new API endpoint for that dataset: http://dev.socrata.com/foundry/#/data.medicare.gov/apyc-v239
It'll allow you to sort by multiple columns and there's no maximum for $limit on the new endpoints, so you can do stuff like this:
https://data.medicare.gov/resource/q7p2-jxeh.json?$order=state,measure_name&$limit=100000000

Related

Get value from json in LogicApp Two/Three Levels Down

Absolute newb question here how do I get a value two levels down in the JSON data with Azure Logic Apps? The web-hook is sending in: Image of incoming JSON
I need the value for shipping/address/city...
I have been trying to run #triggerBody()?['shipping']['address']['city'] but it can't find address (obviously I am doing something wrong).
I just need some syntax to continue and your help is greatly appreciated!
One of the workaround is to use Parse_JSON connector to get the inner values of JSON. Just use the sample Schema in Use sample payload to generate schema and it fetches the inner values.
Below is the screenshot of my logic app.
Result:

What is the URL when I DELETE an object

I'm running a local server playing around with an API using Django. I have a model called 'Users' populated with a few objects, and am using DefaultRouter.
I want to know what the URL would be if I were to DELETE a specific object from this model. For example, if I wanted to GET a user with an ID of 1 in this model, the URL would be: "localhost:8000/Users/1/". What would the equivalent be to DELETE this user?
I found an explanation of this on the REST API website (below), however, I don't understand what any of the syntaxes means.
What is {prefix}, {url_path}, {lookup} and [.format]? If anyone could provide an example of what this might be using a localhost that would be really helpful.
Thanks
Let us take an example of an API (URL) to update book data with id (pk) being 10. It would look something like this:
URL: http://www.example.com/api/v1/book/10/
Method: PUT/PATCH
With some data associated.
If you want to delete you just need to change method to DELETE instead of put or patch.
Regarding your second question lets compare the url with the parameters.
prefix: http://www.example.com/api/v1/book
lookup: 10
format: It specifies what type of data do you expect when you hit the API. Generally it is considered to be json.
url_path: In general, every thing after look up except query string is considered to be url_path.

Getting the value of a particular cell with AJAX

My goal is very simple and I would guess it is a very common goal among web developers.
I am creating a Rails (5.1) application, and I simply want to use AJAX to get the value of a specific cell in a specific row of a specific table in my database (later I am going to use that value to highlight some text on the current page in the user's browser).
I have not been able to find any documentation online explaining how to do this. As I said, it seems like a basic task to ask of jquery and ajax, so I'm confused as to why I'm having so much trouble figuring it out.
For concreteness, say I have a table called 'animals', and I want to get the value of the column 'species' for the animal with 'id' = 99.
How can I construct an AJAX call to query the database for the value of 'species' for the 'animal' with 'id' = 99 .
Though some DBs provide a REST API, what we commonly do is define a route in the app to pull and return data from the DB.
So:
Add a route
Add a controller/action for that route
In that action, fetch the data from the DB and render it in your preferred format
On the client-side, make the AJAX call to that controller/action and do something with the response.

Valid to return different json-response depending on list or retrieve?

I am currently designing a Rest API and is a little stuck on performance matters for 2 of the use cases in the system:
List all campaigns (api/campaigns) - needs to return campaign data needed for listing and paging campaigns. Maybe return up to 1000 records and would take ages to retreive and return detailed data. The needed data can be returned in a single DB call.
Retrieve campaign item (api/campaigns/id) - need to return all data about the campaign and may take up to a second to run. Multiple DB calls is needed to get all campaign data for a single campaign.
My question is: Is it valid to return different json-responses to those 2 calls (if well documented) even if it regards the same resource? I am thinking that the list response is a sub set of the retreive-response. The reason for this is to make to save DB calls and bandwitdh + parsing.
Thanks in advance!
I think it's both fine and expected for /campaigns and /campaigns/{id} to return different information. I would suggest using query parameters to limit the amount of information you need to return. For instance, only return a URI to each player unless you see a ?expand=players query parameter, in which case you return detailed player information.

sfPropelPager reduce queries

i'm working in a symfony project and using sfPropelPager to show a paged list of elements.
The problem is that with a great amount of data to list (i.e. thousands of registers) it makes a query to the database for each page to show!!!! That means about 100 extra queries in my case, and that is unacceptable.
Showing some of my code: the function that returns the pager object
$pager = new sfPropelPager('MyTable',sfConfig::get('sfPropelPagerLines'));
$c = new Criteria();
$c->add('my_table_field',$value);
$c->addDescendingOrderByColumn('date');
$pager->setCriteria($c);
$pager->init();
return $pager;
So, please, if you know a way to get all the results with only one query, it would be a great solution for my problem. Otherwise i must implement that list with an ajax call for every page the user wants to see
Thank you very much for your time.
I'm not sure to get your problem but, anyway, avoid the use of Criteria. Try to make queries with the ModelCriteria API: http://www.propelorm.org/reference/model-criteria.html.
For each paginated page, a query to the database will be done, this is the standard behavior for all pagers I know. If it's related to related objects (assuming you want to display information from relations), you may want to create a query that links those objects before to paginate, that way you'll get one query per page for all your data to display.
Read this doc for instance: http://www.propelorm.org/documentation/03-basic-crud.html#query_termination_methods
At last i did'nt get a solution for the problem, i had to implement the list via AJAX call, calling to a function that returns the requested page, so at the load of the page, no query for this list is slowing the user experience.
Thank you anyway to help me :)