I use ajax to get a JSON object from an external API.
Here is the JSON object that I get :
"pricing_options": {
"7": {
"lodging_id": 7,
"lodging_url": "https://exampleapiurl.com/7",
"price": 520.0,
"name": "Private Room w/EnSuite Bath",
"max_occupancy": 2,
"image_url": "https://exampleapiurl.com/assets/282/2017/09/Karuna.jpg"
},
"552": {
"lodging_id": 552,
"lodging_url": "https://exampleapiurl.com/552",
"price": 400.0,
"name": "Private Room w/Queen Bed & Hall Bath",
"max_occupancy": 2,
"image_url": "https://exampleapiurl.com/assets/282/2019/03/Khandro-300x240.jpg"
},
"80": {
"lodging_id": 80,
"lodging_url": "https://exampleapiurl.com/80",
"price": 400.0,
"name": "Private Room w/Twin Bed & Hall Bath",
"max_occupancy": 1,
"image_url": "https://exampleapiurl.com/assets/282/2017/09/Basement-4-e1624825910642.jpg"
}
}
However, it keeps re-ordered automatically by its ID (7, 80, 552). I want to keep it that way (in the same exact order).
The JSON protocol does not define an ordering for dictionary entries. As such, in all implementations it will be impossible to set or preserve a specific order.
If you want to set an order, modify the JSON schema so that the structure containing these entries is a list instead. For example:
{
"pricing_options":[
{
"lodging_id":7,
"lodging_url":"https://exampleapiurl.com/7",
"price":520.0,
"name":"Private Room w/EnSuite Bath",
"max_occupancy":2,
"image_url":"https://exampleapiurl.com/assets/282/2017/09/Karuna.jpg"
},
{
"lodging_id":80,
"lodging_url":"https://exampleapiurl.com/80",
"price":400.0,
"name":"Private Room w/Twin Bed & Hall Bath",
"max_occupancy":1,
"image_url":"https://exampleapiurl.com/assets/282/2017/09/Basement-4-e1624825910642.jpg"
},
{
"lodging_id":552,
"lodging_url":"https://exampleapiurl.com/552",
"price":400.0,
"name":"Private Room w/Queen Bed & Hall Bath",
"max_occupancy":2,
"image_url":"https://exampleapiurl.com/assets/282/2019/03/Khandro-300x240.jpg"
},
]
}
Alternatively, parse the JSON and build an internal data structure that maintains the entries in a specific order.
Related
When returning a list of objects in a JSON response, say a GET request to a /movies endpoint, is it more common to return a JSON array or an object that wraps a JSON array? I've seen both formats in APIs and I was wondering if the standard. If there isn't, which way is preferable?
i.e.
[
{
"name": "Harry Potter",
"year": 2000
}
]
vs.
{
"movies": [
{
"name": "Harry Potter",
"year": 2000
}
]
}
In general if you have a service that only return a list, the first option is perfect fine:
[
{
"name": "Harry Potter",
"year": 2000
}
]
But if you are thinking in a general way to do it will be better add more context data, as total items counter, pagination variables or status values. So in spite of the first one is perfectly fine, I always prefer the second one, but without the name of the collection/array/table name and with more context info, as for example:
{
"items": [
{
"name": "Harry Potter",
"year": 2000
}
],
"total": 1,
"page": 1,
"pages": 1
"status": 1,
"timestamp: 121344
}
Set the array nested on movies value is a bit redundant. But for my it's only a practical approach that for my experience is more readable and used in all projects which I am related.
I have followed the Postman tutorial for the model derivative API, specifically for extracting metadata. I used a .dxf file, since I want to know if it is possible to retrieve perimeter, length/width properties based off the file.
I received a 200 response and it gave me a massive list of objects w/ their respective objectid's. Basically I got back a ton of these:
{
"objectid": 253,
"name": "Line [108]",
"externalId": "108",
"properties": {
"3D Visualization ": {
"Material": "ByLayer"
},
"General": {
"Color": "ByLayer",
"Handle": "108",
"Layer": "color#000000ff",
"Linetype": "BYLAYER",
"Linetype scale": "1.000",
"Lineweight": "ByLayer",
"Name ": "Line",
"Plot style": "ByColor",
"Thickness": "0.000 mm",
"Transparency": "ByLayer"
},
"Geometry": {
"Angle": "192.931 deg",
"Length": "0.088 mm"
}
}
}
The .dxf file I tested was as simple as possible and it looks like this image:
How can I retrieve the perimeter of this image? Is it possible to retrieve other specific geometric properties that I specify?
How can I know what part of the .dxf file each objectid is referring to?
Although it looks simple, the polyline (?) is probably being tessellated, resulting in a large number of small lines. Have you tried the original DWG file? Can you try that with viewer.autodesk.com?
I am doing an API call every 40 mins to retrieve the current status information of every car in a car fleet. And each call adds one new JSON document to a Cloudant database. Each JSON document defines the current availability status for every car across many locations in many cities. There are currently around 2200 JSON documents in the database. All JSON documents have one field called payload that contains all information; it is a large array of objects. Instead of retrieving the whole payload array of objects I would like to retrieve only the needed info with a query (so, only one or several objects of that array). However, I have difficulty drafting a query that results only in the needed data.
Below, I'll explain my problem in more detail:
When saving the JSON document to Cloudant, a timestamp is defined in the document. The _id parameter is defined to be equal to this timestamp. Below, I show a simplified version of these JSON documents:
{
"_id": "1540914946026",
"_rev": "3-c1834c8a230cf772e41bbcb9cf6b682e",
"timestamp": 1540914946026,
"datetime": "2018-10-30 15:55:46",
"payload": [
{
"cityName": "Abcoude",
"locations": [
{
"address": "asterlaan 28",
"geoPoint": {
"latitude": 52.27312,
"longitude": 4.96768
},
"cars": [
{
"mod": "BMW",
"state": "FREE"
}
]
}
],
"availableCars": 1,
"occupiedCars": 0
},
{
"cityName": "Alkmaar",
"locations": [
{
"address": "Aert de Gelderlaan 14",
"geoPoint": {
"latitude": 52.63131,
"longitude": 4.72329
},
"cars": [
{
"model": "Volswagen",
"state": "FREE"
}
]
},
{
"address": "Ardennenstraat 49",
"geoPoint": {
"latitude": 52.66721,
"longitude": 4.76046
},
"cars": [
{
"mod": "BMW",
"state": "FREE"
}
]
},
{
"address": "Beneluxplein 7",
"geoPoint": {
"latitude": 52.65356,
"longitude": 4.75817
},
"cars": [
{
"mod": "BMW",
"state": "FREE"
}
]
},
{
"address": "Dr. Schaepmankade 1",
"geoPoint": {
"latitude": 52.62595,
"longitude": 4.75122
},
"cars": [
{
"mod": "BMW",
"state": "OCCUPIED"
}
]
},
{
"address": "Kennemerstraatweg",
"geoPoint": {
"latitude": 52.62909,
"longitude": 4.74226
},
"cars": [
{
"model": "Mercedes",
"state": "FREE"
}
]
},
{
"address": "NS Station Alkmaar Noord/Parkeerterrein Noord",
"geoPoint": {
"latitude": 52.64366,
"longitude": 4.7627
},
"cars": [
{
"model": "Tesla",
"state": "FREE"
}
]
},
{
"address": "NS Station Alkmaar/Stationsweg 56",
"geoPoint": {
"latitude": 52.6371,
"longitude": 4.73935
},
"cars": [
{
"model": "Tesla",
"state": "FREE"
}
]
},
{
"address": "Oude Hoeverweg",
"geoPoint": {
"latitude": 52.63943,
"longitude": 4.72928
},
"cars": [
{
"model": "Tesla",
"state": "FREE"
}
]
},
{
"address": "Parkeerterrein Wortelsteeg",
"geoPoint": {
"latitude": 52.63048,
"longitude": 4.75487
},
"cars": [
{
"model": "Tesla",
"state": "OCCUPIED"
}
]
},
{
"address": "Schoklandstraat 38",
"geoPoint": {
"latitude": 52.65812,
"longitude": 4.75359
},
"cars": [
{
"model": "Volkswagen",
"state": "FREE"
}
]
}
],
"availableCars": 8,
"occupiedCars": 2
}
]
}
As you can see, the payload field is an array that has several objects (FYI: every object in this array represents one specific city: there are 1600 cities, so 1600 nested objects inside the payload array). Furthermore, inside each of the 1600 objects mentioned, other arrays and objects are again nested inside. For all objects in the payload array, the first field is cityName.
Furthermore, there is a nested array locations (inside each of the 1600 objects of the payload array) representing all addresses in a specific city. The locations array can be of size 1 to 600, meaning 1 to 600 nested objects / addresses per city. The last two fields in all objects of the payload array are availableCars and occupiedCars.
I want query documents to see how many cars are available and occupied for a specific city during a specific time interval. To do this:
I have to specify a start timestamp (or id) and an end timestamp, resulting in only the JSON documents within this interval.
Furthermore, I will need to specify inside the JSON documents only one or more specific cities by cityName (there are 1600 cities) and then get the number of available cars availableCars and the number of occupiedCars for those cities.
For example, in this simplified example, I would like to query for the status information (availableCars & `occupiedCars) for the city of Alkmaar from 1540914946026 (epoch time) until now. I would like to get the following result:
{
"id":"1540914946026",
"cityName":"Alkmaar",
"availableCars":8,
"occupiedCars":2
}
This is just an example, in reality, I want to be able to query for other cities as well, or query for several cities together and then get for each of those cities the number of available cars availableCars and the number of occupied cars occupiedCars.
Could anyone help me to define a query and index to be able to get the above result? Can I do this with cloudant query?
Your data model does not play to Cloudant's strengths. Let each document group data that changes and is accessed together. Your items in your payload array would be much better stored as discrete documents.
If you find yourself reaching into growing arrays inside documents for subsets of data, this is a warning sign that your data model is not ideal: the document is now mutable and growing (with potential update conflicts as a result), and access becomes more cumbersome over time as Cloudant has no mechanism to only retrieve parts of a document. Moreover, Cloudant has a limit (1M) on document size, so by using your proposed model, you will likely hit that limit, too, and your application would stop working.
With that said, it is possible to create a view index that lets you emit each component of your payload, which would let you look up data per city -- but that solution is still subject to all the limitations above (document model is mutable, documents grow large etc).
Rule of thumb: small documents. Immutable model, where possible. Documents group data that either change, or are accessed as a unit.
I'm making a call to a Wikipedia API which returns a Title, Shot-Text, Image and geo-cordinates of that location. My Wikipedia API is:
https://en.wikipedia.org/w/api.php?format=json&action=query&prop=extracts|pageimages|coordinates&titles=Berlin&redirects=1&formatversion=2&exintro=1&explaintext=1&piprop=thumbnail&pithumbsize=400
Also I'm using another Wikipedia API which returns a list of place names according to their geo-coordinates:
https://en.wikipedia.org/w/api.php?format=json&action=query&list=geosearch&gsradius=1000&gscoord=52.5243700|13.4105300&gslimit=50&gsprop=type|dim|globe
For the second API I get a response like this:
"query": {
"geosearch": [
{
"pageid": 28782169,
"ns": 0,
"title": "1757 Berlin raid",
"lat": 52.523405,
"lon": 13.4114,
"dist": 122.4,
"primary": "",
"type": null,
"dim": 1000
},
{
"pageid": 526195,
"ns": 0,
"title": "Scheunenviertel",
"lat": 52.526111111111,
"lon": 13.41,
"dist": 196.9,
"primary": "",
"type": "landmark",
"dim": 1000
},
...
]
}
Now I want to combine these two searches in one API. I want to add information from my first API within the second API, something like as below:
"query": {
"geosearch": [
{
"pageid": 28782169,
"ns": 0,
"title": "1757 Berlin raid",
"lat": 52.523405,
"lon": 13.4114,
"dist": 122.4,
"primary": "",
"type": null,
"dim": 1000
"pages": [
{
"pageid": 28782169,
"ns": 0,
"title": "1757 Berlin raid",
"extract": "Berlin is the capital of Germany and one of the 16 states of Germany. With a population of 3.5 million people, it is the second most populous city proper and the seventh.........",
"thumbnail": {
"source": "https://upload.wikimedia.org/wikipedia/commons/thumb/3/3b/Siegessaeule_Aussicht_10-13_img4_Tiergarten.jpg/400px-Siegessaeule_Aussicht_10-13_img4_Tiergarten.jpg",
"width": 400,
"height": 267
}
}
]
},
...
]
}
I want to know is it possible in this way?
So, if I understand right, you want with one Wikipedia API request to get a title, shot-text, image and geo-cordinates (your first API) for all the places (Wikipedia articles) that are located in a certain area by given coordinates and radius (your second API). If that correct, you can do it in this way:
main parameters: format=json&action=query
query parameters:
redirects=1
generator=geosearch (your second API: see point 3)
prop=extracts|coordinates|pageimages (your first API: see point 4, 5, and 6)
geosearch parameters (all generator parameters are prefixed with a "g"):
ggslimit=20 - your total results from the query (because the exlimit=20)
ggsradius=1000&ggscoord=52.5243700|13.4105300 - this is your entry point
parameters for extracts: exintro=1&explaintext=1&exlimit=20 (max exlimit is 20)
parameters for coordinates: coprop=type|dim|globe&colimit=20 (max colimit is 500)
parameters for pageimages: piprop=thumbnail&pithumbsize=400&pilimit=20 (max is 50)
How you see, the max colimit is 500 and the max pilimit is 50, but we can't use more than 20, because of exlimit.
Or finally, your request will be the joining of all parameters above:
https://en.wikipedia.org/w/api.php?format=json&action=query&redirects=1&generator=geosearch&prop=extracts|coordinates|pageimages&ggslimit=20&ggsradius=1000&ggscoord=52.5243700|13.4105300&exintro=1&explaintext=1&exlimit=20&coprop=type|dim|globe&colimit=20&piprop=thumbnail&pithumbsize=400&pilimit=20
And here is the response:
"query":{
"pages":{
"2511":{
"pageid":2511,
"ns":0,
"title":"Alexanderplatz",
"extract":"Alexanderplatz (pronounced [\u0294al\u025bk\u02c8sand\u0250\u02ccplats]) is a large public square and transport hub in the central Mitte district of Berlin, near the Fernsehturm. Berliners often call it simply Alex, referring to a larger neighbourhood stretching from Mollstra\u00dfe in the northeast to Spandauer Stra\u00dfe and the Rotes Rathaus in the southwest.",
"coordinates":[
{
"lat":52.52166748,
"lon":13.41333294,
"primary":"",
"type":"landmark",
"dim":"1000",
"globe":"earth"
}
],
"thumbnail":{
"source":"https://upload.wikimedia.org/wikipedia/commons/thumb/d/da/Alexanderplatz_by_the_night_-_ProtoplasmaKid.webm/400px--Alexanderplatz_by_the_night_-_ProtoplasmaKid.webm.jpg",
"width":400,
"height":225
}
},
...
},
}
I think that's the word I'm looking for. I'm trying to get parent info into each of the cards. I think that's what I need to do, but chime in if you have any other ideas.
{
"LEA": {
"name": "Limited Edition Alpha",
"code": "LEA",
"releaseDate": "1993-08-05",
"border": "black",
"type": "core",
"cards": [
{"name": "Air Elemental"},
{"name": "Earth Elemental"},
{"name": "Fire Elemental"},
{"name": "Water Elemental"}
]
},
"LEB": {
"name": "Limited Edition Beta",
"code": "LEB",
"releaseDate": "1993-10-01",
"border": "black",
"type": "core",
"cards": [
{"name": "Armageddon"},
{"name": "Fireball"},
{"name": "Swords to Plowshares"},
{"name": "Wrath of God"}
]
}
}
This is a tiny subset of the data, obviously. LEA and LEB are sets of cards, and inside each set there are a bunch of cards. I'm thinking of denormalizing this into just the cards, with the set info added to each card. Something like this...
{
{
"name": "Air Elemental",
"set": {
"name": "Limited Edition Alpha",
"code": "LEA",
"releaseDate": "1993-08-05",
"border": "black",
"type": "core"
}
},
{
"name": "Earth Elemental",
"set": {
"name": "Limited Edition Alpha",
"code": "LEA",
"releaseDate": "1993-08-05",
"border": "black",
"type": "core"
}
},
{
"name": "Armageddon",
"set": {
"name": "Limited Edition Beta",
"code": "LEB",
"releaseDate": "1993-10-01",
"border": "black",
"type": "core"
}
},
{
"name": "Fireball",
"set": {
"name": "Limited Edition Beta",
"code": "LEB",
"releaseDate": "1993-10-01",
"border": "black",
"type": "core"
}
}
}
Is my thinking right, first and foremost? Would I want a giant collection of cards and have the set information flattened into each card? In SQL, I'd do a table for the sets, and and the cards would belong_to a set. I'm trying to wrap my head around 'document thinking'.
Second, if my thinking is correct, any ideas on how I could achieve this denormalizing?
Here you go =).
OK here is where I would start. Since we've said that cards will never change (since they're based on physical MTG cards), create one collection with all of your cards in it, this will be used for easily populating a user's deck later on. You can search on it by card name or some sort of card ID (like a physical one, stored on the card).
For the user's array of card objects, you shouldn't just store the _id field for a card, because that forces you to join. Since cards will never change, completely denormalize them and just shove them in that card array, so a user object, so far, resembles:
{
name: "Tom Hanks",
skill_level: 0,
decks: [
[
{
card_name: "Balance",
card_description: "LONG_BLOCK_OF_DESCRIP_TEXT",
card_creator: "Sugargirl14",
type: "Normal",
_id: $SOME_MONGO_ID_HERE,
... rest of card data...
}, {
...card 2 complete data...
}
],
[
{ ...another deck here... }
]
]
}
OK, back to set info, I will also assume set info is a constant (based on your SO post, I can't see how it would physically change). So, if that set info is always relevant to the card, I would denormalize and include it, changing our card object to:
{
card_name: "Balance",
card_description: "LONG_BLOCK_OF_DESCRIP_TEXT",
card_creator: "Sugargirl14",
type: "Normal",
_id: $SOME_MONGO_ID_HERE
set: {
"name": "Limited Edition Alpha",
"code": "LEA",
"releaseDate": "1993-08-05",
"border": "black",
"type": "core",
"_id": $SOME_MONGO_ID_HERE
},
... rest of card data...
}
I imagine that storing the other cards in the denormalized object for a given card isn't relevant, if it is, add them. If you'll note, the key that is given in your SO example is dropped, since it seems to always == the "code" field.
OK, now to properly answer your SO question about whether you should embed sets in cards, or vice versa. First off, both collections are relevant. So, even if we embed sets into cards, you'll want those sets in a collection so they can be fetched later and inserted into new cards.
Which gets embedded in which is really determined by business logic, how the data is used and which gets pulled more often. Are you frequently displaying sets and pulling cards from them (like for users to search)? You could embed all of the card data, or any relevant data, in each set's cards array. But with the above data model, each card stores its set ID in its set object. I assume cards belong to only one set, so to get all cards for a set you can query over your card collection where set.id == the Mongo ID of the set you want. Now sets need minimal updates, due to business logic, (hopefully none at all) and your queries are still fast (and you get complete card objects). I'd, honestly, do that latter one and keep my sets clean of cards. As such, a card owns the set it belongs to as opposed to a set owning a card. That's a more SQLy way to think that actually can work fine in Mongo (you'll never join).
So our final data model resembles:
Collection 1, Set:
//data model
{
"name": "Limited Edition Alpha",
"code": "LEA",
"releaseDate": "1993-08-05",
"border": "black",
"type": "core",
"_id": $SOME_MONGO_ID_HERE
}
Collection 2, cards:
//data model
{
_id: $SOME_MONGO_ID_HERE
card_name: "Balance",
card_description: "LONG_BLOCK_OF_DESCRIP_TEXT",
card_creator: "Sugargirl14",
type: "Normal",
set: {
"name": "Limited Edition Alpha",
"code": "LEA",
"releaseDate": "1993-08-05",
"border": "black",
"type": "core",
"_id": $SOME_MONGO_ID_HERE
... rest of card data...
},
}
Collection 3, users:
{
_id: $SOME_MONGO_ID_HERE,
name: "Tom Hanks",
skill_level: 0,
decks: [
[
{
card_name: "Balance",
card_description: "LONG_BLOCK_OF_DESCRIP_TEXT",
card_creator: "Sugargirl14",
type: "Normal",
_id: $SOME_MONGO_ID_HERE,
set: {
"name": "Limited Edition Alpha",
"code": "LEA",
"releaseDate": "1993-08-05",
"border": "black",
"type": "core",
"_id": $SOME_MONGO_ID_HERE
},
}, {
...card 2 complete data...
}
],
[
{ ...another deck here... }
]
]
}
This, obviously, assumes set data for each card is relevant to the user. Now your data is denormalized, sets and cards rarely need updates (according to business logic), so you'll never need cascading updates or deletes. Manipulating users is easy. When you remove a card from a user's deck you can do a $pull from Mongo (I think that's what it's called) on the relevant decks array where a contained item's _id field == the Mongo ID of the card you want to remove. All other updates are easier.
In retrospect, you might want to make the user's decks like so:
decks: {
"SOME_ID_HERE": [
{ ...card 1... },
{ ...card 2... }
]
}
This makes identifying the decks MUCH easier and will make your pulls easier (you'll have more data on the frontend and the pull query will be more precise). It can be a number, random string, anything really, since it gets passed back to the frontend. Or just use their Mongo ID, when looking at a deck, a user will have it's Mongo ID. Then when they pull a card out of it, or add one in, you have a direct identifier to easily grab the deck needed.
Obviously all values with text like: $MONGO_ID_HERE should really be MongoId() objects.
Whew, that was intense, 6800 characters. Hope it makes sense to you and I apologize if any verbiage is confusing or if any of my JSON objects' formatting is fucked up (just let me know if any prose is confusing, I'll reword). Does this make sense/solve your problem?