Deserializing Nested JSON API response with Django - json

Question

I'm pretty new to the DRF and serializing/deserializing. I'm slowly building a dashboard for my business during the corona virus and learning to code. I am in a little deep, but after spending more than $10k on developers on upwork and not really get much result, I figured, what do I have to lose?
Our software provider has a full API for our needs https://developer.myvr.com/api/, but absolutely no dashboard to report statistics about our clients reservation data.
The end result will be a synchronization of some of the data from their API to my database which will be hosted through AWS. I chose to do it this way due to having to do some post processing of data from the API. For example, we need to calculate occupancy rates(which is not an endpoint), expenses from our accounting connection and a few other small calculations in which the data is not already in the provided API. I originally wanted to use the data from the API solely, but I'm hesitant due to the reasons above.
That's the backstory, here are the questions:
The API response is extremely complex and nested multiple times, what is the best practise to extract a replication of the structure of the data to my own Database? Would I have to create models for each field manually?
Example response:
```{
"uri": "https://api.myvr.com/v1/properties/b6b0f2fe278f612b/",
"id": "b6b0f2fe278f612b",
"key": "b6b0f2fe278f612b",
"accessDescription": null,
"accommodates": 11,
"active": false,
"addressOne": "11496 Zermatt Dr",
"addressTwo": null,
"allowTurns": true,
"amenities": "https://api.myvr.com/v1/property-amenities/?propertyId=b6b0f2fe278f612b",
"automaticallyApprove": false,
"baseNightlyRate": "395.00",
"baseRate": {
"uri": "https://api.myvr.com/v1/rates/660c299d4785c32e/",
"id": "660c299d4785c32e",
"key": "660c299d4785c32e",
"externalId": null,
"baseRate": true,
"changeoverDay": null,
"created": "2019-01-19T08:02:36Z",
"currency": "USD",
"endDate": "2020-01-18",
"minStay": 3,
"modified": "2019-01-19T08:02:36Z",
"monthly": 0,
"name": "Base Rate",
"weekNight": 39500,
"nightly": 39500,
"position": 0,
"property": {
"name": "API Demo Property",
"uri": "https://api.myvr.com/v1/properties/b6b0f2fe278f612b/",
"id": "b6b0f2fe278f612b",
"externalId": null,
"key": "b6b0f2fe278f612b",
"slug": "api-demo-property"
},
"ratePlan": {
"uri": "https://api.myvr.com/v1/rate-plans/862caa3f5267602d/",
"key": "862caa3f5267602d",
"name": "Default Rates for Property"
},
"repeat": true,
"startDate": "2020-01-18",
"weekend": 0,
"weekendNight": 0,
"weekly": 250000
},
"bathrooms": "4.0",
"bedrooms": 4,
"bookingUrl": "https://myvr.com/reservation/redirect/booking/b6b0f2fe278f612b/",
"checkInTime": "16:00:00",
"checkOutTime": "10:00:00",
"city": "Truckee",
"commissionStructure": null,
"countryCode": "US",
"created": "2016-01-19T00:01:48Z",
"currency": "USD",
"customFields": {},
"description": "Luxurious living, scenic mountain setting, entertainment galore. Located on a quiet street in Tahoe Donner, our well equipped modern home is nestled into the wilderness. A babbling creek greets visitors approaching the front step as it collects into a small pond with a cascading waterfall. <br/>\n<br/>\nInside, over 3,000 sqft of luxurious living space divides itself between two floors. On the first floor, a beautiful kitchen with granite counters, gas stove and stainless steel appliances opens to a large great room centered around a wood burning fireplace and featuring 30' soaring ceilings. A spacious loft overlooks the great room, showcasing a large poker/card table. Upstairs features a large entertainment room, complete with wet bar, shuffleboard table, and state-of-the-art television setup with surround sound. The scenic backyard is accessible from a large deck featuring a new hot tub with seating for 7.",
"externalId": null,
"feePlan": {
"uri": "https://api.myvr.com/v1/fee-plans/4d1c44383755051b/",
"key": "4d1c44383755051b",
"name": "Default Fees for Listing"
},
"headline": "Beautiful Four Bedroom Lake Front Property",
"houseRules": null,
"instantBookingsEnabled": false,
"lat": "39.3422523000",
"level": "unit",
"localAreaDescription": "Tahoe Donner is a year round activity resort. The amenities include private beach/boat launching facilities, pools, recreation center, tennis, horseback riding, golf, downhill skiing as well as cross country skiing. Truckee is a historical mining town-having a western feel but also has museums, theaters, fine dining plus 2 large supermarkets-all less than 3 miles from the house. Our home is also located within a 15 minute drive to 4 major ski resorts. Downtown Reno is a short 40 minute drive away for those seeking a night on the town or the thrill of a Nevada casino.",
"lon": "-120.2271947000",
"lowestNightlyRate": "395.00",
"manual": "",
"modified": "2019-10-18T17:18:43Z",
"name": "API Demo Property",
"owner": null,
"postalCode": "96161",
"ratePlan": {
"uri": "https://api.myvr.com/v1/rate-plans/862caa3f5267602d/",
"key": "862caa3f5267602d",
"name": "Default Rates for Property"
},
"ratePlanLocked": false,
"region": "CA",
"shortCode": "API",
"size": 3000,
"slug": "api-demo-property",
"suitableElderly": "yes",
"suitableEvents": "unknown",
"suitableGroups": "yes",
"suitableHandicap": "no",
"suitableInfants": "unknown",
"suitableKids": "yes",
"suitablePets": "no",
"suitableSmoking": "no",
"transitDescription": null,
"type": "house",
"weekendNights": [
5,
6
]
}```
I think the best way to populate the database would be to run a custom management command to run a once off script, I've done this previously with another database, however I'm still stuck as I don't really want to write these models manually. Also a concern is if a field is missing or the structure changes.
This project is definitely above my skills and extremely ambitious, but I would appreciate any feedback or advice anyone might have.
Thanks,
Darren

Answer 1

So I didn't really get any interest in this question, but I ended up working it out myself.
I hope someone googles it and might find it helpful.
import requests
from rest_framework.response import Response
from django.core.management.base import BaseCommand, CommandError
from reservation.models import Reservation
import time
MYVR_URL = 'https://api.myvr.com/'
MYVR_RESERVATION = 'v1/reservations/?limit=100'
headers = {
'Authorization': 'Basic SOmeAPiCodeHeRe123=',
}
class Command(BaseCommand):
help = 'Imports new properties and saves the objects in the database'
def handle(self, *args, **options):
url = MYVR_URL + MYVR_RESERVATION
print("Populating Reservations")
def looping_api(url, headers):
while url:
r = requests.request("GET", url, headers=headers)
url = r.json().get('next')
props_data = r.json().get('results')
start_time = time.time()
for prop in props_data:
try:
created = Reservation.objects.update_or_create(
myvr_key=prop.get('key'),
adults=prop.get('adults'),
children=prop.get('children'),
checkin=prop.get('checkIn'),
checkout=prop.get('checkOut'),
checkinTime=prop.get('checkInTime'),
checkoutTime=prop.get('checkOutTime'),
guestFirstName=prop.get('firstName'),
dateCreated=prop.get('created'),
dateBooked=prop.get('dateBooked'),
dateCancelled=prop.get('dateCanceled'),
contact=prop.get('contact').get('name'),
contact_key=prop.get('contact').get('key'),
guest_type=prop.get('guestType'),
property_name=prop.get('property').get('name'),
property_key=prop.get('property').get('key'),
source_code=prop.get('source').get('code'),
source_name=prop.get('source').get('name'),
total_due=prop.get('quote').get('totalDue'),
total_refundables=prop.get(
'quote').get('totalRefundableFees'),
total_nonrefundables=prop.get(
'quote').get('totalNonrefundableFees'),
reference_id=prop.get('referenceId'),
)
print(
f"Added obj {prop.get('key')}")
except AttributeError as error:
print(f"{error} attribute is null or owner booking")
url = r.json().get('next')
print(r.json().get('next'))
print(len(props_data))
end_time = time.time()
duration = (end_time - start_time)
print(duration)
looping_api(url, headers)