Serializing Multiple API Fields into one. Django - json

Question

I have a pre-defined API, like:
{
time : some_time,
height : {1: 154, 2: 300, 3: 24},
color : {1: 'red', 2: 'blue', 3: 'green'},
age : {1: 27, 2: 324, 3: 1},
... many, many more keys.
}
I have no control of this API, so cannot change its structure.
Each integer key inside the sub dictionaries are linked and part of one record. For example the object that is 154 in height, is also colour: red and age: 27.
I am aware one strategy to work with this is to have separate serialisers for each field.
class MySerializer(serializers.ModelSerializer):
# Nested serializers
height = HeightSerializer()
colour = ColourSerializer()
age = AgeSerializer()
etc, etc, etc
But that still gives me messy data to work with, that requires lots of update() logic in the serializer.
What I instead want to do is have one nested serializer that has access to the full request data, and can work with height, colour and age simultaneously and return me something like from the to_internal_value() method:
{
['record' : 1, 'height': 154, 'colour' : 'red', 'age' : 27],
['record' : 2, 'height': 300, 'colour' : 'blue', 'age' : 324],
['record' : 3, 'height': 24, 'colour' : 'green', 'age' : 2],
}
But unfortunately the height serializer only seems to have access to information on fields called height. I am aware I can user source="foo" in the init call, but then it only has access to a field called "foo". I want it to have access to all fields.
I noticed there is a source='*' option, but it doesn't work. My init method of the serializer never gets called unless there is a key "height" in the api call.
Any ideas how I can have a nested serialiser that has access to all the data in the request?
Thanks
Joey