How to filter keys in record_path in pandas json_normalize method? - json

Question

I have a large json/dict:
{
"vegetarian": false,
"vegan": false,
"glutenFree": false,
"dairyFree": false,
"veryHealthy": false,
"cheap": false,
"veryPopular": false,
"sustainable": false,
"weightWatcherSmartPoints": 14,
"gaps": "no",
"lowFodmap": false,
"aggregateLikes": 1,
"spoonacularScore": 57.0,
"healthScore": 21.0,
"creditsText": "BBC Good Food",
"sourceName": "BBC Good Food",
"pricePerServing": 231.2,
"id": 1089577,
"title": "Hungarian Goulash Stew",
"readyInMinutes": 120,
"servings": 4,
"sourceUrl": "https://www.bbcgoodfood.com/recipes/1107653/hungarian-goulash-stew",
"image": "https://spoonacular.com/recipeImages/1089577-312x231.jpg",
"imageType": "jpg",
"nutrition": {
"nutrients": [
{
"name": "Calories",
"title": "Calories",
"amount": 323.18,
"unit": "kcal",
"percentOfDailyNeeds": 16.16
},
{
"name": "Fat",
"title": "Fat",
"amount": 15.14,
"unit": "g",
"percentOfDailyNeeds": 23.3
},
{
"name": "Saturated Fat",
"title": "Saturated Fat",
"amount": 4.43,
"unit": "g",
"percentOfDailyNeeds": 27.69
},
{
"name": "Carbohydrates",
"title": "Carbohydrates",
"amount": 38.95,
"unit": "g",
"percentOfDailyNeeds": 12.98
},
{
"name": "Net Carbohydrates",
"title": "Net Carbohydrates",
"amount": 34.64,
"unit": "g",
"percentOfDailyNeeds": 12.6
},
{
"name": "Sugar",
"title": "Sugar",
"amount": 11.27,
"unit": "g",
"percentOfDailyNeeds": 12.52
},
{
"name": "Cholesterol",
"title": "Cholesterol",
"amount": 102.67,
"unit": "mg",
"percentOfDailyNeeds": 34.22
},
{
"name": "Sodium",
"title": "Sodium",
"amount": 1864.24,
"unit": "mg",
"percentOfDailyNeeds": 81.05
},
{
"name": "Protein",
"title": "Protein",
"amount": 10.3,
"unit": "g",
"percentOfDailyNeeds": 20.6
},
{
"name": "Vitamin A",
"title": "Vitamin A",
"amount": 3866.5,
"unit": "IU",
"percentOfDailyNeeds": 77.33
},
{
"name": "Vitamin E",
"title": "Vitamin E",
"amount": 3.49,
"unit": "mg",
"percentOfDailyNeeds": 23.3
},
{
"name": "Selenium",
"title": "Selenium",
"amount": 13.41,
"unit": "µg",
"percentOfDailyNeeds": 19.16
},
{
"name": "Manganese",
"title": "Manganese",
"amount": 0.38,
"unit": "mg",
"percentOfDailyNeeds": 18.83
},
{
"name": "Iron",
"title": "Iron",
"amount": 3.19,
"unit": "mg",
"percentOfDailyNeeds": 17.74
},
{
"name": "Vitamin B2",
"title": "Vitamin B2",
"amount": 0.3,
"unit": "mg",
"percentOfDailyNeeds": 17.54
},
{
"name": "Fiber",
"title": "Fiber",
"amount": 4.31,
"unit": "g",
"percentOfDailyNeeds": 17.22
},
{
"name": "Folate",
"title": "Folate",
"amount": 61.04,
"unit": "µg",
"percentOfDailyNeeds": 15.26
},
{
"name": "Vitamin B6",
"title": "Vitamin B6",
"amount": 0.3,
"unit": "mg",
"percentOfDailyNeeds": 14.9
},
{
"name": "Vitamin B1",
"title": "Vitamin B1",
"amount": 0.21,
"unit": "mg",
"percentOfDailyNeeds": 13.72
},
{
"name": "Vitamin K",
"title": "Vitamin K",
"amount": 13.78,
"unit": "µg",
"percentOfDailyNeeds": 13.13
},
{
"name": "Potassium",
"title": "Potassium",
"amount": 407.99,
"unit": "mg",
"percentOfDailyNeeds": 11.66
},
{
"name": "Vitamin C",
"title": "Vitamin C",
"amount": 9.53,
"unit": "mg",
"percentOfDailyNeeds": 11.56
},
{
"name": "Phosphorus",
"title": "Phosphorus",
"amount": 111.88,
"unit": "mg",
"percentOfDailyNeeds": 11.19
},
{
"name": "Vitamin B3",
"title": "Vitamin B3",
"amount": 2.01,
"unit": "mg",
"percentOfDailyNeeds": 10.05
},
{
"name": "Copper",
"title": "Copper",
"amount": 0.15,
"unit": "mg",
"percentOfDailyNeeds": 7.53
},
{
"name": "Magnesium",
"title": "Magnesium",
"amount": 29.73,
"unit": "mg",
"percentOfDailyNeeds": 7.43
},
{
"name": "Vitamin B5",
"title": "Vitamin B5",
"amount": 0.69,
"unit": "mg",
"percentOfDailyNeeds": 6.95
},
{
"name": "Zinc",
"title": "Zinc",
"amount": 0.89,
"unit": "mg",
"percentOfDailyNeeds": 5.93
},
{
"name": "Calcium",
"title": "Calcium",
"amount": 58.6,
"unit": "mg",
"percentOfDailyNeeds": 5.86
},
{
"name": "Vitamin B12",
"title": "Vitamin B12",
"amount": 0.2,
"unit": "µg",
"percentOfDailyNeeds": 3.3
},
{
"name": "Vitamin D",
"title": "Vitamin D",
"amount": 0.46,
"unit": "µg",
"percentOfDailyNeeds": 3.06
}
]
}
I would like to keep the id, readyInMinutes and title keys + nutrition, the issue is that I need the nutrition to be parsed (because it's a list of dicts).
I am trying panda's pd.json_normalize function to do this:
keys = ['id', 'title', 'readyInMinutes']
pd.json_normalize(results[0], meta = keys, record_path=['nutrition',['nutrients']], record_prefix='nutrients.')
But if I need to filter which keys I want inside the record_path? For example only title and amount?
Please instruct how can I configure it.

Answer 1

To parse JSON data you can use JMESPath library.
JMESPath search syntax:
search(<jmespath expr>, <JSON document>)
In your case the JMESPath expression will be this:
{id: id, readyInMinutes:readyInMinutes, title: title, nutrition: {nutrients: nutrition.nutrients[*].{title: title, amount: amount}}}
Output:
{
"id": 1089577,
"readyInMinutes": 120,
"title": "Hungarian Goulash Stew",
"nutrition": {
"nutrients": [
{
"title": "Calories",
"amount": 323.18
},
{
"title": "Fat",
"amount": 15.14
},
{
"title": "Saturated Fat",
"amount": 4.43
},
{
"title": "Carbohydrates",
"amount": 38.95
},
{
"title": "Net Carbohydrates",
"amount": 34.64
},
{
"title": "Sugar",
"amount": 11.27
},
{
"title": "Cholesterol",
"amount": 102.67
},
{
"title": "Sodium",
"amount": 1864.24
},
{
"title": "Protein",
"amount": 10.3
},
{
"title": "Vitamin A",
"amount": 3866.5
},
{
"title": "Vitamin E",
"amount": 3.49
},
{
"title": "Selenium",
"amount": 13.41
},
{
"title": "Manganese",
"amount": 0.38
},
{
"title": "Iron",
"amount": 3.19
},
{
"title": "Vitamin B2",
"amount": 0.3
},
{
"title": "Fiber",
"amount": 4.31
},
{
"title": "Folate",
"amount": 61.04
},
{
"title": "Vitamin B6",
"amount": 0.3
},
{
"title": "Vitamin B1",
"amount": 0.21
},
{
"title": "Vitamin K",
"amount": 13.78
},
{
"title": "Potassium",
"amount": 407.99
},
{
"title": "Vitamin C",
"amount": 9.53
},
{
"title": "Phosphorus",
"amount": 111.88
},
{
"title": "Vitamin B3",
"amount": 2.01
},
{
"title": "Copper",
"amount": 0.15
},
{
"title": "Magnesium",
"amount": 29.73
},
{
"title": "Vitamin B5",
"amount": 0.69
},
{
"title": "Zinc",
"amount": 0.89
},
{
"title": "Calcium",
"amount": 58.6
},
{
"title": "Vitamin B12",
"amount": 0.2
},
{
"title": "Vitamin D",
"amount": 0.46
}
]
}
}