How do I query a complex JSON in Jinja2? - json

Question

I'm parsing a complex JSON with a Jinja2 template in Ansible. The JSON has a lot of items and random arrays.
For example, how could I query the value of name & id if the date < 2001 for all motors? (this should return: Premium diesel & 4). Please note that this is an example structure and it could have random and more children in my real scenario. What I need is a query to r search for the above keys in the entire tree.
{
"cars":[
{
"Ford":[
{
"vehicle":{
"motor":[
{
"diesel":[
{
"regular":{
"name":"Regular diesel",
"specs":{
"law":[
{
"id":1,
"date":"2008"
}
],
"gas_station":[
{
"id":2,
"date":"2002"
}
]
}
},
"premium":{
"name":"Premium diesel",
"specs":{
"law":[
{
"id":3,
"date":"2005"
}
],
"gas_station":[
{
"id":4,
"date":"2000"
}
]
}
}
}
]
}
]
}
}
]
}
]
}

Answer 1

Q: "How could I query the value of name & id if the date < 2001?
(this should return: Premium diesel & 4)
A: Create a list of the items where the date is less than '2001'
year: 2001
vehicle_motors_year:
2001:
- brand: Ford
id: 4
name: Premium diesel
type: gas_station
- brand: Ford
id: 6
name: Natural gasoline
type: gas_station
- brand: VW
id: 12
name: Hybrid hydrogen
type: gas_station
Then select and format the items
result: "{{ vehicle_motors_year[year]|
json_query('[].[name, id]')|
map('join', ' & ') }}"
gives the expected result
result:
- Premium diesel & 4
- Natural gasoline & 6
- Hybrid hydrogen & 12
Details:
Given the list of cars for testing
cars:
- Ford:
- vehicle:
motor:
- diesel:
- premium:
name: Premium diesel
specs:
gas_station:
- date: '2000'
id: 4
law:
- date: '2005'
id: 3
regular:
name: Regular diesel
specs:
gas_station:
- date: '2002'
id: 2
law:
- date: '2008'
id: 1
- gasoline:
- natural:
name: Natural gasoline
specs:
gas_station:
- date: '2000'
id: 6
law:
- date: '2005'
id: 5
- VW:
- vehicle:
motor:
- hybrid:
- hydrogen:
name: Hybrid hydrogen
specs:
gas_station:
- date: '2000'
id: 12
law:
- date: '2005'
id: 11
Get the lists of brands and motors
brands: "{{ cars|map('first') }}"
motors: "{{ cars|json_query(query_motors) }}"
query_motors: '[].*[][].vehicle[].motor'
give
brands:
- Ford
- VW
motors:
- - diesel:
- premium:
name: Premium diesel
specs:
gas_station:
- date: '2000'
id: 4
law:
- date: '2005'
id: 3
regular:
name: Regular diesel
specs:
gas_station:
- date: '2002'
id: 2
law:
- date: '2008'
id: 1
- gasoline:
- natural:
name: Natural gasoline
specs:
gas_station:
- date: '2000'
id: 6
law:
- date: '2005'
id: 5
- - hybrid:
- hydrogen:
name: Hybrid hydrogen
specs:
gas_station:
- date: '2000'
id: 12
law:
- date: '2005'
id: 11
Create a dictionary of the brands' vehicles' motors
vehicle_motors_str: |
{% for motor in motors %}
{{ brands[loop.index0] }}:
{% for fuels in motor %}
{% for fuel,types in fuels.items() %}
{% for type in types %}
{% for k,v in type.items() %}
{{ fuel }}_{{ k }}:
{{ v }}
{% endfor %}
{% endfor %}
{% endfor %}
{% endfor %}
{% endfor %}
vehicle_motors: "{{ vehicle_motors_str|from_yaml }}"
gives
vehicle_motors:
Ford:
diesel_premium:
name: Premium diesel
specs:
gas_station:
- date: '2000'
id: 4
law:
- date: '2005'
id: 3
diesel_regular:
name: Regular diesel
specs:
gas_station:
- date: '2002'
id: 2
law:
- date: '2008'
id: 1
gasoline_natural:
name: Natural gasoline
specs:
gas_station:
- date: '2000'
id: 6
law:
- date: '2005'
id: 5
VW:
hybrid_hydrogen:
name: Hybrid hydrogen
specs:
gas_station:
- date: '2000'
id: 12
law:
- date: '2005'
id: 11
Use this dictionary to find items where the date is less than '2001'
year: 2001
vehicle_motors_year_str: |
{{ year }}:
{% for brand,fuels in vehicle_motors.items() %}
{% for fuel,types in fuels.items() %}
{% for k,v in types.specs.items() %}
{% for i in v %}
{% if i.date|int < year|int %}
- name: {{ types.name }}
id: {{ i.id }}
brand: {{ brand }}
type: {{ k }}
{% endif %}
{% endfor %}
{% endfor %}
{% endfor %}
{% endfor %}
vehicle_motors_year: "{{ vehicle_motors_year_str|from_yaml }}"
gives
vehicle_motors_year:
2001:
- brand: Ford
id: 4
name: Premium diesel
type: gas_station
- brand: Ford
id: 6
name: Natural gasoline
type: gas_station
- brand: VW
id: 12
name: Hybrid hydrogen
type: gas_station
Example of a complete playbook for testing
- hosts: localhost
vars:
cars:
- Ford:
- vehicle:
motor:
- diesel:
- premium:
name: Premium diesel
specs:
gas_station:
- date: '2000'
id: 4
law:
- date: '2005'
id: 3
regular:
name: Regular diesel
specs:
gas_station:
- date: '2002'
id: 2
law:
- date: '2008'
id: 1
- gasoline:
- natural:
name: Natural gasoline
specs:
gas_station:
- date: '2000'
id: 6
law:
- date: '2005'
id: 5
- VW:
- vehicle:
motor:
- hybrid:
- hydrogen:
name: Hybrid hydrogen
specs:
gas_station:
- date: '2000'
id: 12
law:
- date: '2005'
id: 11
brands: "{{ cars|map('first') }}"
motors: "{{ cars|json_query(query_motors) }}"
query_motors: '[].*[][].vehicle[].motor'
vehicle_motors_str: |
{% for motor in motors %}
{{ brands[loop.index0] }}:
{% for fuels in motor %}
{% for fuel,types in fuels.items() %}
{% for type in types %}
{% for k,v in type.items() %}
{{ fuel }}_{{ k }}:
{{ v }}
{% endfor %}
{% endfor %}
{% endfor %}
{% endfor %}
{% endfor %}
vehicle_motors: "{{ vehicle_motors_str|from_yaml }}"
year: 2001
vehicle_motors_year_str: |
{{ year }}:
{% for brand,fuels in vehicle_motors.items() %}
{% for fuel,types in fuels.items() %}
{% for k,v in types.specs.items() %}
{% for i in v %}
{% if i.date|int < year|int %}
- name: {{ types.name }}
id: {{ i.id }}
brand: {{ brand }}
type: {{ k }}
{% endif %}
{% endfor %}
{% endfor %}
{% endfor %}
{% endfor %}
vehicle_motors_year: "{{ vehicle_motors_year_str|from_yaml }}"
tasks:
- debug:
var: brands
- debug:
var: motors
- debug:
var: vehicle_motors
- debug:
var: vehicle_motors_year
If you get rid of the redundant lists
cars:
Ford:
vehicle:
motor:
diesel:
premium:
name: Premium diesel
specs:
gas_station:
- {date: 2000, id: 4}
law:
- {date: 2005, id: 3}
regular:
name: Regular diesel
specs:
gas_station:
- {date: 2002, id: 2}
law:
- {date: 2008, id: 1}
gasoline:
natural:
name: Natural gasoline
specs:
gas_station:
- {date: 2000, id: 6}
law:
- {date: 2005, id: 5}
VW:
vehicle:
motor:
hybrid:
hydrogen:
name: Hybrid hydrogen
specs:
gas_station:
- {date: 2000, id: 12}
law:
- {date: 2005, id: 11}
Create the dictionary brands_motors
brands_motors: "{{ dict(cars.keys()|list|
zip(cars|
json_query('*.*.*.*.*')|
flatten(levels=2)|
map('flatten'))) }}"
gives
brands_motors:
Ford:
- name: Premium diesel
specs:
gas_station:
- date: 2000
id: 4
law:
- date: 2005
id: 3
- name: Regular diesel
specs:
gas_station:
- date: 2002
id: 2
law:
- date: 2008
id: 1
- name: Natural gasoline
specs:
gas_station:
- date: 2000
id: 6
law:
- date: 2005
id: 5
VW:
- name: Hybrid hydrogen
specs:
gas_station:
- date: 2000
id: 12
law:
- date: 2005
id: 11
Create a list of the motors
motors_str: |
{% for brand,motors in brands_motors.items() %}
{% for motor in motors %}
{% for spec,types in motor.specs.items() %}
{% for type in types %}
- brand: {{ brand }}
name: {{ motor.name }}
spec: {{ spec }}
date: {{ type.date }}
id: {{ type.id }}
{% endfor %}
{% endfor %}
{% endfor %}
{% endfor %}
motors: "{{ motors_str|from_yaml }}"
gives
motors:
- {brand: Ford, date: 2000, id: 4, name: Premium diesel, spec: gas_station}
- {brand: Ford, date: 2005, id: 3, name: Premium diesel, spec: law}
- {brand: Ford, date: 2002, id: 2, name: Regular diesel, spec: gas_station}
- {brand: Ford, date: 2008, id: 1, name: Regular diesel, spec: law}
- {brand: Ford, date: 2000, id: 6, name: Natural gasoline, spec: gas_station}
- {brand: Ford, date: 2005, id: 5, name: Natural gasoline, spec: law}
- {brand: VW, date: 2000, id: 12, name: Hybrid hydrogen, spec: gas_station}
- {brand: VW, date: 2005, id: 11, name: Hybrid hydrogen, spec: law}
The selection is trivial
year: 2001
result: "{{ motors|
selectattr('date', 'lt', year)|
json_query('[].[name, id]')|
map('join', ' & ') }}"
gives the expected result
result:
- Premium diesel & 4
- Natural gasoline & 6
- Hybrid hydrogen & 12