How do I iterate over CIDR blocks in JQ? - json

Question

Basically I'm trying to iterate through my AWS security groups to find any CIDR's using 0.0.0.0/0.
Here is my example JSON file:
{
"SecurityGroups": [
{
"IpPermissionsEgress": [],
"Description": "AWS OpsWorks load balancer - do not change or delete",
"IpPermissions": [
{
"PrefixListIds": [],
"FromPort": 22,
"IpRanges": [
{
"CidrIp": "0.0.0.0/0"
}
],
"ToPort": 22,
"IpProtocol": "tcp",
"UserIdGroupPairs": [],
"Ipv6Ranges": []
},
{
"PrefixListIds": [],
"FromPort": 80,
"IpRanges": [
{
"CidrIp": "0.0.0.0/0"
}
],
"ToPort": 80,
"IpProtocol": "tcp",
"UserIdGroupPairs": [],
"Ipv6Ranges": []
},
{
"PrefixListIds": [],
"FromPort": 443,
"IpRanges": [
{
"CidrIp": "0.0.0.0/0"
}
],
"ToPort": 443,
"IpProtocol": "tcp",
"UserIdGroupPairs": [],
"Ipv6Ranges": []
}
],
"GroupName": "AWS-OpsWorks-LB-Server",
"OwnerId": "056146032236",
"GroupId": "sg-7dd13739"
},
{
"IpPermissionsEgress": [
{
"IpProtocol": "-1",
"PrefixListIds": [],
"IpRanges": [
{
"CidrIp": "0.0.0.0/0"
}
],
"UserIdGroupPairs": [],
"Ipv6Ranges": []
}
],
"Description": "SG for bastion hosts",
"Tags": [
{
"Value": "bastion-host-sg",
"Key": "Name"
}
],
"IpPermissions": [
{
"PrefixListIds": [],
"FromPort": 80,
"IpRanges": [
{
"CidrIp": "0.0.0.0/0"
}
],
"ToPort": 80,
"IpProtocol": "tcp",
"UserIdGroupPairs": [],
"Ipv6Ranges": []
},
{
"PrefixListIds": [],
"FromPort": 1991,
"IpRanges": [
{
"CidrIp": "0.0.0.0/0"
}
],
"ToPort": 1991,
"IpProtocol": "tcp",
"UserIdGroupPairs": [],
"Ipv6Ranges": []
},
{
"PrefixListIds": [],
"FromPort": 8080,
"IpRanges": [
{
"CidrIp": "0.0.0.0/0"
}
],
"ToPort": 8080,
"IpProtocol": "tcp",
"UserIdGroupPairs": [],
"Ipv6Ranges": []
},
{
"PrefixListIds": [],
"FromPort": 1194,
"IpRanges": [
{
"CidrIp": "0.0.0.0/0"
}
],
"ToPort": 1194,
"IpProtocol": "udp",
"UserIdGroupPairs": [],
"Ipv6Ranges": []
},
{
"PrefixListIds": [],
"FromPort": 22,
"IpRanges": [
{
"CidrIp": "0.0.0.0/0"
}
],
"ToPort": 22,
"IpProtocol": "tcp",
"UserIdGroupPairs": [],
"Ipv6Ranges": [
{
"CidrIpv6": "::/0"
}
]
},
{
"PrefixListIds": [],
"FromPort": 30,
"IpRanges": [
{
"CidrIp": "0.0.0.0/0"
}
],
"ToPort": -1,
"IpProtocol": "icmp",
"UserIdGroupPairs": [],
"Ipv6Ranges": []
},
{
"PrefixListIds": [],
"FromPort": 1194,
"IpRanges": [
{
"CidrIp": "0.0.0.0/0"
}
],
"ToPort": 1194,
"IpProtocol": "tcp",
"UserIdGroupPairs": [],
"Ipv6Ranges": []
},
{
"PrefixListIds": [],
"FromPort": 53,
"IpRanges": [
{
"CidrIp": "0.0.0.0/0"
}
],
"ToPort": 53,
"IpProtocol": "udp",
"UserIdGroupPairs": [],
"Ipv6Ranges": []
},
{
"PrefixListIds": [],
"FromPort": 53,
"IpRanges": [
{
"CidrIp": "0.0.0.0/0"
}
],
"ToPort": 53,
"IpProtocol": "tcp",
"UserIdGroupPairs": [],
"Ipv6Ranges": []
},
{
"PrefixListIds": [],
"FromPort": 443,
"IpRanges": [
{
"CidrIp": "0.0.0.0/0"
}
],
"ToPort": 443,
"IpProtocol": "tcp",
"UserIdGroupPairs": [],
"Ipv6Ranges": []
},
{
"PrefixListIds": [],
"FromPort": 8,
"IpRanges": [
{
"CidrIp": "0.0.0.0/0"
}
],
"ToPort": -1,
"IpProtocol": "icmp",
"UserIdGroupPairs": [],
"Ipv6Ranges": []
}
],
"GroupName": "bastion-host-sg",
"VpcId": "vpc-effd0e8a",
"OwnerId": "056146032236",
"GroupId": "sg-0f60196a"
}
]
}
Here is the command I'm trying to run, but get an error:
$ cat sg-small.json | jq '.SecurityGroups[].IpPermissions[].IpRanges[] | map(select(any(.CidrIp == "0.0.0.0/0")))'
jq: error (at <stdin>:227): Cannot iterate over string ("0.0.0.0/0")
Any thoughts as to why this is happening. That method seems to work for other things that aren't IP addresses.

Answer 1

Your expression:
.SecurityGroups[].IpPermissions[].IpRanges[]
is just a stream of CidrIp objects, so that's not what you want.
The following will select the SecurityGroups that meet the criterion:
.SecurityGroups[]
| select(any( .IpPermissions[].IpRanges[]; .CidrIp == "0.0.0.0/0"))
Whether this is precisely what you want is unclear since you haven't specified that. Please see http://stackoverflow.com/help/mcve

Answer 2

The reason you're seeing the error
Cannot iterate over string ("0.0.0.0/0")
is because with your data the first portion of your filter
.SecurityGroups[].IpPermissions[].IpRanges[]
generates a sequence of objects
{
"CidrIp": "0.0.0.0/0"
}
...
The map portion of your filter iterates over the values in each of these objects, passing
"0.0.0.0/0"
to the expression select(any(.CidrIp == "0.0.0.0/0")) where any tries to iterate over all the values within the string and fails with the error you observe.
If you only want to see a sequence of {"CidrIp":...} objects as above you can eliminate the map and any:
.SecurityGroups[].IpPermissions[].IpRanges[]
| select(.CidrIp == "0.0.0.0/0")
If you would rather collect those objects into an array you could remove the any and move some of the iteration into the map e.g.
.SecurityGroups
| map(.IpPermissions[].IpRanges[] | select(.CidrIp == "0.0.0.0/0"))
producing
[
{
"CidrIp": "0.0.0.0/0"
},
....
The explict construction of the result array is easier to see if you replace map with its definition. Since map(f) is defined as [ .[] | f ] the above filter is the same as this one:
.SecurityGroups
| [ .[] | .IpPermissions[].IpRanges[] | select(.CidrIp == "0.0.0.0/0") ]