Question: I would like to know how I can iterate over an object to print out the name of each subnet and its cidr range in the subnets object.
I know how I can print out the vnet name and cidr range:
jq '.[] | select(.vnet) | {name: .vnet.name, cidr: .vnet.address_space[0]}'
I have looked online and found that you can convert the object to an array, but I am unsure how I can look through that array to print out the values:
jq '.[] | to_entries | map_values(.value)'
Essentially the desired OUTPUT should look something like this:
{
"name": "asdf1",
"cidr": "10....."
},
{
"name": "asdf2",
"cidr": "10....."
}
//...and so forth
Sample INPUT:
{
"route_tables": {
"asdf": {
"disable_bgp_route_propagation": true,
"name": "az_afw",
"resource_group_name": "vnet-spoke",
"route_entries": {
"re1": {
"name": "rt-rfc-10-8",
"next_hop_in_ip_address": "10.0.0.0",
"next_hop_type": "VirtualAppliance",
"prefix": "10.0.0.0/8"
},
"re2": {
"name": "rt-rfc-172-12",
"next_hop_in_ip_address": "10.0.0.0",
"next_hop_type": "VirtualAppliance",
"prefix": "172.16.0.0/12"
}
}
}
},
"vnet_peering_settings": {
"peer1": {
"peer_to_source": {
"allow_forwarded_traffic": true,
"allow_gateway_transit": false,
"allow_virtual_network_access": true,
"use_remote_gateways": true
},
"source_to_peer": {
"allow_forwarded_traffic": true,
"allow_gateway_transit": true,
"allow_virtual_network_access": true,
"use_remote_gateways": false
}
}
},
"vnet_spoke_object": {
"specialsubnets": {},
"subnets": {
"objectAsdf1": {
"cidr": "10.0.0.1/24",
"enforce_private_link_endpoint_network_policies": false,
"enforce_private_link_service_network_policies": false,
"name": "asdf1",
"nsg_creation": true,
"nsg_inbound": [],
"nsg_outbound": [],
"route": null,
"service_endpoints": []
},
"objectAsdf2": {
"cidr": "10.0.0.1/24",
"enforce_private_link_endpoint_network_policies": false,
"enforce_private_link_service_network_policies": false,
"name": "asdf2",
"nsg_creation": true,
"nsg_inbound": [],
"nsg_outbound": [],
"route": "asdf",
"service_endpoints": [
"Microsoft.EventHub"
]
},
"objectAsdf3": {
"cidr": "10.0.0.1/24",
"enforce_private_link_endpoint_network_policies": false,
"enforce_private_link_service_network_policies": false,
"name": "asdf3",
"nsg_creation": true,
"nsg_inbound": [],
"nsg_outbound": [],
"route": "asdf",
"service_endpoints": []
},
"objectAsdf4": {
"cidr": "10.0.0.1/24",
"enforce_private_link_endpoint_network_policies": false,
"enforce_private_link_service_network_policies": false,
"name": "asdf4",
"nsg_creation": true,
"nsg_inbound": [],
"nsg_outbound": [],
"route": "asdf",
"service_endpoints": []
}
},
"vnet": {
"address_space": [
"10.0.0.0/16"
],
"ddos_id": "placeholder",
"dns": [
"10.0.0.1",
"10.0.0.1"
],
"enable_ddos_std": false,
"name": "asdf"
}
}
}
Start with the root path and traverse down and use to_entries to get past the varying key names and use the .value field only
jq '.vnet_spoke_object.subnets | to_entries[].value | { name, cidr }'
or collect it as an array of objects
jq '.vnet_spoke_object.subnets | to_entries | map(.value | { name, cidr })'
jqplay - Demo
Related
my input json is given below , i want select if "Name": "bot-block" then update it's "Action": to "Block": {} from "Allow": {},i performed this this using select command but it filters my json and only returns the item with .Name=bot-block,i want updation in json not filtering .this is my current command jq '.[] | select(.Name=="bot-block") | .Action |= . + { "Block" : {} } ' input.json
[
{
"Name": "searchblock",
"Priority": 3,
"Action": {
"Block": {}
},
"VisibilityConfig": {
"SampledRequestsEnabled": true,
"CloudWatchMetricsEnabled": true,
"MetricName": "searchblock"
}
},
{
"Name": "bot-block",
"Priority": 4,
"Action": {
"Allow": {}
},
"VisibilityConfig": {
"SampledRequestsEnabled": true,
"CloudWatchMetricsEnabled": true,
"MetricName": "user-agent"
}
}
]
expected output
[
{
"Name": "searchblock",
"Priority": 3,
"Action": {
"Block": {}
},
"VisibilityConfig": {
"SampledRequestsEnabled": true,
"CloudWatchMetricsEnabled": true,
"MetricName": "searchblock"
}
},
{
"Name": "bot-block",
"Priority": 4,
"Action": {
"Block": {}
},
"VisibilityConfig": {
"SampledRequestsEnabled": true,
"CloudWatchMetricsEnabled": true,
"MetricName": "user-agent"
}
}
]
You can directly "assign" to the Action element of the selected array item. |= only affects the selected item, but the output of the filter is still the entire input.
jq 'map(select(.Name=="bot-block").Action |= {Block: {}})' input.json
Super close! It's just a precedence issue.
.[] | select(.Name=="bot-block") | .Action |= . + { "Block" : {} }
should be
( .[] | select(.Name=="bot-block") | .Action ) |= . + { "Block" : {} }
because |= has lower precedence than |.
jqplay
Think of foo | bar |= baz has modifying foo | bar but returning foo. As such, the original snippet returns the modified .[] | select(.Name=="bot-block"), while the fixed version returns the modified ..
I'm re-shaping a JSON file which base content is the output of an AWS API call. From the raw output, I'm currently extracting the fields that I'm interested in the format I want except for a small detail that motivated this question.
Specifically, my input comes from the output of aws rds describe-db-instances command and contains the info of multiple RDS instances. Taking an example with 2 instances, this is how it looks:
{
"DBInstances": [
{
"DBInstanceIdentifier": "db1-name",
"DBInstanceClass": "db.m5.xlarge",
"Engine": "oracle-ee",
"DBInstanceStatus": "available",
"MasterUsername": "user",
"DBName": "RANDOM",
"Endpoint": {
"Address": "some-endpoint.rds.amazonaws.com",
"Port": 5698,
"HostedZoneId": "GHDSFHFSDHSDH"
},
"AllocatedStorage": 4000,
"InstanceCreateTime": "2018-07-23T23:21:42.361000+00:00",
"PreferredBackupWindow": "09:30-07:00",
"BackupRetentionPeriod": 14,
"DBSecurityGroups": [],
"VpcSecurityGroups": [
{
"VpcSecurityGroupId": "sg-xxxxxxxxxxxxxxxxx",
"Status": "active"
},
{
"VpcSecurityGroupId": "sg-xxxxxxxxxxxxxxxxx",
"Status": "active"
}
],
"DBParameterGroups": [
{
"DBParameterGroupName": "DB1-parameter",
"ParameterApplyStatus": "in-sync"
}
],
"AvailabilityZone": "ZONE1",
"DBSubnetGroup": {
"DBSubnetGroupName": "dbsubnetgroup-1",
"DBSubnetGroupDescription": "dbsubnetgroup-1",
"VpcId": "vpc-xxxxxxxxxxxxxxxxx",
"SubnetGroupStatus": "Complete",
"Subnets": [
{
"SubnetIdentifier": "subnet-xxxxxxxxxxxxxxxxx",
"SubnetAvailabilityZone": {
"Name": "az1"
},
"SubnetStatus": "Active"
},
{
"SubnetIdentifier": "subnet-xxxxxxxxxxxxxxxxx",
"SubnetAvailabilityZone": {
"Name": "az2"
},
"SubnetStatus": "Active"
}
]
},
"PreferredMaintenanceWindow": "sat:07:00-sat:07:30",
"PendingModifiedValues": {},
"LatestRestorableTime": "2020-03-27T18:54:25+00:00",
"MultiAZ": false,
"EngineVersion": "X.X.X",
"AutoMinorVersionUpgrade": false,
"ReadReplicaDBInstanceIdentifiers": [],
"LicenseModel": "bring-your-own-license",
"Iops": 5000,
"OptionGroupMemberships": [
{
"OptionGroupName": "optiongroupName",
"Status": "in-sync"
}
],
"CharacterSetName": "WE8ISO8859P15",
"PubliclyAccessible": false,
"StorageType": "io1",
"DbInstancePort": 0,
"StorageEncrypted": true,
"KmsKeyId": "someKey",
"DbiResourceId": "db-xxxxxxxxxxxxxxxxxxxxxxxxx",
"CACertificateIdentifier": "rds-ca-2019",
"DomainMemberships": [],
"CopyTagsToSnapshot": true,
"MonitoringInterval": 0,
"DBInstanceArn": "someARN",
"IAMDatabaseAuthenticationEnabled": false,
"PerformanceInsightsEnabled": false,
"DeletionProtection": false,
"AssociatedRoles": []
},
{
"DBInstanceIdentifier": "db2-name",
"DBInstanceClass": "db.m5.large",
"Engine": "oracle-ee",
"DBInstanceStatus": "available",
"MasterUsername": "user2",
"DBName": "XXXX",
"Endpoint": {
"Address": "endpoint2.rds.amazonaws.com",
"Port": 8974,
"HostedZoneId": "FASDFDS54FSA"
},
"AllocatedStorage": 100,
"InstanceCreateTime": "2020-04-23T21:38:53.023000+00:00",
"PreferredBackupWindow": "01:00-05:30",
"BackupRetentionPeriod": 35,
"DBSecurityGroups": [],
"VpcSecurityGroups": [
{
"VpcSecurityGroupId": "sg-xxxxxxxxxxxxxxxxx",
"Status": "active"
}
],
"DBParameterGroups": [
{
"DBParameterGroupName": "default",
"ParameterApplyStatus": "in-sync"
}
],
"AvailabilityZone": "AZ-2",
"DBSubnetGroup": {
"DBSubnetGroupName": "subnet-group",
"DBSubnetGroupDescription": "",
"VpcId": "vpc-xxxxxxxxxxxxxxxxx",
"SubnetGroupStatus": "Complete",
"Subnets": [
{
"SubnetIdentifier": "subnet-xxxxxxxxxxxxxxxxx",
"SubnetAvailabilityZone": {
"Name": "AZ-1"
},
"SubnetStatus": "Active"
},
{
"SubnetIdentifier": "subnet-xxxxxxxxxxxxxxxxx",
"SubnetAvailabilityZone": {
"Name": "AZ-2"
},
"SubnetStatus": "Active"
},
{
"SubnetIdentifier": "subnet-xxxxxxxxxxxxxxxxx",
"SubnetAvailabilityZone": {
"Name": "AZ-3"
},
"SubnetStatus": "Active"
}
]
},
"PreferredMaintenanceWindow": "sun:08:39-sun:09:09",
"PendingModifiedValues": {},
"LatestRestorableTime": "2020-07-27T18:53:18+00:00",
"MultiAZ": false,
"EngineVersion": "X.X.X",
"AutoMinorVersionUpgrade": false,
"ReadReplicaDBInstanceIdentifiers": [],
"LicenseModel": "bring-your-own-license",
"Iops": 2000,
"OptionGroupMemberships": [
{
"OptionGroupName": "optiongroup-name",
"Status": "in-sync"
}
],
"CharacterSetName": "AL32UTF8",
"PubliclyAccessible": false,
"StorageType": "io1",
"DbInstancePort": 0,
"StorageEncrypted": true,
"KmsKeyId": "someARN",
"DbiResourceId": "db-xxxxxxxxxxxxxxxxx",
"CACertificateIdentifier": "rds-ca-2019",
"DomainMemberships": [],
"CopyTagsToSnapshot": false,
"MonitoringInterval": 0,
"DBInstanceArn": "someARN",
"IAMDatabaseAuthenticationEnabled": false,
"PerformanceInsightsEnabled": false,
"DeletionProtection": false,
"AssociatedRoles": []
}
]
}
This is my current output:
[
{
"DBInstancePrefix": {
"DBInstanceClass": "db.m5.xlarge",
"DBInstanceIdentifier": "db1-name",
"DBName": "RANDOM",
"DBParameterGroupName": "DB1-parameter",
"DBSubnetGroupName": "dbsubnetgroup-1",
"KmsKeyId": "someKey",
"OptionGroupName": "optiongroupName",
"VpcSecurityGroupIds": [
"sg-xxxxxxxxxxxxxxxxx",
"sg-xxxxxxxxxxxxxxxxx"
]
}
},
{
"DBInstancePrefix": {
"DBInstanceClass": "db.m5.large",
"DBInstanceIdentifier": "db2-name",
"DBName": "XXXX",
"DBParameterGroupName": "default",
"DBSubnetGroupName": "subnet-group",
"KmsKeyId": "someARN",
"OptionGroupName": "optiongroup-name",
"VpcSecurityGroupIds": [
"sg-xxxxxxxxxxxxxxxxx"
]
}
}
]
This is my current JQ filter:
. | [.[] | map(.) | .[] | {DBInstancePrefix: {DBInstanceClass: .DBInstanceClass, DBInstanceIdentifier: .DBInstanceIdentifier, DBName: .DBName, DBParameterGroupName:.DBParameterGroups[].DBParameterGroupName, DBSubnetGroupName: .DBSubnetGroup.DBSubnetGroupName, KmsKeyId:.KmsKeyId, OptionGroupName: .OptionGroupMemberships[].OptionGroupName, VpcSecurityGroupIds: [.VpcSecurityGroups | .[] | .VpcSecurityGroupId] }}]
You can verify it on this snippet on jqplay.org.
What I need is to turn the parent key "DBInstancePrefix" dynamic based on a substring from "DBInstanceIdentifier" key. So for the example names I wrote would be:
"db1-name" >>> "db1"
"db2-name" >>> "db2"
So, my desired output would be:
[
{
"db1": {
"DBInstanceClass": "db.m5.xlarge",
"DBInstanceIdentifier": "db1-name",
"DBName": "RANDOM",
"DBParameterGroupName": "DB1-parameter",
"DBSubnetGroupName": "dbsubnetgroup-1",
"KmsKeyId": "someKey",
"OptionGroupName": "optiongroupName",
"VpcSecurityGroupIds": [
"sg-xxxxxxxxxxxxxxxxx",
"sg-xxxxxxxxxxxxxxxxx"
]
}
},
{
"db2": {
"DBInstanceClass": "db.m5.large",
"DBInstanceIdentifier": "db2-name",
"DBName": "XXXX",
"DBParameterGroupName": "default",
"DBSubnetGroupName": "subnet-group",
"KmsKeyId": "someARN",
"OptionGroupName": "optiongroup-name",
"VpcSecurityGroupIds": [
"sg-xxxxxxxxxxxxxxxxx"
]
}
}
]
Any ideas or suggestions? Thanks for reading.
To manipulate the object key-name as you desire, you can apply the filter operation inside (..). Any operation done inside it, the result is preserved "literally".
Your case demands the DBInstanceIdentifier to be split by - and using the first element in the resultant array.
With that and few other trivial modifications, you need
.DBInstances |
map
(
{
( .DBInstanceIdentifier | split("-")[0] ): {
DBInstanceClass,
DBInstanceIdentifier,
DBName,
DBParameterGroupName:.DBParameterGroups[].DBParameterGroupName,
DBSubnetGroupName: .DBSubnetGroup.DBSubnetGroupName,
KmsKeyId,
OptionGroupName: .OptionGroupMemberships[].OptionGroupName,
VpcSecurityGroupIds: [.VpcSecurityGroups[] | .VpcSecurityGroupId ]
}
}
)
jqplay - Demo
I took the long path and since I couldn't resolve it on the same filter I did another one that takes the output of the first one and thus achieved the desired result. But it is awful compared with the answer selected as the solution.
Based on Inian answer, I did a small modification since my real DBInstanceIdentifier values have the following format:
<name>-db-<environment> and I need DBInstancePrefix becomes <name>-db.
So, my final filter is:
.DBInstances |
map
(
{
( .DBInstanceIdentifier|split("-")[0:2] | join("-") ): {
DBInstanceClass,
DBInstanceIdentifier,
DBName,
DBParameterGroupName:.DBParameterGroups[].DBParameterGroupName,
DBSubnetGroupName: .DBSubnetGroup.DBSubnetGroupName,
KmsKeyId,
OptionGroupName: .OptionGroupMemberships[].OptionGroupName,
VpcSecurityGroupIds: [.VpcSecurityGroups[] | .VpcSecurityGroupId ]
}
}
)
I have a JSON like this (sample.json):
{
"sheet1": [
{
"hostname": "sv001",
"role": "web",
"ip1": "172.17.0.3"
},
{
"hostname": "sv002",
"role": "web",
"ip1": "172.17.0.4"
},
{
"hostname": "sv003",
"role": "db",
"ip1": "172.17.0.5",
"ip2": "172.18.0.5"
}
],
"sheet2": [
{
"hostname": "sv004",
"role": "web",
"ip1": "172.17.0.6"
},
{
"hostname": "sv005",
"role": "db",
"ip1": "172.17.0.7"
},
{
"hostname": "vsv006",
"role": "db",
"ip1": "172.17.0.8"
}
],
"sheet3": []
}
I want to extract data like this:
sheet1
jq '(something command)' sample.json
{
"web": {
"hosts": [
"172.17.0.3",
"172.17.0.4"
]
},
"db": {
"hosts": [
"172.17.0.5"
]
}
}
Is it possible to perform the reconstruction with jq map?
(I will reuse the result for ansible inventory.)
Here's a short, straight-forward and efficient solution -- efficient in part because it avoids group_by by courtesy of the following generic helper function:
def add_by(f;g): reduce .[] as $x ({}; .[$x|f] += [$x|g]);
.sheet1
| add_by(.role; .ip1)
| map_values( {hosts: .} )
Output
This produces the required output:
{
"web": {
"hosts": [
"172.17.0.3",
"172.17.0.4"
]
},
"db": {
"hosts": [
"172.17.0.5"
]
}
}
If the goal is to regroup the ips by their roles within each sheet you could do this:
map_values(
reduce group_by(.role)[] as $g ({};
.[$g[0].role].hosts = [$g[] | del(.hostname, .role)[]]
)
)
Which produces something like this:
{
"sheet1": {
"db": {
"hosts": [
"172.17.0.5",
"172.18.0.5"
]
},
"web": {
"hosts": [
"172.17.0.3",
"172.17.0.4"
]
}
},
"sheet2": {
"db": {
"hosts": [
"172.17.0.7",
"172.17.0.8"
]
},
"web": {
"hosts": [
"172.17.0.6"
]
}
},
"sheet3": {}
}
https://jqplay.org/s/3VpRc5l4_m
If you want to flatten all to a single object keeping only unique ips, you can keep everything mostly the same, you'll just need to flatten the inputs prior to grouping and remove the map_values/1 call.
$ jq -n '
reduce ([inputs[][]] | group_by(.role)[]) as $g ({};
.[$g[0].role].hosts = ([$g[] | del(.hostname, .role)[]] | unique)
)
'
{
"db": {
"hosts": [
"172.17.0.5",
"172.17.0.7",
"172.17.0.8",
"172.18.0.5"
]
},
"web": {
"hosts": [
"172.17.0.3",
"172.17.0.4",
"172.17.0.6"
]
}
}
https://jqplay.org/s/ZGj1wC8hU3
I want to edit this json file:
{
"shipping_orders": [{
}],
"timestamp": 100,
"shoporders": [{
"buyer_remark": null
}],
"payment_channel_info": {
"shipping_method": 28,
"grouping_info": {
"groups": [{
"display_info": {
"discount": 1
}
}]
}
}
}
And I want to add these json objects to the file:
"headers": {}
"ext_ad_info_mappings": []
"enabled": true
The output I want is like this:
{
"headers": {},
"shipping_orders": [{
}],
"timestamp": 100,
"shoporders": [{
"buyer_remark": null,
"ext_ad_info_mappings": []
}],
"payment_channel_info": {
"shipping_method": 28,
"grouping_info": {
"groups": [{
"display_info": {
"discount": 1,
"enabled": true
}
}]
}
}
}
I tried this command but the result only shows true https://jqplay.org/s/0aVbFzMttK
jq '.+{"headers": {}} and .shoporders[] + {"ext_ad_info_mappings": []} and .payment_channel_info.grouping_info.groups[].display_info + {"enabled": true}'
How should I edit the json file so that it'll output like the one I want above ?
Just locate where in the tree you want to set the values and set them.
.headers = {}
| .shoporders[].ext_ad_info_mappings = []
| .payment_channel_info.grouping_info.groups[].display_info.enabled = true
https://jqplay.org/s/ugF5foO90N
I need to parse the complex JSON (below) IN SCALA to get the values of "expression" and "value" in "measure" key i.e I need List (COUNT, COUNT_DISTINCT ...) and List (1,tbl1.USER_ID ...).
I tried multiple options, but it is not working. Any help is appreciated
{
"uuid": "uuidddd",
"last_modified": 1559080222953,
"version": "2.6.1.0",
"name": "FULL_DAY_2_mand_date",
"is_draft": false,
"model_name": "FULL_DAY_1_may05",
"description": "",
"null_string": null,
"dimensions": [
{
"name": "PLATFORM",
"table": "tbl1",
"column": "PLATFORM",
"derived": null
},
{
"name": "OS_VERSION",
"table": "tbl1",
"column": "OS_VERSION",
"derived": null
}
],
"measures": [
{
"name": "_COUNT_",
"function": {
"expression": "COUNT",
"parameter": {
"type": "constant",
"value": "1"
},
"returntype": "bigint"
}
},
{
"name": "UU",
"function": {
"expression": "COUNT_DISTINCT",
"parameter": {
"type": "column",
"value": "tbl1.USER_ID"
},
"returntype": "hllc(12)"
}
},
{
"name": "CONT_SIZE",
"function": {
"expression": "SUM",
"parameter": {
"type": "column",
"value": "tbl1.SIZE"
},
"returntype": "bigint"
}
},
{
"name": "CONT_COUNT",
"function": {
"expression": "SUM",
"parameter": {
"type": "column",
"value": "tbl1.COUNT"
},
"returntype": "bigint"
}
}
],
"dictionaries": [],
"rowkey": {
"rowkey_columns": [
{
"column": "tbl1.OS_VERSION",
"encoding": "dict",
"encoding_version": 1,
"isShardBy": false
},
{
"column": "tbl1.PLATFORM",
"encoding": "dict",
"encoding_version": 1,
"isShardBy": false
},
{
"column": "tbl1.DEVICE_FAMILY",
"encoding": "dict",
"encoding_version": 1,
"isShardBy": false
}
]
},
"hbase_mapping": {
"column_family": [
{
"name": "F1",
"columns": [
{
"qualifier": "M",
"measure_refs": [
"_COUNT_",
"CONT_SIZE",
"CONT_COUNT"
]
}
]
},
{
"name": "F2",
"columns": [
{
"qualifier": "M",
"measure_refs": [
"UU"
]
}
]
}
]
},
"aggregation_groups": [
{
"includes": [
"tbl1.PLATFORM",
"tbl1.OS_VERSION"
],
"select_rule": {
"hierarchy_dims": [],
"mandatory_dims": [
"tbl1.DATE_HR"
],
"joint_dims": []
}
}
],
"signature": "ttrrs==",
"notify_list": [],
"status_need_notify": [
"ERROR",
"DISCARDED",
"SUCCEED"
],
"partition_date_start": 0,
"partition_date_end": 3153600000000,
"auto_merge_time_ranges": [
604800000,
2419200000
],
"volatile_range": 0,
"retention_range": 0,
"engine_type": 4,
"storage_type": 2,
"override_kylin_properties": {
"job.queuename": "root.production.P0",
"is-mandatory-only-valid": "true"
},
"cuboid_black_list": [],
"parent_forward": 3,
"mandatory_dimension_set_list": [],
"snapshot_table_desc_list": []
}
This is a snippet of the code I tried, and it is giving a null list
import org.json4s._
import org.json4s.jackson.JsonMethods._
implicit val formats = org.json4s.DefaultFormats
case class Function (
expression: String,
parameter: Parameter,
returntype: String
)
case class Parameter (
`type`: String,
value: String
)
case class Measures (
name: String,
function: Function
)
case class AllMeasuresData(uuid: String, measure: List[Measures])
val data = parse(tmp).extract[AllMeasuresData]
val names = data.measure.map(_.name)
println(names)
case class AllMeasuresData(uuid: String, measure: List[Measures])
val data = parse(tmp).extract[AllMeasuresData]
val names = data.measure.map(_.name)
println(names)
There are couple typos in your ADT:
Here is what you need:
case class Function (
expression: String,
parameter: Parameter,
returntype: String
)
case class Parameter (
`type`: String,
value: String
)
case class Measures (
name: String,
function: Function
)
case class AllMeasuresData(uuid: String, measures: List[Measures])
There is also an extra comma int the json, here is the correct:
{
"uuid":"uuidddd",
"last_modified":1559080222953,
"version":"2.6.1.0",
"name":"FULL_DAY_2_mand_date",
"is_draft":false,
"model_name":"FULL_DAY_1_may05",
"description":"",
"null_string":null,
"dimensions":[
{
"name":"PLATFORM",
"table":"tbl1",
"column":"PLATFORM",
"derived":null
},
{
"name":"OS_VERSION",
"table":"tbl1",
"column":"OS_VERSION",
"derived":null
} // There was an extra trailing comma here
],
"measures":[
{
"name":"_COUNT_",
"function":{
"expression":"COUNT",
"parameter":{
"type":"constant",
"value":"1"
},
"returntype":"bigint"
}
},
{
"name":"UU",
"function":{
"expression":"COUNT_DISTINCT",
"parameter":{
"type":"column",
"value":"tbl1.USER_ID"
},
"returntype":"hllc(12)"
}
},
{
"name":"CONT_SIZE",
"function":{
"expression":"SUM",
"parameter":{
"type":"column",
"value":"tbl1.SIZE"
},
"returntype":"bigint"
}
},
{
"name":"CONT_COUNT",
"function":{
"expression":"SUM",
"parameter":{
"type":"column",
"value":"tbl1.COUNT"
},
"returntype":"bigint"
}
}
],
"dictionaries":[
],
"rowkey":{
"rowkey_columns":[
{
"column":"tbl1.OS_VERSION",
"encoding":"dict",
"encoding_version":1,
"isShardBy":false
},
{
"column":"tbl1.PLATFORM",
"encoding":"dict",
"encoding_version":1,
"isShardBy":false
},
{
"column":"tbl1.DEVICE_FAMILY",
"encoding":"dict",
"encoding_version":1,
"isShardBy":false
}
]
},
"hbase_mapping":{
"column_family":[
{
"name":"F1",
"columns":[
{
"qualifier":"M",
"measure_refs":[
"_COUNT_",
"CONT_SIZE",
"CONT_COUNT"
]
}
]
},
{
"name":"F2",
"columns":[
{
"qualifier":"M",
"measure_refs":[
"UU"
]
}
]
}
]
},
"aggregation_groups":[
{
"includes":[
"tbl1.PLATFORM",
"tbl1.OS_VERSION"
],
"select_rule":{
"hierarchy_dims":[
],
"mandatory_dims":[
"tbl1.DATE_HR"
],
"joint_dims":[
]
}
}
],
"signature":"ttrrs==",
"notify_list":[
],
"status_need_notify":[
"ERROR",
"DISCARDED",
"SUCCEED"
],
"partition_date_start":0,
"partition_date_end":3153600000000,
"auto_merge_time_ranges":[
604800000,
2419200000
],
"volatile_range":0,
"retention_range":0,
"engine_type":4,
"storage_type":2,
"override_kylin_properties":{
"job.queuename":"root.production.P0",
"is-mandatory-only-valid":"true"
},
"cuboid_black_list":[
],
"parent_forward":3,
"mandatory_dimension_set_list":[
],
"snapshot_table_desc_list":[
]
}
Now you can run:
val data = parse(tmp).extract[AllMeasuresData]
val names = data.measures.map(_.name)
println(names)
// Displays
// List(_COUNT_, UU, CONT_SIZE, CONT_COUNT)
Your Parameter class does not match the JSON because you have used type1 rather than type as the field name. Use backticks to use "type" as a field name even though it is a reserved word:
case class Parameter (
`type`: String,
value: String
)
You also need to change the Function class as it has returntype1 rather than returntype:
case class Function (
expression: String,
parameter: Parameter,
returntype: String
)
The names of the fields in Scala must exactly match the names of the fields in the JSON. Extra fields in the JSON are ignored, but all the fields in the Scala must have matching fields in the JSON. If there are optional fields in the JSON then the Scala field type should be Option[...].