Error: JSONDecodeError: Expecting property name enclosed in double quotes - json

I'm trying to convert a JSON to CSV but now it is giving an error:
'JSONDecodeError: Expecting property name enclosed in double quotes'
The error starts on the line:
jsonDict = json.loads(jsonObj)
And I gave print (jsonObj).
Could someone help me I don't know what else to do.
Heading:
# versao python #3.7.3
import json
from pandas.io.json import json_normalize
def flatten_dict(d, acc={}):
for k, v in d.items():
if isinstance(v, dict):
flatten_dict(v, acc)
elif isinstance(v, list):
for l in v:
flatten_dict(l, acc)
else:
acc[k] = v
return acc
data = []
with open('testejson.json') as f:
for jsonObj in f:
jsonDict = json.loads(jsonObj)
data.append(jsonDict)
df = json_normalize(data)
df.to_csv('tmp.csv', index=False)
The JSON file and very big plus this and its structure:
{
"CVE_data_type" : "CVE",
"CVE_data_format" : "MITRE",
"CVE_data_version" : "4.0",
"CVE_data_numberOfCVEs" : "1052",
"CVE_data_timestamp" : "2021-03-16T16:00Z",
"CVE_Items" : [ {
"cve" : {
"data_type" : "CVE",
"data_format" : "MITRE",
"data_version" : "4.0",
"CVE_data_meta" : {
"ID" : "CVE-2002-0184",
"ASSIGNER" : "cve#mitre.org"
},
"problemtype" : {
"problemtype_data" : [ {
"description" : [ {
"lang" : "en",
"value" : "NVD-CWE-Other"
} ]
} ]
},
"references" : {
"reference_data" : [ {
"url" : "http://distro.conectiva.com.br/atualizacoes/?id=a&anuncio=000475",
"name" : "CLA-2002:475",
"refsource" : "CONECTIVA",
"tags" : [ ]
}, {
"url" : "http://marc.info/?l=bugtraq&m=101974610509912&w=2",
"name" : "20020425 [Global InterSec 2002041701] Sudo Password Prompt",
"refsource" : "BUGTRAQ",
"tags" : [ ]
}, {
"url" : "http://marc.info/?l=bugtraq&m=101975443619600&w=2",
"name" : "20020425 Sudo version 1.6.6 now available (fwd)",
"refsource" : "BUGTRAQ",
"tags" : [ ]
}, {
"url" : "http://marc.info/?l=bugtraq&m=101979472822196&w=2",
"name" : "20020425 [slackware-security] sudo upgrade fixes a potential vulnerability",
"refsource" : "BUGTRAQ",
"tags" : [ ]
}, {
"url" : "http://marc.info/?l=bugtraq&m=102010164413135&w=2",
"name" : "20020429 TSLSA-2002-0046 - sudo",
"refsource" : "BUGTRAQ",
"tags" : [ ]
}, {
"url" : "http://www.debian.org/security/2002/dsa-128",
"name" : "DSA-128",
"refsource" : "DEBIAN",
"tags" : [ ]
}, {
"url" : "http://www.iss.net/security_center/static/8936.php",
"name" : "sudo-password-expansion-overflow(8936)",
"refsource" : "XF",
"tags" : [ ]
}, {
"url" : "http://www.kb.cert.org/vuls/id/820083",
"name" : "VU#820083",
"refsource" : "CERT-VN",
"tags" : [ "US Government Resource" ]
}, {
"url" : "http://www.linux-mandrake.com/en/security/2002/MDKSA-2002-028.php3",
"name" : "MDKSA-2002:028",
"refsource" : "MANDRAKE",
"tags" : [ ]
}, {
"url" : "http://www.linuxsecurity.com/advisories/other_advisory-2040.html",
"name" : "ESA-20020429-010",
"refsource" : "ENGARDE",
"tags" : [ "Patch", "Vendor Advisory" ]
}, {
"url" : "http://www.novell.com/linux/security/advisories/2002_014_sudo_txt.html",
"name" : "SuSE-SA:2002:014",
"refsource" : "SUSE",
"tags" : [ ]
}, {
"url" : "http://www.redhat.com/support/errata/RHSA-2002-071.html",
"name" : "RHSA-2002:071",
"refsource" : "REDHAT",
"tags" : [ ]
}, {
"url" : "http://www.redhat.com/support/errata/RHSA-2002-072.html",
"name" : "RHSA-2002:072",
"refsource" : "REDHAT",
"tags" : [ ]
}, {
"url" : "http://www.securityfocus.com/bid/4593",
"name" : "4593",
"refsource" : "BID",
"tags" : [ ]
} ]
},
"description" : {
"description_data" : [ {
"lang" : "en",
"value" : "Sudo before 1.6.6 contains an off-by-one error that can result in a heap-based buffer overflow that may allow local users to gain root privileges via special characters in the -p (prompt) argument, which are not properly expanded."
} ]
}
},
"configurations" : {
"CVE_data_version" : "4.0",
"nodes" : [ {
"operator" : "OR",
"cpe_match" : [ {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:todd_miller:sudo:1.5.9:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:todd_miller:sudo:1.6:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:todd_miller:sudo:1.6.1:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:todd_miller:sudo:1.6.2:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:todd_miller:sudo:1.6.3:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:todd_miller:sudo:1.6.3p1:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:todd_miller:sudo:1.6.3p2:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:todd_miller:sudo:1.6.3p3:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:todd_miller:sudo:1.6.3p4:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:todd_miller:sudo:1.6.3p5:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:todd_miller:sudo:1.6.3p6:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:todd_miller:sudo:1.6.3p7:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:todd_miller:sudo:1.6.4:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:todd_miller:sudo:1.6.4p1:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:todd_miller:sudo:1.6.4p2:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:todd_miller:sudo:1.6.5:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:todd_miller:sudo:1.6.5p1:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:todd_miller:sudo:1.6.5p2:*:*:*:*:*:*:*"
} ]
} ]
},
"impact" : {
"baseMetricV2" : {
"cvssV2" : {
"version" : "2.0",
"vectorString" : "AV:L/AC:L/Au:N/C:C/I:C/A:C",
"accessVector" : "LOCAL",
"accessComplexity" : "LOW",
"authentication" : "NONE",
"confidentialityImpact" : "COMPLETE",
"integrityImpact" : "COMPLETE",
"availabilityImpact" : "COMPLETE",
"baseScore" : 7.2
},
"severity" : "HIGH",
"exploitabilityScore" : 3.9,
"impactScore" : 10.0,
"obtainAllPrivilege" : true,
"obtainUserPrivilege" : false,
"obtainOtherPrivilege" : false,
"userInteractionRequired" : false
}
},
"publishedDate" : "2002-05-16T04:00Z",
"lastModifiedDate" : "2021-03-16T14:15Z"
} ]
}

Related

One jq command to get expected output

I want to format below json code and print only specific values.
I tried using below command and got the below json output. But when I further try to parse the code for end result it didn't worked as expected.
So, a little help can make things work for me.
Command that results in below JSON code output:
mgmt_cli -r true -d 192.168.86.201 show access-rulebase name Network --format json | jq --raw-output ' .rulebase[].rulebase[] | {name: .name, rule_number: ."rule-number"}'
JSON Code:
{
"name": "MgmtRule",
"rule_number": 1
}
{
"name": null,
"rule_number": 2
}
{
"name": null,
"rule_number": 3
}
{
"name": "Inside Rules",
"rule_number": 4
}
{
"name": null,
"rule_number": 5
}
{
"name": null,
"rule_number": 6
}
{
"name": null,
"rule_number": 7
}
{
"name": null,
"rule_number": 8
}
{
"name": null,
"rule_number": 9
}
{
"name": "Cleanup rule",
"rule_number": 10
}
Further formatting jq command that I added to the above command to get the end result:
jq --slurp '"\(.[0])-\(.[length -1])"'
Unexpected Output:
"{\"name\":\"MgmtRule\",\"rule_number\":1}-{\"name\":\"Cleanup rule\",\"rule_number\":10}"
Expected Output:
MgmtRule (1-3)
Inside Rules (4-9)
Cleanup rule (10)
Raw output:
Short form of original raw output:
{
"uid" : "38271c2f-ab44-4e25-9aa4-e219cb6e12cf",
"name" : "Network",
"rulebase" : [ {
"uid" : "38acf747-85ee-4962-a3e7-d3cd678c388f",
"name" : "Internal Rules",
"type" : "access-section",
"from" : 1,
"to" : 3,
"rulebase" : [ {
"uid" : "bdd9d868-51b2-4210-8c37-f0237cd560fa",
"name" : "Test",
"type" : "access-rule",
"domain" : {
"uid" : "41e821a0-3720-11e3-aa6e-0800200c9fde",
"name" : "SMC User",
"domain-type" : "domain"
},
"rule-number" : 1,
"track" : {
"type" : "598ead32-aa42-4615-90ed-f51a5928d41d",
"per-session" : false,
"per-connection" : true,
"accounting" : false,
"enable-firewall-session" : false,
"alert" : "none"
},
"source" : [ "6ddb84da-ef72-45c8-b2c4-bb3a849a6aa9" ],
"source-negate" : false,
"destination" : [ "375f4a15-1a7d-0b48-8ffc-2e9eb4e512ae" ],
"destination-negate" : false,
"service" : [ "97aeb443-9aea-11d5-bd16-0090272ccb30", "18ec9eaa-1657-4240-ab97-5f234623336b" ],
"service-negate" : false,
"service-resource" : "",
"vpn" : [ "97aeb369-9aea-11d5-bd16-0090272ccb30" ],
"action" : "6c488338-8eec-4103-ad21-cd461ac2c472",
"action-settings" : {
"enable-identity-captive-portal" : false
},
"content" : [ "97aeb369-9aea-11d5-bd16-0090272ccb30" ],
"content-negate" : false,
"content-direction" : "any",
"time" : [ "97aeb369-9aea-11d5-bd16-0090272ccb30" ],
"custom-fields" : {
"field-1" : "",
"field-2" : "",
"field-3" : ""
},
"meta-info" : {
"lock" : "unlocked",
"validation-state" : "ok",
"last-modify-time" : {
"posix" : 1674486777583,
"iso-8601" : "2023-01-23T20:42+0530"
},
"last-modifier" : "admin",
"creation-time" : {
"posix" : 1674486722309,
"iso-8601" : "2023-01-23T20:42+0530"
},
"creator" : "admin"
},
"comments" : "",
"enabled" : true,
"install-on" : [ "6c488338-8eec-4103-ad21-cd461ac2c476" ],
"available-actions" : {
"edit" : "true",
"delete" : "true",
"clone" : "not_supported"
}
}, {
"uid" : "bb93b38a-c672-49ec-afb0-8a90e8518437",
"type" : "access-rule",
"domain" : {
"uid" : "41e821a0-3720-11e3-aa6e-0800200c9fde",
"name" : "SMC User",
"domain-type" : "domain"
},
"rule-number" : 2,
"track" : {
"type" : "598ead32-aa42-4615-90ed-f51a5928d41d",
"per-session" : false,
"per-connection" : true,
"accounting" : false,
"enable-firewall-session" : false,
"alert" : "none"
},
"source" : [ "6ddb84da-ef72-45c8-b2c4-bb3a849a6aa9" ],
"source-negate" : false,
"destination" : [ "8dbb91f3-786b-4d11-a029-45ee5b7d59cf" ],
"destination-negate" : false,
"service" : [ "97aeb369-9aea-11d5-bd16-0090272ccb30" ],
"service-negate" : false,
"service-resource" : "",
"vpn" : [ "97aeb369-9aea-11d5-bd16-0090272ccb30" ],
"action" : "6c488338-8eec-4103-ad21-cd461ac2c472",
"action-settings" : {
"enable-identity-captive-portal" : false
},
"content" : [ "97aeb369-9aea-11d5-bd16-0090272ccb30" ],
"content-negate" : false,
"content-direction" : "any",
"time" : [ "97aeb369-9aea-11d5-bd16-0090272ccb30" ],
"custom-fields" : {
"field-1" : "",
"field-2" : "",
"field-3" : ""
},
"meta-info" : {
"lock" : "unlocked",
"validation-state" : "ok",
"last-modify-time" : {
"posix" : 1674488952100,
"iso-8601" : "2023-01-23T21:19+0530"
},
"last-modifier" : "admin",
"creation-time" : {
"posix" : 1674488917458,
"iso-8601" : "2023-01-23T21:18+0530"
},
"creator" : "admin"
},
"comments" : "",
"enabled" : true,
"install-on" : [ "6c488338-8eec-4103-ad21-cd461ac2c476" ],
"available-actions" : {
"edit" : "true",
"delete" : "true",
"clone" : "not_supported"
}
}, {
"uid" : "925f210e-ce4f-4139-870e-3cd53f46d832",
"type" : "access-rule",
"domain" : {
"uid" : "41e821a0-3720-11e3-aa6e-0800200c9fde",
"name" : "SMC User",
"domain-type" : "domain"
},
"rule-number" : 3,
"track" : {
"type" : "598ead32-aa42-4615-90ed-f51a5928d41d",
"per-session" : false,
"per-connection" : true,
"accounting" : false,
"enable-firewall-session" : false,
"alert" : "none"
},
"source" : [ "6ddb84da-ef72-45c8-b2c4-bb3a849a6aa9" ],
"source-negate" : false,
"destination" : [ "97aeb369-9aea-11d5-bd16-0090272ccb30" ],
"destination-negate" : false,
"service" : [ "97aeb369-9aea-11d5-bd16-0090272ccb30" ],
"service-negate" : false,
"service-resource" : "",
"vpn" : [ "97aeb369-9aea-11d5-bd16-0090272ccb30" ],
"action" : "6c488338-8eec-4103-ad21-cd461ac2c473",
"action-settings" : { },
"content" : [ "97aeb369-9aea-11d5-bd16-0090272ccb30" ],
"content-negate" : false,
"content-direction" : "any",
"time" : [ "97aeb369-9aea-11d5-bd16-0090272ccb30" ],
"custom-fields" : {
"field-1" : "",
"field-2" : "",
"field-3" : ""
},
"meta-info" : {
"lock" : "unlocked",
"validation-state" : "ok",
"last-modify-time" : {
"posix" : 1674490197006,
"iso-8601" : "2023-01-23T21:39+0530"
},
"last-modifier" : "admin",
"creation-time" : {
"posix" : 1674490169763,
"iso-8601" : "2023-01-23T21:39+0530"
},
"creator" : "admin"
},
"comments" : "",
"enabled" : true,
"install-on" : [ "6c488338-8eec-4103-ad21-cd461ac2c476" ],
"available-actions" : {
"edit" : "true",
"delete" : "true",
"clone" : "not_supported"
}
} ]
}, {
"uid" : "da0fdbf6-360b-44a4-94a8-b01894d3b5a0",
"name" : "cleanup rule",
"type" : "access-section",
"from" : 4,
"to" : 4,
"rulebase" : [ {
"uid" : "2b922948-da96-4c9d-a654-063e0183f9ae",
"name" : "Cleanup rule",
"type" : "access-rule",
"domain" : {
"uid" : "41e821a0-3720-11e3-aa6e-0800200c9fde",
"name" : "SMC User",
"domain-type" : "domain"
},
"rule-number" : 4,
"track" : {
"type" : "598ead32-aa42-4615-90ed-f51a5928d41d",
"per-session" : false,
"per-connection" : true,
"accounting" : false,
"enable-firewall-session" : false,
"alert" : "none"
},
"source" : [ "97aeb369-9aea-11d5-bd16-0090272ccb30" ],
"source-negate" : false,
"destination" : [ "97aeb369-9aea-11d5-bd16-0090272ccb30" ],
"destination-negate" : false,
"service" : [ "97aeb369-9aea-11d5-bd16-0090272ccb30" ],
"service-negate" : false,
"service-resource" : "",
"vpn" : [ "97aeb369-9aea-11d5-bd16-0090272ccb30" ],
"action" : "6c488338-8eec-4103-ad21-cd461ac2c473",
"action-settings" : { },
"content" : [ "97aeb369-9aea-11d5-bd16-0090272ccb30" ],
"content-negate" : false,
"content-direction" : "any",
"time" : [ "97aeb369-9aea-11d5-bd16-0090272ccb30" ],
"custom-fields" : {
"field-1" : "",
"field-2" : "",
"field-3" : ""
},
"meta-info" : {
"lock" : "unlocked",
"validation-state" : "ok",
"last-modify-time" : {
"posix" : 1674486751969,
"iso-8601" : "2023-01-23T20:42+0530"
},
"last-modifier" : "admin",
"creation-time" : {
"posix" : 1668629634218,
"iso-8601" : "2022-11-17T01:43+0530"
},
"creator" : "System"
},
"comments" : "",
"enabled" : true,
"install-on" : [ "6c488338-8eec-4103-ad21-cd461ac2c476" ],
"available-actions" : {
"edit" : "true",
"delete" : "true",
"clone" : "not_supported"
}
} ]
} ],
"objects-dictionary" : [ {
"uid" : "6c488338-8eec-4103-ad21-cd461ac2c472",
"name" : "Accept",
"type" : "RulebaseAction",
"domain" : {
"uid" : "a0bbbc99-adef-4ef8-bb6d-defdefdefdef",
"name" : "Check Point Data",
"domain-type" : "data domain"
},
"icon" : "Actions/actionsAccept",
"color" : "none"
}, {
"uid" : "8dbb91f3-786b-4d11-a029-45ee5b7d59cf",
"name" : "All_Internet",
"type" : "address-range",
"domain" : {
"uid" : "41e821a0-3720-11e3-aa6e-0800200c9fde",
"name" : "SMC User",
"domain-type" : "domain"
},
"ipv4-address-first" : "0.0.0.0",
"ipv4-address-last" : "255.255.255.255",
"icon" : "Objects/ip",
"color" : "sienna"
}, {
"uid" : "97aeb369-9aea-11d5-bd16-0090272ccb30",
"name" : "Any",
"type" : "CpmiAnyObject",
"domain" : {
"uid" : "a0bbbc99-adef-4ef8-bb6d-defdefdefdef",
"name" : "Check Point Data",
"domain-type" : "data domain"
},
"icon" : "General/globalsAny",
"color" : "black"
}, {
"uid" : "6c488338-8eec-4103-ad21-cd461ac2c473",
"name" : "Drop",
"type" : "RulebaseAction",
"domain" : {
"uid" : "a0bbbc99-adef-4ef8-bb6d-defdefdefdef",
"name" : "Check Point Data",
"domain-type" : "data domain"
},
"icon" : "Actions/actionsDrop",
"color" : "none"
}, {
"uid" : "375f4a15-1a7d-0b48-8ffc-2e9eb4e512ae",
"name" : "dummy",
"type" : "simple-gateway",
"domain" : {
"uid" : "41e821a0-3720-11e3-aa6e-0800200c9fde",
"name" : "SMC User",
"domain-type" : "domain"
},
"icon" : "NetworkObjects/management",
"color" : "black"
}, {
"uid" : "6ddb84da-ef72-45c8-b2c4-bb3a849a6aa9",
"name" : "H_192.168.86.111",
"type" : "host",
"domain" : {
"uid" : "41e821a0-3720-11e3-aa6e-0800200c9fde",
"name" : "SMC User",
"domain-type" : "domain"
},
"ipv4-address" : "192.168.86.111",
"icon" : "Objects/host",
"color" : "black"
}, {
"uid" : "97aeb443-9aea-11d5-bd16-0090272ccb30",
"name" : "https",
"type" : "service-tcp",
"domain" : {
"uid" : "a0bbbc99-adef-4ef8-bb6d-defdefdefdef",
"name" : "Check Point Data",
"domain-type" : "data domain"
},
"port" : "443",
"icon" : "Protocols/HTTP",
"color" : "red"
}, {
"uid" : "598ead32-aa42-4615-90ed-f51a5928d41d",
"name" : "Log",
"type" : "Track",
"domain" : {
"uid" : "a0bbbc99-adef-4ef8-bb6d-defdefdefdef",
"name" : "Check Point Data",
"domain-type" : "data domain"
},
"icon" : "Track/tracksLog",
"color" : "none"
}, {
"uid" : "6c488338-8eec-4103-ad21-cd461ac2c476",
"name" : "Policy Targets",
"type" : "Global",
"domain" : {
"uid" : "a0bbbc99-adef-4ef8-bb6d-defdefdefdef",
"name" : "Check Point Data",
"domain-type" : "data domain"
},
"icon" : "General/globalsAny",
"color" : "none"
}, {
"uid" : "18ec9eaa-1657-4240-ab97-5f234623336b",
"name" : "ssh",
"type" : "service-tcp",
"domain" : {
"uid" : "a0bbbc99-adef-4ef8-bb6d-defdefdefdef",
"name" : "Check Point Data",
"domain-type" : "data domain"
},
"port" : "22",
"icon" : "Services/TCPService",
"color" : "red"
} ],
"from" : 1,
"to" : 4,
"total" : 4
}
Elements that I'm trying to fetch and get the end result from above raw output:
"name" & "rule-number"
Expected Output from above raw data:
Internal Rules (1-3)
cleanup rule (4)
By transforming the unprocessed, raw output generated by mgmt_cli, the solution becomes almost trivial since the JSON already contains the "to" and "from" properties.
Input (shortened to the relevant bits):
{
"uid": "38271c2f-ab44-4e25-9aa4-e219cb6e12cf",
"name": "Network",
"rulebase": [
{
"uid": "38acf747-85ee-4962-a3e7-d3cd678c388f",
"name": "Internal Rules",
"type": "access-section",
"from": 1,
"to": 3
},
{
"uid": "da0fdbf6-360b-44a4-94a8-b01894d3b5a0",
"name": "cleanup rule",
"type": "access-section",
"from": 4,
"to": 4
}
],
"from": 1,
"to": 4,
"total": 4
}
Jq transformation:
... | jq -r '.rulebase[]
| if .from == .to then "\(.name) (\(.from))"
else "\(.name) (\(.from)-\(.to))"
end'
Output:
Internal Rules (1-3)
cleanup rule (4)
Or perhaps (but I find the if/then more straightforward):
.rulebase[]
| (select(.from == .to) | "\(.name) (\(.from)-\(.to))")
// "\(.name) (\(.from)"

jq transform outer and inner array

This seems like it should be easy. I have a json file, and all I want is less of it - the structure stays the same. I am missing some ah-ha moment. What I am trying to do is get the some of the main array details and for each member, some of the inner details.
So for this:
{
"objects" : [ {
"uid" : "fc5d20ad-03d5-4684-8836-feae73d1fe",
"name" : "group1",
"type" : "group",
"members" : [ {
"uid" : "7846f4d6-72e0-4cd8-8795-1f5bc2f124",
"name" : "member1",
"type" : "host",
"ipv4-address" : "17.19.49.10",
"comments" : "",
"color" : "black",
"icon" : "Objects/host",
"tags" : [ ],
"meta-info" : {
"lock" : "unlocked",
"validation-state" : "ok",
"last-modify-time" : {
"posix" : 1626407214607,
"iso-8601" : "2021-07-16T13:46+1000"
},
"last-modifier" : "",
"creation-time" : {
"posix" : 1625632725273,
"iso-8601" : "2021-07-07T14:38+1000"
},
"creator" : "user"
},
"read-only" : false
}, {
"uid" : "a5b12fed-e32e-4151-8310-4848f16bef",
"name" : "member2",
"type" : "host",
"domain" : {
"uid" : "41e821a0-3720-11e3-aa6e-00200c9fde",
"name" : "SMC ",
"domain-type" : "domain"
},
"ipv4-address" : "12.19.50.10",
"interfaces" : [ ],
"nat-settings" : {
"auto-rule" : false
},
"comments" : "",
"color" : "black",
"icon" : "Objects/host",
"tags" : [ ],
"meta-info" : {
"lock" : "unlocked",
"validation-state" : "ok",
"last-modify-time" : {
"posix" : 1626407292367,
"iso-8601" : "2021-07-16T13:48+1000"
},
"last-modifier" : "user",
"creation-time" : {
"posix" : 1625632759552,
"iso-8601" : "2021-07-07T14:39+1000"
},
"creator" : "user"
},
"read-only" : false
} ],
"comments" : "",
"color" : "black",
"icon" : "General/group",
"tags" : [ ],
"meta-info" : {
"lock" : "unlocked",
"validation-state" : "ok",
"last-modify-time" : {
"posix" : 1626407413393,
"iso-8601" : "2021-07-16T13:50+1000"
},
"last-modifier" : "user",
"creation-time" : {
"posix" : 1625633894300,
"iso-8601" : "2021-07-07T14:58+1000"
},
"creator" : "user"
},
"read-only" : false
}, {
"uid" : "e8d4f757-98e8-4aae-bc31-772066d",
"name" : "group2",
"type" : "group",
"domain" : {
"uid" : "41e821a0-3720-11e3-aa6e-0800c9fde",
"name" : "SMC ",
"domain-type" : "domain"
},
"members" : [ {
"uid" : "84e89b9c-dbe7-441e-9859-2d0eef",
"name" : "member1",
"type" : "host",
"domain" : {
"uid" : "41e821a0-3720-11e3-aa6e-08fde",
"name" : "SMC ",
"domain-type" : "domain"
},
"ipv4-address" : "82.19.76.10",
"interfaces" : [ ],
"nat-settings" : {
"auto-rule" : false
},
"comments" : "",
"color" : "black",
"icon" : "Objects/host",
"tags" : [ ],
"meta-info" : {
"lock" : "unlocked",
"validation-state" : "ok",
"last-modify-time" : {
"posix" : 1626398340102,
"iso-8601" : "2021-07-16T11:19+1000"
},
"last-modifier" : "user",
"creation-time" : {
"posix" : 1625632518542,
"iso-8601" : "2021-07-07T14:35+1000"
},
"creator" : "user"
},
"read-only" : false
}, {
"uid" : "116f3792-8baa-4e56-888bcbd8194",
"name" : "member2",
"type" : "host",
"domain" : {
"uid" : "41e821a0-3720-e3-aa6e-0c9fde",
"name" : "member3",
"domain-type" : "domain"
},
"ipv4-address" : "84.12.48.10",
"interfaces" : [ ],
"nat-settings" : {
"auto-rule" : false
},
"comments" : "",
"color" : "black",
"icon" : "Objects/host",
"tags" : [ ],
"meta-info" : {
"lock" : "unlocked",
"validation-state" : "ok",
"last-modify-time" : {
"posix" : 1626398367303,
"iso-8601" : "2021-07-16T11:19+1000"
},
"last-modifier" : "user",
"creation-time" : {
"posix" : 1625632577299,
"iso-8601" : "2021-07-07T14:36+1000"
},
"creator" : "user"
},
"read-only" : false
} ],
"comments" : "",
"color" : "black",
"icon" : "General/group",
"tags" : [ ],
"meta-info" : {
"lock" : "unlocked",
"validation-state" : "ok",
"last-modify-time" : {
"posix" : 1626398715255,
"iso-8601" : "2021-07-16T11:25+1000"
},
"last-modifier" : "user",
"creation-time" : {
"posix" : 1625632984944,
"iso-8601" : "2021-07-07T14:43+1000"
},
"creator" : "user"
},
"read-only" : false
} ]
}
I want to end up with this:
{
[ {
"uid" : "fc5d20ad-03d5-4684-8836-feae73d1fe",
"name" : "group1",
"type" : "group",
"members" : [ {
"uid" : "7846f4d6-72e0-4cd8-8795-1f5bc2f124",
"name" : "member1",
"type" : "host",
"ipv4-address" : "17.19.49.10",
}, {
"uid" : "a5b12fed-e32e-4151-8310-4848f16bef",
"name" : "member2",
"type" : "host",
"ipv4-address" : "12.19.50.10",
} ],
}, {
"uid" : "e8d4f757-98e8-4aae-bc31-772066d",
"name" : "group2",
"type" : "group",
"members" : [ {
"uid" : "84e89b9c-dbe7-441e-9859-2d0eef",
"name" : "member1",
"type" : "host",
"ipv4-address" : "82.19.76.10",
},
}, {
"uid" : "116f3792-8baa-4e56-888bcbd8194",
"name" : "member2",
"type" : "host",
"ipv4-address" : "84.12.48.10",
} ]
} ]
}
But I can't figure out how to get the member details out. This is what I have so far:
cat groups.txt | jq '.objects[] | { name: .name, uid: .uid, type: .type, members: .members[] | { name: .name, uid: .uid, "ipv4-address": ."ipv4-address", type: .type } }'
This repeats everything for each member, so I need an "inner loop" to deal with the members and I can't see it:
{
"name": "group1",
"uid": "fc5d20ad-03d5-4684-8836-feae73d1fe",
"type": "group",
"members": {
"name": "member1",
"uid": "7846f4d6-72e0-4cd8-8795-1f5bc2f124",
"ipv4-address": "17.19.49.10",
"type": "host"
}
}
{
"name": "group1",
"uid": "fc5d20ad-03d5-4684-8836-feae73d1fe",
"type": "group",
"members": {
"name": "member2",
"uid": "a5b12fed-e32e-4151-8310-4848f16bef",
"ipv4-address": "12.19.50.10",
"type": "host"
}
}
[snip]
How do I keep them together, and keep the surrounding array?
You were really close.
.objects[] |= {uid, name, type, members: .members | map({uid, name, type, "ipv4-address"})}
Online demo

SwiftyJSON parsing data in array of objects

I am trying to get the value for "name" in the tracks section of JSON. The following JSON is from Spotify. I am using SwiftyJSON for parsing the data.
I am able to pull values from the base json. But once I try to pull anything in the "track" section I get errors.
Here is a sample of the code I got working
let swiftJSON = try? JSON(data: data)
let addedAT = swiftJSON!["items"].arrayValue.map {$0["added_at"].stringValue}
print(addedAT)
{
"items" : [
{
"video_thumbnail" : {
"url" : null
},
"track" : {
"is_playable" : true,
"type" : "track",
"id" : "3F5CgOj3wFlRv51JsHbxhe",
"track" : true,
"duration_ms" : 218364,
"explicit" : true,
"uri" : "spotify:track:3F5CgOj3wFlRv51JsHbxhe",
"is_local" : false,
"artists" : [
{
"external_urls" : {
"spotify" : "https:\/\/open.spotify.com\/artist\/3TVXtAsR1Inumwj472S9r4"
},
"uri" : "spotify:artist:3TVXtAsR1Inumwj472S9r4",
"type" : "artist",
"href" : "https:\/\/api.spotify.com\/v1\/artists\/3TVXtAsR1Inumwj472S9r4",
"name" : "Drake",
"id" : "3TVXtAsR1Inumwj472S9r4"
},
{
"external_urls" : {
"spotify" : "https:\/\/open.spotify.com\/artist\/1URnnhqYAYcrqrcwql10ft"
},
"uri" : "spotify:artist:1URnnhqYAYcrqrcwql10ft",
"type" : "artist",
"href" : "https:\/\/api.spotify.com\/v1\/artists\/1URnnhqYAYcrqrcwql10ft",
"name" : "21 Savage",
"id" : "1URnnhqYAYcrqrcwql10ft"
}
],
"album" : {
"name" : "Honestly, Nevermind",
"total_tracks" : 14,
"external_urls" : {
"spotify" : "https:\/\/open.spotify.com\/album\/3cf4iSSKd8ffTncbtKljXw"
},
"images" : [
{
"height" : 640,
"url" : "https:\/\/i.scdn.co\/image\/ab67616d0000b2738dc0d801766a5aa6a33cbe37",
"width" : 640
},
{
"height" : 300,
"url" : "https:\/\/i.scdn.co\/image\/ab67616d00001e028dc0d801766a5aa6a33cbe37",
"width" : 300
},
{
"height" : 64,
"url" : "https:\/\/i.scdn.co\/image\/ab67616d000048518dc0d801766a5aa6a33cbe37",
"width" : 64
}
],
"artists" : [
{
"external_urls" : {
"spotify" : "https:\/\/open.spotify.com\/artist\/3TVXtAsR1Inumwj472S9r4"
},
"uri" : "spotify:artist:3TVXtAsR1Inumwj472S9r4",
"type" : "artist",
"href" : "https:\/\/api.spotify.com\/v1\/artists\/3TVXtAsR1Inumwj472S9r4",
"name" : "Drake",
"id" : "3TVXtAsR1Inumwj472S9r4"
}
],
"href" : "https:\/\/api.spotify.com\/v1\/albums\/3cf4iSSKd8ffTncbtKljXw",
"release_date_precision" : "day",
"type" : "album",
"id" : "3cf4iSSKd8ffTncbtKljXw",
"album_type" : "album",
"release_date" : "2022-06-17",
"uri" : "spotify:album:3cf4iSSKd8ffTncbtKljXw"
},
"track_number" : 14,
"external_ids" : {
"isrc" : "USUG12204897"
},
"episode" : false,
"external_urls" : {
"spotify" : "https:\/\/open.spotify.com\/track\/3F5CgOj3wFlRv51JsHbxhe"
},
"popularity" : 90,
"disc_number" : 1,
"name" : "Jimmy Cooks (feat. 21 Savage)",
"preview_url" : "https:\/\/p.scdn.co\/mp3-preview\/5feee7c2fed489c933a31ef125aa4ffb5205a457?cid=263c670f8c17484d89d8bd2e6e0964f7",
"href" : "https:\/\/api.spotify.com\/v1\/tracks\/3F5CgOj3wFlRv51JsHbxhe"
},
"added_by" : {
"uri" : "spotify:user:12493212",
"external_urls" : {
"spotify" : "https:\/\/open.spotify.com\/user\/12493212"
},
"type" : "user",
"href" : "https:\/\/api.spotify.com\/v1\/users\/12493212",
"id" : "12493212"
},
"primary_color" : null,
"added_at" : "2022-06-28T16:06:35Z",
"is_local" : false
},
{
"video_thumbnail" : {
"url" : null
},
"track" : {
"is_playable" : true,
"type" : "track",
"id" : "1qMMYpVatbRITKCfq1gasi",
"track" : true,
"duration_ms" : 172933,
"explicit" : true,
"uri" : "spotify:track:1qMMYpVatbRITKCfq1gasi",
"is_local" : false,
"artists" : [
{
"href" : "https:\/\/api.spotify.com\/v1\/artists\/1RyvyyTE3xzB2ZywiAwp0i",
"uri" : "spotify:artist:1RyvyyTE3xzB2ZywiAwp0i",
"name" : "Future",
"id" : "1RyvyyTE3xzB2ZywiAwp0i",
"external_urls" : {
"spotify" : "https:\/\/open.spotify.com\/artist\/1RyvyyTE3xzB2ZywiAwp0i"
},
"type" : "artist"
}
],
"album" : {
"external_urls" : {
"spotify" : "https:\/\/open.spotify.com\/album\/6tE9Dnp2zInFij4jKssysL"
},
"href" : "https:\/\/api.spotify.com\/v1\/albums\/6tE9Dnp2zInFij4jKssysL",
"release_date" : "2022-04-29",
"release_date_precision" : "day",
"artists" : [
{
"href" : "https:\/\/api.spotify.com\/v1\/artists\/1RyvyyTE3xzB2ZywiAwp0i",
"uri" : "spotify:artist:1RyvyyTE3xzB2ZywiAwp0i",
"name" : "Future",
"id" : "1RyvyyTE3xzB2ZywiAwp0i",
"external_urls" : {
"spotify" : "https:\/\/open.spotify.com\/artist\/1RyvyyTE3xzB2ZywiAwp0i"
},
"type" : "artist"
}
],
"name" : "I NEVER LIKED YOU",
"type" : "album",
"album_type" : "album",
"id" : "6tE9Dnp2zInFij4jKssysL",
"uri" : "spotify:album:6tE9Dnp2zInFij4jKssysL",
"images" : [
{
"height" : 640,
"width" : 640,
"url" : "https:\/\/i.scdn.co\/image\/ab67616d0000b27386badd635b69aea887862214"
},
{
"height" : 300,
"width" : 300,
"url" : "https:\/\/i.scdn.co\/image\/ab67616d00001e0286badd635b69aea887862214"
},
{
"height" : 64,
"width" : 64,
"url" : "https:\/\/i.scdn.co\/image\/ab67616d0000485186badd635b69aea887862214"
}
],
"total_tracks" : 16
},
"track_number" : 5,
"external_ids" : {
"isrc" : "USSM12203787"
},
"episode" : false,
"external_urls" : {
"spotify" : "https:\/\/open.spotify.com\/track\/1qMMYpVatbRITKCfq1gasi"
},
"popularity" : 83,
"disc_number" : 1,
"name" : "PUFFIN ON ZOOTIEZ",
"preview_url" : "https:\/\/p.scdn.co\/mp3-preview\/ff3bd626fd7f0bd6af4a429b3119b06ff0aee089?cid=263c670f8c17484d89d8bd2e6e0964f7",
"href" : "https:\/\/api.spotify.com\/v1\/tracks\/1qMMYpVatbRITKCfq1gasi"
},
"added_by" : {
"external_urls" : {
"spotify" : "https:\/\/open.spotify.com\/user\/12493212"
},
"uri" : "spotify:user:12493212",
"type" : "user",
"id" : "12493212",
"href" : "https:\/\/api.spotify.com\/v1\/users\/12493212"
},
"primary_color" : null,
"added_at" : "2022-06-28T16:53:13Z",
"is_local" : false
}
],
"href" : "https:\/\/api.spotify.com\/v1\/playlists\/1nJflwAwbyhpQzKksqwC7H\/tracks?offset=0&limit=100&market=ES&locale=en-US;q=1.0",
"offset" : 0,
"previous" : null,
"limit" : 100,
"next" : null,
"total" : 2
}

Unmarshalling JSON with variable-length data and slices of other JSON objects [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 1 year ago.
Improve this question
I have a question on how Go handles unmarhsalling complex/nested JSON as it appears I have to create the entire data structure in structs prior to unmarshalling.
I am working with a JSON file that follows this format (it is the NVD database, reference here if it is helpful https://nvd.nist.gov/vuln/data-feeds):
"CVE_data_type" : "CVE",
"CVE_data_format" : "MITRE",
"CVE_data_version" : "4.0",
"CVE_data_numberOfCVEs" : "1085",
"CVE_data_timestamp" : "2021-02-24T17:00Z",
"CVE_Items" : [ {
"cve" : {
"data_type" : "CVE",
"data_format" : "MITRE",
"data_version" : "4.0",
"CVE_data_meta" : {
"ID" : "CVE-2011-0762",
"ASSIGNER" : "cve#mitre.org"
},
"problemtype" : {
"problemtype_data" : [ {
"description" : [ {
"lang" : "en",
"value" : "CWE-399"
} ]
} ]
},
"references" : {
"reference_data" : [ {
"url" : "ftp://vsftpd.beasts.org/users/cevans/untar/vsftpd-2.3.4/Changelog",
"name" : "ftp://vsftpd.beasts.org/users/cevans/untar/vsftpd-2.3.4/Changelog",
"refsource" : "CONFIRM",
"tags" : [ ]
}, {
"url" : "http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=622741",
"name" : "http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=622741",
"refsource" : "CONFIRM",
"tags" : [ ]
}, {
"url" : "http://cxib.net/stuff/vspoc232.c",
"name" : "http://cxib.net/stuff/vspoc232.c",
"refsource" : "MISC",
"tags" : [ "Exploit" ]
}, {
"url" : "http://jvn.jp/en/jp/JVN37417423/index.html",
"name" : "JVN#37417423",
"refsource" : "JVN",
"tags" : [ ]
}, {
"url" : "http://lists.fedoraproject.org/pipermail/package-announce/2011-March/055881.html",
"name" : "FEDORA-2011-2615",
"refsource" : "FEDORA",
"tags" : [ ]
}, {
"url" : "http://lists.fedoraproject.org/pipermail/package-announce/2011-March/055882.html",
"name" : "FEDORA-2011-2590",
"refsource" : "FEDORA",
"tags" : [ ]
}, {
"url" : "http://lists.fedoraproject.org/pipermail/package-announce/2011-March/055957.html",
"name" : "FEDORA-2011-2567",
"refsource" : "FEDORA",
"tags" : [ ]
}, {
"url" : "http://lists.opensuse.org/opensuse-security-announce/2011-05/msg00005.html",
"name" : "SUSE-SR:2011:009",
"refsource" : "SUSE",
"tags" : [ ]
}, {
"url" : "http://marc.info/?l=bugtraq&m=133226187115472&w=2",
"name" : "HPSBMU02752",
"refsource" : "HP",
"tags" : [ ]
}, {
"url" : "http://securityreason.com/achievement_securityalert/95",
"name" : "20110301 vsftpd 2.3.2 remote denial-of-service",
"refsource" : "SREASONRES",
"tags" : [ "Exploit" ]
}, {
"url" : "http://securityreason.com/securityalert/8109",
"name" : "8109",
"refsource" : "SREASON",
"tags" : [ ]
}, {
"url" : "http://www.debian.org/security/2011/dsa-2305",
"name" : "DSA-2305",
"refsource" : "DEBIAN",
"tags" : [ ]
}, {
"url" : "http://www.exploit-db.com/exploits/16270",
"name" : "16270",
"refsource" : "EXPLOIT-DB",
"tags" : [ ]
}, {
"url" : "http://www.kb.cert.org/vuls/id/590604",
"name" : "VU#590604",
"refsource" : "CERT-VN",
"tags" : [ "US Government Resource" ]
}, {
"url" : "http://www.mandriva.com/security/advisories?name=MDVSA-2011:049",
"name" : "MDVSA-2011:049",
"refsource" : "MANDRIVA",
"tags" : [ ]
}, {
"url" : "http://www.redhat.com/support/errata/RHSA-2011-0337.html",
"name" : "RHSA-2011:0337",
"refsource" : "REDHAT",
"tags" : [ ]
}, {
"url" : "http://www.securityfocus.com/archive/1/516748/100/0/threaded",
"name" : "20110301 vsftpd 2.3.2 remote denial-of-service",
"refsource" : "BUGTRAQ",
"tags" : [ ]
}, {
"url" : "http://www.securityfocus.com/bid/46617",
"name" : "46617",
"refsource" : "BID",
"tags" : [ "Exploit" ]
}, {
"url" : "http://www.securitytracker.com/id?1025186",
"name" : "1025186",
"refsource" : "SECTRACK",
"tags" : [ ]
}, {
"url" : "http://www.ubuntu.com/usn/USN-1098-1",
"name" : "USN-1098-1",
"refsource" : "UBUNTU",
"tags" : [ ]
}, {
"url" : "http://www.vupen.com/english/advisories/2011/0547",
"name" : "ADV-2011-0547",
"refsource" : "VUPEN",
"tags" : [ ]
}, {
"url" : "http://www.vupen.com/english/advisories/2011/0639",
"name" : "ADV-2011-0639",
"refsource" : "VUPEN",
"tags" : [ ]
}, {
"url" : "http://www.vupen.com/english/advisories/2011/0668",
"name" : "ADV-2011-0668",
"refsource" : "VUPEN",
"tags" : [ ]
}, {
"url" : "http://www.vupen.com/english/advisories/2011/0713",
"name" : "ADV-2011-0713",
"refsource" : "VUPEN",
"tags" : [ ]
}, {
"url" : "https://exchange.xforce.ibmcloud.com/vulnerabilities/65873",
"name" : "vsftpd-vsffilenamepassesfilter-dos(65873)",
"refsource" : "XF",
"tags" : [ ]
} ]
},
"description" : {
"description_data" : [ {
"lang" : "en",
"value" : "The vsf_filename_passes_filter function in ls.c in vsftpd before 2.3.3 allows remote authenticated users to cause a denial of service (CPU consumption and process slot exhaustion) via crafted glob expressions in STAT commands in multiple FTP sessions, a different vulnerability than CVE-2010-2632."
} ]
}
},
"configurations" : {
"CVE_data_version" : "4.0",
"nodes" : [ {
"operator" : "OR",
"cpe_match" : [ {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:beasts:vsftpd:0.0.1:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:beasts:vsftpd:0.0.2:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:beasts:vsftpd:0.0.3:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:beasts:vsftpd:0.0.4:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:beasts:vsftpd:0.0.5:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:beasts:vsftpd:0.0.6:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:beasts:vsftpd:0.0.7:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:beasts:vsftpd:0.0.8:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:beasts:vsftpd:0.0.9:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:beasts:vsftpd:0.0.10:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:beasts:vsftpd:0.0.11:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:beasts:vsftpd:0.0.12:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:beasts:vsftpd:0.0.13:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:beasts:vsftpd:0.0.14:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:beasts:vsftpd:0.0.15:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:beasts:vsftpd:0.9.0:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:beasts:vsftpd:0.9.1:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:beasts:vsftpd:0.9.2:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:beasts:vsftpd:0.9.3:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:beasts:vsftpd:1.0.0:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:beasts:vsftpd:1.0.1:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:beasts:vsftpd:1.1.0:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:beasts:vsftpd:1.1.1:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:beasts:vsftpd:1.1.2:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:beasts:vsftpd:1.1.3:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:beasts:vsftpd:1.2.0:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:beasts:vsftpd:1.2.1:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:beasts:vsftpd:1.2.2:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:beasts:vsftpd:2.0.0:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:beasts:vsftpd:2.0.1:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:beasts:vsftpd:2.0.2:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:beasts:vsftpd:2.0.3:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:beasts:vsftpd:2.0.4:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:beasts:vsftpd:2.0.5:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:beasts:vsftpd:2.0.6:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:beasts:vsftpd:2.0.7:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:beasts:vsftpd:2.1.0:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:beasts:vsftpd:2.1.1:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:beasts:vsftpd:2.1.2:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:beasts:vsftpd:2.2.0:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:beasts:vsftpd:2.2.1:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:beasts:vsftpd:2.2.2:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:beasts:vsftpd:2.3.0:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:beasts:vsftpd:2.3.1:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:beasts:vsftpd:*:*:*:*:*:*:*:*",
"versionEndIncluding" : "2.3.2"
} ]
} ]
},
"impact" : {
"baseMetricV2" : {
"cvssV2" : {
"version" : "2.0",
"vectorString" : "AV:N/AC:L/Au:S/C:N/I:N/A:P",
"accessVector" : "NETWORK",
"accessComplexity" : "LOW",
"authentication" : "SINGLE",
"confidentialityImpact" : "NONE",
"integrityImpact" : "NONE",
"availabilityImpact" : "PARTIAL",
"baseScore" : 4.0
},
"severity" : "MEDIUM",
"exploitabilityScore" : 8.0,
"impactScore" : 2.9,
"obtainAllPrivilege" : false,
"obtainUserPrivilege" : false,
"obtainOtherPrivilege" : false,
"userInteractionRequired" : false
}
},
"publishedDate" : "2011-03-02T20:00Z",
"lastModifiedDate" : "2021-02-19T05:15Z"
}, {
"cve" : {
"data_type" : "CVE",
"data_format" : "MITRE",
"data_version" : "4.0",
"CVE_data_meta" : {
"ID" : "CVE-2011-4362",
"ASSIGNER" : "cve#mitre.org"
},
"problemtype" : {
"problemtype_data" : [ {
"description" : [ {
"lang" : "en",
"value" : "CWE-189"
} ]
} ]
},
"references" : {
"reference_data" : [ {
"url" : "http://archives.neohapsis.com/archives/bugtraq/2011-12/0167.html",
"name" : "20111224 Lighttpd Proof of Concept code for CVE-2011-4362",
"refsource" : "BUGTRAQ",
"tags" : [ "Broken Link" ]
}, {
"url" : "http://blog.pi3.com.pl/?p=277",
"name" : "http://blog.pi3.com.pl/?p=277",
"refsource" : "MISC",
"tags" : [ "Third Party Advisory" ]
}, {
"url" : "http://download.lighttpd.net/lighttpd/security/lighttpd_sa_2011_01.txt",
"name" : "http://download.lighttpd.net/lighttpd/security/lighttpd_sa_2011_01.txt",
"refsource" : "CONFIRM",
"tags" : [ "Vendor Advisory" ]
}, {
"url" : "http://jvn.jp/en/jp/JVN37417423/index.html",
"name" : "JVN#37417423",
"refsource" : "JVN",
"tags" : [ ]
}, {
"url" : "http://redmine.lighttpd.net/issues/2370",
"name" : "http://redmine.lighttpd.net/issues/2370",
"refsource" : "CONFIRM",
"tags" : [ "Vendor Advisory" ]
}, {
"url" : "http://secunia.com/advisories/47260",
"name" : "47260",
"refsource" : "SECUNIA",
"tags" : [ "Third Party Advisory" ]
}, {
"url" : "http://www.debian.org/security/2011/dsa-2368",
"name" : "DSA-2368",
"refsource" : "DEBIAN",
"tags" : [ "Third Party Advisory" ]
}, {
"url" : "http://www.exploit-db.com/exploits/18295",
"name" : "18295",
"refsource" : "EXPLOIT-DB",
"tags" : [ "Third Party Advisory", "VDB Entry" ]
}, {
"url" : "http://www.openwall.com/lists/oss-security/2011/11/29/13",
"name" : "[oss-security] 20111129 Re: CVE Request: lighttpd/mod_auth out-of-bounds read due to signedness error",
"refsource" : "MLIST",
"tags" : [ "Mailing List", "Third Party Advisory" ]
}, {
"url" : "http://www.openwall.com/lists/oss-security/2011/11/29/8",
"name" : "[oss-security] 20111129 CVE Request: lighttpd/mod_auth out-of-bounds read due to signedness error",
"refsource" : "MLIST",
"tags" : [ "Mailing List", "Third Party Advisory" ]
}, {
"url" : "http://www.securitytracker.com/id?1026359",
"name" : "1026359",
"refsource" : "SECTRACK",
"tags" : [ "Third Party Advisory", "VDB Entry" ]
}, {
"url" : "https://bugzilla.redhat.com/show_bug.cgi?id=758624",
"name" : "https://bugzilla.redhat.com/show_bug.cgi?id=758624",
"refsource" : "CONFIRM",
"tags" : [ "Issue Tracking", "Third Party Advisory" ]
}, {
"url" : "https://exchange.xforce.ibmcloud.com/vulnerabilities/71536",
"name" : "lighttpd-base64-dos(71536)",
"refsource" : "XF",
"tags" : [ "Third Party Advisory", "VDB Entry" ]
} ]
},
"description" : {
"description_data" : [ {
"lang" : "en",
"value" : "Integer signedness error in the base64_decode function in the HTTP authentication functionality (http_auth.c) in lighttpd 1.4 before 1.4.30 and 1.5 before SVN revision 2806 allows remote attackers to cause a denial of service (segmentation fault) via crafted base64 input that triggers an out-of-bounds read with a negative index."
} ]
}
},
"configurations" : {
"CVE_data_version" : "4.0",
"nodes" : [ {
"operator" : "OR",
"cpe_match" : [ {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:lighttpd:lighttpd:*:*:*:*:*:*:*:*",
"versionStartIncluding" : "1.4.1",
"versionEndExcluding" : "1.4.30"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:a:lighttpd:lighttpd:1.5.0:*:*:*:*:*:*:*"
} ]
}, {
"operator" : "OR",
"cpe_match" : [ {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:o:debian:debian_linux:5.0:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:o:debian:debian_linux:6.0:*:*:*:*:*:*:*"
}, {
"vulnerable" : true,
"cpe23Uri" : "cpe:2.3:o:debian:debian_linux:7.0:*:*:*:*:*:*:*"
} ]
} ]
},
"impact" : {
"baseMetricV2" : {
"cvssV2" : {
"version" : "2.0",
"vectorString" : "AV:N/AC:L/Au:N/C:N/I:N/A:P",
"accessVector" : "NETWORK",
"accessComplexity" : "LOW",
"authentication" : "NONE",
"confidentialityImpact" : "NONE",
"integrityImpact" : "NONE",
"availabilityImpact" : "PARTIAL",
"baseScore" : 5.0
},
"severity" : "MEDIUM",
"exploitabilityScore" : 10.0,
"impactScore" : 2.9,
"obtainAllPrivilege" : false,
"obtainUserPrivilege" : false,
"obtainOtherPrivilege" : false,
"userInteractionRequired" : false
}
},
"publishedDate" : "2011-12-24T19:55Z",
"lastModifiedDate" : "2021-02-19T05:15Z"
},
As you can see, there is a lot of data here, and I am struggling to get this unmarshalled into something that I can work with. The first few fields are straightforward, so I have begun creating a struct as such:
type NvdData struct {
CveDataType string `json:"CVE_data_type"`
CveDataFormat string `json:"CVE_data_format"`
CveDataVersion string `json:"CVE_data_version"`
CveDataNumberCves string `json:"CVE_data_numberOfCVEs"`
}
Where I am confused is when we get to the "CVE_Items" field which contains a list/slice of "CVE" objects that then all have their own various fields, some of which are lists, some which aren't.
My question is, how do I construct my NvdData struct to handle this? Do I need to create an additional struct, CVE , which has all of the various fields associated with each CVE entry, and include that data structure as part of my NvdData definition? Something like:
type NvdData struct {
CveDataType string `json:"CVE_data_type"`
CveDataFormat string `json:"CVE_data_format"`
CveDataVersion string `json:"CVE_data_version"`
CveDataNumberCves string `json:"CVE_data_numberOfCVEs"`
CveItems []Cve{} `json:"CVE_Items`
}
type Cve struct {
DataType string `json:"data_type"`
DataFormat string `json:"data_format"`
...
}
If that makes sense? And then for any subsequent data structures that are lists/slices, I would have to describe those as a struct as well and the type for that json field would be a slice of that type?
One last one-off question I have as well is, if there are fields that I don't care about can I drop them from the type definition and they will be ignored by the Unmarshalling process, or do I need to describe them as well and just ignore them when I iterate through the data?
I saw that there is a schema online for this: https://csrc.nist.gov/schema/nvd/feed/1.1/nvd_cve_feed_json_1.1.schema
Which has this, which defines a CVE item (it references other definitions in the same file). You could in theory use this to create the various structs. There is an attempt to auto generate the structs here: https://adrianhesketh.com/2016/07/19/json-schema-to-go-struct-generator-roundup/
"def_cve_item": {
"description": "Defines a vulnerability in the NVD data feed.",
"properties": {
"cve": {"$ref": "CVE_JSON_4.0_min_1.1.schema"},
"configurations": {"$ref": "#/definitions/def_configurations"},
"impact": {"$ref": "#/definitions/def_impact"},
"publishedDate": {"type": "string"},
"lastModifiedDate": {"type": "string"}
},
"required": ["cve"]
}
I think the simplest (albeit not the most robust) would be to unmarshall into a map[string]interface{}
type CVEItem struct {
CVE map[string]interface{} `json:"cve"`
Configurations map[string]interface{} `json:"configurations"`
Impact map[string]interface{} `json:"impact"`
PublishedDate string `json:"publishedDate"`
LastModifiedDate string `json:"lastModifiedDate"`
}
type DataStruct struct {
DataType string `json:"CVE_data_type"`
CVEItems []CVEItem `json:"CVE_items"`
}
var result DataStruct
err = json.Unmarshal([]byte(byteValue), &result)
if err != nil {
fmt.Println(err)
}
Then from this you can loop through the various items and build concrete structs from what is there (by looking at the keys). If there are many optional or nested items... then yes, it will be a pain to do.

How to convert JSON to POJO

Anyway to create a Java POJO template from a JSON string? I'm using Jersey
{
"reviewData" : [ {
"projectKey" : "CR-FOO",
"name" : "Example review.",
"description" : "Description or statement of objectives for this example review.",
"author" : {
"userName" : "joe",
"displayName" : "Joe Krustofski",
"avatarUrl" : "http://foo.com/avatar"
},
"moderator" : {
"userName" : "scott",
"displayName" : "Scott the Moderator",
"avatarUrl" : "http://foo.com/avatar"
},
"creator" : {
"userName" : "joe",
"displayName" : "Joe Krustofski",
"avatarUrl" : "http://foo.com/avatar"
},
"permaId" : {
"id" : "CR-FOO-21"
},
"permaIdHistory" : [ "CR-FOO-21" ],
"type" : "REVIEW",
"allowReviewersToJoin" : true,
"metricsVersion" : 4,
"createDate" : "2013-10-08T15:46:11.022+0200",
"dueDate" : "2013-10-09T15:46:11.022+0200",
"jiraIssueKey" : "FOO-6754"
}, {
"projectKey" : "CR-FOO",
"name" : "Example review.",
"description" : "Description or statement of objectives for this example review.",
"author" : {
"userName" : "joe",
"displayName" : "Joe Krustofski",
"avatarUrl" : "http://foo.com/avatar"
},
"moderator" : {
"userName" : "scott",
"displayName" : "Scott the Moderator",
"avatarUrl" : "http://foo.com/avatar"
},
"creator" : {
"userName" : "joe",
"displayName" : "Joe Krustofski",
"avatarUrl" : "http://foo.com/avatar"
},
"permaId" : {
"id" : "CR-FOO-21"
},
"permaIdHistory" : [ "CR-FOO-21" ],
"type" : "REVIEW",
"allowReviewersToJoin" : true,
"metricsVersion" : 4,
"createDate" : "2013-10-08T15:46:11.022+0200",
"dueDate" : "2013-10-09T15:46:11.022+0200",
"jiraIssueKey" : "FOO-6754"
} ]
}
The best solution I found was http://www.jsonschema2pojo.org/
It has support for Jackson and Maven and an online tool as well.