Unmarshal single-item JSON list to single nested struct - json

I have JSON coming from a graph database and it only returns lists. Here is an example query and response:
{
Course(func: eq(XID, "1")) {
Name
Holes : Hole {
Number
Tee {
Color
Basket #facets(distance) {
Designation
}
}
}
}
}
This query looks up by ID so it can only return one result but the nature of the database backend means it will always return a list like so:
{
"Course": [
{
"Name": "NAD",
"Holes": [
{
"Number": 1,
"Tee": [
{
"Color": "Blue",
"Basket": [
{
"Designation": "A",
"Basket|distance": "70"
}
]
},
{
"Color": "Red",
"Basket": [
{
"Designation": "A",
"Basket|distance": "69"
}
]
}
]
},
{
"Number": 2,
"Tee": [
{
"Color": "Blue",
"Basket": [
{
"Designation": "A",
"Basket|distance": "79"
},
{
"Designation": "B",
"Basket|distance": "89"
}
]
},
{
"Color": "Red",
"Basket": [
{
"Designation": "A",
"Basket|distance": "79"
},
{
"Designation": "B",
"Basket|distance": "95"
}
]
}
]
}
]
}
]
}
I want to unmarshal that single Course to a struct:
type Course struct {
Name string `json:"Name"`
Holes []struct {
Number int `json:"Number"`
Tee []struct {
Color string `json:"Color"`
Basket []struct {
Designation string `json:"Designation"`
BasketDistance string `json:"Basket|distance"`
} `json:"Basket"`
} `json:"Tee"`
} `json:"Holes"`
}
I tried making a temporary struct that is just a list of courses to pull out the correct JSON by index but I get a really generic runtime error that the object can't be unmarshaled to that type.
I believe it would be relatively easy to convert it to a []map[string]interface{} then unmarshal to a Course from there by specifying the JSONRawMessage type but doesn't that suffer a performance penalty? What's the most performant way to solve this problem?

Related

Scala parsing nested json with Json4s

I am trying to fetch data from nested JSON, I need only a few fields from the JSON,
I have created case classes for the required data, the solution I found from google suggested to use read function, but I get an empty Object
I tried to google with no success, What I am missing?
my code
val rawDataFromFile = Source.fromFile(path).mkString
case class Data(listOfPersons: List[Person])
case class Person(bio: Bio, terms: List[Term])
case class Bio(birthday: String, gender: String)
case class Term(`type`: String, start: String, end: String)
read[Data](rawDataFromFile)
res >> Data(List())
and the JSON
[
{
"id": {
"not_intresting_field_1": "B000944",
"not_intresting_field_4": [
"H2OH13033",
"S6OH00163"
]
},
"name": {
"first": "first_name_1",
"last": "last_name_1"
},
"bio": {
"birthday": "1952-11-09",
"gender": "M"
},
"terms": [
{
"type": "rep",
"start": "1993-01-05",
"end": "1995-01-03"
},
{
"type": "rep",
"start": "1995-01-04",
"end": "1997-01-03"
}
]
},
{
"id": {
"not_intresting_field_1": "C000127",
"not_intresting_field_4": [
"S8WA00194",
"H2WA01054"
]
},
"name": {
"first": "first_name_1",
"last": "last_name_1"
},
"bio": {
"birthday": "1958-10-13",
"gender": "F"
},
"terms": [
{
"type": "rep",
"start": "1993-01-05",
"end": "1995-01-03"
},
{
"type": "sen",
"start": "2001-01-03",
"end": "2007-01-03"
}
]
}
]
Your case class is not the same as your json structure.
Here your define Data type which will read json like following
{
"listOfPersons": [
{
"id": {
"not_intresting_field_1": "B000944",
"not_intresting_field_4": [
"H2OH13033",
"S6OH00163"
]
},
"name": {
"first": "first_name_1",
"last": "last_name_1"
},
"bio": {
"birthday": "1952-11-09",
"gender": "M"
},
... //your original json
}
]
}
Try this
read[List[Person]](rawDataFromFile)

JQ: Delete key:value pairs anywhere in JSON tree

I am looking for a way to delete key:value pairs (where key is a known string) anywhere in the JSON tree with jq.
Specifically: I would like to delete "type": "Link" and "linkType": "Entry" everywhere (in all parts of the hierarchy, where ever they appear). My JSON file is +800000 rows long and is nested deeply.
Snippet:
{
"entries": [
{
"sys": {
"id": "vcLKKhJ3mZNfGMvVZZi07",
"contentType": {
"sys": {
"id": "page"
}
}
},
"fields": {
"title": {
"de-DE": "Startseite",
"en-US": "Home"
},
"description": {
"en-US": "foo"
},
"keywords": {
"en-US": "bar"
},
"stageModules": {
"en-US": [
{
"sys": {
"type": "Link",
"linkType": "Entry",
"id": "11AfBBuNK8bx3EygAS3WTY"
}
}
]
}
}
}
]
}
I've tried so many different things to iterate through the file to remove these two from the output, I constantly end up with "Cannot index array with string". E.g.
[ .[] | select(.linkType != "Entry", .type != "Link" ) ]
Ideal result:
{
"entries": [
{
"sys": {
"id": "vcLKKhJ3mZNfGMvVZZi07",
"contentType": {
"sys": {
"id": "page"
}
}
},
"fields": {
"title": {
"de-DE": "Startseite",
"en-US": "Home"
},
"description": {
"en-US": "foo"
},
"keywords": {
"en-US": "bar"
},
"stageModules": {
"en-US": [
{
"sys": {
"id": "11AfBBuNK8bx3EygAS3WTY"
}
}
]
}
}
}
]
}
Can someone help me out?
Thank you for your help.
From the jq manual:
The walk(f) function applies f recursively to every component of the
input entity.
Use this to recursively go through all items, check your conditions and use del if necessary:
walk(
if type == "object" and .type == "Link" then del(.type) else . end
)
Edit: If "matching is less important", as you state, then just use del on any object:
walk(if type == "object" then del(.type, .linkType) else . end)

Nested dicts and lists / glom lib python

I am trying to access deep-nested lists and dictionaries. I am experimenting with the glom library however my Third_KV key doesn't work on the below JSON object when trying to retrieve the "Country"
from glom import glom
target = {
"Result": {
"Topics": [
{
"A": "abc",
"D": 0,
"Questions": [
{
"E": "jklm",
"P": "dsfs",
"Answers": [
{
"first": "string",
"second": "string",
"Country": "CH"
},
{
"first": "string",
"second": "string",
"Country": "NL"
}
]
}
]
}
]
}
}
path = {
"First_KV": ("Result.Topics", ["Questions"]),
"Second_KV": ("Result.Topics", [("Questions", ["Answers"])]),
"Third_KV": ("Result.Topics", [("Questions", "Answers", ["Country"])])
}
countries = glom(target, path["Third_KV"])
Not very clear what final json/array/structure you want, but without relying on any library, can you not use simple map() e.g.
const jsonTest = {
"Result": {
"Topics": [{
"A": "abc",
"D": 0,
"Questions": [{
"E": "jklm",
"P": "dsfs",
"Answers": [{
"first": "CHfirstCountry",
"second": "CHsecondCountry",
"Country": "CH"
},
{
"first": "NLfirstCountry",
"second": "NLsecondCountry",
"Country": "NL"
}
]
}]
}]
}
};
const AnswersArray = jsonTest.Result.Topics[0].Questions[0].Answers;
let dictPerCountry = new Object();
AnswersArray.map((eachElement) => {
dictPerCountry[eachElement.Country] = [eachElement.first, eachElement.second];
});
console.log({
dictPerCountry
});
dictPerCountry will look like so:
{
"dictPerCountry": {
"CH": [
"CHfirstCountry",
"CHsecondCountry"
],
"NL": [
"NLfirstCountry",
"NLsecondCountry"
]
}
}
Answers are of "list" type too and you are missing its square brackets. check below pattern to get the country
pattern = ('Result.Topics', [('Questions', [('Answers', ['Country'])])])
So you need to change your dictionary "path" to be
path = {
"First_KV": ("Result.Topics", ["Questions"]),
"Second_KV": ("Result.Topics", [("Questions", ["Answers"])]),
"Third_KV": ('Result.Topics', [('Questions', [('Answers', ['Country'])])])
}

How to iterate on json fields and insert new values using json4s?

I have a simple json file:
val myJson = {
"field1": [
{
"name": "john",
"lname": "knight"
},
{
"name": "jack",
"lname": "samuel"
},
{
"name": "elinor",
"lname": "cooper"
}
],
"field2": [
{
...
},
{
...
},
{
...
}
],
"field3": [
{
...
},
{
...
},
{
...
}
]
}
and what i want is to be able to iterate on "field1" and for each name to call a method that returns some value and insert this value to the json under "fiel1".
// this returns a list of strings
val kids = getKids("john")
// this is will me the returned value
kids = List("amy", "tom")
now I want to insert it:
{
"field1": [
{
"name": "john",
"lname": "knight"
"kids": ["amy", "tom"]
},
{
"name": "jack",
"lname": "samuel"
"kids": ["edi", "keren"]
},
{
"name": "elinor",
"lname": "cooper"
"kids": ["lilly", "mag"]
}
]
...
but I want to iterate on all the names and do this for each one...how can I accomplish this with json4s?
so lets say i have the parsed json:
val myParsedJson = JsonMethods.parse(myJson)
how do I go from here?
thanks!

JSON: Serialize a dictionary with complex object in key

What is the best way to serialize a dictionary when key is a complex type. For example consider this invalid json:
[
{ParentId:1, ParentName:'X'}:
[{'ChildId':'1', 'ChildName':'a'}, {'ChildId':'2', 'ChildName':'b'}],
{ParentId:2, ParentName:'Y'}:
[{'ChildId':"3", 'ChildName':'c'}]}
]
Is there any way to correctly serialize this?
You can wrap the json with keys as strings and values as your complex object.
In addition, your keys of the complex type should be strings as well.
For example:
[
{
"name": {
"ParentId": 1,
"ParentName": "X"
},
"value": [
{
"ChildId": "1",
"ChildName": "a"
},
{
"ChildId": "2",
"ChildName": "b"
}
]
},
{
"name": {
"ParentId": 2,
"ParentName": "Y"
},
"value": [
{
"ChildId": "3",
"ChildName": "c"
}
]
}
]
Unfortunately this is not a valid json, so the answer would be no :-( keys have to be strings. You could reorganize your data structure to wrap your children list into a "parent" object which has an id and a name !
Basically it would look like :
[
{
"ParentId": 1,
"ParentName": "X",
"children": [
{
"ChildId": 1,
"ChildName": "a"
},
{
"ChildId": 2,
"ChildName": "b"
}
]
},
{
"ParentId": 2,
"ParentName": "Y",
"children": [
{
"ChildId": 3,
"ChildName": "c"
}
]
}
]
You can find the specifications of JSON here : https://www.rfc-editor.org/rfc/rfc4627