golang recursive json to struct? - json

I used to write python, just started to contact golang
my json for example,children unknow numbers,may be three ,may be ten。
[{
"id": 1,
"name": "aaa",
"children": [{
"id": 2,
"name": "bbb",
"children": [{
"id": 3,
"name": "ccc",
"children": [{
"id": 4,
"name": "ddd",
"children": []
}]
}]
}]
}]
i write struct
type AutoGenerated []struct {
ID int `json:"id"`
Name string `json:"name"`
Children []struct {
ID int `json:"id"`
Name string `json:"name"`
Children []struct {
ID int `json:"id"`
Name string `json:"name"`
Children []struct {
ID int `json:"id"`
Name string `json:"name"`
Children []interface{} `json:"children"`
} `json:"children"`
} `json:"children"`
} `json:"children"`
}
but i think this too stupid。
how to optimize?

You can reuse the AutoGenerated type in its definition:
type AutoGenerated []struct {
ID int `json:"id"`
Name string `json:"name"`
Children AutoGenerated `json:"children"`
}
Testing it:
var o AutoGenerated
if err := json.Unmarshal([]byte(src), &o); err != nil {
panic(err)
}
fmt.Println(o)
(src is your JSON input string.)
Output (try it on the Go Playground):
[{1 aaa [{2 bbb [{3 ccc [{4 ddd []}]}]}]}]
Also it's easier to understand and work with if AutoGenerated itself is not a slice:
type AutoGenerated struct {
ID int `json:"id"`
Name string `json:"name"`
Children []AutoGenerated `json:"children"`
}
Then using it / testing it:
var o []AutoGenerated
if err := json.Unmarshal([]byte(src), &o); err != nil {
panic(err)
}
fmt.Println(o)
Outputs the same. Try this one on the Go Playground.

Related

Unmarshal nested JSON with different structure but same keys

I'm trying to unmarshal some nested JSON which looks like:
{
"id": "aRandomId",
"type": "aRandomType",
"aRandomField": {
"type": "someType",
"createdAt": "2020-07-07T15:50:02",
"object": "anObject",
"modifiedAt": "2020-07-07T15:50:02"
},
"aParameter": {
"type": "Property",
"createdAt": "2020-07-07T15:50:02",
"value": "myValue",
"modifiedAt": "2020-07-07T15:50:02"
},
"location": {
"type": "GeoProperty",
"value": {
"type": "Point",
"coordinates": [
7.0054,
40.9999
]
}
},
... other things with type, value ...
"createdAt": "2020-07-07T15:50:02",
"modifiedAt": "2020-07-07T15:50:02",
}
I would like to get all keys and values which are: type, createdAt, Value (also if they are nested)
Actually, I have 2 structs:
type Attribute struct {
Type string `json:"type"`
CreatedAt string `json:"createdAt"`
Value string `json:"value"`
ModifiedAt string `json:"modifiedAt"`
}
type Entity struct {
Id string `json:"id"`
Type string `json:"type"`
CreatedAt string `json:"createdAt"`
Attribute Attribute
}
in := []byte(buf.String())
var entity Entity
err := json.Unmarshal(in, &entity)
if err != nil {
panic(err)
}
frame.Fields = append(frame.Fields,
data.NewField("key", nil, []string{"type : ", "createdAt : ", "name : "}),
)
frame.Fields = append(frame.Fields,
data.NewField("value", nil, []string{entity.Type, entity.CreatedAt, entity.Attribute.Value}),
)
The problem is there can be several different Attribute struct andIi can't provide them all.
I would like to display all key (only type, createdAt and Value) in one frame and all their value in another.
Maybe have something like that?
type Entity struct {
attribute List<Attribute>
}
type Attribute struct{
Type string
CreatedAt string
Value string
ModifiedAt string
}
The problem is there can be several different Attribute struct andIi can't provide them all
It looks like your JSON data can have a set of keys with similar values (Attribute) and you can't know how many of them might be in the data.
For this case, you can use a map[string]json.RawMessage as the starting entity to unmarshal to
var e map[string]json.RawMessage
if err := json.Unmarshal([]byte(jsonData), &e); err != nil {
panic(err)
}
You can then range over the values to see if you can unmarshal them into Attribute type
for k, v := range e {
var a Attribute
if err := json.Unmarshal(v, &a); err == nil {
log.Printf("Got attribute %s: %s", k, string(v))
}
}
Run it on playground

Dynamic JSON struct in Golang doesn't behave as expected

I'm trying to create a struct with some basic fields which are always present and some optional fields which are structs by themselves.
I'm wondering why the following code:
package main
import (
"encoding/json"
"fmt"
"time"
)
type DataManagement struct {
DataManagement struct {
Type string
Asset struct {
LocalAssetUID string
Type string
}
*ContentProductionOrder
*ContentItem
TimeStamp time.Time
Hash string
}
}
type ContentProductionOrder struct {
ProductionOrderNo int
ItemNo int
StartDate time.Time
FinishDate time.Time
StatusID int
StatusDate time.Time
SourceTypeID int
TrackingID int
}
type ContentItem struct {
ItemNo int
ItemText string
Quantity int
}
func main() {
var contentItem ContentItem
contentItem.ItemNo = 123
contentItem.ItemText = "aaaaaaaa"
contentItem.Quantity = 3
var dataManagement DataManagement
dataManagement.DataManagement.Type = "asd"
dataManagement.DataManagement.Asset.LocalAssetUID = "asd"
dataManagement.DataManagement.Asset.Type = "asd"
dataManagement.DataManagement.ContentItem = &contentItem
dataManagement.DataManagement.TimeStamp = time.Now().UTC()
dataManagement.DataManagement.Hash = "123"
xy, _ := json.MarshalIndent(dataManagement, "", " ")
fmt.Println(string(xy))
xy, _ = json.MarshalIndent(contentItem, "", " ")
fmt.Println(string(xy))
}
outputs to the following:
{
"DataManagement": {
"Type": "asd",
"Asset": {
"LocalAssetUID": "asd",
"Type": "asd"
},
"ItemText": "aaaaaaaa",
"Quantity": 3,
"TimeStamp": "2009-11-10T23:00:00Z",
"Hash": "123"
}
}
{
"ItemNo": 123,
"ItemText": "aaaaaaaa",
"Quantity": 3
}
and not to:
{
"DataManagement": {
"Type": "asd",
"Asset": {
"LocalAssetUID": "asd",
"Type": "asd"
},
"ContentItem": {
"ItemNo": 123,
"ItemText": "aaaaaaaa",
"Quantity": 3
},
"TimeStamp": "2009-11-10T23:00:00Z",
"Hash": "123"
}
}
{
"ItemNo": 123,
"ItemText": "aaaaaaaa",
"Quantity": 3
}
Any ideas? It's probably pretty easy to explain; I'm not that experienced in Golang.
Here's a Playground link: https://play.golang.org/p/iRDcaRIZ_ZU
The output you are not getting which you want is because you have used embedded struct for ContentItem in DataManagement rather than field name to add to the struct.
A field declared with a type but no explicit field name is called an
embedded field. An embedded field must be specified as a type name T
or as a pointer to a non-interface type name *T, and T itself may not
be a pointer type. The unqualified type name acts as the field name.
A field declaration will sove your issue. You should change the struct DataManagement to:
type DataManagement struct {
DataManagement struct {
Type string
Asset struct {
LocalAssetUID string
Type string
}
*ContentProductionOrder // this is embedded struct
ContentItem *ContentItem
TimeStamp time.Time
Hash string
}
}
Working Code on Go Playground
For more information, Have a look at Golang Spec for Struct Types

Golang & mgo: How to create a generic entity with common fields like _id, creation time, and last update

Given the following struct:
package models
import (
"time"
"gopkg.in/mgo.v2/bson"
)
type User struct {
Id bson.ObjectId `json:"id" bson:"_id"`
Name string `json:"name" bson:"name"`
BirthDate time.Time `json:"birth_date" bson:"birth_date"`
InsertedAt time.Time `json:"inserted_at" bson:"inserted_at"`
LastUpdate time.Time `json:"last_update" bson:"last_update"`
}
... here is how I insert a new user into a Mongo collection:
user := &models.User{
bson.NewObjectId(),
"John Belushi",
time.Date(1949, 01, 24),
time.now().UTC(),
time.now().UTC(),
}
dao.session.DB("test").C("users").Insert(user)
Is it possible to have a generic Entity all other entities inherit from? I've tried this...
type Entity struct {
Id bson.ObjectId `json:"id" bson:"_id"`
InsertedAt time.Time `json:"inserted_at" bson:"inserted_at"`
LastUpdate time.Time `json:"last_update" bson:"last_update"`
}
type User struct {
Entity
Name string `json:"name" bson:"name"`
BirthDate time.Time `json:"birth_date" bson:"birth_date"`
}
... but this implies a final result like this:
{
"Entity": {
"_id": "...",
"inserted_at": "...",
"last_update": "..."
},
"name": "John Belushi",
"birth_date": "1949-01-24..."
}
How do I get the following result without repeating common fields in each struct?
{
"_id": "...",
"inserted_at": "...",
"last_update": "...",
"name": "John Belushi",
"birth_date": "1949-01-24..."
}
This was already answered in Storing nested structs with mgo, but it is very easy all you need to do is add bson:",inline" on the anonymous inner struct and initialize as normal...
Here a quick example:
package main
import (
"gopkg.in/mgo.v2"
"gopkg.in/mgo.v2/bson"
)
type Entity struct {
Id bson.ObjectId `json:"id" bson:"_id"`
InsertedAt time.Time `json:"inserted_at" bson:"inserted_at"`
LastUpdate time.Time `json:"last_update" bson:"last_update"`
}
type User struct {
Entity `bson:",inline"`
Name string `json:"name" bson:"name"`
BirthDate time.Time `json:"birth_date" bson:"birth_date"`
}
func main() {
info := &mgo.DialInfo{
Addrs: []string{"localhost:27017"},
Timeout: 60 * time.Second,
Database: "test",
}
session, err := mgo.DialWithInfo(info)
if err != nil {
panic(err)
}
defer session.Close()
session.SetMode(mgo.Monotonic, true)
// c := session.DB("test").C("users")
user := User{
Entity: Entity{"123456789098", time.Now().UTC(), time.Now().UTC()},
Name: "John Belushi",
BirthDate: time.Date(1959, time.February, 28, 0, 0, 0, 0, time.UTC),
}
session.DB("test").C("users").Insert(user)
}

Parsing JSON in GoLang into struct

So, I'm having some trouble parsing this data in golang:
{
"gateways": [
{
"token": "my_token_here",
"gateway_type": "test",
"description": null,
"payment_methods": [
"credit_card",
"sprel",
"third_party_token",
"bank_account",
"apple_pay"
],
"state": "retained",
"created_at": "2016-03-12T18:52:37Z",
"updated_at": "2016-03-12T18:52:37Z",
"name": "Spreedly Test",
"characteristics": [
"purchase",
"authorize",
"capture",
"credit",
"general_credit",
"void",
"verify",
"reference_purchase",
"purchase_via_preauthorization",
"offsite_purchase",
"offsite_authorize",
"3dsecure_purchase",
"3dsecure_authorize",
"store",
"remove",
"disburse",
"reference_authorization"
],
"credentials": [],
"gateway_specific_fields": [],
"redacted": false
}
]
}
When using this struct I can get it to output pretty easily.
type gateways struct {
Gateways []struct {
Characteristics []string `json:"characteristics"`
CreatedAt string `json:"created_at"`
Credentials []interface{} `json:"credentials"`
Description interface{} `json:"description"`
GatewaySpecificFields []interface{} `json:"gateway_specific_fields"`
GatewayType string `json:"gateway_type"`
Name string `json:"name"`
PaymentMethods []string `json:"payment_methods"`
Redacted bool `json:"redacted"`
State string `json:"state"`
Token string `json:"token"`
UpdatedAt string `json:"updated_at"`
} `json:"gateways"`
}
But as soon as I seperate the "Gateways []struct" into its own struct then it returns an empty array...
Full source.
type gateway struct {
Characteristics []string `json:"characteristics"`
CreatedAt string `json:"created_at"`
Credentials []interface{} `json:"credentials"`
Description interface{} `json:"description"`
GatewaySpecificFields []interface{} `json:"gateway_specific_fields"`
GatewayType string `json:"gateway_type"`
Name string `json:"name"`
PaymentMethods []string `json:"payment_methods"`
Redacted bool `json:"redacted"`
State string `json:"state"`
Token string `json:"token"`
UpdatedAt string `json:"updated_at"`
}
type gateways struct {
Gateways []gateway `json:"gateways"`
}
func ParseResponse() {
var parsed gateways
json.Unmarshal(json, &parsed)
}
There's a problem with your ParseResponse function, you're calling json.Unmarshal passing as first parameter json, that's a packge name: that's ambiguous.
As you can see, your code works well changing the ParseResponse function.
package main
import (
"encoding/json"
"fmt"
)
type gateway struct {
Characteristics []string `json:"characteristics"`
CreatedAt string `json:"created_at"`
Credentials []interface{} `json:"credentials"`
Description interface{} `json:"description"`
GatewaySpecificFields []interface{} `json:"gateway_specific_fields"`
GatewayType string `json:"gateway_type"`
Name string `json:"name"`
PaymentMethods []string `json:"payment_methods"`
Redacted bool `json:"redacted"`
State string `json:"state"`
Token string `json:"token"`
UpdatedAt string `json:"updated_at"`
}
type gateways struct {
Gateways []gateway `json:"gateways"`
}
func ParseResponse(js []byte) {
var parsed gateways
json.Unmarshal(js, &parsed)
fmt.Println(parsed)
}
func main() {
var js []byte = []byte(`{
"gateways": [
{
"token": "my_token_here",
"gateway_type": "test",
"description": null,
"payment_methods": [
"credit_card",
"sprel",
"third_party_token",
"bank_account",
"apple_pay"
],
"state": "retained",
"created_at": "2016-03-12T18:52:37Z",
"updated_at": "2016-03-12T18:52:37Z",
"name": "Spreedly Test",
"characteristics": [
"purchase",
"authorize",
"capture",
"credit",
"general_credit",
"void",
"verify",
"reference_purchase",
"purchase_via_preauthorization",
"offsite_purchase",
"offsite_authorize",
"3dsecure_purchase",
"3dsecure_authorize",
"store",
"remove",
"disburse",
"reference_authorization"
],
"credentials": [],
"gateway_specific_fields": [],
"redacted": false
}
]
}`)
/*
var parsed gateways
e := json.Unmarshal(js, &parsed)
if e != nil {
fmt.Println(e.Error())
} else {
fmt.Println(parsed)
}
*/
ParseResponse(js)
}
Outputs:
{[{[purchase authorize capture credit general_credit void verify reference_purchase purchase_via_preauthorization offsite_purchase offsite_authorize 3dsecure_purchase 3dsecure_authorize store remove disburse reference_authorization] 2016-03-12T18:52:37Z [] <nil> [] test Spreedly Test [credit_card sprel third_party_token bank_account apple_pay] false retained my_token_here 2016-03-12T18:52:37Z}]}
Take a look at http://play.golang.org/p/3xJHBmhuei - unmarshalling into your second definition of gateways completes successfully, so it must be something little that you're missing.
Check if the json.Unmarshal call returns an error.
P.S. It probably isn't the problem if you're unmarshalling successfully into the first version of gateways, but the JSON string that you've given above is missing a closing "}" bracket.

Encoding nested JSON in Go

I've had a lot of trouble finding an example of this. Most of the information on the internet is about decoding JSON.
I'd like to serialize some data into nested JSON, like for example:
{
"item": {
"title": "Items",
"properties": [
{
"num": 1,
"name": "Item 1"
},
{
"num": 2,
"name": "Item 2"
}
]
}
}
I know how to marshal data with a flat struct, but how do I put data into a struct that can be serialized with nesting?
http://play.golang.org/p/nDKmv1myTD
I found this tool that generates a struct from a JSON schema, but I don't understand how to get data into the sub structs.
http://mholt.github.io/json-to-go/
type Example struct {
Item struct {
Title string `json:"title"`
Properties []struct {
Num int `json:"num"`
Name string `json:"name"`
} `json:"properties"`
} `json:"item"`
}
This tool you found is nice, but I would not use it. It makes it difficult to initialize the structs.
Init example with your snippet: (http://play.golang.org/p/_Qw3Qp8XZh)
package main
import (
"encoding/json"
"fmt"
)
type Example struct {
Item struct {
Title string `json:"title"`
Properties []struct {
Num int `json:"num"`
Name string `json:"name"`
} `json:"properties"`
} `json:"item"`
}
func main() {
info := &Example{
Item: struct {
Title string `json:"title"`
Properties []struct {
Num int `json:"num"`
Name string `json:"name"`
} `json:"properties"`
}{
Title: "title",
Properties: []struct {
Num int `json:"num"`
Name string `json:"name"`
}{
{Num: 0, Name: "name0"},
{Num: 1, Name: "name1"},
},
},
}
b, err := json.Marshal(info)
if err != nil {
fmt.Println(err)
return
}
fmt.Println(string(b))
}
Result (pretty printed):
{
"item": {
"title": "title",
"properties": [
{
"num": 0,
"name": "name0"
},
{
"num": 1,
"name": "name1"
}
]
}
}
I think it is better to use named struct vs anonymous nested ones.
Same example with named structs: http://play.golang.org/p/xm7BXxEGTC
package main
import (
"encoding/json"
"fmt"
)
type Example struct {
Item Item `json:"item"`
}
type Item struct {
Title string `json:"title"`
Properties []Property `json:"properties"`
}
type Property struct {
Num int `json:"num"`
Name string `json:"name"`
}
func main() {
info := &Example{
Item: Item{
Title: "title",
Properties: []Property{
{Num: 0, Name: "name0"},
{Num: 1, Name: "name1"},
},
},
}
b, err := json.Marshal(info)
if err != nil {
fmt.Println(err)
return
}
fmt.Println(string(b))
}
It is the exact same thing, but I find it more clear and easy to use.