Get Inner JSON Value in Go - json

Simple question that I'm having a difficult time how to structure a struct for JSON decoding.
How can I copy an inner field of a struct to another field of a struct?
I have JSON
{
"Trains": [{
"Car": "6",
"Destination": "SilvrSpg",
"DestinationCode": "B08",
"DestinationName": "Silver Spring",
"Group": "1",
"Line": "RD",
"LocationCode": "A13",
"LocationName": "Twinbrook",
"Min": "1"
}]
}
And I have structs
type Trains struct {
Min string `json:"Min"`
DestName string `json:"DestinationName"`
DestCode string `json:"DestinationCode"`
LocName string `json:"LocationName"`
LocCode string `json:"LocationCode"`
Line string `json:"Line"`
}
type AllData struct {
Data []Trains `json:"Trains"`
}
How Can I get the value of the Trains.LocationCode to a struct like
type AllData struct {
Id Trains[0].LocCode value
Data []Trains `json:"Trains"`
}
So I basically just need to have JSON like this
{
"Id":"A13",
"Data": [{
"Car": "6",
"Destination": "SilvrSpg",
"DestinationCode": "B08",
"DestinationName": "Silver Spring",
"Group": "1",
"Line": "RD",
"LocationCode": "A13",
"LocationName": "Twinbrook",
"Min": "1"
}]
}
Where the Id is the inner value of the Trains struct.
How can I structure a struct to reflect this?

The JSON decoder does not have this capability. You must write the line of code in your application.
package main
import (
"encoding/json"
"fmt"
"log"
)
var s = `
{
"Trains": [{
"Car": "6",
"Destination": "SilvrSpg",
"DestinationCode": "B08",
"DestinationName": "Silver Spring",
"Group": "1",
"Line": "RD",
"LocationCode": "A13",
"LocationName": "Twinbrook",
"Min": "1"
}]
}`
type Train struct {
Min string `json:"Min"`
DestName string `json:"DestinationName"`
DestCode string `json:"DestinationCode"`
LocName string `json:"LocationName"`
LocCode string `json:"LocationCode"`
Line string `json:"Line"`
}
type Data struct {
// The name "-" tells the JSON decoder to ignore this field.
ID string `json:"-"`
Trains []Train
}
func main() {
var d Data
if err := json.Unmarshal([]byte(s), &d); err != nil {
log.Fatal(err)
}
if len(d.Trains) < 1 {
log.Fatal("No trains")
}
// Copy value from inner to outer.
d.ID = d.Trains[0].LocCode
fmt.Printf("%+v\n", &d)
}

Related

Is there any way to unmarshal JSON into a map and preserve keys order

I need to unmarshal JSON data into the map by preserving the order of keys and values.
This is a JSON data example:
{
"31ded736-4076-4b1c-b38f-7e8d9fa78b41": {
"Name": "Requested",
"Items": [
{ "ID": "c7ac5b7f-59b0-45e3-82fb-b3b0afc05f55", "GroupName": "First task" , "NumQuestion": "10" ,"Score":"10"},
{"ID": "17acf9a1-b2c7-46c6-b975-759b9d9f538d", "GroupName": "Test1" , "NumQuestion": "20" ,"Score":"5" }
]
},
"115f7d04-3075-408a-b8ce-c6e46fe6053f": {
"Name": "To do",
"Items": [
{ "ID": "c7ac5b7f-59b0-45e3-82fb-b3b0afc05f56", "GroupName": "First task" , "NumQuestion": "5" ,"Score":"10"}
]
},
"9bcf1415-3a41-43b6-a871-8de1939a75c4": {
"Name": "In Progress",
"Items": [
{ "ID": "c7ac5b7f-59b0-45e3-82fb-b3b0afc05f57", "GroupName": "Second task" , "NumQuestion": "10" ,"Score":"5"}
]
},
"2f6c1455-6cf9-4009-b86b-de0a0d2204a1": {
"Name": "Done",
"Items": [
{ "ID": "c7ac5b7f-59b0-45e3-82fb-b3b0afc05f58", "GroupName": "Third task" , "NumQuestion": "20" , "Score":"7"}
]
}
}
This is my code:
var GroupTestListUpdate = http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
type Item struct {
ID string `json:"id"`
GroupName string `json:"groupName"`
NumQuestion string `json:"numQuestion"`
Score string `json:"score"`
}
type Input struct {
Name string `json:"name"`
Items []Item `array:"item"`
}
var objmap map[string]Input
reqBody, err := ioutil.ReadAll(r.Body)
if err != nil{
http.Error(w, "can't read body", http.StatusBadRequest)
return
}
json.Unmarshal(reqBody, &objmap)
fmt.Println(objmap)
})
Output:
map[115f7d04-3075-408a-b8ce-c6e46fe6053f:{To do [{c7ac5b7f-59b0-45e3-82fb-b3b0afc05f56 First task 5 10}]} 2f6c1455-6cf9-4009-b86b-de0a0d2204a1:{Done [{c7ac5b7f-59b0-45e3-82fb-b3b0afc05f58 Third task 20 7}]} 31ded736-4076-4b1c-b38f-7e8d9fa78b41:{Requested [{c7ac5b7f-59b0-45e3-82fb-b3b0afc05f55 First task 10 10} {17acf9a1-b2c7-46c6-b975-759b9d9f538d Test1 20 5}]} 9bcf1415-3a41-43b6-a871-8de1939a75c4:{In Progress [{c7ac5b7f-59b0-45e3-82fb-b3b0afc05f57 Second task 10 5}]}]
Question:
I want to know how to preserve the order of keys and values.
I got the same output every time I tried this code, So I wonder that the order in the map is not completely random?
Yes.
You can override the UnmarshalJSON method on a custom map type that will allow you to preserve the order.
As an example, here is a library that does just that: https://github.com/iancoleman/orderedmap.

Creating dynamic json

I need to create dynamic json i.e whose key value varies, below mentioned is the json
[{"email":"xxx#gmail.com","location":{"set":"Redmond"},"fname":{"set":"xxxxx"},"clicked_time":{"set":"zz"},"domain":{"add":"ttt"}},{"email":"zzz#gmail.com","location":{"set":"Greece"},"fname":{"set":"zzzzz"},"clicked_time":{"set":"zzz"},"domain":{"add":"zxxxx"}}]
I tried using below code:
rows := []map[string]string{}
if i > 0 {
row := make(map[string]string)
for j:=0;j<len(record);j++ {
key := header[j]
value := record[j]
row[key] = value
}
rows = append(rows, row)
}
How may I add set to location and add to domain to create a nested structure as map can have only one type string or nested structure?
Perhaps I have missed the point a little here, but I am not seeing why this is so dynamic in a way that can't be handled by a struct and the json unmarshal method.
Please see the following for an example
https://play.golang.org/p/8nrO36HQGhy
package main
import (
"encoding/json"
"fmt"
)
type (
Details struct {
Email string `json:"email"`
Location Entry `json:"location"`
FName Entry `json:"fname"`
ClickedTime Entry `json:"clicked_time"`
Domain Entry `json:"domain"`
}
Entry struct {
Set string `json:"set"`
Add string `json:"add"`
}
)
func main() {
d := []byte(`[{
"email": "xxx#gmail.com",
"location": {
"set": "Redmond"
},
"fname": {
"set": "xxxxx"
},
"clicked_time": {
"set": "zz"
},
"domain": {
"add": "ttt"
}
}, {
"email": "zzz#gmail.com",
"location": {
"set": "Greece"
},
"fname": {
"set": "zzzzz"
},
"clicked_time": {
"set": "zzz"
},
"domain": {
"add": "zxxxx"
}
}]`)
x := []Details{}
_ = json.Unmarshal(d, &x)
fmt.Printf("%+v\n", x)
}

Why can this data not be properly unmarshalled into my object model?

I have a (non)working example here: https://play.golang.org/p/qaYhKvJ65J3
I'm not sure why the following data:
alertData := `{
"Id": 0,
"Version": 0,
"OrgId": 1,
"DashboardId": 61,
"PanelId": 84,
"Name": "{qa-dev}{stats-pipeline} Topology Message Age (aggregator) alert",
"Message": "",
"Severity": "",
"State": "",
"Handler": 1,
"Silenced": false,
"ExecutionError": "",
"Frequency": 10,
"EvalData": null,
"NewStateDate": "0001-01-01T00:00:00Z",
"PrevStateDate": "0001-01-01T00:00:00Z",
"StateChanges": 0,
"Created": "0001-01-01T00:00:00Z",
"Updated": "0001-01-01T00:00:00Z",
"Settings": {
"conditions": [
{
"evaluator": {
"params": [
10000
],
"type": "gt"
},
"operator": {
"type": "and"
},
"query": {
"datasourceId": 2,
"model": {
"hide": true,
"refCount": 0,
"refId": "C",
"textEditor": false
},
"params": [
"C",
"5m",
"now"
]
},
"reducer": {
"params": [],
"type": "avg"
},
"type": "query"
}
],
"executionErrorState": "keep_state",
"frequency": "10s",
"handler": 1,
"name": "{qa-dev}{stats-pipeline} Topology Message Age (aggregator) alert",
"noDataState": "keep_state",
"notifications": []
}
}`
Can't be unmarshalled into the following object model:
type Condition struct {
Evaluator struct {
Params []int `json:"params"`
Type string `json:"type"`
} `json:"evaluator"`
Operator struct {
Type string `json:"type"`
} `json:"operator"`
Query struct {
Params []string `json:"params"`
} `json:"query"`
Reducer struct {
Params []interface{} `json:"params"`
Type string `json:"type"`
} `json:"reducer"`
Type string `json:"type"`
}
When I do the following:
condition := Condition{}
err := json.Unmarshal([]byte(alertData), &condition)
if err != nil {
panic(err)
}
fmt.Printf("\n\n json object:::: %+v", condition)
I just get: json object:::: {Evaluator:{Params:[] Type:} Operator:{Type:} Query:{Params:[]} Reducer:{Params:[] Type:} Type:}
Ideally I'd be able to parse it into something like type Conditions []struct{ } but I'm not sure if you can define models as lists?
It looks like you are trying to access the "conditions" property nested under the root "Settings" property. As such, you need to define that root-level type and enough fields to tell the unmarshaler how to find your target property. As such, you would just need to create a new "AlertData" type with the necessary "Settings/conditions" fields.
For example (Go Playground):
type AlertData struct {
Settings struct {
Conditions []Condition `json:"conditions"`
}
}
func main() {
alert := AlertData{}
err := json.Unmarshal([]byte(alertData), &alert)
if err != nil {
panic(err)
}
fmt.Printf("OK: conditions=%#v\n", alert.Settings.Conditions)
// OK: conditions=[]main.Condition{main.Condition{Evaluator:struct { Params []int "json:\"params\""; Type string "json:\"type\"" }{Params:[]int{10000}, Type:"gt"}, Operator:struct { Type string "json:\"type\"" }{Type:"and"}, Query:struct { Params []string "json:\"params\"" }{Params:[]string{"C", "5m", "now"}}, Reducer:struct { Params []interface {} "json:\"params\""; Type string "json:\"type\"" }{Params:[]interface {}{}, Type:"avg"}, Type:"query"}}
}
Note that the printed listing includes so much type information because the "Condition" type uses anonymous structs as field types. If you were to extract them into named structs it will be easier to work with the data, e.g.:
type Condition struct {
Evaluator Evaluator `json:"evaluator"`
Operator Operator `json:"operator"`
// ...
}
type Evaluator struct {
Params []int `json:"params"`
Type string `json:"type"`
}
type Operator struct {
Type string `json:"type"`
}
//...
// OK: conditions=[]main.Condition{
// main.Condition{
// Evaluator:main.Evaluator{Params:[]int{10000}, Type:"gt"},
// Operator:main.Operator{Type:"and"},
// Query:main.Query{Params:[]string{"C", "5m", "now"}},
// Reducer:main.Reducer{Params:[]interface {}{}, Type:"avg"},
// Type:"query",
// },
// }
Go Playground example here...
Maerics explanation is correct, here is an alternative approach which wraps access around struct methods, the data structure is also fully defined. If you're new to Go it's good to get handle on creating the data structures yourself, but here is a handy utility for helping create structs from valid JSON
https://mholt.github.io/json-to-go/
package main
import (
"encoding/json"
"fmt"
"log"
"time"
)
type Data struct {
ID int `json:"Id"`
Version int `json:"Version"`
OrgID int `json:"OrgId"`
DashboardID int `json:"DashboardId"`
PanelID int `json:"PanelId"`
Name string `json:"Name"`
Message string `json:"Message"`
Severity string `json:"Severity"`
State string `json:"State"`
Handler int `json:"Handler"`
Silenced bool `json:"Silenced"`
ExecutionError string `json:"ExecutionError"`
Frequency int `json:"Frequency"`
EvalData interface{} `json:"EvalData"`
NewStateDate time.Time `json:"NewStateDate"`
PrevStateDate time.Time `json:"PrevStateDate"`
StateChanges int `json:"StateChanges"`
Created time.Time `json:"Created"`
Updated time.Time `json:"Updated"`
Settings struct {
Conditions []Condition `json:"conditions"`
ExecutionErrorState string `json:"executionErrorState"`
Frequency string `json:"frequency"`
Handler int `json:"handler"`
Name string `json:"name"`
NoDataState string `json:"noDataState"`
Notifications []interface{} `json:"notifications"`
} `json:"Settings"`
}
type Condition struct {
Evaluator struct {
Params []int `json:"params"`
Type string `json:"type"`
} `json:"evaluator"`
Operator struct {
Type string `json:"type"`
} `json:"operator"`
Query struct {
DatasourceID int `json:"datasourceId"`
Model struct {
Hide bool `json:"hide"`
RefCount int `json:"refCount"`
RefID string `json:"refId"`
TextEditor bool `json:"textEditor"`
} `json:"model"`
Params []string `json:"params"`
} `json:"query"`
Reducer struct {
Params []interface{} `json:"params"`
Type string `json:"type"`
} `json:"reducer"`
Type string `json:"type"`
}
func (d Data) GetFirstCondition() (Condition, error) {
if len(d.Settings.Conditions) > 0 {
return d.Settings.Conditions[0], nil
}
return Condition{}, fmt.Errorf("no conditions found")
}
func (d Data) GetConditionByIndex(index uint) (Condition, error) {
if len(d.Settings.Conditions) == 0 {
return Condition{}, fmt.Errorf("no conditions found")
}
if int(index) > len(d.Settings.Conditions)-1 {
return Condition{}, fmt.Errorf("index out of bounds")
}
return d.Settings.Conditions[index], nil
}
var alertData = `{
"Id": 0,
"Version": 0,
"OrgId": 1,
"DashboardId": 61,
"PanelId": 84,
"Name": "{qa-dev}{stats-pipeline} Topology Message Age (aggregator) alert",
"Message": "",
"Severity": "",
"State": "",
"Handler": 1,
"Silenced": false,
"ExecutionError": "",
"Frequency": 10,
"EvalData": null,
"NewStateDate": "0001-01-01T00:00:00Z",
"PrevStateDate": "0001-01-01T00:00:00Z",
"StateChanges": 0,
"Created": "0001-01-01T00:00:00Z",
"Updated": "0001-01-01T00:00:00Z",
"Settings": {
"conditions": [
{
"evaluator": {
"params": [
10000
],
"type": "gt"
},
"operator": {
"type": "and"
},
"query": {
"datasourceId": 2,
"model": {
"hide": true,
"refCount": 0,
"refId": "C",
"textEditor": false
},
"params": [
"C",
"5m",
"now"
]
},
"reducer": {
"params": [],
"type": "avg"
},
"type": "query"
}
],
"executionErrorState": "keep_state",
"frequency": "10s",
"handler": 1,
"name": "{qa-dev}{stats-pipeline} Topology Message Age (aggregator) alert",
"noDataState": "keep_state",
"notifications": []
}
}`
func main() {
var res Data
err := json.Unmarshal([]byte(alertData), &res)
if err != nil {
log.Fatal(err)
}
fmt.Println(res.GetFirstCondition())
fmt.Println(res.GetConditionByIndex(0))
// should fail :-)
fmt.Println(res.GetConditionByIndex(1))
}

Parse JSON in Go

This is an example of JSON output when calling 'ListObjects' for AWS S3
{
"Contents": [{
"ETag": "9e2bc2894b23742b7bb688c646c6fee9",
"Key": "DSC-0237.jpg",
"LastModified": "2017-09-06 21:53:15 +0000 UTC",
"Owner": {
"DisplayName": "demo-user",
"ID": "a9e2f170a6880f1d61852df8e523e88ca2a2b7abd093476cc93f1239ab5063c6"
},
"Size": 117904,
"StorageClass": "STANDARD"
}, {
"ETag": "\"9e2bc2894b23742b7bb688c646c6fee9\"",
"Key": "DSC-0238.jpg",
"LastModified": "2017-09-06 21:52:24 +0000 UTC",
"Owner": {
"DisplayName": "demo-user",
"ID": "a9e2f170a6880f1d61852df8e523e88ca2a2b7abd093476cc93f1239ab5063c6"
},
"Size": 117904,
"StorageClass": "STANDARD"
}, {
"ETag": "\"9e2bc2894b23742b7bb688c646c6fee9\"",
"Key": "DSC-0239.jpg",
"LastModified": "2017-09-06 21:53:01 +0000 UTC",
"Owner": {
"DisplayName": "demo-user",
"ID": "a9e2f170a6880f1d61852df8e523e88ca2a2b7abd093476cc93f1239ab5063c6"
},
"Size": 117904,
"StorageClass": "STANDARD"
}],
"IsTruncated": false,
"Marker": "",
"MaxKeys": 5,
"Name": "test-bucket-x011pp3",
"Prefix": ""
}
How do I parse this in Go? Chiefly I am interested in collecting:
Bucket Name
Key
Size
Owner's DisplayName
LastModified
I am coming from Python and in Python it would be something really simple like:
json_result = json.loads(json_string)
bucket_name = json_result['Name']
for idx, obj in enumerate(json_result['Contents']):
key = obj['Key']
size = obj['Size']
lastmod = obj['LastModified']
owner= obj['Owner']['DisplayName']
Thank you for the help!
It is something like this
package main
import (
"bytes"
"encoding/json"
"fmt"
"log"
)
var jsonData = []byte(`
{
"Contents": [{
"ETag": "9e2bc2894b23742b7bb688c646c6fee9",
"Key": "DSC-0237.jpg",
"LastModified": "2017-09-06 21:53:15 +0000 UTC",
"Owner": {
"DisplayName": "demo-user",
"ID": "a9e2f170a6880f1d61852df8e523e88ca2a2b7abd093476cc93f1239ab5063c6"
},
"Size": 117904,
"StorageClass": "STANDARD"
}, {
"ETag": "\"9e2bc2894b23742b7bb688c646c6fee9\"",
"Key": "DSC-0238.jpg",
"LastModified": "2017-09-06 21:52:24 +0000 UTC",
"Owner": {
"DisplayName": "demo-user",
"ID": "a9e2f170a6880f1d61852df8e523e88ca2a2b7abd093476cc93f1239ab5063c6"
},
"Size": 117904,
"StorageClass": "STANDARD"
}, {
"ETag": "\"9e2bc2894b23742b7bb688c646c6fee9\"",
"Key": "DSC-0239.jpg",
"LastModified": "2017-09-06 21:53:01 +0000 UTC",
"Owner": {
"DisplayName": "demo-user",
"ID": "a9e2f170a6880f1d61852df8e523e88ca2a2b7abd093476cc93f1239ab5063c6"
},
"Size": 117904,
"StorageClass": "STANDARD"
}],
"IsTruncated": false,
"Marker": "",
"MaxKeys": 5,
"Name": "test-bucket-x011pp3",
"Prefix": ""
}`,
)
type Response struct {
Contents []*Content
IsTruncated bool
Marker string
MaxKeys int
Name string
Prefix string
}
type Content struct {
ETag string
Key string
LastModified string
Owner *Owner
Size int
StroageClass string
}
type Owner struct {
DisplayName string
ID string
}
func main() {
resp := &Response{}
if err := json.NewDecoder(bytes.NewBuffer(jsonData)).Decode(resp); err != nil {
log.Fatal(err)
}
fmt.Printf("%#v", resp)
}
Try it in the Go Playground
And you should read this https://golang.org/pkg/encoding/json/
See https://golang.org/pkg/encoding/json
You annotate a struct with JSON fields that should be unmarshalled from the buffer of JSON, something like this:
type AWSObject struct {
Size int `json:"Size"`
Key string `json:"key"`
Owner AWSObjectOwner `json:"Owner"`
}
type AWSObjectOwner struct {
DisplayName `json:"DisplayName"`
}
var awsObjects []AWSObject
err = json.Unmarshal(jsonBuffer, &awsObjects)
if err != nil {
fmt.Printf("Error unmarshaling objects: " + err.Error() + "\n")
...
}
Using quicktype, I generated your model, marshaling code, and usage instructions:
// To parse this JSON data, add this code to your project and do:
//
// r, err := UnmarshalListObjectsResponse(bytes)
// bytes, err = r.Marshal()
package main
import "encoding/json"
func UnmarshalListObjectsResponse(data []byte) (ListObjectsResponse, error) {
var r ListObjectsResponse
err := json.Unmarshal(data, &r)
return r, err
}
func (r *ListObjectsResponse) Marshal() ([]byte, error) {
return json.Marshal(r)
}
type ListObjectsResponse struct {
Contents []Content `json:"Contents"`
IsTruncated bool `json:"IsTruncated"`
Marker string `json:"Marker"`
MaxKeys int64 `json:"MaxKeys"`
Name string `json:"Name"`
Prefix string `json:"Prefix"`
}
type Content struct {
ETag string `json:"ETag"`
Key string `json:"Key"`
LastModified string `json:"LastModified"`
Owner Owner `json:"Owner"`
Size int64 `json:"Size"`
StorageClass string `json:"StorageClass"`
}
type Owner struct {
DisplayName string `json:"DisplayName"`
ID string `json:"ID"`
}
As others suggested, it's probably best to use the AWS SDK for Go, but this method may be useful in general the next time you need to parse arbitrary JSON.

MGO return bson field instead of json field

The bson name is used when performing pipe in mgo.
Struct :
type Training struct {
Id bson.ObjectId `json:"id" bson:"_id"`
Name string `json:"name" bson:"name"`
Description string `json:"description"`
Level *TrainingLevel `json:"level"`
Preworks []bson.ObjectId `json:"preworks"`
PrePostTests []bson.ObjectId `json:"preposttests" bson:"preposttests"`
TrainingEvaluations []bson.ObjectId `json:"training_evaluations" bson:"training_evaluations"`
TrainerEvaluations []bson.ObjectId `json:"trainer_evaluations" bson:"trainer_evaluations"`
AppCompanyId bson.ObjectId `json:"app_company_id" bson:"app_company_id"`
Company *Company `json:"company"`
}
Function :
func (this *TrainingClass) GetAllTraining() (interface{}, error) {
if !this.tokenInfo.IsAllowed(this.c) {
return nil, tlib.NewTError(common.Error_NoAccess, "You don't have the right!")
}
sess, db := GetDB()
defer sess.Close()
pipeline := []bson.M{
{"$match": bson.M{
"app_company_id": this.tokenInfo.AppCompanyId}},
{"$lookup": bson.M{
"from": "trainingbatch",
"localField": "_id",
"foreignField": "training._id",
"as": "trainingbatches"}},
}
resp := []bson.M{}
db.C(common.C_TRAINING).Pipe(pipeline).All(&resp)
return bson.M{"data": resp}, nil
}
Json result :
{
"data": [
{
"_id": "5995a749dbcfbe4e8cc31378",
"app_company_id": "58b24756e65bd121f6b1a923",
"description": "Description First Training",
"name": "First Training",
"trainingbatches": [
{
"_id": "5995a74adbcfbe4e8cc31379",
"app_company_id": "58b24756e65bd121f6b1a923",
"company": {
"_id": "58b24756e65bd121f6b1a923",
"address": "",
"app_company_id": "58b24756e65bd121f6b1a923",
"fullname": "",
"name": "Tandem",
"phone": ""
},
}
]
}
]
}
As you can see field _id is generated instead of id. That's not happen if I use find or findId. Is there any way to keep using json field no matter what's the query?
The way you're reading the result, it has no idea what the JSON field names are. In order for it to use those tags, it must actually deserialize into the struct where the tags have been specified. When you do:
resp := []bson.M{}
db.C(common.C_TRAINING).Pipe(pipeline).All(&resp)
You're explicitly telling mgo to return BSON results. The object you pass in (a slice of bson.M) has no json tags on it. In order to control the serialization to JSON, you must pass a struct with the JSON tags specified to All:
resp := []Training
db.C(common.C_TRAINING).Pipe(pipeline).All(&resp)