Umarshalling this JSON in Golang - json

I'm teaching myself how to use the JSON package in Golang. It seems straightforward for a lot of things, but I'm having troubles parsing some JSON data I retrieved from my 3D printer. The JSON looks like this:
{
"tasks": [
{
"class": "Task",
"id": "5fee231a",
"instances": {
"28253266": {
"class": "StateInstance",
"id": "28253266",
"progress": 1,
"stateType": "Y-EdgeAvoiding"
},
"1d774b49": {
"class": "StateInstance",
"id": "1d774b49",
"progress": 1,
"stateType": "X-Calibration"
},
},
"stateType": "StartingUp"
}
]
}
(NB: There's a few more "instances", but I didn't include them for brevity, but they all follow this pattern, but with a different stateType)
Basically, the printer has a task that it is doing (in the example above, the task has an ID of 5fee231a) and within it, sub-tasks (e.g. Task 28253266).
If I use this code:
var vals interface{}
err = json.Unmarshal([]byte(myJSON), &vals)
if err != nil {
fmt.Println("Error:", err)
}
spew.Dump(&vals)
(using github.com/davecgh/go-spew to dump the variable), then I get some output (NB: This isn't the whole output, it's snipped for brevity :)):
(*map[string]interface {})(0xc0820068e0)((len=1) {
(string) (len=5) "tasks": ([]interface {}) (len=1 cap=1) {
(map[string]interface {}) (len=4) {
(string) (len=5) "class": (string) (len=4) "Task",
(string) (len=2) "id": (string) (len=8) "5fee231a",
(string) (len=9) "instances": (map[string]interface {}) (len=13) {
(string) (len=8) "bd65d028": (map[string]interface {}) (len=4) {
(string) (len=5) "class": (string) (len=13) "StateInstance",
(string) (len=2) "id": (string) (len=8) "bd65d028",
(string) (len=8) "progress": (float64) 1,
(string) (len=9) "stateType": (string) (len=17) "CenteringPosition"
},
(string) (len=8) "d1e225e7": (map[string]interface {}) (len=4) {
(string) (len=2) "id": (string) (len=8) "d1e225e7",
(string) (len=8) "progress": (float64) 1,
(string) (len=9) "stateType": (string) (len=10) "TargetCold",
(string) (len=5) "class": (string) (len=13) "StateInstance"
},
This is nice, but I'd like to be be able to grab the status of a given instance (e.g. ask for the progress of X-Calibration, get 1 in return). So I created some structs:
type Message struct {
Tasks []struct {
Class string
Id string
Instances []struct {
Something []struct {
Class string
Id string
StateType string
}
}
}
}
Tasks gets unmarshalled, but none of the instances:
====================== Error: json: cannot unmarshal object into Go value of type []struct { Something struct { Class string; Id string; StateType string } }
(*buccaneer.Message)(0xc082002c40)({
Tasks: ([]struct { Class string; Id string; Instances []struct { Something struct { Class string; Id string; StateType string } } }) (len=1 cap=4) {
(struct { Class string; Id string; Instances []struct { Something struct { Class string; Id string; StateType string } } }) {
Class: (string) (len=4) "Task",
Id: (string) (len=8) "5fee231a",
Instances: ([]struct { Something struct { Class string; Id string; StateType string } }) <nil>
}
}
})
And note the something struct in there. I don't know what to name that struct, as the instances have unique names (i.e. each "sub-task" has a different ID)
Any help is much appreciated :)
EDIT: Updated to show that unmarshalling is partially working, except for the Instances

Instances needs to be a map, because it's an object.
You don't need Something inside of it, since it's already a map[string]struct{...}.
(Also Id should really be ID, but that's a stylistic nitpick.)
The type:
type Message struct {
Tasks []struct {
Class string
ID string
Instances map[string]struct {
Class string
ID string
StateType string
}
}
}
Playground: http://play.golang.org/p/r-wjaEiwP0.

In addition to the answer posted by Ainar-G, I discovered this website which generates structs for you, based on whatever JSON you paste in.
When I pasted in the JSON from my printer, the resulting code was a total mess, not nearly as compact and complete as the answers provided above, but if you have some simple JSON that you'd like to quickly convert to a struct, this might be of use to you.

Related

Unwinding nested data responses into structs when a slice is part of the response

I'd like to get some input into how you would all go about unwinding nested response data into custom structs. Below is an example of the data i'm being returned. I'm trying to get to the user data.
{
"_links": {
"self": {
"href": "/api/v2/user-search/default/test?after=1585612800000&limit=20&offset=0&q=johnsmith%40test.com",
"type": "application/json"
}
},
"totalCount": 1,
"items": [
{
"lastPing": "2020-04-30T02:56:10.430867577Z",
"environmentId": "xxxx",
"ownerId": "xxxx",
"user": {
"key": "johnsmith#test.com",
"email": "johnsmith#test.com",
"firstName": "john",
"lastName": "smith"
},
"_links": {
"parent": {
"href": "/api/v2/users/default/test",
"type": "application/json"
},
"self": {
"href": "/api/v2/users/default/test/johnsmith#test.com",
"type": "application/json"
},
"settings": {
"href": "/api/v2/users/default/test/johnsmith#test.com/flags",
"type": "text/html"
},
"site": {
"href": "/default/test/users/johnsmith#test.com",
"type": "text/html"
}
}
}
]
}
Currently I'm doing the below
respData := map[string][]map[string]map[string]interface{}{}
json.Unmarshal(respBody, &respData)
userData := respData["items"][0]["user"]
I'd love to be able to unmarshal it into a custom struct but I can't seem to get it to work. The nested slice that the user object sits within is what keeps throwing me off.
type User struct {
Key string `json:"key"`
Email string `json:"email"`
FirstName string `json:"firstName"`
LastName string `json:"lastName"`
}
type LinkInfo struct {
Href string `json:"href"`
Type string `json:"type"`
}
type Item struct {
LastPing time.Time `json:"lastPing"`
EnvironmentID string `json:"environmentId"`
OwnerID string `json:"ownerId"`
User User `json:"user"`
Links LinkInfo `json:"_links"`
Self LinkInfo `json:"self"`
Settings LinkInfo `json:"settings"`
Parent LinkInfo `json:"parent"`
Site LinkInfo `json:"site"`
}
type ItemDetails struct {
Links LinkInfo `json:"_links"`
TotalCount int `json:"total_count"`
Items []Item
}
Can you try this?
https://play.golang.org/p/S_CUN0XEh-d
From what you mentioned it sounds like you were on the right track. Your JSON is pretty large, so let me give you a smaller example similar to the part you mentioned you're having trouble with (the user object inside the items list).
type response struct {
TotalCount int `json:"totalCount"`
Items []*itemStruct `json:"items"`
}
type itemStruct struct {
LastPing string `json:"lastPing"`
User *userStruct `json:"user"`
}
type userStruct struct {
Key string `json:"key"`
}
Basically to map to a JSON list of objects, just put a field like this in your struct: Objects []*structWhichMapsToMyObject
Edit: Here's the code running in Go Playground: https://play.golang.org/p/EvSvv-2s8y8
If you want this:
"user": {
"key": "johnsmith#test.com",
"email": "johnsmith#test.com",
"firstName": "john",
"lastName": "smith"
}
Declare a matching Go struct:
type User struct {
Key string `json:"key"`
Email string `json:"email"`
FirstName string `json:"firstName"`
LastName string `json:"lastName"`
}
Then, since the user's parent object looks like this:
"items": [
{
"lastPing": "2020-04-30T02:56:10.430867577Z",
"environmentId": "xxxx",
"ownerId": "xxxx",
"user": { ... },
"_links": { ... }
}
]
you also need to declare a matching Go struct for that (you can omit fields you don't need):
type Item struct {
User User `json:"user"`
}
and then the parent of the parent:
{
"_links": {
"self": {
"href": "/api/v2/user-search/default/test?after=1585612800000&limit=20&offset=0&q=johnsmith%40test.com",
"type": "application/json"
}
},
"totalCount": 1,
"items": [ ... ]
}
and the matching Go struct for the grandparent, again, include only the fields you need:
type ResponseData struct {
Items []Item `json:"items"`
}
Once you have this you can decode the json into an instance of ResponseData:
var rd ResponseData
if err := json.Unmarshal(data, &rd); err != nil {
panic(err)
}
for _, item := range rd.Items {
fmt.Println(item.User)
}
https://play.golang.com/p/7yavVSBcHQP

Nested Go Structs for JSON marshaling with optional structs

I'm trying to initialize a nested struct to then marshal into json for an API response. The challenge I'm hitting is one of the components (a slice of structs) can have n number of members but of one of two possible types (text, image).
The JSON I want to create looks like this:
{
"messages": [
{
"message_parts": [
{
"text": {
"content": "dfdffd"
}
},
{
"image": {
"url": "https://image.jpg"
}
}
],
"actor_id": "44444444",
"actor_type": "agent"
}
],
"channel_id": "44444444",
"users": [
{
"id": "44444444"
}
]
}
In the message_parts slice, that can contain at least one of text or image but possibly one of each.
My structs look like this currently:
Type messagePayload struct {
Messages []Messages `json:"messages"`
Status string `json:"status,omitempty"`
ChannelID string `json:"channel_id"`
Users []Users `json:"users"`
}
type Messages struct {
MessageParts []MessageParts `json:"message_parts"`
ActorID string `json:"actor_id"`
ActorType string `json:"actor_type"`
}
type Users struct {
ID string `json:"id"`
}
type Text struct {
Content string `json:"content,omitempty"`
}
type MessageParts struct {
Text *Text `json:"text,omitempty"`
Image *Image `json:"image,omitempty"`
}
type Image struct {
URL string `json:"url,omitempty"`
}
I'm really struggling to initialize this in a way that not show up in the json if they're not present.
here's where I'm at but it obviously doesn't work:
payload := &messagePayload{
Messages: []Messages{
{
MessageParts: []MessageParts{
{
&Text{
Content: text,
},
},
{
&Image{
URL: mediaurl,
},
},
},
ActorID: agentID,
ActorType: "agent",
}},
ChannelID: channelid,
Users: []Users{
{
ID: user,
},
},
}
EDIT:
Thanks to hint below and a few other findings, I've found the best way is to initalize the payload and then add the slices for text and images as needed:
https://play.golang.org/p/Pmmv00spcI6
As noted above, I was able to find a solution- you need to initialize the payload without the text or image data, then append them to the MessageParts slice:
package main
import (
"encoding/json"
"fmt"
)
type messagePayload struct {
Messages []Messages `json:"messages"`
Status string `json:"status,omitempty"`
ChannelID string `json:"channel_id"`
Users []Users `json:"users"`
}
type Messages struct {
MessageParts []MessageParts `json:"message_parts"`
ActorID string `json:"actor_id"`
ActorType string `json:"actor_type"`
}
type Users struct {
ID string `json:"id"`
}
type Text struct {
Content string `json:"content,omitempty"`
}
type MessageParts struct {
Text *Text `json:"text,omitempty"`
Image *Image `json:"image,omitempty"`
}
type Image struct {
URL string `json:"url,omitempty"`
}
func main() {
payload := &messagePayload{
Messages: []Messages{
{
MessageParts: []MessageParts{
},
ActorID: "id",
ActorType: "agent",
}},
ChannelID: "cid",
Users: []Users{
{
ID: "user1",
},
},
}
var text= new(MessageParts)
text.Text = &Text{Content: "LOL"}
var image = new(MessageParts)
image.Image= &Image{URL: "https://"}
payload.Messages[0].MessageParts = append(payload.Messages[0].MessageParts, *text)
payload.Messages[0].MessageParts = append(payload.Messages[0].MessageParts, *image)
m, err := json.Marshal(payload)
if err != nil {
fmt.Println("Error, ", err)
return
}
fmt.Println(string(m))
}

JSON string array without Keys into a GoLang struct

I am trying to parse JSON that is submitted from a POST request into a struct in a web service to send email. The following JSON is submitted in the body of the request.
{
"body": {
"template": "abctemplate.html",
"params": {
"name": "Chase",
"email": "1234#gmail.com"
}
},
"to": [
"abc#gmail.com",
"xyz#gmail.com"
],
"cc": [
"xxx#example.com",
"yyy#example.com"
],
"replyTo": {
"email": "aaa#gmail.com",
"name": "Jack"
},
"bcc": "ccc#gmail.com",
"subject": "Hello, world!"
}
This is mapped and read into the following struct
type emailData struct {
Body struct {
Template string `json:"template"`
Params map[string]string `json:"params"`
} `json:"body"`
To map[string]string `json:"To"` // TODO This is wrong
CC string `json:"cc"` // TODO this is wrong
ReplyTo struct {
Email string `json:"email"`
Name string `json:"name"`
}
BCC string `json:"bcc"`
Subject string `json:"subject"`
}
Both the 'to' and 'cc' JSON fields are string arrays of unknown length and do not have keys. Is there a way to map the string arrays into the struct fields? I've tried the two different ways where there are // TODO tags with no luck. Thanks!
Both cc and to are json arrays which you can unmarshal into Go slices without worrying about the length.
type emailData struct {
Body struct {
Template string `json:"template"`
Params map[string]string `json:"params"`
} `json:"body"`
To []string `json:"to"`
CC []string `json:"cc"`
ReplyTo struct {
Email string `json:"email"`
Name string `json:"name"`
}
BCC string `json:"bcc"`
Subject string `json:"subject"`
}
https://play.golang.org/p/Pi_5aSs922
Use the below to convert the json to struct in golang:
Json-goStruct
Take care of the maps, which might be go struct and vice-versa.

Encoding nested JSON in Go

I've had a lot of trouble finding an example of this. Most of the information on the internet is about decoding JSON.
I'd like to serialize some data into nested JSON, like for example:
{
"item": {
"title": "Items",
"properties": [
{
"num": 1,
"name": "Item 1"
},
{
"num": 2,
"name": "Item 2"
}
]
}
}
I know how to marshal data with a flat struct, but how do I put data into a struct that can be serialized with nesting?
http://play.golang.org/p/nDKmv1myTD
I found this tool that generates a struct from a JSON schema, but I don't understand how to get data into the sub structs.
http://mholt.github.io/json-to-go/
type Example struct {
Item struct {
Title string `json:"title"`
Properties []struct {
Num int `json:"num"`
Name string `json:"name"`
} `json:"properties"`
} `json:"item"`
}
This tool you found is nice, but I would not use it. It makes it difficult to initialize the structs.
Init example with your snippet: (http://play.golang.org/p/_Qw3Qp8XZh)
package main
import (
"encoding/json"
"fmt"
)
type Example struct {
Item struct {
Title string `json:"title"`
Properties []struct {
Num int `json:"num"`
Name string `json:"name"`
} `json:"properties"`
} `json:"item"`
}
func main() {
info := &Example{
Item: struct {
Title string `json:"title"`
Properties []struct {
Num int `json:"num"`
Name string `json:"name"`
} `json:"properties"`
}{
Title: "title",
Properties: []struct {
Num int `json:"num"`
Name string `json:"name"`
}{
{Num: 0, Name: "name0"},
{Num: 1, Name: "name1"},
},
},
}
b, err := json.Marshal(info)
if err != nil {
fmt.Println(err)
return
}
fmt.Println(string(b))
}
Result (pretty printed):
{
"item": {
"title": "title",
"properties": [
{
"num": 0,
"name": "name0"
},
{
"num": 1,
"name": "name1"
}
]
}
}
I think it is better to use named struct vs anonymous nested ones.
Same example with named structs: http://play.golang.org/p/xm7BXxEGTC
package main
import (
"encoding/json"
"fmt"
)
type Example struct {
Item Item `json:"item"`
}
type Item struct {
Title string `json:"title"`
Properties []Property `json:"properties"`
}
type Property struct {
Num int `json:"num"`
Name string `json:"name"`
}
func main() {
info := &Example{
Item: Item{
Title: "title",
Properties: []Property{
{Num: 0, Name: "name0"},
{Num: 1, Name: "name1"},
},
},
}
b, err := json.Marshal(info)
if err != nil {
fmt.Println(err)
return
}
fmt.Println(string(b))
}
It is the exact same thing, but I find it more clear and easy to use.

Unable to decode the JSON response

I have the following response from a graph api
{
"data": [
{
"name": "Mohamed Galib",
"id": "502008940"
},
{
"name": "Mebin Joseph",
"id": "503453614"
},
{
"name": "Rohith Raveendranath",
"id": "507482441"
}
],
"paging": {
"next": "https://some_url"
}
}
I have a struct as follows
type Item struct {
Name, Id string
}
I wanted to parse the response and get an array of Item, How do I do that?
You need to update your struct like so:
type Item struct {
Name string `json:"name"`
Id string `json:"id"`
}
and add a struct to represent the wrapper:
type Data struct {
Data []Item `json:"data"`
}
You can then use json.Unmarshal to populate a Data instance.
See the example in the docs.