Cannot Unmarshal JSON String to struct in Go - json

I have a string in my Go module which is the body of a HTTP response. it looks something like this:
bodyString = `{"firstname": "foo", "lastname": "bar", "username": "foobar"}`
I want to convert it to the following Go struct:
type Params struct {
FirstName string
LastName string
Username string
PasswordHash string
EmailAddress string
}
I attempt to do so using the following:
var jsonMap map[string]interface{}
json.Unmarshal([]byte(bodyString), &jsonMap)
paramsIn.FirstName = jsonMap["firstname"].(string)
paramsIn.LastName = jsonMap["lastname"].(string)
paramsIn.Username = jsonMap["username"].(string)
paramsIn.PasswordHash = jsonMap["passwordhash"].(string)
paramsIn.EmailAddress = jsonMap["emailaddress"].(string)
However the unmarshal fails to match the data in the string to the appropriate keys. i.e. what is stored in the jsonMap variable is only empty strings.
I clearly am doing something wrong and I haven't worked with json in Go very much. If any one can help or show me the correct way to unmarshal json data from a string that would be great, thanks.

Golang will convert your struct's field name (CammelCase) to snake_case (default). So, if you have struct like :
type Params struct {
FirstName string
LastName string
Username string
PasswordHash string
EmailAddress string
}
The JSON from the struct will be :
{
"first_name":"bla",
"last_name":"bla",
"user_name":"bla",
"password_hash":"ewedsads",
"email_address":"bla#gmail.com"
}
But you can customize the JSON field name by json tag, example :
type Params struct {
FirstName string `json:"firstname"`
LastName string `json:"lastname"`
Username string `json:"username"`
PasswordHash string `json:"passwordhash"`
EmailAddress string `json:"emailaddress"`
}
Then you can change your code like this :
package main
import (
"encoding/json"
"fmt"
)
type Params struct {
FirstName string `json:"firstname"`
LastName string `json:"lastname"`
Username string `json:"username"`
PasswordHash string `json:"passwordhash"`
EmailAddress string `json:"emailaddress"`
}
func main() {
paramsIn := Params{}
bodyString := `{"firstname": "foo", "lastname": "bar", "username": "foobar"}`
json.Unmarshal([]byte(bodyString), &paramsIn)
fmt.Println(paramsIn)
}

Use go tag like this:
type Params struct {
FirstName string `json:"firstname"`
LastName string `json:"lastname"`
Username string `json:"username"`
PasswordHash string
EmailAddress string
}

This may not be the solution to everyone who is experience this problem, and you should check out the other answers on this post and other similar SO posts.
For me the issue was how the double quote character was being treated ("). My json string was being created from the body of a http response, which was being fed by a curl command in another terminal window. I was writing my curl commands in MacOS's standard text editor but I was in RTF (Rich Text Format) which formatted my " character as something else (“) (The difference is miniscule but it was enough to completely stump me for hours!). There is no issue with the code above once I used the proper double quote character in the http request body.
If I am unclear, please comment and I can clarify more.

Related

golang how i can get value from []interface {} type

i have var (name result["error_type"]) with type
[]interface {}
and value
[map[reason:map[phone:empty] send_at:1.636697291e+09 status:error]]
how i cat get value from type []interface {}
example result["error_type"]["128"]["reason"]["phone"]
this type i got from
var result map[string]interface{}
json.NewDecoder(r.Body).Decode(&result)
r.Body has Json
{
"offer_name":"EbcBankruptcy",
"offer_id":"288",
"partner_name":"середов",
"partner_id":"1",
"type_system":"gb",
"status":"success",
"date_request":"2021-01-02 11:03",
"bank_name":"alfa",
"bank_id":"1",
"type_product":"1",
"error_type":{"128": [{"reason": {"phone": "Отсутствует обязательное поле номер телефона"}, "status": "error", "send_at": 1636697291}], "200": [{"reason": {"phone": "Отсутствует обязательное поле номер телефона"}, "status": "error", "send_at": 1636697281}]},
"request_id":"1"
}
also i dont create structure error_type for json.NewDecoder parse because i dont know what kind id will be in error_type in json (128, 200, 300)
i try get value
test["reason"]["phone"]
but, its not work
also cast to
map[string]interface{}
its not work
As far as the understanding from the question, what I understand is you can format the data as following.
type Payload struct {
OfferName string `json:"offer_name"`
OfferID string `json:"offer_id"`
PartnerName string `json:"partner_name"`
PartnerID string `json:"partner_id"`
TypeSystem string `json:"type_system"`
Status string `json:"status"`
DateRequest string `json:"date_request"`
BankName string `json:"bank_name"`
BankID string `json:"bank_id"`
TypeProduct string `json:"type_product"`
// you can use the type map of array of error data here
ErrorType map[string][]ErrorData `json:"error_type"`
RequestID string `json:"request_id"`
}
type ErrorData struct {
Reason Reason `json:"reason"`
Status string `json:"status"`
SendAt int `json:"send_at"`
}
type Reason struct {
Phone string `json:"phone"`
}
With the following you can unmarshal the data to
fmt.Printf("%+v", p.ErrorType["128"][0].Reason)
In the case you are unable to know the keys of the map, you can still range over the map values and get the data.
Here is the playground link https://go.dev/play/p/oTF0JUwOj0D

How to modify big JSON file

I have a large json file with multiple levels of nesting. Now I need to modify the value of each key in this file with the Go code. I know of two methods: the first is to obtain each key and then modify its value, but there is no doubt that this method is too complicated and prone to errors. The second method is to serialize the entire json file into a struct, then modify the struct field, and then deserialize it. However, this case needs to define a struct of several hundred lines, which is also very complicated.
Is there any other way?
for example my json is like this, but more bigger, 100+ lines :
{
"user": [{
"cdb_id":"",
"firstname":"Tom",
"lastname":"Bradley",
"phone":14155555555,
"email":"tom#gmail.com",
"address":[{
"street":"4343 shoemaker ave",
"city":"Brea",
"zip":"92821",
"country":"USA"
}],
"authenticators":[{
"name":"Lisa Hayden",
"phone":15625555555
},{
"name":"Pavan M",
"phone":17145555555
}],
"voice_sig":"242y5-4546-555kk54-437879ek545",
"voicesig_created_time":"2017-08-02T21:27:44+0000",
"status":"verified"
}]
}
I need modify "cdb_id"/"lastname"/"street"/"phone"/ "voice_sig".....all these keys' value, Except make a struct or get the keys' value one by one and modify, do i have any other way?
The new values for those keys will be POST request from Web Pages.
You can use json pointer:
https://godoc.org/github.com/go-openapi/jsonpointer
Or, you can read it in a map[string]interface{} and work your way through, but that gets tedious.
Now my way is use this website, switch my Json to struct, and modify one by one. But it just think this is not so good way, so i am eagering for a better way.
http://json2struct.mervine.net/
type MyJsonName struct {
User []struct {
Address []struct {
City string `json:"city"`
Country string `json:"country"`
Street string `json:"street"`
Zip string `json:"zip"`
} `json:"address"`
Authenticators []struct {
Name string `json:"name"`
Phone int `json:"phone"`
} `json:"authenticators"`
CdbID string `json:"cdb_id"`
Email string `json:"email"`
Firstname string `json:"firstname"`
Lastname string `json:"lastname"`
Phone int `json:"phone"`
Status string `json:"status"`
VoiceSig string `json:"voice_sig"`
VoicesigCreatedTime string `json:"voicesig_created_time"`
} `json:"user"`
}

Empty fields while doing json unmarshal

I have written this piece of code. I can't figure out why it is giving empty fields when i run it. I have tried Uppercase variables. Still doesn't work.
package main
import (
"encoding/json"
"fmt"
)
type FirstName struct {
Name1 string
}
type SecondName struct {
Name2 string
}
type Person struct {
Details FirstName
MoreDetails SecondName
}
func main() {
var someGuy Person
var data = []byte(`{"details": {"firstName": "alan"}, "moreDetails": {"surName": "Turing"}}`)
json.Unmarshal(data, &someGuy)
fmt.Println(someGuy)
}
Your JSON to Go mapping is incorrect. To have automatic matching between struct fields and the JSON properties during the unmarshaling process, the field names (and not the name of their types) must match the properties in JSON (in a case in-sensitive manner):
type FirstName struct {
FirstName string
}
type SecondName struct {
SurName string
}
type Person struct {
Details FirstName
MoreDetails SecondName
}
With this, the output is (try it on the Go Playground):
{{alan} {Turing}}
If you want to keep those names in Go, another option is to provide a mapping between Go struct fields and JSON property names. For this you may use struct tags:
type FirstName struct {
Name1 string `json:"firstName"`
}
type SecondName struct {
Name2 string `json:"surName"`
}
Output will be the same. Try this one on the Go Playground.
For an introduction and more details about struct tags, see What are the use(s) for tags in Go?

Creating struct to read from API in Go

I'm working on a project and it is my first time using Go.
The project queries a number of APIs and for the most part I have had no trouble getting this working.
Coming from a PHP background, creating Go type definitions for my JSON responses is a little different.
I am stuck on one API, a Magento API, that returns a JSON response like so:
{
"66937": {
"entity_id": "66937",
"website_id": "1",
"email": "email#email.com",
"group_id": "1",
"created_at": "2017-08-11 02:09:18",
"disable_auto_group_change": "0",
"firstname": "Joe",
"lastname": "Bloggs",
"created_in": "New Zealand Store View"
},
"66938": {
"entity_id": "66938",
"website_id": "1",
"email": "email1#email.comm",
"group_id": "1",
"created_at": "2017-08-11 02:16:41",
"disable_auto_group_change": "0",
"firstname": "Jane",
"lastname": "Doe",
"created_in": "New Zealand Store View"
}
}
I have been using a tool, JSON-to-Go, to help me create the struct types, however it doesn't look quite right for this style of response:
type AutoGenerated struct {
Num0 struct {
EntityID string `json:"entity_id"`
WebsiteID string `json:"website_id"`
Email string `json:"email"`
GroupID string `json:"group_id"`
CreatedAt string `json:"created_at"`
DisableAutoGroupChange string `json:"disable_auto_group_change"`
Firstname string `json:"firstname"`
Lastname string `json:"lastname"`
CreatedIn string `json:"created_in"`
} `json:"0"`
Num1 struct {
EntityID string `json:"entity_id"`
WebsiteID string `json:"website_id"`
Email string `json:"email"`
GroupID string `json:"group_id"`
CreatedAt string `json:"created_at"`
DisableAutoGroupChange string `json:"disable_auto_group_change"`
Firstname string `json:"firstname"`
Lastname string `json:"lastname"`
CreatedIn string `json:"created_in"`
} `json:"1"`
}
All I am interested in is the inner JSON - the stuff to actually do with the customer. I am looping over this to extract some information.
How do I create the required struct to read from this?
I have looked at any number of documents or articles but they tend to use more simple JSON responses as examples.
For your JSON structure following might suit well.
Play Link: https://play.golang.org/p/ygXsdYALCb
Create a struct called Info or name you prefer also customize your field names as you like.
type Info struct {
EntityID string `json:"entity_id"`
WebsiteID string `json:"website_id"`
Email string `json:"email"`
GroupID string `json:"group_id"`
CreatedAt string `json:"created_at"`
DisableAutoGroupChange string `json:"disable_auto_group_change"`
Firstname string `json:"firstname"`
Lastname string `json:"lastname"`
CreatedIn string `json:"created_in"`
}
And create map of Info struct and the unmarshal it.
var result map[string]Info
if err := json.Unmarshal(jsonBytes, &result); err != nil {
fmt.Println(err)
}
fmt.Printf("%+v", result)
EDIT:
As asked in the comment, adding for example:
fmt.Println("Accessing unmarshal values:")
for key, info := range result {
fmt.Println("Key:", key)
fmt.Printf("Complete Object: %+v\n", info)
fmt.Println("Individual value, typical object field access:")
fmt.Println("EntityID:", info.EntityID)
fmt.Println("Email:", info.Email)
}
Well, first, I don't like the auto-generated struct definitions there. I would change that to look like this
type Customer struct {
EntityID string `json:"entity_id"`
WebsiteID string `json:"website_id"`
Email string `json:"email"`
GroupID string `json:"group_id"`
CreatedAt string `json:"created_at"`
DisableAutoGroupChange string `json:"disable_auto_group_change"`
Firstname string `json:"firstname"`
Lastname string `json:"lastname"`
CreatedIn string `json:"created_in"`
}
You may want to create a wrapper type
type Customers map[string]Customer
This should work with your json that you've provided. To put this together
customers := Customers{}
err := json.Unmarshal(jsonBytes, &customers)
Both #Xibz and #jeevatkm provided great solutions. However, in some cases, not all JSON structures can be unmarshalled into Go structures. You may have to define your own decoding functions.
You can also try gorilla's schema package if you have to define your own decoding function for particular data type or structures.
https://github.com/gorilla/schema

Golang and JSON with array of struct [duplicate]

This question already has answers here:
My structures are not marshalling into json [duplicate]
(3 answers)
Closed 7 months ago.
I would like to create a JSON of a GatewayInfo where the type are defined like this:
type SpanInfo struct {
imsi string
network string
network_status string
signal_quality int
slot int
state string
}
type GatewayInfo []SpanInfo
The gateway information is created with:
var gatewayInfo = make(GatewayInfo, nb_spans)
To create the JSON, I use the json.Marshal function:
gatewayInfo := getGatewayInfo(spans)
log.Printf("Polling content: %s\n", gatewayInfo)
jsonInfo, _ := json.Marshal(gatewayInfo)
log.Printf("jsonInfo: %s\n", jsonInfo)
Unfortunately the result is not what I was expecting:
2015/02/09 13:48:26 Polling content: [{652020105829193 20801 Registered (Roaming) %!s(int=17) %!s(int=2) } {652020105829194 20801 Registered (Roaming) %!s(int=16) %!s(int=3) } {652020105829192 20801 Registered (Roaming) %!s(int=19) %!s(int=1) } {652020105829197 20801 Registered (Roaming) %!s(int=19) %!s(int=4) }]
2015/02/09 13:48:26 jsonInfo: [{},{},{},{}]
As we can see, the GatewayInfo instance has the SpanInfo, but in the JSON I have empty SpanInfo.
Your struct fields must be exported (field is exported if it begins with a capital letter) or they won't be encoded:
Struct values encode as JSON objects. Each exported struct field
becomes a member of the object
To get the JSON representation as probably expected change the code to this:
type SpanInfo struct {
IMSI string `json:"imsi"`
Network string `json:"network"`
NetworkStatus string `json:"network_status"`
SignalQuality int `json:"signal_quality"`
Slot int `json:slot"`
State string `json:"state"`
}
type GatewayInfo []SpanInfo
The json package can only serialize exported fields of your struct. Change your struct to start all fields with an uppercase letter so they can be included in the output:
type SpanInfo struct {
Imsi string
Network string
Network_status string
Signal_quality int
Slot int
State string
}
Read the documentation of json.Marshal() for details and more information.
This is not a new answer. It is just consolidation of comments on the
accepted answer.
From ORIGINAL Query
type SpanInfo struct {
imsi string
network string
network_status string
signal_quality int
slot int
state string
}
From Answer and comments - Please note that the first char of each field in struct is now in UPPER case along with json representation added to each field
type SpanInfo struct {
IMSI string `json:"imsi"`
Network string `json:"network"`
NetworkStatus string `json:"network_status"`
SignalQuality int `json:"signal_quality"`
Slot int `json:slot"`
State string `json:"state"`
}