(un)marshalling json golang not working - json

I'm playing with Go and am stumped as to why json encode and decode don't work for me
I think i copied the examples almost verbatim, but the output says both marshal and unmarshal return no data. They also don't give an error.
can anyone hint to where i'm going wrong?
my sample code: Go playground
package main
import "fmt"
import "encoding/json"
type testStruct struct {
clip string `json:"clip"`
}
func main() {
//unmarshal test
var testJson = "{\"clip\":\"test\"}"
var t testStruct
var jsonData = []byte(testJson)
err := json.Unmarshal(jsonData, &t)
if err != nil {
fmt.Printf("There was an error decoding the json. err = %s", err)
return
}
fmt.Printf("contents of decoded json is: %#v\r\n", t)
//marshal test
t.clip = "test2"
data, err := json.Marshal(&t)
if err != nil {
fmt.Printf("There was an error encoding the json. err = %s", err)
return
}
fmt.Printf("encoded json = %s\r\n", string(data))
}
output:
contents of decoded json is: main.testStruct{clip:""}
encoded json = {}
in both outputs I would have expected to see the decoded or encoded json

For example,
package main
import "fmt"
import "encoding/json"
type testStruct struct {
Clip string `json:"clip"`
}
func main() {
//unmarshal test
var testJson = "{\"clip\":\"test\"}"
var t testStruct
var jsonData = []byte(testJson)
err := json.Unmarshal(jsonData, &t)
if err != nil {
fmt.Printf("There was an error decoding the json. err = %s", err)
return
}
fmt.Printf("contents of decoded json is: %#v\r\n", t)
//marshal test
t.Clip = "test2"
data, err := json.Marshal(&t)
if err != nil {
fmt.Printf("There was an error encoding the json. err = %s", err)
return
}
fmt.Printf("encoded json = %s\r\n", string(data))
}
Output:
contents of decoded json is: main.testStruct{Clip:"test"}
encoded json = {"clip":"test2"}
Playground:
http://play.golang.org/p/3XaVougMTE
Export the struct fields.
type testStruct struct {
Clip string `json:"clip"`
}
Exported identifiers
An identifier may be exported to permit access to it from another
package. An identifier is exported if both:
the first character of the identifier's name is a Unicode upper case letter (Unicode class "Lu"); and
the identifier is declared in the package block or it is a field name or method name.
All other identifiers are not exported.

Capitalize names of structure fields
type testStruct struct {
clip string `json:"clip"` // Wrong. Lowercase - other packages can't access it
}
Change to:
type testStruct struct {
Clip string `json:"clip"`
}

In my case, my struct fields were capitalized but I was still getting the same error.
Then I noticed that the casing of my fields was different. I had to use underscores in my request.
For eg:
My request body was:
{
"method": "register",
"userInfo": {
"fullname": "Karan",
"email": "email#email.com",
"password": "random"
}
}
But my golang struct was:
type AuthRequest struct {
Method string `json:"method,omitempty"`
UserInfo UserInfo `json:"user_info,omitempty"`
}
I solved this by modifying my request body to:
{
"method": "register",
"user_info": {
"fullname": "Karan",
"email": "email#email.com",
"password": "random"
}
}

Related

How to write unit test failure for json.NewDecoder.Decode?

I have to write unit tests for a function and this function uses json.NewDecoder.Decode
var infos models.RegisterInfos // struct with json fields
err := json.NewDecoder(r.Body).Decode(&infos)
if err != nil {
// do something
}
How can I simulate an error in a unit test (using the testing package) for json.NewDecoder(r.Body).Decode(&infos) ? I tried looking in the NewDecoder and Decode source code but I couldn't find anything that can generate an error in just a few lines.
you could send a body like <invalid json> as example:
func main() {
body := "<invalid json>"
var infos RegisterInfos // struct with json fields
err := json.NewDecoder(strings.NewReader(body)).Decode(&infos)
if err != nil {
fmt.Println(err)
}
}
See https://go.dev/play/p/44E99D0eQou
Feed it an invalid input, or decode into an invalid output:
package main
import (
"encoding/json"
"fmt"
"strings"
"testing"
)
type Message struct {
Name string
}
func TestDecodeFail(t *testing.T) {
for _, tc := range []struct {
in string
desc string
out any
}{
{`{Name: "Bobby"}`, "key without quotes", &Message{}},
{`{"Name": "Foo"a}`, "extra character", &Message{}},
{`{"Name": "Foo"}`, "bad destination", &struct{ Name int64 }{}},
{`{"Name": "Foo` + "\u001a" + `"}`, "invalid character", &Message{}},
{`{"Name": "Foo"}`, "unmarshal to nil", (*Message)(nil)},
} {
err := decode(tc.in, tc.out)
if err != nil {
fmt.Printf("%s -> %s, %T\n", tc.desc, err.Error(), err)
}
}
}
func decode(in string, out any) error {
return json.NewDecoder(strings.NewReader(in)).Decode(out)
}
Outputs:
key without quotes -> invalid character 'N' looking for beginning of object key string, *json.SyntaxError
extra character -> invalid character 'a' after object key:value pair, *json.SyntaxError
bad destination -> json: cannot unmarshal string into Go struct field .Name of type int64, *json.UnmarshalTypeError
invalid character -> invalid character '\x1a' in string literal, *json.SyntaxError
unmarshal to nil -> json: Unmarshal(nil *main.Message), *json.InvalidUnmarshalError

Can't unmarshal json with base64 encoded byte slice in go

I'm having a hard time unmarshalling a struct that is sent to me from an AWS lambda function.
All of the pkidata fields are empty no matter what I do. I created this code sample to isolate the process :
package main
import (
"encoding/base64"
"encoding/json"
"fmt"
)
func main() {
mystring := `{"error":{"errormessage": "errormessagetest", "errortype": "errortypetest"}, "pkidata":{"certificate":"Q2VydGlmaWNhdGU6CiAgICBEYXRhOgogICAgICAgIFZlcnNpb246IDMgKDB4MikKICAgICAgICBTZXJpYWwgTnVtYmVyOgogICAgICAgICAgICBhYTo5MzpiMTo4MDphYjpiYTplYzo5ZTo4MzpmOTo2OTo5NToyNDpkZjo5Njo2OQogICAgU2lnbmF0dXJlIEFsZ29yaXRobTogc2hhMjU2V2l0aFJTQUVuY3J5cHRpb24KICAgICAgICBJc3N1ZXI6IENOPXJhcGhvZXN0ZXIKICAgICAgICBWYWxpZGl0eQogICAgICAgICAgICBOb3QgQmVmb3JlOiBNYXIgMjcgMTk6NDM6MTQgMjAyMiBHTVQKICAgICAgICAgICAgTm90IEFmdGVyIDogSnVuIDI5IDE5OjQzOjE0IDIwMjQgR01UCiAgICAgICAgU3ViamVjdDogQ049cmFwaG9lc3RlcgogICAgICAgIFN1YmplY3QgUHVibGljIEtleSBJbmZvOgogICAgICAgICAgICBQdWJsaWMgS2V5IEFsZ29yaXRobTogcnNhRW5jcnlwdGlvbgogICAgICAgICAgICAgICAgUHVibGljLUtleTogKDIwNDggYml0KQogICAgICAgICAgICAgICAgTW9kdWx1czoKICAgICAgICAgICAgICAgICAgICAwMDpjNjplZjpkNjoxYTo0Mjo0Mjo2ZTo5MTpiODpmNTpiZjoyNToyNTpkODoKICAgICAgICAgICAgICAgICAgICBhNjo4Yjo4MzpmYjo2ZDo4MzpiNjowZTo3Mzo4ZjowYTozMjo5ZjpjNjplMjoKICAgICAgICAgICAgICAgICAgICBmMTo4Njo5NDoxMzo5MjplYjo0MTozNTo4NTpiZDo0ZDpkNzowNzo4ZTo5YjoKICAgICAgICAgICAgICAgICAgICBjYzoyYzpkMjo2Nzo5ZDpjOTo1Mzo4Yjo4ODpjZToxMTozOTpkODpkZTo3MjoKICAgICAgICAgICAgICAgICAgICBhMTphODo4MDozMDpmOTpiYTpiOTo3NTpjNTphMjoxYjpiZToyMDo3Yzo1ZjoKICAgICAgICAgICAgICAgICAgICBhMjpjZjpiNzphMjo2ODo4MzpiMzoxMTplODoyNTo3Mzo2MzoyNjphNzpiODoKICAgICAgICAgICAgICAgICAgICBjMzpjZDpjODplZTozNjphZjo4ZTo4ZjowZTo5NDowMToyMDpmZjowZTo0MToKICAgICAgICAgICAgICAgICAgICBjYjplZDpmYjo3NDowNToyZDo0NDphOToxNjoxZDo0ODpkNDphNDo3NTo0ZjoKICAgICAgICAgICAgICAgICAgICBlZjo0NDozYTphMDpmMDo1ZTo5ZTo5YjoxOTphZTo4MTozMzpkMDpkMTpmYzoKICAgICAgICAgICAgICAgICAgICA0MTphMTo1OTo5Zjo3MDoxZDphYTo3MzoyNTpiNTpiNjo3NDo2Mzo1NzpjZjoKICAgICAgICAgICAgICAgICAgICAzNTplNzpiZToyNjo1MzoxNDpkNTplNzpiMzpmYToyZDpjYzo5NToyMzo0ODoKICAgICAgICAgICAgICAgICAgICBkOTpkNTozNDplMjowNzoxYjoyNzoyMzpiZTpiODpmNzpiNTpkOTo3MDo1ZDoKICAgICAgICAgICAgICAgICAgICAyOTo4NTo3NToyYjpmYTo0MjpmNTo2Mjo1OTozZjowODo3Njo0Nzo2NjpmNzoKICAgICAgICAgICAgICAgICAgICAxYzpiYzozNzo1MjpiMDo3ODoyNzozNDo3MTo3ZTpiNjozZDowNDpjZDoyMzoKICAgICAgICAgICAgICAgICAgICAzMjphNjoxYTo2NzphNDo4ODoyYTowYTpmMjoxMjphNTo3ODpkYjpiZDpjOToKICAgICAgICAgICAgICAgICAgICA1ZToxMTo4Njo0MDpkMTo0MTplZjpkYzo2Yjo4Zjo1NjpmZDo1MjpkYTo0ZjoKICAgICAgICAgICAgICAgICAgICBiNTpmMDozYTpmZTpjNzoxMDo3ZTpjNDo3OTo2MTo1ZTpkYjo5Zjo4NjpjOToKICAgICAgICAgICAgICAgICAgICBmYjpiNwogICAgICAgICAgICAgICAgRXhwb25lbnQ6IDY1NTM3ICgweDEwMDAxKQogICAgICAgIFg1MDl2MyBleHRlbnNpb25zOgogICAgICAgICAgICBYNTA5djMgQmFzaWMgQ29uc3RyYWludHM6IAogICAgICAgICAgICAgICAgQ0E6RkFMU0UKICAgICAgICAgICAgWDUwOXYzIFN1YmplY3QgS2V5IElkZW50aWZpZXI6IAogICAgICAgICAgICAgICAgMDU6M0Y6RDk6MjE6Rjg6MUE6QjM6NDM6OEQ6QzI6NDA6NzI6OUY6MDE6NDg6MTc6NDE6Nzc6Q0U6MzkKICAgICAgICAgICAgWDUwOXYzIEF1dGhvcml0eSBLZXkgSWRlbnRpZmllcjogCiAgICAgICAgICAgICAgICBrZXlpZDozNTpCRjpBOTozRDo1RjozMzozODowQjowNjo3QzpDQTpFQzo0ODo3QzpCRTo3Njo3MzpCRTpEQjpGRQogICAgICAgICAgICAgICAgRGlyTmFtZTovQ049cmFwaG9lc3RlcgogICAgICAgICAgICAgICAgc2VyaWFsOjhGOjE3OkY1OjJBOjhBOkI0OkQwOjY5CgogICAgICAgICAgICBYNTA5djMgRXh0ZW5kZWQgS2V5IFVzYWdlOiAKICAgICAgICAgICAgICAgIFRMUyBXZWIgU2VydmVyIEF1dGhlbnRpY2F0aW9uCiAgICAgICAgICAgIFg1MDl2MyBLZXkgVXNhZ2U6IAogICAgICAgICAgICAgICAgRGlnaXRhbCBTaWduYXR1cmUsIEtleSBFbmNpcGhlcm1lbnQKICAgICAgICAgICAgWDUwOXYzIFN1YmplY3QgQWx0ZXJuYXRpdmUgTmFtZTogCiAgICAgICAgICAgICAgICBETlM6cmFwaG9lc3RlcgogICAgU2lnbmF0dXJlIEFsZ29yaXRobTogc2hhMjU2V2l0aFJTQUVuY3J5cHRpb24KICAgICAgICAgNTg6M2U6MTE6ZmQ6OWM6ZWQ6MTA6YmQ6NmM6NWE6OWY6NzY6OGY6OTc6MGE6YTY6Mzg6ODI6CiAgICAgICAgIDFhOmYwOmJlOjRiOmU4OjEwOmQ4OjNlOmFhOmFlOjIwOmNiOmMzOjI2OjY1OmU3OmI3OmM3OgogICAgICAgICBjNzo2MToxZTpjNjplMjpjYTpmODo1NTo4ZTpkODoxNjozODphZjo3ZjphZDo3Mjo1MzoxYzoKICAgICAgICAgNmE6MzI6NjE6M2U6ZDM6M2E6NWI6NTg6OTU6NDU6ZGE6MGQ6ZTg6ZjQ6NDM6ZWY6NmY6ZmM6CiAgICAgICAgIDE4OmQxOjA4OjA5OjZlOmQ5OjRmOmQ3OjljOmU2OjliOjAzOmU3OjczOjhkOjExOjNhOjkyOgogICAgICAgICBiMDozYTpjOTpiNDo2MjpmNDphODpiMTphMDo0MzpkMzplODoyZjphZDo5Yjo4ZjoyZjpjZjoKICAgICAgICAgZTA6ZTY6YzE6MWM6MDc6N2E6OWY6YjM6NTI6NWU6MzM6ZWU6MjE6NjY6YTU6MDE6YjA6MzQ6CiAgICAgICAgIDUzOjY5OmRjOjIwOmYyOmY4OmJjOjEyOjJiOjNkOjkyOjM3OjdhOjY3OjZiOmJkOjkzOmZlOgogICAgICAgICA1NTpiMTo5YTo4Yjo5NTpiOTpjMDpkYzpiOToxMTo1MDo5ODoyNjo0MzpkOTpkZTo5ZTo2NzoKICAgICAgICAgZTE6NWE6MmI6YTI6ZmU6YWY6ZDE6ODY6NTQ6N2Y6YzE6NjE6Y2E6ZWQ6NjY6N2Y6YWY6ODQ6CiAgICAgICAgIGFiOjAyOmY5OjBhOmUyOmJkOjFhOjQ1OjgzOmU3OjM1Ojc4OjNkOjAzOmRkOjY4OjkyOjhlOgogICAgICAgICA2YTpkODpjZDoyYToxMzplYzphOTowMjo5OTo1Yzo3ZDplZDowMzo2Mzo5Mzo2MDo1ZTpiYToKICAgICAgICAgNDg6ODM6Zjc6ODA6YzA6YzQ6ZDU6YjM6NjI6NGE6MmM6Njc6NTE6YzA6MmM6MDE6NmU6OWM6CiAgICAgICAgIDNkOmExOjQ2OjcwOjZmOmUwOjIxOmYxOjE5OjE1OjJiOmZmOjQwOjNmOjk4OjdlOjMwOjI3OgogICAgICAgICAxYzplZTo5ZDo3ZQotLS0tLUJFR0lOIENFUlRJRklDQVRFLS0tLS0KTUlJRFl6Q0NBa3VnQXdJQkFnSVJBS3FUc1lDcnV1eWVnL2xwbFNUZmxta3dEUVlKS29aSWh2Y05BUUVMQlFBdwpGVEVUTUJFR0ExVUVBd3dLY21Gd2FHOWxjM1JsY2pBZUZ3MHlNakF6TWpjeE9UUXpNVFJhRncweU5EQTJNamt4Ck9UUXpNVFJhTUJVeEV6QVJCZ05WQkFNTUNuSmhjR2h2WlhOMFpYSXdnZ0VpTUEwR0NTcUdTSWIzRFFFQkFRVUEKQTRJQkR3QXdnZ0VLQW9JQkFRREc3OVlhUWtKdWtiajF2eVVsMkthTGcvdHRnN1lPYzQ4S01wL0c0dkdHbEJPUwo2MEUxaGIxTjF3ZU9tOHdzMG1lZHlWT0xpTTRST2RqZWNxR29nREQ1dXJsMXhhSWJ2aUI4WDZMUHQ2Sm9nN01SCjZDVnpZeWFudU1QTnlPNDJyNDZQRHBRQklQOE9RY3Z0KzNRRkxVU3BGaDFJMUtSMVQrOUVPcUR3WHA2YkdhNkIKTTlEUi9FR2hXWjl3SGFwekpiVzJkR05YenpYbnZpWlRGTlhucy9vdHpKVWpTTm5WTk9JSEd5Y2p2cmozdGRsdwpYU21GZFN2NlF2VmlXVDhJZGtkbTl4eThOMUt3ZUNjMGNYNjJQUVROSXpLbUdtZWtpQ29LOGhLbGVOdTl5VjRSCmhrRFJRZS9jYTQ5Vy9WTGFUN1h3T3Y3SEVIN0VlV0ZlMjUrR3lmdTNBZ01CQUFHamdhMHdnYW93Q1FZRFZSMFQKQkFJd0FEQWRCZ05WSFE0RUZnUVVCVC9aSWZnYXMwT053a0J5bndGSUYwRjN6amt3UlFZRFZSMGpCRDR3UElBVQpOYitwUFY4ek9Bc0dmTXJzU0h5K2RuTysyLzZoR2FRWE1CVXhFekFSQmdOVkJBTU1DbkpoY0dodlpYTjBaWEtDCkNRQ1BGL1VxaXJUUWFUQVRCZ05WSFNVRUREQUtCZ2dyQmdFRkJRY0RBVEFMQmdOVkhROEVCQU1DQmFBd0ZRWUQKVlIwUkJBNHdESUlLY21Gd2FHOWxjM1JsY2pBTkJna3Foa2lHOXcwQkFRc0ZBQU9DQVFFQVdENFIvWnp0RUwxcwpXcDkyajVjS3BqaUNHdkMrUytnUTJENnFyaURMd3labDU3Zkh4MkVleHVMSytGV08yQlk0cjMrdGNsTWNhakpoClB0TTZXMWlWUmRvTjZQUkQ3Mi84R05FSUNXN1pUOWVjNXBzRDUzT05FVHFTc0RySnRHTDBxTEdnUTlQb0w2MmIKankvUDRPYkJIQWQ2bjdOU1hqUHVJV2FsQWJBMFUybmNJUEw0dkJJclBaSTNlbWRydlpQK1ZiR2FpNVc1d055NQpFVkNZSmtQWjNwNW40Vm9yb3Y2djBZWlVmOEZoeXUxbWY2K0Vxd0w1Q3VLOUdrV0Q1elY0UFFQZGFKS09hdGpOCktoUHNxUUtaWEgzdEEyT1RZRjY2U0lQM2dNREUxYk5pU2l4blVjQXNBVzZjUGFGR2NHL2dJZkVaRlN2L1FEK1kKZmpBbkhPNmRmZz09Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K","privatekey":"LS0tLS1CRUdJTiBQUklWQVRFIEtFWS0tLS0tCk1JSUV2QUlCQURBTkJna3Foa2lHOXcwQkFRRUZBQVNDQktZd2dnU2lBZ0VBQW9JQkFRREc3OVlhUWtKdWtiajEKdnlVbDJLYUxnL3R0ZzdZT2M0OEtNcC9HNHZHR2xCT1M2MEUxaGIxTjF3ZU9tOHdzMG1lZHlWT0xpTTRST2RqZQpjcUdvZ0RENXVybDF4YUlidmlCOFg2TFB0NkpvZzdNUjZDVnpZeWFudU1QTnlPNDJyNDZQRHBRQklQOE9RY3Z0CiszUUZMVVNwRmgxSTFLUjFUKzlFT3FEd1hwNmJHYTZCTTlEUi9FR2hXWjl3SGFwekpiVzJkR05YenpYbnZpWlQKRk5YbnMvb3R6SlVqU05uVk5PSUhHeWNqdnJqM3RkbHdYU21GZFN2NlF2VmlXVDhJZGtkbTl4eThOMUt3ZUNjMApjWDYyUFFUTkl6S21HbWVraUNvSzhoS2xlTnU5eVY0UmhrRFJRZS9jYTQ5Vy9WTGFUN1h3T3Y3SEVIN0VlV0ZlCjI1K0d5ZnUzQWdNQkFBRUNnZ0VBV0loMzhpdTJ3TnBkUkJDVENhckhRNk96QzJQb2E3b0YySFg3SVZueVB3QTcKVGRFZ3JoOGN4ajJDQVhkWWdFSUxGdTk0SE1yL1dpOHlVcDBWVUpYc05kZGhuZ1ljUk5oeVFLZVhIM1lMOHhOWgphQWkyVWdtNE5FQkg4NWNPWEl2NGw3WFhMRUZ4QVBsTE5sZ0hwQjRuUmxDdks3Qm1tRWJBcHM2b3ZSQis2V3JVCkRBWTc0UDNPTi9EMmxyczVNSzlCdlA1d0FsZml1L0x3MTV4NE1UVEE2MkxHNU1remUxV2d0VGtXdnN0VUNXZUUKYW9zQ21tbVhwUGJvR2ZzRXcvSGpWdnZ6UFNKMDhid0U0Mlk2VUFtMGgzWjdMQ1o5ajJtazBsS2NpdStzYndoKwpGL3VvZHdMc2d3bGErNGFGN3g4ZW9rb0RCeVRjVSt1VUVaZWR3MEVKb1FLQmdRRC92ejhFTkM3Q3pSR2VBWjR0Cjhobkdnd2FjbXRjRmVHTHVWVG5iY3ZIZ2EwbVpabk5jdHdpRVlHMm9CYWtkWFcwTmN5WU1tQWlFZXczNTk2Q3QKakxzY0ViSEhMYmV4OGNHWTZlM3gwRFhPTkVGYlBNUnFUTWtIZytIS2tHbWxES3EzUkNxc2RzYnd3T3EyaEd6dApCNnp2SHNuUXlXeEtnMlE2bXVhckpXaUpYd0tCZ1FESElqVEUrQ2pTWkVvZ0E3dG1zN1NkR0cyY0NjNnB0RWRrCmZPZWFGVzJPWDlia2JjakxXcm9zRmpSVm1nbVRXSFZNcnBmR3p5aVBKTWNtanZxbnQzYklVU2JXK2xKMWdZL0sKSTkyL3Q0K09PRTRUL0xYZ09Hc2lOajh1R3ZTMkt2VnNXOE4yMW8vMkE3TklDcEZlTlFuRnFpcW1vazRzV1FjdAprUCtVc3lBMHFRS0JnRUViUXNPak5UUktXRzNOSTZPMnVuRWE4aGYwSTBFMVFKcEVBQktwMUlHNVRtZEsrWnRrCnpUcEdBaVNBTEdsWlcyWE5KcWFXSGJUOFRyYVFIOHVPeENPRkxhanpMb3lTYXlLdWl2REFmTEllQTNWaXB1NFkKMWlTZm9sK2JIQW4waTVGVVBUamlsQXErVWdKYy9BM3JvazdObTdzWFRBWThKTXYybXpwK3Izb3BBb0dBZit1bApZbkxZSm05dWZxcG5GRTZVMCtZQlk1K1pOc01zcVl6amFzUWhJTGpJWHUxM2M4clFsY21kek8rcS94QndLZFp1CmdiUktKY3BHZEtxY3NtalUwVkh3VldGdzN6TmZuMGJ6Q0ZsdXlybTR6bllraitrdVNVZjN4akppeVROSGZWaGwKZEUzWkpQeXppcS9mT2xsdjhaNml6M0NzTHlqeHgzL0I3MUF5ZkNFQ2dZQk5KckJEdnRSZDRCbmVXQjI5Mk84MQpBU0JpWnNPaHFDMnhSWWg1YnYyVFA2clpQd0h0cGV5MmJEK21iQVpLd003TDRKeHJxK3c0aERTN2EwRlowUm1rCi9yRlpvSjR3MDZWNGM2ZndyZG9WZmRvWE5zNkViQmd1ekxab0YyOFNUc3pOZGhFelJkdFNyWWxlalM1UnVDV2UKbUJ1L0g1d1lBNFFtUU9hRzMranFVZz09Ci0tLS0tRU5EIFBSSVZBVEUgS0VZLS0tLS0K","certificatechain":"LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURNakNDQWhxZ0F3SUJBZ0lKQUk4WDlTcUt0TkJwTUEwR0NTcUdTSWIzRFFFQkN3VUFNQlV4RXpBUkJnTlYKQkFNTUNuSmhjR2h2WlhOMFpYSXdIaGNOTWpJd016STNNVGswTXpBMVdoY05Nekl3TXpJME1UazBNekExV2pBVgpNUk13RVFZRFZRUUREQXB5WVhCb2IyVnpkR1Z5TUlJQklqQU5CZ2txaGtpRzl3MEJBUUVGQUFPQ0FROEFNSUlCCkNnS0NBUUVBdUlqRXJzc0hHSFR3TmNFbjhVTmlKQjErQ0lzL2Q2aklVaXRjUlFJcEhseVJPYzMxaDNyOGlkV0oKK2pGNkFSSndXQThIcmRKYTVCTFhYK3BMWWtMblQ1V2xBVjZkNi9FT2xoUDlCM2tyNHE2ODB0TFhUM3J3UitOdAo0ZTZ0MnUzbjFIS0tsMGM0U25xRGY2b1NjRzE5SGh5OWcvbmp6eFpLVXhZd21tWDRZZXdsSnZNSXZ0VnA5M2JFClQ1MjdJdk5lcFN4MXBBTmhtLzVTRW1zQ3MwQ2pENmZZOXNrcWUra3piZE1TOFhNZlZQYkVXUzdjcG9lZERpSFEKdTVRQVhjSmxnRUVCTjMvbG5JZEpxQ0xQOEFTV2hWd3puSmNEemc5UEcrTlZ4S1A1MHV0bDdROHdsd0Z0R2d4UAp5REhvamNsbVBtQ291UmdHb2RrMnJBQ0lYME9PTFFJREFRQUJvNEdFTUlHQk1CMEdBMVVkRGdRV0JCUTF2Nms5Clh6TTRDd1o4eXV4SWZMNTJjNzdiL2pCRkJnTlZIU01FUGpBOGdCUTF2Nms5WHpNNEN3Wjh5dXhJZkw1MmM3N2IKL3FFWnBCY3dGVEVUTUJFR0ExVUVBd3dLY21Gd2FHOWxjM1JsY29JSkFJOFg5U3FLdE5CcE1Bd0dBMVVkRXdRRgpNQU1CQWY4d0N3WURWUjBQQkFRREFnRUdNQTBHQ1NxR1NJYjNEUUVCQ3dVQUE0SUJBUUN6YXU1QlNnSituZy92Cnd2SHVDTnEyZWg5c1M1M3pmVUlQeTZqTUZrZHprelVZTmI1WjFRK1hjb21GbU9xOVVmaWcxN2dZdUR3WC9nMlMKVklDMVJDdWxwVTNORnNLOEhrSDdjdVFPOGNjSnlWc3JtUndPSFhabmtwTnBoYXI4RFQvSjU5d3FNU09ZZUJQYgpuQU41T0piaUF0NlRBNmRsUXJidkFvWG04dXdHMjFSU1N1VHQ5UElsY3NGallQazhiQWJ0dXdldEVnYjVuNkhoClhhczA1S0dteENmZDZxNHAxYUxITkVpNTVSejg0bEVsVHVNZlA0ZEwyZjIzUnFncmpEQmNDRmEwN1owZHJNdjcKbERYNEovaVpXY08zWXJJWXppcVg5M015UzNSUCtrYXAvV0dIYlZDcWtpZ2lTODZwQkY1NDh3OWxXQ0xJai9kTgozRTI2V24zNgotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg=="}}`
// yeah the string is long lol
var body ResponseBody
if err := json.Unmarshal([]byte(mystring), &body); err != nil {
fmt.Println(fmt.Errorf("failed unmarshaling response | %s", err.Error()))
}
fmt.Println(body.Error.ErrorMessage)
certChain, err := base64.StdEncoding.DecodeString(mystring)
if err != nil {
fmt.Println(err.Error())
}
fmt.Println(certChain)
}
type PkiGenerationInput struct {
TeacherUsername string `json:"teacherusername"`
}
type Response struct {
Headers ResponseHeaders `json:"headers"`
Body ResponseBody `json:"body"`
}
type ResponseHeaders struct {
ContentType string `json:"Content-Type"`
}
type ResponseBody struct {
PkiData ParsablePkiData `json:"pkidata"`
Error struct {
ErrorMessage string `json:"errormessage"`
ErrorType string `json:"errortype"`
} `json:"error"`
}
type ParsablePkiData struct {
Certificate string `json:"certificate"`
PrivateKey string `json:"privatekey"`
CertificateChain string `json:"certificatechain"`
}
Here is the output
errormessagetest
illegal base64 data at input byte 0
[]
the fact I can read the errormessagetest shows that the unmarshalling kinda works, so that's weird.
I checked for any unexported fields but we can clearly see they are all capitalized. I also reduced the string size "just to see" and it didn't change anything.
Answer to synthetize all of the comments :
First of all, my "dumb mistake" was because I tried to decode the wrong variable :
// mystring needs to be replaced by body.PkiData.Certificate
base64.StdEncoding.DecodeString(mystring)
But the research led us to discover that the json.Unmarshal() function decodes base64 by itself when the result type matches what the declared type. Given the fact the encoded data was a []byte, I just replaced the string type by a []byte in the struct.
That allows to create a much more simple version of the previous code :
type ResponseBody struct {
PkiData PkiData `json:"pkidata"`
}
type PkiData struct {
Certificate []byte
PrivateKey []byte
CertificateChain []byte
}
var body ResponseBody
if err := json.Unmarshal(rawResp.Payload, &body); err != nil {
return PkiData{}, fmt.Errorf("failed unmarshaling response | %s", err.Error())
}

JSON Unmarshal does not work with dynamic struct created by relect

I'm trying to parse JSON files by using dynamically created structs, but apparently I'm doing something wrong. Can somebody please tell we what am I doing wrong here:
structured := make(map[string][]reflect.StructField)
structured["Amqp1"] = []reflect.StructField{
reflect.StructField{
Name: "Test",
Type: reflect.TypeOf(""),
Tag: reflect.StructTag(`json:"test"`),
},
reflect.StructField{
Name: "Float",
Type: reflect.TypeOf(5.5),
Tag: reflect.StructTag(`json:"float"`),
},
reflect.StructField{
Name: "Connections",
Type: reflect.TypeOf([]Connection{}),
Tag: reflect.StructTag(`json:"connections"`),
},
}
sections := []reflect.StructField{}
for sect, params := range structured {
sections = append(sections,
reflect.StructField{
Name: sect,
Type: reflect.StructOf(params),
},
)
}
parsed := reflect.New(reflect.StructOf(sections)).Elem()
if err := json.Unmarshal([]byte(JSONConfigContent), &parsed); err != nil {
fmt.Printf("unable to parse data from provided configuration file: %s\n", err)
os.Exit(1)
}
https://play.golang.org/p/C2I4Pduduyg
Thanks in advance.
You want to use .Interface() to return the actual, underlying value, which should be a pointer to the concrete anonymous struct.
Note that the reflect.New function returns a reflect.Value representing a pointer to a new zero value for the specified type. The Interface method, in this case, returns that pointer as interface{} which is all you need for json.Unmarshal.
If, after unmarshaling, you need a non-pointer of the struct you can turn to reflect again and use reflect.ValueOf(parsed).Elem().Interface() to effectively dereference the pointer.
parsed := reflect.New(reflect.StructOf(sections)).Interface()
if err := json.Unmarshal([]byte(JSONConfigContent), parsed); err != nil {
fmt.Printf("unable to parse data from provided configuration file: %s\n", err)
os.Exit(1)
}
https://play.golang.org/p/Bzu1hUyKjvM
reflect.New returns a value representing a pointer.
Change line 69:
fmt.Printf(">>> %v", &parsed)
Result:
>>> <struct { Amqp1 struct { Test string "json:\"test\""; Float float64 "json:\"float\""; Connections []main.Connection "json:\"connections\"" } } Value>
I would recommend something very different. I generally avoid reflection where possible. You can accomplish what you are trying to do by simply providing structs for each type of config you expect and then do an initial "pre-unmarshalling" to determine what type of config you should actually use by name (which, in this case is a key of your JSON object):
package main
import (
"encoding/json"
"fmt"
"os"
)
//Amqp1 config struct
type Amqp1 struct {
Config struct {
Test string `json:"test"`
Float float64 `json:"float"`
Connections []Connection `json:"connections"`
} `json:"Amqp1"`
}
//Connection struct
type Connection struct {
Type string `json:"type"`
URL string `json:"url"`
}
//JSONConfigContent json
const JSONConfigContent = `{
"Amqp1": {
"test": "woobalooba",
"float": 5.5,
"connections": [
{"type": "test1", "url": "booyaka"},
{"type": "test2", "url": "foobar"}
]
}
}`
func main() {
configMap := make(map[string]interface{})
if err := json.Unmarshal([]byte(JSONConfigContent), &configMap); err != nil {
fmt.Printf("unable to parse data from provided configuration file: %s\n", err)
os.Exit(1)
}
//get config name
var configName string
for cfg := range configMap {
configName = cfg
break
}
//unmarshal appropriately
switch configName {
case "Amqp1":
var amqp1 Amqp1
if err := json.Unmarshal([]byte(JSONConfigContent), &amqp1); err != nil {
fmt.Printf("unable to parse data from provided configuration file: %s\n", err)
os.Exit(1)
}
fmt.Printf("%s >>\n", configName)
fmt.Printf("Test: %s\n", amqp1.Config.Test)
fmt.Printf("Float: %v\n", amqp1.Config.Float)
fmt.Printf("Connections: %#v\n", amqp1.Config.Connections)
default:
fmt.Printf("unknown config encountered: %s\n", configName)
os.Exit(1)
}
}
The initial "pre-unmarshalling" happens on the 2nd line of main(), to a plain map where the key is a string and the value is interface{} because you don't care. You're just doing that to get the actual type of config that it is, which is the key of the first nested object.
You can run this on playground as well.

Golang json Unmarshal "unexpected end of JSON input"

I am working on some code to parse the JSON data from an HTTP response. The code I have looks something like this:
type ResultStruct struct {
result []map[string]string
}
var jsonData ResultStruct
err = json.Unmarshal(respBytes, &jsonData)
The json in the respBytes variable looks like this:
{
"result": [
{
"id": "ID 1"
},
{
"id": "ID 2"
}
]
}
However, err is not nil. When I print it out it says unexpected end of JSON input. What is causing this? The JSON seems to valid. Does this error have something to do with my custom struct?
Thanks in advance!
The unexpected end of JSON input is the result of a syntax error in the JSON input (likely a missing ", }, or ]). The error does not depend on the type of the value that you are decoding to.
I ran the code with the example JSON input on the playground. It runs without error.
The code does not decode anything because the result field is not exported. If you export the result field:
type ResultStruct struct {
Result []map[string]string
}
then the input is decoded as shown in this playground example.
I suspect that you are not reading the entire response body in your application. I suggest decoding the JSON input using:
err := json.NewDecoder(resp.Body).Decode(&jsonData)
The decoder reads directly from the response body.
You can also get this error if you're using json.RawMessage in an unexported field. For example, the following code produces the same error:
package main
import (
"encoding/json"
"fmt"
)
type MyJson struct {
Foo bool `json:"foo"`
bar json.RawMessage `json:"bar"`
}
type Bar struct {
X int `json:"x"`
}
var respBytes = []byte(`
{
"foo": true,
"bar": { "x": 10 }
}`)
func main() {
var myJson MyJson
err := json.Unmarshal(respBytes, &myJson)
if err != nil {
fmt.Println(err)
return
}
myBar := new(Bar)
err = json.Unmarshal(myJson.bar, myBar)
fmt.Println(err)
}
If you export "MyJson.bar" field (e.g. -> "MyJson.Bar", then the code works.
it is not the case here; but if you are getting this error loading json from a file it Will occur if the byte slice for the buffer is not initialized the the byte size of the file. [when you're new like me that happens! ] Since this is the first search result I got it still took some digging to figure out. In this use case the error is a bit misleading.
type GenesisResultStruct []GenesisField
fileinfo, _ := genesis.Stat()
bs := make([]byte, fileinfo.Size())
//bs := []byte {} // wrong!!
_, error := genesis.Read(bs)
if error != nil {
fmt.Println("genesis read error: ", error)
os.Exit(1)
}
var jsonData GenesisResultStruct
eGen = json.Unmarshal(bs, &jsonData)
if eGen != nil {
fmt.Println("genesis unmarshal error: ", eGen)
os.Exit(1)
}
Faced the same issue today.
You can also get this error if the respBytes is nil or there are no brackets [] if you are unmarshalling it to a slice.
In that case, you need to explicitly set respBytes.
As you are unmarshalling it to a slice, brackets [] are expected in the byte-slice
if src == nil {
src = []byte("[]")
}

Handling JSON Post Request in Go

So I have the following, which seems incredibly hacky, and I've been thinking to myself that Go has better designed libraries than this, but I can't find an example of Go handling a POST request of JSON data. They are all form POSTs.
Here is an example request: curl -X POST -d "{\"test\": \"that\"}" http://localhost:8082/test
And here is the code, with the logs embedded:
package main
import (
"encoding/json"
"log"
"net/http"
)
type test_struct struct {
Test string
}
func test(rw http.ResponseWriter, req *http.Request) {
req.ParseForm()
log.Println(req.Form)
//LOG: map[{"test": "that"}:[]]
var t test_struct
for key, _ := range req.Form {
log.Println(key)
//LOG: {"test": "that"}
err := json.Unmarshal([]byte(key), &t)
if err != nil {
log.Println(err.Error())
}
}
log.Println(t.Test)
//LOG: that
}
func main() {
http.HandleFunc("/test", test)
log.Fatal(http.ListenAndServe(":8082", nil))
}
There's got to be a better way, right? I'm just stumped in finding what the best practice could be.
(Go is also known as Golang to the search engines, and mentioned here so others can find it.)
Please use json.Decoder instead of json.Unmarshal.
func test(rw http.ResponseWriter, req *http.Request) {
decoder := json.NewDecoder(req.Body)
var t test_struct
err := decoder.Decode(&t)
if err != nil {
panic(err)
}
log.Println(t.Test)
}
You need to read from req.Body. The ParseForm method is reading from the req.Body and then parsing it in standard HTTP encoded format. What you want is to read the body and parse it in JSON format.
Here's your code updated.
package main
import (
"encoding/json"
"log"
"net/http"
"io/ioutil"
)
type test_struct struct {
Test string
}
func test(rw http.ResponseWriter, req *http.Request) {
body, err := ioutil.ReadAll(req.Body)
if err != nil {
panic(err)
}
log.Println(string(body))
var t test_struct
err = json.Unmarshal(body, &t)
if err != nil {
panic(err)
}
log.Println(t.Test)
}
func main() {
http.HandleFunc("/test", test)
log.Fatal(http.ListenAndServe(":8082", nil))
}
There are two reasons why json.Decoder should be preferred over json.Unmarshal - that are not addressed in the most popular answer from 2013:
February 2018, go 1.10 introduced a new method json.Decoder.DisallowUnknownFields() which addresses the concern of detecting unwanted JSON-input
req.Body is already an io.Reader. Reading its entire contents and then performing json.Unmarshal wastes resources if the stream was, say a 10MB block of invalid JSON. Parsing the request body, with json.Decoder, as it streams in would trigger an early parse error if invalid JSON was encountered. Processing I/O streams in realtime is the preferred go-way.
Addressing some of the user comments about detecting bad user input:
To enforce mandatory fields, and other sanitation checks, try:
d := json.NewDecoder(req.Body)
d.DisallowUnknownFields() // catch unwanted fields
// anonymous struct type: handy for one-time use
t := struct {
Test *string `json:"test"` // pointer so we can test for field absence
}{}
err := d.Decode(&t)
if err != nil {
// bad JSON or unrecognized json field
http.Error(rw, err.Error(), http.StatusBadRequest)
return
}
if t.Test == nil {
http.Error(rw, "missing field 'test' from JSON object", http.StatusBadRequest)
return
}
// optional extra check
if d.More() {
http.Error(rw, "extraneous data after JSON object", http.StatusBadRequest)
return
}
// got the input we expected: no more, no less
log.Println(*t.Test)
Playground
Typical output:
$ curl -X POST -d "{}" http://localhost:8082/strict_test
expected json field 'test'
$ curl -X POST -d "{\"Test\":\"maybe?\",\"Unwanted\":\"1\"}" http://localhost:8082/strict_test
json: unknown field "Unwanted"
$ curl -X POST -d "{\"Test\":\"oops\"}g4rB4g3##$%^&*" http://localhost:8082/strict_test
extraneous data after JSON
$ curl -X POST -d "{\"Test\":\"Works\"}" http://localhost:8082/strict_test
log: 2019/03/07 16:03:13 Works
I was driving myself crazy with this exact problem. My JSON Marshaller and Unmarshaller were not populating my Go struct. Then I found the solution at https://eager.io/blog/go-and-json:
"As with all structs in Go, it’s important to remember that only
fields with a capital first letter are visible to external programs
like the JSON Marshaller."
After that, my Marshaller and Unmarshaller worked perfectly!
I found the following example from the docs really helpful (source here).
package main
import (
"encoding/json"
"fmt"
"io"
"log"
"strings"
)
func main() {
const jsonStream = `
{"Name": "Ed", "Text": "Knock knock."}
{"Name": "Sam", "Text": "Who's there?"}
{"Name": "Ed", "Text": "Go fmt."}
{"Name": "Sam", "Text": "Go fmt who?"}
{"Name": "Ed", "Text": "Go fmt yourself!"}
`
type Message struct {
Name, Text string
}
dec := json.NewDecoder(strings.NewReader(jsonStream))
for {
var m Message
if err := dec.Decode(&m); err == io.EOF {
break
} else if err != nil {
log.Fatal(err)
}
fmt.Printf("%s: %s\n", m.Name, m.Text)
}
}
The key here being that the OP was looking to decode
type test_struct struct {
Test string
}
...in which case we would drop the const jsonStream, and replace the Message struct with the test_struct:
func test(rw http.ResponseWriter, req *http.Request) {
dec := json.NewDecoder(req.Body)
for {
var t test_struct
if err := dec.Decode(&t); err == io.EOF {
break
} else if err != nil {
log.Fatal(err)
}
log.Printf("%s\n", t.Test)
}
}
Update: I would also add that this post provides some great data about responding with JSON as well. The author explains struct tags, which I was not aware of.
Since JSON does not normally look like {"Test": "test", "SomeKey": "SomeVal"}, but rather {"test": "test", "somekey": "some value"}, you can restructure your struct like this:
type test_struct struct {
Test string `json:"test"`
SomeKey string `json:"some-key"`
}
...and now your handler will parse JSON using "some-key" as opposed to "SomeKey" (which you will be using internally).
I like to define custom structs locally. So:
// my handler func
func addImage(w http.ResponseWriter, r *http.Request) {
// define custom type
type Input struct {
Url string `json:"url"`
Name string `json:"name"`
Priority int8 `json:"priority"`
}
// define a var
var input Input
// decode input or return error
err := json.NewDecoder(r.Body).Decode(&input)
if err != nil {
w.WriteHeader(400)
fmt.Fprintf(w, "Decode error! please check your JSON formating.")
return
}
// print user inputs
fmt.Fprintf(w, "Inputed name: %s", input.Name)
}
type test struct {
Test string `json:"test"`
}
func test(w http.ResponseWriter, req *http.Request) {
var t test_struct
body, _ := ioutil.ReadAll(req.Body)
json.Unmarshal(body, &t)
fmt.Println(t)
}