Store struct as string in redis - json

As Redis only stores strings I would like to know how I can do the equivalent of Javascript's JSON.stringify using Go to convert a Struct into a string.
I have tried typecasting:
string(the_struct)
but this results in an error.

The encoding/json package can be used to easily convert a struct to JSON string and vice versa (parse a JSON string into a struct).
Simple example (try it on the Go Playground):
type Person struct {
Name string
Age int
}
func main() {
p := Person{"Bob", 23}
// Struct -> JSON
data, err := json.Marshal(&p)
if err != nil {
panic(err)
}
fmt.Println(string(data))
// JSON -> JSON
var p2 Person
err = json.Unmarshal(data, &p2)
if err != nil {
panic(err)
}
fmt.Printf("%+v", p2)
}
Output:
{"Name":"Bob","Age":23}
{Name:Bob Age:23}
Notes:
The fields of the struct must be exported (start them with capital letter), else the json package (which uses reflection) will not be able to read/write them.
You can also specify tags for the struct fields to control/fine tune the json marshaling/unmarshaling process, for example to change the names in the JSON text:
type Person struct {
Name string `json:"name"`
Age int `json:"years"`
}
With this change the output of the above application is the following:
{"name":"Bob","years":23}
{Name:Bob Age:23}
The documentation of the json.Marshal() function details the possibilities provided by the tags.
And by implementing the json.Marshaler and json.Unmarshaler interfaces you can fully customize the marshaling / unmarshaling process.
Also if your struct is not pre-defined (e.g. you don't know the fields in advance), you can use a map[string]interface{}. See this answer for details and examples.

Related

Unmarshal remaining JSON after performing custom unmarshalling

I have a JSON object That contains an implementation of an interface within it. I'm attempting to take that JSON and marshal it into a struct whilst creating the implementation of the interface.
I've managed to get it to implement the interface with a custom JSON unmarshal function however I'm struggling to piece together how to then marshal the rest of the fields
I've created an example in the Go playground
https://play.golang.org/p/ztF7H7etdjM
My JSON being passed into my application is
{
"address":"1FYuJ4MsVmpzPoFJ6svJMJfygn91Eubid9",
"nonce":13,
"network_id":"qadre.demo.balance",
"challenge":"f2b19e71876c087e681fc092ea3a34d5680bbfe772e40883563e1d5513bb593f",
"type":"verifying_key",
"verifying_key":{
"verifying_key":"3b6a27bcceb6a42d62a3a8d02a6f0d73653215771de243a63ac048a18b59da29",
"fqdn":"huski.service.key"
},
"signature":"a3bf8ee202a508d5a5632f50b140b70b7095d8836493dc7ac4159f6f3350280078b3a58b2162a240bc8c7485894554976a9c7b5d279d3f5bf49fec950f024e02",
"fqdn":"huski.service.SingleKeyProof"
}
I've attempted to do a json.Unmarshal and pass in a new struct for the remaining fields however it seems to put me in an infinite loop, my application hangs and then crashes
The best solution I've come up with so far is to marshal the JSON into a `map[string]interface{} and do each field separately, this feels very clunky though
var m map[string]interface{}
if err := json.Unmarshal(data, &m); err != nil {
return err
}
ad, ok := m["address"]
if ok {
s.Address = ad.(string)
}
fqdn, ok := m["fqdn"]
if ok {
s.FQDN = fqdn.(string)
}
n, ok := m["nonce"]
if ok {
s.Nonce = int64(n.(float64))
}
c, ok := m["challenge"]
if ok {
s.Challenge = []byte(c.(string))
}
network, ok := m["network_id"]
if ok {
s.NetworkID = network.(string)
}
sig, ok := m["signature"]
if ok {
s.Signature = []byte(sig.(string))
}
The reason your code gets into an infinite loop when you try to unmarshal the rest of the fields is because, I presume, the implementation of UnmarshalJSON after its done unmarshaling the verifying key, calls json.Unmarshal with the receiver, which in turn calls the UnmarshalJSON method on the receiver and so they invoke each other ad infinitum.
What you can do is to create a temporary type using the existing type as its definition, this will "keep the structure" but "drop the methods", then unmarshal the rest of the fields into an instance of the new type, and, after unmarshal is done, convert the instance to the original type and assign that to the receiver.
While this fixes the infinite loop, it also re-introduces the original problem of json.Unmarshal not being able to unmarshal into a non-empty interface type. To fix that you can embed the new type in another temporary struct that has a field with the same json tag as the problematic field which will cause it to be "overshadowed" while json.Unmarshal is doing its work.
type SingleKey struct {
FQDN string `json:"fqdn"`
Address string `json:"address"`
Nonce int64 `json:"nonce"`
Challenge []byte `json:"challenge"`
NetworkID string `json:"network_id"`
Type string `json:"type"`
VerifyingKey PublicKey `json:"verifying_key"`
Signature []byte `json:"signature"`
}
func (s *SingleKey) UnmarshalJSON(data []byte) error {
type _SingleKey SingleKey
var temp struct {
RawKey json.RawMessage `json:"verifying_key"`
_SingleKey
}
if err := json.Unmarshal(data, &temp); err != nil {
return err
}
*s = SingleKey(temp._SingleKey)
switch s.Type {
case "verifying_key":
s.VerifyingKey = &PublicKeyImpl{}
// other cases ...
}
return json.Unmarshal([]byte(temp.RawKey), s.VerifyingKey)
}
https://play.golang.org/p/L3gdQZF47uN
Looking at what you've done in your custom unmarshalling function, you seem to be passing in a map with the name of fields as index, and the reflect.Type you want to unmarshal said value into. That, to me, suggests that the keys might be different for different payloads, but that each key has a distinct type associated with it. You can perfectly handle data like this with a simple wrapper type:
type WrappedSingleKey struct {
FQDN string `json:"fqdn"`
Address string `json:"address"`
Nonce int64 `json:"nonce"`
Challenge []byte `json:"challenge"`
NetworkID string `json:"network_id"`
Type string `json:"type"`
VerifyingKey json.RawMessage `json:"verifying_key"`
OtherKey json.RawMessage `json:"other_key"`
Signature []byte `json:"signature"`
}
type SingleKey struct {
FQDN string `json:"fqdn"`
Address string `json:"address"`
Nonce int64 `json:"nonce"`
Challenge []byte `json:"challenge"`
NetworkID string `json:"network_id"`
Type string `json:"type"`
VerifyingKey *PublicKey `json:"verifying_key,omitempty"`
OtherType *OtherKey `json:"other_key,omitempty"`
Signature []byte `json:"signature"`
}
So I've changed the type of your VerifyingKey field to a json.RawMessage. That's basically telling json.Unmarshal to leave that as raw JSON input. For every custom/optional field, add a corresponding RawMessage field.
In the unwrapped type, I've changed VerifyingKey to a pointer and added the omitempty bit to the tag. That's just to accomodate mutliple types, and not have to worry about custom marshalling to avoid empty fields, like the included OtherType field I have. To get what you need, then:
func (s *SingleKey) UnmarshalJSON(data []byte) error {
w := WrappedSingleKey{} // create wrapped instance
if err := json.Unmarshal(data, &w); err != nil {
return err
}
switch w.Type {
case "verifying_key":
var pk PublicKey
if err := json.Unmarshal([]byte(w.VerifyingKey), &pk); err != nil {
return err
}
s.VerifyingKey = &pk // assign
case "other_key":
var ok OtherKey
if err := json.Unmarshal([]byte(w.OtherKey), &ok); err != nil {
return err
}
s.OtherKey = &ok
}
// copy over the fields that didn't require anything special
s.FQDN = w.FQDN
s.Address = w.Address
}
This is a fairly simple approach, does away with the reflection, tons of functions, and is quite commonly used. It's something that lends itself quite well to code generation, too. The individual assignment of the fields is a bit tedious, though. You might think that you can solve that by embedding the SingleKey type into the wrapper, but be careful: this will recursively call your custom unmarshaller function.
You could, for example, update all the fields in the WRapped type to be pointers, and have them point to fields on your actual type. That does away with the manual copying of fields... It's up to you, really.
Note
I didn't test this code, just wrote it as I went along. It's something I've used in the past, and I believe what I wrote here should work, but no guarantees (as in: you might need to debug it a bit)

Is Go able to unmarshal to map[string][]interface{}?

Currently, I try to parse JSON to map[string][]interface{}, but unmarshalling returns an error. According to (https://golang.org/pkg/encoding/json/), to unmarshal JSON into an interface value, Unmarshal stores one of these in the interface value:
bool, for JSON booleans
float64, for JSON numbers
string, for JSON strings
-[]interface{}, for JSON arrays
map[string]interface{}, for JSON objects
nil for JSON null
I wonder if golang is able to unmarshal map[string][]interface{}. The following is code snippet. I am new to Golang, thanks for help in advance.
// emailsStr looks like "{"isSchemaConforming":true,"schemaVersion":0,"unknown.0":[{"email_address":"test1#uber.com"},{"email_address":"test2#uber.com"}]}"
emailsRaw := make(map[string][]*entities.Email)
err := json.Unmarshal([]byte(emailsStr), &emailsRaw)
Error message:
&json.UnmarshalTypeError{Value:"number", Type:(*reflect.rtype)(0x151c7a0), Offset:44, Struct:"", Field:""}
The Go encoding/json package will only unmarshal dynamically to a map[string]interface{}. From there, you will need to use type assertions and casting to pull out the values you want, like so:
func main() {
jsonStr := `{"isSchemaConforming":true,"schemaVersion":0,"unknown.0":[{"email_address":"test1#uber.com"},{"email_address":"test2#uber.com"}]}`
dynamic := make(map[string]interface{})
json.Unmarshal([]byte(jsonStr), &dynamic)
firstEmail := dynamic["unknown.0"].([]interface{})[0].(map[string]interface{})["email_address"]
fmt.Println(firstEmail)
}
(https://play.golang.org/p/VEUEIwj3CIC)
Each time, Go's .(<type>) operator is used to assert and cast the dynamic value to a specific type. This particular code will panic if anything happens to be the wrong type at runtime, like if the contents of unknown.0 aren't an array of JSON objects.
The more idiomatic (and robust) way to do this in Go is to annotate a couple structs with json:"" tags and have encoding/json unmarshal into them. This avoids all the nasty brittle .([]interface{}) type casting:
type Email struct {
Email string `json:"email_address"`
}
type EmailsList struct {
IsSchemaConforming bool `json:"isSchemaConforming"`
SchemaVersion int `json:"schemaVersion"`
Emails []Email `json:"unknown.0"`
}
func main() {
jsonStr := `{"isSchemaConforming":true,"schemaVersion":0,"unknown.0":[{"email_address":"test1#uber.com"},{"email_address":"test2#uber.com"}]}`
emails := EmailsList{}
json.Unmarshal([]byte(jsonStr), &emails)
fmt.Printf("%+v\n", emails)
}
(https://play.golang.org/p/iS6e0_87P2J)
A better approach will be to use struct for main schema and then use an slice of email struct for fetching the data for email entities get the values from the same according to requirements. Please find the solution below :-
package main
import (
"fmt"
"encoding/json"
)
type Data struct{
IsSchemaConforming bool `json:"isSchemaConforming"`
SchemaVersion float64 `json:"schemaVersion"`
EmailEntity []Email `json:"unknown.0"`
}
// Email struct
type Email struct{
EmailAddress string `json:"email_address"`
}
func main() {
jsonStr := `{"isSchemaConforming":true,"schemaVersion":0,"unknown.0":[{"email_address":"test1#uber.com"},{"email_address":"test2#uber.com"}]}`
var dynamic Data
json.Unmarshal([]byte(jsonStr), &dynamic)
fmt.Printf("%#v", dynamic)
}

Go: decoding json with one set of json tags, and encoding to a different set of json tags

I have an application that consumes data from a third-party api. I need to decode the json into a struct, which requires the struct to have json tags of the "incoming" json fields. The outgoing json fields have a different naming convention, so I need different json tags for the encoding.
I will have to do this with many different structs, and each struct might have many fields.
What is the best way to accomplish this without repeating a lot of code?
Example Structs:
// incoming "schema" field names
type AccountIn struct {
OpenDate string `json:"accountStartDate"`
CloseDate string `json:"cancelDate"`
}
// outgoing "schema" field names
type AccountOut struct {
OpenDate string `json:"openDate"`
CloseDate string `json:"closeDate"`
}
Maybe the coming change on Go 1.8 would help you, it will allow to 'cast' types even if its JSON tags definition is different: This https://play.golang.org/p/Xbsoa8SsEk works as expected on 1.8beta, I guess this would simplify your current solution
A bit an uncommon but probably quite well working method would be to use a intermediate format so u can use different readers and writers and therefore different tags. For example https://github.com/mitchellh/mapstructure which allows to convert a nested map structure into struct
types. Pretty similar like json unmarshal, just from a map.
// incoming "schema" field names
type AccountIn struct {
OpenDate string `mapstructure:"accountStartDate" json:"openDate"`
CloseDate string `mapstructure:"cancelDate" json:"closeDate"`
}
// from json to map with no name changes
temporaryMap := map[string]interface{}{}
err := json.Unmarshal(jsonBlob, &temporaryMap)
// from map to structs using mapstructure tags
accountIn := &AccountIn{}
mapstructure.Decode(temporaryMap, accountIn)
Later when writing (or reading) u will use directly the json functions which will then use the json tags.
If it's acceptable to take another round trip through json.Unmarshal and json.Marshal, and you don't have any ambiguous field names within your various types, you could translate all the json keys in one pass by unmarshaling into the generic structures used by the json package:
// map incoming to outgoing json identifiers
var translation = map[string]string{
"accountStartDate": "openDate",
"cancelDate": "closeDate",
}
func translateJS(js []byte) ([]byte, error) {
var m map[string]interface{}
if err := json.Unmarshal(js, &m); err != nil {
return nil, err
}
translateKeys(m)
return json.MarshalIndent(m, "", " ")
}
func translateKeys(m map[string]interface{}) {
for _, v := range m {
if v, ok := v.(map[string]interface{}); ok {
translateKeys(v)
}
}
keys := make([]string, 0, len(m))
for k := range m {
keys = append(keys, k)
}
for _, k := range keys {
if newKey, ok := translation[k]; ok {
m[newKey] = m[k]
delete(m, k)
}
}
}
https://play.golang.org/p/nXmWlj7qH9
This might be a Naive Approach but is fairly easy to implement:-
func ConvertAccountInToAccountOut(AccountIn incoming) (AccountOut outcoming){
var outcoming AccountOut
outcoming.OpenDate = incoming.OpenDate
outcoming.CloseDate = incoming.CloseDate
return outcoming
}
var IncomingJSONData AccountIn
resp := getJSONDataFromSource() // Some method that gives you the Input JSON
err1 := json.UnMarshall(resp,&IncomingJSONData)
OutGoingJSONData := ConvertAccountInToAccountOut(IncomingJSONData)
if err1 != nil {
fmt.Println("Error in UnMarshalling JSON ",err1)
}
fmt.Println("Outgoing JSON Data: ",OutGoingJSONData)

How to access interface fields on json decode?

I have a json document and I'm using a client which decodes the document in an interface (instead of struct) as below:
var jsonR interface{}
err = json.Unmarshal(res, &jsonR)
How can I access the interface fields? I've read the go doc and blog but my head still can't get it. Their example seem to show only that you can decode the json in an interface but doesn't explain how its fields can be used.
I've tried to use a range loop but it seems the story ends when I reach a map[string]interface. The fields that I need seem to be in the interface.
for k, v := range jsonR {
if k == "topfield" {
fmt.Printf("k is %v, v is %v", k, v)
}
}
The value inside the interface depends on the json structure you're parsing. If you have a json dictionary, the dynamic type of jsonR will be: map[string]interface{}.
Here's an example.
package main
import (
"encoding/json"
"fmt"
"log"
)
func main() {
a := []byte(`{"topfield": 123}`)
var v interface{}
if err := json.Unmarshal(a, &v); err != nil {
log.Fatalf("unmarshal failed: %s", err)
}
fmt.Printf("value is %v", v.(map[string]interface{})["topfield"])
}
Parsing json like this can be very difficult. The default type of a parse is map[string]interface{}. The Problem arises when you have another complex data structure within the main json(like another list or object). The best way to go about decoding json is defining a struct to hold data. Not only will the values be of the correct type but you can extract the specific data you actually care about.
Your struct can look like this:
type Top struct {
Topfield int `json:"topfield"`
}
which can be decoded like this:
a := []byte(`{"topfield": 123}`)
var data Top
json.Unmarshal(a, &data) //parse the json into data
now you can use regular struct operations to access your data like this:
value := data.Topfield
json which contains more complex data can also be easyli decoded. Perhaps you have a list in your data somewhere, you can use a struct like the following to extract it:
type Data struct {
States []string `json:"states"`
PopulationData []Country `json:"popdata"`
}
type Country struct {
Id int `json:"id"`
LastCensusPop int `json:"lcensuspopulation"`
Gdp float64 `json:"gdp"`
}
such a structure can not only parse list but also parse objects withing fields.

json: cannot unmarshal object into Go value of type

I can't decode the json code below ... any ideas why it doesn't work ? See also play.golang
package main
import (
"encoding/json"
)
type LocationReadable struct {
District string
City string
State string
}
type Locale struct {
Location string
CountryCode string
CurrencyId string
CurrencySymbol string
LocationReadable LocationReadable
}
type Media struct {
Image string
Video string
}
type Variations struct {
FixedPrice float64
Media Media
Quantity int
}
type PaymentData struct {
PaymentName string
PaymentService string
}
type Payment struct {
Online PaymentData
Offline PaymentData
}
type Shipping struct {
ShippingService string
ShippingName string
ShippingCost float64
HandlingTimeMax int
DispatchTimeMin int
DispatchTimeMax int
ShippingAdditionalCost int
}
type Item []struct {
_version string
CategoryId string
Title string
Media Media
SellerId string
Locale Locale
ListingType string
Payment Payment
StartTime string
EndTime string
Shipping Shipping
TitleSlug string
Variations Variations
_fpaiStatus string
}
func main() {
itemInfoR := `{"locale":{"location":"51.51121389999999,-0.11982439999997041","countryCode":"GB","currencyId":"GBP","currencySymbol":"£","locationReadable":{"district":"City of Westminster","city":"London","state":"Greater London"}},"_version":"serving","categoryId":["Root","Cameras \u0026 Photo","Digital Cameras"],"title":"many pictures","media":{"image":["//lh5.ggpht.com/O_o_N6CFkClY5AV0-LqntpyFjor7Of4u23ZcK7lYwc2uY1ea7GWi61VDJZCB7UCb79svkjKPHIenqwEUhjHi0jdIQnnl6z_p03yktPUB1FBHezIQ","//lh6.ggpht.com/ih3q2d7CenGLPyupH9FpfsoJQWQpw1i8wWA2Kd26bFnSF2fbnKyGU9WePIhCgEeqw5p6YMVmFi1c9oS0Ag93aF_oZ3ZiwK7fQuSYIrZ9VhgXbrTHkw","//lh6.ggpht.com/7RJRsapsnwWL3_KiLIjMz4QojDzUvsztXtvKTFvIfde_AHccDnOibAvXRN73tTB4SeHzlj8S1LWxbYwwWFGn9elfCKdSb8BUIU5QJY1LO791HutQ","//lh6.ggpht.com/qAtjgyHAB734Ox_4NC_fa-ZRqrCjCmJu0Tp8bo-HMO88duv8l4hhuv2REBkB--yneFzOL7annecVlGty-YsKouondiOFVnAZWzjpdrfsGfbL6wh2","//lh3.ggpht.com/dWUbASepwHF4lHaXIPnpv4BNm2pCml9MlJt7s86s1cpu-PsYNmS0yQmKFKTM38q_oMLW_YJMJ19civ2gVViKAGYcZylRW7jN3w77AJvhzS6JE2g","//lh6.ggpht.com/9aXLmPRVeZnxkwvNb3mWTF8kvfEY_lho_lOVVc9AbNqLb8GQmiS_XXVZ3OKqMv2pxgYSayMYPPRh6ACYyh0H8KtS8mPD6MKUkEajwxkTtp5Q4Lo","//lh3.ggpht.com/FG_QXZPHJ2tTYwI_t5Fg1KqivglVg9RlJn0JRsu9Ox8vJ7IcBirb2IV_I1LL_WVOMxfTuBBSDLMlrw9v0MCAdmnPCR29sCbRGjhm6zEfIH-3q2QSdw","//lh4.ggpht.com/Y23DqORrVkM2m55f-rq5_BBrlkvQg4uX7AsAt-ixhMobjK_SFgFaDfktgLhkNsyKwSr9HcF8iiGY3Nw0xOKXG1sn6wyAWg_qsolmKjVOrM5V5mIR","//lh6.ggpht.com/mQ62Ly-DjMKPMzU1OcSPJ7SLBqym0uBjawlkTHfmb-HOKaD56dnitk1duwPFJVdbi0GUpd63RQvr2VMpHp6S1OQ3di-hq4-JPeRoS5FJzksXSvW_","//lh3.ggpht.com/dqWjWPcNsvlR1tMC_agizX19f9MDiNGWFYTYVn4kjJxzIIkEe0mLzNcvS62zVJxAOaitT-IgaUfZ-Ze23BgzbqYY-l600i_LbVe35Uinz6sXIyoB","//lh6.ggpht.com/xhSdFc9uHgghs_6gf3seUWYM-PG2oLmjTrpF7ptEEMqaIrQIa8VPfC6tXE7f3M13eZvDXYqMW_k0AHO5vwCEPNp-iObixskd_lBaKNfz3MH3SNQ","//lh5.ggpht.com/kYeoKPoZGJCow-G1FhnD8kzVjNjbQA8-Kyj8eAh0HL-fMZX9tTeFPQikTZdSU0kks4-5Ui54cZF2CjGut9vfMJAVDKIq3T-bAQewCxvfl2120tH5zQ","//lh5.ggpht.com/4qUl3d-G9EPBzcYKrimNsWhQw7CmONV0jgfVhxFgB9mEU_QLRCyNJTWs2A3xf6wc7AUF2DXrKEkoX-SNLMZ6s-O4aXXV9WOjOPcWdAYreMRBld0E","//lh5.ggpht.com/z-0C4G6EWYkelAF1LjPfl_UQcsp92H4joIPt8NfsOl0nPJ2VpzZYahWadKqTLfl6kq3C6aDBcwfGQyMWSozYoZIAOAW0yRvZrwxia321PlsKTxbZ","//lh4.ggpht.com/U7I12JrDYmMC_pUXpw8DVBjBilU67BvbM8qT8gJE0bQfkhHo7FOdMttiz3syP5IR-LyO4J1WBlfmZjvMjRr4GIBt4o3Vqp-hKz7q2_OGwGtsN5s","//lh3.ggpht.com/fF2XWEtqG23ybhzClhC_p8gvKJalf1vg7k3H7UkuAaIVubil7EgOvJUCwAZk2KiCtlPYp1E5Ep2xaxZjJRmg5EFSEAjqlMHJS_Wd1Bcje6xre4s","//lh3.ggpht.com/jgOebMihBoIZvHE4EOklJvZ_k-9egjNIlUKfKFcLkvXJs8g2FXjPvdFUbwqGrkHrMtyis8uOvgt-E51Vm11hq4bieh7h0cegca0VI4vFtFaAemU","//lh3.ggpht.com/MOrI-zKNMNrQE_aHj5hzbojP3T0hEMJKK6K8UO3e1NBC-nkcQeIM1QnvtJdT_G-W4e7-qv4BiqwdWcNHBpZXOmmX3tcuYEV8u_ANEoa9_aUIfeyg","//lh6.ggpht.com/SyIS5sGOkTG7k_jFF14wzH9Evrblv6o4pHBI6z6X070-xhAeyut_kRO6xHtDID4KLcWFvItjQy-plPcJ6K1T9tlFOrtaryEPvuAYdMVx8e0TTw","//lh6.ggpht.com/2Pp9kLYFhDT3USwHinU5OxnzcWWOLI0nOWe29gOD5KMzyEcXoHkTN-AutJV9M8F_9eqAP379XB9O1d0BWPanhr-MguzKxfHeUvYTs6yHzDkxyfe0NA","//lh4.ggpht.com/7aofqklSkF3AMDfF19yqsA9J3EfEiKy1NdOelEGKNnW0Cv5tGEpq2PF_jZO1MVoBbrrmVVRv0Tdq7I8KyZbIlyHdbTs1jMl7dEFqVMvsPcyaORyHlQ","//lh4.ggpht.com/anYJHqkMCkuhmIHQTBspLtWcDTyx1ZRe84_q5pAgVEOVmsKkaKhS725N4YFoj2zpJrBP7iTC2vf1GUtrp6H7kkm8c1k6zkW6I_Gf5f9A3re_I8Ex","//lh3.ggpht.com/OtSw0rU-DvfoXgoWrQdkln6Kz7O14TF9qrPNJSGJnZLeDqUEctOn1DT09pdwwVpNQV-cXmVYQL-PX4XPhpZLWH1ciSkVT6WHNmTz1D9pHphBwJUv","//lh3.ggpht.com/cTCZnXPIjI-EO2bvQdLgeoSLOSlMFcv805n347Zyci9XDYUdcVDC_5H7SFVYDr4pC5HtQDYnrOHL6AinLW7hWtfSCLlvVhVUNQ-DlDn0NwZ-1iCO-g","//lh4.ggpht.com/i-mL_JcF9rwjQq6HnuKzuAHU41_UGxQ62IOPZvaDrATXaPFbhe-EbT7ZIpboyNA5PXRCsxNsZ9hu58edRvNs5ScgKN8Lg-00J2LhlwMAbdEsv7b0nw","//lh6.ggpht.com/D_YV2BG1WWwl67xNloP3sxzRkqhcVTgJi58L-A8nLrOcMR_tBqLz4fHEGQ-qiNcG_-32MNy3dlSPWrTBKzBcweJxgMnRVet5yuGfelUlwehDtXX_3w"],"video":[]},"sellerId":"mihai","listingType":"fixedPrice","payment":{"online":[{"paymentName":"PayPal","paymentService":"paypal"}],"offline":[{"paymentName":"Pay on Pick-up","paymentService":"payOnPickup"}]},"startTime":"2014-01-04T10:02:18+00:00","endTime":"2014-04-04T10:02:18+00:00","shipping":[{"shippingService":"economy","shippingName":"Economy","shippingCost":1.0,"handlingTimeMax":4,"dispatchTimeMin":1,"dispatchTimeMax":10,"shippingAdditionalCost":"2"},{"shippingService":"localPickup","shippingName":"Local Pick-Up","shippingCost":0.0,"handlingTimeMax":2,"dispatchTimeMin":0,"dispatchTimeMax":0,"shippingAdditionalCost":"0"}],"titleSlug":"many-pictures","variations":[{"fixedPrice":222999.0,"media":{"image":["//lh6.ggpht.com/ih3q2d7CenGLPyupH9FpfsoJQWQpw1i8wWA2Kd26bFnSF2fbnKyGU9WePIhCgEeqw5p6YMVmFi1c9oS0Ag93aF_oZ3ZiwK7fQuSYIrZ9VhgXbrTHkw","//lh6.ggpht.com/9aXLmPRVeZnxkwvNb3mWTF8kvfEY_lho_lOVVc9AbNqLb8GQmiS_XXVZ3OKqMv2pxgYSayMYPPRh6ACYyh0H8KtS8mPD6MKUkEajwxkTtp5Q4Lo","//lh3.ggpht.com/FG_QXZPHJ2tTYwI_t5Fg1KqivglVg9RlJn0JRsu9Ox8vJ7IcBirb2IV_I1LL_WVOMxfTuBBSDLMlrw9v0MCAdmnPCR29sCbRGjhm6zEfIH-3q2QSdw"],"video":[]},"quantity":1121,"Brand":"Bell \u0026 Howell"},{"fixedPrice":211.0,"media":{"image":["//lh6.ggpht.com/qAtjgyHAB734Ox_4NC_fa-ZRqrCjCmJu0Tp8bo-HMO88duv8l4hhuv2REBkB--yneFzOL7annecVlGty-YsKouondiOFVnAZWzjpdrfsGfbL6wh2","//lh3.ggpht.com/FG_QXZPHJ2tTYwI_t5Fg1KqivglVg9RlJn0JRsu9Ox8vJ7IcBirb2IV_I1LL_WVOMxfTuBBSDLMlrw9v0MCAdmnPCR29sCbRGjhm6zEfIH-3q2QSdw","//lh6.ggpht.com/9aXLmPRVeZnxkwvNb3mWTF8kvfEY_lho_lOVVc9AbNqLb8GQmiS_XXVZ3OKqMv2pxgYSayMYPPRh6ACYyh0H8KtS8mPD6MKUkEajwxkTtp5Q4Lo","//lh3.ggpht.com/MOrI-zKNMNrQE_aHj5hzbojP3T0hEMJKK6K8UO3e1NBC-nkcQeIM1QnvtJdT_G-W4e7-qv4BiqwdWcNHBpZXOmmX3tcuYEV8u_ANEoa9_aUIfeyg"],"video":[]},"quantity":2,"Brand":"Fujifilm"},{"fixedPrice":22.0,"media":{"image":["//lh3.ggpht.com/jgOebMihBoIZvHE4EOklJvZ_k-9egjNIlUKfKFcLkvXJs8g2FXjPvdFUbwqGrkHrMtyis8uOvgt-E51Vm11hq4bieh7h0cegca0VI4vFtFaAemU","//lh3.ggpht.com/MOrI-zKNMNrQE_aHj5hzbojP3T0hEMJKK6K8UO3e1NBC-nkcQeIM1QnvtJdT_G-W4e7-qv4BiqwdWcNHBpZXOmmX3tcuYEV8u_ANEoa9_aUIfeyg","//lh4.ggpht.com/anYJHqkMCkuhmIHQTBspLtWcDTyx1ZRe84_q5pAgVEOVmsKkaKhS725N4YFoj2zpJrBP7iTC2vf1GUtrp6H7kkm8c1k6zkW6I_Gf5f9A3re_I8Ex"],"video":[]},"quantity":12,"Brand":"Gateway"}],"_fpaiStatus":"published"}`
itemInfoBytes := []byte(itemInfoR)
var ItemInfo Item
er := json.Unmarshal(itemInfoBytes, &ItemInfo)
if er != nil {
panic(er)
}
}
Here's a fixed version of it: http://play.golang.org/p/w2ZcOzGHKR
The biggest fix that was needed is when Unmarshalling an array, that property needs to be an array/slice in the struct as well.
For example:
{ "things": ["a", "b", "c"] }
Would Unmarshal into a:
type Item struct {
Things []string
}
And not into:
type Item struct {
Things string
}
The other thing to watch out for when Unmarshaling is that the types line up exactly. It will fail when Unmarshalling a JSON string representation of a number into an int or float field -- "1" needs to Unmarshal into a string, not into an int like we saw with ShippingAdditionalCost int
Determining of root cause is not an issue since Go 1.8; field name now is shown in the error message:
json: cannot unmarshal object into Go struct field Comment.author of type string
You JSON doesn't match your struct fields: E.g. "district" in JSON and "District" as the field.
Also: Your Item is a slice type but your JSON is a dict value. Do not mix this up. Slices decode from arrays.
Just in case you're looking for an answer to this from an AWS perspective, particularly if you're trying to use UnmarshalListOfMaps() on the items coming back from a DynamoDB query(), then watch out for a simple typo I made.
Given that this function UnmarshalListOfMaps() takes in...a list of maps :-) then it needs to unmarshal into a List of whatever struct you're trying to build, and not just the plain struct itself. This was throwing me off because I was trying to start by querying for something that should only return one row.
movie := Movie{}
// Run the DynamoDB query
resp, err := session.Query(input) // type QueryInput
if err != nil {
log.WithError(err).Error("Error running DynamoDB query")
}
// Unmarshal all the attribute values into a Movie struct
err = dynamodbattribute.UnmarshalListOfMaps(resp.Items, &movie); if err != nil {
log.WithError(err).Error("Error marshaling DynamoDB result into Movie")
return link, err
}
The problem is in the first line. It should be movies := []Movie{} and then the reference &movie needs to change to &movies as well. If you leave it just as it is above, then the AWS Go SDK will throw this error:
cannot unmarshal list into Go value of type (etc).
If you ever face this while typing yml file not copy-pasting, it might be a problem with your yml file - mine was wrong indent.
This happened while making pipeline on concourse.