I'm using GoLang to get a data from redis hash and then map into a struct.
type Person struct {
ID string `json:"id"`
FirstName string `json:"firstName"`
LastName string `json:"lastName"`
Filters interface{} `json:"filters"`
Type string `json:"type"`
}
In Redis, a hash field contains a stringified JSON.
HGET hashname fieldname
Above returns a stringified JSON.
Now "filters" key can be array or map based on the type (That's why I defined Filters type as interface in struct).
I marshall the JSON like below:
var p Person
content, err := redis.HGet("hashName", "id").Result()
_ = json.Unmarshal([]byte(content), &p)
Now I have to loop over filters like below but this is giving error that cannot range over p.Filters (type interface {}) (I understand why this error is coming)
for _, filter := range p.Filters {
fmt.Println(filter)
}
Is there any way we can handle this situation?
Thanks,
Shashank
You need to convert Filters type interface{} into the expected slice. If you don't really know what type it will be, you can use map[string]interface{}. Therefore change your Filters type to map[string]interface{}.
Per more information
If Filters can be an array (or not), then you might consider a type switch:
A simple example:
package main
import (
"encoding/json"
"fmt"
)
func main() {
var i interface{}
//json.Unmarshal([]byte(`["hi"]`), &i)
json.Unmarshal([]byte(`{"a":"hi"}`), &i)
switch i.(type) {
case []interface{}:
println("ARRAY")
case map[string]interface{}:
println("NOT ARRAY")
default:
fmt.Printf("%T\n", i)
}
}
Related
I'm using Mysql 8. I'm also utilizing 99designs/gqlgen to autogenerate the structs based on the GraphQL schema.
I was to re-use the same structs when scanning MySql responses. And on top of that, while prototyping, I want to have some JSONs in my table.
So the struct is:
type CustomizedItemInput struct {
Sku string `json:"sku"`
Name string `json:"name"`
Skus []*CustomizedComponent `json:"skus"`
...
Since storing(providing Value()) is simpler I managed to store Skus into DB as a top-level JSON successfully. Looks like this:
[{"sku": "123", "position": "LEFT"}, {"sku": "456", "position": "RIGHT"}]
Now, how do I get this value out of DB and back into a array of pointers inside the struct without much hustle?
Of course, ideally it should be done without changing the underlying struct because it's autogenerated.
UPDATE:
Adding debugging information. I need to read a DB row into CustomizedItemView which basically mirrors CustomizedItemInput from above:
type CustomizedItemView struct {
Sku string `json:"sku"`
Name string `json:"name"`
Skus []*CustomizedComponentView `json:"skus"`
...
Of course, when I say "without hustle" I mean having the DB row extracted into a struct seamlessly. I can add map[string]interface{}{} with all bells and whistles and get the value. But I want to have it neat, like:
var storedCustItem = model.CustomizedItemView{}
err := udb.Get(&storedCustItem, database.SelectCustomizationQuery, userID, custItem.Sku, createdAt)
The error I get is:
2020/10/10 20:38:24 sql: Scan error on column index 8, name "skus": unsupported Scan, storing driver.Value type []uint8 into type *[]*model.CustomizedComponentView
(8 because I removed some fields for the example).
The main problem is that I can't create Scan() for an unnamed type. I have created wrappers for Value() because my inserts are more verbose and I do type conversion with the wrapper type in them:
type CustomizedComponentsIn []*CustomizedComponent
...
func (customizedComponents CustomizedComponentsIn) Value() (driver.Value, error)
...
tx.MustExec(database.SaveCustomizationCommand,
custItem.Sku,
custItem.Name,
model.CustomizedComponentsIn(custItem.Skus)
...
,which is Ok for inserts because there will be some values that do not belong to the input struct.
But I hoped to at least get the value scanned into a View struct automatically.
If you can change the type of the Skus field, the common approach would be to declare a slice type that implements the sql.Scanner and driver.Valuer interfaces and use that instead of the unnamed []*CustomizedComponent type.
For example:
type CustomizedItemInput struct {
Sku string `json:"sku"`
Name string `json:"name"`
Skus CustomizedComponentSlice `json:"skus"`
// ...
}
type CustomizedComponentSlice []*CustomizedComponent
// Value implements driver.Valuer interface.
func (s CustomizedComponentSlice) Value() (driver.Value, error) {
return json.Marshal(s)
}
// Scan implements sql.Scanner interface.
func (s *CustomizedComponentSlice) Scan(src interface{}) error {
var data []byte
switch v := src.(type) {
case string:
data = []byte(v)
case []byte:
data = v
default:
return nil
}
return json.Unmarshal(data, s)
}
If you can't change type of the Skus field you will have to explicitly convert the field during scanning.
For example, given the above named slice type, you could do something like this:
v := new(CustomizedItemView)
row := db.QueryRow("SELECT sku, name, skus FROM customized_item_view WHERE sku = ? LIMIT 1", sku)
err := row.Scan(
&v.Sku,
&v.Name,
// do the conversion here, and any other place where you're scanning Skus...
(*CustomizedComponentSlice)(&v.Skus),
)
if err != nil {
return err
}
fmt.Println(v.Skus) // result
Currently, I try to parse JSON to map[string][]interface{}, but unmarshalling returns an error. According to (https://golang.org/pkg/encoding/json/), to unmarshal JSON into an interface value, Unmarshal stores one of these in the interface value:
bool, for JSON booleans
float64, for JSON numbers
string, for JSON strings
-[]interface{}, for JSON arrays
map[string]interface{}, for JSON objects
nil for JSON null
I wonder if golang is able to unmarshal map[string][]interface{}. The following is code snippet. I am new to Golang, thanks for help in advance.
// emailsStr looks like "{"isSchemaConforming":true,"schemaVersion":0,"unknown.0":[{"email_address":"test1#uber.com"},{"email_address":"test2#uber.com"}]}"
emailsRaw := make(map[string][]*entities.Email)
err := json.Unmarshal([]byte(emailsStr), &emailsRaw)
Error message:
&json.UnmarshalTypeError{Value:"number", Type:(*reflect.rtype)(0x151c7a0), Offset:44, Struct:"", Field:""}
The Go encoding/json package will only unmarshal dynamically to a map[string]interface{}. From there, you will need to use type assertions and casting to pull out the values you want, like so:
func main() {
jsonStr := `{"isSchemaConforming":true,"schemaVersion":0,"unknown.0":[{"email_address":"test1#uber.com"},{"email_address":"test2#uber.com"}]}`
dynamic := make(map[string]interface{})
json.Unmarshal([]byte(jsonStr), &dynamic)
firstEmail := dynamic["unknown.0"].([]interface{})[0].(map[string]interface{})["email_address"]
fmt.Println(firstEmail)
}
(https://play.golang.org/p/VEUEIwj3CIC)
Each time, Go's .(<type>) operator is used to assert and cast the dynamic value to a specific type. This particular code will panic if anything happens to be the wrong type at runtime, like if the contents of unknown.0 aren't an array of JSON objects.
The more idiomatic (and robust) way to do this in Go is to annotate a couple structs with json:"" tags and have encoding/json unmarshal into them. This avoids all the nasty brittle .([]interface{}) type casting:
type Email struct {
Email string `json:"email_address"`
}
type EmailsList struct {
IsSchemaConforming bool `json:"isSchemaConforming"`
SchemaVersion int `json:"schemaVersion"`
Emails []Email `json:"unknown.0"`
}
func main() {
jsonStr := `{"isSchemaConforming":true,"schemaVersion":0,"unknown.0":[{"email_address":"test1#uber.com"},{"email_address":"test2#uber.com"}]}`
emails := EmailsList{}
json.Unmarshal([]byte(jsonStr), &emails)
fmt.Printf("%+v\n", emails)
}
(https://play.golang.org/p/iS6e0_87P2J)
A better approach will be to use struct for main schema and then use an slice of email struct for fetching the data for email entities get the values from the same according to requirements. Please find the solution below :-
package main
import (
"fmt"
"encoding/json"
)
type Data struct{
IsSchemaConforming bool `json:"isSchemaConforming"`
SchemaVersion float64 `json:"schemaVersion"`
EmailEntity []Email `json:"unknown.0"`
}
// Email struct
type Email struct{
EmailAddress string `json:"email_address"`
}
func main() {
jsonStr := `{"isSchemaConforming":true,"schemaVersion":0,"unknown.0":[{"email_address":"test1#uber.com"},{"email_address":"test2#uber.com"}]}`
var dynamic Data
json.Unmarshal([]byte(jsonStr), &dynamic)
fmt.Printf("%#v", dynamic)
}
As Redis only stores strings I would like to know how I can do the equivalent of Javascript's JSON.stringify using Go to convert a Struct into a string.
I have tried typecasting:
string(the_struct)
but this results in an error.
The encoding/json package can be used to easily convert a struct to JSON string and vice versa (parse a JSON string into a struct).
Simple example (try it on the Go Playground):
type Person struct {
Name string
Age int
}
func main() {
p := Person{"Bob", 23}
// Struct -> JSON
data, err := json.Marshal(&p)
if err != nil {
panic(err)
}
fmt.Println(string(data))
// JSON -> JSON
var p2 Person
err = json.Unmarshal(data, &p2)
if err != nil {
panic(err)
}
fmt.Printf("%+v", p2)
}
Output:
{"Name":"Bob","Age":23}
{Name:Bob Age:23}
Notes:
The fields of the struct must be exported (start them with capital letter), else the json package (which uses reflection) will not be able to read/write them.
You can also specify tags for the struct fields to control/fine tune the json marshaling/unmarshaling process, for example to change the names in the JSON text:
type Person struct {
Name string `json:"name"`
Age int `json:"years"`
}
With this change the output of the above application is the following:
{"name":"Bob","years":23}
{Name:Bob Age:23}
The documentation of the json.Marshal() function details the possibilities provided by the tags.
And by implementing the json.Marshaler and json.Unmarshaler interfaces you can fully customize the marshaling / unmarshaling process.
Also if your struct is not pre-defined (e.g. you don't know the fields in advance), you can use a map[string]interface{}. See this answer for details and examples.
I have this struct with a field of type interface{}. In the process of caching it using memcached (https://github.com/bradfitz/gomemcache), the struct is marshalled into a JSON, which is then unmarshalled back into the struct when retrieved from the cache. The resulting interface{} field inevitably points to an object of type map[string]interface{} (as in, the interface{} field can only type asserted as map[string]interface{}), the marshalling and unmarshalling process not having preserved the type information. Is there any way to save this information in the marshalling process, in such a way that it can be unmarshalled properly? Or do I have to use some other codec or something?
type A struct {
value interface{}
}
type B struct {
name string
id string
}
func main() {
a := A{value: B{name: "hi", id: "12345"}}
cache.Set("a", a) // Marshals 'a' into JSON and stores in cache
result = cache.Get("a") // Retrieves 'a' from cache and unmarshals
fmt.Printf("%s", result.value.(B).name) // Generates error saying that
// map[string]interface{} cannot be type asserted as a 'B' struct
fmt.Printf("%s", result.value.(map[string]interface{})["name"].(string)) // Correctly prints "12345"
}
Short version, no you can't do that, you have few options though.
Change A.Value to use B instead of interface{}.
Add a function to A that converts A.Value from a map to B if it isn't already B.
Use encoding/gob and store the bytes in memcache then convert it back with a function like NewA(b []byte) *A.
For using gob, you have to register each struct with it first before encoding / decoding, example:
func init() {
//where you should register your types, just once
gob.Register(A{})
gob.Register(B{})
}
func main() {
var (
buf bytes.Buffer
enc = gob.NewEncoder(&buf)
dec = gob.NewDecoder(&buf)
val = A{B{"name", "id"}}
r A
)
fmt.Println(enc.Encode(&val))
fmt.Println(dec.Decode(&r))
fmt.Printf("%#v", r)
}
JSON isn't able to encode the depth of type information that you can in Go, so you'll always get back the following basic types when unmarshalling:
bool, for JSON booleans
float64, for JSON numbers
string, for JSON strings
[]interface{}, for JSON arrays
map[string]interface{}, for JSON objects
nil for JSON null
From go docs: http://golang.org/pkg/encoding/json/#Unmarshal
If you have that knowledge about the types you need, you can write some methods to construct the right variables on unmarshalling perhaps?
I have a json document and I'm using a client which decodes the document in an interface (instead of struct) as below:
var jsonR interface{}
err = json.Unmarshal(res, &jsonR)
How can I access the interface fields? I've read the go doc and blog but my head still can't get it. Their example seem to show only that you can decode the json in an interface but doesn't explain how its fields can be used.
I've tried to use a range loop but it seems the story ends when I reach a map[string]interface. The fields that I need seem to be in the interface.
for k, v := range jsonR {
if k == "topfield" {
fmt.Printf("k is %v, v is %v", k, v)
}
}
The value inside the interface depends on the json structure you're parsing. If you have a json dictionary, the dynamic type of jsonR will be: map[string]interface{}.
Here's an example.
package main
import (
"encoding/json"
"fmt"
"log"
)
func main() {
a := []byte(`{"topfield": 123}`)
var v interface{}
if err := json.Unmarshal(a, &v); err != nil {
log.Fatalf("unmarshal failed: %s", err)
}
fmt.Printf("value is %v", v.(map[string]interface{})["topfield"])
}
Parsing json like this can be very difficult. The default type of a parse is map[string]interface{}. The Problem arises when you have another complex data structure within the main json(like another list or object). The best way to go about decoding json is defining a struct to hold data. Not only will the values be of the correct type but you can extract the specific data you actually care about.
Your struct can look like this:
type Top struct {
Topfield int `json:"topfield"`
}
which can be decoded like this:
a := []byte(`{"topfield": 123}`)
var data Top
json.Unmarshal(a, &data) //parse the json into data
now you can use regular struct operations to access your data like this:
value := data.Topfield
json which contains more complex data can also be easyli decoded. Perhaps you have a list in your data somewhere, you can use a struct like the following to extract it:
type Data struct {
States []string `json:"states"`
PopulationData []Country `json:"popdata"`
}
type Country struct {
Id int `json:"id"`
LastCensusPop int `json:"lcensuspopulation"`
Gdp float64 `json:"gdp"`
}
such a structure can not only parse list but also parse objects withing fields.