My package needs to be able to let my users define the fields backend database column name explicitly, if they want to.
By default I will use the fields name - but sometimes they will need to manually specify the column name, just like the JSON package - unmarshal uses the explicit name if required.
How can I consume this explicit value in my code? I don't even know what it's called so Google is really failing me at the moment.
Here's what JSON's unmarshal function needs for example:
type User struct {
Name string
LastName string `json:"last_name"`
CategoryId int `json:"category_id"`
}
What would I need to use something like this?
// Paprika is my package name.
type User struct {
Name string
LastName string `paprika:"last_name"`
CategoryId int `paprika:"category_id"`
}
My package will be constructing SQL queries and I can't just rely on the field's name - I need to be able to let them manually set the column name. So this is working with only the defined columns at the moment.
// Extracts resource information using reflection and
// saves field names and types.
func loadTypeToSchema(resource interface{}) {
resourceType := reflect.TypeOf(resource)
// Grab the resource struct Name.
fullName := resourceType.String()
name := fullName[strings.Index(fullName, ".")+1:]
// Grabs the resource struct fields and types.
fieldCount := resourceType.NumField()
fields := make(map[string]string)
for i := 0; i <= fieldCount-1; i++ {
field := resourceType.Field(i)
fields[field.Name] = field.Type.Name()
}
// Add resource information to schema map.
schema[name] = fields
}
First of all, what you call decorators are actually called tags. You can read these tags using reflection. The reflect package even has its own example for that.
Nevertheless, here's another example that prints all tags of the members of a struct (Click to play):
type Foo struct {
A int `tag for A`
B int `tag for B`
C int
}
func main() {
f := Foo{}
t := reflect.TypeOf(f)
for i := 0; i < t.NumField(); i++ {
fmt.Println(t.Field(i).Tag)
}
}
Note that in case f is a pointer, e.g. a *Foo, you'd have to indirect (dereference) that value first or else the type returned by TypeOf is not a struct but a pointer and NumField as well as Field() would not work.
Related
I'm using Mysql 8. I'm also utilizing 99designs/gqlgen to autogenerate the structs based on the GraphQL schema.
I was to re-use the same structs when scanning MySql responses. And on top of that, while prototyping, I want to have some JSONs in my table.
So the struct is:
type CustomizedItemInput struct {
Sku string `json:"sku"`
Name string `json:"name"`
Skus []*CustomizedComponent `json:"skus"`
...
Since storing(providing Value()) is simpler I managed to store Skus into DB as a top-level JSON successfully. Looks like this:
[{"sku": "123", "position": "LEFT"}, {"sku": "456", "position": "RIGHT"}]
Now, how do I get this value out of DB and back into a array of pointers inside the struct without much hustle?
Of course, ideally it should be done without changing the underlying struct because it's autogenerated.
UPDATE:
Adding debugging information. I need to read a DB row into CustomizedItemView which basically mirrors CustomizedItemInput from above:
type CustomizedItemView struct {
Sku string `json:"sku"`
Name string `json:"name"`
Skus []*CustomizedComponentView `json:"skus"`
...
Of course, when I say "without hustle" I mean having the DB row extracted into a struct seamlessly. I can add map[string]interface{}{} with all bells and whistles and get the value. But I want to have it neat, like:
var storedCustItem = model.CustomizedItemView{}
err := udb.Get(&storedCustItem, database.SelectCustomizationQuery, userID, custItem.Sku, createdAt)
The error I get is:
2020/10/10 20:38:24 sql: Scan error on column index 8, name "skus": unsupported Scan, storing driver.Value type []uint8 into type *[]*model.CustomizedComponentView
(8 because I removed some fields for the example).
The main problem is that I can't create Scan() for an unnamed type. I have created wrappers for Value() because my inserts are more verbose and I do type conversion with the wrapper type in them:
type CustomizedComponentsIn []*CustomizedComponent
...
func (customizedComponents CustomizedComponentsIn) Value() (driver.Value, error)
...
tx.MustExec(database.SaveCustomizationCommand,
custItem.Sku,
custItem.Name,
model.CustomizedComponentsIn(custItem.Skus)
...
,which is Ok for inserts because there will be some values that do not belong to the input struct.
But I hoped to at least get the value scanned into a View struct automatically.
If you can change the type of the Skus field, the common approach would be to declare a slice type that implements the sql.Scanner and driver.Valuer interfaces and use that instead of the unnamed []*CustomizedComponent type.
For example:
type CustomizedItemInput struct {
Sku string `json:"sku"`
Name string `json:"name"`
Skus CustomizedComponentSlice `json:"skus"`
// ...
}
type CustomizedComponentSlice []*CustomizedComponent
// Value implements driver.Valuer interface.
func (s CustomizedComponentSlice) Value() (driver.Value, error) {
return json.Marshal(s)
}
// Scan implements sql.Scanner interface.
func (s *CustomizedComponentSlice) Scan(src interface{}) error {
var data []byte
switch v := src.(type) {
case string:
data = []byte(v)
case []byte:
data = v
default:
return nil
}
return json.Unmarshal(data, s)
}
If you can't change type of the Skus field you will have to explicitly convert the field during scanning.
For example, given the above named slice type, you could do something like this:
v := new(CustomizedItemView)
row := db.QueryRow("SELECT sku, name, skus FROM customized_item_view WHERE sku = ? LIMIT 1", sku)
err := row.Scan(
&v.Sku,
&v.Name,
// do the conversion here, and any other place where you're scanning Skus...
(*CustomizedComponentSlice)(&v.Skus),
)
if err != nil {
return err
}
fmt.Println(v.Skus) // result
I'm building a web server with Go and I don't know how to process JSON with Go.
Saying that I have a struct as below:
type User struct{
Id int
Name string
Password string
Status int
}
and now I have had an object of the struct User:
user := User{1, "Test", "password", 1}
Now I need to convert user to a JSON object. Here is what I've found:
b, err := json.Marshal(user)
fmt.Println(string(b))
It works well.
Now I want to do two things:
1) remove the Password from the JSON object
2) add a new filed: "code": 200 into the JSON object
What should I do?
If you want to keep the Password property accessible to outer packages, you can set a tag: json:"-" on it. As specified in the docs:
The encoding of each struct field can be customized by the format
string stored under the "json" key in the struct field's tag. The
format string gives the name of the field, possibly followed by a
comma-separated list of options. The name may be empty in order to
specify options without overriding the default field name.
The "omitempty" option specifies that the field should be omitted from
the encoding if the field has an empty value, defined as false, 0, a
nil pointer, a nil interface value, and any empty array, slice, map,
or string.
As a special case, if the field tag is "-", the field is always
omitted. Note that a field with name "-" can still be generated using
the tag "-,".
type User struct {
Id int
Name string
Password string `json:"-"`
Status int
Code int `json:"code"`
}
make Password lower case (and add Code int to your struct):
Try this:
package main
import (
"encoding/json"
"fmt"
)
func main() {
user := User{1, "Test", "password", 1, 200}
b, err := json.Marshal(user)
if err != nil {
panic(err)
}
fmt.Println(string(b))
}
type User struct {
Id int
Name string
password string
Status int
Code int
}
output:
{"Id":1,"Name":"Test","Status":1,"Code":200}
I try to write simple message protocol in go and i've encountered a problem. I have a lot of message types and i want to have a dictionary like this to manipulate with messages:
var dict map[reflect.Type]int = map[reflect.Type]int{
reflect.TypeOf(DataMessage{}): 1000,
reflect.TypeOf(TextMessage{}): 1001,
//....
}
func GetMessageTypeId(value interface{}) int {
if id, ok := dict[reflect.TypeOf(value)]; ok {
return id
} else {
return -1
}
}
func GetValueByTypeId(typeId int) interface{} {
for typeDec, id := range dict {
if id == typeId {
return reflect.Zero(typeDec).Interface()
}
}
fmt.Println("Unknown message type", typeId)
return nil
}
It works fine, but when i instantiate message with GetValueByTypeId and try to unmarshall json into it - i receive map[string]interface instead of my message.
I've made simple example to reproduce the problem:
http://play.golang.org/p/QEyDN9vztr
Please read this article: http://research.swtch.com/interfaces, especially the "Memory Optimizations".
The interface{} by definition consists of two pointers - to method table (e.g. type) and to data it holds. So for
var destination3 interface{} = reflect.Zero(reflect.TypeOf(Message{})).Interface()
it is empty method table (as interface{} has no methods) and reference to Message{}. Taking reference from it returns the reference to this struct so the unmarhal overwrites it with whatever matches interface{}.
If the data interface{} variable holds is a pointer itself, then it is optimized in a way that this pointer is used instead creating interface{} structure. So getting reference to it gives the reference to original variable.
http://play.golang.org/p/KsIS29rUAX
package main
import "fmt"
func main() {
var m1 struct{ Data string }
var m2 interface{}
var m3 interface{}
m2 = &m1
m3 = m1
fmt.Printf("&m1=%p m2=%p &m3=%p\n", &m1, m2, &m3)
}
In your case, using Zero is equivalent to m3 in the example above. Using New is equivalent to m2.
I've found the way how to do what i need
val := reflect.New(reflect.TypeOf(Message{}))
json.Unmarshal(data, val.Interface())
return val.Elem().Interface()
http://play.golang.org/p/8g9FSg3MSj
But was was wrong wit the first version???
It Looks like reflect.Zero(type) should be equivalent to reflect.New(type).Elem() - am i wrong?
I'm using this library to access CouchDB (cloudant to be specific) "github.com/mikebell-org/go-couchdb" and I've noticed a problem.
When I go to add a file to the database and pass in a struct, only the fields of the struct which started with a capital letter get added.
For example
type Person struct {
name string
Age int
}
func main() {
db, _ := couchdb.Database(host, database, username, password)
joe := Person{
name: "mike",
Age: 190,
}
m, _ := db.PostDocument(joe)
}
In this case, only the "age" field got updated and inserted into my database.
I've noticed this problem in another case also - when I'm doing something like this :
type Sample struct {
Name string
age int
}
joe := Sample{
Name: "xx",
age: 23,
}
byt, _ := json.Marshal(joe)
post_data := strings.NewReader(string(byt))
fmt.Println(post_data)
in this case, only Name would be printed out :
output : &{{"Name":"xx"} 0 -1}
Why is this? and If I would like to have a field with a lowercase and be inside the database, is that possible?
This is because only fields starting with a capital letter are exported, or in other words visible outside the curent package (and in the json package in this case).
Here is the part of the specifications refering to this: http://golang.org/ref/spec#Exported_identifiers
Still, you can unmarshall json fields that do no start with a capital letters using what is called "tags". With the json package, this is the syntax to use:
type Sample struct {
Name string `json:"name"`
Age int `json:"age"`
}
Refer to the documentation for more information about this.
json package only stringfiy fields start with capital letter.
see http://golang.org/pkg/encoding/json/
Struct values encode as JSON objects. Each exported struct field becomes a member of the object, using the field name as the object key, unless the field is omitted for one of the reasons given below.
You need define the struct like this:
type Sample struct{
Name string `json:"name"`
Age int `json:"age"`
}
json.Marshal method struct-in field-i only accepts fields that start with a capital letter
The json package only accesses the exported fields of struct types (those that begin with an uppercase letter). Therefore only the exported fields of a struct will be present in the JSON output.
type Sample struct {
Name string
Age int
}
In a web service implemented in Go, I want to be able to restrict fields returned in a JSON response based on a user's role.
For example I may have a currently logged in user who has a role of guest and another with the role of admin
For an admin I want json to have all the keys eg
{
id: 1,
name: "John",
role: "admin"
}
and for a guest to not have the role key eg
{
id: 1,
name: "John"
}
I can currently marshal the json and it returns all fields. I need to be able to restrict it.
You can go by the suggestion #Volker made and clear struct fields for which the user has no permissions. This is probably the easiest to implement.
A second option in a similar vein is to create a custom JSON encoder. One which encodes fields only if a role struct tag matches the current user's role. Here is some pseudo code to illustrate:
type T struct {
currentRole Role `json:"-"`
FieldA string `json:"field_a,omitempty", role:"guest"`
FieldB string `json:"field_b,omitempty", role:"guest"`
FieldC int `json:"field_c,omitempty", role:"admin"`
}
// Have T implement the encoding/json.Marshaler interface.
func (t *T) MarshalJSON() ([]byte, error) {
var buf bytes.Buffer
// Use some reflection magic to iterate over struct fields.
for _, field := range getStructFields(t) {
// More reflection magic to extract field tag data.
role := getFieldTag(field, "role")
// If the field tag's role matches our current role,
// we are good to go. otherwise, skip this field.
if !matchingRole(role, t.currentRole) {
continue // skip this field
}
data, err := json.Marshal(fieldValue(field))
...
_, err = buf.Write(data)
...
}
return buf.Bytes(), nil
}
This is going to be a pain to maintain if you need new roles though. So this would not be something I would lightly consider doing.
Security concerns
I am not entirely sure that what you are looking for is the right solution to your problem. This depends on the context in which you use your code, which is not clear from your question. But if this concerns a website where a user's abilities on the website are defined solely by the value of the role JSON field, then you are looking at a security hole. They can simply go into a browser debugger and change the value of this JSON object to include the "role: "admin" field. And presto! Instant administrative powers. Whether or not to render certain parts of a page, based on user role, should really be handled by the server, during template processing. Just like any and all data posted to the server should be checked and checked again to ensure it came from a trusted source.
If none of this is applicable to you, then by all means, disregard this paragraph.
This question seems old, but I recently wanted to do the same thing. Maybe this will help someone in the future. Here is another method: you can define your own Marshal interface and use anonymous structs.
//User holds all variables
//even private ones
type User struct {
ID int64
Name string
Role string
}
//MarshalJSON gives back json user
//but only the public fields!
func (u *User) MarshalJSON() ([]byte, error) {
return json.Marshal(struct {
ID string `json:"id"`
Name string `json:"name"`
}{u.ID, u.Name})
}
it would be pretty easy to put an if u.Role == "admin" statement in the block to decide whether to marshal the rest.
Another option that also works to define the set of fields in the output for a list of struct that comes from an appengine datastore query.
// Setting different JSON output field for the same struct, using anonymous
// fields (inspired by inspired by http://choly.ca/post/go-json-marshalling/)
// This alternative could be used to load a resultset from an appengine datastore
// query and returned a custom field combination for the list items.
package main
import (
"encoding/json"
"fmt"
)
type User struct {
ID string `json:"id"`
Name string `json:"name"`
Role string `json:"-"`
LaunchCode string `json:"-"`
}
type AdminOutputUser User
func (user *AdminOutputUser) MarshalJSON() ([]byte, error) {
type Alias AdminOutputUser
return json.Marshal(&struct {
*Alias
Role string `json:"role"`
}{
(*Alias)(user),
user.Role,
})
}
type SuperadminOutputUser User
func (user *SuperadminOutputUser) MarshalJSON() ([]byte, error) {
type Alias SuperadminOutputUser
return json.Marshal(&struct {
*Alias
Role string `json:"role"`
LaunchCode string `json:"code"`
}{
(*Alias)(user),
user.Role,
user.LaunchCode,
})
}
func main() {
user := User{"007", "James Bond", "admin", "12345678"}
adminOutput := AdminOutputUser(user)
superadminOutput := SuperadminOutputUser(user)
b, _ := json.Marshal(&user)
fmt.Printf("%s\n\n", string(b))
// {"id":"007","name":"James Bond"}
b, _ = json.Marshal(&adminOutput)
fmt.Printf("%s\n\n", string(b))
// {"id":"007","name":"James Bond","role":"admin"}
b, _ = json.Marshal(&superadminOutput)
fmt.Printf("%s\n\n", string(b))
// {"id":"007","name":"James Bond","role":"admin","code":"12345678"}
}
// for appengine could do something like
// ...
// var users []AdminOutputUser // or User or SuperadminOutputUser
// q := datastore.NewQuery("User")
// keys, err := q.GetAll(ctx, &users)
// ...
https://play.golang.org/p/ignIz0hP0z
You might just define your struct like this
type User struct {
ID int64 `json:"id"`
Name string `json:"name"`
Role string `json:"role,omitempty"`
}
And then set them like this
normalUser := User{ID: "boring", Name: "Rubber"}
adminUser := User{ID: "powers", Name: "Ruler", Role: "admin"}
Then json.Marshal() or json.NewEncoder().Encode() as usual
Found in How To Use Struct Tags in Go
Note: I know that omitempty was mentioned in a comment and is even
part of #jimt's code example and mentioned as first option, albeit without a simple example. To me being still pretty new to
Go wasn't clear that that would just work as expected. So I figured it might help others as well 🤓