I have a scenario where the JSON that has dynamic set of fields that need to get unmarshalled in to a struct.
const jsonStream = `{
"name": "john",
"age": 23,
"bvu62fu6dq": {
"status": true
}
}`
type Status struct {
Status bool
}
type Person struct {
Name string `json:"name"`
Age int `json:"age"`
Status map[string]Status `json:"status"`
}
func main() {
dec := json.NewDecoder(strings.NewReader(jsonStream))
for {
var person Person
if err := dec.Decode(&person); err == io.EOF {
break
} else if err != nil {
log.Fatal(err)
}
fmt.Println(person)
fmt.Println(person.Status["bvu62fu6dq"])
}
}
The output:
{john 23 map[]}
{false}
When it gets unmarshalled, the nested status struct is not being correctly resolved to the value in the JSON (shows false even with true value in JSON), is there any issue in the code?
Your types don't really match with the JSON you have:
type Status struct {
Status bool
}
type Person struct {
Name string `json:"name"`
Age int `json:"age"`
Status map[string]Status `json:"status"`
}
Maps to JSON that looks something like this:
{
"name": "foo",
"age": 12,
"status": {
"some-string": {
"Status": true
}
}
}
The easiest way to unmarshal data with a mix of known/unknown fields in a go type is to have something like this:
type Person struct {
Name string `json:"name"`
Age int `json:"age"`
Random map[string]interface{} `json:"-"` // skip this key
}
Then, first unmarshal the known data:
var p Person
if err := json.Unmarshal([]byte(jsonStream), &p); err != nil {
panic(err)
}
// then unmarshal the rest of the data
if err := json.Unmarshal([]byte(jsonStream), &p.Random); err != nil {
panic(err)
}
Now the Random map will contain every and all data, including the name and age fields. Seeing as you've got those tagged on the struct, these keys are known, so you can easily delete them from the map:
delete(p.Random, "name")
delete(p.Random, "age")
Now p.Random will contain all the unknown keys and their respective values. These values apparently will be an object with a field status, which is expected to be a boolean. You can set about using type assertions and convert them all over to a more sensible type, or you can take a shortcut and marshal/unmarshal the values. Update your Person type like so:
type Person struct {
Name string `json:"name"`
Age int `json:"age"`
Random map[string]interface{} `json:"-"`
Statuses map[string]Status `json:"-"`
}
Now take the clean Random value, marshal it and unmarshal it back into the Statuses field:
b, err := json.Marshal(p.Random)
if err != nil {
panic(err)
}
if err := json.Unmarshal(b, &p.Statuses); err != nil {
panic(err)
}
// remove Random map
p.Random = nil
The result is Person.Statuses["bvu62fu6dq"].Status is set to true
Demo
Cleaning this all up, and marshalling the data back
Now because our Random and Statuses fields are tagged to be ignored for JSON marshalling (json:"-"), marshalling this Person type won't play nice when you want to output the original JSON from these types. It's best to wrap this logic up in a custom JSON (un)-Marshaller interface. You can either use some intermediary types in your MarshalJSON and UnmarshalJSON methods on the Person type, or just create a map and set the keys you need:
func (p Person) MarshalJSON() ([]byte, error) {
data := make(map[string]interface{}, len(p.Statuses) + 2) // 2 being the extra fields
// copy status fields
for k, v := range p.Statuses {
data[k] = v
}
// add known keys
data["name"] = p.Name
data["age"] = p.Age
return json.Marshal(data) // return the marshalled map
}
Similarly, you can do the same thing for UnmarshalJSON, but you'll need to create a version of the Person type that doesn't have the custom handling:
type intermediaryPerson struct {
Name string `json:"name"`
Age int `json:"age"`
Random map[string]interface{} `json:"-"`
}
// no need for the tags and helper fields anymore
type Person struct {
Name string
Age int
Statuses map[string]Status // Status type doesn't change
}
func (p *Person) UnmarshalJSON(data []byte) error {
i := intermediaryPerson{}
if err := json.Unmarshal(data, &i); err != nil {
return err
}
if err := json.Unmarshal(data, &i.Random); err != nil {
return err
}
delete(i.Random, "name")
delete(i.Random, "age")
stat, err := json.Marshal(i.Random)
if err != nil {
return err
}
// copy known fields
p.Name = i.Name
p.Age = i.Age
return json.Unmarshal(stat, &p.Statuses) // set status fields
}
In cases like this, it's common to create a type that handles the known fields and embed that, though:
type BasePerson struct {
Name string `json:"name"`
Age int `json:"age"`
}
and embed that in both the intermediary and the "main"/exported type:
type interPerson struct {
BasePerson
Random map[string]interface{} `json:"-"`
}
type Person struct {
BasePerson
Statuses map[string]Status
}
That way, you can just unmarshal the known fields directly into the BasePerson type, assign it, and then handle the map:
func (p *Person) UnmarshalJSON(data []byte) error {
base := BasePerson{}
if err := json.Unmarshal(data, &base); err != nil {
return err
}
p.BasePerson = base // takes care of all known fields
unknown := map[string]interface{}{}
if err := json.Unmarshal(data, unknown); err != nil {
return err
}
// handle status stuff same as before
delete(unknown, "name") // remove known fields
// marshal unknown key map, then unmarshal into p.Statuses
}
Demo 2
This is how I'd go about it. It allows for calls to json.Marshal and json.Unmarshal to look just like any other type, it centralises the handling of unknown fields in a single place (the implementation of the marshaller/unmarshaller interface), and leaves you with a single Person type where every field contains the required data, in a usable format. It's a tad inefficient in that it relies on unmarshalling/marshalling/unmarshalling the unknown keys. You could do away with that, like I said, using type assertions and iterating over the unknown map instead, faffing around with something like this:
for k, v := range unknown {
m, ok := v.(map[string]interface{})
if !ok {
continue // not {"status": bool}
}
s, ok := m["status"]
if !ok {
continue // status key did not exist, ignore
}
if sb, ok := s.(bool); ok {
// ok, we have a status bool value
p.Statuses[k] = Status{
Status: sb,
}
}
}
But truth be told, the performance difference won't be that great (it's micro optimisation IMO), and the code is a tad too verbose to my liking. Be lazy, optimise when needed, not whenever
Type doesn't meet with your json value.
const jsonStream = `{
"name": "john",
"age": 23,
"bvu62fu6dq": {
"status": true
}
}`
For above json your code should look like below snnipet to work (some modifications in your existing code).
package main
import (
"encoding/json"
"fmt"
"io"
"log"
"strings"
)
const jsonStream = `{
"name": "john",
"age": 23,
"bvu62fu6dq": {
"status": true
}
}`
type bvu62fu6dq struct {
Status bool
}
type Person struct {
Name string `json:"name"`
Age int `json:"age"`
Status bvu62fu6dq `json:"bvu62fu6dq"`
}
func main() {
dec := json.NewDecoder(strings.NewReader(jsonStream))
for {
var person Person
if err := dec.Decode(&person); err == io.EOF {
break
} else if err != nil {
log.Fatal(err)
}
fmt.Println(person)
fmt.Println(person.Status)
}
}
Based on your json data you have to map with type fields.
Run code snippet
I wish to be able to use .Decode() on a response body to populate a struct without first having to attempt to figure out which type of struct I should decode to.
I have a generic struct Match to hold information about a game that was played e.g. a match in Fortnite. Within this struct, I use MatchData to hold the entirety of the game's match data.
When decoding into the MatchData struct, I'm finding the underlying embedded type is initialised, but with all default values, and not the values from the respose.
type Match struct {
MatchID int `json:"match_id"`
GameType int `json:"game_type"`
MatchData *MatchData `json:"match_data"`
}
type MatchData struct {
MatchGame1
MatchGame2
}
type MatchGame1 struct {
X int `json:"x"`
Y int `json:"y"`
}
type MatchGame2 struct {
X int `json:"x"`
Y int `json:"y"`
}
func populateData(m *Match) (Match, error) {
response, err := http.Get("game1.com/path")
if err != nil {
return nil, err
}
// Here, m.MatchData is set with X and Y equal to 0
// when response contains values > 0
err = json.NewDecoder(response.Body).Decode(&m.MatchData)
if err != nil {
return nil, err
}
return m, nil
}
Edit
Example expected JSON payload.
{
"x": 10,
"y": 20
}
I can solve the issue by checking m.GameType, creating a struct that corresponds and then assigning it to m.MatchData, but if I wanted to add another 100 game APIs, I'd prefer if the function could be agnostic of it.
I'm not sure if this is even possible, but thanks in advance.
The approach in the question will not work because the embedded structs share field names. Try this approach.
Declare a map that associates game type identifiers with the associated Go types. This is only code related to decoding that knows about the hundreds of game types.
var gameTypes = map[int]reflect.Type{
1: reflect.TypeOf(&MatchGame1{}),
2: reflect.TypeOf(&MatchGame2{}),
}
Decode the match data to a raw message. Use the game type to create a match data value and decode to that value.
func decodeMatch(r io.Reader) (*Match, error) {
// Start with match data set to a raw messae.
var raw json.RawMessage
m := &Match{MatchData: &raw}
err := json.NewDecoder(r).Decode(m)
if err != nil {
return nil, err
}
m.MatchData = nil
// We are done if there's no raw message.
if len(raw) == 0 {
return m, nil
}
// Create the required match data value.
t := gameTypes[m.GameType]
if t == nil {
return nil, errors.New("unknown game type")
}
m.MatchData = reflect.New(t.Elem()).Interface()
// Decode the raw message to the match data.
return m, json.Unmarshal(raw, m.MatchData)
}
Run it on the playground.
I have some JSON of the form:
[{
"type": "car",
"color": "red",
"hp": 85,
"doors": 4
}, {
"type": "plane",
"color": "blue",
"engines": 3
}]
I have types car and plane that satisfy a vehicle interface; I'd like to be able to write:
var v []vehicle
e := json.Unmarshal(myJSON, &v)
... and have JSON fill my slice of vehicles with a car and a plane; instead (and unsurprisingly) I just get "cannot unmarshal object into Go value of type main.vehicle".
For reference, here are suitable definitions of the types involved:
type vehicle interface {
vehicle()
}
type car struct {
Type string
Color string
HP int
Doors int
}
func (car) vehicle() { return }
type plane struct {
Type string
Color string
Engines int
}
func (plane) vehicle() { return }
var _ vehicle = (*car)(nil)
var _ vehicle = (*plane)(nil)
(Note that I'm actually totally uninterested in the t field on car and plane - it could be omitted because this information will, if someone successfully answers this question, be implicit in the dynamic type of the objects in v.)
Is there a way to have the JSON umarhsaller choose which type to use based on some part of the contents (in this case, the type field) of the data being decoded?
(Note that this is not a duplicate of Unmarshal JSON with unknown fields because I want each item in the slice to have a different dynamic type, and from the value of the 'type' property I know exactly what fields to expect—I just don't know how to tell json.Unmarshal how to map 'type' property values onto Go types.)
Taking the answers from the similar question: Unmarshal JSON with unknown fields, we can construct a few ways to unamrshal this JSON object in a []vehicle data structure.
The "Unmarshal with Manual Handling" version can be done by using a generic []map[string]interface{} data structure, then building the correct vehicles from the slice of maps. For brevity, this example does leave out the error checking for missing or incorrectly typed fields which the json package would have done.
https://play.golang.org/p/fAY9JwVp-4
func NewVehicle(m map[string]interface{}) vehicle {
switch m["type"].(string) {
case "car":
return NewCar(m)
case "plane":
return NewPlane(m)
}
return nil
}
func NewCar(m map[string]interface{}) *car {
return &car{
Type: m["type"].(string),
Color: m["color"].(string),
HP: int(m["hp"].(float64)),
Doors: int(m["doors"].(float64)),
}
}
func NewPlane(m map[string]interface{}) *plane {
return &plane{
Type: m["type"].(string),
Color: m["color"].(string),
Engines: int(m["engines"].(float64)),
}
}
func main() {
var vehicles []vehicle
objs := []map[string]interface{}{}
err := json.Unmarshal(js, &objs)
if err != nil {
log.Fatal(err)
}
for _, obj := range objs {
vehicles = append(vehicles, NewVehicle(obj))
}
fmt.Printf("%#v\n", vehicles)
}
We could leverage the json package again to take care of the unmarshaling and type checking of the individual structs by unmarshaling a second time directly into the correct type. This could all be wrapped up into a json.Unmarshaler implementation by defining an UnmarshalJSON method on the []vehicle type to first split up the JSON objects into raw messages.
https://play.golang.org/p/zQyL0JeB3b
type Vehicles []vehicle
func (v *Vehicles) UnmarshalJSON(data []byte) error {
// this just splits up the JSON array into the raw JSON for each object
var raw []json.RawMessage
err := json.Unmarshal(data, &raw)
if err != nil {
return err
}
for _, r := range raw {
// unamrshal into a map to check the "type" field
var obj map[string]interface{}
err := json.Unmarshal(r, &obj)
if err != nil {
return err
}
vehicleType := ""
if t, ok := obj["type"].(string); ok {
vehicleType = t
}
// unmarshal again into the correct type
var actual vehicle
switch vehicleType {
case "car":
actual = &car{}
case "plane":
actual = &plane{}
}
err = json.Unmarshal(r, actual)
if err != nil {
return err
}
*v = append(*v, actual)
}
return nil
}
JSON decoding and encoding in Go is actually surprisingly well at recognizing fields inside embedded structs. E.g. decoding or encoding the following structure works when there is no overlapping fields between type A and type B:
type T struct{
Type string `json:"type"`
*A
*B
}
type A struct{
Baz int `json:"baz"`
}
type B struct{
Bar int `json:"bar"`
}
Be aware that if both "baz" and "bar" are set in the JSON for the example above, both the T.A and T.B properties will be set.
If there is overlapping fields between A and B, or just to be able to better discard invalid combinations of fields and type, you need to implement the json.Unmarshaler interface. To not have to first decode fields into a map, you can extend the trick of using embedded structs.
type TypeSwitch struct {
Type string `json:"type"`
}
type T struct {
TypeSwitch
*A
*B
}
func (t *T) UnmarshalJSON(data []byte) error {
if err := json.Unmarshal(data, &t.TypeSwitch); err != nil {
return err
}
switch t.Type {
case "a":
t.A = &A{}
return json.Unmarshal(data, t.A)
case "b":
t.B = &B{}
return json.Unmarshal(data, t.B)
default:
return fmt.Errorf("unrecognized type value %q", t.Type)
}
}
type A struct {
Foo string `json:"bar"`
Baz int `json:"baz"`
}
type B struct {
Foo string `json:"foo"`
Bar int `json:"bar"`
}
For marshaling back, json.Marshaler must also be implemented if there is overlapping fields.
Full example: https://play.golang.org/p/UHAdxlVdFQQ
The two passes approach works fine, but there is also the option of the mapstructure package, that was created to do exactly this.
I was facing the same problem.
I'm using the lib github.com/mitchellh/mapstructure together the encoding/json.
I first, unmarshal the json to a map, and use mapstructure to convert the map to my struct, e.g.:
type (
Foo struct {
Foo string `json:"foo"`
}
Bar struct {
Bar string `json:"bar"`
}
)
func Load(jsonStr string, makeInstance func(typ string) any) (any, error) {
// json to map
m := make(map[string]any)
e := json.Unmarshal([]byte(jsonStr), &m)
if e != nil {
return nil, e
}
data := makeInstance(m["type"].(string))
// decoder to copy map values to my struct using json tags
cfg := &mapstructure.DecoderConfig{
Metadata: nil,
Result: &data,
TagName: "json",
Squash: true,
}
decoder, e := mapstructure.NewDecoder(cfg)
if e != nil {
return nil, e
}
// copy map to struct
e = decoder.Decode(m)
return data, e
}
Using:
f, _ := Load(`{"type": "Foo", "foo": "bar"}`, func(typ string) any {
switch typ {
case "Foo":
return &Foo{}
}
return nil
})
If the property is a string you can use .(string) for casting the property because the origin is an interface.
You can use it the next way:
v["type"].(string)
I have a simple type that implements conversion of subtyped integer consts to strings and vice versa in Go. I want to be able to automatically unmarshal strings in JSON to values of this type. I can't, because UnmarshalJSON doesn't give me a way to return or modify the scalar value. It's expecting a struct, whose members are set by UnmarshalJSON. The ",string" method doesn't work either for other than builtin scalar types. Is there a way to implement UnmarshalJSON correctly for a derived scalar type?
Here's an example of what I'm after. I want it to print "Hello Ralph" four times, but it prints "Hello Bob" four times because the PersonID isn't being changed.
package main
import (
"encoding/json"
"fmt"
)
type PersonID int
const (
Bob PersonID = iota
Jane
Ralph
Nobody = -1
)
var nameMap = map[string]PersonID{
"Bob": Bob,
"Jane": Jane,
"Ralph": Ralph,
"Nobody": Nobody,
}
var idMap = map[PersonID]string{
Bob: "Bob",
Jane: "Jane",
Ralph: "Ralph",
Nobody: "Nobody",
}
func (intValue PersonID) Name() string {
return idMap[intValue]
}
func Lookup(name string) PersonID {
return nameMap[name]
}
func (intValue PersonID) UnmarshalJSON(data []byte) error {
// The following line is not correct
intValue = Lookup(string(data))
return nil
}
type MyType struct {
Person PersonID `json: "person"`
Count int `json: "count"`
Greeting string `json: "greeting"`
}
func main() {
var m MyType
if err := json.Unmarshal([]byte(`{"person": "Ralph", "count": 4, "greeting": "Hello"}`), &m); err != nil {
fmt.Println(err)
} else {
for i := 0; i < m.Count; i++ {
fmt.Println(m.Greeting, m.Person.Name())
}
}
}
Use a pointer receiver for the unmarshal method. If a value receiver is used, changes to the receiver are lost when the method returns.
The argument to the unmarshal method is JSON text. Unmarshal the JSON text to get a plain string with all JSON quoting removed.
func (intValue *PersonID) UnmarshalJSON(data []byte) error {
var s string
if err := json.Unmarshal(data, &s); err != nil {
return err
}
*intValue = Lookup(s)
return nil
}
There's a mismatch between the JSON tags an the example JSON. I changed the JSON to match the tag, but you can change it the other way.
if err := json.Unmarshal([]byte(`{"person": "Ralph", "count": 4, "greeting": "Hello"}`), &m); err != nil {
playground example
Here's an answer based on my comment. I'm not sure this is exactly what you want to do as some of your questions wording confuses me however the basic idea is to separate unmarshalling and transformation into two different steps. First unmarshal raw data into a compatible type, after do a transformation to another type or enrich the type you already have like in the example below. You're welcome to hide this behavior in a custom implementation of UnmarshalJSON if you'd like but I would personally advise against it. Here's my two reasons; 1) it's just not consistent with Go's explicit verbose coding style 2) I despise highly obfuscated packages/libraries/languages that do stuff like this for you because sooner or later it bites you in the ass an costs you a lot more than adding that 1 line of extra code in a few places (like hours trying to debug something that makes no sense to you).
type MyType struct {
Id PersonID
Name string `json: "name"`
Count int `json: "count"`
Greeting string `json: "greeting"`
}
func main() {
var m MyType
if err := json.Unmarshal([]byte(`{"name": "Ralph", "count": 4, "greeting": "Hello"}`), &m); err != nil {
fmt.Println(err)
} else {
m.Id = Lookup(m.Name) // see this isn't unmarshalling
// better to take the data as is and do transformation separate
for i := 0; i < m.Count; i++ {
fmt.Println(m.Greeting, m.Person.Name())
}
}
}
I've created an API in Go that, upon being called, performs a query, creates an instance of a struct, and then encodes that struct as JSON before sending back to the caller. I'd now like to allow the caller to be able to select the specific fields they would like returned by passing in a "fields" GET parameter.
This means depending on the fields value(s), my struct would change. Is there any way to remove fields from a struct? Or at least hide them in the JSON response dynamically? (Note: Sometimes I have empty values so the JSON omitEmpty tag will not work here) If neither of these are possible, is there a suggestion on a better way to handle this?
A smaller version of the structs I'm using are below:
type SearchResult struct {
Date string `json:"date"`
IdCompany int `json:"idCompany"`
Company string `json:"company"`
IdIndustry interface{} `json:"idIndustry"`
Industry string `json:"industry"`
IdContinent interface{} `json:"idContinent"`
Continent string `json:"continent"`
IdCountry interface{} `json:"idCountry"`
Country string `json:"country"`
IdState interface{} `json:"idState"`
State string `json:"state"`
IdCity interface{} `json:"idCity"`
City string `json:"city"`
} //SearchResult
type SearchResults struct {
NumberResults int `json:"numberResults"`
Results []SearchResult `json:"results"`
} //type SearchResults
I then encode and output the response like so:
err := json.NewEncoder(c.ResponseWriter).Encode(&msg)
The question is asking for fields to be dynamically selected based on the caller-provided list of fields. This isn't possible to be done with the statically-defined json struct tag.
If what you want is to always skip a field to json-encode, then of course use json:"-" to ignore the field. (Note also that this is not required if your field is unexported; those fields are always ignored by the json encoder.) This isn't what the question asks.
To quote the comment on the json:"-" answer:
This [the json:"-" answer] is the answer most people ending up here from searching would want, but it's not the answer to the question.
I'd use a map[string]interface{} instead of a struct in this case. You can easily remove fields by calling the delete built-in on the map for the fields to remove.
That is, if you can't query only for the requested fields in the first place.
use `json:"-"`
// Field is ignored by this package.
Field int `json:"-"`
// Field appears in JSON as key "myName".
Field int `json:"myName"`
// Field appears in JSON as key "myName" and
// the field is omitted from the object if its value is empty,
// as defined above.
Field int `json:"myName,omitempty"`
// Field appears in JSON as key "Field" (the default), but
// the field is skipped if empty.
// Note the leading comma.
Field int `json:",omitempty"`
doc : http://golang.org/pkg/encoding/json/#Marshal
Another way to do this is to have a struct of pointers with the ,omitempty tag. If the pointers are nil, the fields won't be Marshalled.
This method will not require additional reflection or inefficient use of maps.
Same example as jorelli using this method: http://play.golang.org/p/JJNa0m2_nw
You can use the reflect package to select the fields that you want by reflecting on the field tags and selecting the json tag values. Define a method on your SearchResults type that selects the fields you want and returns them as a map[string]interface{}, and then marshal that instead of the SearchResults struct itself. Here's an example of how you might define that method:
func fieldSet(fields ...string) map[string]bool {
set := make(map[string]bool, len(fields))
for _, s := range fields {
set[s] = true
}
return set
}
func (s *SearchResult) SelectFields(fields ...string) map[string]interface{} {
fs := fieldSet(fields...)
rt, rv := reflect.TypeOf(*s), reflect.ValueOf(*s)
out := make(map[string]interface{}, rt.NumField())
for i := 0; i < rt.NumField(); i++ {
field := rt.Field(i)
jsonKey := field.Tag.Get("json")
if fs[jsonKey] {
out[jsonKey] = rv.Field(i).Interface()
}
}
return out
}
and here's a runnable solution that shows how you would call this method and marshal your selection: http://play.golang.org/p/1K9xjQRnO8
I just published sheriff, which transforms structs to a map based on tags annotated on the struct fields. You can then marshal (JSON or others) the generated map. It probably doesn't allow you to only serialize the set of fields the caller requested, but I imagine using a set of groups would allow you to cover most cases. Using groups instead of the fields directly would most likely also increase cache-ability.
Example:
package main
import (
"encoding/json"
"fmt"
"log"
"github.com/hashicorp/go-version"
"github.com/liip/sheriff"
)
type User struct {
Username string `json:"username" groups:"api"`
Email string `json:"email" groups:"personal"`
Name string `json:"name" groups:"api"`
Roles []string `json:"roles" groups:"api" since:"2"`
}
func main() {
user := User{
Username: "alice",
Email: "alice#example.org",
Name: "Alice",
Roles: []string{"user", "admin"},
}
v2, err := version.NewVersion("2.0.0")
if err != nil {
log.Panic(err)
}
o := &sheriff.Options{
Groups: []string{"api"},
ApiVersion: v2,
}
data, err := sheriff.Marshal(o, user)
if err != nil {
log.Panic(err)
}
output, err := json.MarshalIndent(data, "", " ")
if err != nil {
log.Panic(err)
}
fmt.Printf("%s", output)
}
Take three ingredients:
The reflect package to loop over all the fields of a struct.
An if statement to pick up the fields you want to Marshal, and
The encoding/json package to Marshal the fields of your liking.
Preparation:
Blend them in a good proportion. Use reflect.TypeOf(your_struct).Field(i).Name() to get a name of the ith field of your_struct.
Use reflect.ValueOf(your_struct).Field(i) to get a type Value representation of an ith field of your_struct.
Use fieldValue.Interface() to retrieve the actual value (upcasted to type interface{}) of the fieldValue of type Value (note the bracket use - the Interface() method produces interface{}
If you luckily manage not to burn any transistors or circuit-breakers in the process you should get something like this:
func MarshalOnlyFields(structa interface{},
includeFields map[string]bool) (jsona []byte, status error) {
value := reflect.ValueOf(structa)
typa := reflect.TypeOf(structa)
size := value.NumField()
jsona = append(jsona, '{')
for i := 0; i < size; i++ {
structValue := value.Field(i)
var fieldName string = typa.Field(i).Name
if marshalledField, marshalStatus := json.Marshal((structValue).Interface()); marshalStatus != nil {
return []byte{}, marshalStatus
} else {
if includeFields[fieldName] {
jsona = append(jsona, '"')
jsona = append(jsona, []byte(fieldName)...)
jsona = append(jsona, '"')
jsona = append(jsona, ':')
jsona = append(jsona, (marshalledField)...)
if i+1 != len(includeFields) {
jsona = append(jsona, ',')
}
}
}
}
jsona = append(jsona, '}')
return
}
Serving:
serve with an arbitrary struct and a map[string]bool of fields you want to include, for example
type magic struct {
Magic1 int
Magic2 string
Magic3 [2]int
}
func main() {
var magic = magic{0, "tusia", [2]int{0, 1}}
if json, status := MarshalOnlyFields(magic, map[string]bool{"Magic1": true}); status != nil {
println("error")
} else {
fmt.Println(string(json))
}
}
Bon Appetit!
I created this function to convert struct to JSON string by ignoring some fields. Hope it will help.
func GetJSONString(obj interface{}, ignoreFields ...string) (string, error) {
toJson, err := json.Marshal(obj)
if err != nil {
return "", err
}
if len(ignoreFields) == 0 {
return string(toJson), nil
}
toMap := map[string]interface{}{}
json.Unmarshal([]byte(string(toJson)), &toMap)
for _, field := range ignoreFields {
delete(toMap, field)
}
toJson, err = json.Marshal(toMap)
if err != nil {
return "", err
}
return string(toJson), nil
}
Example: https://play.golang.org/p/nmq7MFF47Gp
You can use tagging attribute "omitifempty" or make optional fields pointers and leave those you want skipped uninitialized.
Here is how I defined my structure.
type User struct {
Username string `json:"username" bson:"username"`
Email string `json:"email" bson:"email"`
Password *string `json:"password,omitempty" bson:"password"`
FullName string `json:"fullname" bson:"fullname"`
}
And inside my function set user.Password = nil for not to be Marshalled.
I didn't have the same problem but similar. Below code solves your problem too, of course if you don't mind performance issue. Before implement that kind of solution to your system I recommend you to redesign your structure if you can. Sending variable structure response is over-engineering. I believe a response structure represents a contract between a request and resource and it should't be depend requests.(you can make un-wanted fields null, I do). In some cases we have to implement this design, if you believe you are in that cases here is the play link and code I use.
type User2 struct {
ID int `groups:"id" json:"id,omitempty"`
Username string `groups:"username" json:"username,omitempty"`
Nickname string `groups:"nickname" json:"nickname,omitempty"`
}
type User struct {
ID int `groups:"private,public" json:"id,omitempty"`
Username string `groups:"private" json:"username,omitempty"`
Nickname string `groups:"public" json:"nickname,omitempty"`
}
var (
tagName = "groups"
)
//OmitFields sets fields nil by checking their tag group value and access control tags(acTags)
func OmitFields(obj interface{}, acTags []string) {
//nilV := reflect.Value{}
sv := reflect.ValueOf(obj).Elem()
st := sv.Type()
if sv.Kind() == reflect.Struct {
for i := 0; i < st.NumField(); i++ {
fieldVal := sv.Field(i)
if fieldVal.CanSet() {
tagStr := st.Field(i).Tag.Get(tagName)
if len(tagStr) == 0 {
continue
}
tagList := strings.Split(strings.Replace(tagStr, " ", "", -1), ",")
//fmt.Println(tagList)
// ContainsCommonItem checks whether there is at least one common item in arrays
if !ContainsCommonItem(tagList, acTags) {
fieldVal.Set(reflect.Zero(fieldVal.Type()))
}
}
}
}
}
//ContainsCommonItem checks if arrays have at least one equal item
func ContainsCommonItem(arr1 []string, arr2 []string) bool {
for i := 0; i < len(arr1); i++ {
for j := 0; j < len(arr2); j++ {
if arr1[i] == arr2[j] {
return true
}
}
}
return false
}
func main() {
u := User{ID: 1, Username: "very secret", Nickname: "hinzir"}
//assume authenticated user doesn't has permission to access private fields
OmitFields(&u, []string{"public"})
bytes, _ := json.Marshal(&u)
fmt.Println(string(bytes))
u2 := User2{ID: 1, Username: "very secret", Nickname: "hinzir"}
//you want to filter fields by field names
OmitFields(&u2, []string{"id", "nickname"})
bytes, _ = json.Marshal(&u2)
fmt.Println(string(bytes))
}
I also faced this problem, at first I just wanted to specialize the responses in my http handler. My first approach was creating a package that copies the information of a struct to another struct and then marshal that second struct. I did that package using reflection, so, never liked that approach and also I wasn't dynamically.
So I decided to modify the encoding/json package to do this. The functions Marshal, MarshalIndent and (Encoder) Encode additionally receives a
type F map[string]F
I wanted to simulate a JSON of the fields that are needed to marshal, so it only marshals the fields that are in the map.
https://github.com/jtorz/jsont
package main
import (
"fmt"
"log"
"net/http"
"github.com/jtorz/jsont/v2"
)
type SearchResult struct {
Date string `json:"date"`
IdCompany int `json:"idCompany"`
Company string `json:"company"`
IdIndustry interface{} `json:"idIndustry"`
Industry string `json:"industry"`
IdContinent interface{} `json:"idContinent"`
Continent string `json:"continent"`
IdCountry interface{} `json:"idCountry"`
Country string `json:"country"`
IdState interface{} `json:"idState"`
State string `json:"state"`
IdCity interface{} `json:"idCity"`
City string `json:"city"`
} //SearchResult
type SearchResults struct {
NumberResults int `json:"numberResults"`
Results []SearchResult `json:"results"`
} //type SearchResults
func main() {
msg := SearchResults{
NumberResults: 2,
Results: []SearchResult{
{
Date: "12-12-12",
IdCompany: 1,
Company: "alfa",
IdIndustry: 1,
Industry: "IT",
IdContinent: 1,
Continent: "america",
IdCountry: 1,
Country: "México",
IdState: 1,
State: "CDMX",
IdCity: 1,
City: "Atz",
},
{
Date: "12-12-12",
IdCompany: 2,
Company: "beta",
IdIndustry: 1,
Industry: "IT",
IdContinent: 1,
Continent: "america",
IdCountry: 2,
Country: "USA",
IdState: 2,
State: "TX",
IdCity: 2,
City: "XYZ",
},
},
}
fmt.Println(msg)
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
//{"numberResults":2,"results":[{"date":"12-12-12","idCompany":1,"idIndustry":1,"country":"México"},{"date":"12-12-12","idCompany":2,"idIndustry":1,"country":"USA"}]}
err := jsont.NewEncoder(w).Encode(msg, jsont.F{
"numberResults": nil,
"results": jsont.F{
"date": nil,
"idCompany": nil,
"idIndustry": nil,
"country": nil,
},
})
if err != nil {
log.Fatal(err)
}
})
http.ListenAndServe(":3009", nil)
}
The question is now a bit old, but I came across the same issue a little while ago, and as I found no easy way to do this, I built a library fulfilling this purpose.
It allows to easily generate a map[string]interface{} from a static struct.
https://github.com/tuvistavie/structomap
To extend chhaileng answer, here is the version that remove all occurrences of a field with recursion
// GetJSONWithOutFields - Description: return a string representation of an interface with specified fields removed
func GetJSONWithOutFields(obj interface{}, ignoreFields ...string) (string, error) {
toJson, err := json.Marshal(obj)
if err != nil {
return "", err
}
if len(ignoreFields) == 0 {
return string(toJson), nil
}
toMap := map[string]interface{}{}
err = json.Unmarshal(toJson, &toMap)
if err != nil {
return "", err
}
for _, field := range ignoreFields {
DeleteField(toMap, field)
}
toJson, err = json.Marshal(toMap)
if err != nil {
return "", err
}
return string(toJson), nil
}
// DeleteField - Description: recursively delete field
func DeleteField(toMap map[string]interface{}, field string) {
delete(toMap, field)
for _, v := range toMap {
if m, isMap := v.(map[string]interface{}); isMap {
DeleteField(m, field)
}
}
}