I'd like to marshal part of this struct:
type ValueSet struct {
Id string `json:"id" bson:"_id"`
Url string `bson:"url,omitempty" json:"url,omitempty"`
Identifier *Identifier `bson:"identifier,omitempty" json:"identifier,omitempty"`
Version string `bson:"version,omitempty" json:"version,omitempty"`
Name string `bson:"name,omitempty" json:"name,omitempty"`
Status string `bson:"status,omitempty" json:"status,omitempty"`
Experimental *bool `bson:"experimental,omitempty" json:"experimental,omitempty"`
Publisher string `bson:"publisher,omitempty" json:"publisher,omitempty"`
Contact []ValueSetContactComponent `bson:"contact,omitempty" json:"contact,omitempty"`
Date *FHIRDateTime `bson:"date,omitempty" json:"date,omitempty"`
LockedDate *FHIRDateTime `bson:"lockedDate,omitempty" json:"lockedDate,omitempty"`
Description string `bson:"description,omitempty" json:"description,omitempty"`
UseContext []CodeableConcept `bson:"useContext,omitempty" json:"useContext,omitempty"`
Immutable *bool `bson:"immutable,omitempty" json:"immutable,omitempty"`
Requirements string `bson:"requirements,omitempty" json:"requirements,omitempty"`
Copyright string `bson:"copyright,omitempty" json:"copyright,omitempty"`
Extensible *bool `bson:"extensible,omitempty" json:"extensible,omitempty"`
CodeSystem *ValueSetCodeSystemComponent `bson:"codeSystem,omitempty" json:"codeSystem,omitempty"`
Compose *ValueSetComposeComponent `bson:"compose,omitempty" json:"compose,omitempty"`
Expansion *ValueSetExpansionComponent `bson:"expansion,omitempty" json:"expansion,omitempty"`
}
which is part of a Go implementation of HL7 FHIR, including only the metadata fields, and omitting the three content three fields (codeSystem, compose and expansion). I can't (and shouldn't) change the JSON tags in the original source code, since other code strongly depends on it working the way it's written. How can I tell json.Marshal to override the existing JSON tags on these struct elements?
You can't change it, but you don't have to.
Easiest solution is to create your own struct, define your own json tags (how you want them to appear in the output), copy the fields, and marshal a value of your own struct.
E.g. let's say you want to marshal the Id and Urlfields, then:
type MyValueSet struct {
Id string `json:"MyId"`
Url string `json:"MyUrl"`
}
var vs ValueSet = ... // Comes from somewhere
mvs := MyValueSet {
Id: vs.Id,
Url: vs.Url,
}
data, err := json.Marshal(&mvs)
// Check err
Related
I'm working on unmarshaling some nested json data that I have already written a struct for. I've used a tool that will generate a struct based off json data, but am a bit confused how to work with accessing nested json data (and fields can sometimes be emtpy).
Here is an example of struct:
type SomeJson struct {
status string `json:"status"`
message string `json:"message"`
someMoreData []struct {
constant bool `json:"constant,omitempty"`
inputs []struct {
name string `json:"name"`
type string `json:"type"`
} `json:"inputs,omitempty"`
Name string `json:"name,omitempty"`
Outputs []struct {
Name string `json:"name"`
Type string `json:"type"`
} `json:"outputs,omitempty"`
I'm able to unmarshal the json data and access the top level fields (such as status and message), but am having trouble accessing any data under the someMoreData field. I understand this field is a (I assume an unknown) map of structs, but only have experience working with basic single level json blobs.
For reference this is the code I have to unmarshal the json and am able to access the top level fields.
someData := someJson{}
json.Unmarshal(body, &someData)
So what is exactly the best to access some nested fields such as inputs.name or outputs.name?
to iterate over your particular struct you can use:
for _, md := range someData.someMoreData {
println(md.Name)
for _, out := range md.Outputs {
println(out.Name, out.Type)
}
}
to access specific field:
someData.someMoreData[0].Outputs[0].Name
Couple of things to note:
The struct definition is syntactically incorrect. There are couple of closing braces missing.
type is a keyword.
The status and message and other fields with lower case first letter fields are unexported. So, the Json parser will not throw error, but you will get zero values as output. Not sure that's what you observed.
someMoreData is an array of structs not map.
I use AWS Lambda with DynamoDB using golang.
My DynamoDB table use lowercase attribute names such as id or name.
In Go, if I want to be able to marshal a struct correctly, I have to name fields starting with a capital letter.
type Item struct {
ID string
Name string
}
To put an item into my DynamoDB table, I have to marshal it into a map[string]*dynamodb.AttributeValue, using dynamodbattribute.MarshalMap function.
item := Item{
ID: "xxxx",
Name: "yyyy"
}
av, _ := dynamodbattribute.MarshalMap(item)
Of course, this will create a map using names written as ID and Name, which are incompatible with id and name from the dynamodb table.
Reading the documentation, I found that you can use a custom encoder, and enable json tags.
type Item struct {
ID string `json="id"`
Name string `json="name"`
}
func setOpts(encoder *dynamodbattribute.Encoder) {
// Not sure which one I sould set
// So I set them both :)
encoder.SupportJSONTags = true
encoder.MarshalOptions.SupportJSONTags = true
}
func main() {
encoder := dynamodbattribute.NewEncoder(setOpts)
encoder.Encode(...)
}
But here the encoder.Encode() method is only used to create a dynamodb.AttributeValue, and not a map[string]*dynamodb.AttributeValue.
Is there a way to use a custom encoder with MarshalMap? Or am I using it in a wrong way?
EDIT:
Okay so as Zak pointed out, there is a dynamodbav tag that can be used.
I also found out that I was using json tags in a wrong way. I should use the syntax json:"id" instead of json="id".
Indeed, DynamoDB SDK uses the json tag if available, and this one can be overrided by the dynamodbav.
So all I had to do was to make my structure looks like this and it worked
type Item struct {
ID string `json:"id"`
Name string `json:"name"`
}
Dynamo's built in marshalling, from MarshalMap(...) can support struct tags, similar to json.
You can add them to the type that you are marshalling, like so:
type Item struct {
ID string `dynamodbav:"id"`
Name string `dynamodbav:"name"`
}
See the docs here
I want to be able to access a struct field that's resulted from a JSON unmarshal, BUT I want to use that same struct to hide the field when it is marshaled.
Example:
type MyStruct struct {
GoodField string `json:"goodField"`
SecretField string `json:"secret"`
}
Incoming JSON gets unmarshaled and the secret field is accessible
Use the same MyStruct in the server response but hide the secret field.
I've looked at using omitempty and - tags, not working.
you were on the right track with omitempty you just have to set SecretField to "" for it to take effect
package main
import (
"fmt"
"encoding/json"
)
type MyStruct struct {
GoodField string `json:"goodField"`
SecretField string `json:"secret,omitempty"`
}
func main() {
data := MyStruct{}
s := `{"goodField": "xxx", "secret": "yyy"}`
json.Unmarshal([]byte(s), &data);
fmt.Println(data.GoodField, data.SecretField);
data.SecretField = ""
response, _ := json.Marshal(data)
fmt.Println(string(response))
}
In general I think the best pattern is to use separate request and response types, rather than mutating data just to prevent it from being marshaled; that's an intrusive and destructive approach, and the side-effect could bite you. In line with the Go proverb "a little duplication is better than a little dependency," you're better off just separating those concerns rather than try to have one type serve double duty.
If you want, though, you can try using struct embedding to DRY out the field declarations. With a combination of omitempty and - json tags, you can get the desired results:
type payload struct {
A string `json:"a"`
B string `json:"b,omitempty"`
}
type request struct {
payload
}
type response struct {
payload
B string `json:"-"`
}
Here's a worked example of the above.
It's not without its own issues to be aware of -- there are two fields named B here, not one. It works fine if you're just using dot-operator field access as in my example. If you use struct literals, you've got to be careful not to put the value in the wrong spot. Probably a good idea to write constructors for these types, to centralize that concern in one spot and get it right.
Also if you need this field accessible only from this package (which helps you to control access to it at all) you can make it unexported with first lowercase letter. Use this structure for all operations, except Marshalling/unmarshalling.
type MyStruct struct {
GoodField string `json:"goodField"`
secretField string `json:"secret"`
}
And make one more structure what will be used only for Marshalling.
If you can't, or don't want, to set the field to empty, you can accomplish this with a custom marshaler, as well:
type MyStruct struct {
GoodField string `json:"goodField"`
SecretField string `json:"secret"`
}
func (ms *MyStruct) MarshalJSON() ([]byte, error) {
type MyStructWithoutSecretFields struct {
GoodField string `json:"goodField"`
}
noSecrets := MyStructWithoutSecretFields{
GoodField: ms.GoodField,
}
return json.Marshal(noSecrets)
}
I am using thrift in my project, thrift will generate code as follow:
type CvJdRelationInfo struct {
JdId string `thrift:"jdId,1" json:"jdId"`
CvId string `thrift:"cvId,2" json:"cvId"`
Status int16 `thrift:"status,3" json:"status"`
AcceptTimestamp int64 `thrift:"acceptTimestamp,4" json:"acceptTimestamp"`
}
as you see thrift already generate json tags(but no bson tags), when I use mgo save record, mgo will auto convert:
JdId -> jdid
CvId -> cvid
Status -> status
AcceptTimeStamp -> accepttimestamp
what I needed is:
type CvJdRelationInfo struct {
JdId string `thrift:"jdId,1" json:"jdId" bson:"jdId"`
CvId string `thrift:"cvId,2" json:"cvId" bson:"cvId"`
Status int16 `thrift:"status,3" json:"status" bson:"status"`
AcceptTimestamp int64 `thrift:"acceptTimestamp,4" json:"acceptTimestamp" bson:"acceptTimestamp"`
}
as your can see, bson tags is the same as json tags. Can I use json tags as bson tags?
MongoDB actually stores the data as binary JSON (bson), which is distinct from JSON. It's slightly confusing because if you access the database using the mongo shell, you get back raw JSON, but it's actually a conversion, it's not the storage format. So in storing the data to the database, the "mgo" driver serializes as bson.
This serializing ignores the json export keys, and chooses the appropriate name by defaulting to the lowercase version of your struct field. (See the bson.Marshal go doc.) If you specify a bson export key, it will then ignore the struct field name and go with whatever you have specified as the bson export key.
For example,
type User struct {
Name string
UserAge int `bson:"age"`
Phone string `json:"phoneNumber"`
}
will result in the following structure in MongoDB:
{
"name": "",
"age": 0,
"phone": ""
}
So it seems like your struct fields should handle most things for you.
The one 'gotcha' that you might not see until it bites you is, if you don't specify the bson export keys, you don't have the ability to do bson:",omitempty" for leaving out blank fields, or bson:",inline" for marshaling embedded (or nested) structs.
For instance, this is how you would handle embedded structs:
type Employee struct {
User `bson:",inline"`
JobTitle string
EmployeeId string
Salary int
}
These sorts of things are specified in that link I provided on bson.Marshal. Hope that helps!
you can change the default encoder/decoder in the following way:
mongo_addr := "mongodb://localhost:27017/"
structcodec, _ := bsoncodec.NewStructCodec(bsoncodec.JSONFallbackStructTagParser)
rb := bson.NewRegistryBuilder()
// register struct codec
rb.RegisterDefaultEncoder(reflect.Struct, structcodec)
rb.RegisterDefaultDecoder(reflect.Struct, structcodec)
clientOptions := options.Client().SetRegistry(rb.Build()).ApplyURI(mongo_addr)
mgclient, err := mongo.Connect(ctx, clientOptions)
if err != nil {
panic(err)
}
Now during marshaling/unmarshalling json tag is used instead of bson.
This trick is a particularly useful when you use tags that both bson and json belong to.
By the way, those clientOptions can be set on collection and database levels too
You can use the following (From thrift test file git.apache.org/thrift.git/lib/go/test/GoTagTest.thrift)
struct tagged {
1: string string_thing,
2: i64 int_thing (go.tag = "json:\"int_thing,string\""),
3: optional i64 optional_int_thing
}
I'm trying to pull reddit content from the json API into a custom structure for the client. the structure i've come up with in go for this is
type Subreddit struct {
offset int
num_of_posts int
subscribers: int
thumbnail string
children []post
}
type post struct {
type string
url string
thumbnail string
submitted_by string
upvotes int
downvotes int
}
unfortunately the reddit json isn't formatted even close to this and in addition i'll want to filter out url's i can't support etc.
The only way i know to do it this is to create an interface for each of the "children" in the source data, and iterate through each child manually, creating an individual "post" for each interface. and pushing them into the subreddit object's post array.
For reference the data is formatted like http://www.reddit.com/r/web_design/.json
Is this the right way to do this? Or is there a faster way. It seems like a lot of overhead for such a small task, but i'm a PHP Javascript dev, so It's just unusual for me I suppose.
Before I even start to answer the question:
Remember that your struct fields must be exported in order to be used with the encoding/json package.
Secondly I must admit I am not entirely sure what you meant with the entire create an interface for each of the "children" part. But it sounded complicated ;)
Anyway, to your answer:
If you wish to use the standard encoding/json package to unmarshal the json, you must use an intermediate structure unless you will use a similar structure as the one used by Reddit.
Below you can find an example of how parts of the Reddit structure might be mapped to Go structs. By Unmarshalling the json into an instance of RedditRoot, you can then easily iterate over the Children , remove any unwanted child, and populate your Subreddit struct:
type RedditRoot struct {
Kind string `json:"kind"`
Data RedditData `json:"data"`
}
type RedditData struct {
Children []RedditDataChild `json:"children"`
}
type RedditDataChild struct {
Kind string `json:"kind"`
Data *Post `json:"data"`
}
type Post struct {
Type string `json:"-"` // Is this equal to data.children[].data.kind?
Url string `json:"url"`
Thumbnail string `json:"thumbnail"`
Submitted_by string `json:"author"`
Upvotes int `json:"ups"`
Downvotes int `json:"downs"`
}