I use AWS Lambda with DynamoDB using golang.
My DynamoDB table use lowercase attribute names such as id or name.
In Go, if I want to be able to marshal a struct correctly, I have to name fields starting with a capital letter.
type Item struct {
ID string
Name string
}
To put an item into my DynamoDB table, I have to marshal it into a map[string]*dynamodb.AttributeValue, using dynamodbattribute.MarshalMap function.
item := Item{
ID: "xxxx",
Name: "yyyy"
}
av, _ := dynamodbattribute.MarshalMap(item)
Of course, this will create a map using names written as ID and Name, which are incompatible with id and name from the dynamodb table.
Reading the documentation, I found that you can use a custom encoder, and enable json tags.
type Item struct {
ID string `json="id"`
Name string `json="name"`
}
func setOpts(encoder *dynamodbattribute.Encoder) {
// Not sure which one I sould set
// So I set them both :)
encoder.SupportJSONTags = true
encoder.MarshalOptions.SupportJSONTags = true
}
func main() {
encoder := dynamodbattribute.NewEncoder(setOpts)
encoder.Encode(...)
}
But here the encoder.Encode() method is only used to create a dynamodb.AttributeValue, and not a map[string]*dynamodb.AttributeValue.
Is there a way to use a custom encoder with MarshalMap? Or am I using it in a wrong way?
EDIT:
Okay so as Zak pointed out, there is a dynamodbav tag that can be used.
I also found out that I was using json tags in a wrong way. I should use the syntax json:"id" instead of json="id".
Indeed, DynamoDB SDK uses the json tag if available, and this one can be overrided by the dynamodbav.
So all I had to do was to make my structure looks like this and it worked
type Item struct {
ID string `json:"id"`
Name string `json:"name"`
}
Dynamo's built in marshalling, from MarshalMap(...) can support struct tags, similar to json.
You can add them to the type that you are marshalling, like so:
type Item struct {
ID string `dynamodbav:"id"`
Name string `dynamodbav:"name"`
}
See the docs here
Related
I have a struct -
type User struct {
Uid string `firestore:"uid"`
FcmToken string `firestore:"fcmtoken"
}
How do I convert It to map using json.Marshal(user) ,
I know it can be done when struct fields are annotated with json:"fieldname" but I don't know how to do the same when it is annotated with firestore, or is it even possible?
I have used the word annotation, which may not be what it is called, please correct me!
A field tag can contain multiple key/value pairs. See the struct tag documentation for more details.
Edit the field tags to include whatever JSON configuration you want:
type User struct {
Uid string `firestore:"uid" json:"uid"`
FcmToken string `firestore:"fcmtoken" json:"tid"`
}
It is not possible to make the JSON package use the firestore tags or vice versa.
I'm trying to create a go struct for my project. I want the struct content to be defined in a separate JSON file, as example below :
{
"structA":{
"column_name" :["id","name","occupation","age"],
"column_type" :["int","string","string","int"]
}
And when we compile go, the struct will be somehow like
type structA struct {
id int
name string
occupation string
age int
}
I want the structA definition to be written in a way like looping though the json and generate it afterwards. Is it possible to do so in Go?
You can use JSON schema to define your types and generate Go code with your types using this package: https://github.com/idubinskiy/schematyper
I'm writing an application in Go that will recieve two forms of JSON:
Example 1:
{"book_data":{"title":"book-title","page_number":457}}
Example 2:
{"book_data":{"collection":214},"books":{"data":[{"title":"book-title","page_number":457},{"title":"book-title","page_number":354}]}}
I thought that I could create a struct like the following and unmarshal JSON into it:
type Book struct {
Title string `json:"title"`
PageNumber int `json:"page_number"`
}
but that only works for the first example.
How can I handle JSON from both examples?
You can first unmarshal partly in json.RawMessage to next decide depending of unmarshalled payload. And you also can just unmarshal in more generic structure. Something like
type Book struct {
Title string `json:"title"`
PageNumber int `json:"page_number"`
}
type BookShelf struct {
BookData struct {
Book
Collection int `json:"collection"`
} `json:"book_data"`
Books struct {
Data []Book `json:"data"`
} `json:"books"`
}
which for me looks readable, meaningful and handy enough for further processing.
Why not unmarshal to a map[string]interface{} and then use the result to see which form you need to handle? You can then deserialize with a specific struct type for each form.
Another way would be to use the following package to check for differing attributes, so you can decide which struct to use for the real unmarshaling.
https://github.com/go-xmlpath/xmlpath/tree/v2
You can unmarshal to map because your key is string and value may be anything like - map[string]interface{}. If you are not sure any data type or value then use interface{} beacause it can store any value. Then use result to see which form it is, And deserialize to specific struct type.
Another way to convert JSON to go struct is use this tool.
https://mholt.github.io/json-to-go/
I'd like to marshal part of this struct:
type ValueSet struct {
Id string `json:"id" bson:"_id"`
Url string `bson:"url,omitempty" json:"url,omitempty"`
Identifier *Identifier `bson:"identifier,omitempty" json:"identifier,omitempty"`
Version string `bson:"version,omitempty" json:"version,omitempty"`
Name string `bson:"name,omitempty" json:"name,omitempty"`
Status string `bson:"status,omitempty" json:"status,omitempty"`
Experimental *bool `bson:"experimental,omitempty" json:"experimental,omitempty"`
Publisher string `bson:"publisher,omitempty" json:"publisher,omitempty"`
Contact []ValueSetContactComponent `bson:"contact,omitempty" json:"contact,omitempty"`
Date *FHIRDateTime `bson:"date,omitempty" json:"date,omitempty"`
LockedDate *FHIRDateTime `bson:"lockedDate,omitempty" json:"lockedDate,omitempty"`
Description string `bson:"description,omitempty" json:"description,omitempty"`
UseContext []CodeableConcept `bson:"useContext,omitempty" json:"useContext,omitempty"`
Immutable *bool `bson:"immutable,omitempty" json:"immutable,omitempty"`
Requirements string `bson:"requirements,omitempty" json:"requirements,omitempty"`
Copyright string `bson:"copyright,omitempty" json:"copyright,omitempty"`
Extensible *bool `bson:"extensible,omitempty" json:"extensible,omitempty"`
CodeSystem *ValueSetCodeSystemComponent `bson:"codeSystem,omitempty" json:"codeSystem,omitempty"`
Compose *ValueSetComposeComponent `bson:"compose,omitempty" json:"compose,omitempty"`
Expansion *ValueSetExpansionComponent `bson:"expansion,omitempty" json:"expansion,omitempty"`
}
which is part of a Go implementation of HL7 FHIR, including only the metadata fields, and omitting the three content three fields (codeSystem, compose and expansion). I can't (and shouldn't) change the JSON tags in the original source code, since other code strongly depends on it working the way it's written. How can I tell json.Marshal to override the existing JSON tags on these struct elements?
You can't change it, but you don't have to.
Easiest solution is to create your own struct, define your own json tags (how you want them to appear in the output), copy the fields, and marshal a value of your own struct.
E.g. let's say you want to marshal the Id and Urlfields, then:
type MyValueSet struct {
Id string `json:"MyId"`
Url string `json:"MyUrl"`
}
var vs ValueSet = ... // Comes from somewhere
mvs := MyValueSet {
Id: vs.Id,
Url: vs.Url,
}
data, err := json.Marshal(&mvs)
// Check err
I am using thrift in my project, thrift will generate code as follow:
type CvJdRelationInfo struct {
JdId string `thrift:"jdId,1" json:"jdId"`
CvId string `thrift:"cvId,2" json:"cvId"`
Status int16 `thrift:"status,3" json:"status"`
AcceptTimestamp int64 `thrift:"acceptTimestamp,4" json:"acceptTimestamp"`
}
as you see thrift already generate json tags(but no bson tags), when I use mgo save record, mgo will auto convert:
JdId -> jdid
CvId -> cvid
Status -> status
AcceptTimeStamp -> accepttimestamp
what I needed is:
type CvJdRelationInfo struct {
JdId string `thrift:"jdId,1" json:"jdId" bson:"jdId"`
CvId string `thrift:"cvId,2" json:"cvId" bson:"cvId"`
Status int16 `thrift:"status,3" json:"status" bson:"status"`
AcceptTimestamp int64 `thrift:"acceptTimestamp,4" json:"acceptTimestamp" bson:"acceptTimestamp"`
}
as your can see, bson tags is the same as json tags. Can I use json tags as bson tags?
MongoDB actually stores the data as binary JSON (bson), which is distinct from JSON. It's slightly confusing because if you access the database using the mongo shell, you get back raw JSON, but it's actually a conversion, it's not the storage format. So in storing the data to the database, the "mgo" driver serializes as bson.
This serializing ignores the json export keys, and chooses the appropriate name by defaulting to the lowercase version of your struct field. (See the bson.Marshal go doc.) If you specify a bson export key, it will then ignore the struct field name and go with whatever you have specified as the bson export key.
For example,
type User struct {
Name string
UserAge int `bson:"age"`
Phone string `json:"phoneNumber"`
}
will result in the following structure in MongoDB:
{
"name": "",
"age": 0,
"phone": ""
}
So it seems like your struct fields should handle most things for you.
The one 'gotcha' that you might not see until it bites you is, if you don't specify the bson export keys, you don't have the ability to do bson:",omitempty" for leaving out blank fields, or bson:",inline" for marshaling embedded (or nested) structs.
For instance, this is how you would handle embedded structs:
type Employee struct {
User `bson:",inline"`
JobTitle string
EmployeeId string
Salary int
}
These sorts of things are specified in that link I provided on bson.Marshal. Hope that helps!
you can change the default encoder/decoder in the following way:
mongo_addr := "mongodb://localhost:27017/"
structcodec, _ := bsoncodec.NewStructCodec(bsoncodec.JSONFallbackStructTagParser)
rb := bson.NewRegistryBuilder()
// register struct codec
rb.RegisterDefaultEncoder(reflect.Struct, structcodec)
rb.RegisterDefaultDecoder(reflect.Struct, structcodec)
clientOptions := options.Client().SetRegistry(rb.Build()).ApplyURI(mongo_addr)
mgclient, err := mongo.Connect(ctx, clientOptions)
if err != nil {
panic(err)
}
Now during marshaling/unmarshalling json tag is used instead of bson.
This trick is a particularly useful when you use tags that both bson and json belong to.
By the way, those clientOptions can be set on collection and database levels too
You can use the following (From thrift test file git.apache.org/thrift.git/lib/go/test/GoTagTest.thrift)
struct tagged {
1: string string_thing,
2: i64 int_thing (go.tag = "json:\"int_thing,string\""),
3: optional i64 optional_int_thing
}