let me start by telling that I'm pretty recent in the Go world.
What I'm trying to do is to read the json I get from a JSON API (I don't control). Everything is working fine, I can show the received ID and Tags too. But the fields field is a little bit different, because its a dynamic array.
I can receive from the api this:
{
"id":"M7DHM98AD2-32E3223F",
"tags": [
{
"id":"9M23X2Z0",
"name":"History"
},
{
"id":"123123123",
"name":"Theory"
}
],
"fields": {
"title":"Title of the item",
"description":"Description of the item"
}
}
Or instead of title and description I could receive only description, or receive another random object like long_title. The objects return may differ completly and can be an infinite possibility of objects. But it always returns objects with a key and a string content like in the example.
This is my code so far:
type Item struct {
ID string `json:"id"`
Tags []Tag `json:"tags"`
//Fields []Field `json:"fields"`
}
// Tag data from the call
type Tag struct {
ID string `json:"id"`
Name string `json:"name"`
}
// AllEntries gets all entries from the session
func AllEntries() {
resp, _ := client.Get(APIURL)
body, _ := ioutil.ReadAll(resp.Body)
item := new(Item)
_ = json.Unmarshal(body, &item)
fmt.Println(i, "->", item.ID)
}
So the Item.Fields is dynamic, there is no way to predict what will be the key names, and therefore as far I can tell, there is no way to create a struct for it. But again, I'm pretty newbie with Go, could someone give me any tips? Thanks
If the data in "fields" is always going to be a flat-dict then you can use map[string]string as type for the Fields.
For arbitrary data specify Fields as a RawMessage type and parse it later on based on its content. Example from docs: https://play.golang.org/p/IR1_O87SHv
If the fields are way too unpredictable then you can keep this field as is([]byte) or if there are fields that are always going to common then you can parse those and leave the rest(but this would result in loss of data present in other fields).
Related
I want to omit certain structs nested in a JSON request. I've created a rest API on golang which reads a message body from an http request, decodes it into the struct defined in the code and inserts it into Mongo DB
My structs are as follows. Note that for the nested structure C, I use a pointer in order to be able to omit it.
type A struct {
Title string `json:"title"`
Text string `json:"text"`
Data B `json:"data"`
}
type B struct {
Product *C `json:"product,omitempty"`
ExternalLink string `json:"external_link,omitempty"`
}
type C struct {
Name string `json:"name"`
Id int `json:"id"`
}
Here is how I decode it (Didn't go for Json.Unmarshall as I read that for http bodies, decoding should be used over unmarshall)
func NewMessage(req *http.Request) *A {
var newMessage *A
json.NewDecoder(req.Body).Decode(&newMessage)
messageInData := newMessage
return newMessage
}
The "newMessage" upon return is inserted into Mongo directly. However, Even if the request payload contains no such object as the struct C for instance like below
{
"title": "First message from GoLang",
"text": "Hello Dastgyr",
"data": {
"external_link": "some link here"
//no product object (C struct) here
}
}
The object inserted into Mongo still contains the struct C as having a null value as shown below
{
"title": "First message from GoLang",
"text": "Hello Dastgyr",
"data": {
"product": null,
"external_link": "some link here"
}
}
I've also tried using B as a pointer in Struct A but to no avail
type A struct {
Title string `json:"title"`
Text string `json:"text"`
Data *B `json:"data,omitempty"`
}
I want to be able to omit certain nested structs. Despite using pointers, the struct I want is still not omitting. What mistake am I making in defining structs?
Still new to golang so a nudge in the right direction would help
You are using json tags for json un-marshaling, it seems to be un-marshaling correctly (concluding this as you didn't mention any errors, and moved on to MongoDB)
How you add data to MongoDB is a totally different matter, and has nothing to do with your JSON tags. It uses bson tags, and you will need to add them, if you wish to use this same struct as a mongo DB model representation. Something like this:
type A struct {
Title string `json:"title" bson:"title"`
Text string `json:"text" bson:"text"`
Data *B `json:"data,omitempty" bson:"data,omitempty"`
}
Remember that tags in golang, are just some metadata being added with a structure, which some code actually reads and acts on. json library identifies & processes json:"" tag, while official go mongodb library that you might be using, will process the bson:"" tags.
I'm working on unmarshaling some nested json data that I have already written a struct for. I've used a tool that will generate a struct based off json data, but am a bit confused how to work with accessing nested json data (and fields can sometimes be emtpy).
Here is an example of struct:
type SomeJson struct {
status string `json:"status"`
message string `json:"message"`
someMoreData []struct {
constant bool `json:"constant,omitempty"`
inputs []struct {
name string `json:"name"`
type string `json:"type"`
} `json:"inputs,omitempty"`
Name string `json:"name,omitempty"`
Outputs []struct {
Name string `json:"name"`
Type string `json:"type"`
} `json:"outputs,omitempty"`
I'm able to unmarshal the json data and access the top level fields (such as status and message), but am having trouble accessing any data under the someMoreData field. I understand this field is a (I assume an unknown) map of structs, but only have experience working with basic single level json blobs.
For reference this is the code I have to unmarshal the json and am able to access the top level fields.
someData := someJson{}
json.Unmarshal(body, &someData)
So what is exactly the best to access some nested fields such as inputs.name or outputs.name?
to iterate over your particular struct you can use:
for _, md := range someData.someMoreData {
println(md.Name)
for _, out := range md.Outputs {
println(out.Name, out.Type)
}
}
to access specific field:
someData.someMoreData[0].Outputs[0].Name
Couple of things to note:
The struct definition is syntactically incorrect. There are couple of closing braces missing.
type is a keyword.
The status and message and other fields with lower case first letter fields are unexported. So, the Json parser will not throw error, but you will get zero values as output. Not sure that's what you observed.
someMoreData is an array of structs not map.
I'm currently developing a JSON API for a blog in golang, and I've run into a roadblock trying to handle the serialization and deserialization of blog posts. I want my posts to contain an array of Post Sections that could be a number of things (such as normal paragraphs, images, quotes, etc.). I'm using Mongo for storage (with the amazing mgo library) and I want to save the posts like this:
{
"title": "Blog post",
"sections": [
{
"type": "text",
"content": { "en": "English content", "de": "Deutscher Inhalt" }
},
{
"type": "image",
"content": "https://dummyimage.com/100x100"
},
...more sections
],
...other fields
}
I've tried several solutions to implement this in go and none have really seemed like the "right way" to do it:
Not caring about the content
This seemed like the obvious solution, just using a simple struct:
type PostSection struct{
Type string
Content interface{}
}
This way, I can pass through whatever the frontend POSTS and save it. However, manipulating the data or validating it becomes impossible, so it's not a good solution.
Using custom interface serialization
I found this article about serializing interfaces in golang. This seemed great at first, because I could have an interface like this:
type PostSection interface{
Type() string
Content() interface{}
}
and then implement every type like this:
type PostImage string
func (p *PostImage) Type() string {
return "image"
}
func (p *PostImage) Content() interface{} {
return p
}
Optimally, that would've been it, and after implementing MarshalJSON and UnmarshalJSON for all my types, it was working fine when using json.Marshal directly on a PostSection object.
However, when serializing or deserializing an entire Post object containing an array of PostSections, my custom code was just being ignored and the PostSections would just be treated as the underlying objects (string or map[string]string in the examples) when serializing, or result in empty objects when deserializing.
Writing custom serialization for the entire Post struct
So, the solution I'm currently using but would like to change is custom serialization for the entire Post object. This leads to super ugly code, as I only really need custom code for a single field and so I'm passing through the rest, making the deserialization look similar to this:
p.ID = decoded.ID
p.Author = decoded.Author
p.Title = decoded.Title
p.Intro = decoded.Intro
p.Slug = decoded.Slug
p.TitleImage = decoded.TitleImage
p.Images = decoded.Images
...more fields...
and then, decoding the sections like this:
sections := make([]PostSection, len(decoded.Sections))
for i, s := range decoded.Sections {
if s["type"] == "text" {
content := s["content"].(map[string]interface{})
langs := make(PostText, len(content))
for lang, langContent := range content {
langString := langContent.(string)
langs[lang] = langString
}
sections[i] = &langs
} else if s["type"] == "image" {
content := s["content"].(string)
contentString := PostImage(content)
sections[i] = &contentString
}
}
p.Sections = sections
This is a whole lot of code I'll have to use every time I wanna include PostSections in another form somewhere else (for example in a Newsletter) and it doesn't feel like idiomatic go code by a long shot. Also, there is no error handling for malformed sections - They just cause a panic like this.
Is there a clean solution to this problem?
To avoid writing UnmarshalJSON for the whole Post you can wrap your PostSection in a concrete type and have it implement the Unmarshaler interface.
type Post struct {
ID int
Author string
Title string
Intro string
Slug string
TitleImage string
Images []string
Sections []*PostSection
}
type SectionContent interface {
Type() string
Content() interface{}
}
type PostSection struct {
Content SectionContent
}
func (s *PostSection) UnmarshalJSON(data []byte) error {
// ...
return nil
}
I have JSON (that I cannot control) like this:
{
"foo1":{
"a":{
"up":10,
"down":5
}
},
"foo2":{
"a":{
"up":1,
"down":1
}
},
"bar":{
"up":11,
"down":6
}
}
"foo1" and "foo2" are dynamic.
How can I properly unmarshal this structure in go?
It would be okay if I could just tell go to not try to deserialize "bar" (the inconsistent property).
Go will by default ignore fields unspecified in the struct you unmarshal into.
In this case, your structure would be set up like this:
type NestedProp2 struct {
Up int
Down int
}
type NestedProp struct {
A NestedProp2
}
type Prop struct {
Foo1 NestedProp
Foo2 NestedProp
}
When you call the the json.Unmarshal function, the extra property will not be deserialized:
var prop Prop
err := json.Unmarshal(jsonBlob, &prop)
if err != nil {
fmt.Println("error:", err)
}
fmt.Printf("%+v", prop)
So you get the following output:
{Foo1:{A:{Up:10 Down:5}} Foo2:{A:{Up:1 Down:1}}}
You can see it in action here.
You said:
I have JSON (that I cannot control)
So to what extent you could control? Here I could provide you with some scenario, and hope some of them match your purpose :)
Remember the general rule first:
In Golang, if a JSON key failed to find a matched field in struct, it will not be unmarshalled.
This means, for a key name in a JSON struct, when unmarshalling, it will look for a field in a golang struct at the same level with the same name case-insensitively. If this search failed, this key won't be unmarshalled.
For example, a key named foo1 will look for a field name foo1 in a golang struct at the same indent level. However it also matches with Foo1 or FoO1, since this matching is case-insensitive.
Remember, you could use field tag to specify the field name as well. Please take a look at the official page.
The value of some of the JSON fields are not consistent, and they could be ignored.
This is the case #gnalck solved in his answer. According to the general rule, if those inconsistent field failed to find a match, they will not be unmarshalled. Therefore, just don't put those inconsistent fields in the struct and you will be fine.
The value of some of the JSON fields are not consistent, but they could not be ignored.
In this case, #gnalck failed since those fields could not be ignored. Now a better way is to unmarshal bar into a json.RawMessage, so that you could unmarshal later.
The keys of the JSON object is undetermined, and their value is undetermined as well.
In this case, we could unmarshal the whole JSON object into a map[string]json.RawMessage, and unmarshal each fields later. When unmarshalling to a map, you could iterate through the map to get all the fields, and unmarshal them into a proper struct later.
I'm using the neoism library (https://github.com/jmcvetta/neois) to talk to a local neo4j database - I'm fairly new to go and am new to neo4j so the gap in my understanding could be on either side of the problem.
I have a simple database, a single "page" node which is related to a single "template" node. I was hoping to be able to have struct representing each node and nest them inside one another but I'm strugging to get this to work.
Creating the simple DB:
template, err := ioutil.ReadFile(viewPath + "templates/default.mustache")
if err != nil{
panic(err)
}
defaultTemplate, _ := db.CreateNode(neoism.Props{
"name": "default",
"content": string(template),
})
defaultTemplate.AddLabel("Template")
n0, _ := db.CreateNode(neoism.Props{
"name": "Home",
"slug": "home",
"title": "Home Page",
"content" : "here I am",
})
n0.AddLabel("Page")
n0.Relate("TEMPLATE", template.Id(), neoism.Props{})
Now to the business of trying to get the data back out...
Here is my query which works just fine:
type PageStruct struct{
Name string `json:"p.name"`
Slug string `json:"p.slug"`
Title string `json:"p.title"`
Content string `json:"p.content"`
TemplateName string `json:"t.name"`
TemplateContent string `json:"t.content"`
}
res := []PageStruct{}
cq := neoism.CypherQuery{
Statement: `
MATCH (p:Page)-[r:TEMPLATE]->(t:Template)
WHERE p.slug = {slug}
RETURN p.name, p.slug, p.title, p.content, t.name, t.content
`,
Parameters: neoism.Props{"slug": pageSlug},
Result: &res,
}
db.Cypher(&cq)
page := res[0]
But ideally what I want is to unmarshall into a nested struct, something like this:
type PageStruct struct{
Name string `json:"p.name"`
Slug string `json:"p.slug"`
Title string `json:"p.title"`
Content string `json:"p.content"`
Template struct {
Name string `json:"t.name"`
Content string `json:"t.content"`
} `json:"t"`
}
I've been trying various things with no success, could anyone give me any advice on how to achieve this abitious feat of computer engineering...
Additionally I'm slightly unclear as to how to handle relationships in neo4j in the sense of having no enforcement (that I'm aware of) as to the type of relationship (e.g one-to-one, one-to-many) so how to handle this - do we always just assume an array of data?
Any help/advice is greatly appreciated.
Use embedding.
type Page struct {
Name string `json:"p.name"`
// ...
Template
}
type Template struct {
Name string `json:"t.name"`
// ...
}
Playground: http://play.golang.org/p/B3ro3wgsGS.