unmarshal generic json in Go [duplicate] - json

This question already has answers here:
JSON and dealing with unexported fields
(2 answers)
(un)marshalling json golang not working
(3 answers)
json.Marshal(struct) returns "{}"
(3 answers)
Printing Empty Json as a result [duplicate]
(1 answer)
Parsing JSON in Golang doesn't Populate Object [duplicate]
(1 answer)
Closed 10 months ago.
I'm a new Go programmer (From Java) and I would like to reproduce a generic way which is esay to use in Java.
I want to create some function which allow me to do an Unmarshal on a JSON string in order to avoid code duplicity.
This is my current code which is not working :
type myStruct1 struct {
id string
name string
}
func (obj myStruct1) toString() string {
var result bytes.Buffer
result.WriteString("id : ")
result.WriteString(obj.id)
result.WriteString("\n")
result.WriteString("name : ")
result.WriteString(obj.name)
return result.String()
}
func main() {
content := `{id:"id1",name="myName"}`
object := myStruct1{}
parseJSON(content, object)
fmt.Println(object.toString())
}
func parseJSON(content string, object interface{}) {
var parsed interface{}
json.Unmarshal([]byte(content), &parsed)
}
This code, on run, returns me this :
id :
name :
Do you have any idea ?
Thanks

The issue is you want to write to a generic type? You probably want a string map. This works with BSON anyways:
var anyJson map[string]interface{}
json.Unmarshal(bytes, &anyJson)
You'll be able to access the fields like so:
anyJson["id"].(string)
Don't forget to type assert your values, and they must be the correct type or they'll panic. (You can read more about type assertions on the golang site)

To parse "generic JSON" when you have no idea what schema it has:
var parsed any
err := json.Unmarshal(jsonText, &parsed)
The returned any in parsed will be a map[string]any or []any or nil or single values float64, bool, string.
You can test the type and react accordingly.
import (
"encoding/json"
"fmt"
)
func test(jsonText []byte) {
// parsing
var parsed any
err := json.Unmarshal(jsonText, &parsed)
if err != nil {
panic(err) // malformed input
}
// type-specific logic
switch val := parsed.(type) {
case nil:
fmt.Println("json specifies null")
case map[string]any:
fmt.Printf("id:%s name:%s\n", val["id"], val["name"])
case []any:
fmt.Printf("list of %d items\n", len(val))
case float64:
fmt.Printf("single number %f\n", val)
case bool:
fmt.Printf("single bool %v\n", val)
case string:
fmt.Printf("single string %s\n", val)
default:
panic(fmt.Errorf("type %T unexpected", parsed))
}
}

Unmarshal will only set exported fields of the struct.
Which means you need to modify the json struct to use capital case letters:
type myStruct1 struct {
Id string
Name string
}
The reason behind this is that the json library does not have the ability to view fields using reflect unless they are exported.

You have to export your fields:
type myStruct1 struct {
Id string
Name string
}
See Exported Identifiers from documentation.

There are a few changes you need to make in your code to make it work:
The function json.Unmarshal can only set variables inside your struct which are exported, that is, which start with capital letters. Use something like ID and Name for your variable names inside myStruct1.
Your content is invalid JSON. What you actually want is {"ID":"id1","Name":"myName"}.
You're passing object to parseJSON but you're using parsed instead, not object. Make parseJSON receive a *myStruct (instead of an interface{}), and use that variable instead of parsed when unmarshalling the string. Also, always handle the error returns, like err := json.Unmarshal(content, object), and check err.
I'd suggest you to do the Golang Tour ;)

You can also set the file as an Object with dynamic properties inside another struct. This will let you add metadata and you read it the same way.
type MyFile struct {
Version string
Data map[string]interface{}
}

Related

How to render a json string from bson document [duplicate]

This question already has answers here:
How to convert bson to json effectively with mongo-go-driver?
(1 answer)
convert result into JSON without structs using mongo-go-driver
(1 answer)
How can I get JSON from BSON without my keys all being named "Key"?
(1 answer)
Closed 3 years ago.
I am struggling to create a valid JSON string from a BSON document in Go for an API.
Let's say I have an object like this:
type MyObject struct {
Name string
}
I call my database which returns to me a cursor containing many documents as: [{"Name": "object_name"}, ...]
I am able to retrieve all the documents via a loop like
for cur.Next(ctx) {
var obj MyObject
err := cur.Decode(&obj)
//then display error if there's one
}
And now I would like to end up with a JSON string containing all the documents my database returned in order to send it via HTTP.
Because, if use I fmt.Println(obj)I end up with something like this: [{object1_name} {object2_name} ...] which is, according to me, not a valid format that I can use for an API.
I know json.Marshal(obj) can actually encode to valid JSON and I can decode it with os.Stdout.Write(obj) but I didn't manage to store this valid string in a variable. How can I manage to do this?
From Golang documentation for json package
package main
import (
"encoding/json"
"fmt"
)
func main() {
type ColorGroup struct {
ID int `json:"id"`
Name string `json:"name"`
Colors []string `json:"colors"`
}
group := ColorGroup{
ID: 1,
Name: "Reds",
Colors: []string{"Crimson", "Red", "Ruby", "Maroon"},
}
b, err := json.Marshal(group)
if err != nil {
fmt.Println("error:", err)
} else {
str := string(b)
fmt.Println("stringified json is:", str)
}
}
Output
stringified json is: {"id":1,"name":"Reds","colors":["Crimson","Red","Ruby","Maroon"]}
The json.Marshal return two values - a byte array and error
If error is nil then you can obtain the string by converting byte array to string using
str := string(b)

[Go]: Parsing JSON [duplicate]

This question already has answers here:
Lowercase JSON key names with JSON Marshal in Go
(4 answers)
Closed 5 years ago.
What I am trying to do
I am parsing a JSON HTTP response based on this answer to a similar question. My code is able to parse the JSON without any error but is unable to read the values and store them in the provided variable.
This has been puzzling me for the last 2 hours and it might be due to a trivial reason that I am overlooking here.
CODE
type ImporterResponse struct {
results []packagemeta `json:"results"`
}
type packagemeta struct {
path string `json:"path"`
synopsis string `json:"synopsis,omitempty"`
count int `json:"import_count,omitempty`
}
func main() {
res := []byte(`{"results":[{"path":"4d63.com/randstr/lib/randstr","import_count":0,"synopsis":"Package randstr generates random strings (e.g."},{"path":"bitbucket.org/pcas/tool/mathutil","import_count":0}]}`)
fmt.Println("Decoding the JSON")
r := bytes.NewReader(res)
decoder := json.NewDecoder(r)
packageimporters := &ImporterResponse{}
err := decoder.Decode(packageimporters)
if err != nil {
log.Fatal(err)
}
fmt.Printf("Packageimporters: %+v", packageimporters)
fmt.Println(len(packageimporters.results))
}
Link to Playground: https://play.golang.org/p/NzLl7Ujo2IJ
What I want:
How to fix this?
Why is no error message raised if JSON is not parsed properly?
P.S: I understand that this question has been asked before and there are possible solutions available but none of them work for me. Hence, I have made this post.
You need to make your struct fields exported, otherwise the json package cannot access them.
Please read JSON and go for more details, specifically this paragraph:
The json package only accesses the exported fields of struct types
(those that begin with an uppercase letter). Therefore only the the
exported fields of a struct will be present in the JSON output.
And this one for more details:
How does Unmarshal identify the fields in which to store the decoded
data? For a given JSON key "Foo", Unmarshal will look through the
destination struct's fields to find (in order of preference):
An exported field with a tag of "Foo" (see the Go spec for more on
struct tags),
An exported field named "Foo", or
An exported field
named "FOO" or "FoO" or some other case-insensitive match of "Foo".
So your struct should really be:
type Packagemeta struct {
Path string `json:"path"`
Synopsis string `json:"synopsis,omitempty"`
Count int `json:"import_count,omitempty`
}

How to deal with Struct having different json key than json response

I have a struct VideoInfo that has a key in it called embedCode. The API I am querying returns the embed code as embed_code. During unmarshalling the response how do I ensure embed_code goes into embedCode?
Also is there an easy way to take a large json string and automatically turn it into a struct, or can one only use a map?
With respect to remapping the field names use the corresponding annotation in the structure declaration:
type VideoInfo struct {
EmbedCode string `json:"embed_code"`
}
The marshaller/un-marshaller will only process public field, so you need to capitalise the field name.
With respect to converting the whole structure, yes it is easy. Declare an instance to un-marshal into and pass a reference to the json.Unmarshal method (from a test):
data, _ := json.Marshal(request)
var resp response.VideoInfo
if err := json.Unmarshal(data, &resp); err != nil {
t.Errorf("unexpected error, %v", err)
}
At first, struct's field must be start from capital letter to be public. So you need something like that:
type VideoInfo struct {
EmbedCode string `json:"embed_code"`
}
And look at documentation for more info.

JSON Unmarshall not working as expected with structs [duplicate]

This question already has answers here:
My structures are not marshalling into json [duplicate]
(3 answers)
Closed 7 years ago.
I have the following code:
package main
import "encoding/json"
import "fmt"
type SuperNum struct {
num string
}
func main() {
byt := []byte(`{"num":"6.13"}`)
var dat SuperNum
if err := json.Unmarshal(byt, &dat); err != nil {
panic(err)
}
fmt.Printf("%+v", dat) // I expect this to have a `num` attribute
}
Output:
{num:}
Program exited.
You can run this code in the golang playground.
Because I'm setting a num property in the struct and in the JSON and they're both strings, I would have expected the dat struct to have a num property, with 'hello', but it doesn't.
What am I doing wrong? What in my mental model of how this should work is incorrect?
EDIT
I tried adding the json signature to the struct, but it makes no difference (no idea what that actually does).
type SuperNum struct {
num string `json:"num"`
}
num is by convention not exported as it is lower case. Change it to Num and you are able to inspect the result.
type SuperNum struct {
Num string
}
Just change num to Num. The lowercase properties of the structures are not visible.
Go playground
When unmarhalling JSON structures, the properties that you're mapping on must be public (remember that in Go, public and private visibility of struct and module members is denoted by the member's name being upper or lower camel case.
So, first of all, your struct must be defined like this:
type SuperNum struct {
Num string // <- note the capital "N"
}
With this struct, the JSON marshaller will expect the JSON property to be also named Num. In order to configure a different property name (like the lowercased num in your example), use the json annotation for that struct member:
type SuperNum struct {
Num string `json:"num"`
}

How to access interface fields on json decode?

I have a json document and I'm using a client which decodes the document in an interface (instead of struct) as below:
var jsonR interface{}
err = json.Unmarshal(res, &jsonR)
How can I access the interface fields? I've read the go doc and blog but my head still can't get it. Their example seem to show only that you can decode the json in an interface but doesn't explain how its fields can be used.
I've tried to use a range loop but it seems the story ends when I reach a map[string]interface. The fields that I need seem to be in the interface.
for k, v := range jsonR {
if k == "topfield" {
fmt.Printf("k is %v, v is %v", k, v)
}
}
The value inside the interface depends on the json structure you're parsing. If you have a json dictionary, the dynamic type of jsonR will be: map[string]interface{}.
Here's an example.
package main
import (
"encoding/json"
"fmt"
"log"
)
func main() {
a := []byte(`{"topfield": 123}`)
var v interface{}
if err := json.Unmarshal(a, &v); err != nil {
log.Fatalf("unmarshal failed: %s", err)
}
fmt.Printf("value is %v", v.(map[string]interface{})["topfield"])
}
Parsing json like this can be very difficult. The default type of a parse is map[string]interface{}. The Problem arises when you have another complex data structure within the main json(like another list or object). The best way to go about decoding json is defining a struct to hold data. Not only will the values be of the correct type but you can extract the specific data you actually care about.
Your struct can look like this:
type Top struct {
Topfield int `json:"topfield"`
}
which can be decoded like this:
a := []byte(`{"topfield": 123}`)
var data Top
json.Unmarshal(a, &data) //parse the json into data
now you can use regular struct operations to access your data like this:
value := data.Topfield
json which contains more complex data can also be easyli decoded. Perhaps you have a list in your data somewhere, you can use a struct like the following to extract it:
type Data struct {
States []string `json:"states"`
PopulationData []Country `json:"popdata"`
}
type Country struct {
Id int `json:"id"`
LastCensusPop int `json:"lcensuspopulation"`
Gdp float64 `json:"gdp"`
}
such a structure can not only parse list but also parse objects withing fields.