I would like to save an elasticsearch.Config struct to a JSON file. The code looks like this:
package main
import (
"encoding/json"
"fmt"
"github.com/elastic/go-elasticsearch"
"io/ioutil"
)
var EsConfig elasticsearch.Config
func main() {
EsConfig.Addresses = append(EsConfig.Addresses, "http://localhost:9200")
EsConfig.Username = "testuser"
EsConfig.Password = "testpwd"
js, err := json.MarshalIndent(EsConfig, "", " ") //js is nil
if err != nil {
fmt.Printf("After marshalling: %v\n", err)
}
err = ioutil.WriteFile("testconfig.json", js, 0644) //output fine is empty
if err != nil {
fmt.Printf("After WriteFile %v\n", err)
}
}
however the js variable is always nil and getting error
After marshalling: json: unsupported type: func(int) time.Duration
Please, what am I missing here?
elasticsearch.Config contains exported RetryBackoff field of said type func(int) time.Duration. By default, json.Marshall tries to marshal all exported fields and, for obvious reason, fails to do so.
AFAIK, the only way to customize struct fields marshaling is struct tags, and you can not add them to another package type. So the only way I see is to make your own struct with necessary fields from elasticsearch.Config, fill and then marshal it.
Related
I am trying to read the following JSON file:
{
"a":1,
"b":2,
"c":3
}
I have tried this but I found that I had to write each field of the JSON file into a struct but I really don't want to have all my JSON file in my Go code.
import (
"fmt"
"encoding/json"
"io/ioutil"
)
type Data struct {
A string `json:"a"`
B string `json:"b"`
C string `json:"c"`
}
func main() {
file, _ := ioutil.ReadFile("/path/to/file.json")
data := Data{}
if err := json.Unmarshal(file ,&data); err != nil {
panic(err)
}
for _, letter := range data.Letter {
fmt.Println(letter)
}
}
Is there a way to bypass this thing with something like json.load(file) in Python?
If you only want to support integer values, you could unmarshal your data into a map[string]int. Note that the order of a map is not defined, so the below program's output is non-deterministic for the input.
package main
import (
"fmt"
"encoding/json"
"io/ioutil"
)
func main() {
file, _ := ioutil.ReadFile("/path/to/file.json")
var data map[string]int
if err := json.Unmarshal(file ,&data); err != nil {
panic(err)
}
for letter := range data {
fmt.Println(letter)
}
}
You can unmarshal any JSON data in this way:
var data interface{}
if err := json.Unmarshal(..., &data); err != nil {
// handle error
}
Though, in this way you should handle all the reflection-related stuffs
since you don't know what type the root data is, and its fields.
Even worse, your data might not be map at all.
It can be any valid JSON data type like array, string, integer, etc.
Here's a playground link: https://play.golang.org/p/DiceOv4sATO
It's impossible to do anything as simple as in Python, because Go is strictly typed, so it's necessary to pass your target into the unmarshal function.
What you've written could otherwise be shortened, slightly, to something like this:
func UnmarshalJSONFile(path string, i interface{}) error {
f, err := os.Open(path)
if err != nil {
return err
}
defer f.Close()
return json.NewDecoder(f).Decode(i)
}
But then to use it, you would do this:
func main() {
data := Data{}
if err := UnmarshalJSONFile("/path/to/file.json", &data); err != nil {
panic(err)
}
}
But you can see that the UnmarshalJSONFile is so simple, it hardly warrants a standard library function.
Because I often unmarshal http.Response.Body, I thought I could write a function which handles all the hassle of reading, closing and unmarshaling into various different structs. That's why I introduced a function func unmarhalInterface(closer *io.ReadCloser, v *interface{}) error and can then assert the return value with t:=i.(T).
According to this answer I already wrapped it into a value of type *interface{}, but because the overlying type is interface{}and not myStruct, the json package implementation chooses map[string]interface{}. After that a type assertion fails (of course). Is there anything i am missing or requires this implementation a type assertion "by hand", that means look for all fields in the map and assign those that I want into my struct.
Code below has minimal example with notation in comments. If my explanation is not sufficient, please ask away.
package main
import (
"bytes"
"encoding/json"
"fmt"
"io"
"io/ioutil"
"log"
)
type myStruct struct {
A string `json:"a"`
B string `json:"b"`
}
func main() {
jsonBlob := []byte(`{"a":"test","b":"test2"}`)
var foo = interface{}(myStruct{})
closer := ioutil.NopCloser(bytes.NewReader(jsonBlob))
err := unmarshalCloser(&closer, &foo)
if err != nil {
log.Fatal(err)
}
fmt.Println(fmt.Sprintf("%v", foo))
// That´s what i want:
foo2 := foo.(myStruct)
fmt.Println(foo2.A)
}
func unmarshalCloser(closer *io.ReadCloser, v *interface{}) error {
defer func() { _ = (*closer).Close() }()
data, err := ioutil.ReadAll(*closer)
if err != nil {
return err
}
err = json.Unmarshal(data, v)
if err != nil {
return err
}
return nil
}
Golang Playground
An empty interface isn't an actual type, it's basically something that matches anything. As stated in the comments, a pointer to an empty interface doesn't really make sense as a pointer already matches an empty interface since everything matches an empty interface. To make your code work, you should remove the interface wrapper around your struct, since that's just messing up the json type checking, and the whole point of an empty interface is that you can pass anything to it.
package main
import (
"bytes"
"encoding/json"
"fmt"
"io"
"io/ioutil"
"log"
)
type myStruct struct {
A string `json:"a"`
B string `json:"b"`
}
func main() {
jsonBlob := []byte(`{"a":"test","b":"test2"}`)
var foo = &myStruct{} // This need to be a pointer so its attributes can be assigned
closer := ioutil.NopCloser(bytes.NewReader(jsonBlob))
err := unmarshalCloser(closer, foo)
if err != nil {
log.Fatal(err)
}
fmt.Println(fmt.Sprintf("%v", foo))
// That´s what i want:
fmt.Println(foo.A)
}
// You don't need to declare either of these arguments as pointers since they're both interfaces
func unmarshalCloser(closer io.ReadCloser, v interface{}) error {
defer closer.Close()
// v NEEDS to be a pointer or the json stuff will barf
// Simplified with the decoder
return json.NewDecoder(closer).Decode(v)
}
I have a hard time understanding why the code below, which uses the unmarshal method does not work, but then almost the same I write with NewDecoder and it works fine.
package conf
import (
"os"
"io/ioutil"
"encoding/json"
)
type Configuration struct {
Agents []Agent `json:"agents"`
IbmWmqFolder string `json:"ibmWmqFolder"`
}
type Agent struct {
AgentName string `json:"agentName"`
Folders []string `json:"folders"`
}
func LoadConfiguration() (configuration Configuration) {
jsonFile, err := os.Open("config.json")
if err != nil {
panic(err)
}
defer jsonFile.Close()
byteValue, _ := ioutil.ReadAll(jsonFile)
json.Unmarshal(byteValue, configuration)
return
}
but if I do all the same but instead of the two last lines with the byteValue and the unmarshal itself, but use the decoder, it works,
jsonParser := json.NewDecoder(jsonFile)
jsonParser.Decode(&configuration)
return
Thanks!
I would guess that you need to pass a pointer to the configuration, like so:
json.Unmarshal(byteValue, &configuration)
You should also check the error value returned by Unmarshal, e.g.:
err = json.Unmarshal(byteValue, &configuration)
if err != nil {
panic(err)
}
See the the docs.
I an new to programming in Go so apologies if this is something obvious. I have a JSON file named foo.json:
{"type":"fruit","name":"apple","color":"red"}
and I am writing a Go program that has to do something when the "name" value in the JSON file is "apple". It needs no other information from that JSON file as that file is used for a completely different purpose in another area of the code.
I have read documentation on Decode() and Unmarshal() and abut 30 different web pages describing how to read the whole file into structures, etc. but it all seems extremely complicated for what I want to do which is just write the correct code to implement the first 2 lines of this pseudo-code:
file, _ := os.Open("foo.json")
name = file["name"]
if (name == "apple") {
do stuff
}
such that I end up with a Go variable named name that contains the string value apple. What is the right way to do this in Go?
The easiest method to do what you want is to decode into a struct.
Provided the format remains similar to {"type":"fruit","name":"apple","color":"red"}
type Name struct {
Name string `json:"name"`
}
var data []byte
data, _ = ioutil.ReadFile("foo.json")
var str Name
_ = json.Unmarshal(data, &str)
if str.Name == "apple" {
// Do Stuff
}
Your other option is to use third party libraries such as gabs or jason.
Gabs :
jsonParsed, err := gabs.ParseJSON(data)
name, ok := jsonParsed.Path("name").Data().(string)
Jason :
v, _ := jason.NewObjectFromBytes(data)
name, _ := v.GetString("name")
Update :
The structure
type Name struct {
Name string `json:"name"`
}
is the json equivalent of {"name":"foo"}.
So unmarshaling won't work for the following json with different formats.
[{"name":"foo"}]
{"bar":{"name":"foo"}}
PS : As mentioned by W.K.S. In your case an anonymous struct would suffice since you're not using this structure for anything else.
One thing is reading a file and other one is decoding a JSON document. I leave you a full example doing both. To run it you have to have a file called file.json in the same directory of your code or binary executable:
package main
import (
"encoding/json"
"io/ioutil"
"log"
"os"
)
func main() {
f, err := os.Open("file.json") // file.json has the json content
if err != nil {
log.Fatal(err)
}
bb, err := ioutil.ReadAll(f)
if err != nil {
log.Fatal(err)
}
doc := make(map[string]interface{})
if err := json.Unmarshal(bb, &doc); err != nil {
log.Fatal(err)
}
if name, contains := doc["name"]; contains {
log.Printf("Happy end we have a name: %q\n", name)
} else {
log.Println("Json document doesn't have a name field.")
}
log.Printf("full json: %s", string(bb))
}
https://play.golang.org/p/0u04MwwGfn
I have also tried to find a simple solution such as $d = json_decode($json, true) in PHP and came to the conclusion that there is no such simple way in Golang. The following is the simplest solution I could make (the checks are skipped for clarity):
var f interface{}
err = json.Unmarshal(file, &f)
m := f.(map[string]interface{})
if (m["name"] == "apple") {
// Do something
}
where
file is an array of bytes of JSON string,
f interface serves as a generic container for unknown JSON structure,
m is a map returned by the type assertion.
We can assert that f is a map of strings, because Unmarshal() builds a variable of that type for any JSON input. At least, I couldn't make it return something different. It is possible to detect the type of a variable by means of run-time reflection:
fmt.Printf("Type of f = %s\n", reflect.TypeOf(f))
For the f variable above, the code will print Type of f = map[string]interface {}.
Example
And this is the full code with necessary checks:
package main
import (
"fmt"
"os"
"io/ioutil"
"encoding/json"
)
func main() {
// Read entire file into an array of bytes
file, err := ioutil.ReadFile("foo.json")
if (err != nil) {
fmt.Fprintf(os.Stderr, "Failed read file: %s\n", err)
os.Exit(1)
}
var f interface{}
err = json.Unmarshal(file, &f)
if (err != nil) {
fmt.Fprintf(os.Stderr, "Failed to parse JSON: %s\n", err)
os.Exit(1)
}
// Type-cast `f` to a map by means of type assertion.
m := f.(map[string]interface{})
fmt.Printf("Parsed data: %v\n", m)
// Now we can check if the parsed data contains 'name' key
if (m["name"] == "apple") {
fmt.Print("Apple found\n")
}
}
Output
Parsed data: map[type:fruit name:apple color:red]
Apple found
The proper Go way of doing this would be to decode into an instance of an anonymous struct containing only the field you need.
func main() {
myStruct := struct{ Name string }{}
json.Unmarshal([]byte(`{"type":"fruit","name":"apple","color":"red"}`), &myStruct)
fmt.Print(myStruct.Name)
}
Playground Link
Alternatively, You could use Jeffails/gabs JSON Parser:
jsonParsed,_ := gabs.ParseJSON([]byte(`{"type":"fruit","name":"apple","color":"red"}`));
value, ok = jsonParsed.Path("name").Data().(string)
I am trying to convert a Go struct to JSON using the json package but all I get is {}. I am certain it is something totally obvious but I don't see it.
package main
import (
"fmt"
"encoding/json"
)
type User struct {
name string
}
func main() {
user := &User{name:"Frank"}
b, err := json.Marshal(user)
if err != nil {
fmt.Printf("Error: %s", err)
return;
}
fmt.Println(string(b))
}
Then when I try to run it I get this:
$ 6g test.go && 6l -o test test.6 && ./test
{}
You need to export the User.name field so that the json package can see it. Rename the name field to Name.
package main
import (
"fmt"
"encoding/json"
)
type User struct {
Name string
}
func main() {
user := &User{Name: "Frank"}
b, err := json.Marshal(user)
if err != nil {
fmt.Println(err)
return
}
fmt.Println(string(b))
}
Output:
{"Name":"Frank"}
Related issue:
I was having trouble converting struct to JSON, sending it as response from Golang, then, later catch the same in JavaScript via Ajax.
Wasted a lot of time, so posting solution here.
In Go:
// web server
type Foo struct {
Number int `json:"number"`
Title string `json:"title"`
}
foo_marshalled, err := json.Marshal(Foo{Number: 1, Title: "test"})
fmt.Fprint(w, string(foo_marshalled)) // write response to ResponseWriter (w)
In JavaScript:
// web call & receive in "data", thru Ajax/ other
var Foo = JSON.parse(data);
console.log("number: " + Foo.number);
console.log("title: " + Foo.title);
This is an interesting question, it is very easy using the new go versions. You should do this:
package main
import (
"fmt"
"encoding/json"
)
type User struct {
Name string `json:"name"`
}
func main() {
user := &User{name:"Frank"}
b, err := json.Marshal(user)
if err != nil {
fmt.Printf("Error: %s", err)
return;
}
fmt.Println(string(b))
}
Change this name to Name.
You can define your own custom MarshalJSON and UnmarshalJSON methods and intentionally control what should be included, ex:
package main
import (
"fmt"
"encoding/json"
)
type User struct {
name string
}
func (u *User) MarshalJSON() ([]byte, error) {
return json.Marshal(&struct {
Name string `json:"name"`
}{
Name: "customized" + u.name,
})
}
func main() {
user := &User{name: "Frank"}
b, err := json.Marshal(user)
if err != nil {
fmt.Println(err)
return
}
fmt.Println(string(b))
}
Struct values encode as JSON objects. Each exported struct field becomes
a member of the object unless:
the field's tag is "-", or
the field is empty and its tag specifies the "omitempty" option.
The empty values are false, 0, any nil pointer or interface value, and any array, slice, map, or string of length zero. The object's default key string is the struct field name but can be specified in the struct field's tag value. The "json" key in the struct field's tag value is the key name, followed by an optional comma and options.