I'm trying to configure my Go program by creating a JSON file and parsing it into a struct:
var settings struct {
serverMode bool
sourceDir string
targetDir string
}
func main() {
// then config file settings
configFile, err := os.Open("config.json")
if err != nil {
printError("opening config file", err.Error())
}
jsonParser := json.NewDecoder(configFile)
if err = jsonParser.Decode(&settings); err != nil {
printError("parsing config file", err.Error())
}
fmt.Printf("%v %s %s", settings.serverMode, settings.sourceDir, settings.targetDir)
return
}
The config.json file:
{
"serverMode": true,
"sourceDir": ".",
"targetDir": "."
}
The Program compiles and runs without any errors, but the print statement outputs:
false
(false and two empty strings)
I've also tried with json.Unmarshal(..) but had the same result.
How do I parse the JSON in a way that fills the struct values?
You're not exporting your struct elements. They all begin with a lower case letter.
var settings struct {
ServerMode bool `json:"serverMode"`
SourceDir string `json:"sourceDir"`
TargetDir string `json:"targetDir"`
}
Make the first letter of your stuct elements upper case to export them. The JSON encoder/decoder wont use struct elements which are not exported.
Related
I'm trying to parse JSON files by using dynamically created structs, but apparently I'm doing something wrong. Can somebody please tell we what am I doing wrong here:
structured := make(map[string][]reflect.StructField)
structured["Amqp1"] = []reflect.StructField{
reflect.StructField{
Name: "Test",
Type: reflect.TypeOf(""),
Tag: reflect.StructTag(`json:"test"`),
},
reflect.StructField{
Name: "Float",
Type: reflect.TypeOf(5.5),
Tag: reflect.StructTag(`json:"float"`),
},
reflect.StructField{
Name: "Connections",
Type: reflect.TypeOf([]Connection{}),
Tag: reflect.StructTag(`json:"connections"`),
},
}
sections := []reflect.StructField{}
for sect, params := range structured {
sections = append(sections,
reflect.StructField{
Name: sect,
Type: reflect.StructOf(params),
},
)
}
parsed := reflect.New(reflect.StructOf(sections)).Elem()
if err := json.Unmarshal([]byte(JSONConfigContent), &parsed); err != nil {
fmt.Printf("unable to parse data from provided configuration file: %s\n", err)
os.Exit(1)
}
https://play.golang.org/p/C2I4Pduduyg
Thanks in advance.
You want to use .Interface() to return the actual, underlying value, which should be a pointer to the concrete anonymous struct.
Note that the reflect.New function returns a reflect.Value representing a pointer to a new zero value for the specified type. The Interface method, in this case, returns that pointer as interface{} which is all you need for json.Unmarshal.
If, after unmarshaling, you need a non-pointer of the struct you can turn to reflect again and use reflect.ValueOf(parsed).Elem().Interface() to effectively dereference the pointer.
parsed := reflect.New(reflect.StructOf(sections)).Interface()
if err := json.Unmarshal([]byte(JSONConfigContent), parsed); err != nil {
fmt.Printf("unable to parse data from provided configuration file: %s\n", err)
os.Exit(1)
}
https://play.golang.org/p/Bzu1hUyKjvM
reflect.New returns a value representing a pointer.
Change line 69:
fmt.Printf(">>> %v", &parsed)
Result:
>>> <struct { Amqp1 struct { Test string "json:\"test\""; Float float64 "json:\"float\""; Connections []main.Connection "json:\"connections\"" } } Value>
I would recommend something very different. I generally avoid reflection where possible. You can accomplish what you are trying to do by simply providing structs for each type of config you expect and then do an initial "pre-unmarshalling" to determine what type of config you should actually use by name (which, in this case is a key of your JSON object):
package main
import (
"encoding/json"
"fmt"
"os"
)
//Amqp1 config struct
type Amqp1 struct {
Config struct {
Test string `json:"test"`
Float float64 `json:"float"`
Connections []Connection `json:"connections"`
} `json:"Amqp1"`
}
//Connection struct
type Connection struct {
Type string `json:"type"`
URL string `json:"url"`
}
//JSONConfigContent json
const JSONConfigContent = `{
"Amqp1": {
"test": "woobalooba",
"float": 5.5,
"connections": [
{"type": "test1", "url": "booyaka"},
{"type": "test2", "url": "foobar"}
]
}
}`
func main() {
configMap := make(map[string]interface{})
if err := json.Unmarshal([]byte(JSONConfigContent), &configMap); err != nil {
fmt.Printf("unable to parse data from provided configuration file: %s\n", err)
os.Exit(1)
}
//get config name
var configName string
for cfg := range configMap {
configName = cfg
break
}
//unmarshal appropriately
switch configName {
case "Amqp1":
var amqp1 Amqp1
if err := json.Unmarshal([]byte(JSONConfigContent), &amqp1); err != nil {
fmt.Printf("unable to parse data from provided configuration file: %s\n", err)
os.Exit(1)
}
fmt.Printf("%s >>\n", configName)
fmt.Printf("Test: %s\n", amqp1.Config.Test)
fmt.Printf("Float: %v\n", amqp1.Config.Float)
fmt.Printf("Connections: %#v\n", amqp1.Config.Connections)
default:
fmt.Printf("unknown config encountered: %s\n", configName)
os.Exit(1)
}
}
The initial "pre-unmarshalling" happens on the 2nd line of main(), to a plain map where the key is a string and the value is interface{} because you don't care. You're just doing that to get the actual type of config that it is, which is the key of the first nested object.
You can run this on playground as well.
I'm using the Decoder from package encoding/json to decode a JSON configuration file into a struct. The field names have a different case (lowercase first character in struct because of visibility concerns) in the file and struct so I'm using struct field tags as described in the documentation. The Problem is that Decoder seems to ignore these tags and the struct fields are empty. Any ideas what's wrong with my code?
config.json
{
"DataSourceName": "simple-blog.db"
}
Config struct
type Config struct {
dataSourceName string `json:"DataSourceName"`
}
Loading config
func loadConfig(fileName string) {
file, err := os.Open(fileName)
if err != nil {
log.Fatalf("Opening config file failed: %s", err)
}
defer file.Close()
decoder := json.NewDecoder(file)
config = &Config{} // Variable config is defined outside
err = decoder.Decode(config)
if err != nil {
log.Fatalf("Decoding config file failed: %s", err)
}
log.Print("Configuration successfully loaded")
}
Usage
loadConfig("config.json")
log.Printf("DataSourceName: %s", config.dataSourceName)
Output
2017/10/15 21:04:11 DB Name:
You need to export your dataSourceName field as encoding/json package requires them to be so
My original data format:
id=1, name=peter, age=12
I converted it to JSON string:
{"id" : "1", "name" : "peter", "age" : "12"}
I use the following golang statement to do the conversion:
Regex, err = regexp.Compile(`([^,\s]*)=([^,\s]*)`)
JSON := fmt.Sprintf("{%s}", Regex.ReplaceAllString(inp, `"$1" : "$2"`))
inp is the variable that holds the original data format.
However, now I get a new format:
id=1 name=peter age=12
and I also want to convert to JSON string using similar method that I used above, i.e., use regex to do a one pass formatting.
{"id"="1", "name"="peter", "age"="12"}
How can I achieve that?
UPDATE: One additional requirement. if the input format is
id=1, name=peter, age="12"
I need to get rid of the "" to be or escape \" so I can process in the next step. The double quote can appear at the beginning and the end of any value field.
There are two parts to the question: The easy part is serialising to JSON, Go has standard library methods for doing that. I would use that library rather than trying to encode the JSON myself.
The slightly trickier part of your question is parsing the input into a struct or map that can be easily serialised out, and making it flexible enough to accept different input formats.
I would do it with a general interface for converting text to a struct or map, and then implementing the interface for parsing each new input type.
Sample code: (You can run it here)
package main
import (
"encoding/json"
"errors"
"fmt"
"strings"
)
// parseFn describes the function for converting input into a map.
// This could be a struct or something else if the format is well known.
// In real code this would return map[string]interface{}, but for this
// demo I'm just using string
type parseFn func(string) (map[string]string, error)
// parseFormat1 is for fields separated by commas
func parseFormat1(in string) (map[string]string, error) {
data := map[string]string{}
fields := strings.Split(in, ",")
for _, field := range fields {
pair := strings.Split(field, "=")
if len(pair) != 2 {
return nil, errors.New("invalid input")
}
data[strings.Trim(pair[0], ` "`)] = strings.Trim(pair[1], ` "`)
}
return data, nil
}
// parseFormat2 is for lines with no commas
func parseFormat2(in string) (map[string]string, error) {
data := map[string]string{}
fields := strings.Split(in, " ")
for _, field := range fields {
pair := strings.Split(field, "=")
if len(pair) != 2 {
return nil, errors.New("invalid input")
}
data[strings.Trim(pair[0], ` "`)] = strings.Trim(pair[1], ` "`)
}
return data, nil
}
// nullFormat is what we fall back on when we just don't know
func nullFormat(in string) (map[string]string, error) { return nil, errors.New("invalid format") }
// classify just tries to guess the parser to use for the input
func classify(in string) parseFn {
switch {
case strings.Count(in, ", ") > 1:
return parseFormat1
case strings.Count(in, " ") > 1:
return parseFormat2
default:
return nullFormat
}
}
func main() {
testCases := []string{
`id=1, name=peter, age=12`,
`id=1, name=peter, age="12"`,
`id=1 name=peter age=12`,
`id=1;name=peter;age="12"`,
}
for ix, tc := range testCases {
pfn := classify(tc)
d, err := pfn(tc)
if err != nil {
fmt.Printf("\nerror parsing on line %d: %v\n", ix, err)
continue
}
b, err := json.Marshal(d)
if err != nil {
fmt.Printf("\nerror marshaling on line %d: %v\n", ix, err)
continue
}
fmt.Printf("\nSuccess on line %d:\n INPUT: %s\nOUTPUT: %s\n", ix, tc, string(b))
}
}
I an new to programming in Go so apologies if this is something obvious. I have a JSON file named foo.json:
{"type":"fruit","name":"apple","color":"red"}
and I am writing a Go program that has to do something when the "name" value in the JSON file is "apple". It needs no other information from that JSON file as that file is used for a completely different purpose in another area of the code.
I have read documentation on Decode() and Unmarshal() and abut 30 different web pages describing how to read the whole file into structures, etc. but it all seems extremely complicated for what I want to do which is just write the correct code to implement the first 2 lines of this pseudo-code:
file, _ := os.Open("foo.json")
name = file["name"]
if (name == "apple") {
do stuff
}
such that I end up with a Go variable named name that contains the string value apple. What is the right way to do this in Go?
The easiest method to do what you want is to decode into a struct.
Provided the format remains similar to {"type":"fruit","name":"apple","color":"red"}
type Name struct {
Name string `json:"name"`
}
var data []byte
data, _ = ioutil.ReadFile("foo.json")
var str Name
_ = json.Unmarshal(data, &str)
if str.Name == "apple" {
// Do Stuff
}
Your other option is to use third party libraries such as gabs or jason.
Gabs :
jsonParsed, err := gabs.ParseJSON(data)
name, ok := jsonParsed.Path("name").Data().(string)
Jason :
v, _ := jason.NewObjectFromBytes(data)
name, _ := v.GetString("name")
Update :
The structure
type Name struct {
Name string `json:"name"`
}
is the json equivalent of {"name":"foo"}.
So unmarshaling won't work for the following json with different formats.
[{"name":"foo"}]
{"bar":{"name":"foo"}}
PS : As mentioned by W.K.S. In your case an anonymous struct would suffice since you're not using this structure for anything else.
One thing is reading a file and other one is decoding a JSON document. I leave you a full example doing both. To run it you have to have a file called file.json in the same directory of your code or binary executable:
package main
import (
"encoding/json"
"io/ioutil"
"log"
"os"
)
func main() {
f, err := os.Open("file.json") // file.json has the json content
if err != nil {
log.Fatal(err)
}
bb, err := ioutil.ReadAll(f)
if err != nil {
log.Fatal(err)
}
doc := make(map[string]interface{})
if err := json.Unmarshal(bb, &doc); err != nil {
log.Fatal(err)
}
if name, contains := doc["name"]; contains {
log.Printf("Happy end we have a name: %q\n", name)
} else {
log.Println("Json document doesn't have a name field.")
}
log.Printf("full json: %s", string(bb))
}
https://play.golang.org/p/0u04MwwGfn
I have also tried to find a simple solution such as $d = json_decode($json, true) in PHP and came to the conclusion that there is no such simple way in Golang. The following is the simplest solution I could make (the checks are skipped for clarity):
var f interface{}
err = json.Unmarshal(file, &f)
m := f.(map[string]interface{})
if (m["name"] == "apple") {
// Do something
}
where
file is an array of bytes of JSON string,
f interface serves as a generic container for unknown JSON structure,
m is a map returned by the type assertion.
We can assert that f is a map of strings, because Unmarshal() builds a variable of that type for any JSON input. At least, I couldn't make it return something different. It is possible to detect the type of a variable by means of run-time reflection:
fmt.Printf("Type of f = %s\n", reflect.TypeOf(f))
For the f variable above, the code will print Type of f = map[string]interface {}.
Example
And this is the full code with necessary checks:
package main
import (
"fmt"
"os"
"io/ioutil"
"encoding/json"
)
func main() {
// Read entire file into an array of bytes
file, err := ioutil.ReadFile("foo.json")
if (err != nil) {
fmt.Fprintf(os.Stderr, "Failed read file: %s\n", err)
os.Exit(1)
}
var f interface{}
err = json.Unmarshal(file, &f)
if (err != nil) {
fmt.Fprintf(os.Stderr, "Failed to parse JSON: %s\n", err)
os.Exit(1)
}
// Type-cast `f` to a map by means of type assertion.
m := f.(map[string]interface{})
fmt.Printf("Parsed data: %v\n", m)
// Now we can check if the parsed data contains 'name' key
if (m["name"] == "apple") {
fmt.Print("Apple found\n")
}
}
Output
Parsed data: map[type:fruit name:apple color:red]
Apple found
The proper Go way of doing this would be to decode into an instance of an anonymous struct containing only the field you need.
func main() {
myStruct := struct{ Name string }{}
json.Unmarshal([]byte(`{"type":"fruit","name":"apple","color":"red"}`), &myStruct)
fmt.Print(myStruct.Name)
}
Playground Link
Alternatively, You could use Jeffails/gabs JSON Parser:
jsonParsed,_ := gabs.ParseJSON([]byte(`{"type":"fruit","name":"apple","color":"red"}`));
value, ok = jsonParsed.Path("name").Data().(string)
I'm playing with Go and am stumped as to why json encode and decode don't work for me
I think i copied the examples almost verbatim, but the output says both marshal and unmarshal return no data. They also don't give an error.
can anyone hint to where i'm going wrong?
my sample code: Go playground
package main
import "fmt"
import "encoding/json"
type testStruct struct {
clip string `json:"clip"`
}
func main() {
//unmarshal test
var testJson = "{\"clip\":\"test\"}"
var t testStruct
var jsonData = []byte(testJson)
err := json.Unmarshal(jsonData, &t)
if err != nil {
fmt.Printf("There was an error decoding the json. err = %s", err)
return
}
fmt.Printf("contents of decoded json is: %#v\r\n", t)
//marshal test
t.clip = "test2"
data, err := json.Marshal(&t)
if err != nil {
fmt.Printf("There was an error encoding the json. err = %s", err)
return
}
fmt.Printf("encoded json = %s\r\n", string(data))
}
output:
contents of decoded json is: main.testStruct{clip:""}
encoded json = {}
in both outputs I would have expected to see the decoded or encoded json
For example,
package main
import "fmt"
import "encoding/json"
type testStruct struct {
Clip string `json:"clip"`
}
func main() {
//unmarshal test
var testJson = "{\"clip\":\"test\"}"
var t testStruct
var jsonData = []byte(testJson)
err := json.Unmarshal(jsonData, &t)
if err != nil {
fmt.Printf("There was an error decoding the json. err = %s", err)
return
}
fmt.Printf("contents of decoded json is: %#v\r\n", t)
//marshal test
t.Clip = "test2"
data, err := json.Marshal(&t)
if err != nil {
fmt.Printf("There was an error encoding the json. err = %s", err)
return
}
fmt.Printf("encoded json = %s\r\n", string(data))
}
Output:
contents of decoded json is: main.testStruct{Clip:"test"}
encoded json = {"clip":"test2"}
Playground:
http://play.golang.org/p/3XaVougMTE
Export the struct fields.
type testStruct struct {
Clip string `json:"clip"`
}
Exported identifiers
An identifier may be exported to permit access to it from another
package. An identifier is exported if both:
the first character of the identifier's name is a Unicode upper case letter (Unicode class "Lu"); and
the identifier is declared in the package block or it is a field name or method name.
All other identifiers are not exported.
Capitalize names of structure fields
type testStruct struct {
clip string `json:"clip"` // Wrong. Lowercase - other packages can't access it
}
Change to:
type testStruct struct {
Clip string `json:"clip"`
}
In my case, my struct fields were capitalized but I was still getting the same error.
Then I noticed that the casing of my fields was different. I had to use underscores in my request.
For eg:
My request body was:
{
"method": "register",
"userInfo": {
"fullname": "Karan",
"email": "email#email.com",
"password": "random"
}
}
But my golang struct was:
type AuthRequest struct {
Method string `json:"method,omitempty"`
UserInfo UserInfo `json:"user_info,omitempty"`
}
I solved this by modifying my request body to:
{
"method": "register",
"user_info": {
"fullname": "Karan",
"email": "email#email.com",
"password": "random"
}
}