I am trying to port some code written against go1.3 to current versions and ran into a case where the json parsing behavior is different between versions. We are using a custom unmarshaller for parsing some specific date format. It looks like recent versions pass in the string with additional quotes which 1.3 did not.
Is this a bug or an intentional change? And whats the best way of writing code which is compatible with different versions in this situation. Just go looking for all places where a custom unmarshaller is in use always strip out extra quotes if any? It would be a pity to have to do that - so I am hoping there is a better way.
package main
import "encoding/json"
import "fmt"
import "time"
type Timestamp1 time.Time
func (t *Timestamp1) UnmarshalJSON(b []byte) (err error) {
fmt.Println("String to parse as timestamp:", string(b))
parsedTime, err := time.Parse("2006-01-02T15:04:05", string(b))
if err == nil {
*t = Timestamp1(parsedTime)
return nil
} else {
return err
}
}
type S struct {
LastUpdatedDate Timestamp1 `json:"last_updated_date,string"`
}
func main() {
s := `{"last_updated_date" : "2015-11-03T10:00:00"}`
var s1 S
err := json.Unmarshal([]byte(s), &s1)
fmt.Println(err)
fmt.Println(s1)
}
There was a bug concerning json:",string" tag that was fixed in 1.5. If there isn't a particular reason you need it, you can remove it and simply adjust your format:
// N.B. time is in quotes.
parsedTime, err := time.Parse(`"2006-01-02T15:04:05"`, string(b))
Playground: http://play.golang.org/p/LgWuKcPEuI.
This should work in 1.3 as well as 1.5.
Related
Following this tutorial I'm trying to read a json file in Golang. It says there are two ways of doing that:
unmarshal the JSON using a set of predefined structs
or unmarshal the JSON using a map[string]interface{}
Since I'll probably have a lot of different json formats I prefer to interpret it on the fly. So I now have the following code:
package main
import (
"fmt"
"os"
"io/ioutil"
"encoding/json"
)
func main() {
// Open our jsonFile
jsonFile, err := os.Open("users.json")
// if we os.Open returns an error then handle it
if err != nil {
fmt.Println(err)
}
fmt.Println("Successfully Opened users.json")
// defer the closing of our jsonFile so that we can parse it later on
defer jsonFile.Close()
byteValue, _ := ioutil.ReadAll(jsonFile)
var result map[string]interface{}
json.Unmarshal([]byte(byteValue), &result)
fmt.Println(result["users"])
fmt.Printf("%T\n", result["users"])
}
This prints out:
Successfully Opened users.json
[map[type:Reader age:23 social:map[facebook:https://facebook.com twitter:https://twitter.com] name:Elliot] map[name:Fraser type:Author age:17 social:map[facebook:https://facebook.com twitter:https://twitter.com]]]
[]interface {}
At this point I don't understand how I can read the age of the first user (23). I tried some variations:
fmt.Println(result["users"][0])
fmt.Println(result["users"][0].age)
But apparently, type interface {} does not support indexing.
Is there a way that I can access the items in the json without defining the structure?
Probably you want
fmt.Println(result["users"].(map[string]interface{})["age"])
or
fmt.Println(result[0].(map[string]interface{})["age"])
As the JSON is a map of maps the type of the leaf nodes is interface{} and so has to be converted to map[string]interface{} in order to lookup a key
Defining a struct is much easier. My top tip for doing this is to use a website that converts JSON to a Go struct definition, like Json-To-Go
This question already has an answer here:
JSON Unmarshal struct case-sensitively
(1 answer)
Closed 3 years ago.
Golang does not support case sensitive unmarshalling of JSON using the standard packages. This seems like a very common need though.
Is there any way to get a precise matching of cases when marshalling and unmarshalling JSON?
Example: I don't want to match "id" with "ID".
The standard library encoding/json does in fact support case sensitive encoding/decoding if your data structures define the members that way and include tags appropriately.
For example:
package main
import (
"encoding/json"
"fmt"
)
type image struct {
Url string `json:"url"`
}
type images struct {
Image1 image `json:"image"`
Image2 image `json:"Image"`
}
func main() {
i := images{Image1: image{Url: "test.jpg"}, Image2: image{Url: "test2.jpg"}}
data, err := json.Marshal(i)
if err != nil {
fmt.Printf("error: %s", err)
}
fmt.Println(string(data))
var i2 images
err = json.Unmarshal(data, &i2)
if err != nil {
fmt.Printf("error: %s", err)
}
fmt.Printf("%#v\n", i2)
}
https://play.golang.org/p/GWUWYUc-T9t
Which will output:
{"image":{"url":"test.jpg"},"Image":{"url":"test2.jpg"}}
main.images{Image1:main.image{Url:"test.jpg"}, Image2:main.image{Url:"test2.jpg"}}
However, another good json encoding package is jsonparser: https://github.com/buger/jsonparser
I don't know if it supports case sensativity differentaly then the standard library package.
I am new to golang and json and currently struggle to parse the json out from a system.
I've read a couple of blog posts on dynamic json in go and also tried the tools like json2GoStructs
Parsing my json file with this tools just gave me a huge structs which I found a bit messy. Also I had no idea how to get the info im interested in.
So, here are my problems:
How do I get to the info I am interested in?
What is the best approach to parse complex json?
I am only interested into the following 3 json fields:
Name
Guid
Task -> Property -> Name: Error
I'm thankful for every tip, code snippet or explanation!
This is what I got so far (mostly from a tutorial):
package main
import (
"encoding/json"
"fmt"
"io/ioutil"
)
func checkErr(err error) {
if err != nil {
panic(err)
}
}
func readFile(filePath string) []byte {
data, err := ioutil.ReadFile(filePath)
checkErr(err)
return data
}
func main() {
path := "/Users/andi/Documents/tmp/wfsJob.json"
data := readFile(path)
var f interface{}
err := json.Unmarshal(data, &f)
checkErr(err)
m := f.(map[string]interface{})
for k, v := range m {
switch vv := v.(type) {
case string:
fmt.Println(k, "is string", vv)
case int:
fmt.Println(k, "is int", vv)
case []interface{}:
fmt.Println(k, "is an array:")
for i, u := range vv {
fmt.Println(i, u)
}
default:
fmt.Println(k, "is of a type I don't know how to handle")
}
}
}
I can offer you this easy way to using JSON in Golang. With this tool you don't need to parse the whole json file, and you can use it without struct.
Gjson is a great solution for fetching a few fields from JSON string. But it may become slow when many (more than 2) fields must be fetched from distinct parts of the JSON, since it re-parses the JSON on each Get call. Additionally, it requires calling gjson.Valid for validating the incoming JSON, since other methods assume the caller provides valid JSON.
There is an alternative package - fastjson. Like gsjon, it is fast and has nice API. Unlike gjson it validates the input JSON and works faster when many unrelated fields must be obtained from the JSON. Here is a sample code for obtaining fields from the original question:
var p fastjson.Parser
v, err := p.ParseBytes(data)
if err != nil {
log.Fatal(err)
}
fmt.Printf("Name: %s", v.GetStringBytes("Name"))
fmt.Printf("Guid: %s", v.GetStringBytes("Guid"))
fmt.Printf("Error: %s", v.GetStringBytes("Task", "Property", "Name"))
Background
I am learning Go and I'm trying to do some JSON unmarshaling of a datetime.
I have some JSON produced by a program I wrote in C, I am outputting what I thought was a valid ISO8601 / RFC3339 timezone offset. I'm using strftime with the following format string:
%Y-%m-%dT%H:%M:%S.%f%z
(Note that %f is not supported by strftime natively, I have a wrapper that replaces it with the nanoseconds).
This will then produce the following result:
2016-08-08T21:35:14.052975+0200
Unmarshaling this in Go however will not work:
https://play.golang.org/p/vzOXbzAwdW
package main
import (
"fmt"
"time"
)
func main() {
t, err := time.Parse(time.RFC3339Nano, "2016-08-08T21:35:14.052975+0200")
if err != nil {
panic(err)
}
fmt.Println(t)
}
Output:
panic: parsing time "2016-08-08T21:35:14.052975+0200" as "2006-01-02T15:04:05.999999999Z07:00": cannot parse "+0200" as "Z07:00"
(Working example: https://play.golang.org/p/5xcM0aHsSw)
This is because RFC3339 expects the timezone offset to be in the format 02:00 with a :, but strftime outputs it as 0200.
So I need to fix this in my C program to output the correct format.
%z The +hhmm or -hhmm numeric timezone (that is, the hour and
minute offset from UTC). (SU)
Question
However, now I have a bunch of JSON files with this incorrect format:
2016-08-08T21:35:14.052975+0200
instead of the correct (with the : in the timezone offset):
2016-08-08T21:35:14.052975+02:00
but I still want to be able to unmarshal it correctly in my Go program. Preferably two different JSON files with only this difference should parse in the exact same way.
Regarding marshaling back to JSON, the correct format should be used.
This is how I have defined it in my struct:
Time time.Time `json:"time"`
So the question is, what is the "Go" way of doing this?
Also in my code example I am using RFC3339Nano. How would I specify that in the metadata for the struct as well? As I have it now with just json:"time" will that ignore the nano seconds?
You can define your own time field type that supports both formats:
type MyTime struct {
time.Time
}
func (self *MyTime) UnmarshalJSON(b []byte) (err error) {
s := string(b)
// Get rid of the quotes "" around the value.
// A second option would be to include them
// in the date format string instead, like so below:
// time.Parse(`"`+time.RFC3339Nano+`"`, s)
s = s[1:len(s)-1]
t, err := time.Parse(time.RFC3339Nano, s)
if err != nil {
t, err = time.Parse("2006-01-02T15:04:05.999999999Z0700", s)
}
self.Time = t
return
}
type Test struct {
Time MyTime `json:"time"`
}
Try on Go Playground
In the example above we take the predefined format time.RFC3339Nano, which is defined like this:
RFC3339Nano = "2006-01-02T15:04:05.999999999Z07:00"
and remove the :
"2006-01-02T15:04:05.999999999Z0700"
This time format used by time.Parse is described here:
https://golang.org/pkg/time/#pkg-constants
Also see the documentation for time.Parse
https://golang.org/pkg/time/#Parse
P.S. The fact that the year 2006 is used in the time format strings is probably because the first version of Golang was released that year.
You can try https://play.golang.org/p/IsUpuTKENg
package main
import (
"fmt"
"time"
)
func main() {
t, err := time.Parse("2006-01-02T15:04:05.999999999Z0700", "2016-08-08T21:35:14.052975-0200")
if err != nil {
panic(err)
}
fmt.Println(t)
}
I'm trying to work out a bit of code to pull in config from a JSON file.
When I attempt to build, I get this error
type ConfigVars is not an expression
Below is the config and program code I'm trying to work with. Every example I've found so far is similar to the below code. Any suggestion of what I'm doing incorrectly?
-- Config File
{"beaconUrl":"http://test.com/?id=1"}
-- Program Code
package main
import (
"encoding/json"
"fmt"
"os"
)
type ConfigVars struct {
BeaconUrl string
}
func main() {
configFile, err := os.Open("config.json")
defer configFile.Close()
if err != nil {
fmt.Println("Opening config file", err.Error())
}
jsonParser := json.NewDecoder(configFile)
if err = jsonParser.Decode(&ConfigVars); err != nil {
fmt.Println("Parsing config file", err.Error())
}
}
What you're doing there is trying to pass a pointer to the ConfigVars type (which obviously doesn't really mean anything). What you want to do is make a variable whose type is ConfigVars and pass a pointer to that instead:
var cfg ConfigVars
err = jsonParser.Decode(&cfg)
...
For others who come onto this problem, you may find that you've forgotten to initialize the variable during assignment using the := operator, as described in Point 3 at the end of this GoLang tutorial.
var cfg ConfigVars
err := jsonParser.Decode(&cfg)