JSON omitempty With time.Time Field - json

Trying to json Marshal a struct that contains 2 time fields. But I only want the field to come through if it has a time value. So I'm using json:",omitempty" but it's not working.
What can I set the Date value to so json.Marshal will treat it like an empty (zero) value and not include it in the json string?
Playground: http://play.golang.org/p/QJwh7yBJlo
Actual Outcome:
{"Timestamp":"2015-09-18T00:00:00Z","Date":"0001-01-01T00:00:00Z"}
Desired Outcome:
{"Timestamp":"2015-09-18T00:00:00Z"}
Code:
package main
import (
"encoding/json"
"fmt"
"time"
)
type MyStruct struct {
Timestamp time.Time `json:",omitempty"`
Date time.Time `json:",omitempty"`
Field string `json:",omitempty"`
}
func main() {
ms := MyStruct{
Timestamp: time.Date(2015, 9, 18, 0, 0, 0, 0, time.UTC),
Field: "",
}
bb, err := json.Marshal(ms)
if err != nil {
panic(err)
}
fmt.Println(string(bb))
}

The omitempty tag option does not work with time.Time as it is a struct. There is a "zero" value for structs, but that is a struct value where all fields have their zero values. This is a "valid" value, so it is not treated as "empty".
But by simply changing it to a pointer: *time.Time, it will work (nil pointers are treated as "empty" for json marshaling/unmarshaling). So no need to write custom Marshaler in this case:
type MyStruct struct {
Timestamp *time.Time `json:",omitempty"`
Date *time.Time `json:",omitempty"`
Field string `json:",omitempty"`
}
Using it:
ts := time.Date(2015, 9, 18, 0, 0, 0, 0, time.UTC)
ms := MyStruct{
Timestamp: &ts,
Field: "",
}
Output (as desired):
{"Timestamp":"2015-09-18T00:00:00Z"}
Try it on the Go Playground.
If you can't or don't want to change it to a pointer, you can still achieve what you want by implementing a custom Marshaler and Unmarshaler. If you do so, you can use the Time.IsZero() method to decide if a time.Time value is the zero value.

You may define you self Time type for custom marshal format, and use it everywhere instead time.Time
https://play.golang.org/p/C8nIR1uZAok
package main
import (
"bytes"
"encoding/json"
"fmt"
"time"
)
type MyTime struct {
*time.Time
}
func (t MyTime) MarshalJSON() ([]byte, error) {
return []byte(t.Format("\"" + time.RFC3339 + "\"")), nil
}
// UnmarshalJSON implements the json.Unmarshaler interface.
// The time is expected to be a quoted string in RFC 3339 format.
func (t *MyTime) UnmarshalJSON(data []byte) (err error) {
// by convention, unmarshalers implement UnmarshalJSON([]byte("null")) as a no-op.
if bytes.Equal(data, []byte("null")) {
return nil
}
// Fractional seconds are handled implicitly by Parse.
tt, err := time.Parse("\""+time.RFC3339+"\"", string(data))
*t = MyTime{&tt}
return
}
func main() {
t := time.Now()
d, err := json.Marshal(MyTime{&t})
fmt.Println(string(d), err)
var mt MyTime
json.Unmarshal(d, &mt)
fmt.Println(mt)
}

As a follow up to icza's answer here is a custom marshaller that omits an empty date field but keeps the rest of the fields unchanged.
func (ms *MyStruct) MarshalJSON() ([]byte, error) {
type Alias MyStruct
if ms.Timestamp.IsZero() {
return json.Marshal(&struct {
Timestamp int64 `json:",omitempty"`
*Alias
}{
Timestamp: 0,
Alias: (*Alias)(ms),
})
} else {
return json.Marshal(&struct {
*Alias
}{
Alias: (*Alias)(ms),
})
}
}
This borrows from http://choly.ca/post/go-json-marshalling/
The OPs case has two time fields which would make it much more complicated. (you'd have to check for neither, either and both being empty!)
There may be better ways to achieve this, so comments are welcome.

Related

hh:mm:ss to time.Time while parsing from JSON in Go

I have a struct that I can't change and an array of these structs in a separate JSON file.
I could have parsed data from JSON file easily, but there are mismatched types in same fields:
(main.go)
import "time"
type SomeType struct {
name string `json: "name"`
time time.Time `json: "someTime"`
}
(someData.json)
[
{
"name": "some name",
"someTime": "15:20:00"
},
{
"name": "some other name",
"someTime": "23:15:00"
}
]
If "time" field was a type of a string, I would simply use json.Unmarshal and parse all of the data from json into []SomeType, but since types mismatch, I can't find a way to do it correctly.
You should add the UnmarshalJSON method to your struct
type SomeType struct {
Name string `json:"name"`
Time time.Time `json:"someTime"`
}
func (st *SomeType) UnmarshalJSON(data []byte) error {
type parseType struct {
Name string `json:"name"`
Time string `json:"someTime"`
}
var res parseType
if err := json.Unmarshal(data, &res); err != nil {
return err
}
parsed, err := time.Parse("15:04:05", res.Time)
if err != nil {
return err
}
now := time.Now()
st.Name = res.Name
st.Time = time.Date(now.Year(), now.Month(), now.Day(), parsed.Hour(), parsed.Minute(), parsed.Second(), 0, now.Location())
return nil
}
GoPlay
I think you should define a custom time type, with a custom unmarshaller as follows:
type CustomTime struct {
time.Time
}
func (t *CustomTime) UnmarshalJSON(b []byte) error {
formattedTime, err := time.Parse(`"15:04:05"`, string(b))
t.Time = formattedTime
return err
}
And have another struct, which is basically the exact struct you have, instead it uses your custom time type rather than the original struct which uses time.Time:
type SameTypeWithDifferentTime struct {
Name string `json:"name"`
SomeTime CustomTime `json:"someTime"`
}
func (s SameTypeWithDifferentTime) ToOriginal() SomeType {
return SomeType {
Name: s.Name,
SomeTime: s.SomeTime.Time,
}
}
The rest of the process is pretty straight forward, you just deserialize your json to the new type, and use ToOriginal() receiver function to convert it to your original struct. Another point to mention here is that your not exporting your struct fields.

Unmarshalling nested JSON objects with dates in Golang

I'm a noob with Golang. I managed to get some things done with lots of effort.
I'm dealing with JSON files containing dates in a nested way.
I came across some workaround to unmarshal dates from JSON data into time.Time but I'm having a hard time dealing with nested ones.
The following code (obtained here in StackOverflow) is easy to understand since creates a user-defined function to parse the time objects first to a string and then to time.Time with time.Parse.
package main
import (
"encoding/json"
"fmt"
"log"
"time"
)
const dateFormat = "2006-01-02"
const data = `{
"name": "Gopher",
"join_date": "2007-09-20"
}`
type User struct {
Name string `json:"name"`
JoinDate time.Time `json:"join_date"`
}
func (u *User) UnmarshalJSON(p []byte) error {
var aux struct {
Name string `json:"name"`
JoinDate string `json:"join_date"`
}
err := json.Unmarshal(p, &aux)
if err != nil {
return err
}
t, err := time.Parse(dateFormat, aux.JoinDate)
if err != nil {
return err
}
u.Name = aux.Name
u.JoinDate = t
return nil
}
func main() {
var u User
err := json.Unmarshal([]byte(data), &u)
if err != nil {
log.Fatal(err)
}
fmt.Println(u.JoinDate.Format(time.RFC3339))
}
So far, so good.
Now I would like to extend it in order to handle the nested date fields in the JSON, like the example below:
[{
"name": "Gopher",
"join_date": "2007-09-20",
"cashflow": [
{"date": "2021-02-25",
"amount": 100},
{"date": "2021-03-25",
"amount": 105}
]
}]
The struct that I would like to get is:
type Record []struct {
Name string `json:"name"`
JoinDate time.Time `json:"join_date"`
Cashflow []struct {
Date time.Time `json:"date"`
Amount int `json:"amount"`
} `json:"cashflow"`
}
Thanks for the help.
To solve this using the patterns you've already got, you can write a separate unmarshalling function for the inner struct. You can do that by hoisting the inner struct to its own named struct, and then writing the function.
type CashflowRec struct {
Date time.Time `json:"date"`
Amount int `json:"amount"`
}
type Record struct {
Name string `json:"name"`
JoinDate time.Time `json:"join_date"`
Cashflow []CashflowRec `json:"cashflow"`
}
You've already shown how to write the unmarshalling function for CashflowRec, it looks almost the same as your User function. The unmarshalling function for Record will make use of that when it calls
func (u *Record) UnmarshalJSON(p []byte) error {
var aux struct {
Name string `json:"name"`
JoinDate string `json:"join_date"`
Cashflow []CashflowRec `json:"cashflow"`
}
err := json.Unmarshal(p, &aux)
Working example: https://go.dev/play/p/1X7BJ4NETM0
aside 1 Something amusing I learned while looking at this: because you've provided your own unmarshalling function, you don't actually need the json tags in your original structs. Those are hints for the unmarshaller that the json package provides. You should probably still leave them in, in case you have to marshal the struct later. Here's it working without those tags: https://go.dev/play/p/G2VWopO_A3t
aside 2 You might find it simpler not to use time.Time, but instead create a new type of your own, and then give that type its own unmarshaller. This gives you the interesting choice for writing only that one unmarshaller, but whether or not this is a win depends on what else you do with the struct later on. Working example that still uses your nested anonymous structs: https://go.dev/play/p/bJUcaw3_r41
type dateType time.Time
type Record struct {
Name string `json:"name"`
JoinDate dateType `json:"join_date"`
Cashflow []struct {
Date dateType `json:"date"`
Amount int `json:"amount"`
} `json:"cashflow"`
}
func (c *dateType) UnmarshalJSON(p []byte) error {
var s string
if err := json.Unmarshal(p, &s); err != nil {
return err
}
t, err := time.Parse(dateFormat, s)
if err != nil {
return err
}
*c = dateType(t)
return nil
}

Is there any posibilities to change the default Bind timestamp format in gin-gionic?

I have a question in Go especially with gin-gionic and gorm.
Let's say I have model like this
// Classroom struct.
type Classroom struct {
gorm.Model
Name string `json:"name"`
Code string `json:"code"`
StartedAt time.Time `json:"started_at"`
}
I want to create data of Classroom Model with this JSON
{
"name": "Math",
"code": "math-mr-robie",
"started_at": "2020-10-10 10:00:00"
}
But when I bind the JSON data, I got this following error
parsing time ""2020-10-10 10:00:00"" as ""2006-01-02T15:04:05Z07:00"": cannot parse " 10:00:00"" as "T"
I know that error appear because of the format that I sent was not the exact format of time.Time?
Is it possible to set default format of time.Time?
How to do that?
Because I've try to add .Format in after time.Time but error occurs.
// Classroom struct.
type Classroom struct {
gorm.Model
Name string `json:"name"`
Code string `json:"code"`
StartedAt time.Time.Format("2006-01-02 15:04:05") `json:"started_at"`
}
I resolve this issue by creating new struct JSONData that contain time inside it.
// JSONData struct.
type JSONData struct {
Time time.Time
}
After I red Customize Data Types in Gorm and see some examples here then I add some methods
// Scan JSONDate.
func (j *JSONDate) Scan(value interface{}) (err error) {
nullTime := &sql.NullTime{}
err = nullTime.Scan(value)
*j = JSONDate{nullTime.Time}
return
}
// Value JSONDate.
func (j JSONDate) Value() (driver.Value, error) {
y, m, d := time.Time(j.Time).Date()
return time.Date(y, m, d, 0, 0, 0, 0, time.Time(j.Time).Location()), nil
}
// GormDataType gorm common data type
func (j JSONDate) GormDataType() string {
return "timestamp"
}
For the gin things. Another resource #Eklavya given. So I add another methods.
// UnmarshalJSON JSONDate.
func (j *JSONDate) UnmarshalJSON(b []byte) error {
s := strings.Trim(string(b), "\"")
t, err := time.Parse(helpers.YMDHIS, s)
if err != nil {
return err
}
*j = JSONDate{
Time: t,
}
return nil
}
// MarshalJSON JSONDate.
func (j JSONDate) MarshalJSON() ([]byte, error) {
return []byte("\"" + j.Time.Format(helpers.YMDHIS) + "\""), nil
}
// Format method.
func (j JSONDate) Format(s string) string {
t := time.Time(j.Time)
return t.Format(helpers.YMDHIS)
}
And it's works!
I came across this question with the same issue and found that if you are looking for a specific time format like the ISOStrings sent from the browser you can have something like this;
type Classroom struct {
gorm.Model
Name string `json:"name"`
Code string `json:"code"`
StartedAt time.Time `json:"started_at" time_format:"RFC3339"`
}
With the time_format I did not need to define marshal functions to handle the formatting. However, if you need to do a completely custom format for your date and time then I believe you will need to define these functions.

Stop json.Marshal() from stripping trailing zero from floating point number

I got the following problem:
My golang program converts some information into JSON.
For example it results in the following json:
{
"value":40,
"unit":"some_string"
}
The problem is the "input" for value is 40.0 and the marshalling strips the trailing zero. It would be no problem if the EPL which reads the JSON would be able to read 40 as float without the .0
So the JSON output should look like:
{
"value":40.0,
"unit":"some_string"
}
Is there a possibility to "stop" json.Marshal() from removing the zero?
Edit: Value must be a Float
#icza provided a good answer, but just to offer another option, you can define your own float type and define your own serialization for it. Like this
type KeepZero float64
func (f KeepZero) MarshalJSON() ([]byte, error) {
if float64(f) == float64(int(f)) {
return []byte(strconv.FormatFloat(float64(f), 'f', 1, 32)), nil
}
return []byte(strconv.FormatFloat(float64(f), 'f', -1, 32)), nil
}
type Pt struct {
Value KeepZero
Unit string
}
func main() {
data, err := json.Marshal(Pt{40.0, "some_string"})
fmt.Println(string(data), err)
}
This results in {"Value":40.0,"Unit":"some_string"} <nil>. Check it out in playground.
By default floating point numbers are rendered without a decimal point and fractions if its value is an integer value. The representation is shorter, and it means the same number.
If you want control over how a number appears in the JSON representation, use the json.Number type.
Example:
type Pt struct {
Value json.Number
Unit string
}
func main() {
data, err := json.Marshal(Pt{json.Number("40.0"), "some_string"})
fmt.Println(string(data), err)
}
Output (try it on the Go Playground):
{"Value":40.0,"Unit":"some_string"} <nil>
If you have a number as a float64 value, you may convert it to json.Number like this:
func toNumber(f float64) json.Number {
var s string
if f == float64(int64(f)) {
s = fmt.Sprintf("%.1f", f) // 1 decimal if integer
} else {
s = fmt.Sprint(f)
}
return json.Number(s)
}
Testing it:
f := 40.0
data, err := json.Marshal(Pt{toNumber(f), "some_string"})
fmt.Println(string(data), err)
f = 40.123
data, err = json.Marshal(Pt{toNumber(f), "some_string"})
fmt.Println(string(data), err)
Output (try it on the Go Playground):
{"Value":40.0,"Unit":"some_string"} <nil>
{"Value":40.123,"Unit":"some_string"} <nil>
The other direction, if you want the float64 value of a json.Number, simply call its Number.Float64() method.
I had a similar issue where I wanted to marshal a map[string]interface{} with float values f.x 1.0 to JSON as 1.0. I solved it by adding a custom Marshal function for a custom float type and then replace the floats in the map with the custom type:
type customFloat float64
func (f customFloat) MarshalJSON() ([]byte, error) {
if float64(f) == math.Trunc(float64(f)) {
return []byte(fmt.Sprintf("%.1f", f)), nil
}
return json.Marshal(float64(f))
}
func replaceFloat(value map[string]interface{}) {
for k, v := range value {
switch val := v.(type) {
case map[string]interface{}:
replaceFloat(val)
case float64:
value[k] = customFloat(val)
}
}
}
Then replace all float64 nodes:
replaceFloat(myValue)
bytes, err := json.Marshal(myValue)
This will print the floats like 1.0
type MyFloat float64
func (mf MyFloat) MarshalJSON() ([]byte, error) {
return []byte(fmt.Sprintf("%.1f", float64(mf))), nil
}
used like this
type PricePoint struct {
Price MyFloat `json:"price"`
From time.Time `json:"valid_from"`
To time.Time `json:"valid_to"`
}
Replace the 1 in "%.1f" with what ever precision you need
Save the value as a string and cast it back if you need it.

How to get JSON when decoder fails to unmarshal?

I am using a json.Decoder to decode JSON delivered over a network stream. It works fine, but whenever someone sends data that doesn't fit the schema (e.g. sending a negative integer when the struct's field type is unsigned) it returns an error with a vague message that doesn't pinpoint the offending property. This makes debugging more difficult.
Is there any way to extract the JSON that the decoder was trying to unmarshal when it errored? Here's a small reproducable snippet:
package main
import (
"bytes"
"fmt"
"encoding/json"
)
func main() {
buff := bytes.NewBufferString("{\"bar\": -123}")
decoder := json.NewDecoder(buff)
var foo struct{
Bar uint32
}
if err := decoder.Decode(&foo); err != nil {
fmt.Println(err)
fmt.Println("TODO: how to get JSON that caused this error?")
} else {
fmt.Println(foo.Bar)
}
}
Or on playground: https://play.golang.org/p/-WdYBkYEzJ
Some information is in the error, which is of type *json.UnamrshalTypeError
type UnmarshalTypeError struct {
Value string // description of JSON value - "bool", "array", "number -5"
Type reflect.Type // type of Go value it could not be assigned to
Offset int64 // error occurred after reading Offset bytes
}
You can get the offset in the json string, the reflect.Type of the field being unmarshaled into, and the json description of the Value. This can still pose a problem for types that implement their own unmarshaling logic, which is referenced by issue 11858
As of Go 1.8 this is now possible. The UnmarshalTypeError type now contains Struct and Field values which provide the name of the struct and field which caused a type mismatch.
package main
import (
"bytes"
"fmt"
"encoding/json"
)
func main() {
buff := bytes.NewBufferString("{\"bar\": -123}")
decoder := json.NewDecoder(buff)
var foo struct{
Bar uint32
}
if err := decoder.Decode(&foo); err != nil {
if terr, ok := err.(*json.UnmarshalTypeError); ok {
fmt.Printf("Failed to unmarshal field %s \n", terr.Field)
} else {
fmt.Println(err)
}
} else {
fmt.Println(foo.Bar)
}
}
The error message string also was changed to contain this new information.
Go 1.8:
json: cannot unmarshal number -123 into Go struct field .Bar of type uint32
Go 1.7 and earlier:
json: cannot unmarshal number -123 into Go value of type uint32
You can get at each element, key, value, and even delimiters using decode.Token() such as this in the playground, and below (modified from your example):
package main
import (
"bytes"
"encoding/json"
"fmt"
)
func main() {
buff := bytes.NewBufferString(`{"foo": 123, "bar": -123, "baz": "123"}`)
decoder := json.NewDecoder(buff)
for {
t, err := decoder.Token()
if _, ok := t.(json.Delim); ok {
continue
}
fmt.Printf("type:%11T | value:%5v //", t, t)
switch t.(type) {
case uint32:
fmt.Println("you don't see any uints")
case int:
fmt.Println("you don't see any ints")
case string:
fmt.Println("handle strings as you will")
case float64:
fmt.Println("handle numbers as you will")
}
if !decoder.More() {
break
}
if err != nil {
fmt.Println(err)
}
}
}
This will output
type: string | value: foo //handle strings as you will
type: float64 | value: 123 //handle numbers as you will
type: string | value: bar //handle strings as you will
type: float64 | value: -123 //handle numbers as you will
type: string | value: baz //handle strings as you will
type: string | value: 123 //handle strings as you will
You can switch on the type and handle each one as you wish. I have shown a simple example of that as well, each of the "//comments" in the result are conditional based on the type.
You'll also notice that each numbers' type is float64, although they would fit into an int, or even uint in the case of the "foo" value, and I check for those types in the switch but they never get used. You would have to provide your own logic in order to convert the float64 values to the types you wanted if they could be and handle types that wouldn't convert as special cases, or errors or whatever you wanted.