I am struggling with deserializing a integer into a string struct field.
The struct field is a string and is expected to be assignable from users of my library. That's why I want it to be a string, since for the purpose of writing it to the database I actually don't care about the value inside.
The users can supply text, but some just assign integers.
Consider this struct:
type Test struct {
Foo string
}
Sometimes I end up with a JSON value that is valid but won't deserialize into the struct due to the Foo field being a integer instead of a string:
{ "foo": "1" } // works
{ "foo": 1 } // doesn't
json.Unmarshal will blow up with the following error:
json: cannot unmarshal number into Go struct field test.Foo of type string
See the reproduction: https://play.golang.org/p/4Qau3umaVm
Now in every other JSON library (in other languages) I have worked in so far, if the target field is a string and you get a integer the deserializer will usually just wrap the int in a string and be done with it. Can this be achieved in Go?
Since I can't really control how the data comes in I need to make json.Unmarshal unsensitive to this - the other solution would be to define Foo as interface{} which needlessly complicates my code with type assertions etc..
Any ideas on how to do this? I basically need the inverse of json:",string"
To handle big structs you can use embedding.
Updated to not discard possibly previously set field values.
func (t *T) UnmarshalJSON(d []byte) error {
type T2 T // create new type with same structure as T but without its method set!
x := struct{
T2 // embed
Foo json.Number `json:"foo"`
}{T2: T2(*t)} // don't forget this, if you do and 't' already has some fields set you would lose them
if err := json.Unmarshal(d, &x); err != nil {
return err
}
*t = T(x.T2)
t.Foo = x.Foo.String()
return nil
}
https://play.golang.org/p/BytXCeHMvt
You can customize how the data structure is Unmarshaler by implementing the json.Unmarshaler interface.
The simplest way to handle unknown types is to unnmarshal the JSON into an intermediate structure, and handle the type assertions and validation during deserialization:
type test struct {
Foo string `json:"foo"`
}
func (t *test) UnmarshalJSON(d []byte) error {
tmp := struct {
Foo interface{} `json:"foo"`
}{}
if err := json.Unmarshal(d, &tmp); err != nil {
return err
}
switch v := tmp.Foo.(type) {
case float64:
t.Foo = strconv.Itoa(int(v))
case string:
t.Foo = v
default:
return fmt.Errorf("invalid value for Foo: %v", v)
}
return nil
}
https://play.golang.org/p/t0eI4wCxdB
Related
I want to unmarshal a JSON object where one field contains a JSON string into one coherent object. How do I do that in Go?
Example:
Input:
{
"foo":1,
"bar":"{\\"a\\":\\"Hello\\"}"
}
Go type:
type Child struct {
A string `json:"a"`
}
type Main struct {
Foo int `json:"foo"`
Bar Child `json:"bar"`
}
I guess I'd need to implement a custom UnmarshalJSON implementation on one of the types, but its twisting my head to figure out on which one and how.
I guess you want to treat this as if the JSON String were just part of the surrounding JSON object? If so, then yes, as you suggest, a custom UnmarshalJSON method on Child should accomplish this.
func (c *Child) UnmarshalJSON(p []byte) error {
var jsonString string
if err := json.Unmarshal(p, &jsonString); err != nil {
return err // Means the string was invalid
}
type C Child // A new type that doesn't have UnmarshalJSON method
return json.Unmarshal([]byte(jsonString), (*C)(c))
}
See it in the playground
if i were to create a custom UnmarshalJson for that data, I would create an auxiliary struct auxMain that has the same fields as the main struct but with Bar field as string. Then it unmarshals the JSON data into this auxiliary struct, extracting the Foo field and the Bar field as a string. After that, it unmarshals the Bar field as string into the Child struct, and assigns the extracted Foo field and the Child struct to the Main struct.
It's a round about way but seems to work in the playground.
func (m *Main) UnmarshalJSON(b []byte) error {
type auxMain struct {
Foo int `json:"foo"`
Bar string `json:"bar"`
}
var a auxMain
if err := json.Unmarshal(b, &a); err != nil {
return err
}
var child Child
if err := json.Unmarshal([]byte(a.Bar), &child); err != nil {
return err
}
m.Foo = a.Foo
m.Bar = child
return nil
}
try it out in the PlayGround and see: https://go.dev/play/p/wWIceUxu1tj
Don't know if this is what you are looking for.
What I want to do is commented:
type foo struct {
Message string `json:"message"`
}
func bar() {
//The "message" field contains a bool type which does not
//match the equivalent "message" field in foo, a string type
jsonData := byte[]("{
\"message\": true
}")
var baz foo
//Because the type of "message" in the json bytes is
//bool and not a string, put "" inside baz.Message
json.Unmarshal(jsonData, &baz)
}
How do I Unmarshal the json byte array, then only populate the particular field if the field matches the type in the json byte array field? And if the field does type does not match the json byte array field type, put in a placeholder value of nil, "" etc. ?
There are many ways to handle this, once you fix your question to something that can be answered. The one I like is to unmarshal to a variable of type interface{}, which you can then inspect:
package main
import (
"encoding/json"
"fmt"
)
type foo struct {
Message interface{} `json:"message"`
}
func bar() {
//The "message" field contains a bool type which does not
//match the equivalent "message" field in foo, a string type
jsonData := []byte(`{
"message": true
}`)
var baz foo
//Because the type of "message" in the json bytes is
//bool and not a string, put "" inside baz.Message
err := json.Unmarshal(jsonData, &baz)
fmt.Printf("unmarhal error is: %v\n", err)
if err == nil {
fmt.Printf("baz.Message is now: %T = %v\n", baz.Message, baz.Message)
}
}
func main() {
bar()
}
(Go Playground link)
It should now be fairly obvious how to switch on the type (after decoding) and see if what you got is what you want. If so, use it. If not, use your default. If necessary, decode the incoming json to more-generic Go types first, then fill in the specific type you really want to handle.
The Unmarshaler interface in the encoding/json package allows you to have full control over how the data is decoded into your types.
You only need to implement one method on your type - UnmarshalJSON(data []byte).
In the example below, I've declared a temporary anonymous type in my UnmarshalJSON method and decoded the json into an empty interface.
I can then type assert the interface into a string, and on success I set the Message field on my original Foo type. We don't need to handle the failure case, since strings in Go have a default value of "".
Here's a working example.
type Foo struct {
Message string
}
func (this *Foo) UnmarshalJSON(data []byte) error {
raw := struct {
Message interface{} `json:"message"`
}{}
err := json.Unmarshal(data, &raw)
if err != nil {
return err
}
if msgString, ok := raw.Message.(string); ok {
this.Message = msgString
}
return nil
}
My struct:
type User struct {
FirstName string `json:"firstname, omitempty" validate:"required"`
LastName string `json:"lastname, omitempty" validate:"required"`
NumberofDays int `json:"numberofdays, string" validate:"min=0,max=100"`
}
Value for NumberofDays is passed as string from the server but I want to check if it is within range and store as int.
Ex: user := &User{"Michael","Msk","3"}
I'm getting 'cannot unmarshal string into Go value of type int'.
I'm not sure how to typecast to int and do the validation
Remove the space after the comma in the struct tags, e.g. json:"numberofdays, string" should be json:"numberofdays,string". With the space, the json package ignores the string part, hence the error message you're getting. Without the space, the error goes away and behavior is as expected.
Demo comparing the two here: https://play.golang.org/p/AUImnw_PIS
Documentation of json package struct tags: https://golang.org/pkg/encoding/json/#Marshal
You can use a custom type to unmarshal a string
to an integer value using whatever set of parsing rules
you want this mapping to use:
package main
import (
"encoding/json"
"fmt"
"strconv"
)
type User struct {
FirstName string `json:"firstname,omitempty" validate:"required"`
LastName string `json:"lastname,omitempty" validate:"required"`
NumberofDays StrInt `json:"numberofdays" validate:"min=0,max=100"`
}
type StrInt int
func (si *StrInt) UnmarshalJSON(b []byte) error {
var s string
err := json.Unmarshal(b, &s)
if err != nil {
return err
}
n, err := strconv.ParseInt(s, 10, 0)
if err != nil {
return err
}
*si = StrInt(n)
return nil
}
const data = `{
"FirstName": "John",
"LastName": "Doe",
"NumberOfDays": "42"
}`
func main() {
var u User
err := json.Unmarshal([]byte(data), &u)
if err != nil {
panic(err)
}
fmt.Println(u)
}
Playground.
The idea is as follows:
Have a custom type which is mostly int but allows defining custom methods on it.
Have that type satisfy the encoding/json.Unmarshaler interface by implementing the UnmarshalJSON() method on a pointer to that type.
In the method, first unmarshal the value as its native type—string—and
then parse it as an integer.
This approach might appear to be weird, but if you'll think of it
a bit more, there's no single "true" string representation of an integer:
there are different bases and different conventions at representing
integers even in a particular base.
In your unmarshaling method you implement the policy of the text representation
of your integers.
The json package defines a custom type json.Number for these cases.
I am trying to unmarshal JSON into a struct. However, the struct has a field with a tag. Using reflection, and I try to see if the tag has the string "json" in it. If it does, then the json to unmarshal should simply be unmarshaled into the field as a string.
Example:
const data = `{"I":3, "S":{"phone": {"sales": "2223334444"}}}`
type A struct {
I int64
S string `sql:"type:json"`
}
Problem is simple - unmarshal "S" in the json as a string into the struct A.
This is how far I have come. But I am stuck here.
http://play.golang.org/p/YzrhjuXxGN
This is the go way of doing it - no reflection requred. Create a new type RawString and create MarshalJSON and UnmarshalJSON methods for it. (playground)
// RawString is a raw encoded JSON object.
// It implements Marshaler and Unmarshaler and can
// be used to delay JSON decoding or precompute a JSON encoding.
type RawString string
// MarshalJSON returns *m as the JSON encoding of m.
func (m *RawString) MarshalJSON() ([]byte, error) {
return []byte(*m), nil
}
// UnmarshalJSON sets *m to a copy of data.
func (m *RawString) UnmarshalJSON(data []byte) error {
if m == nil {
return errors.New("RawString: UnmarshalJSON on nil pointer")
}
*m += RawString(data)
return nil
}
const data = `{"i":3, "S":{"phone": {"sales": "2223334444"}}}`
type A struct {
I int64
S RawString `sql:"type:json"`
}
func main() {
a := A{}
err := json.Unmarshal([]byte(data), &a)
if err != nil {
log.Fatal("Unmarshal failed", err)
}
fmt.Println("Done", a)
}
I modified the implementation of RawMessage to create the above.
The problem here is that you are using one encoding format for two protocols and probably there is something wrong in your model.
That is a valid Json payload and should be handled as such. One trick that you can use is to create
another field to handle the "string" json and one to handle the "json struct".
See this modified example: I first unmarshal json and then marshal back to json to create the final string to upload to the database. One field is used for unmarshaling and the other to communicate with the DB.
package main
import(
"fmt"
"encoding/json"
)
const data = `{"i":3, "S":{"phone": {"sales": "2223334444"}}}`
type A struct {
I int64
Sjson struct {
Phone struct {
Sales string `json:"sales"`
} `json:"phone"`
} `json:"S", sql:"-"`
S string `sql:"type:json",json:"-"`
}
func main() {
msg := A{}
_ = json.Unmarshal([]byte(data), &msg)
data, _ := json.Marshal(msg.Sjson)
msg.S = string(data)
fmt.Println("Done", msg)
}
Looks like the problem is that s interface{} in your code was not addressable. For Value.SetString the Value has to be addressable and with Kind String. you can check the documentation for it - http://golang.org/pkg/reflect/#Value.SetString
How i understand it SetString would not change the value in a, since you are only working with interface s. in Laws of Reflection you can find "reflect.ValueOf is a copy of x, not x itself"(3rd Law).
To make your code work I made some type assertions, and I used reflect.ValueOf on a pointer to asserted struct.
To check if Value is settable or addressable you can use Value.CanSet ad Value.CanAddr
working code: http://play.golang.org/p/DTriENkzA8
No idea whether its correct way to do this
I have struct
type tySurvey struct {
Id int64 `json:"id,omitempty"`
Name string `json:"name,omitempty"`
}
I do json.Marshal write JSON bytes in HTML page. jQuery modifies name field in object and encodes object using jQueries JSON.stringify and jQuery posts string to Go handler.
id field encoded as string.
Sent: {"id":1} Received: {"id":"1"}
Problem is that json.Unmarshal fails to unmarshal that JSON because id is not integer anymore.
json: cannot unmarshal string into Go value of type int64
What is best way to handle such data? I do not wish to manually convert every field. I wish to write compact, bug free code.
Quotes is not too bad. JavaScript does not work well with int64.
I would like to learn the easy way to unmarshal json with string values in int64 values.
This is handled by adding ,string to your tag as follows:
type tySurvey struct {
Id int64 `json:"id,string,omitempty"`
Name string `json:"name,omitempty"`
}
This can be found about halfway through the documentation for Marshal.
Please note that you cannot decode the empty string by specifying omitempty as it is only used when encoding.
use json.Number
type tySurvey struct {
Id json.Number `json:"id,omitempty"`
Name string `json:"name,omitempty"`
}
You could also create a type alias for int or int64 and create a custom json unmarshaler
Sample code:
Reference
// StringInt create a type alias for type int
type StringInt int
// UnmarshalJSON create a custom unmarshal for the StringInt
/// this helps us check the type of our value before unmarshalling it
func (st *StringInt) UnmarshalJSON(b []byte) error {
//convert the bytes into an interface
//this will help us check the type of our value
//if it is a string that can be converted into a int we convert it
///otherwise we return an error
var item interface{}
if err := json.Unmarshal(b, &item); err != nil {
return err
}
switch v := item.(type) {
case int:
*st = StringInt(v)
case float64:
*st = StringInt(int(v))
case string:
///here convert the string into
///an integer
i, err := strconv.Atoi(v)
if err != nil {
///the string might not be of integer type
///so return an error
return err
}
*st = StringInt(i)
}
return nil
}
func main() {
type Item struct {
Name string `json:"name"`
ItemId StringInt `json:"item_id"`
}
jsonData := []byte(`{"name":"item 1","item_id":"30"}`)
var item Item
err := json.Unmarshal(jsonData, &item)
if err != nil {
log.Fatal(err)
}
fmt.Printf("%+v\n", item)
}