JSON unmarshaling with long numbers gives floating point number - json

I was marshaling and unmarshaling JSONs using golang and when I want to do it with number fields golang transforms it in floating point numbers instead of use long numbers, for example.
I have the following JSON:
{
"id": 12423434,
"Name": "Fernando"
}
After marshal it to a map and unmarshal again to a json string I get:
{
"id":1.2423434e+07,
"Name":"Fernando"
}
As you can see the "id" field is in floating point notation.
The code that I am using is the following:
package main
import (
"encoding/json"
"fmt"
"os"
)
func main() {
//Create the Json string
var b = []byte(`
{
"id": 12423434,
"Name": "Fernando"
}
`)
//Marshal the json to a map
var f interface{}
json.Unmarshal(b, &f)
m := f.(map[string]interface{})
//print the map
fmt.Println(m)
//unmarshal the map to json
result,_:= json.Marshal(m)
//print the json
os.Stdout.Write(result)
}
It prints:
map[id:1.2423434e+07 Name:Fernando]
{"Name":"Fernando","id":1.2423434e+07}
It appears to be that the first marshal to the map generates the FP. How can I fix it to a long?
This is the link to the program in the goland playground:
http://play.golang.org/p/RRJ6uU4Uw-

There are times when you cannot define a struct in advance but still require numbers to pass through the marshal-unmarshal process unchanged.
In that case you can use the UseNumber method on json.Decoder, which causes all numbers to unmarshal as json.Number (which is just the original string representation of the number). This can also useful for storing very big integers in JSON.
For example:
package main
import (
"strings"
"encoding/json"
"fmt"
"log"
)
var data = `{
"id": 12423434,
"Name": "Fernando"
}`
func main() {
d := json.NewDecoder(strings.NewReader(data))
d.UseNumber()
var x interface{}
if err := d.Decode(&x); err != nil {
log.Fatal(err)
}
fmt.Printf("decoded to %#v\n", x)
result, err := json.Marshal(x)
if err != nil {
log.Fatal(err)
}
fmt.Printf("encoded to %s\n", result)
}
Result:
decoded to map[string]interface {}{"id":"12423434", "Name":"Fernando"}
encoded to {"Name":"Fernando","id":12423434}

The JSON standard doesn't have longs or floats, it only has numbers. The json package will assume float64 when you haven't defined anything else (meaning, only provided Unmarshal with an interface{}).
What you should do is to create a proper struct (as Volker mentioned):
package main
import (
"encoding/json"
"fmt"
"os"
)
type Person struct {
Id int64 `json:"id"`
Name string `json:"name"`
}
func main() {
//Create the Json string
var b = []byte(`{"id": 12423434, "Name": "Fernando"}`)
//Marshal the json to a proper struct
var f Person
json.Unmarshal(b, &f)
//print the person
fmt.Println(f)
//unmarshal the struct to json
result, _ := json.Marshal(f)
//print the json
os.Stdout.Write(result)
}
Result:
{12423434 Fernando}
{"id":12423434,"name":"Fernando"}
Playground: http://play.golang.org/p/2R76DYVgMK
Edit:
In case you have a dynamic json structure and wish to use the benefits of a struct, you can solve it using json.RawMessage. A variable of type json.RawMessage will store the raw JSON string so that you later on, when you know what kind of object it contains, can unmarshal it into the proper struct. No matter what solution you use, you will in any case need some if or switch statement where you determine what type of structure it is.
It is also useful when parts of the JSON data will only be copied to the another JSON object such as with the id-value of a JSON RPC request.
Example of container struct using json.RawMessage and the corresponding JSON data:
type Container struct {
Type string `json:"type"`
Data json.RawMessage `json:"data"`
}
var b = []byte(`{"type": "person", "data":{"id": 12423434, "Name": "Fernando"}}`)
A modified version of your example on Playground: http://play.golang.org/p/85s130Sthu
Edit2:
If the structure of your JSON value is based on the name of a name/value pair, you can do the same with a:
type Container map[string]json.RawMessage

Related

Why doesn't JSON parsing fail with completely different type passed to Decode()?

I have the following data structures which I'd like to parse from an API:
type OrderBook struct {
Pair string `json:"pair"`
UpdateTime int64 `json:"update_time"`
}
type depthResponse struct {
Result OrderBook `json:"result"`
// doesn't matter here
//Cmd string `json:"-"`
}
and when I parse the following:
data := `{"error":{"code":"3016","msg":"交易对错误"},"cmd":"depth"}`
It doesn't fail. Why?
Full source code (playground)
package main
import (
"encoding/json"
"fmt"
"log"
"strings"
)
type OrderBook struct {
Pair string `json:"pair"`
UpdateTime int64 `json:"update_time"`
}
type depthResponse struct {
Result OrderBook `json:"result"`
}
func main() {
data := `{"error":{"code":"3016","msg":"交易对错误"},"cmd":"depth"}`
r := strings.NewReader(data)
var resp depthResponse
if err := json.NewDecoder(r).Decode(&resp); err != nil {
log.Fatalf("We should end up here: %v", err)
}
fmt.Printf("%+v\n", resp)
}
That's the expected behaviour of Decode (as documented in the Unmarshal function):
https://golang.org/pkg/encoding/json/#Unmarshal
By default, object keys which don't have a corresponding struct field are ignored.
You can however use the DisallowUnknownFields() function (as described in the docs as well) to have it fail if the input JSON has fields not contained in the destination struct.
dec := json.NewDecoder(r)
dec.DisallowUnknownFields()
In that case, you'll get an error as you expect.
Modified playground here: https://play.golang.org/p/A0f6dxTXV34

Does json.Unmarshal require your result structure to match exactly the JSON passed in?

I have a JSON string I want to unmarshal:
{
"id":1720,
"alertId":1,
"alertName":"{stats} Test Lambda Alert",
"dashboardId":5,
"panelId":2,
"userId":0,
"newState":"alerting",
"prevState":"ok",
"time":1523983581000,
"text":"",
"regionId":0,
"tags":[],
"login":"",
"email":"",
"avatarUrl":"",
"data":{
"evalMatches":[
{
"metric":"{prod}{stats} Lambda Alert Test",
"tags":null,
"value":16.525333333333332
}
]
}
}
I get the raw stream via a request: bodyBytes, err := ioutil.ReadAll(resp.Body)
I was hoping I could just specify a struct that pulls the values I care about, e.g.,
type Result struct {
ID string `json:"id"`
Time int64 `json:"time"`
}
However, when I try this, I get errors.
type Result struct {
ID string `json:"id"`
Time string `json:"time"`
}
var result Result
err2 := json.Unmarshal(bodyBytes, &result)
if err2 != nil {
log.Fatal(fmt.Sprintf(`Error Unmarshalling: %s`, err2))
}
fmt.Println(result.ID)
Error Unmarshalling: json: cannot unmarshal array into Go value of type main.Result
I suspect this error may be due to what's actually returned from ioutil.ReadAll(), since it has the above JSON string wrapped in [ ] if I do a fmt.Println(string(bodyBytes)), but if I try to json.Unmarshal(bodyBytes[0], &result), I just get compile errors, so I'm not sure.
If I want to unmarshal a JSON string, do I have to specify the full structure in my type Result struct? Is there a way around this? I don't want to be bound to the JSON object I receive (if the API changes upstream, it requires us to modify our code to recognize that, etc.).
You can unmarshal into structs that represent only some fields of your JSON document, but the field types have to match, as the error clearly states:
cannot unmarshal number into Go struct field Result.id of type string
You cannot unmarshal a number into a string. If you define the ID field as any numeric type it'll work just fine:
package main
import (
"encoding/json"
"fmt"
)
var j = []byte(`
{
"id":1720,
"prevState":"ok",
"time":1523983581000,
"text":"",
"regionId":0
}
`)
type Result struct {
ID int `json:"id"` // or any other integer type, or float{32,64}, or json.Number
Time int64 `json:"time"`
}
func main() {
var r Result
err := json.Unmarshal(j, &r)
fmt.Println(r, err)
}
Try it on the playground: https://play.golang.org/p/lqsQwLW2dHZ
Update
You have just edited your question with the actual error you receive. You have to unmarshal JSON arrays into slices. So if the HTTP response in fact returns a JSON array, unmarshal into []Result:
var j = []byte(`
[
{
"id":1720,
"prevState":"ok",
"time":1523983581000,
"text":"",
"regionId":0
}
]
`)
var r []Result
err := json.Unmarshal(j, &r)
fmt.Println(r[0], err)
https://play.golang.org/p/EbOVA8CbcFO
To generate Go types that match your JSON document pretty well, use https://mholt.github.io/json-to-go/.

Unmarshalling array of bytes to an interface and type cast that interface into struct doesn't work?

I have been coding in golang for a while now. I have come across something I thought would work perfectly fine.
When I JSON Marshal a nested struct in golang I get the array of bytes, when I UnMarshal the same into an interface and convert the interface into the respective nested struct, it gives me a panic stating interface conversion: interface is map[string]interface but not the nested struct.
Please go through the link below.
https://play.golang.org/p/apdR4TKjee-
Can someone explain to me what is that I am missing?
When you unmarshall JSON into interface{}, it has no way to know what type you want it to use, so it defaults to map[string]interface{} as indicated in the documentation:
To unmarshal JSON into an interface value, Unmarshal stores one of these in the interface value:
...
map[string]interface{}, for JSON objects
If you want to unmarshal to a specific type, pass an instance of that type to Unmarshal:
var result sample
err = json.Unmarshal(data,&result)
You can not unmarshal to nil interface, however, if interface has a pointer to undelying structure it will work see below your updated code:
package main
import (
"encoding/json"
"fmt"
"github.com/viant/toolbox"
"log"
)
type sample struct {
Ping string `json:"ping"`
Pong sample1 `json:"long"`
}
type sample1 struct {
Play string `json:"plong"`
}
func main() {
var ans sample
var result interface{} = &sample{}
ans.Ping = "asda"
var ans1 sample1
ans1.Play = "asasd"
ans.Pong = ans1
fmt.Println(ans)
converter := toolbox.NewColumnConverter(toolbox.DefaultDateLayout)
var aMap = make(map[string]interface{})
converter.AssignConverted(&aMap, ans)
data, err := json.Marshal(ans)
fmt.Println(err)
fmt.Println(string(data), err)
err = json.Unmarshal(data,&result)
fmt.Println(result)
if err!= nil{
log.Fatal(err)
}
fmt.Println(result.(*sample))
fmt.Println(result)
}

Golang parse JSON array into data structure

I am trying to parse a file which contains JSON data:
[
{"a" : "1"},
{"b" : "2"},
{"c" : "3"}
]
Since this is a JSON array with dynamic keys, I thought I could use:
type data map[string]string
However, I cannot parse the file using a map:
c, _ := ioutil.ReadFile("c")
dec := json.NewDecoder(bytes.NewReader(c))
var d data
dec.Decode(&d)
json: cannot unmarshal array into Go value of type main.data
What would be the most simple way to parse a file containing a JSON data is an array (only string to string types) into a Go struct?
EDIT: To further elaborate on the accepted answer -- it's true that my JSON is an array of maps. To make my code work, the file should contain:
{
"a":"1",
"b":"2",
"c":"3"
}
Then it can be read into a map[string]string
Try this: http://play.golang.org/p/8nkpAbRzAD
package main
import (
"encoding/json"
"fmt"
"io/ioutil"
"log"
)
type mytype []map[string]string
func main() {
var data mytype
file, err := ioutil.ReadFile("test.json")
if err != nil {
log.Fatal(err)
}
err = json.Unmarshal(file, &data)
if err != nil {
log.Fatal(err)
}
fmt.Println(data)
}
It's because your json is actually an array of maps, but you're trying to unmarshall into just a map. Try using the following:
type YourJson struct {
YourSample []struct {
data map[string]string
}
}
you can try the bitly's simplejson package
https://github.com/bitly/go-simplejson
it's much easier.

golang json unmarshal part of map[string]interface{}

I have the following code to try to Unmarshal this json file, however the line json.Unmarshal([]byte(msg["restaurant"]), &restaurant) always gives an error. How can I make Unmarshal ignore the "restaurant" or pass only the "restaurant" data to the Unmarshal function?
Thanks!
{
"restaurant": {
"name": "Tickets",
"owner": {
"name": "Ferran"
}
}
}
file, e := ioutil.ReadFile("./rest_read.json")
if e != nil {
fmt.Println("file error")
os.Exit(1)
}
var data interface{}
json.Unmarshal(file, &data)
msg := data.(map[string]interface{})
log.Println(msg)
log.Println(msg["restaurant"])
log.Println(reflect.TypeOf(msg["restaurant"]))
var restaurant Restaurant
json.Unmarshal([]byte(msg["restaurant"]), &restaurant)
log.Println("RName: ", restaurant.Name)
log.Println("Name: ", restaurant.Owner.Name)
It is possible to do generic unmarshalling ala gson by decoding into an interface and then extracting a top level map from the result, e.g:
var msgMapTemplate interface{}
err := json.Unmarshal([]byte(t.ResponseBody), &msgMapTemplate)
t.AssertEqual(err, nil)
msgMap := msgMapTemplate.(map[string]interface{})
See "decoding arbitrary data" in http://blog.golang.org/json-and-go for more into.
I would propose to construct a proper model for your data. This will enable you to cleanly unmarshal your data into a Go struct.
package main
import (
"encoding/json"
"fmt"
)
type Restaurant struct {
Restaurant RestaurantData `json:"restaurant"`
}
type RestaurantData struct {
Name string `json:"name"`
Owner Owner `json:"owner"`
}
type Owner struct {
Name string `json:"name"`
}
func main() {
data := `{"restaurant":{"name":"Tickets","owner":{"name":"Ferran"}}}`
r := Restaurant{}
json.Unmarshal([]byte(data), &r)
fmt.Printf("%+v", r)
}
Unmarshalling occurs recursively, so msg["restaurant"] is no longer a json string - it is another map[string]interface{}. If you want to unmarshall directly into a Restaurant object, you will have to provide a simple wrapper object with a Restaurant member and unmarshall into that.