I am trying to parse a file which contains JSON data:
[
{"a" : "1"},
{"b" : "2"},
{"c" : "3"}
]
Since this is a JSON array with dynamic keys, I thought I could use:
type data map[string]string
However, I cannot parse the file using a map:
c, _ := ioutil.ReadFile("c")
dec := json.NewDecoder(bytes.NewReader(c))
var d data
dec.Decode(&d)
json: cannot unmarshal array into Go value of type main.data
What would be the most simple way to parse a file containing a JSON data is an array (only string to string types) into a Go struct?
EDIT: To further elaborate on the accepted answer -- it's true that my JSON is an array of maps. To make my code work, the file should contain:
{
"a":"1",
"b":"2",
"c":"3"
}
Then it can be read into a map[string]string
Try this: http://play.golang.org/p/8nkpAbRzAD
package main
import (
"encoding/json"
"fmt"
"io/ioutil"
"log"
)
type mytype []map[string]string
func main() {
var data mytype
file, err := ioutil.ReadFile("test.json")
if err != nil {
log.Fatal(err)
}
err = json.Unmarshal(file, &data)
if err != nil {
log.Fatal(err)
}
fmt.Println(data)
}
It's because your json is actually an array of maps, but you're trying to unmarshall into just a map. Try using the following:
type YourJson struct {
YourSample []struct {
data map[string]string
}
}
you can try the bitly's simplejson package
https://github.com/bitly/go-simplejson
it's much easier.
Related
This is a very weird situation but I need to convert a stringified json to something valid that I can unmarshall with:
"{\"hello\": \"hi\"}"
I want to be able to unmarshall this into a struct like this:
type mystruct struct {
Hello string `json:"hello,string"`
}
I know normally the unmarshall takes bytes but Im trying to convert what I currently get into something structified.
Any suggestions?
The issue is that the encoding/json package accepts well-formed JSON, in this case the initial JSON that you have has escaped quotes, first you have to unescape them, one way to do this is by using the strconv.Unquote function, here's a sample snippet:
package main
import (
"encoding/json"
"fmt"
"strconv"
)
type mystruct struct {
Hello string `json:"hello,omitempty"`
}
func main() {
var rawJSON []byte = []byte(`"{\"hello\": \"hi\"}"`)
s, _ := strconv.Unquote(string(rawJSON))
var val mystruct
if err := json.Unmarshal([]byte(s), &val); err != nil {
// handle error
}
fmt.Println(s)
fmt.Println(err)
fmt.Println(val.Hello)
}
I'm trying to create a JSON representation within Go using a map[string]interface{} type. I'm dealing with JSON strings and I'm having a hard time figuring out how to avoid the JSON unmarshaler to automatically deal with numbers as float64s. As a result the following error occurs.
Ex.
"{ 'a' : 9223372036854775807}" should be map[string]interface{} = [a 9223372036854775807 but in reality it is map[string]interface{} = [a 9.2233720368547758088E18]
I searched how structs can be used to avoid this by using json.Number but I'd really prefer using the map type designated above.
The go json.Unmarshal(...) function automatically uses float64 for JSON numbers. If you want to unmarshal numbers into a different type then you'll have to use a custom type with a custom unmarshaler. There is no way to force the unmarshaler to deserialize custom values into a generic map.
For example, here's how you could parse values of the "a" property as a big.Int.
package main
import (
"encoding/json"
"fmt"
"math/big"
)
type MyDoc struct {
A BigA `json:"a"`
}
type BigA struct{ *big.Int }
func (a BigA) UnmarshalJSON(bs []byte) error {
_, ok := a.SetString(string(bs), 10)
if !ok {
return fmt.Errorf("invalid integer %s", bs)
}
return nil
}
func main() {
jsonstr := `{"a":9223372036854775807}`
mydoc := MyDoc{A: BigA{new(big.Int)}}
err := json.Unmarshal([]byte(jsonstr), &mydoc)
if err != nil {
panic(err)
}
fmt.Printf("OK: mydoc=%#v\n", mydoc)
// OK: mydoc=main.MyDoc{A:9223372036854775807}
}
func jsonToMap(jsonStr string) map[string]interface{} {
result := make(map[string]interface{})
json.Unmarshal([]byte(jsonStr), &result)
return result
}
Example - https://goplay.space/#ra7Gv8A5Heh
Related questions - create a JSON data as map[string]interface with the given data
All of the ways I'm seeing involve building structs and unmarshalling the data into the struct. But what if I'm getting JSON responses with hundreds of fields? I don't want to have to create 100 field structs just to be able to get to the data I want. Coming from a Java background there are easy ways to simply get the http response as a string and then pass the JSON string into a JSON object that allows for easy traversal. It's very painless. Is there anything like this in Go?
Java example in pseudo code:
String json = httpResponse.getBody();
JsonObject object = new JsonObject(json);
object.get("desiredKey");
Golang: fetch JSON from an HTTP response without using structs as helpers
This is a typical scenario we come across. This is achieved by json.Unmarshal.
Here is a simple json
{"textfield":"I'm a text.","num":1234,"list":[1,2,3]}
which is serialized to send across the network and unmarshaled at Golang end.
package main
import (
"fmt"
"encoding/json"
)
func main() {
// replace this by fetching actual response body
responseBody := `{"textfield":"I'm a text.","num":1234,"list":[1,2,3]}`
var data map[string]interface{}
err := json.Unmarshal([]byte(responseBody), &data)
if err != nil {
panic(err)
}
fmt.Println(data["list"])
fmt.Println(data["textfield"])
}
Hope this was helpful.
The json.Unmarshal method will unmarshal to a struct that does not contain all the fields in the original JSON object. In other words, you can cherry-pick your fields. Here is an example where FirstName and LastName are cherry-picked and MiddleName is ignored from the json string:
package main
import (
"encoding/json"
"fmt"
)
type Person struct {
FirstName string `json:"first_name"`
LastName string `json:"last_name"`
}
func main() {
jsonString := []byte("{\"first_name\": \"John\", \"last_name\": \"Doe\", \"middle_name\": \"Anderson\"}")
var person Person
if err := json.Unmarshal(jsonString, &person); err != nil {
panic(err)
}
fmt.Println(person)
}
The other answers here are misleading, as they don't show you what happens if you try to go deeper in the Map. This example works fine enough:
package main
import (
"encoding/json"
"fmt"
"net/http"
)
func main() {
r, e := http.Get("https://github.com/manifest.json")
if e != nil {
panic(e)
}
body := map[string]interface{}{}
json.NewDecoder(r.Body).Decode(&body)
/*
[map[
id:com.github.android
platform:play
url:https://play.google.com/store/apps/details?id=com.github.android
]]
*/
fmt.Println(body["related_applications"])
}
but if you try to go one level deeper, it fails:
/*
invalid operation: body["related_applications"][0] (type interface {} does not
support indexing)
*/
fmt.Println(body["related_applications"][0])
Instead, you would need to assert type at each level of depth:
/*
map[
id:com.github.android
platform:play
url:https://play.google.com/store/apps/details?id=com.github.android
]
*/
fmt.Println(body["related_applications"].([]interface{})[0])
You can as well unmarshal it into a map[string]interface{}
body, err := ioutil.ReadAll(resp.Body)
map := &map[string]interface{}{}
json.Unmarshal(body, map)
desiredValue := map["desiredKey"]
The received json must have an object as the most outer element. The map can also contain lists or nested maps, depending on the json.
I have the following code to try to Unmarshal this json file, however the line json.Unmarshal([]byte(msg["restaurant"]), &restaurant) always gives an error. How can I make Unmarshal ignore the "restaurant" or pass only the "restaurant" data to the Unmarshal function?
Thanks!
{
"restaurant": {
"name": "Tickets",
"owner": {
"name": "Ferran"
}
}
}
file, e := ioutil.ReadFile("./rest_read.json")
if e != nil {
fmt.Println("file error")
os.Exit(1)
}
var data interface{}
json.Unmarshal(file, &data)
msg := data.(map[string]interface{})
log.Println(msg)
log.Println(msg["restaurant"])
log.Println(reflect.TypeOf(msg["restaurant"]))
var restaurant Restaurant
json.Unmarshal([]byte(msg["restaurant"]), &restaurant)
log.Println("RName: ", restaurant.Name)
log.Println("Name: ", restaurant.Owner.Name)
It is possible to do generic unmarshalling ala gson by decoding into an interface and then extracting a top level map from the result, e.g:
var msgMapTemplate interface{}
err := json.Unmarshal([]byte(t.ResponseBody), &msgMapTemplate)
t.AssertEqual(err, nil)
msgMap := msgMapTemplate.(map[string]interface{})
See "decoding arbitrary data" in http://blog.golang.org/json-and-go for more into.
I would propose to construct a proper model for your data. This will enable you to cleanly unmarshal your data into a Go struct.
package main
import (
"encoding/json"
"fmt"
)
type Restaurant struct {
Restaurant RestaurantData `json:"restaurant"`
}
type RestaurantData struct {
Name string `json:"name"`
Owner Owner `json:"owner"`
}
type Owner struct {
Name string `json:"name"`
}
func main() {
data := `{"restaurant":{"name":"Tickets","owner":{"name":"Ferran"}}}`
r := Restaurant{}
json.Unmarshal([]byte(data), &r)
fmt.Printf("%+v", r)
}
Unmarshalling occurs recursively, so msg["restaurant"] is no longer a json string - it is another map[string]interface{}. If you want to unmarshall directly into a Restaurant object, you will have to provide a simple wrapper object with a Restaurant member and unmarshall into that.
I was marshaling and unmarshaling JSONs using golang and when I want to do it with number fields golang transforms it in floating point numbers instead of use long numbers, for example.
I have the following JSON:
{
"id": 12423434,
"Name": "Fernando"
}
After marshal it to a map and unmarshal again to a json string I get:
{
"id":1.2423434e+07,
"Name":"Fernando"
}
As you can see the "id" field is in floating point notation.
The code that I am using is the following:
package main
import (
"encoding/json"
"fmt"
"os"
)
func main() {
//Create the Json string
var b = []byte(`
{
"id": 12423434,
"Name": "Fernando"
}
`)
//Marshal the json to a map
var f interface{}
json.Unmarshal(b, &f)
m := f.(map[string]interface{})
//print the map
fmt.Println(m)
//unmarshal the map to json
result,_:= json.Marshal(m)
//print the json
os.Stdout.Write(result)
}
It prints:
map[id:1.2423434e+07 Name:Fernando]
{"Name":"Fernando","id":1.2423434e+07}
It appears to be that the first marshal to the map generates the FP. How can I fix it to a long?
This is the link to the program in the goland playground:
http://play.golang.org/p/RRJ6uU4Uw-
There are times when you cannot define a struct in advance but still require numbers to pass through the marshal-unmarshal process unchanged.
In that case you can use the UseNumber method on json.Decoder, which causes all numbers to unmarshal as json.Number (which is just the original string representation of the number). This can also useful for storing very big integers in JSON.
For example:
package main
import (
"strings"
"encoding/json"
"fmt"
"log"
)
var data = `{
"id": 12423434,
"Name": "Fernando"
}`
func main() {
d := json.NewDecoder(strings.NewReader(data))
d.UseNumber()
var x interface{}
if err := d.Decode(&x); err != nil {
log.Fatal(err)
}
fmt.Printf("decoded to %#v\n", x)
result, err := json.Marshal(x)
if err != nil {
log.Fatal(err)
}
fmt.Printf("encoded to %s\n", result)
}
Result:
decoded to map[string]interface {}{"id":"12423434", "Name":"Fernando"}
encoded to {"Name":"Fernando","id":12423434}
The JSON standard doesn't have longs or floats, it only has numbers. The json package will assume float64 when you haven't defined anything else (meaning, only provided Unmarshal with an interface{}).
What you should do is to create a proper struct (as Volker mentioned):
package main
import (
"encoding/json"
"fmt"
"os"
)
type Person struct {
Id int64 `json:"id"`
Name string `json:"name"`
}
func main() {
//Create the Json string
var b = []byte(`{"id": 12423434, "Name": "Fernando"}`)
//Marshal the json to a proper struct
var f Person
json.Unmarshal(b, &f)
//print the person
fmt.Println(f)
//unmarshal the struct to json
result, _ := json.Marshal(f)
//print the json
os.Stdout.Write(result)
}
Result:
{12423434 Fernando}
{"id":12423434,"name":"Fernando"}
Playground: http://play.golang.org/p/2R76DYVgMK
Edit:
In case you have a dynamic json structure and wish to use the benefits of a struct, you can solve it using json.RawMessage. A variable of type json.RawMessage will store the raw JSON string so that you later on, when you know what kind of object it contains, can unmarshal it into the proper struct. No matter what solution you use, you will in any case need some if or switch statement where you determine what type of structure it is.
It is also useful when parts of the JSON data will only be copied to the another JSON object such as with the id-value of a JSON RPC request.
Example of container struct using json.RawMessage and the corresponding JSON data:
type Container struct {
Type string `json:"type"`
Data json.RawMessage `json:"data"`
}
var b = []byte(`{"type": "person", "data":{"id": 12423434, "Name": "Fernando"}}`)
A modified version of your example on Playground: http://play.golang.org/p/85s130Sthu
Edit2:
If the structure of your JSON value is based on the name of a name/value pair, you can do the same with a:
type Container map[string]json.RawMessage