So I'm making a Go service that makes a call to a restful API, I have no control over the API I'm calling.
I know that Go has a nice built in deserializer in NewDecoder->Decode, but it only works for struct fields that start with capital letters (aka public fields). Which poses a problem because the JSON I'm trying to consume looks like this:
{
"_next": "someValue",
"data": [{/*a collection of objects*/}],
"message": "success"
}
How the heck would I map "_next"?
Use tags to specify the field name in JSON. The JSON object you posted above can be modeled like this:
type Something struct {
Next string `json:"_next"`
Data []interface{} `json:"data"`
Message string `json:"message"`
}
Testing it:
func main() {
var sg Something
if err := json.Unmarshal([]byte(s), &sg); err != nil {
panic(err)
}
fmt.Printf("%+v", sg)
}
const s = `{
"_next": "someValue",
"data": ["one", 2],
"message": "success"
}`
Output (try it on the Go Playground):
{Next:someValue Data:[one 2] Message:success}
Also note that you may also unmarshal into maps or interface{} values, so you don't even have to create structs, but it won't be as convenient using it as the structs:
func main() {
var m map[string]interface{}
if err := json.Unmarshal([]byte(s), &m); err != nil {
panic(err)
}
fmt.Printf("%+v", m)
}
const s = `{
"_next": "someValue",
"data": ["one", 2],
"message": "success"
}`
Output (try it on the Go Playground):
map[_next:someValue data:[one 2] message:success]
Tags will solve your problem.
Hoping it may help others who come here, you can make use of https://mholt.github.io/json-to-go/ to generate Go structs. Paste a JSON structure on the left and the equivalent Go type will be generated to the right, which you can paste into your program.
Related
After I make a request to a server - I get JSON like this:
{
"actions": [
{
"class": "...",
"parameters": [
{ ... }
{ ... }
]
}
]
...
}
The type of the variable I've put this data in, is map[string]interface{}.
I want to access the parameters array in the first object of the actions array. I can successfully get the class property - data.(map[string]interface{})["class"].
However, if I try the same for the parameters property - I get nil ...
I tried data.(map[string][]interface{})["parameters"] - but I get error panic: interface conversion: interface {} is map[string]interface {}, not map[string][]interface {}.
Any idea what I'm missing here?
EDIT:
The Golang code for this is this:
func main() {
var jsonResult map[string]interface{}
errorFromJsonFetching := utils.GetJSON(theUrl, &jsonResult)
if errorFromJsonFetching != nil {
fmt.Printf("Error from checking deploy build: %#v\n", errorFromJsonFetching)
}
// get the top level "actions" prop ..
actionZero := jsonResult["actions"].([]interface{})[0]
fmt.Printf("Class prop: /%v %T\n", actionZero.(map[string]interface{})["class"], actionZero)
fmt.Printf("Parameters prop: /%v %T\n", actionZero.(map[string]interface{})["parameters"], actionZero)
}
The GetJSON function is from another file in the project:
func GetJSON (url string, result interface{}) error {
fmt.Printf("Getting JSON from %v\n", url)
resp, err := http.Get(url)
if err != nil {
return fmt.Errorf("Cannot fetch URL %q: %v\n", url, err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return fmt.Errorf("Bad resposne status code: %s\n", resp.Status)
}
// attempt to put the JSON in the `result` ..
err = json.NewDecoder(resp.Body).Decode(&result)
if err != nil {
return fmt.Errorf("Could not decode the JSON from response, err: %v\n", err)
}
return nil;
}
EDIT2:
Thanks to the ideas of #larsks in the comments - I created a minimal code sample with JSON in a string, directly in main.go file - and all worked fine.
Then I went ahead to the browser again and tried to fetch the data again - from directly hitting the URL or with $.getJSON from a page - and both returned one and the same JSON data.
However, in my Go code, when I dump the JSON data - I see this for the first member of actions:
map[_class:hudson.model.ParametersDefinitionProperty parameterDefinitions:[map[...
So when I try to get the parameters array by the key parameterDefinitions - then I get the array of object ... :O
Sooooo ... I don't know what's happening ... either Go itself modifies the JSON data when it gets it from the backend, or the backend itself returns different things, depending on how the data is being fetched.
(The backend is the Jenkins API by the way ... so I don't know why I get parameterDefinitions instead of parameters in Go ... :( ...)
Thanks for updating your question; having the correct data makes it easier to answer.
It's best if the code you include in your question is something we can just grab and run -- that means it compiles and runs, and when given the sample data in your question, it produces the behavior you're asking about.
I've modified your code so that I can run it locally, primarily by replacing the GetJSON method with something that reads from a file named data.json instead of hitting an API:
package main
import (
"encoding/json"
"fmt"
"os"
)
func GetJSON(result interface{}) error {
datafile, err := os.Open("data.json")
if err != nil {
panic(err)
}
return json.NewDecoder(datafile).Decode(&result)
}
func main() {
var jsonResult map[string]interface{}
errorFromJsonFetching := GetJSON(&jsonResult)
if errorFromJsonFetching != nil {
fmt.Printf("Error from checking deploy build: %#v\n", errorFromJsonFetching)
}
// get the top level "actions" prop ..
actionZero := jsonResult["actions"].([]interface{})[0]
fmt.Printf("Class prop: /%v %T\n", actionZero.(map[string]interface{})["class"], actionZero)
fmt.Printf("Parameters prop: /%v %T\n", actionZero.(map[string]interface{})["parameters"], actionZero)
}
If I feed it this sample data, which I believe matches the structure of what you show in your question:
{
"actions": [
{
"class": "Name of class",
"parameters": [
{
"name": "alice",
"color": "blue"
},
{
"name": "bob",
"count": 10,
},
{
"name": "mallory",
"valid": false,
}
]
}
]
}
It produces the following output:
Class prop: /Name of class map[string]interface {}
Parameters prop: /[map[color:blue name:alice] map[count:10 name:bob] map[name:mallory valid:false]] map[string]interface {}
This doesn't produce a nil value for the parameters key. In order to more fully answer your question, can you update it so that it has sample data and code that reproduces the behavior you're asking baout?
I am sorry if this is a silly question because i am very new in Go.
I am calling couple of apis on the base of business logic, different types of response coming like json array, nest json and single json object.
i need to wrap a api response that called according to business logic in a common format like:
{
"data":"api response here",
"statusCode":200
}
i tried some but its not expect output
type Model[T any] struct {
Data T
StatusCode int
}
model := Model[string]{Data: apiResponse, StatusCode: 200}
out, err := json.Marshal(model)
out put of this code is
{
"Data": "[{\"name\":\"Harry Potter\",\"city\":\"London\"},{\"name\":\"Don Quixote\",\"city\":\"Madrid\"},{\"name\":\"Joan of Arc\",\"city\":\"Paris\"},{\"name\":\"Rosa Park\",\"city\":\"Alabama\"}]",
"StatusCode": 200
}
this i made these changes
var result []map[string]interface{}
json.Unmarshal([]byte(body), &result)
out, err := json.Marshal(result)
output was as expected, above api response was in proper json when use []map[string]interface
problem is, its only for those api that return array of json. those apis returning single json object then to make it work i need to do this
map[string]interface`
means remove the array map.
i need to make it generic so that any kind of json response map into it.
Use type of field Data as an interface{}
type APIResponse struct {
Data interface{} `json:"data"`
StatusCode int `json:"statusCode"`
}
And then you can assign any API Response type to the Data field and marshal it.
func main() {
r := []Person{
{
Name: "Harry Porter",
City: "London",
},
{
Name: "Don Quixote",
City: "Madrid",
},
}
res := APIResponse{
Data: r,
StatusCode: 200,
}
resByt, err := json.Marshal(res)
if err != nil {
panic(err)
}
fmt.Println(string(resByt))
}
Output
{"data":[{"name":"Harry Porter","city":"London"},{"name":"Don Quixote","city":"Madrid"}],"statusCode":200}
Run the full code here in Playground.
You can simply do:
result:=map[string]interface{} {
"data": apiResponse,
"statusCode": 200,
}
out, err:=json.Marshal(result)
The docs (https://github.com/mitchellh/mapstructure) mention the benefits of mapstructure for decoding
{
"type": "person",
"name": "Mitchell"
}
I'm looking for an example that show the decoding process, e.g. to unmarshal either a Person or Pet class
That library only maps maps to structures. It does not do type-specific decoding. Quoting from its doc:
Perhaps we can't populate a specific structure without first reading the "type" field from the JSON. We could always do two passes over the decoding of the JSON (reading the "type" first, and the rest later). However, it is much simpler to just decode this into a map[string]interface{} structure, read the "type" key, then use something like this library to decode it into the proper structure.
All it offers is that you only have to parse the JSON source once, into a map, then you can decide yourself what type you need to map or unmarshal into. Then you can use the already constructed map to fill the type you need.
See this example:
type Person struct {
Name string `json:"name"`
}
type Credentials struct {
User string `json:"user"`
Password string `json:"password"`
}
func main() {
for _, src := range []string{srcPerson, srcCredentials} {
var m map[string]interface{}
if err := json.Unmarshal([]byte(src), &m); err != nil {
panic(err)
}
switch m["type"] {
case "person":
var p Person
if err := mapstructure.Decode(m, &p); err != nil {
panic(err)
}
fmt.Printf("%T %+v\n", p, p)
case "credentials":
var c Credentials
if err := mapstructure.Decode(m, &c); err != nil {
panic(err)
}
fmt.Printf("%T %+v\n", c, c)
}
}
}
const srcPerson = `{
"type": "person",
"name": "Mitchell"
}`
const srcCredentials = `{
"type": "credentials",
"user": "bob",
"password": "secret"
}`
Output:
main.Person {Name:Mitchell}
main.Credentials {User:bob Password:secret}
I am working on some code to parse the JSON data from an HTTP response. The code I have looks something like this:
type ResultStruct struct {
result []map[string]string
}
var jsonData ResultStruct
err = json.Unmarshal(respBytes, &jsonData)
The json in the respBytes variable looks like this:
{
"result": [
{
"id": "ID 1"
},
{
"id": "ID 2"
}
]
}
However, err is not nil. When I print it out it says unexpected end of JSON input. What is causing this? The JSON seems to valid. Does this error have something to do with my custom struct?
Thanks in advance!
The unexpected end of JSON input is the result of a syntax error in the JSON input (likely a missing ", }, or ]). The error does not depend on the type of the value that you are decoding to.
I ran the code with the example JSON input on the playground. It runs without error.
The code does not decode anything because the result field is not exported. If you export the result field:
type ResultStruct struct {
Result []map[string]string
}
then the input is decoded as shown in this playground example.
I suspect that you are not reading the entire response body in your application. I suggest decoding the JSON input using:
err := json.NewDecoder(resp.Body).Decode(&jsonData)
The decoder reads directly from the response body.
You can also get this error if you're using json.RawMessage in an unexported field. For example, the following code produces the same error:
package main
import (
"encoding/json"
"fmt"
)
type MyJson struct {
Foo bool `json:"foo"`
bar json.RawMessage `json:"bar"`
}
type Bar struct {
X int `json:"x"`
}
var respBytes = []byte(`
{
"foo": true,
"bar": { "x": 10 }
}`)
func main() {
var myJson MyJson
err := json.Unmarshal(respBytes, &myJson)
if err != nil {
fmt.Println(err)
return
}
myBar := new(Bar)
err = json.Unmarshal(myJson.bar, myBar)
fmt.Println(err)
}
If you export "MyJson.bar" field (e.g. -> "MyJson.Bar", then the code works.
it is not the case here; but if you are getting this error loading json from a file it Will occur if the byte slice for the buffer is not initialized the the byte size of the file. [when you're new like me that happens! ] Since this is the first search result I got it still took some digging to figure out. In this use case the error is a bit misleading.
type GenesisResultStruct []GenesisField
fileinfo, _ := genesis.Stat()
bs := make([]byte, fileinfo.Size())
//bs := []byte {} // wrong!!
_, error := genesis.Read(bs)
if error != nil {
fmt.Println("genesis read error: ", error)
os.Exit(1)
}
var jsonData GenesisResultStruct
eGen = json.Unmarshal(bs, &jsonData)
if eGen != nil {
fmt.Println("genesis unmarshal error: ", eGen)
os.Exit(1)
}
Faced the same issue today.
You can also get this error if the respBytes is nil or there are no brackets [] if you are unmarshalling it to a slice.
In that case, you need to explicitly set respBytes.
As you are unmarshalling it to a slice, brackets [] are expected in the byte-slice
if src == nil {
src = []byte("[]")
}
I have a map that uses string for both key and value. I have an array of keys that specifies the order of the values of the map.
I want to serialize that map to a JSON, but keeping the order defined on the array.
There is a sample code here: http://play.golang.org/p/A52GTDY6Wx
I want to serialize it as:
{
"name": "John",
"age": "20"
}
But if I serialize the map directly, the keys are ordered alphabetically:
{
"age": "20",
"name": "John"
}
I can serialize it as an array of maps, thus keeping the order, however that generates a lot of undesired characters:
[
{
"name": "John"
},
{
"age": "20"
}
]
In my real code I need to serialize the results of a database query which is specified in a text file, and I need to maintain the column order. I cannot use structs because the columns are not known at compile time.
EDIT: I don't need to read the JSON later in the specified order. The generated JSON is meant to be read by people, so I just want it to be as humanly readable as possible.
I could use a custom format but JSON suits me perfectly for this.
Thanks!
You need to implement the json.Marshaler interface on a custom type. This has the advantage of playing well within other struct types.
Sorry, you're always going to have to write a little bit of JSON encoding code.
package main
import (
"bytes"
"encoding/json"
"fmt"
"log"
)
type KeyVal struct {
Key string
Val interface{}
}
// Define an ordered map
type OrderedMap []KeyVal
// Implement the json.Marshaler interface
func (omap OrderedMap) MarshalJSON() ([]byte, error) {
var buf bytes.Buffer
buf.WriteString("{")
for i, kv := range omap {
if i != 0 {
buf.WriteString(",")
}
// marshal key
key, err := json.Marshal(kv.Key)
if err != nil {
return nil, err
}
buf.Write(key)
buf.WriteString(":")
// marshal value
val, err := json.Marshal(kv.Val)
if err != nil {
return nil, err
}
buf.Write(val)
}
buf.WriteString("}")
return buf.Bytes(), nil
}
func main() {
dict := map[string]interface{}{
"orderedMap": OrderedMap{
{"name", "John"},
{"age", 20},
},
}
dump, err := json.Marshal(dict)
if err != nil {
log.Fatal(err)
}
fmt.Printf("%s\n", dump)
}
Outputs
{"orderedMap":{"name":"John","age":20}}
For that specific requirement you really don't need to use json.Marshal at all, you can simply implement your own function like this:
type OrderedMap map[string]string
func (om OrderedMap) ToJson(order ...string) string {
buf := &bytes.Buffer{}
buf.Write([]byte{'{', '\n'})
l := len(order)
for i, k := range order {
fmt.Fprintf(buf, "\t\"%s\": \"%v\"", k, om[k])
if i < l-1 {
buf.WriteByte(',')
}
buf.WriteByte('\n')
}
buf.Write([]byte{'}', '\n'})
return buf.String()
}
func main() {
om := OrderedMap{
"age": "20",
"name": "John",
}
fmt.Println(om.ToJson("name", "age"))
}
Probably the easiest solution: https://github.com/iancoleman/orderedmap
Although it might be slow as it's mentioned here
Here is a MapSlice implementation similar to what YAML v2 offers. It can do both Marshal and Unmarshal.
https://github.com/mickep76/mapslice-json