I'm building a REST API using Golang but I'm having some troubles trying to correctly Marshalling a Json Slice. I've been scratching my head for a while now, even after looking to several questions and answer and on the web.
Essentially, I have a Redis client that called after a call -X GET /todo/ spits up a slice of todos
[{"content":"test6","id":"46"} {"content":"test5","id":"45"}] //[]string
Now, I want to return a given Response based on the fact that I found todos or not, so I have a Struct like
type Response struct {
Status string
Data []string
}
Then, If I found some todos I just Marshal a json with
if(len(todos) > 0){
res := SliceResponse{"Ok", todos}
response, _ = json.Marshal(res)
}
And, In order to remove unnecessary \ inside the response, I use bytes.Replace like
response = bytes.Replace(response, []byte("\\"), []byte(""), -1)
Finally, getting
{
"Status" : "Ok",
"Data" : [
"{"content":"test6","id":"46"}",
"{"content":"test5","id":"45"}"
]
}
As you can see each " before each { and after each }, excluding the first and the last ones, are clearly wrong.
While the correct JSON would be
{
"Status": "Ok",
"Data": [{
"content ": "test6",
"id ": "46"
}, {
"content ": "test5",
"id ": "45"
}]
}
I successfully managed to get them off by finding their index and trim them off and
also with regex but I was wondering.
Is there a clean and better way to achieve that?
Whenever possible you should marshal from go objects that match your desired json. I'd recommend parsing the json from redis:
type Response struct {
Status string
Data []*Info
}
type Info struct {
Content string `json:"content"`
ID string `json:"id"`
}
func main() {
r := &Response{Status: "OK"}
for _, d := range data {
info := &Info{}
json.Unmarshal([]byte(d), info)
//handle error
r.Data = append(r.Data, info)
}
dat, _ := json.Marshal(r)
fmt.Println(string(dat))
}
Playground Link
Related
After I make a request to a server - I get JSON like this:
{
"actions": [
{
"class": "...",
"parameters": [
{ ... }
{ ... }
]
}
]
...
}
The type of the variable I've put this data in, is map[string]interface{}.
I want to access the parameters array in the first object of the actions array. I can successfully get the class property - data.(map[string]interface{})["class"].
However, if I try the same for the parameters property - I get nil ...
I tried data.(map[string][]interface{})["parameters"] - but I get error panic: interface conversion: interface {} is map[string]interface {}, not map[string][]interface {}.
Any idea what I'm missing here?
EDIT:
The Golang code for this is this:
func main() {
var jsonResult map[string]interface{}
errorFromJsonFetching := utils.GetJSON(theUrl, &jsonResult)
if errorFromJsonFetching != nil {
fmt.Printf("Error from checking deploy build: %#v\n", errorFromJsonFetching)
}
// get the top level "actions" prop ..
actionZero := jsonResult["actions"].([]interface{})[0]
fmt.Printf("Class prop: /%v %T\n", actionZero.(map[string]interface{})["class"], actionZero)
fmt.Printf("Parameters prop: /%v %T\n", actionZero.(map[string]interface{})["parameters"], actionZero)
}
The GetJSON function is from another file in the project:
func GetJSON (url string, result interface{}) error {
fmt.Printf("Getting JSON from %v\n", url)
resp, err := http.Get(url)
if err != nil {
return fmt.Errorf("Cannot fetch URL %q: %v\n", url, err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return fmt.Errorf("Bad resposne status code: %s\n", resp.Status)
}
// attempt to put the JSON in the `result` ..
err = json.NewDecoder(resp.Body).Decode(&result)
if err != nil {
return fmt.Errorf("Could not decode the JSON from response, err: %v\n", err)
}
return nil;
}
EDIT2:
Thanks to the ideas of #larsks in the comments - I created a minimal code sample with JSON in a string, directly in main.go file - and all worked fine.
Then I went ahead to the browser again and tried to fetch the data again - from directly hitting the URL or with $.getJSON from a page - and both returned one and the same JSON data.
However, in my Go code, when I dump the JSON data - I see this for the first member of actions:
map[_class:hudson.model.ParametersDefinitionProperty parameterDefinitions:[map[...
So when I try to get the parameters array by the key parameterDefinitions - then I get the array of object ... :O
Sooooo ... I don't know what's happening ... either Go itself modifies the JSON data when it gets it from the backend, or the backend itself returns different things, depending on how the data is being fetched.
(The backend is the Jenkins API by the way ... so I don't know why I get parameterDefinitions instead of parameters in Go ... :( ...)
Thanks for updating your question; having the correct data makes it easier to answer.
It's best if the code you include in your question is something we can just grab and run -- that means it compiles and runs, and when given the sample data in your question, it produces the behavior you're asking about.
I've modified your code so that I can run it locally, primarily by replacing the GetJSON method with something that reads from a file named data.json instead of hitting an API:
package main
import (
"encoding/json"
"fmt"
"os"
)
func GetJSON(result interface{}) error {
datafile, err := os.Open("data.json")
if err != nil {
panic(err)
}
return json.NewDecoder(datafile).Decode(&result)
}
func main() {
var jsonResult map[string]interface{}
errorFromJsonFetching := GetJSON(&jsonResult)
if errorFromJsonFetching != nil {
fmt.Printf("Error from checking deploy build: %#v\n", errorFromJsonFetching)
}
// get the top level "actions" prop ..
actionZero := jsonResult["actions"].([]interface{})[0]
fmt.Printf("Class prop: /%v %T\n", actionZero.(map[string]interface{})["class"], actionZero)
fmt.Printf("Parameters prop: /%v %T\n", actionZero.(map[string]interface{})["parameters"], actionZero)
}
If I feed it this sample data, which I believe matches the structure of what you show in your question:
{
"actions": [
{
"class": "Name of class",
"parameters": [
{
"name": "alice",
"color": "blue"
},
{
"name": "bob",
"count": 10,
},
{
"name": "mallory",
"valid": false,
}
]
}
]
}
It produces the following output:
Class prop: /Name of class map[string]interface {}
Parameters prop: /[map[color:blue name:alice] map[count:10 name:bob] map[name:mallory valid:false]] map[string]interface {}
This doesn't produce a nil value for the parameters key. In order to more fully answer your question, can you update it so that it has sample data and code that reproduces the behavior you're asking baout?
I am sorry if this is a silly question because i am very new in Go.
I am calling couple of apis on the base of business logic, different types of response coming like json array, nest json and single json object.
i need to wrap a api response that called according to business logic in a common format like:
{
"data":"api response here",
"statusCode":200
}
i tried some but its not expect output
type Model[T any] struct {
Data T
StatusCode int
}
model := Model[string]{Data: apiResponse, StatusCode: 200}
out, err := json.Marshal(model)
out put of this code is
{
"Data": "[{\"name\":\"Harry Potter\",\"city\":\"London\"},{\"name\":\"Don Quixote\",\"city\":\"Madrid\"},{\"name\":\"Joan of Arc\",\"city\":\"Paris\"},{\"name\":\"Rosa Park\",\"city\":\"Alabama\"}]",
"StatusCode": 200
}
this i made these changes
var result []map[string]interface{}
json.Unmarshal([]byte(body), &result)
out, err := json.Marshal(result)
output was as expected, above api response was in proper json when use []map[string]interface
problem is, its only for those api that return array of json. those apis returning single json object then to make it work i need to do this
map[string]interface`
means remove the array map.
i need to make it generic so that any kind of json response map into it.
Use type of field Data as an interface{}
type APIResponse struct {
Data interface{} `json:"data"`
StatusCode int `json:"statusCode"`
}
And then you can assign any API Response type to the Data field and marshal it.
func main() {
r := []Person{
{
Name: "Harry Porter",
City: "London",
},
{
Name: "Don Quixote",
City: "Madrid",
},
}
res := APIResponse{
Data: r,
StatusCode: 200,
}
resByt, err := json.Marshal(res)
if err != nil {
panic(err)
}
fmt.Println(string(resByt))
}
Output
{"data":[{"name":"Harry Porter","city":"London"},{"name":"Don Quixote","city":"Madrid"}],"statusCode":200}
Run the full code here in Playground.
You can simply do:
result:=map[string]interface{} {
"data": apiResponse,
"statusCode": 200,
}
out, err:=json.Marshal(result)
As part of an application we are building, one of the process steps is an AWS Lamda that captures a post request does some work with it, and then moves one. It has an API Gateway Request as a trigger and the body of this request would be a JSON String. I am having trouble parsing the JSON String to GoLang Object. Here is what I have:
The function that catches request:
func HandleRequest(ctx context.Context, event events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error) {
log.Print(fmt.Sprintf("body:[%s] ", event.Body))
parseResponseStringToTypedObject(event.Body)
return events.APIGatewayProxyResponse{
StatusCode: http.StatusOK,
Body: "OK",
}, nil
}
Then the parseResponseStringToTypedObject function :
func parseResponseStringToTypedObject(responseString string) {
b := []byte(responseString)
var resp SimpleType
err := json.Unmarshal(b, &resp)
if err == nil {
log.Print(fmt.Sprintf("Account Name: [%s]", resp.accountName))
} else {
log.Print(fmt.Sprintf("Could not unmarshall JSON string: [%s]", err.Error()))
}
}
Here is the SimpleType struct:
type SimpleType struct {
accountName string `json:accountName`
amount int `json:amount`
}
I then, as a test, posted this JSON Body via Postman:
I opened up the CloudWatch Logs (where my lamda logs to) and see that the body is present in the event.Body property, and then logging out a field in the unmarshalled object (resp.accountName) I note that the field is blank. Why is this? Here is log output for the request:
Your SimpleType struct needs 2 things here...
1) The properties need to be "public" or "exported". Meaning they need to start with an upper cased character.
AND
2) The properties need proper tags for the serialization and de serialization of JSON. e.g. each JSON tag surrounded by "
type SimpleType struct {
AccountName string `json:"accountName"`
Amount int `json:"amount"`
}
Hope this helps!
I'm building a RESTful API with Go and MongoDB, and I'm running into some difficulty with embedding the JSON for one document inside the JSON for another. Here's a toy example of what I'm trying to accomplish. I have the following schemas:
type Post struct {
ID bson.ObjectId `json:"id,omitempty"`
Title string `json:"title,omitempty"`
Owner bson.ObjectId `json:"owner,omitempty"` // references a User
}
type User struct {
ID bson.ObjectId `json:"id,omitempty"`
Name string `json:"name,omitempty"`
}
When creating the JSON for a post, I'd like to first look up the owner of the post in MongoDB and embed the resulting user inside said post's JSON (in-place of the original ObjectId), like so:
{
"id": "...",
"title": "My awesome post",
"owner": {
"id": "...",
"name": "Cody"
}
}
I'm not quite sure how to accomplish this, other than manually constructing the JSON using map[string]interface{}, like so:
post := LookupPost(...)
user := LookupUser(post.Owner)
m := map[string]interface{}{
"id": post.ID,
"title": post.Title,
"owner": map[string]interface{}{
"id": user.ID,
"name": user.Name,
},
}
b, _ := json.Marshal(m)
Obviously this doesn't scale very well isn't very DRY -- ideally, I'd be able to utilize the json tags in each struct definition and have the fields inserted automatically.
Am I missing something, or is what I'm trying to do impossible? Or am I simply not approaching MongoDB/JSON in Go correctly? To put things in perspective, I'm coming from a Node.js background, where this sort of functionality is trivial.
Edit
To clarify things, here's some incorrect Go code that shows what I'd like to do
func getPostJSON() []byte {
p := LookupPost(...)
u := LookupUser(p.Owner, ...)
uj, _ := json.Marshal(u)
p.Owner = uj // can't do this in Go
pj, _ := json.Marshal(p)
return pj
}
I'm not familar with MongoDB or bson.ObjectId, but can you substitute your own type for your User field and have MongoDB easily fill that in for you from a user's bson.ObjectId?
If so you can just wrap user object id's into their own type that implements the json.Marshaler interface. E.g.:
// Embedded (instead of `type x bson.ObjectId`) so that we
// get all the methods and satisfy all the interfaces that
// bson.ObjectId does. Hopefully that's engough to allow MongoDB
// to fill in fields of this type from a database??
type ownerObjID struct{ bson.ObjectId }
// Here we marshal the results of looking up the user from the id
// rather than just the ID itself.
func (oid ownerObjID) MarshalJSON() ([]byte, error) {
user, err := LookupUser(oid.ObjectId)
if err != nil {
return nil, err
}
return json.Marshal(user)
}
type Post struct {
ID bson.ObjectId `json:"id,omitempty"`
Title string `json:"title,omitempty"`
Owner ownerObjID `json:"owner,omitempty"` // <-- is this type wrapping doable/easy with MongoDB?
}
type User struct {
ID bson.ObjectId `json:"id,omitempty"`
Name string `json:"name,omitempty"`
}
func main() {
post := LookupPost()
b, err := json.MarshalIndent(post, "", " ")
if err != nil {
log.Fatal(err)
}
fmt.Printf("JSON:\n%s\n", b)
}
// Some stubs for demo:
func LookupPost() Post {
return Post{
ID: "postID001",
Title: "Ima Test",
Owner: ownerObjID{"ownerID002"},
}
}
func LookupUser(id bson.ObjectId) (User, error) {
return User{
ID: id,
Name: "name for " + string(id),
}, nil
}
Playground
Gives me:
JSON:
{
"id": "postID001",
"title": "Ima Test",
"owner": {
"id": "ownerID002",
"name": "name for ownerID002"
}
}
So I actually discovered a much cleaner solution to this problem:
type Post struct {
ID bson.ObjectId `bson:"_id,omitempty" json:"id,omitempty"`
Title string `bson:"title,omitempty" json:"title,omitempty"`
Owner UserRef `bson:"owner,omitempty" json:"owner,omitempty"`
}
type User struct {
ID bson.ObjectId `json:"id,omitempty"`
Name string `json:"name,omitempty"`
}
type UserRef bson.ObjectId
func (ref UserRef) GetBSON() (interface{}, error) {
return bson.ObjectId(ref), nil
}
func (ref UserRef) MarshalJSON() ([]byte, error) {
u := LookupUserInMongoDB(ref)
return json.Marshal(u)
}
Here's how it works -- mgo can't store UserRef as an ObjectId when converting a Post to bson, so we can implement the GetBSON method for UserRef to return the underlying ObjectId. This allows us to store Owner as an ObjectId in the database. And, like in #DaveC's answer, we implement the MarshalJSON method for UserRef so that when converting a Post to json, we can replace the ObjectId with an actual embedded user.
I'm asking this about Go's encoding/json, but I guess it also applies to any other JSON libraries that map JSON blobs to objects in whatever language.
Here's an example. If you want to a shorten a URL using the goo.gl URL shortener API, you get back either a successful response:
{
"kind": "urlshortener#url",
"id": "http://goo.gl/fbsS",
"longUrl": "http://www.google.com/"
}
Or an error response:
{
"error": {
"errors": [
{
"domain": "global",
"reason": "required",
"message": "Required",
"locationType": "parameter",
"location": "resource.longUrl"
}
],
"code": 400,
"message": "Required"
}
}
Is there an idiomatic way of dealing with this -- a response that could adhere to two completely different schemas?
Normally I deal with JSON using maps/lists; I know that's possible in Go. I could unmarshal to a map[string]interface{} and then check if the map has "error" as a key. But then I'd have to decode again into a proper struct, I think. (Am I wrong?)
I'm doing something like this. I have one type for each kind of response:
type successResponse struct {
Kind string
Id string
LongUrl string
}
type errorResponse struct {
Error struct {
Errors []struct {
Domain string
Reason string
Message string
LocationType string
Location string
}
Code int
Message string
}
}
And decoding looks like this:
s := new(successResponse)
err := json.Unmarshal(blob, s)
if err == nil {
// handle success
} else {
e := new(errorResponse)
err = json.Unmarshal(blob, e)
if err == nil {
// handle error response
} else {
// handle actual error
}
}
But that seems kind of ugly. How should I approach this?
Since the fields in your json responses are distinct from each other you can just create one struct with the union of all the fields. The json decoder will ignore fields that are not present in the json string and you can test the existence of the fields to know which type of response you are getting back.
I was confused about this, too, and thought I had to decode it again. You don't, though. You just have to typecast the interface{} data into the appropriate structure.
For example if the json package has put the value into a generic interface{}, you can typecast it into ErrorType with error := val.(ErrorType).
You can use foo.(type) in a switch statement to "do the right thing", if you are parsing based on what type the value is.
I've only been learning Go this week so it's not the prettiest code, but there are some examples in the geodns JSON configuration parsing.
Have you tried Go-SimpleJSON? I think this might solve your issue.
type Response struct {
Kind string
Id string
LongUrl string
Error struct {
Errors []struct {
Domain string
Reason string
Message string
LocationType string
Location string
}
Code int
Message string
}
}
s := Response{}
if err := json.Unmarshal(blob, &s); err == nil {
if s.Error == nil {
// success
} else {
// error
}
} else {
// something went wrong
}