This question already has an answer here:
json.Unmarshal returning blank structure
(1 answer)
Closed 3 years ago.
I am new to Go, and I am trying to practice with building a simple HTTP server. However I met some problems with JSON responses. I wrote following code, then try postman to send some JSON data. However, my postman always gets an empty response and the content-type is text/plain; charset=utf-8. Then I checked a sample in http://www.alexedwards.net/blog/golang-response-snippets#json. I copied and pasted the sample, and it was working well. But I cannot see any difference between mine and the sample. Can someone give some help?
package main
import (
"encoding/json"
"net/http"
)
type ResponseCommands struct {
key string
value bool
}
func main() {
http.HandleFunc("/", handler)
http.ListenAndServe(":5432", nil)
}
func handler(rw http.ResponseWriter, req *http.Request) {
responseBody := ResponseCommands{"BackOff", false}
data, err := json.Marshal(responseBody)
if err != nil {
http.Error(rw, err.Error(), http.StatusInternalServerError)
return
}
rw.WriteHeader(200)
rw.Header().Set("Content-Type", "application/json")
rw.Write(data)
}
The main difference is that the variable in the struct are public (exported)
type Profile struct {
Name string
Hobbies []string
}
In your case, they are not (lowercase).
type ResponseCommands struct {
key string
value bool
}
See "Lowercase JSON key names with JSON Marshal in Go".
As VonC already answered correct. Just want to add that IDEA can help with such 'small' problems.
I'm using Gogland and it let me know that json tag cannot be applied to lowercase field.
Related
I'm new to Go so please bear with me if this is a trivial problem. I am using a home grown "type registry" to map type names to their type, so as to generate them dynamically based on use cases that point to the various type names (I'm basically trying for a simple solution to polymorphic Aggregation JSON response structures in Elasticsearch, but of course this could apply to many other dynamic/polymorphic situations).
I'm using the solution provided by dolmen in this question: is there a way to create an instance of a struct from a string? :
var typeRegistry = make(map[string]reflect.Type)
func registerType(typedNil interface{}) {
t := reflect.TypeOf(typedNil).Elem()
typeRegistry[t.Name()] = t
}
func init() {
registerType((*playlistIDAggregation)(nil))
registerType((*srcIDAggregation)(nil))
registerType((*assetIDAggregation)(nil))
}
func makeInstance(name string) interface{} {
return reflect.New(typeRegistry[name]).Elem().Interface()
}
I then want to use my dynamically generated struct as the target for the JSON unmarshalling of the Aggregations node in my ES response:
playlistIDAgg := makeInstance("playlistIDAggregation")
err = json.Unmarshal(esResponse.Aggregations, &playlistIDAgg)
This isn't working like I want it to, as the Unmarshal is trying to unmarshall into an empty interface instead of the underlying struct type. it's putting the data under "data" nodes in the playlistIDAgg variable, and those data fields are of course map[string]interface{}. Am I just missing the way to type assert my playlistIDAgg interface or is there a better way to do this?
EDIT---
The questions in the comments made me realize an edit to this question was long overdue.
In my particular case, the structs I defined, to bind to my Bucket aggregations returned by Elasticsearch, have a similar structure and vary only by their root JSON tag, which ES uses to name the aggregation and strongly type it. E.g.
type <name>Aggregation struct {
Agg BucketAggregationWithCamIDCardinality `json:"<name>"`
}
So, rather than a type registry, my particular problem could be solved by dynamically setting the JSON tag on the struct based on the particular use case.
In addition, a heavier but more robust option would be to leverage Oliver Eilhard's Elasticsearch Go client lib, called Elastic, which has built-in support for all the ES aggregation response structures:
https://github.com/olivere/elastic/
I changed makeInstance function by getting address of element and add target structs with exposed fields.
func makeInstance(name string) interface{} {
return reflect.New(typeRegistry[name]).Elem().Addr().Interface()
}
Here is the working code
package main
import (
"encoding/json"
"fmt"
"reflect"
)
type playlistIDAggregation struct {
PlaylistID string
}
type srcIDAggregation struct {
SrcID string
}
type assetIDAggregation struct {
AssetID string
}
var typeRegistry = make(map[string]reflect.Type)
func registerType(typedNil interface{}) {
t := reflect.TypeOf(typedNil).Elem()
typeRegistry[t.Name()] = t
}
func init() {
registerType((*playlistIDAggregation)(nil))
registerType((*srcIDAggregation)(nil))
registerType((*assetIDAggregation)(nil))
}
func makeInstance(name string) interface{} {
return reflect.New(typeRegistry[name]).Elem().Addr().Interface()
}
func main() {
playlistIDAgg := makeInstance("playlistIDAggregation")
fmt.Printf("Type = %[1]T => %#[1]v\n", playlistIDAgg)
err := json.Unmarshal([]byte(`{"PlayListID": "dummy-id"}`), &playlistIDAgg)
if err != nil {
panic(err)
}
fmt.Printf("Type = %[1]T => %#[1]v\n", playlistIDAgg)
}
https://play.golang.org/p/dn19_iG5Xjz
I have an issue printing the JSON from the POST.
I use gorilla/mux for routing
r := mux.NewRouter()
r.HandleFunc("/test", Point).Methods("POST")
http.ListenAndServe(":80", r)`
and in Point function I have
func Point(w http.ResponseWriter, r *http.Request) {
var callback Decoder
json.NewDecoder(r.Body).Decode(&callback)
}
But I can use this method only when I know the structure and I want to figure out how to log.Print the whole JSON as a string. I tried
func Point(w http.ResponseWriter, r *http.Request) {
r.ParseForm()
log.Println(r.Form)
}
But it prints an empty map. Please help to figure out this.
In the assumption that you are building a standard API endpoint which receives some JSON and you would like to do something with it you should approach it the following way.
Edit:
As mentioned in the comments when you use the ioutil.ReadAll()
function as in this example it will read everything that is sent in the
post request into the application memory. It's a good idea to check
this in a production application (e.g limiting payload size).
1.) Make a struct which will hold the data coming from an API post request in GoLang
2.) Convert your request body into a byte array []byte
3.) Unmarshal your []byte into a single instance of your struct made earlier.
I'll put an example below:
1. Make a struct which you'll put your JSON into.
Let's take an example of a simple blog post.
The JSON object looks like this and has a slug, a title, and description
{
"slug": "test-slug",
"title": "This is the title",
"body": "This is the body of the page"
}
So your struct would look like this:
type Page struct {
Slug string `json:"slug"`
Title string `json:"title"`
Body string `json:"body"`
}
2 - 3. Get the body of your request and convert it to byte[] Then take that string and Unmarshal it into an instance of your struct.
The data of a post request is the request 'Body'.
In Golang the request in almost all cases (unless you're using something fancy outside of the defaults) will be an http.Request object. This is the 'r' which you normally have in your normal code and it holds the 'Body' of our POST request.
import (
"encoding/json"
"github.com/go-chi/chi" // you can remove
"github.com/go-chi/render" // you can remove but be sure to remove where it is used as well below.
"io/ioutil"
"net/http"
)
func GiveMeAPage(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("A page"))
}
So what we are going to do here is convert an io.ReadCloser, which is what the http.Request.Body is, to a []byte as the Unmarshal function takes a []byte type. I've commented inline below for you.
func Create(w http.ResponseWriter, r *http.Request) {
var p Page //Create an instance of our struct
//Read all the data in r.Body from a byte[], convert it to a string, and assign store it in 's'.
s, err := ioutil.ReadAll(r.Body)
if err != nil {
panic(err) // This would normally be a normal Error http response but I've put this here so it's easy for you to test.
}
// use the built in Unmarshal function to put the string we got above into the empty page we created at the top. Notice the &p. The & is important, if you don't understand it go and do the 'Tour of Go' again.
err = json.Unmarshal(s, &p)
if err != nil {
panic(err) // This would normally be a normal Error http response but I've put this here so it's easy for you to test.
}
// From here you have a proper Page object which is filled. Do what you want with it.
render.JSON(w, r, p) // This is me using a useful helper function from go-chi which 'Marshals' a struct to a json string and returns it to using the http.ResponseWriter.
}
As a side note. Please don't use Decoder to parse JSON unless you are using JSON streams. You are not here, and it's unlikely you will
for a while. You can read about why that is
here
If you just want the raw JSON data without parsing it, http.Request.Body implements io.Reader, so you can just Read from it. For example with ioutil.ReadAll.
Something like (untested):
func Point(w http.ResponseWriter, r *http.Request) {
data, err := ioutil.ReadAll(r.Body)
// check error
// do whatever you want with the raw data (you can `json.Unmarshal` it too)
}
This question already has an answer here:
JSON Unmarshal struct case-sensitively
(1 answer)
Closed 3 years ago.
Golang does not support case sensitive unmarshalling of JSON using the standard packages. This seems like a very common need though.
Is there any way to get a precise matching of cases when marshalling and unmarshalling JSON?
Example: I don't want to match "id" with "ID".
The standard library encoding/json does in fact support case sensitive encoding/decoding if your data structures define the members that way and include tags appropriately.
For example:
package main
import (
"encoding/json"
"fmt"
)
type image struct {
Url string `json:"url"`
}
type images struct {
Image1 image `json:"image"`
Image2 image `json:"Image"`
}
func main() {
i := images{Image1: image{Url: "test.jpg"}, Image2: image{Url: "test2.jpg"}}
data, err := json.Marshal(i)
if err != nil {
fmt.Printf("error: %s", err)
}
fmt.Println(string(data))
var i2 images
err = json.Unmarshal(data, &i2)
if err != nil {
fmt.Printf("error: %s", err)
}
fmt.Printf("%#v\n", i2)
}
https://play.golang.org/p/GWUWYUc-T9t
Which will output:
{"image":{"url":"test.jpg"},"Image":{"url":"test2.jpg"}}
main.images{Image1:main.image{Url:"test.jpg"}, Image2:main.image{Url:"test2.jpg"}}
However, another good json encoding package is jsonparser: https://github.com/buger/jsonparser
I don't know if it supports case sensativity differentaly then the standard library package.
I am facing a rather simple problem in Go since I am totally new to it. I want to fetch and print data from a REST api. the code I have written:
package main
import (
_"bytes"
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
)
type Headers struct {
Headers HttpHeaders `json:"headers"`
}
type HttpHeaders struct {
Accept string
Accept_Encoding string `json:"Accept-Encoding"`
Accept_Language string `json:"Accept-Language"`
Connection string
Cookie string
Host string
Referer string
Upgrade_Insecure_Requests bool `json:"Upgrade-Insecure-Requests"`
User_Agent string `json:"User-Agent"`
}
func main() {
url := "http://httpbin.org/headers"
res, err := http.Get(url)
if err != nil {
panic(err)
}
defer res.Body.Close()
body, err := ioutil.ReadAll(res.Body)
if err != nil {
panic(err)
}
var head Headers
err = json.Unmarshal(body, &head)
if err != nil {
panic(err)
}
fmt.Printf("%+v\n", head)
}
After the adjustments I made, the response looks like:
{Headers:{Accept: Accept_Encoding:gzip Accept_Language: Connection:close Cookie: Host:httpbin.org Referer: Upgrade_Insecure_Requests:false User_Agent:}}
I have still some fields without a value and the value in Upgrade_Insecure_requests does not seem to match the one that is returned via the API.
2nd edit:
Removed the spaces in the tags. The response now still looks not good.
{Headers:{Accept: Accept_Encoding:gzip Accept_Language: Connection:close Cookie: Host:httpbin.org Referer: Upgrade_Insecure_Requests:false User_Agent:Go-http-client/1.1}}
The Upgrade_insecure_Requests is still 0 instead of 1 and the other fields are still blank.
Change your last line to
fmt.Printf("%+v\n", head)
to see all of your fields. As you can see all of strings are empty. It's because in Headers struct headers field is not exported (first letter must be capital) and no data will be unmarshaled into it. Also some fields name don't match the data in http://httpbin.org/headers.
For example change Accept_Encoding field in struct as follows:
Accept_Encoding string `json:"Accept-Encoding"`
to read from Accept-Encoding instead of Accept_Encoding.
A couple of issues here, first you must export the "HttpHeaders" field of the "Headers" struct, and second, you must tag the fields with JSON names if their JSON key isn't the same (case insensitive):
type Headers struct {
// The "Headers" member must have a capital "H" so that the
// JSON marshaler knows that it can be de/serialized.
Headers HttpHeaders
}
type HttpHeaders {
Accept string
// Since the JSON key is the same as the field name we don't need a tag.
Accept_Encoding string `json:"Accept-Encoding"`
// Here we need a tag since the member name cannot contain a dash.
// ...
}
Note that the "Headers" field of the "Headers" struct doesn't need a JSON tag because the unmarshaler ignores capitalization when matching JSON document keys to go struct field names. That is, the "Header" field would be populated by the JSON key "header", or "Header", or even "HEADER".
This question already has answers here:
JSON and dealing with unexported fields
(2 answers)
(un)marshalling json golang not working
(3 answers)
json.Marshal(struct) returns "{}"
(3 answers)
Printing Empty Json as a result [duplicate]
(1 answer)
Parsing JSON in Golang doesn't Populate Object [duplicate]
(1 answer)
Closed 10 months ago.
I'm a new Go programmer (From Java) and I would like to reproduce a generic way which is esay to use in Java.
I want to create some function which allow me to do an Unmarshal on a JSON string in order to avoid code duplicity.
This is my current code which is not working :
type myStruct1 struct {
id string
name string
}
func (obj myStruct1) toString() string {
var result bytes.Buffer
result.WriteString("id : ")
result.WriteString(obj.id)
result.WriteString("\n")
result.WriteString("name : ")
result.WriteString(obj.name)
return result.String()
}
func main() {
content := `{id:"id1",name="myName"}`
object := myStruct1{}
parseJSON(content, object)
fmt.Println(object.toString())
}
func parseJSON(content string, object interface{}) {
var parsed interface{}
json.Unmarshal([]byte(content), &parsed)
}
This code, on run, returns me this :
id :
name :
Do you have any idea ?
Thanks
The issue is you want to write to a generic type? You probably want a string map. This works with BSON anyways:
var anyJson map[string]interface{}
json.Unmarshal(bytes, &anyJson)
You'll be able to access the fields like so:
anyJson["id"].(string)
Don't forget to type assert your values, and they must be the correct type or they'll panic. (You can read more about type assertions on the golang site)
To parse "generic JSON" when you have no idea what schema it has:
var parsed any
err := json.Unmarshal(jsonText, &parsed)
The returned any in parsed will be a map[string]any or []any or nil or single values float64, bool, string.
You can test the type and react accordingly.
import (
"encoding/json"
"fmt"
)
func test(jsonText []byte) {
// parsing
var parsed any
err := json.Unmarshal(jsonText, &parsed)
if err != nil {
panic(err) // malformed input
}
// type-specific logic
switch val := parsed.(type) {
case nil:
fmt.Println("json specifies null")
case map[string]any:
fmt.Printf("id:%s name:%s\n", val["id"], val["name"])
case []any:
fmt.Printf("list of %d items\n", len(val))
case float64:
fmt.Printf("single number %f\n", val)
case bool:
fmt.Printf("single bool %v\n", val)
case string:
fmt.Printf("single string %s\n", val)
default:
panic(fmt.Errorf("type %T unexpected", parsed))
}
}
Unmarshal will only set exported fields of the struct.
Which means you need to modify the json struct to use capital case letters:
type myStruct1 struct {
Id string
Name string
}
The reason behind this is that the json library does not have the ability to view fields using reflect unless they are exported.
You have to export your fields:
type myStruct1 struct {
Id string
Name string
}
See Exported Identifiers from documentation.
There are a few changes you need to make in your code to make it work:
The function json.Unmarshal can only set variables inside your struct which are exported, that is, which start with capital letters. Use something like ID and Name for your variable names inside myStruct1.
Your content is invalid JSON. What you actually want is {"ID":"id1","Name":"myName"}.
You're passing object to parseJSON but you're using parsed instead, not object. Make parseJSON receive a *myStruct (instead of an interface{}), and use that variable instead of parsed when unmarshalling the string. Also, always handle the error returns, like err := json.Unmarshal(content, object), and check err.
I'd suggest you to do the Golang Tour ;)
You can also set the file as an Object with dynamic properties inside another struct. This will let you add metadata and you read it the same way.
type MyFile struct {
Version string
Data map[string]interface{}
}