How to parse JSON string to struct - json

I have struct of Request, value is optional:
type Request struct {
Operation string `json:"operation"`
Key string `json:"key"`
Value string `json:"value"`
}
And function that should parse json string to struct^
go func() {
s := string("{'operation': 'get', 'key': 'example'}")
data := Request{}
json.Unmarshal([]byte(s), data)
log.Printf("Operation: %s", data.Operation)
}
For some reason data.Operation is empty. What is wrong here?

Two problems, first, your json is invalid, it needs to use " instead of '
Second, you have to unmarshal into &data and not to data
https://play.golang.org/p/zdMq5_ex8G
package main
import (
"fmt"
"encoding/json"
)
type Request struct {
Operation string `json:"operation"`
Key string `json:"key"`
Value string `json:"value"`
}
func main() {
s := string(`{"operation": "get", "key": "example"}`)
data := Request{}
json.Unmarshal([]byte(s), &data)
fmt.Printf("Operation: %s", data.Operation)
}
Side note, you would have seen this, if you would have been checking your errors:
s := string("{'operation': 'get', 'key': 'example'}")
err := json.Unmarshal([]byte(s), data)
if err != nil {
fmt.Println(err.Error())
//json: Unmarshal(non-pointer main.Request)
}
err := json.Unmarshal([]byte(s), &data)
if err != nil {
fmt.Println(err.Error())
//invalid character '\'' looking for beginning of object key string
}

Related

How to write unit test failure for json.NewDecoder.Decode?

I have to write unit tests for a function and this function uses json.NewDecoder.Decode
var infos models.RegisterInfos // struct with json fields
err := json.NewDecoder(r.Body).Decode(&infos)
if err != nil {
// do something
}
How can I simulate an error in a unit test (using the testing package) for json.NewDecoder(r.Body).Decode(&infos) ? I tried looking in the NewDecoder and Decode source code but I couldn't find anything that can generate an error in just a few lines.
you could send a body like <invalid json> as example:
func main() {
body := "<invalid json>"
var infos RegisterInfos // struct with json fields
err := json.NewDecoder(strings.NewReader(body)).Decode(&infos)
if err != nil {
fmt.Println(err)
}
}
See https://go.dev/play/p/44E99D0eQou
Feed it an invalid input, or decode into an invalid output:
package main
import (
"encoding/json"
"fmt"
"strings"
"testing"
)
type Message struct {
Name string
}
func TestDecodeFail(t *testing.T) {
for _, tc := range []struct {
in string
desc string
out any
}{
{`{Name: "Bobby"}`, "key without quotes", &Message{}},
{`{"Name": "Foo"a}`, "extra character", &Message{}},
{`{"Name": "Foo"}`, "bad destination", &struct{ Name int64 }{}},
{`{"Name": "Foo` + "\u001a" + `"}`, "invalid character", &Message{}},
{`{"Name": "Foo"}`, "unmarshal to nil", (*Message)(nil)},
} {
err := decode(tc.in, tc.out)
if err != nil {
fmt.Printf("%s -> %s, %T\n", tc.desc, err.Error(), err)
}
}
}
func decode(in string, out any) error {
return json.NewDecoder(strings.NewReader(in)).Decode(out)
}
Outputs:
key without quotes -> invalid character 'N' looking for beginning of object key string, *json.SyntaxError
extra character -> invalid character 'a' after object key:value pair, *json.SyntaxError
bad destination -> json: cannot unmarshal string into Go struct field .Name of type int64, *json.UnmarshalTypeError
invalid character -> invalid character '\x1a' in string literal, *json.SyntaxError
unmarshal to nil -> json: Unmarshal(nil *main.Message), *json.InvalidUnmarshalError

Unmarshaling JSON in golang

i'm having a lot of trouble getting my program to work. I want to unmarshal something pretty simple, but it's giving me a lot of issues, unfortunately.
Here is the response that I want to unmarshal:
{"error":[],"result":{"XXBTZUSD":[[1647365820,"39192.0","39192.0","39191.9","39191.9","39191.9","0.18008008",10],[1647365880,"39186.1","39186.1","39172.0","39176.0","39174.4","0.13120077",10]],"last":1647408900}}
I've wrote these structs to help with unmarshalling
type Resp struct {
Error []string `json:"error"`
Result Trades `json:"result"`
}
type Trades struct {
Pair []OHLC `json:"XXBTZUSD"`
Last float64 `json:"last"`
}
type OHLC struct {
Time float64
Open string
High string
Low string
Close string
Vwa string
Volume string
Count float64
}
I have a function call that makes the http request and then unmarshals the data. For whatever reason, my code will end before even starting the function call for the http request and subsequent unmarshalling when the Pair type is []OHLC or []*OHLC. If I change the Pair type to interface{}, then it runs. i want to make it work with the OHLC struct instead though. Below is the complete code:
package main
import (
"fmt"
"net/http"
//"strings"
"io/ioutil"
"encoding/json"
)
type Resp struct {
Error []string `json:"error"`
Result Trades `json:"result"`
}
type Trades struct {
Pair []OHLC `json:"XXBTZUSD"`
Last float64 `json:"last"`
}
type OHLC struct {
TT float64
Open string
High string
Low string
Close string
Vwap string
Volume string
Count float64
}
/*func main() {
var data = [...]Trade{
Trade{5, "op", "hi", "lo", "cl", "vw", "vo", 2},
Trade{5, "op", "hi", "lo", "cl", "vw", "vo", 2},
}
}*/
func main() {
fmt.Println("in main");
getOhlc()
}
func getOhlc() {
fmt.Println("in ohlc func")
resp, err := http.Get("https://api.kraken.com/0/public/OHLC?pair=XXBTZUSD");
if err != nil {
fmt.Errorf("error after request")
return;
}
defer resp.Body.Close();
body, err := ioutil.ReadAll(resp.Body);
if err != nil {
fmt.Errorf("error when reading")
return;
}
var jsonData Resp;
err = json.Unmarshal(body, &jsonData);
if err != nil {
fmt.Errorf("error when unmarshalling")
return
}
if(len(jsonData.Error) > 0) {
fmt.Errorf("error");
return;
}
fmt.Println(jsonData);
}
Any ideas about what might be happening?
"Any ideas about what might be happening?"
The elements in the "XXBTZUSD" JSON array are arrays themselves, i.e. "XXBTZUSD" is an array of arrays. The OHLC type is a struct type. The stdlib will not, by itself, unmarshal a JSON array into a Go struct. Go structs can be used to unmarshal JSON objects. JSON arrays can be unmarshaled into Go slices or arrays.
You would clearly see that that's the issue if you would just print the error from json.Unmarshal:
json: cannot unmarshal array into Go struct field
Trades.result.XXBTZUSD of type main.OHLC
https://go.dev/play/p/D4tjXZVzDI_w
If you want to unmarshal a JSON array into a Go struct you have to have the Go struct type implement a the json.Unmarshaler interface.
func (o *OHLC) UnmarshalJSON(data []byte) error {
// first unmarshal the array into a slice of raw json
raw := []json.RawMessage{}
if err := json.Unmarshal(data, &raw); err != nil {
return err
}
// create a function that unmarshals each raw json element into a field
unmarshalFields := func(raw []json.RawMessage, fields ...interface{}) error {
if len(raw) != len(fields) {
return errors.New("bad number of elements in json array")
}
for i := range raw {
if err := json.Unmarshal([]byte(raw[i]), fields[i]); err != nil {
return err
}
}
return nil
}
// call the function
return unmarshalFields(
raw,
&o.Time,
&o.Open,
&o.High,
&o.Low,
&o.Close,
&o.Vwa,
&o.Volume,
&o.Count,
)
}
https://go.dev/play/p/fkFKLkaNaSU
Your code had some issues:
Remove semicolons from end of lines, it's redundant.
fmt.Errorf return error, and not print it, every time check your error and propagate it.
We can convert array of numbers and string to struct in golang.
for achieving your desired output we need to first convert to intermediate container and then convert to our wanted output:
package main
import (
"errors"
"fmt"
"log"
"net/http"
//"strings"
"encoding/json"
"io/ioutil"
)
type Resp struct {
Error []string `json:"error"`
Result Trades `json:"result"`
}
type IntermediateResp struct {
Error []string `json:"error"`
Result IntermediateTrades `json:"result"`
}
type IntermediateTrades struct {
Pair [][]interface{} `json:"XXBTZUSD"`
Last int `json:"last"`
}
type Trades struct {
Pair []OHLC `json:"result"`
Last int `json:"last"`
}
type OHLC struct {
TT float64
Open string
High string
Low string
Close string
Vwap string
Volume string
Count float64
}
/*func main() {
var data = [...]Trade{
Trade{5, "op", "hi", "lo", "cl", "vw", "vo", 2},
Trade{5, "op", "hi", "lo", "cl", "vw", "vo", 2},
}
}*/
func main() {
fmt.Println("in main")
err := getOhlc()
if err != nil {
log.Fatal(err)
}
}
func buildOHLC(l []interface{}) (*OHLC, error) {
if len(l) < 8 {
return nil, errors.New("short list")
}
return &OHLC{
TT: l[0].(float64),
Open: l[1].(string),
High: l[2].(string),
Low: l[3].(string),
Close: l[4].(string),
Vwap: l[5].(string),
Volume: l[6].(string),
Count: l[7].(float64),
}, nil
}
func convert(r IntermediateResp) (*Resp, error) {
result := &Resp{Error: r.Error, Result: Trades{Pair: make([]OHLC, len(r.Result.Pair)), Last: r.Result.Last}}
for i, v := range r.Result.Pair {
ohlc, err := buildOHLC(v)
if err != nil {
return nil, err
}
result.Result.Pair[i] = *ohlc
}
return result, nil
}
func getOhlc() error {
fmt.Println("in ohlc func")
resp, err := http.Get("https://api.kraken.com/0/public/OHLC?pair=XXBTZUSD")
if err != nil {
return fmt.Errorf("error after request, %v", err)
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
fmt.Println(string(body))
if err != nil {
return fmt.Errorf("error when reading %v", err)
}
var jsonData IntermediateResp
err = json.Unmarshal(body, &jsonData)
if err != nil {
return fmt.Errorf("error when unmarshalling %v", err)
}
if len(jsonData.Error) > 0 {
return fmt.Errorf("error")
}
convertedOhlc, err := convert(jsonData)
if err != nil {
return fmt.Errorf("error when convertedOhlc %v", err)
}
fmt.Println(convertedOhlc)
return nil
}
We define IntermediateResp and IntermediateTrades for Unmarshaling json and then convert it to actual Resp.
I think aother way is using custom Unmarshal for Trades struct.

Unmarshalling a single element in an array

Is there a way to unmarshall JSON arrays into single objects in Go?
I have a json response from an endpoint:
{
"results": [
{
"key": "value"
}
]
}
I have a Go struct for the object inside the array:
type Object struct {
Key string `json:"key"`
}
...and a struct for the response object:
type Response struct {
Objects []Object `json:"results"`
}
results is an array of objects, but I know that the endpoint will only ever return an array with 1 object. Is there a way to unmarshall the data and avoid having reference the object by an index? I was hoping I could use something like json:"results[0]" as a field tag.
I'd prefer to be able to:
decoder.Decode(&response)
response.Object.Key
Rather than
decoder.Decode(&response)
response.Objects[0].Key
To does this you need to customize unmarshalling.
An way is create a ResponseCustom like:
//Response json (customized) that match with Unmarshaler interface
type ResponseCustom struct {
Object
}
func (r *ResponseCustom) UnmarshalJSON(b []byte) error{
rsp := &Response{}
err := json.Unmarshal(b, rsp)
if err != nil {
log.Fatalln("error:", err)
} else {
r.Object = rsp.Objects[0]
}
//
return err
}
So you can use ResponseCustom instead of you Response for get Object value.
Look:
func main() {
//
data := []byte(jsondata)
resp := &ResponseCustom{}
//
json.Unmarshal(data, resp)
//
fmt.Println("My Object.value is: " + resp.Object.Key)
}
The result is:
My Object.value is: value
In playground: https://play.golang.org/p/zo7wOSacA4w
Implement unmarshaler interface to convert array of object to object. Fetch the value for key and then unmarshal your json as
package main
import(
"fmt"
"encoding/json"
)
type Object struct {
Key string `json:"key"`
}
func (obj *Object) UnmarshalJSON(data []byte) error {
var v map[string]interface{}
if err := json.Unmarshal(data, &v); err != nil {
fmt.Printf("Error whilde decoding %v\n", err)
return err
}
obj.Key = v["results"].(interface{}).([]interface{})[0].(interface{}).(map[string]interface{})["key"].(interface{}).(string)
return nil
}
func main(){
var obj Object
data := []byte(`{
"results": [
{
"key": "value"
}
]
}`)
if err := json.Unmarshal(data, &obj); err!=nil{
fmt.Println(err)
}
fmt.Printf("%#v", obj)
}
Playground

Want to send empty json string "{}" instead of "null"

I'm trying to handle http requests in my go api, in which i wish to send empty json string like
{"result": {}, "status": "failed"}
but when the mysql query returns zero rows, it return an output as
{"result": null, "status": "failed"}
Edit : This is the response struct i'm using:
type Resp struct {
Result []map[string]interface{} `json:"result"`
Status string `json:"status"`
}
How can i handle this situation?
The Result field is a slice, which can be nil. This is rendered as null in JSON. To make it not nil, you will have to initialise it.
In addition, since Result is a slice, it will be marshalled to a JSON array ([]), not a JSON object ({}).
Example:
package main
import (
"encoding/json"
"fmt"
"log"
"os"
)
type Resp struct {
Result []map[string]interface{} `json:"result"`
Status string `json:"status"`
}
func main() {
enc := json.NewEncoder(os.Stdout)
fmt.Println("Empty Resp struct:")
if err := enc.Encode(Resp{}); err != nil {
log.Fatal(err)
}
fmt.Println()
fmt.Println("Initialised Result field:")
if err := enc.Encode(Resp{Result: []map[string]interface{}{}}); err != nil {
log.Fatal(err)
}
}
Output:
Empty Resp struct:
{"result":null,"status":""}
Initialised Result field:
{"result":[],"status":""}
https://play.golang.org/p/9zmfH-180Zk

go "encoding/json" : marshal json field

I have a PostgreSQL schema with json field's (DisplayInfo, and FormatInfo). Structure of this field's is dynamic.
I'can read and render it only as string (string type in render struct) :
[
{
"ID":9,
"Name":"120 №1",
"DisplayInfo":"{\"path\": \"http://path/to/img.png\"}",
"Format":{
"Code":"frame-120",
"Width":120,
"Height":60,
"FormatInfo":"[{\"name\": \"\\u0413\\u043b\\u0430\\u0432\\u043d\\u043e\\u0435 \\u0438\\u0437\\u043e\\u0431\\u0440\\u0430\\u0436\\u0435\\u043d\\u0438\\u0435\", \"field_type\": \"img\", \"key\": \"path\"}]"
},
"Weight":0.075,
"Application":8,
"Url":"//path/to/game",
"Referrer":""
}
]
but i want output field DisplayInfo as JSON object. How ?
My render code:
func renderJSON(w http.ResponseWriter, obj models.Model) {
js, err := json.Marshal(obj)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
w.Header().Set("Content-Type", "application/json; charset=utf-8")
w.Header().Set("Access-Control-Allow-Origin", "*")
w.Write(js)
}
UPD 1 : Structure of this field's is dynamic. DisplayInfo may have 'path' field, or may not. They may have additional fields.
UPD 2. I wana output DisplayInfo and FormatInfo as json-object(not string), as part of whole object, like this:
[
{
"ID":9,
"Name":"120 №1",
"DisplayInfo":{"path": "http://path/to/img.png"},
"Format":{
"Code":"frame-120",
"Width":120,
"Height":60,
"FormatInfo":[{"name": "\\u0413\\u043b\\u0430\\u0432\\u043d\\u043e\\u0435 \\u0438\\u0437\\u043e\\u0431\\u0440\\u0430\\u0436\\u0435\\u043d\\u0438\\u0435", "field_type": "img", "key": "path"}]
},
"Weight":0.075,
"Application":8,
"Url":"//path/to/game",
"Referrer":""
}
]
UPD 3: Structures
Actual structure is :
type BannerSerializer struct {
ID int
Name string
DisplayInfo string
Format formatSerializer
Weight float32
Application int
Url string
Referrer string
}
Then i trying this structure:
type BannerSerializer struct {
ID int
Name string
DisplayInfo json.RawMessage
Format formatSerializer
Weight float32
Application int
Url string
Referrer string
}
DisplayInfo serialize as base64 string (or like base64, don't know)
Use a pointer to json.RawMessage:
type Data struct {
Obj *json.RawMessage
}
Playground: http://play.golang.org/p/Qq9IUBDLzJ.
Assuming you have access to change models.Model, you can create your own type with a custom Unmarshaler that just returns the raw string:
type JSONString string
func (s JSONString) MarshalJSON() ([]byte, error) {
return []byte(s), nil
}
Working example:
package main
import (
"encoding/json"
"fmt"
)
type JSONString string
func (s JSONString) MarshalJSON() ([]byte, error) {
return []byte(s), nil
}
type Model struct {
ID int
Name string
DisplayInfo JSONString
}
func main() {
data := []byte(`{
"ID":9,
"Name":"120 №1",
"DisplayInfo":"{\"path\": \"http://path/to/img.png\"}"
}`)
var obj Model
err := json.Unmarshal(data, &obj)
if err != nil {
panic(err)
}
// Here comes your code
js, err := json.Marshal(obj)
if err != nil {
panic(err)
}
fmt.Println(string(js))
}
Output:
{"ID":9,"Name":"120 №1","DisplayInfo":{"path":"http://path/to/img.png"}}
Playground: http://play.golang.org/p/6bcnuGjlU8
You'd have to unmarshal it, here's an example:
var data []*struct {
ID int
DisplayInfo string
}
if err := json.Unmarshal([]byte(j), &data); err != nil {
log.Fatal(err)
}
for _, d := range data {
var displayInfo struct{ Path string }
if err := json.Unmarshal([]byte(d.DisplayInfo), &displayInfo); err != nil {
log.Fatal(err)
}
fmt.Println(d.ID, displayInfo.Path)
}
playground