I have following code:
func policyDocumentToStr(doc map[string]interface{}) (*string, error) {
policy, err := json.Marshal(doc)
if err != nil {
log.Debugf("Error converting policy document to string. Error %s", err)
return nil, err
}
policyAsString := string(policy)
return &policyAsString, nil
}
I want to write a unit test which would cover the case of json.Marshal(doc) returning an error. Can anybody suggest how can I generate an error? What kind of input to function would result in error at line policy, err := json.Marshal(doc)?
Feed it a value that cannot be represented in JSON. There are many ways to do this, including creating a type with a custom marshaler that always returns an error. But one of the simplest ways is to try to marshal a channel:
x := map[string]interface{}{
"foo": make(chan int),
}
_, err := json.Marshal(x)
fmt.Printf("Marshal error: %s\n", err)
Playground link
Define a type that implements json.Marshaler. Use this type to produce any error you want (including error values from the json package):
type FakeValue struct {
err error
}
func (v FakeValue) MarshalJSON() ([]byte, error) {
if v.err != nil {
return nil, v.err
}
return []byte(`null`), v.err
}
func TestPolicyString(t *testing.T) {
doc := map[string]interface{}{
"fake_error": FakeValue{errors.New("fail!")},
}
_, err := policyDocumentToStr(doc)
if err == nil {
t.Fatal("Got nil, want error")
}
}
Monkey patching is one of the ways to facilitate testing by changing a program at runtime, either by replacing a function or a variable. Here we are swapping the json.Marshal and we can mock it while testing.
var (
jsonMarshal = json.Marshal
)
This is your piece of code
func policyDocumentToStr(doc map[string]interface{}) (*string, error) {
policy, err := jsonMarshal(doc)
if err != nil {
fmt.Printf("Error converting policy document to string. Error %s", err)
return nil, err
}
policyAsString := string(policy)
return &policyAsString, nil
}
The test code would look something like this
import (
"testing"
"errors"
)
func fakemarshal(v interface{}) ([]byte, error) {
return []byte{}, errors.New("Marshalling failed")
}
func restoremarshal(replace func(v interface{}) ([]byte, error)) {
jsonMarshal = replace
}
func TestPolicyDocumentToStr(t * testing.T){
storedMarshal := jsonMarshal
jsonMarshal = fakemarshal
defer restoremarshal(storedMarshal)
input := map[string]interface{} {
"test": "test1",
}
tests := []struct {
name string
arg map[string]interface{}
wantErr string
}{
{
name: "Test if JSON Marshalling fails",
arg: input,
wantErr: "Marshalling failed",
},
}
for _, tt := range tests {
_, gotErr := policyDocumentToStr(tt.arg)
if gotErr != nil && gotErr.Error() != tt.wantErr {
t.Errorf("Expected %s but got %s", tt.wantErr, gotErr.Error())
}
}
}
Related
For the following JSON response {"table_contents":[{"id":100,"description":"text100"},{"id":101,"description":"text101"},{"id":1,"description":"text1"}]}
All you have to do is to produce the following code to execute it properly and be able to reads fields from the struct, such as:
package main
import (
"fmt"
"encoding/json"
)
type MyStruct1 struct {
TableContents []struct {
ID int
Description string
} `json:"table_contents"`
}
func main() {
result:= []byte(`{"table_contents":[{"id":100,"description":"text100"},{"id":101,"description":"text101"},{"id":1,"description":"text1"}]}`)
var container MyStruct1
err := json.Unmarshal(result, &container)
if err != nil {
fmt.Println(" [0] Error message: " + err.Error())
return
}
for i := range container.TableContents {
fmt.Println(container.TableContents[i].Description)
}
}
But how do you deal with the following JSON response? {"table_contents":[[{"id":100,"description":"text100"},{"id":101,"description":"text101"}],{"id":1,"description":"text1"}]} You can either get this response or the one above, it is important to modify the struct to accept both.
I did something like this, with the help of internet:
package main
import (
"fmt"
"encoding/json"
)
type MyStruct1 struct {
TableContents []TableContentUnion `json:"table_contents"`
}
type TableContentClass struct {
ID int
Description string
}
type TableContentUnion struct {
TableContentClass *TableContentClass
TableContentClassArray []TableContentClass
}
func main() {
result:= []byte(`{"table_contents":[[{"id":100,"description":"text100"},{"id":101,"description":"text101"}],{"id":1,"description":"text1"}]}`)
var container MyStruct1
err := json.Unmarshal(result, &container)
if err != nil {
fmt.Println(" [0] Error message: " + err.Error())
return
}
for i := range container.TableContents {
fmt.Println(container.TableContents[i])
}
}
but it does not go past the error message :(
[0] Error message: json: cannot unmarshal array into Go struct field MyStruct1.table_contents of type main.TableContentUnion*
Been struggling to come up with a solution for hours. If someone could help I would be happy. Thank you for reading. Let me know if you have questions
Inside table_contents you have two type options (json object or list of json objects). What you can do is to unmarshall into an interface and then run type-check on it when using it:
type MyStruct1 struct {
TableContents []interface{} `json:"table_contents"`
}
...
for i := range container.TableContents {
switch container.TableContents[i].(type){
case map[string]interface{}:
fmt.Println("json object")
case []interface{}:
fmt.Println("list")
}
}
From there you can use some library (e.g. https://github.com/mitchellh/mapstructure) to map unmarshalled struct to your TableContentClass type. See PoC playground here: https://play.golang.org/p/NhVUhQayeL_C
Custom UnmarshalJSON function
You can also create a custom UnmarshalJSON function on the object that has the 2 possibilities. In you case that would be TableContentUnion.
In the custom unmarshaller you can then decide how to unmarshal the content.
func (s *TableContentUnion) UnmarshalJSON(b []byte) error {
// Note that we get `b` as bytes, so we can also manually check to see
// if it is an array (starts with `[`) or an object (starts with `{`)
var jsonObj interface{}
if err := json.Unmarshal(b, &jsonObj); err != nil {
return err
}
switch jsonObj.(type) {
case map[string]interface{}:
// Note: instead of using json.Unmarshal again, we could also cast the interface
// and build the values as in the example above
var tableContentClass TableContentClass
if err := json.Unmarshal(b, &tableContentClass); err != nil {
return err
}
s.TableContentClass = &tableContentClass
case []interface{}:
// Note: instead of using json.Unmarshal again, we could also cast the interface
// and build the values as in the example above
if err := json.Unmarshal(b, &s.TableContentClassArray); err != nil {
return err
}
default:
return errors.New("TableContentUnion.UnmarshalJSON: unknown content type")
}
return nil
}
The rest then works like in your test code that was failing before. Here the working Go Playground
Unmarshal to map and manually build struct
You can always unmarshal a json (with an object at the root) into a map[string]interface{}. Then you can iterate things and further unmarshal them after checking what type they are.
Working example:
func main() {
result := []byte(`{"table_contents":[[{"id":100,"description":"text100"},{"id":101,"description":"text101"}],{"id":1,"description":"text1"}]}`)
var jsonMap map[string]interface{}
err := json.Unmarshal(result, &jsonMap)
if err != nil {
fmt.Println(" [0] Error message: " + err.Error())
return
}
cts, ok := jsonMap["table_contents"].([]interface{})
if !ok {
// Note: nil or missing 'table_contents" will also lead to this path.
fmt.Println("table_contents is not a slice")
return
}
var unions []TableContentUnion
for _, content := range cts {
var union TableContentUnion
if contents, ok := content.([]interface{}); ok {
for _, content := range contents {
contCls := parseContentClass(content)
if contCls == nil {
continue
}
union.TableContentClassArray = append(union.TableContentClassArray, *contCls)
}
} else {
contCls := parseContentClass(content)
union.TableContentClass = contCls
}
unions = append(unions, union)
}
container := MyStruct1{
TableContents: unions,
}
for i := range container.TableContents {
fmt.Println(container.TableContents[i])
}
}
func parseContentClass(value interface{}) *TableContentClass {
m, ok := value.(map[string]interface{})
if !ok {
return nil
}
return &TableContentClass{
ID: int(m["id"].(float64)),
Description: m["description"].(string),
}
}
This is most useful if the json has too many variations. For cases like this it might also make sense sometimes to switch to a json package that works differently like https://github.com/tidwall/gjson which gets values based on their path.
Use json.RawMessage to capture the varying parts of the JSON document. Unmarshal each raw message as appropriate.
func (ms *MyStruct1) UnmarshalJSON(data []byte) error {
// Declare new type with same base type as MyStruct1.
// This breaks recursion in call to json.Unmarshal below.
type x MyStruct1
v := struct {
*x
// Override TableContents field with raw message.
TableContents []json.RawMessage `json:"table_contents"`
}{
// Unmarshal all but TableContents directly to the
// receiver.
x: (*x)(ms),
}
err := json.Unmarshal(data, &v)
if err != nil {
return err
}
// Unmarahal raw elements as appropriate.
for _, tcData := range v.TableContents {
if bytes.HasPrefix(tcData, []byte{'{'}) {
var v TableContentClass
if err := json.Unmarshal(tcData, &v); err != nil {
return err
}
ms.TableContents = append(ms.TableContents, v)
} else {
var v []TableContentClass
if err := json.Unmarshal(tcData, &v); err != nil {
return err
}
ms.TableContents = append(ms.TableContents, v)
}
}
return nil
}
Use it like this:
var container MyStruct1
err := json.Unmarshal(result, &container)
if err != nil {
// handle error
}
Run it on the Go playground.
This approach does not add any outside dependencies. The function code does not need to modified when fields are added or removed from MyStruct1 or TableContentClass.
I'm looking for a way to unmarshal a JSON body without having to specify targets for all fields. And then be a able to "remarshal" the body with implicit fields untouched.
Something like this would be good, but doesn't work as a expected: (https://play.golang.org/p/fnVOKrmiFj)
package main
import (
"encoding/json"
"fmt"
)
type Transaction struct {
Field1 string `json:"field1"`
X map[string]interface{} `json:"-"`
}
func main() {
body := []byte(`{"field1": "value1", "field2": "value2"}`)
fmt.Printf("%+v\n", string(body))
var unmarshalledTransaction Transaction
json.Unmarshal(body, &unmarshalledTransaction)
fmt.Printf("%+v\n", unmarshalledTransaction)
remarshalledTransaction, _ := json.Marshal(&unmarshalledTransaction)
fmt.Printf("%+v\n", string(remarshalledTransaction))
}
Gives the output
{"field1": "value1", "field2": "value2"}
{Field1:value1 X:map[]}
{"field1":"value1"}
My expected result would be that unmarshalledTransaction contains the "leftover" fields in the X fields. And they are then restored when Marshalling again.
Can this be done?
You would need to implement the MarshalJSON and UnmarshalJSON interfaces, and write your own logic to remap the fields to the appropriate spots:
func (t *Transaction) MarshalJSON() ([]byte, error) {
data := t.X
data["field1"] = t.Field1
return json.Marshal(data)
}
func (t *Transaction) UnmarshalJSON(data []byte) error {
m := make(map[string]interface{})
json.Unmarshal(data, &m)
t.Field1 = m["field1"].(string)
delete(m, "field1")
t.X = m
return nil
}
https://play.golang.org/p/KBGAsXB0xA
If you want a generic solution (that would work with any struct without knowing the fields in advance), you can implement a function that would un-marshal the body into a struct and also return the "leftover" fields.
For that you'd also need to implement a function that would convert any given struct to a map (to be used to then manipulate maps in a generic way instead of known-in-advance structs).
Like so:
func structToMap(object interface{}) (map[string]interface{}, error) {
tempJson, err := json.Marshal(object)
if err != nil {
return nil, err
}
var theMap map[string]interface{}
err = json.Unmarshal(tempJson, &theMap)
if err != nil {
return nil, err
}
return theMap, nil
}
And then:
func unmarshalWithLeftovers(jsonBody []byte, target interface{}) (map[string]interface{}, error) {
err := json.Unmarshal(jsonBody, target)
if err != nil {
return nil, err
}
structMap, err := structToMap(target)
if err != nil {
return nil, err
}
var leftOvers map[string]interface{}
err = json.Unmarshal(jsonBody, &leftOvers)
if err != nil {
return nil, err
}
for k, _ := range structMap {
delete(leftOvers, k)
}
return leftOvers, nil
}
You can then combine the struct and the leftovers map in a similar fashion to re-marshal everything.
See here a working example with the same type and json string that you used in your question:
https://play.golang.org/p/Fot6YVurHH
func good(json) string {
\\do something
err = json.Unmarshal(body, &list)
if err != nil {
panic(fmt.Sprintf("Unable to parse json %s",err))
}
}
func Testgood_PanicStatement(t *testing.T) {
Convey("And Invalid Json return error",t, func() {
actual := good("garbage json")
So(func() {},shoulPanic)
So(actual ,ShouldEqual,"")
}
}
Outcome
Line 34: - Unable to parse json ,{%!e(string=invalid character '{' looking for beginning of object key string) %!e(int64=50)}
goroutine 8 [running]:
Question:It seems like when I am passing garbage json file.It is panicking and doesn't execute any of the So statements?How to fix it?
Use recover().
func Testgood_PanicStatement(t *testing.T) {
Convey("And Invalid Json return error",t, func() {
defer func() {
if r := recover(); r != nil {
So(func() {},shouldPanic)
So(actual ,ShouldEqual,"")
}
}()
actual := good("garbage json")
}
}
Lear more about:
Golang blog
Upvoting the answer of sadlil as correct I want to point out, that panicking functions are not good practice. Rather convert the panic into an error INSIDE the function and test the error instead.
func good(json) (s string, err error) {
defer func() {
if r := recover(); r != nil {
err = fmt.Errorf("Error in good: %v", r)
}
}()
\\do something
err = json.Unmarshal(body, &list)
if err != nil {
# leaving the panic statement for demonstration. In this case
# you could just: return "", err
panic(fmt.Sprintf("Unable to parse json %s",err))
}
return
}
I got the following custom type:
type TimeWithoutZone struct {
time.Time
}
The Marshaling works fine:
const timeWithoutZoneFormat = "2006-01-02T15:04:05"
func (t *TimeWithoutZone) MarshalJSON() ([]byte, error) {
stamp := fmt.Sprintf(`"%s"`, t.Time.Format(timeWithoutZoneFormat ))
return []byte(stamp), nil
}
But here the date can not be parsed:
func (t *TimeWithoutZone) UnmarshalJSON(data []byte) (err error) {
log.Println("Parsing: " + string(data))
t.Time, err = time.Parse(`"` + timeWithoutZoneFormat + `"`, string(data))
if err != nil {
return err
}
return nil
}
It logs: Parsing: {"time":"2016-09-06T11:06:16"} but I would expect it to parse just the value of time
What am I doing wrong? here is the related test:
type TimeTestObj struct {
Time TimeWithoutZone `json:"time"`
}
func TestParseDataWithoutTimezone(t *testing.T) {
parsed := TimeWithoutZone{}
data := `{"time":"2016-09-06T11:06:16"}`
err := json.Unmarshal([]byte(data), &parsed)
if err != nil {
t.Error(err)
}
if parsed.Unix() != 1473152776 {
t.Error(parsed.Unix(), "!=", 1473152776)
}
}
All the examples I find, and even the default parser from the Go time package seem to work that way...
Wow I just have the wrong type in this line:
parsed := TimeWithoutZone{}
must be
parsed := TimeTestObj{}
...
Basically, what I want to achieve is to get the content of a directory via os.ReadDir() and then encode the result into json.
Directly doing json.Marshal() cause no exception but gave me an empty result.
So I tried this:
func (f *os.FileInfo) MarshalerJSON() ([]byte, error) {
return f.Name(), nil
}
Then Go tells me that os.FileInfo() is an interface and cannot be extended this way.
What's the best way to do this?
Pack the data into a struct that can be serialized:
http://play.golang.org/p/qDeg2bfik_
type FileInfo struct {
Name string
Size int64
Mode os.FileMode
ModTime time.Time
IsDir bool
}
func main() {
dir, err := os.Open(".")
if err != nil {
log.Fatal(err)
}
entries, err := dir.Readdir(0)
if err != nil {
log.Fatal(err)
}
list := []FileInfo{}
for _, entry := range entries {
f := FileInfo{
Name: entry.Name(),
Size: entry.Size(),
Mode: entry.Mode(),
ModTime: entry.ModTime(),
IsDir: entry.IsDir(),
}
list = append(list, f)
}
output, err := json.Marshal(list)
if err != nil {
log.Fatal(err)
}
log.Println(string(output))
}
Here is a different version that makes the usage seemingly simple. Though you cannot unmarshall it back to the object. This is only applicable if you are sending it to client-end or something.
http://play.golang.org/p/vmm3temCUn
Usage
output, err := json.Marshal(FileInfo{entry})
output, err := json.Marshal(FileInfoList{entries})
Code
type FileInfo struct {
os.FileInfo
}
func (f FileInfo) MarshalJSON() ([]byte, error) {
return json.Marshal(map[string]interface{}{
"Name": f.Name(),
"Size": f.Size(),
"Mode": f.Mode(),
"ModTime": f.ModTime(),
"IsDir": f.IsDir(),
})
}
type FileInfoList struct {
fileInfoList []os.FileInfo
}
//This is inefficient to call multiple times for the same struct
func (f FileInfoList) MarshalJSON() ([]byte, error) {
fileInfoList := make([]FileInfo, 0, len(f.fileInfoList))
for _, val := range f.fileInfoList {
fileInfoList = append(fileInfoList, FileInfo{val})
}
return json.Marshal(fileInfoList)
}