My original problem is I want to parse URL.Values to a generic type (map[interface{}]interface{}) edit/add some values then convert it to JSON string and put it to PostgreSQL JSON column.
I tried this code to parse it but content seems to be null whereas err is false. request.URL.Query() prints a nice map object.
v := reflect.ValueOf(request.URL.Query())
i := v.Interface()
content, err := i.(map[interface{}]interface{})
// Do some operations
jsonString, _ := json.Marshal(content)
// Add to DB
Why is it null? Also am I thinking too generic?
content, err := i.(map[interface{}]interface{}), this isn't a cast, it's a type assertion. You're saying (asserting) that interface is of type map[interface{}]interface{}, it's not. It's of type map[string][]string. You get null as the value because it fails. I highly doubt error is false.
Are you thinking too generic? Of course you are. I can't think of any reason why the collections type needs to change... Append what you want to it, write it to your db. There's nothing preventing that afaik?
Related
I have a general enough function for going through a map[string] and getting all keys:
i := 0
keys := make([]string, len(input))
for k := range input {
keys[i] = k
i++
}
return keys
My problem is I have two different inputs I want to throw in here, a map[string]MyStruct and map[string][][]float64. Whenever I've tried having the input to the func as map[string]interface{}, go resists all my attempts to try to cast the map[string]MyStruct as a map[string]interface{}. Is there a way I can do this without needing to have two functions, one with map[string]MyStruct as input, and one with map[string][][]float64? The contents of the map[string] don't matter at this point, because I'm just trying to get all the keys of them for use later in the code. This needs to be a function that's called; We're using Sonar, and it's set to refuse code duplication, so I can't have this code snippet duplicated.
Until next Go version brings us generics there're several ways to cope with it.
Duplicate code
Use code generation - design some template and then go on build will fill it for you.
Use interface{} as an input type of a function and then use reflection to guess which type was given to a function.
I'm pretty sure in this case general code will be more complicated than 2 separate functions.
func getKeys(input interface{}) []string {
switch inp := input.(type) {
case map[string]MyStruct:
keys := make([]string, 0, len(inp))
for k := range inp {
keys = append(keys, k)
}
return keys
case map[string][]float64:
...
default:
fmt.Printf("I don't know about type %T!\n", v)
}
You cannot use input type map[string]interface{} to place map[string]string or map[string][]string go won't copy each value to new type. You may take the most general type interface{} and then cast it.
Lets say I have this type:
type Foo struct{
Bar string `json:"bar"`
}
and I want to unmarshal this json into it:
in := []byte(`{"bar":"aaa", "baz":123}`)
foo := &Foo{}
json.Unmarshal(in,foo)
will succeed just fine. I would like to at least know that there were fields that were skipped in the processing. Is there any good way to access that information?
playground snippet
As you're probably aware you can unmarshal any valid json into a map[string]interface{}. Having unmarsahled into an instance of Foo already there is no meta data available where you could check fields that were excluded or anything like that. You could however, unmarshal into both types and then check the map for keys that don't correspond to fields on Foo.
in := []byte(`{"bar":"aaa", "baz":123}`)
foo := &Foo{}
json.Unmarshal(in,foo)
allFields := &map[string]interface{}
json.Unmarshal(in, allFields)
for k, _ := range allFields {
fmt.Println(k)
// could also use reflect to get field names as string from Foo
// the get the symmetric difference by nesting another loop here
// and appending any key that is in allFields but not on Foo to a slice
}
I'm developing a web service and part of that I read the Request.Body and try to unmarshal it.
if err := json.NewDecoder(body).Decode(r); err !=nil{
log.Error(err)
return err
}
The issue is that sometimes the client is sending an empty body and I get a panic runtime error: index out of range
goroutine 7 [running]:
How am I supposed to mitigate this?
I am decomposing your code:
NewDecoder: -
func NewDecoder(r io.Reader) *Decoder
NewDecoder returns a new decoder that reads from r. The decoder
introduces its own buffering and may read data from r beyond the JSON
values requested.
So NewDecoder only reads data from r. it's not care, is r empty...
Decode:-
func (dec *Decoder) Decode(v interface{}) error
Decode reads the next JSON-encoded value from its input and stores it
in the value pointed to by v.
See the documentation for Unmarshal for details about the conversion of JSON into a Go value.
To unmarshal JSON into an interface value, Unmarshal stores one of these in the interface value:
bool, for JSON booleans
float64, for JSON numbers
string, for JSON strings
[]interface{}, for JSON arrays
map[string]interface{}, for JSON objects
nil for JSON null
If a JSON value is not appropriate for a given target type, or if a
JSON number overflows the target type, Unmarshal skips that field and
completes the unmarshalling as best it can. If no more serious errors
are encountered, Unmarshal returns an UnmarshalTypeError describing
the earliest such error.
The JSON null value unmarshals into an interface, map, pointer, or
slice by setting that Go value to nil. Because null is often used in
JSON to mean “not present,” unmarshaling a JSON null into any other Go
type has no effect on the value and produces no error.
Reading above statement, it's clear that there is not chances, we get run time panic error. I was experimenting with a sample code to reproduce this ERROR. May error coming from inside the JSON package or your own code.
var dummy []byte
dummy = make([]byte, 10)
size, _ := body.Read(dummy)
if size > 0 {
if err := json.NewDecoder(body).Decode(r); err != nil {
log.Error(err)
return err
}
fmt.Fprintf(w, "%s", "Json cannot be empty")// where w is http.ResponseWriter
I don't want to specify the type of my json since they are so messy and so complicated, I just want them to load into memory and I perform the lookup when needed.
It is easy with dynamic language such as python, e.g.
data = json.loads(str)
if "foo" in data:
...
How to do the same in go?
You can unmarshal into an interface{} value to decode arbitrary JSON.
Taking the example from http://blog.golang.org/json-and-go
b := []byte(`{"Name":"Wednesday","Age":6,"Parents":["Gomez","Morticia"]}`)
var f interface{}
if err := json.Unmarshal(b, &f); err != nil {
... handle error
}
You need to use a type switch to access data decoded in this way. For example:
age := f.(map[string)interface{})["Age"].(int)
Here's an example which seems easier to understand for me, I hope it works for you too:
https://gobyexample.com/json . Look for the word "arbitrary"
I have a io.Reader which I get from http.Request.Body that reads a JSON byte slice from a server.
I would like to stream this to json.NewDecoder. However I would also like to intercept the JSON before it hits json.NewDecoder and substitute certain parts of it. For example, the JSON string contains empty hashes "{}" which I would like to remove due to a bug in the server's JSON output.
I am currently achieving my goal using json.Unmarshal but not using the JSON streaming parser:
data, _ := ioutil.ReadAll(r.Body)
data = bytes.Replace(data, []byte("{}"), "", -1)
json.Unmarshal(data, [my struct])
How can I achieve the same thing as above but using json.NewDecoder so I can save the many times the above code has to parse through r.Body's data? Here's some code using a pseudo function ReplaceStream(r io.Reader, old, new []byte):
reader := ReplaceStream(r.Body, []byte("{}"), "")
dec := json.NewDecoder(reader)
dec.Decode([my struct])
I know ReplaceStream might be fairly trivial to make, but is there anything in the standard library to do this that I am unaware of?
My advice is to just treat that kind of message as a special case and avoid the extra parsing / substituting for all the other requests
data, _ := ioutil.ReadAll(r.Body)
// FIXME: overcome bug #12312 of json server
if data == `{"list": [{}]}` {
return []
}
// Normal datastruct ..