Is it possible in Go, given the below function, to unmarshall jsonString without knowing the type of c at runtime?
func findChargedItems(fs financialService, conditions []string) ([]*models.ChargedItem, error) {
var jsonResult []string
f := getChargedItemsQuery(conditions)
q, _, _ := f.ToSql()
err := fs.db.Select(&jsonResult, q)
if err != nil {
return nil, err
}
jsonString := fmt.Sprintf("[%v]", strings.Join(jsonResult, ","))
c := make([]*models.ChargedItem, 0)
err = json.Unmarshal([]byte(jsonString), &c)
if err != nil {
return nil, err
}
return c, nil
}
The problem is that I have tons of models that need to do this exact process and I'm repeating myself in order to achieve this. I would like to just have a "generic" function findEntities that operates agnostic of ChargedItem and getChargedItemsQuery. I realize I can just pass a function in for getChargeditemsQuery so that takes care of the that problem, but I am having issues with json.Unmarshal, for instance, when I try to use an interface, the json fields do not map correctly. Is there a way to achieve what I'm trying to do without affecting the data models?
I'm not sure what you're trying to do, but it's probably not a good idea. At any rate, this should do what you are wanting:
package main
import (
"encoding/json"
"fmt"
)
func main() {
// do what youre trying to do
var (
a = []byte("[10, 11]")
b []interface{}
)
json.Unmarshal(a, &b)
// then fix it later
c := make([]float64, len(b))
for n := range c {
c[n] = b[n].(float64)
}
fmt.Println(c)
}
Related
I am trying to read the following JSON file:
{
"a":1,
"b":2,
"c":3
}
I have tried this but I found that I had to write each field of the JSON file into a struct but I really don't want to have all my JSON file in my Go code.
import (
"fmt"
"encoding/json"
"io/ioutil"
)
type Data struct {
A string `json:"a"`
B string `json:"b"`
C string `json:"c"`
}
func main() {
file, _ := ioutil.ReadFile("/path/to/file.json")
data := Data{}
if err := json.Unmarshal(file ,&data); err != nil {
panic(err)
}
for _, letter := range data.Letter {
fmt.Println(letter)
}
}
Is there a way to bypass this thing with something like json.load(file) in Python?
If you only want to support integer values, you could unmarshal your data into a map[string]int. Note that the order of a map is not defined, so the below program's output is non-deterministic for the input.
package main
import (
"fmt"
"encoding/json"
"io/ioutil"
)
func main() {
file, _ := ioutil.ReadFile("/path/to/file.json")
var data map[string]int
if err := json.Unmarshal(file ,&data); err != nil {
panic(err)
}
for letter := range data {
fmt.Println(letter)
}
}
You can unmarshal any JSON data in this way:
var data interface{}
if err := json.Unmarshal(..., &data); err != nil {
// handle error
}
Though, in this way you should handle all the reflection-related stuffs
since you don't know what type the root data is, and its fields.
Even worse, your data might not be map at all.
It can be any valid JSON data type like array, string, integer, etc.
Here's a playground link: https://play.golang.org/p/DiceOv4sATO
It's impossible to do anything as simple as in Python, because Go is strictly typed, so it's necessary to pass your target into the unmarshal function.
What you've written could otherwise be shortened, slightly, to something like this:
func UnmarshalJSONFile(path string, i interface{}) error {
f, err := os.Open(path)
if err != nil {
return err
}
defer f.Close()
return json.NewDecoder(f).Decode(i)
}
But then to use it, you would do this:
func main() {
data := Data{}
if err := UnmarshalJSONFile("/path/to/file.json", &data); err != nil {
panic(err)
}
}
But you can see that the UnmarshalJSONFile is so simple, it hardly warrants a standard library function.
Currently I'm using the following code to convert nested json into flattened json:
import (
"fmt"
"github.com/nytlabs/gojsonexplode"
)
func main() {
input := `{"person":{"name":"Joe", "address":{"street":"123 Main St."}}}`
out, err := gojsonexplode.Explodejsonstr(input, ".")
if err != nil {
// handle error
}
fmt.Println(out)
}
This is the output: {"person.address.street":"123 Main St.","person.name":"Joe"}
After some processing, now I want to restore this data into normal nested json, but I'm unable to do so.
My closest guess is usage of nested maps, but I don't know how to create nested map with N levels.
EDIT: Reason why I need this: I'm storing data in Redis, and if I store json into Redis then I can't search for keys, that's why I convert keys into key1:key2:key3: some_value
In order to "unflatten" the data you need to split each of the keys at the dot and create nested objects. Here is an example with your data on the Go Playground.
func unflatten(flat map[string]interface{}) (map[string]interface{}, error) {
unflat := map[string]interface{}{}
for key, value := range flat {
keyParts := strings.Split(key, ".")
// Walk the keys until we get to a leaf node.
m := unflat
for i, k := range keyParts[:len(keyParts)-1] {
v, exists := m[k]
if !exists {
newMap := map[string]interface{}{}
m[k] = newMap
m = newMap
continue
}
innerMap, ok := v.(map[string]interface{})
if !ok {
return nil, fmt.Errorf("key=%v is not an object", strings.Join(keyParts[0:i+1], "."))
}
m = innerMap
}
leafKey := keyParts[len(keyParts)-1]
if _, exists := m[leafKey]; exists {
return nil, fmt.Errorf("key=%v already exists", key)
}
m[keyParts[len(keyParts)-1]] = value
}
return unflat, nil
}
json.MarshalIndent is your friend.
j, err := json.MarshalIndent(x, "", "\t")
if err != nil {
log.Println(err)
}
log.Println(string(j))
It will print your data (x) in indented manner.
Here is the code
package main
import (
"fmt"
"encoding/json"
"reflect"
)
var (
datajson []byte
//ref mapp
)
type mapp map[string]reflect.Type
type User struct {
Name string
//Type map[string]reflect.Type
}
func MustJSONEncode(i interface{}) []byte {
result, err := json.Marshal(i)
if err != nil {
panic(err)
}
return result
}
func MustJSONDecode(b []byte, i interface{}) {
err := json.Unmarshal(b, i)
if err != nil {
panic(err)
}
}
func Store(a interface{}) {
datajson = MustJSONEncode(a)
//fmt.Println(datajson)
}
func Get(a []byte, b interface{}) {
objType := reflect.TypeOf(b).Elem()
obj := reflect.New(objType)
//fmt.Println(obj)
MustJSONDecode(a, &obj)
fmt.Printf("%s", obj)
}
func main() {
dummy := &User{}
david := User{Name: "DavidMahon"}
Store(david)
Get(datajson, dummy)
}
In the Get function
func Get(a []byte, b interface{}) {
objType := reflect.TypeOf(b).Elem()
obj := reflect.New(objType)
//fmt.Println(obj)
MustJSONDecode(a, &obj)
fmt.Printf("%s", obj)
}
I am unable to unmarshal the json into the underlying object type.
Whats wrong here? I am so stuck here. Something very simple yet so difficult to figure out.
Thanks
UPDATE::Goal of this problem is to retreive a fully formed object of type passed in Get function.
The approach mentioned by Nick on the comment below doesnot get me the actual object which I already tried before. I can anyways retrieve the data (even when the object has recursive objects underneath) in a map like this
func Get(a []byte) {
var f interface{}
//buf := bytes.NewBuffer(a)
//v := buf.String()
//usr := &User{}
MustJSONDecode(a, &f)
fmt.Printf("\n %v \n", f)
}
However I need the actual object back not just the data. Something like user := &User{"SomeName"} where I need user object back from Unmarshall. The trick is somewhere in reflection but dont know how.
I'm confused as to why you want to do this, but here is how to fix it
func Get(a []byte, b interface{}) {
objType := reflect.TypeOf(b).Elem()
obj := reflect.New(objType).Interface()
//fmt.Println(obj)
MustJSONDecode(a, &obj)
fmt.Printf("obj = %#v\n", obj)
}
Note the call to Interface().
Playground link
It seems to me that you are going to a lot of trouble to make an empty &User when you already have one in b, eg
func Get(a []byte, b interface{}) {
MustJSONDecode(a, &b)
fmt.Printf("obj = %#v\n", b)
}
But I'm guessing there is some more to this plan which isn't apparent here!
reflect.New(objType) returns a reflect.Value Which is not the thing as the interface you passed. According to the docs for Value It is a struct with only unexported fields. the json package can't work with unexported fields. Since it's not the same object as you passed in and it's not even json encodable/decodable the json package will fail.
You will probably find the Laws of Reflection article useful while trying to use the reflect package.
I'm processing a json POST in Go that contains an array of objects containing 64bit integers. When using json.Unmarshal these values seem to be converted to a float64 which isn't very helpful.
body := []byte(`{"tags":[{"id":4418489049307132905},{"id":4418489049307132906}]}`)
var dat map[string]interface{}
if err := json.Unmarshal(body, &dat); err != nil {
panic(err)
}
tags := dat["tags"].([]interface{})
for i, tag := range tags {
fmt.Println("tag: ", i, " id: ", tag.(map[string]interface{})["id"].(int64))
}
Is there any way to preserve the original int64 in the output of json.Unmarshal?
Go Playground of above code
Solution 1
You can use a Decoder and UseNumber to decode your numbers without loss :
The Number type is defined like this :
// A Number represents a JSON number literal.
type Number string
which means you can easily convert it :
package main
import (
"encoding/json"
"fmt"
"bytes"
"strconv"
)
func main() {
body := []byte("{\"tags\":[{\"id\":4418489049307132905},{\"id\":4418489049307132906}]}")
dat := make(map[string]interface{})
d := json.NewDecoder(bytes.NewBuffer(body))
d.UseNumber()
if err := d.Decode(&dat); err != nil {
panic(err)
}
tags := dat["tags"].([]interface{})
n := tags[0].(map[string]interface{})["id"].(json.Number)
i64, _ := strconv.ParseUint(string(n), 10, 64)
fmt.Println(i64) // prints 4418489049307132905
}
Solution 2
You can also decode into a specific structure tailored to your needs :
package main
import (
"encoding/json"
"fmt"
)
type A struct {
Tags []map[string]uint64 // "tags"
}
func main() {
body := []byte("{\"tags\":[{\"id\":4418489049307132905},{\"id\":4418489049307132906}]}")
var a A
if err := json.Unmarshal(body, &a); err != nil {
panic(err)
}
fmt.Println(a.Tags[0]["id"]) // logs 4418489049307132905
}
Personally I generally prefer this solution which feels more structured and easier to maintain.
Caution
A small note if you use JSON because your application is partly in JavaScript : JavaScript has no 64 bits integers but only one number type, which is the IEEE754 double precision float. So you wouldn't be able to parse this JSON in JavaScript without loss using the standard parsing function.
easier one:
body := []byte(`{"tags":[{"id":4418489049307132905},{"id":4418489049307132906}]}`)
var dat map[string]interface{}
if err := json.Unmarshal(body, &dat); err != nil {
panic(err)
}
tags := dat["tags"].([]interface{})
for i, tag := range tags {
fmt.Printf("tag: %v, id: %.0f", i, tag.(map[string]interface{})["id"].(float64))
}
I realize this is very old, but this is the solution I ended up using
/*
skipping previous code, this is just converting the float
to an int, if the value is the same with or without what's
after the decimal points
*/
f := tag.(map[string]interface{})["id"].(float64)
if math.Floor(f) == f {
fmt.Println("int tag: ", i, " id: ", int64(f))
} else {
fmt.Println("tag: ", i, " id: ", f)
}
In Go, I have some JSON from third-party API and then I'm trying to parse it:
b := []byte(`{"age":21,"married":true}`)
var response_hash map[string]string
_ = json.Unmarshal(b, &response_hash)
But it fails (response_hash becomes empty) and Unmarshal can handle only JSON with quotes:
b := []byte(`{"age":"21","married":"true"}`)
var response_hash map[string]string
_ = json.Unmarshal(b, &response_hash)
Is there any way to read the first version of JSON from third-party API?
Go is a strongly typed language. You need to specify what types the JSON encoder is to expect. You're creating a map of string values but your two json values are an integer and a boolean value.
Here's a working example with a struct.
http://play.golang.org/p/oI1JD1UUhu
package main
import (
"fmt"
"encoding/json"
)
type user struct {
Age int
Married bool
}
func main() {
src_json := []byte(`{"age":21,"married":true}`)
u := user{}
err := json.Unmarshal(src_json, &u)
if err != nil {
panic(err)
}
fmt.Printf("Age: %d\n", u.Age)
fmt.Printf("Married: %v\n", u.Married)
}
If you want to do it in a more dynamic manner you can use a map of interface{} values. You just have to do type assertions when you use the values.
http://play.golang.org/p/zmT3sPimZC
package main
import (
"fmt"
"encoding/json"
)
func main() {
src_json := []byte(`{"age":21,"married":true}`)
// Map of interfaces can receive any value types
u := map[string]interface{}{}
err := json.Unmarshal(src_json, &u)
if err != nil {
panic(err)
}
// Type assert values
// Unmarshal stores "age" as a float even though it's an int.
fmt.Printf("Age: %1.0f\n", u["age"].(float64))
fmt.Printf("Married: %v\n", u["married"].(bool))
}