Convert flattened json into nested json - json

Currently I'm using the following code to convert nested json into flattened json:
import (
"fmt"
"github.com/nytlabs/gojsonexplode"
)
func main() {
input := `{"person":{"name":"Joe", "address":{"street":"123 Main St."}}}`
out, err := gojsonexplode.Explodejsonstr(input, ".")
if err != nil {
// handle error
}
fmt.Println(out)
}
This is the output: {"person.address.street":"123 Main St.","person.name":"Joe"}
After some processing, now I want to restore this data into normal nested json, but I'm unable to do so.
My closest guess is usage of nested maps, but I don't know how to create nested map with N levels.
EDIT: Reason why I need this: I'm storing data in Redis, and if I store json into Redis then I can't search for keys, that's why I convert keys into key1:key2:key3: some_value

In order to "unflatten" the data you need to split each of the keys at the dot and create nested objects. Here is an example with your data on the Go Playground.
func unflatten(flat map[string]interface{}) (map[string]interface{}, error) {
unflat := map[string]interface{}{}
for key, value := range flat {
keyParts := strings.Split(key, ".")
// Walk the keys until we get to a leaf node.
m := unflat
for i, k := range keyParts[:len(keyParts)-1] {
v, exists := m[k]
if !exists {
newMap := map[string]interface{}{}
m[k] = newMap
m = newMap
continue
}
innerMap, ok := v.(map[string]interface{})
if !ok {
return nil, fmt.Errorf("key=%v is not an object", strings.Join(keyParts[0:i+1], "."))
}
m = innerMap
}
leafKey := keyParts[len(keyParts)-1]
if _, exists := m[leafKey]; exists {
return nil, fmt.Errorf("key=%v already exists", key)
}
m[keyParts[len(keyParts)-1]] = value
}
return unflat, nil
}

json.MarshalIndent is your friend.
j, err := json.MarshalIndent(x, "", "\t")
if err != nil {
log.Println(err)
}
log.Println(string(j))
It will print your data (x) in indented manner.

Related

golang : how to convert type interface {} to array

I have a JSON array with objects in Redis that I want to loop through it, but when I fetch the data, the type is interface{}, so I cannot range over type interface{}
array := redis.Do(ctx, "JSON.GET", "key")
arrayResult, e := array.Result()
if e != nil {
log.Printf("could not get json with command %s", e)
}
for _, i := range arrayResult {
fmt.Printf(i)
}
I believe you should be able to do
for _, i := range arrayResult.([]byte) {
// do work here
}
Thank you guys, I found a solution. so at first I needed to convert the arrayResult to byte. Then I unmarshal it into a strcut, so now I am able to range over it.
array := redis.Do(ctx, "JSON.GET", "key")
arrayResult, e := array.Result()
if e != nil {
log.Printf("could not get json with command %s", e)
}
byteKey := []byte(fmt.Sprintf("%v", arrayResult.(interface{})))
RedisResult := struct{}
errUnmarshalRedisResult := json.Unmarshal(byteKey, &RedisResult)
if errUnmarshalRedisResult != nil {
log.Printf("cannot Unmarshal msg %s", errUnmarshalRedisResult)
}
for _, i := range RedisResult {
fmt.Printf(i)
}

merge two map[string]interface{} from json

I have two json inputs built this way
"count: 1 result: fields"
I would like to concatenate the fields that I find within result without using a defined structure. I have tried in many ways but most of the time the result is an error about the type Interface {} or the last map overwritten the data
I would like both the "result" and the first and second map fields to be merged within the result in output.
oracle, err := http.Get("http://XXX:8080/XXXX/"+id)
if err != nil {
panic(err)
}
defer oracle.Body.Close()
mysql, err := http.Get("http://XXX:3000/XXX/"+id)
if err != nil {
panic(err)
}
defer mysql.Body.Close()
oracleJSON, err := ioutil.ReadAll(oracle.Body)
if err != nil {
panic(err)
}
mysqlJSON, err := ioutil.ReadAll(mysql.Body)
if err != nil {
panic(err)
}
var oracleOUT map[string]interface{}
var mysqlOUT map[string]interface{}
json.Unmarshal(oracleJSON, &oracleOUT)
json.Unmarshal(mysqlJSON, &mysqlOUT)
a := oracleOUT["result"]
b := mysqlOUT["result"]
c.JSON(http.StatusOK, gin.H{"result": ????})
this is an example of json
{"count":1,"result":{"COD_DIPENDENTE":"00060636","MATRICOLA":"60636","COGNOME":"PIPPO"}}
If i have two json like this the result of the function it should be
`"result":{"COD_DIPENDENTE":"00060636","MATRICOLA":"60636","COGNOME":"PIPPO","COD_DIPENDENTE":"00060636","MATRICOLA":"60636","COGNOME":"PIPPO"}}`
The output you are looking for is not valid JSON. However with a small change you can output something very similar to your example that is valid JSON.
You probably do want to use a defined structure for the portion of the input that has a known structure, so that you can extract the more abstract "result" section more easily.
If you start at the top of the input structure using a map[string]interface{} then you'll have to do a type assertion on the "result" key. For example:
var input map[string]interface{}
err = json.Unmarshal(data, &input)
if err != nil {
return err
}
keys, ok := input["result"].(map[string]interface{})
if !ok {
return errors.New("wasn't the type we expected")
}
However if you used a defined structure for the top level you can do it like the following which feels much cleaner.
type Input struct {
Count int `json:"count"`
Result map[string]interface{} `json:"result"`
}
var input Input
err = json.Unmarshal(data, &input)
if err != nil {
return err
}
// from here you can use input.Result directly without a type assertion
To generate output that has duplicate keys, you could use an array of objects with a single key/value pair in each, then you end up with a valid JSON structure that does not overwrite keys. Here's how to do that (playground link):
package main
import (
"encoding/json"
"fmt"
)
type Input struct {
Count int `json:"count"`
Result map[string]interface{} `json:"result"`
}
type Output struct {
Count int `json:"count"`
Result []map[string]interface{} `json:"result"`
}
var inputdata = [][]byte{
[]byte(`{"count":1,"result":{"COD_DIPENDENTE":"00060636", "MATRICOLA":"60636", "COGNOME":"PIPPO"}}`),
[]byte(`{"count":1,"result":{"COD_DIPENDENTE":"00060636", "MATRICOLA":"60636", "COGNOME":"PIPPO"}}`),
}
func main() {
inputs := make([]Input, len(inputdata))
for i := range inputs {
err := json.Unmarshal(inputdata[i], &inputs[i])
if err != nil {
panic(err)
}
}
var out Output
out.Count = len(inputs)
for _, input := range inputs {
for k, v := range input.Result {
out.Result = append(out.Result, map[string]interface{}{k: v})
}
}
outdata, _ := json.Marshal(out)
fmt.Println(string(outdata))
}
Which produces output that looks like this when formatted:
{
"count": 2,
"result": [
{"MATRICOLA": "60636"},
{"COGNOME": "PIPPO"},
{"COD_DIPENDENTE": "00060636"},
{"COGNOME": "PIPPO"},
{"COD_DIPENDENTE": "00060636"},
{"MATRICOLA": "60636"}
]
}

Cannot convert string map to json

I'd like to make a json out of a hash received from redis using redigo:
func showHashtags(c *gin.Context) {
hashMap, err := redis.StringMap(conn.Do("HGETALL", MyDict))
if err != nil {
fmt.Println(err)
}
fmt.Println(hashMap) //works fine and shows the map
m := make(map[string]string)
for k, v := range hashMap {
m[k] = v
}
jmap, _ := json.Marshal(m)
c.JSON(200, jmap)
}
However the result in browser is gibberish like:
"eyIgIjoiMiIsIjExX9iq24zYsSAiOiIxIiwiQWxsNFJhbWluICI6IjEiLCJCSUhFICI6IjMiLCJCVFNBUk1ZICI6IjIiLCJDTUJZTiAiOiIxI....
What is wrong here? How can I fix it?
The variable jmap is type []byte. The call to JSON encoder in c.JSON() marshals []byte as a base64 encoded string as you see in the output.
To fix the problem, use one level of JSON encoding by passing the map directly to c.JSON:
hashMap, err := redis.StringMap(conn.Do("HGETALL", MyDict))
if err != nil {
// handle error
}
m := make(map[string]string)
for k, v := range hashMap {
m[k] = v
}
c.JSON(200, m)
Because hashMap is a map[string]string, you can use it directly:
hashMap, err := redis.StringMap(conn.Do("HGETALL", MyDict))
if err != nil {
// handle error
}
c.JSON(200, hashMap)

invalid operation: type interface {} does not support indexing

I'm new to the golang and I have problem while reading the nested JSON response.
var d interface{}
json.NewDecoder(response.Body).Decode(&d)
test :=d["data"].(map[string]interface{})["type"]
response.Body looks like this
{
"links": {
"self": "/domains/test.one"
},
"data": {
"type": "domains",
"id": "test.one",
"attributes": {
"product": " Website",
"package": "Professional",
"created_at": "2016-08-19T11:37:01Z"
}
}
}
The Error I'm getting is this:
invalid operation: d["data"] (type interface {} does not support indexing)
d is of type interface{}, so you cannot index it like d["data"], you need another type assertion:
test := d.(map[string]interface{})["data"].(map[string]interface{})["type"]
fmt.Println(test)
Then it will work. Output will be "domains". See a working example on the Go Playground.
Also note that if you declare d to be of type map[string]interface{}, you can spare the first type assertion:
var d map[string]interface{}
if err := json.NewDecoder(response.Body).Decode(&d); err != nil {
panic(err)
}
test := d["data"].(map[string]interface{})["type"]
fmt.Println(test)
Output is the same. Try this one on the Go Playground.
If you need to do these and similar operations many times, you may find my github.com/icza/dyno library useful (whose primary goal is to aid working with dynamic objects).
You need some tricks to handle your situation.
Like using reflect and you may ref Marshall&&UnMarshall code about bson.M in golang mongo driver mgo
code sample using reflect decode nested as following:
package main
import (
"encoding/json"
"fmt"
"reflect"
)
func main() {
keys := []string{"hello", "world", "dude", "kind", "cool"}
a := make(map[string]interface{})
a[keys[4]] = "perfect"
b := make(map[string]interface{})
b[keys[3]] = a
c := make(map[string]interface{})
c[keys[2]] = b
d := make(map[string]interface{})
d[keys[1]] = c
e := make(map[string]interface{})
e[keys[0]] = d
fmt.Println(e)
if buf, err := json.Marshal(e); nil == err {
dd := make(map[string]interface{})
err = json.Unmarshal(buf, &dd)
if nil != err {
fmt.Println("failed", err)
}
for k, v := range dd {
travel(dd, k, v)
}
fmt.Println(dd)
} else {
fmt.Println("failed marshal")
}
}
func travel(dd map[string]interface{}, key string, value interface{}) {
vv := reflect.ValueOf(value)
switch vv.Kind() {
case reflect.Map:
m := value.(map[string]interface{})
dd[key] = m
for k, v := range m {
travel(m, k, v)
}
case reflect.String:
dd[key] = value.(string)
}
}
You can do this also:
var inputJson string = "..."
var decoded map[string]map[string]interface{}
json.Unmarshal([]byte(inputJson), &decoded)
test := decoded["data"]["type"]
fmt.Println(test)
// output: domains

JSON single value parsing

In python you can take a json object and grab a specific item from it without declaring a struct, saving to a struct then obtaining the value like in Go. Is there a package or easier way to store a specific value from json in Go?
python
res = res.json()
return res['results'][0]
Go
type Quotes struct {
AskPrice string `json:"ask_price"`
}
quote := new(Quotes)
errJson := json.Unmarshal(content, &quote)
if errJson != nil {
return "nil", fmt.Errorf("cannot read json body: %v", errJson)
}
You can decode into a map[string]interface{} and then get the element by key.
func main() {
b := []byte(`{"ask_price":"1.0"}`)
data := make(map[string]interface{})
err := json.Unmarshal(b, &data)
if err != nil {
panic(err)
}
if price, ok := data["ask_price"].(string); ok {
fmt.Println(price)
} else {
panic("wrong type")
}
}
Structs are often preferred as they are more explicit about the type. You only have to declare the fields in the JSON you care about, and you don't need to type assert the values as you would with a map (encoding/json handles that implicitly).
Try either fastjson or jsonparser. jsonparser is optimized for the case when a single JSON field must be selected, while fastjson is optimized for the case when multiple unrelated JSON fields must be selected.
Below is an example code for fastjson:
var p fastjson.Parser
v, err := p.Parse(content)
if err != nil {
log.Fatal(err)
}
// obtain v["ask_price"] as float64
price := v.GetFloat64("ask_price")
// obtain v["results"][0] as generic JSON value
result0 := v.Get("results", "0")