From last 2 days I'm somehow stuck with JSON and Go. My aim is very simple, one Go program that can read a JSON file, output it correctly and append some items to that JSON and then rewrite it back to disk.
Saved JSON File.
{
"Category": ["food","music"],
"Time(min)": "351",
"Channel": {
"d2d": 10,
"mkbhd": 8,
"coding Train": 24
},
"Info": {
"Date":{
"date":["vid_id1","vid_id2","vid_id3"],
"02/11/2019":["id1","id2","id3"],
"03/11/2019":["SonwZ6MF5BE","8mP5xOg7ijs","sc2ysHjSaXU"]
},
"Videos":{
"videos": ["Title","Category","Channel","length"],
"sc2ysHjSaXU":["Bob Marley - as melhores - so saudade","Music","So Saudade","82"],
"SonwZ6MF5BE":["Golang REST API With Mux","Science & Technology","Traversy Media","44"],
"8mP5xOg7ijs":["Top 15 Funniest Friends Moments","Entertainment","DjLj11","61"]
}
}
}
I have successfully parsed the JSON in Go but then when I try to get JSON["Info"]["Date"] it throws interface error. I can't make a specific struct because all the items will dynamically change whenever the code/API gets called.
The Code that I'm using to parse the data
// Open our jsonFile
jsonFile, err := os.Open("yt.json")
if err != nil {fmt.Println(err)}
fmt.Println("Successfully Opened yt.json")
defer jsonFile.Close()
byteValue, _ := ioutil.ReadAll(jsonFile)
var result map[string]interface{}
json.Unmarshal([]byte(byteValue), &result)
json_data := result["Category"] //returns correct ans
json_data := result["Info"]["Date"] // returns error - type interface {} does not support indexing
Any help/lead is highly appreciated. Thanks a-lot in advance.
Unfortunately, you have to assert types every time you access the parsed data:
date := result["Info"].(map[string]interface{})["Date"]
Now date is map[string]interface{}, but its statically known type is still interface{}.
That means you either need to assume the type in advance or have some kind of a type switch if the structure may vary.
you can't access inner properties with result[][]. You need to do something like follows,
info:= result["Info"]
v := info.(map[string]interface{})
json_data = v["Date"]
Related
I am trying to build a JSON to Excel converter in the Go language. Here is some example JSON:
[{
"App": "Instagram",
"Company": "Facebook",
"Category": "Social Media"
},
{
"App": "WeChat",
"Company": "Tencent",
"Category": "Social Media"
}
}]
I am not using structure here, as Data in JSON can be random, hence using interface to store data, unmarshaling JSON and then using assertion technique tried to extract data, but as data structure used as map, hence while iterating data columns are not stored in sequential manners in excel sheet, with every output columns sequence is random hence data stored inside excel sheet is not appropriate. As data is not fixed type in JSON can not use structure in go program.
With the below code, I am able to get column and set column in Excel as App, Company, Category. But with every iterations sequence of columns is not same, which should be as further step is to feed the value, but as traversing map always gives random output sequence, hence could not move further.
How to handle this part?
file, err := ioutil.ReadFile("test.json")
if err != nil {
panic(err)
}
var data interface{}
json.Unmarshal(file, &data) // reading all json data
fmt.Println(data)
t, ok := data.([]interface{}) // assertion Interface
_ = ok
fmt.Printf("%[1]v %[1]T\n", t) //map[string]interface {}
myMap := t[0] // aim here to get APP, Company and Category which will be the column name of Excel sheet
columnData, _ := myMap.(map[string]interface {}) // extract the underlying concrete data from interface
keys := make([]string, 0, len(columnData)) // creating and initializing slice to store column
for k := range columnData {
fmt.Printf("%[1]v %[1]T\n", k)
keys = append(keys, k)
}
xlsx := excelize.NewFile()
sheetName := "Sheet1"
xlsx.SetSheetName(xlsx.GetSheetName(1), sheetName)
c := 'A'
asciiValue := int(c)
var a string
for i := 0; i < len(keys); i++ {
a = string(asciiValue)
xlsx.SetCellValue(sheetName, a + "1", keys[i])
asciiValue++
}
err := xlsx.SaveAs("./Onkar.xlsx")
if err != nil {
fmt.Println(err)
return
}
fmt.Println("Excel file generated sucessfully")
By looking the code, you are using for :=range, with on maps, does not guarantee order, so since order is important to you i suggest you use, sort the keys.
Here you can find more info.
How to iterate through a map in Golang in order?
In powershell, if I make a REST call and receive any kind of json response, I can easily $json | ConvertFrom-Json into a proper object so I can make modifications, render specific values, whatever.
It seems like in Go I have to either define a struct, or "dynamically" convert using a map[string]interface{}.
The issue with a struct is that I am writing a rest handler for a platform that, depending on the endpoint, serves wildly different JSON responses, like most REST APIs. I don't want to define a struct for all of the dozens of possible responses.
The problem with map[string]interface{} is that it pollutes the data by generating a string with a bunch of ridiculous 'map' prefixes and unwanted [brackets].
ala: [map[current_user_role:admin id:1]]
Is there a way to convert a JSON response like:
{
"current_user_role": "admin",
"id": 1
}
To return a basic:
current_user_role: admin
id: 1
... WITHOUT defining a struct?
Your approach of using a map is right if you don't wish to specify the structure of the data you're receiving. You don't like how it is output from fmt.Println, but I suspect you're confusing the output format with the data representation. Printing them out in the format you find acceptable takes a couple of lines of code, and is not as convenient as in python or powershell, which you may find annoying.
Here's a working example (playground link):
package main
import (
"encoding/json"
"fmt"
"log"
)
var data = []byte(`{
"current_user_role": "admin",
"id": 1
}`)
func main() {
var a map[string]interface{}
if err := json.Unmarshal(data, &a); err != nil {
log.Fatal(err)
}
for k, v := range a {
fmt.Printf("%s: %v\n", k, v)
}
}
I retrieve JSON as GET response from and endpoint
response, _ := http.Get("https://website-returning-json-value.com")
data, _ := ioutil.ReadAll(response.Body)
w.Write(data)
It returns me a JSON value, which is OK, but it is very ugly (no indents etc.). I would like to make it pretty. I've read that there is util function like MarshalIndent which does the job, but this works for JSON object (?) and ReadAll function returns []byte, so it does not work. I read the documentation regarding encoding/json package but there's a lot of information and I got a little bit stuck/confused.
As far as I understand it should be done, I should get []byte via ReadAll function -> convert it to the JSON -> prettify it -> turn to []byte again.
There is json.Indent() for this purpose. Example using it:
src := []byte(`{"foo":"bar","x":1}`)
dst := &bytes.Buffer{}
if err := json.Indent(dst, src, "", " "); err != nil {
panic(err)
}
fmt.Println(dst.String())
Output (try it on the Go Playground):
{
"foo": "bar",
"x": 1
}
But indentation is just for human eyes, it carries the same information, and libraries don't need indented JSON.
Also see: Is there a jq wrapper for golang that can produce human readable JSON output?
I'm trying to decode dynamic/random JSON responses in GO, with nested data
body, _ := ioutil.ReadAll(response.Body)
resp := make(map[string]interface{})
err = json.Unmarshal(body, &resp)
fmt.Printf("BODY: %T<\n", body)
fmt.Printf("BODY: %s<\n", body)
fmt.Printf("RESP: %s<\n", resp)
fmt.Printf("RESP: %T<\n", resp)
fmt.Printf("RESP[results]: %T<\n", resp["results"])
fmt.Printf("RESP[results]: %s<\n", resp["results"])
body is the JSON result from the HTTP server and I unmarshall it and the result looks to be a slice of bytes.
BODY: []uint8
BODY: {"results":[{"code":500.0,"errors":["Configuration file 'c2-web-2.conf' already exists."],"status":"Object could not be created."}]}
So I unmarshall it into resp and that works as expected.
RESP: map[string]interface {}
RESP: map[results:[map[code:%!s(float64=500) errors:[Configuration file 'c2-web-2.conf' already exists.] status:Object could not be created.]]]<
I'm able to access the map with the key results.
RESP[results]: []interface {}
RESP[results]: [map[code:%!s(float64=500) errors:[Configuration file 'conf.d/hosts/c2-web-2.conf' already exists.] status:Object could not be created.]]<
Now what i want to access it the "code", "errors" and "status" which is in resp["results"] This looks like an array or slice and I've tried indexing it but I get the error at compile time
./create_host.go:62: invalid operation: resp["results"][0] (type interface {} does not support indexing)
I've done a lot of googling, tried unmarshalling the data within resp["results"] etc, but after a few days I have not made much progress.
How should I access the map which seems to be a member of an array? The data structure is not guaranteed so I can't create a structure and unmarshall into that.
Thanks
A co-worker provided the code fragement below which made it possible to access the map entries I was looking for.
respBody, _ := ioutil.ReadAll(response.Body)
var rsp interface{}
if err := json.Unmarshal(respBody, &rsp); err != nil {
log.Fatal(err)
}
resultMap := rsp.(map[string]interface{})["results"].([]interface{})[0].(map[string]interface{})
fmt.Printf("test: %s<\n", resultMap["errors"] )
test: [Configuration file 'c2-web-2.conf' already exists.]<
I believe you need to do a type assertion. You have an interface{}, but you need some sort of slice to index into. Try resp["results"].([]interface{})[0]? (Sorry, haven't had a chance to test this myself.)
Hi everyone I'm trying to see what the proper way of accessing fields of a json object from a http.get request in go.
I first do an http.get call get the JSON and then print it (which works) but is there a way to access just a field?
for example:
response, err:= http.Get("URL")
//Error checking is done between
contents, err:=ioutil.Readall(response.Body)
//Now at this point I have a json that looks like
{"id": "someID",
"name": "someName",
"test": [{"Name":"Test1",
"Result": "Success"},
{"Name":"Test2",
"Result": "Success"},
{...},
]}
Is there a way to only print the "test" of the Json? What is the proper way of accessing that field?
Use encoding/json package to Unmarshal data into struct, like following.
type Result struct {
ID string `json:"id"`
Name string `json:"name"`
Test []interface{} `json:"test"`
}
var result Result
json.Unmarshal(contents, &result)
fmt.Println(result.Test)
You can also parse Test to specific struct.
Same as the previous answer, use encoding/json package to Unmarshal data. But if you don't want to specify the structure, you could use map[string]interface/bson.M{} to receive the data, and get the field, then cast into types your want.
m := make(map[string]interface{})
err := json.Unmarshal(data, &m)
if err != nil {
log.Fatal(err)
}
fmt.Println(m["id"])
You may want to try gabs container, if you are not sure how depth JSON hierarchy can be. Have a look at below resources
https://github.com/Jeffail/gabs
https://godoc.org/github.com/Jeffail/gabs
If you just want to access one field then you can use the jsonq module https://godoc.org/github.com/jmoiron/jsonq
For your example you could get the test object with code similar to
jq.Object("test")
Where jq is a jsonq query object constructed from your JSON above (see the godoc page for instructions on how to create a query object from a JSON stream or string).
You can also use this library for retrieving specific String, Integer, Float and Bool values at an arbitrary depth inside a JSON object.
Since you are starting with a URL, Decode is a better option than Unmarshal:
package main
import (
"encoding/json"
"net/http"
)
func main() {
r, e := http.Get("https://github.com/manifest.json")
if e != nil {
panic(e)
}
defer r.Body.Close()
var s struct { Name string }
json.NewDecoder(r.Body).Decode(&s)
println(s.Name == "GitHub")
}
https://golang.org/pkg/encoding/json#Decoder.Decode
You may check this https://github.com/valyala/fastjson
s := []byte(`{"foo": [123, "bar"]}`)
fmt.Printf("foo.0=%d\n", fastjson.GetInt(s, "foo", "0"))
// Output:
// foo.0=123