I have a JSON file stored on the local machine. I need to read it in a variable and loop through it to fetch the JSON object values. If I use the Marshal command after reading the file using the ioutil.Readfile method, it gives some numbers as an output. These are my few failed attempts,
Attempt 1:
plan, _ := ioutil.ReadFile(filename) // filename is the JSON file to read
var data interface{}
err := json.Unmarshal(plan, data)
if err != nil {
log.Error("Cannot unmarshal the json ", err)
}
fmt.Println(data)
It gave me following error,
time="2016-12-13T22:13:05-08:00" level=error msg="Cannot unmarshal the json json: Unmarshal(nil)"
<nil>
Attempt 2: I tried to store the JSON values in a struct and then using MarshalIndent
generatePlan, _ := json.MarshalIndent(plan, "", " ") // plan is a pointer to a struct
fmt.Println(string(generatePlan))
It give me the output as string. But if I cast the output to string then I won't be able to loop it as JSON object.
How can we read a JSON file as JSON object in golang? Is it possible to do that?
Any help is appreciated. Thanks in advance!
The value to be populated by json.Unmarshal needs to be a pointer.
From GoDoc :
Unmarshal parses the JSON-encoded data and stores the result in the value pointed to by v.
So you need to do the following :
plan, _ := ioutil.ReadFile(filename)
var data interface{}
err := json.Unmarshal(plan, &data)
Your error (Unmarshal(nil)) indicates that there was some problem in reading the file , please check the error returned by ioutil.ReadFile
Also please note that when using an empty interface in unmarshal, you would need to use type assertion to get the underlying values as go primitive types.
To unmarshal JSON into an interface value, Unmarshal stores one of
these in the interface value:
bool, for JSON booleans
float64, for JSON numbers
string, for JSON strings
[]interface{}, for JSON arrays
map[string]interface{}, for JSON objects
nil for JSON null
It is always a much better approach to use a concrete structure to populate your json using Unmarshal.
If you're looking at this in 2022, the ioutil package has been deprecated. You can still use it but you'll get annoying errors.
Instead you can use the os package.
someStruct := SomeStruct{} // Read errors caught by unmarshal
fileBytes, _ := os.ReadFile(filename)
err := json.Unmarshal(fileBytes, spec)
Note, I'm specifically ignoring the error from os.ReadFile since it will also cause an error in json.Unmarshal for the sake of the example.
Related
I would like to take an arbitrary http.Request and get the body as a json string. I know this involves the json package, but it seems json.Decode needs a specific struct passed in by reference. How can I decode an arbitrary request body (and then stringify the result)?
func RequestBodyJsonString(r *http.Request) string {
}
Use ioutil.ReadAll to get data in byte slice then type conversion to string to get json string
bytedata, err := ioutil.ReadAll(r.Body)
reqBodyString := string(data)
An example in go playground here
I am connecting to an API which gives a rather large json payload. I need to add a key and value to the root object.Once I do ioutil.Readall from the package "net/http" the JSON is a byte array.
My goal is to just simply add to the structure and move on. As an example, the following the pretty similar to what I am doing: https://tutorialedge.net/golang/consuming-restful-api-with-go/
So how can I simply add to a JSON structure another element (key: value)?
If all you want to do is add a key and value to the root object and produce new JSON, and you don't care about having the data in a structure, you can unmarshal into map[string]interface{}, add your value, and then marshal again:
var m map[string]interface{}
err := json.Unmarshal(data, &m)
m["new_key"] = newValue
newData, err := json.Marshal(m)
(I'm not checking for errors, but you should do that of course.) Take a look at https://golang.org/pkg/encoding/json/ for more information about how to deal with JSON in Go.
Here's how you can do it in an efficient way while preserving the order of keys in the original JSON. The idea is to use json.RawMessage
// original JSON
bytes := []byte(`{"name": "Adele", "age": 24}`)
// let's try to add the kv pair "country": "USA" to it
type Person struct {
Bio json.RawMessage `json:"bio"`
Country string `json:"country"`
}
p := Person{
Bio: json.RawMessage(bytes),
Country: "USA",
}
// ignoring error for brevity
modifiedBytes, _ := json.Marshal(p)
fmt.Println(string(modifiedBytes))
Output:
{"bio":{"name":"Adele","age":24},"country":"USA"}
You can see that the ordering of original JSON is preserved, which wouldn't have been the case if you marshalled the JSON to a map[string]interface{}. This is also more efficient when you're dealing with huge JSONs since there's no reflection involved.
Complete code - https://play.golang.org/p/3hAPVbrAo_w
Since you have a byte data, you need to parse it and store the result in a variable that has your json structure using json.Marshal.
Then after, to add a new key value pair, you can define a new struct with the key and its data type
var variable type1
// Unmarshal the byte data in a variable
json.Unmarshall(data, &variable)
// to add a new key value you can define a new type
type type2 struct {
type1
key type
}
// and you can add
variable2 := type2{variable, newValueToAdd}
While deserializing & reserializing is the more "correct" approach, it may be overkill for just adding a value at the root, which can be done with simple string manipulation (really byte slice manipulation, but the semantics are similar and it's arguably easier):
data := []byte(`{"foo": 1, "bar": true}`)
ins := []byte(`, "baz": "hello"`) // Note the leading comma.
closingBraceIdx := bytes.LastIndexByte(data, '}')
data = append(data[:closingBraceIdx], ins...)
data = append(data, '}')
This is more error-prone, because it is unaware of JSON syntax entirely, but it's safe enough for most cases and for large JSON documents it is more efficient than a parse, insert, and reserialize.
Playground example: https://play.golang.org/p/h8kL4Zzp7rq
if you want to add key-value of json bytes to new json object, you can use json.RawMessage.
type Res struct {
Data interface{}
Message string
}
var row json.RawMessage
row = []byte(`{"Name":"xxx","Sex":1}`)
res := Res{Data:row,Message:"user"}
resBytes ,err := json.Marshal(res)
println(string(resBytes))
//print result："Data":{"Name":"xxx","Sex":1},"Message":"user"}
SJSON package is another way to modify JSON values.
In this example :
json:= `{
"name": {"first":"James","last":"Bond"},
"age' :40,
"license" {"Diamond","Gold","Silver"}
}`
To replace a "Diamond" with "Ultimate"
value, _ := sjson.Set(json, "license.0", "Ultimate")
fmt.Println(value)
json:= `{
"name": {"first":"James","last":"Bond"},
"age' :40,
"license" {"Ultimate","Gold","Silver"}
}`
Or to add a value "Ultimate" in the end:
value, _ := sjson.Set(json, "license.-1", "Ultimate")
fmt.Println(value)
json:= `{
"name": {"first":"James","last":"Bond"},
"age' :40,
"license" {"Diamond","Gold","Silver","Ultimate"}
}`
How do you parse YAML data, in Go, without knowing its structure in advance? All of the examples I've seen assume you want to decode a marshaled map whose keys you already know. What if you don't know the keys? What if it's not a map but a marshaled list, a scalar, or some other common type?
Though I'm principally concerned with YAML, here, it seems like the technique might be generally useful for JSON, etc.. since there's a general pattern for parsing structured data (tagged structs, obviously).
For JSON, unmarshal the data to an interface{} value. Use type assertions to determine what's in the value.
var v interface{}
err := json.Unmarshal(data, &v)
if err != nil {
// handle error
}
switch v := v.(type) {
case string:
fmt.Println("string:", v)
case float64:
fmt.Println("number:", v)
case []interface{}:
fmt.Println("array:", v)
case map[string]interface{}:
fmt.Println("object:", v)
case nil:
fmt.Println("nil")
}
In the case of JSON, the standard library json.Unmarshal function will unmarshal arbitrary JSON if you like, if you pass it a pointer to an uninitialized empty interface. (See this example.)
The official docs even say as much:
To unmarshal JSON into an interface value, Unmarshal stores one of these in the interface value:
bool, for JSON booleans
float64, for JSON numbers
string, for JSON strings
[]interface{}, for JSON arrays
map[string]interface{}, for JSON objects
nil for JSON null
Edit: Though not documented, the same is true of the yaml package; I tested on my workstation, and passing in a pointer to an uninitialized empty interface results in the initialization of correct arrays, maps, and primitives.
Hi everyone I'm trying to see what the proper way of accessing fields of a json object from a http.get request in go.
I first do an http.get call get the JSON and then print it (which works) but is there a way to access just a field?
for example:
response, err:= http.Get("URL")
//Error checking is done between
contents, err:=ioutil.Readall(response.Body)
//Now at this point I have a json that looks like
{"id": "someID",
"name": "someName",
"test": [{"Name":"Test1",
"Result": "Success"},
{"Name":"Test2",
"Result": "Success"},
{...},
]}
Is there a way to only print the "test" of the Json? What is the proper way of accessing that field?
Use encoding/json package to Unmarshal data into struct, like following.
type Result struct {
ID string `json:"id"`
Name string `json:"name"`
Test []interface{} `json:"test"`
}
var result Result
json.Unmarshal(contents, &result)
fmt.Println(result.Test)
You can also parse Test to specific struct.
Same as the previous answer, use encoding/json package to Unmarshal data. But if you don't want to specify the structure, you could use map[string]interface/bson.M{} to receive the data, and get the field, then cast into types your want.
m := make(map[string]interface{})
err := json.Unmarshal(data, &m)
if err != nil {
log.Fatal(err)
}
fmt.Println(m["id"])
You may want to try gabs container, if you are not sure how depth JSON hierarchy can be. Have a look at below resources
https://github.com/Jeffail/gabs
https://godoc.org/github.com/Jeffail/gabs
If you just want to access one field then you can use the jsonq module https://godoc.org/github.com/jmoiron/jsonq
For your example you could get the test object with code similar to
jq.Object("test")
Where jq is a jsonq query object constructed from your JSON above (see the godoc page for instructions on how to create a query object from a JSON stream or string).
You can also use this library for retrieving specific String, Integer, Float and Bool values at an arbitrary depth inside a JSON object.
Since you are starting with a URL, Decode is a better option than Unmarshal:
package main
import (
"encoding/json"
"net/http"
)
func main() {
r, e := http.Get("https://github.com/manifest.json")
if e != nil {
panic(e)
}
defer r.Body.Close()
var s struct { Name string }
json.NewDecoder(r.Body).Decode(&s)
println(s.Name == "GitHub")
}
https://golang.org/pkg/encoding/json#Decoder.Decode
You may check this https://github.com/valyala/fastjson
s := []byte(`{"foo": [123, "bar"]}`)
fmt.Printf("foo.0=%d\n", fastjson.GetInt(s, "foo", "0"))
// Output:
// foo.0=123
I'm developing a web service and part of that I read the Request.Body and try to unmarshal it.
if err := json.NewDecoder(body).Decode(r); err !=nil{
log.Error(err)
return err
}
The issue is that sometimes the client is sending an empty body and I get a panic runtime error: index out of range
goroutine 7 [running]:
How am I supposed to mitigate this?
I am decomposing your code:
NewDecoder: -
func NewDecoder(r io.Reader) *Decoder
NewDecoder returns a new decoder that reads from r. The decoder
introduces its own buffering and may read data from r beyond the JSON
values requested.
So NewDecoder only reads data from r. it's not care, is r empty...
Decode:-
func (dec *Decoder) Decode(v interface{}) error
Decode reads the next JSON-encoded value from its input and stores it
in the value pointed to by v.
See the documentation for Unmarshal for details about the conversion of JSON into a Go value.
To unmarshal JSON into an interface value, Unmarshal stores one of these in the interface value:
bool, for JSON booleans
float64, for JSON numbers
string, for JSON strings
[]interface{}, for JSON arrays
map[string]interface{}, for JSON objects
nil for JSON null
If a JSON value is not appropriate for a given target type, or if a
JSON number overflows the target type, Unmarshal skips that field and
completes the unmarshalling as best it can. If no more serious errors
are encountered, Unmarshal returns an UnmarshalTypeError describing
the earliest such error.
The JSON null value unmarshals into an interface, map, pointer, or
slice by setting that Go value to nil. Because null is often used in
JSON to mean “not present,” unmarshaling a JSON null into any other Go
type has no effect on the value and produces no error.
Reading above statement, it's clear that there is not chances, we get run time panic error. I was experimenting with a sample code to reproduce this ERROR. May error coming from inside the JSON package or your own code.
var dummy []byte
dummy = make([]byte, 10)
size, _ := body.Read(dummy)
if size > 0 {
if err := json.NewDecoder(body).Decode(r); err != nil {
log.Error(err)
return err
}
fmt.Fprintf(w, "%s", "Json cannot be empty")// where w is http.ResponseWriter