I have a json that I receive by post
{"endpoint": "assistance"}
I receive this like this
json_map: = make (map[string]interface{})
Now I need to assign it to a variable as a string but I don't know how to do it.
endpoint: = c.String (json_map ["endpoint"])
A type safe way to do this would be creating a struct that represents your request object and unmarshalling it.
This gives you a panic on unexpected requests.
package main
import (
"encoding/json"
"fmt"
)
type response struct {
Endpoint string
}
func main() {
jsonBody := []byte(`{"endpoint": "assistance"}`)
data := response{}
if err := json.Unmarshal(jsonBody, &data); err != nil {
panic(err)
}
fmt.Println(data.Endpoint)
}
// assistance
This program as an example safely decodes the JSON into a struct and prints the value.
What you are trying to achieve is not to convert a JSON to a string but an empty interface interface{} to a string You can achieve this by doing a type assertion:
endpoint, ok := json_map["endpoint"].(string)
if !ok {
// handle the error if the underlying type was not a string
}
Also as #Lex mentionned, it would probably be much safer to have a Go struct defining your JSON data. This way all your fields will be typed and you will no longer need this kind of type assertion.
I am collecting logs generated by an agent. It generates one large JSON output which I need to decompose into smaller JSON documents and use sarama to write to kafka. Due to MAX size constraint of kafka message, I have problems in demposing into several individual JSON documents. Any suggestions would be greatly appreciated. The log messages do not have any fixed fields or data types except date/time field indicating of logactivity
Sample #1
[{"date":1596206786.847531,"rand_value":11885153394315023285},{"date":1596206787.847446,"rand_value":6208802038498064748},{"date":1596206788.847526,"rand_value":932964293334035461},{"date":1596206789.847568,"rand_value":13217490172547025909}]
Sample 2
[{"date":1596206786.847743,"cpu_p":0,"user_p":0,"system_p":0,"cpu0.p_cpu":0,"cpu0.p_user":0,"cpu0.p_system":0,"cpu1.p_cpu":0,"cpu1.p_user":0,"cpu1.p_system":0,"cpu2.p_cpu":0,"cpu2.p_user":0,"cpu2.p_system":0,"cpu3.p_cpu":0,"cpu3.p_user":0,"cpu3.p_system":0,"cpu4.p_cpu":0,"cpu4.p_user":0,"cpu4.p_system":0,"cpu5.p_cpu":0,"cpu5.p_user":0,"cpu5.p_system":0,"cpu6.p_cpu":0,"cpu6.p_user":0,"cpu6.p_system":0,"cpu7.p_cpu":0,"cpu7.p_user":0,"cpu7.p_system":0},{"date":1596206787.847689,"cpu_p":1.25,"user_p":0.75,"system_p":0.5,"cpu0.p_cpu":2,"cpu0.p_user":1,"cpu0.p_system":1,"cpu1.p_cpu":1,"cpu1.p_user":0,"cpu1.p_system":1,"cpu2.p_cpu":2,"cpu2.p_user":1,"cpu2.p_system":1,"cpu3.p_cpu":3,"cpu3.p_user":2,"cpu3.p_system":1,"cpu4.p_cpu":1,"cpu4.p_user":0,"cpu4.p_system":1,"cpu5.p_cpu":1,"cpu5.p_user":1,"cpu5.p_system":0,"cpu6.p_cpu":2,"cpu6.p_user":2,"cpu6.p_system":0,"cpu7.p_cpu":0,"cpu7.p_user":0,"cpu7.p_system":0},{"date":1596206788.847754,"cpu_p":0.75,"user_p":0.5,"system_p":0.25,"cpu0.p_cpu":0,"cpu0.p_user":0,"cpu0.p_system":0,"cpu1.p_cpu":1,"cpu1.p_user":0,"cpu1.p_system":1,"cpu2.p_cpu":2,"cpu2.p_user":1,"cpu2.p_system":1,"cpu3.p_cpu":0,"cpu3.p_user":0,"cpu3.p_system":0,"cpu4.p_cpu":0,"cpu4.p_user":0,"cpu4.p_system":0,"cpu5.p_cpu":1,"cpu5.p_user":1,"cpu5.p_system":0,"cpu6.p_cpu":1,"cpu6.p_user":0,"cpu6.p_system":1,"cpu7.p_cpu":1,"cpu7.p_user":0,"cpu7.p_system":1},{"date":1596206789.847805,"cpu_p":0.8750000000000001,"user_p":0.5,"system_p":0.375,"cpu0.p_cpu":1,"cpu0.p_user":0,"cpu0.p_system":1,"cpu1.p_cpu":1,"cpu1.p_user":1,"cpu1.p_system":0,"cpu2.p_cpu":2,"cpu2.p_user":1,"cpu2.p_system":1,"cpu3.p_cpu":0,"cpu3.p_user":0,"cpu3.p_system":0,"cpu4.p_cpu":1,"cpu4.p_user":1,"cpu4.p_system":0,"cpu5.p_cpu":0,"cpu5.p_user":0,"cpu5.p_system":0,"cpu6.p_cpu":2,"cpu6.p_user":2,"cpu6.p_system":0,"cpu7.p_cpu":0,"cpu7.p_user":0,"cpu7.p_system":0}]
package main
import (
"encoding/json"
"fmt"
"io/ioutil"
"os"
)
func main() {
ibytes, err := ioutil.ReadFile("hello.json")
if err != nil {
fmt.Println(err)
os.Exit(-1)
}
var msgs []map[string]interface{}
err = json.Unmarshal(ibytes, &msgs)
if err != nil {
fmt.Println("Serialization Error", err)
os.Exit(-1)
}
for _,msg:=range msgs {
fmt.Println("%s",msg)
}
}
I am able to iterate over the individual messages but not in a friendly format to write into kafka.
I managed to find solution by myself using below code
var PlaceHolder []interface{}
err=json.Unmarshal(dbytes,&PlaceHolder)
if err!=nil {
return errors.New(fmt.Sprintf("Error during JSON Unmarshalling (%s) ",err))
}
for _,doc:=range PlaceHolder {
event,_:=json.Marshal(doc)
if err!=nil{
log.Println("Skipping: Error during JSON Marshaling (%s) ",err)
continue
}
KafkaMessage:= &sarama.ProducerMessage{
Topic: this.Topic,
Value: sarama.StringEncoder(event),
}
msgs=append(msgs,KafkaMessage)
}
All of the ways I'm seeing involve building structs and unmarshalling the data into the struct. But what if I'm getting JSON responses with hundreds of fields? I don't want to have to create 100 field structs just to be able to get to the data I want. Coming from a Java background there are easy ways to simply get the http response as a string and then pass the JSON string into a JSON object that allows for easy traversal. It's very painless. Is there anything like this in Go?
Java example in pseudo code:
String json = httpResponse.getBody();
JsonObject object = new JsonObject(json);
object.get("desiredKey");
Golang: fetch JSON from an HTTP response without using structs as helpers
This is a typical scenario we come across. This is achieved by json.Unmarshal.
Here is a simple json
{"textfield":"I'm a text.","num":1234,"list":[1,2,3]}
which is serialized to send across the network and unmarshaled at Golang end.
package main
import (
"fmt"
"encoding/json"
)
func main() {
// replace this by fetching actual response body
responseBody := `{"textfield":"I'm a text.","num":1234,"list":[1,2,3]}`
var data map[string]interface{}
err := json.Unmarshal([]byte(responseBody), &data)
if err != nil {
panic(err)
}
fmt.Println(data["list"])
fmt.Println(data["textfield"])
}
Hope this was helpful.
The json.Unmarshal method will unmarshal to a struct that does not contain all the fields in the original JSON object. In other words, you can cherry-pick your fields. Here is an example where FirstName and LastName are cherry-picked and MiddleName is ignored from the json string:
package main
import (
"encoding/json"
"fmt"
)
type Person struct {
FirstName string `json:"first_name"`
LastName string `json:"last_name"`
}
func main() {
jsonString := []byte("{\"first_name\": \"John\", \"last_name\": \"Doe\", \"middle_name\": \"Anderson\"}")
var person Person
if err := json.Unmarshal(jsonString, &person); err != nil {
panic(err)
}
fmt.Println(person)
}
The other answers here are misleading, as they don't show you what happens if you try to go deeper in the Map. This example works fine enough:
package main
import (
"encoding/json"
"fmt"
"net/http"
)
func main() {
r, e := http.Get("https://github.com/manifest.json")
if e != nil {
panic(e)
}
body := map[string]interface{}{}
json.NewDecoder(r.Body).Decode(&body)
/*
[map[
id:com.github.android
platform:play
url:https://play.google.com/store/apps/details?id=com.github.android
]]
*/
fmt.Println(body["related_applications"])
}
but if you try to go one level deeper, it fails:
/*
invalid operation: body["related_applications"][0] (type interface {} does not
support indexing)
*/
fmt.Println(body["related_applications"][0])
Instead, you would need to assert type at each level of depth:
/*
map[
id:com.github.android
platform:play
url:https://play.google.com/store/apps/details?id=com.github.android
]
*/
fmt.Println(body["related_applications"].([]interface{})[0])
You can as well unmarshal it into a map[string]interface{}
body, err := ioutil.ReadAll(resp.Body)
map := &map[string]interface{}{}
json.Unmarshal(body, map)
desiredValue := map["desiredKey"]
The received json must have an object as the most outer element. The map can also contain lists or nested maps, depending on the json.
I'm making a Go service that gathers JSON objects from different sources and aggregates them in a single JSON object.
I was wondering if there was any way to aggregate the child objects without having to unmarshal and re-marshal them again or having to manually build a JSON string.
I was thinking of using a struct containing the already marshalled parts, such as this:
type Event struct {
Place string `json:"place"`
Attendees string `json:"attendees"`
}
Where Place and Attendees are JSON strings themselves. I'd like to somehow mark them as "already marshalled" so they don't end up as escaped JSON strings but get used as is instead.
Is there any way to achieve this?
You can use json.RawMessage
RawMessage is a raw encoded JSON object. It implements Marshaler and Unmarshaler and can be used to delay JSON decoding or precompute a JSON encoding.
Also, json.RawMessage is an alias to []byte so you can values to it this way:
v := json.RawMessage(`{"foo":"bar"}`)
Example:
package main
import (
"encoding/json"
"fmt"
)
type Event struct {
Place json.RawMessage `json:"place"`
Attendees json.RawMessage `json:"attendees"`
}
func main() {
e := Event{
Place: json.RawMessage(`{"address":"somewhere"}`),
Attendees: json.RawMessage(`{"key":"value"}`),
}
c, err := json.Marshal(&e)
if err != nil {
panic(err)
}
fmt.Println(string(c))
// {"place":{"address":"somewhere"},"attendees":{"key":"value"}}
}
Yes, you can use a custom type that implements Marshaler interface.
https://play.golang.org/p/YB_eKlfOND
package main
import (
"fmt"
"encoding/json"
)
type Event struct {
Place RawString `json:"place"`
Attendees RawString `json:"attendees,omitempty"`
}
type RawString string
func (s RawString) MarshalJSON() ([]byte, error) {
return []byte(s), nil
}
func main() {
event := Event{
Place: RawString(`{"name":"Paris"}`),
Attendees: RawString(`[{"name":"John"}, {"name":"Juli"}]`),
}
s, err := json.Marshal(event)
fmt.Println(fmt.Sprintf("event: %v; err: %v", string(s), err))
}
I am trying to simultaneously unmarshal and strip fields from a number of different JSON responses into appropriate Go structs. To do this, I created a Wrappable interface that defines the Unwrap method (which strips the appropriate fields) and pass that interface to the code that unmarshals and unwraps. It looks like the following example (also at http://play.golang.org/p/fUGveHwiz9):
package main
import (
"encoding/json"
"fmt"
)
type Data struct {
A string `json:"a"`
B string `json:"b"`
}
type DataWrapper struct {
Elements []Data `json:"elems"`
}
type Wrapper interface {
Unwrap() []interface{}
}
func (dw DataWrapper) Unwrap() []interface{} {
result := make([]interface{}, len(dw.Elements))
for i := range dw.Elements {
result[i] = dw.Elements[i]
}
return result
}
func unmarshalAndUnwrap(data []byte, wrapper Wrapper) []interface{} {
err := json.Unmarshal(data, &wrapper)
if err != nil {
panic(err)
}
return wrapper.Unwrap()
}
func main() {
data := `{"elems": [{"a": "data", "b": "data"}, {"a": "data", "b": "data"}]}`
res := unmarshalAndUnwrap([]byte(data), DataWrapper{})
fmt.Println(res)
}
However, when I run the code, Go panics with the following error:
panic: json: cannot unmarshal object into Go value of type main.Wrapper
It seems the unmarshaller doesn't want to be passed a pointer to an interface. I am somewhat surprised by this given that I can get at the underlying type and fields using the reflect package within the unmarshalAndUnwrap method. Can anyone provide insight into this problem and how I might work around it?
As you stated, passing a non-pointer fails. Why are you trying to do this anyway?
Replace
res := unmarshalAndUnwrap([]byte(data), DataWrapper{})
by
res := unmarshalAndUnwrap([]byte(data), &DataWrapper{})
It should do the trick and it avoid unnecessary copy.
This error should help you understand: http://play.golang.org/p/jXxCxPQDOw