With PHP and JavaScript (and Node) parsing JSON is a very trivial operation. From the looks of it Go is rather more complicated. Consider the example below:
package main
import ("encoding/json";"fmt")
type fileData struct{
tn string
size int
}
type jMapA map[string] string
type jMapB map[string] fileData
func parseMapA(){
var dat jMapA
s := `{"lang":"Node","compiled":"N","fast":"maybe"}`
if err := json.Unmarshal([]byte(s), &dat); err != nil {
panic(err)
}
fmt.Println(dat);
for k,v := range dat{
fmt.Println(k,v)
}
}
func parseMapB(){
var dat jMapB
s := `{"f1":{"tn":"F1","size":1024},"f2":{"tn":"F2","size":2048}}`
if err := json.Unmarshal([]byte(s), &dat); err != nil {
panic(err)
}
fmt.Println(dat);
for k,v := range dat{
fmt.Println(k,v)
}
}
func main() {
parseMapA()
parseMapB()
}
The parseMapA() call obligingly returns:
map[lang:Node Compiled:N fast:maybe]
lang Node
compiled N
fast maybe
However, parseMapB() returns:
map[f1:{ 0}, f2:{ 0}]
f2 { 0}
f1 { 0}
I am into my first few hours with Go so I imagine I am doing something wrong here. However, I have no idea what that might be. More generally, what would the Go equivalent of the Node code
for(p in obj){
doSomethingWith(obj[p]);
}
be in Go?
In Go, unmarshaling works only if a struct has exported fields, ie. fields that begin with a capital letter.
Change your first structure to:
type fileData struct{
Tn string
Size int
}
See http://play.golang.org/p/qO3U7ZNrNs for a fixed example.
Moreover, if you intend to marshal this struct back to JSON, you'll notice that the resulting JSON use capitalized fields, which is not what you want.
You need to add field tags:
type fileData struct{
Tn string `json:"tn"`
Size int `json:"size"`
}
so the Marshal function will use the correct names.
Related
I thought I understood unmarshalling by now, but I guess not. I'm having a little bit of trouble unmarshalling a map in go. Here is the code that I have so far
type OHLC_RESS struct {
Pair map[string][]Candles
Last int64 `json:"last"`
}
type Candles struct {
Time uint64
Open string
High string
Low string
Close string
VWAP string
Volume string
Count int
}
func (c *Candles) UnmarshalJSON(d []byte) error {
tmp := []interface{}{&c.Time, &c.Open, &c.High, &c.Low, &c.Close, &c.VWAP, &c.Volume, &c.Count}
length := len(tmp)
err := json.Unmarshal(d, &tmp)
if err != nil {
return err
}
g := len(tmp)
if g != length {
return fmt.Errorf("Lengths don't match: %d != %d", g, length)
}
return nil
}
func main() {
response := []byte(`{"XXBTZUSD":[[1616662740,"52591.9","52599.9","52591.8","52599.9","52599.1","0.11091626",5],[1616662740,"52591.9","52599.9","52591.8","52599.9","52599.1","0.11091626",5]],"last":15}`)
var resp OHLC_RESS
err := json.Unmarshal(response, &resp)
fmt.Println("resp: ", resp)
}
after running the code, the last field will unmarshal fine, but for whatever reason, the map is left without any value. Any help?
The expedient solution, for the specific example JSON, would be to NOT use a map at all but instead change the structure of OHLC_RESS so that it matches the structure of the JSON, i.e.
type OHLC_RESS struct {
Pair []Candles `json:"XXBTZUSD"`
Last int64 `json:"last"`
}
https://go.dev/play/p/Z9PhJt3wX33
However it's safe to assume, I think, that the reason you've opted to use a map is because the JSON object's key(s) that hold the "pairs" can vary and so hardcoding them into the field's tag is out of the question.
To understand why your code doesn't produce the desired result, you have to realize two things. First, the order of a struct's fields has no bearing on how the keys of a JSON object will be decoded. Second, the name Pair holds no special meaning for the unmarshaler. Therefore, by default, the unmarshaler has no way of knowing that your wish is to decode the "XXBTZUSD": [ ... ] element into the Pair map.
So, to get your desired result, you can have the OHLC_RESS implement the json.Unmarshaler interface and do the following:
func (r *OHLC_RESS) UnmarshalJSON(d []byte) error {
// first, decode just the object's keys and leave
// the values as raw, non-decoded JSON
var obj map[string]json.RawMessage
if err := json.Unmarshal(d, &obj); err != nil {
return err
}
// next, look up the "last" element's raw, non-decoded value
// and, if it is present, then decode it into the Last field
if last, ok := obj["last"]; ok {
if err := json.Unmarshal(last, &r.Last); err != nil {
return err
}
// remove the element so it's not in
// the way when decoding the rest below
delete(obj, "last")
}
// finally, decode the rest of the element values
// in the object and store them in the Pair field
r.Pair = make(map[string][]Candles, len(obj))
for key, val := range obj {
cc := []Candles{}
if err := json.Unmarshal(val, &cc); err != nil {
return err
}
r.Pair[key] = cc
}
return nil
}
https://go.dev/play/p/Lj8a8Gx9fWH
The problem I'm currently having is after saving a struct to a json file and then opening the struct from the json file, somehow the properties of the struct have changed slightly.
In the struct N, sometimes A and B can point to the same J. However, after encoding then decoding they point to different Js of the value.
before encoding this returns true (expected). After decoding it, it returns false (not expected)
fmt.Println("is same pointer", n.A[0] == n.B[0])
Is this supposed to happen? Is there a way around this. Thanks.
type N struct {
A []*J
B []*J
C []*J
}
func (n *N) Save(name string) {
name = "radacted.json"
err := os.Remove(name)
file, err := os.Create(name)
defer file.Close()
if err != nil {
fmt.Println(err)
}
bytes, err := json.Marshal(n)
file.Write(bytes)
}
func Open(name string) *N {
bytes, err := ioutil.ReadFile("redacted.json")
if err != nil {
log.Fatal("decode error:", err)
}
var n NeuralNetwork
json.Unmarshal(bytes, &n)
return &n
}
It's expected and documented behaviour
Pointer values encode as the value pointed to.
You can assert values equality
*n.A[0] == *n.B[0] //should stay
fmt.Println("is same pointer", n.A[0] == n.B[0])
you are comparing the address value here so it will not be the same. Let me give you an example
suppose you have struct like this :
type Test struct {
ValueA *int
ValueB *int
}
and on your main function you add the same value but with different address in this case with different variable :
func main() {
hello := 12
hello2 := 12
testObject := Test{ValueA: &hello, ValueB: &hello2}
if *testObject.ValueA == *testObject.ValueB {
fmt.Println("Equal Value")
} else {
fmt.Println("Different Value")
}
}
Notice that the *testObject.ValueA and *testObject.ValueBis getting the exact value not the value address. If you are not using * the result would be different.
so as uvelichitel said you just need to use * when comparing your struct value.
I an new to programming in Go so apologies if this is something obvious. I have a JSON file named foo.json:
{"type":"fruit","name":"apple","color":"red"}
and I am writing a Go program that has to do something when the "name" value in the JSON file is "apple". It needs no other information from that JSON file as that file is used for a completely different purpose in another area of the code.
I have read documentation on Decode() and Unmarshal() and abut 30 different web pages describing how to read the whole file into structures, etc. but it all seems extremely complicated for what I want to do which is just write the correct code to implement the first 2 lines of this pseudo-code:
file, _ := os.Open("foo.json")
name = file["name"]
if (name == "apple") {
do stuff
}
such that I end up with a Go variable named name that contains the string value apple. What is the right way to do this in Go?
The easiest method to do what you want is to decode into a struct.
Provided the format remains similar to {"type":"fruit","name":"apple","color":"red"}
type Name struct {
Name string `json:"name"`
}
var data []byte
data, _ = ioutil.ReadFile("foo.json")
var str Name
_ = json.Unmarshal(data, &str)
if str.Name == "apple" {
// Do Stuff
}
Your other option is to use third party libraries such as gabs or jason.
Gabs :
jsonParsed, err := gabs.ParseJSON(data)
name, ok := jsonParsed.Path("name").Data().(string)
Jason :
v, _ := jason.NewObjectFromBytes(data)
name, _ := v.GetString("name")
Update :
The structure
type Name struct {
Name string `json:"name"`
}
is the json equivalent of {"name":"foo"}.
So unmarshaling won't work for the following json with different formats.
[{"name":"foo"}]
{"bar":{"name":"foo"}}
PS : As mentioned by W.K.S. In your case an anonymous struct would suffice since you're not using this structure for anything else.
One thing is reading a file and other one is decoding a JSON document. I leave you a full example doing both. To run it you have to have a file called file.json in the same directory of your code or binary executable:
package main
import (
"encoding/json"
"io/ioutil"
"log"
"os"
)
func main() {
f, err := os.Open("file.json") // file.json has the json content
if err != nil {
log.Fatal(err)
}
bb, err := ioutil.ReadAll(f)
if err != nil {
log.Fatal(err)
}
doc := make(map[string]interface{})
if err := json.Unmarshal(bb, &doc); err != nil {
log.Fatal(err)
}
if name, contains := doc["name"]; contains {
log.Printf("Happy end we have a name: %q\n", name)
} else {
log.Println("Json document doesn't have a name field.")
}
log.Printf("full json: %s", string(bb))
}
https://play.golang.org/p/0u04MwwGfn
I have also tried to find a simple solution such as $d = json_decode($json, true) in PHP and came to the conclusion that there is no such simple way in Golang. The following is the simplest solution I could make (the checks are skipped for clarity):
var f interface{}
err = json.Unmarshal(file, &f)
m := f.(map[string]interface{})
if (m["name"] == "apple") {
// Do something
}
where
file is an array of bytes of JSON string,
f interface serves as a generic container for unknown JSON structure,
m is a map returned by the type assertion.
We can assert that f is a map of strings, because Unmarshal() builds a variable of that type for any JSON input. At least, I couldn't make it return something different. It is possible to detect the type of a variable by means of run-time reflection:
fmt.Printf("Type of f = %s\n", reflect.TypeOf(f))
For the f variable above, the code will print Type of f = map[string]interface {}.
Example
And this is the full code with necessary checks:
package main
import (
"fmt"
"os"
"io/ioutil"
"encoding/json"
)
func main() {
// Read entire file into an array of bytes
file, err := ioutil.ReadFile("foo.json")
if (err != nil) {
fmt.Fprintf(os.Stderr, "Failed read file: %s\n", err)
os.Exit(1)
}
var f interface{}
err = json.Unmarshal(file, &f)
if (err != nil) {
fmt.Fprintf(os.Stderr, "Failed to parse JSON: %s\n", err)
os.Exit(1)
}
// Type-cast `f` to a map by means of type assertion.
m := f.(map[string]interface{})
fmt.Printf("Parsed data: %v\n", m)
// Now we can check if the parsed data contains 'name' key
if (m["name"] == "apple") {
fmt.Print("Apple found\n")
}
}
Output
Parsed data: map[type:fruit name:apple color:red]
Apple found
The proper Go way of doing this would be to decode into an instance of an anonymous struct containing only the field you need.
func main() {
myStruct := struct{ Name string }{}
json.Unmarshal([]byte(`{"type":"fruit","name":"apple","color":"red"}`), &myStruct)
fmt.Print(myStruct.Name)
}
Playground Link
Alternatively, You could use Jeffails/gabs JSON Parser:
jsonParsed,_ := gabs.ParseJSON([]byte(`{"type":"fruit","name":"apple","color":"red"}`));
value, ok = jsonParsed.Path("name").Data().(string)
I'm calling an API which will return Json objects like this:
{
name: "XXX"
type: "TYPE_1"
shared_fields: {...}
type_1_fields: {...}
..
type_2_fields: {...}
}
Based on different types, this object will have different kinds of fields, but these fields are certain for different types.
So, I unmarshal the Json string to map[string]interface{} to fetch the the different type but how can I turn these map[string]interface{} to a certain struct?
var f map[string]interface{}
err := json.Unmarshal(b, &f)
type := f["type"]
switch type {
case "type_1":
//initialize struct of type_1
case "type_2":
//initialize struct of type_2
}
For this sort of two-step json decoding you will probably want to check out json.RawMessage. It allows you to defer processing of parts of your json response. The example in the docs shows how.
One way to do it would be to have a constructor function (one that starts with New…) that takes a map as input parameter.
A second way, much slower in my opinion, would be to redo the unmarshalling to the right struct type.
If the types are different enough and you want to be lazy you can just try to decode it in each format:
f1 := type1{}
err := json.Unmarshal(b, &f1)
if err == nil {
return f1
}
f2 := type2{}
err := json.Unmarshal(b, &f2)
if err == nil {
return f2
}
...
If the objects are similar or you want to be less lazy, you could just decode the type and then do something:
type BasicInfo struct {
Type string `json:"type"`
}
f := BasicInfo{}
err := json.Unmarshal(b, &f)
switch f.Type {
case "type_1":
//initialize struct of type_1
case "type_2":
//initialize struct of type_2
}
I have this struct
type SyncInfo struct {
Target string
}
Now I query some json data from ElasticSearch. Source is of type json.RawMessage.
All I want is to map source to my SyncInfo which I created the variable mySyncInfo for.
I even figured out how to do that...but it seems weird. I first call MarshalJSON() to get a []byte and then feed that to json.Unmarshal() which takes an []byte and a pointer to my struct.
This works fine but it feels as if I'm doing an extra hop. Am I missing something or is that the intended way to get from a json.RawMessage to a struct?
var mySyncInfo SyncInfo
jsonStr, _ := out.Hits.Hits[0].Source.MarshalJSON()
json.Unmarshal(jsonStr, &mySyncInfo)
fmt.Print(mySyncInfo.Target)
As said, the underlying type of json.RawMessage is []byte, so you can use a json.RawMessage as the data parameter to json.Unmarshal.
However, your problem is that you have a pointer (*json.RawMessage) and not a value. All you have to do is to dereference it:
err := json.Unmarshal(*out.Hits.Hits[0].Source, &mySyncInfo)
Working example:
package main
import (
"encoding/json"
"fmt"
)
type SyncInfo struct {
Target string
}
func main() {
data := []byte(`{"target": "localhost"}`)
Source := (*json.RawMessage)(&data)
var mySyncInfo SyncInfo
// Notice the dereferencing asterisk *
err := json.Unmarshal(*Source, &mySyncInfo)
if err != nil {
panic(err)
}
fmt.Printf("%+v\n", mySyncInfo)
}
Output:
{Target:localhost}
Playground: http://play.golang.org/p/J8R3Qrjrzx
json.RawMessage is really just a slice of bytes. You should be able to feed it directly into json.Unmarshal directly, like so:
json.Unmarshal(out.Hits.Hits[0].Source, &mySyncInfo)
Also, somewhat unrelated, but json.Unmarshal can return an error and you want to handle that.
err := json.Unmarshal(*out.Hits.Hits[0].Source, &mySyncInfo)
if err != nil {
// Handle
}