I am trying to process a song using API.I have tried using a specific URL in http.get and further unmarshalling the data but the only element returned in the console is {}. Any help to send me in the right direction is appreciated.
Edit: here is some code. I have this in my main file.
var data [2]Data
if err != nil {
log.Fatal(err)
}
defer response.Body.Close()
if response.StatusCode != 200 {
log.Fatal("Didn't get 200")
}
rawData, err := ioutil.ReadAll(response.Body)
if err != nil {
log.Fatal(err)
}
json.Unmarshal(rawData, &data)
fmt.Println(data[0])
I have a struct created in a separate file for JSON tags/keys.
type Data struct {
SongID string `json:id`
Name string `json:name`
}
type data struct {
Error bool `json:"error"`
Response struct {
Results []struct {
ID int `json:"id"`
Name string `json:"name"`
} `json:"results"`
} `json:"response"`
}
This should be the data structure.
Use curl to request
curl https://searchly.asuarez.dev/api/v1/song/search?query=hello
use https://mholt.github.io/json-to-go/ to convert the JSON response to Golang struct.
Related
Sorry If i'm posting a question that has already been answered, but I can't seem to find any similar situations on here. I have a websocket client that receives dynamic json data with overlapping fields. The fact that the fields overlap has has made Unmarshalling very difficult for me.
I have structs for the data types I receive, but I need a way to check the json data before I unmarshal it to a specific struct. I was hoping that an interface could act as a temporary holder and I would then be able to match the interface to the specific struct I want to unmarshal to, but that doesn't seem possible, or I just don't know how to go about it. Here are a few examples of the data types I'm receiving and structs to go along with it in case that helps.
response 1: {"connectionID":17973829270596587247,"event":"systemStatus","status":"online","version":"1.9.0"}
response 2: {"channelID":328,"channelName":"ohlc-5","event":"subscriptionStatus","pair":"XBT/USD","status":"subscribed","subscription":{"interval":5,"name":"ohlc"}}
response 3: [328,["1649576721.042916","1649577000.000000","42641.50000","42641.50000","42641.50000","42641.50000","42641.50000","0.00335101",2],"ohlc-5","XBT/USD"]
response 4: {"event":"heartbeat"}
structs below
import (
"time"
"encoding/json"
)
type ConnStatus struct {
ConnectionID uint64 `json:"connectionID"`
Event string `json:"event"`
Status string `json:"status"`
Version string `json:"version"`
}
type HeartBeat struct {
Event string `json:"event"`
}
type OHLCsuccess struct {
ChannelID int `json:"channelID"`
ChannelName string `json:"channelName"`
Event string `json:"event"`
Pair string `json:"pair"`
Status string `json:"status"`
Subscription OHLC `json:"subscription"`
}
type OHLC struct {
Interval int `json:"interval"`
Name string `json:"name"`
}
type OHLCUpdates struct {
ChannelID int
OHLCArray OHLCNewTrade
ChannelName string
Pair string
}
type OHLCNewTrade struct {
StartTime UnixTime
EndTime UnixTime
Open float64
High float64
Low float64
Close float64
VWAP float64
Volume float64
Count int
}
type UnixTime struct {
time.Time
}
func (u *UnixTime) UnmarshalJSON(d []byte) error {
var ts int64
err := json.Unmarshal(d, &ts)
if err != nil {
return err
}
u.Time = time.Unix(ts, 0).UTC()
return nil
}
Any idea(s) on how to go about this? Thanks in advance for the help!
Are you in control of the different responses? If so, wow about adding a "type" field to the top level?
See "How to put everything at the top level" section on https://eagain.net/articles/go-dynamic-json/ for more info.
E.g. (untested):
func UnmarshalJSON(d []byte) error {
var jsonValue map[string]interface{}
err := json.Unmarshal(d, &jsonValue)
if err != nil {
return err
}
switch jsonValue["type"] {
case 1:
// unmarshal into struct type 1
case 2:
// unmarshal into struct type 2
default:
// throw err
}
// or if you don't have access to type:
if jsonValue["connectionID"] != nil {
// unmarshal into struct type 1
}
return nil
}
Alternatively you could try to (strictly) unmarshal into each struct, until you don't get an error, e.g. something like:
func DetermineStruct(d []byte) int {
var connStatus *ConnStatus
reader := bytes.NewReader(d)
decoder := json.NewDecoder(reader)
decoder.DisallowUnknownFields()
err := decoder.Decode(connStatus)
if err == nil {
panic(err)
}
err = json.Unmarshal(d, &connStatus)
if err == nil {
return 1
}
var ohlcSuccess OHLCsuccess
err = json.Unmarshal(d, &ohlcSuccess)
if err == nil {
return 2
}
}
I'm making a JSON request to a 3rd party API. If my auth token has expired, I receive an {"error": "message"}. If everything checks out, I receive a valid response.
Right now, I call json.Unmarshal twice to parse the response: once to check for the error and once to parse the real data.
Is there any way to avoid calling Unmarshal twice?
type Unknown map[string]interface{}
type Response struct {
Status string `json:"status"`
Strategy string `json:"strategy"`
Items []Item `json:"items"`
}
unknown := Unknown{}
json.Unmarshal(jsonData, &unknown)
if val, ok := unknown["error"]; ok {
fmt.Println(val)
return
}
response := Response{}
err := json.Unmarshal(jsonData, &response)
if err != nil {
fmt.Println("There was an error")
fmt.Println(err)
return
}
You can use embedding to decode everything at once:
type Response struct {
Status string `json:"status"`
Strategy string `json:"strategy"`
Items []Item `json:"items"`
}
var dest struct {
Error string `json:"error"`
Response // embedded Response
}
if err := json.Unmarshal(jsonData, &dest); err != nil {
fmt.Println(err)
return
} else if len(dest.Error) > 0 {
fmt.Println(dest.Error)
return
}
response := dest.Response
// ...
See an example on playground: https://play.golang.com/p/eAhFt99n07k
I need to unmarshal json object which may have the following formats:
Format1:
{
"contactType": 2,
"value": "0123456789"
}
Format2:
{
"contactType": "MobileNumber",
"value": "0123456789"
}
The structure I'm using for unmarshalling is:-
type Contact struct {
ContactType int `json:"contactType"`
Value string `json:"value"`
}
But this works only for format 1. I don't want to change the datatype of ContactType but I want to accommodate the 2nd format as well. I heard about json.RawMarshal and tried using it.
type Contact struct {
ContactType int
Value string `json:"value"`
Type json.RawMessage `json:"contactType"`
}
type StringContact struct {
Type string `json:"contactType"`
}
type IntContact struct {
Type int `json:"contactType"`
}
This gets the unmarshalling done, but I'm unable to set the ContactType variable which depends on the type of json.RawMessage. How do I model my structure so that this problem gets solved?
You will need to do the unmarshalling yourself. There is a very good article that shows how to use the json.RawMessage right and a number of other solutions to this very problem, Like using interfaces, RawMessage, implemention your own unmarshal and decode functions etc.
You will find the article here: JSON decoding in GO by Attila Oláh
Note: Attila has made a few errors on his code examples.
I taken the liberty to put together (using some of the code from Attila) a working example using RawMessage to delay the unmarshaling so we can do it on our own version of the Decode func.
Link to GOLANG Playground
package main
import (
"fmt"
"encoding/json"
"io"
)
type Record struct {
AuthorRaw json.RawMessage `json:"author"`
Title string `json:"title"`
URL string `json:"url"`
Author Author
}
type Author struct {
ID uint64 `json:"id"`
Email string `json:"email"`
}
func Decode(r io.Reader) (x *Record, err error) {
x = new(Record)
if err = json.NewDecoder(r).Decode(x); err != nil {
return
}
if err = json.Unmarshal(x.AuthorRaw, &x.Author); err == nil {
return
}
var s string
if err = json.Unmarshal(x.AuthorRaw, &s); err == nil {
x.Author.Email = s
return
}
var n uint64
if err = json.Unmarshal(x.AuthorRaw, &n); err == nil {
x.Author.ID = n
}
return
}
func main() {
byt_1 := []byte(`{"author": 2,"title": "some things","url": "https://stackoverflow.com"}`)
byt_2 := []byte(`{"author": "Mad Scientist","title": "some things","url": "https://stackoverflow.com"}`)
var dat Record
if err := json.Unmarshal(byt_1, &dat); err != nil {
panic(err)
}
fmt.Printf("%#s\r\n", dat)
if err := json.Unmarshal(byt_2, &dat); err != nil {
panic(err)
}
fmt.Printf("%#s\r\n", dat)
}
Hope this helps.
I have an end-point that receives data from a Google PubSub request. As per this repo, the object is as so:
type pushRequest struct {
Message struct {
Attributes map[string]string
Data []byte
ID string `json:"message_id"`
}
Subscription string
}
The Data field is consistently formatted as so:
type Data struct {
Key string `json:"key"`
Body string `json:"body"`
Meta map[string]interface{} `json:"meta"`
}
I can obviously unmarshal the JSON request with something like this:
f := &pushRequest{}
json.Unmarshal(msg, &f)
That leaves with the the []bytes field. Which I can do something like this to convert to a string, as per the docs
messages = append(messages, string(f.Message.Data))
Which doesn't help, since I need it as a struct.
I can Unmarshal the array again:
var m Data
json.Unmarshal(f.Message.Data, &m)
Have tried changing the field type in the pushRequest struct to Data without success. Blank...
Is there a way I can unpack things in a single pass? Doing it twice seems ridiculous.
If it's obvious, I just can't see it!
decoder := json.NewDecoder(r.Body)
psmsg := &PushRequest{}
decoderErr := decoder.Decode(&psmsg)
if decoderErr != nil {
// Error...
return
}
data := Data{}
unmarshalErr := json.Unmarshal([]byte(string(psmsg.Message.Data)), &data)
if unmarshalErr != nil {
// Error...
return
}
Here is a snippet from my cloud function, which serves as a pub/sub push endpoint. The key is that you first have to decode the body using the PushRequest struct. Next, you can transform the message data into a struct. According to the documentation, the Data field within Message is a base-64 encoded string, therefore you have to decode it first.
type PushRequest struct {
Message pubsub.PubsubMessage `json:"message"`
Subscription string `json:"subscription"`
}
type Example struct {
ID string `json:"id" firestore:"id"`
}
func HTTPEndpoint(w http.ResponseWriter, r *http.Request) {
var pr common.PushRequest
if err := json.NewDecoder(r.Body).Decode(&pr); err != nil {
log.Fatalf("Could not decode body: %v", err)
return
}
data, err := base64.StdEncoding.DecodeString(pr.Message.Data)
if err != nil {
log.Fatalf("Base64: %v", err)
return
}
var example Example
if err := json.Unmarshal(data, &example); err != nil {
log.Fatalf("Json: %v", err)
return
}
// Do something useful with the struct
}
I'm trying to unmarshal this json https://www.reddit.com/r/videos/comments/3vgdsb/recruitment_2016.json
It starts as an array of two different objects, and I only need data on the second object.
I want to retrieve the comments body, it doesn't give me any error when i'm trying to decode it, but it doesn't capture the data I want.
This is the output I get from running this:
//Response struct when initialized: []
//Response struct decoded: [{{{[]}}} {{{[]}}}]
////
type Response []struct {
Parent struct {
Data struct {
Children []struct {
Com Comment
}
}
}
}
type Comment struct {
Name string `json:"body"`
}
func init() {
http.HandleFunc("/api/getcomments", getComments)
}
func getComments(w http.ResponseWriter, r *http.Request) {
url := "https://www.reddit.com/r/videos/comments/3vgdsb/recruitment_2016.json"
c := appengine.NewContext(r)
client := urlfetch.Client(c)
resp, err := client.Get(url)
if err != nil { fmt.Fprint(w, "Error client.Get(): ", err) }
re := new(Response)
fmt.Fprint(w, "Response struct: ", re, "\n")
errTwo := json.NewDecoder(resp.Body).Decode(&re)
if errTwo != nil { fmt.Fprint(w, "Error decoding: ", errTwo, "\n") }
fmt.Fprint(w, "Response struct: ", re)
}
For everyone that is struggling to create the right structure for JSON unmarshalling, here's a cool website that convert any JSON to the right Go struct: JSON-to-Go
The json data you are unmarshaling does not conform with your data, and if the names of the fields are not like in your struct, you should use struct tags as well. It should be more like this:
type Response []struct {
Kind string `json:"kind"`
Data struct {
Children []struct {
Data struct {
Replies []struct {
// whatever...
} `json:"replies"`
} `json:"data"`
} `json:"children"`
} `json:"data"`
}
}
Of course I'd replace the inline types with real, named types, but I'm just making a point here in regards to the data hierarchy.
Goddamn, that's some ugly bloated JSON BTW.