I am tring to get League of Legends champion informations from LOL static database. Link is given below.
Get info for specific hero
The problem is that i can only make request by hero names and all JSON responses are different from each other by only one field which is a "main" field; hero name. You can find problematic field as highlighted below:
Also tree respresentation:
My goal is to get all hero informations with iteration by range of known hero names as slice. You can check the code. I need only a couple of fields but the main tag is varies with every new request.
func GetHeroInfo(heroName string) *LolHeroInfo {
getUrl := fmt.Sprintf("http://ddragon.leagueoflegends.com/cdn/12.2.1/data/en_US/champion/%s.json", heroName)
fmt.Println(getUrl)
resp, err := http.Get(getUrl)
if err != nil {
fmt.Println(err)
return nil
}
defer resp.Body.Close()
read, err := io.ReadAll(resp.Body)
if err != nil {
fmt.Println(err)
return nil
}
heroGoverted := LolHeroInfo{}
err = json.Unmarshal(read, &heroGoverted)
if err != nil {
fmt.Println("unmarshall failed:", err)
return nil
}
return &heroGoverted }
Struct type LolHeroInfo is structifyed from this link: mholt's JSON to struct
You can check JSON response for another hero eg: JSON response for Annie .
Is there any way to make an agnostic struct field/tag for a JSON field. I believe this will be very hard because encoding/json package needs to check for field for particular tag in that JSON but maybe you encountered this kind of problem before. Creating separate struct for each hero is impossible so i will drop this case if i can't find a solution.
Thanks in advance for help.
Since you know it's just a single unknown key, you could just decode into a map[string]LolHeroInfo for the Data field, then do
heroGoverted := LolHeroInfo{}
for _, v := range decoded.Data {
heroGoverted = v
}
To solve problem, I used #dave 's solution.
I breake main struct into two separate struct. This way varying JSON field eliminated:
type LolHeroInfo struct {
Type string `json:"type"`
Format string `json:"format"`
Version string `json:"version"`
Data map[string]HeroInfoStruct `json:"data"`
}
heroInfo := lib.GetHeroInfo(lolHeroes[i])
for _, v := range heroInfo.Data { //unmarshalled to first struct
a := lib.HeroInfoStruct{} //data field; second struct
a = v
fmt.Println(a.Lore)// now i can reach to every field that i need
}
Related
I have a JSON object That contains an implementation of an interface within it. I'm attempting to take that JSON and marshal it into a struct whilst creating the implementation of the interface.
I've managed to get it to implement the interface with a custom JSON unmarshal function however I'm struggling to piece together how to then marshal the rest of the fields
I've created an example in the Go playground
https://play.golang.org/p/ztF7H7etdjM
My JSON being passed into my application is
{
"address":"1FYuJ4MsVmpzPoFJ6svJMJfygn91Eubid9",
"nonce":13,
"network_id":"qadre.demo.balance",
"challenge":"f2b19e71876c087e681fc092ea3a34d5680bbfe772e40883563e1d5513bb593f",
"type":"verifying_key",
"verifying_key":{
"verifying_key":"3b6a27bcceb6a42d62a3a8d02a6f0d73653215771de243a63ac048a18b59da29",
"fqdn":"huski.service.key"
},
"signature":"a3bf8ee202a508d5a5632f50b140b70b7095d8836493dc7ac4159f6f3350280078b3a58b2162a240bc8c7485894554976a9c7b5d279d3f5bf49fec950f024e02",
"fqdn":"huski.service.SingleKeyProof"
}
I've attempted to do a json.Unmarshal and pass in a new struct for the remaining fields however it seems to put me in an infinite loop, my application hangs and then crashes
The best solution I've come up with so far is to marshal the JSON into a `map[string]interface{} and do each field separately, this feels very clunky though
var m map[string]interface{}
if err := json.Unmarshal(data, &m); err != nil {
return err
}
ad, ok := m["address"]
if ok {
s.Address = ad.(string)
}
fqdn, ok := m["fqdn"]
if ok {
s.FQDN = fqdn.(string)
}
n, ok := m["nonce"]
if ok {
s.Nonce = int64(n.(float64))
}
c, ok := m["challenge"]
if ok {
s.Challenge = []byte(c.(string))
}
network, ok := m["network_id"]
if ok {
s.NetworkID = network.(string)
}
sig, ok := m["signature"]
if ok {
s.Signature = []byte(sig.(string))
}
The reason your code gets into an infinite loop when you try to unmarshal the rest of the fields is because, I presume, the implementation of UnmarshalJSON after its done unmarshaling the verifying key, calls json.Unmarshal with the receiver, which in turn calls the UnmarshalJSON method on the receiver and so they invoke each other ad infinitum.
What you can do is to create a temporary type using the existing type as its definition, this will "keep the structure" but "drop the methods", then unmarshal the rest of the fields into an instance of the new type, and, after unmarshal is done, convert the instance to the original type and assign that to the receiver.
While this fixes the infinite loop, it also re-introduces the original problem of json.Unmarshal not being able to unmarshal into a non-empty interface type. To fix that you can embed the new type in another temporary struct that has a field with the same json tag as the problematic field which will cause it to be "overshadowed" while json.Unmarshal is doing its work.
type SingleKey struct {
FQDN string `json:"fqdn"`
Address string `json:"address"`
Nonce int64 `json:"nonce"`
Challenge []byte `json:"challenge"`
NetworkID string `json:"network_id"`
Type string `json:"type"`
VerifyingKey PublicKey `json:"verifying_key"`
Signature []byte `json:"signature"`
}
func (s *SingleKey) UnmarshalJSON(data []byte) error {
type _SingleKey SingleKey
var temp struct {
RawKey json.RawMessage `json:"verifying_key"`
_SingleKey
}
if err := json.Unmarshal(data, &temp); err != nil {
return err
}
*s = SingleKey(temp._SingleKey)
switch s.Type {
case "verifying_key":
s.VerifyingKey = &PublicKeyImpl{}
// other cases ...
}
return json.Unmarshal([]byte(temp.RawKey), s.VerifyingKey)
}
https://play.golang.org/p/L3gdQZF47uN
Looking at what you've done in your custom unmarshalling function, you seem to be passing in a map with the name of fields as index, and the reflect.Type you want to unmarshal said value into. That, to me, suggests that the keys might be different for different payloads, but that each key has a distinct type associated with it. You can perfectly handle data like this with a simple wrapper type:
type WrappedSingleKey struct {
FQDN string `json:"fqdn"`
Address string `json:"address"`
Nonce int64 `json:"nonce"`
Challenge []byte `json:"challenge"`
NetworkID string `json:"network_id"`
Type string `json:"type"`
VerifyingKey json.RawMessage `json:"verifying_key"`
OtherKey json.RawMessage `json:"other_key"`
Signature []byte `json:"signature"`
}
type SingleKey struct {
FQDN string `json:"fqdn"`
Address string `json:"address"`
Nonce int64 `json:"nonce"`
Challenge []byte `json:"challenge"`
NetworkID string `json:"network_id"`
Type string `json:"type"`
VerifyingKey *PublicKey `json:"verifying_key,omitempty"`
OtherType *OtherKey `json:"other_key,omitempty"`
Signature []byte `json:"signature"`
}
So I've changed the type of your VerifyingKey field to a json.RawMessage. That's basically telling json.Unmarshal to leave that as raw JSON input. For every custom/optional field, add a corresponding RawMessage field.
In the unwrapped type, I've changed VerifyingKey to a pointer and added the omitempty bit to the tag. That's just to accomodate mutliple types, and not have to worry about custom marshalling to avoid empty fields, like the included OtherType field I have. To get what you need, then:
func (s *SingleKey) UnmarshalJSON(data []byte) error {
w := WrappedSingleKey{} // create wrapped instance
if err := json.Unmarshal(data, &w); err != nil {
return err
}
switch w.Type {
case "verifying_key":
var pk PublicKey
if err := json.Unmarshal([]byte(w.VerifyingKey), &pk); err != nil {
return err
}
s.VerifyingKey = &pk // assign
case "other_key":
var ok OtherKey
if err := json.Unmarshal([]byte(w.OtherKey), &ok); err != nil {
return err
}
s.OtherKey = &ok
}
// copy over the fields that didn't require anything special
s.FQDN = w.FQDN
s.Address = w.Address
}
This is a fairly simple approach, does away with the reflection, tons of functions, and is quite commonly used. It's something that lends itself quite well to code generation, too. The individual assignment of the fields is a bit tedious, though. You might think that you can solve that by embedding the SingleKey type into the wrapper, but be careful: this will recursively call your custom unmarshaller function.
You could, for example, update all the fields in the WRapped type to be pointers, and have them point to fields on your actual type. That does away with the manual copying of fields... It's up to you, really.
Note
I didn't test this code, just wrote it as I went along. It's something I've used in the past, and I believe what I wrote here should work, but no guarantees (as in: you might need to debug it a bit)
I have the following structures to export onto json:
type ExportedIncident struct {
Title string `json:"title"`
Host string `json:"host"`
Status string `json:"status"`
Date string `json:"date"`
Notes []ExportedNote `json:"notes"`
LogEntries []ExportedLogEntry `json:"log_entries"`
}
And I want underscore cased fields, so I had to define each field just for that as described in this answer: https://stackoverflow.com/a/11694255/1731473
But it's really cumbersome and I believe there is a simpler solution in Go, but I can't find it.
How can I set default letter-case (underscore, snake, camel...) for JSON export?
Unfortunately there are no opportunities to export your fields into snake_case so you have to maintain tags by yourself.
Technically you can use method MarshalJSON and perform all manipulations inside this method, but it isn't easier way...
As mentioned by #Vladimir Kovpak, you can't do this with the standard library, at least at the moment.
Though, inspired by this, you can achieve something close to what you want to do. Check out MarshalIndentSnakeCase:
func MarshalIndentSnakeCase(v interface{}, prefix, indent string) ([]byte, error) {
b, err := json.MarshalIndent(v, prefix, indent)
if err != nil {
return nil, err
}
x := convertKeys(b) // Here convert all keys from CamelCase to snake_case
buf := &bytes.Buffer{}
err = json.Indent(buf, []byte(x), prefix, indent)
if err != nil {
return nil, err
}
return buf.Bytes(), nil
}
Down Sides:
You have to do the same in the opposite direction for Unmashalling to work.
The order of elements is lost due to using a map inside convertKeys().
Try it on Go Playground.
I have an application that consumes data from a third-party api. I need to decode the json into a struct, which requires the struct to have json tags of the "incoming" json fields. The outgoing json fields have a different naming convention, so I need different json tags for the encoding.
I will have to do this with many different structs, and each struct might have many fields.
What is the best way to accomplish this without repeating a lot of code?
Example Structs:
// incoming "schema" field names
type AccountIn struct {
OpenDate string `json:"accountStartDate"`
CloseDate string `json:"cancelDate"`
}
// outgoing "schema" field names
type AccountOut struct {
OpenDate string `json:"openDate"`
CloseDate string `json:"closeDate"`
}
Maybe the coming change on Go 1.8 would help you, it will allow to 'cast' types even if its JSON tags definition is different: This https://play.golang.org/p/Xbsoa8SsEk works as expected on 1.8beta, I guess this would simplify your current solution
A bit an uncommon but probably quite well working method would be to use a intermediate format so u can use different readers and writers and therefore different tags. For example https://github.com/mitchellh/mapstructure which allows to convert a nested map structure into struct
types. Pretty similar like json unmarshal, just from a map.
// incoming "schema" field names
type AccountIn struct {
OpenDate string `mapstructure:"accountStartDate" json:"openDate"`
CloseDate string `mapstructure:"cancelDate" json:"closeDate"`
}
// from json to map with no name changes
temporaryMap := map[string]interface{}{}
err := json.Unmarshal(jsonBlob, &temporaryMap)
// from map to structs using mapstructure tags
accountIn := &AccountIn{}
mapstructure.Decode(temporaryMap, accountIn)
Later when writing (or reading) u will use directly the json functions which will then use the json tags.
If it's acceptable to take another round trip through json.Unmarshal and json.Marshal, and you don't have any ambiguous field names within your various types, you could translate all the json keys in one pass by unmarshaling into the generic structures used by the json package:
// map incoming to outgoing json identifiers
var translation = map[string]string{
"accountStartDate": "openDate",
"cancelDate": "closeDate",
}
func translateJS(js []byte) ([]byte, error) {
var m map[string]interface{}
if err := json.Unmarshal(js, &m); err != nil {
return nil, err
}
translateKeys(m)
return json.MarshalIndent(m, "", " ")
}
func translateKeys(m map[string]interface{}) {
for _, v := range m {
if v, ok := v.(map[string]interface{}); ok {
translateKeys(v)
}
}
keys := make([]string, 0, len(m))
for k := range m {
keys = append(keys, k)
}
for _, k := range keys {
if newKey, ok := translation[k]; ok {
m[newKey] = m[k]
delete(m, k)
}
}
}
https://play.golang.org/p/nXmWlj7qH9
This might be a Naive Approach but is fairly easy to implement:-
func ConvertAccountInToAccountOut(AccountIn incoming) (AccountOut outcoming){
var outcoming AccountOut
outcoming.OpenDate = incoming.OpenDate
outcoming.CloseDate = incoming.CloseDate
return outcoming
}
var IncomingJSONData AccountIn
resp := getJSONDataFromSource() // Some method that gives you the Input JSON
err1 := json.UnMarshall(resp,&IncomingJSONData)
OutGoingJSONData := ConvertAccountInToAccountOut(IncomingJSONData)
if err1 != nil {
fmt.Println("Error in UnMarshalling JSON ",err1)
}
fmt.Println("Outgoing JSON Data: ",OutGoingJSONData)
So I would like to fill any struct via the StructScan method and so read any data I get from the db into the regarding struct I feed the test function.
This script doesn't give any compile error (if you implement the other stuff like a db connection and so on) but still the StructScan method returns an error and tells me that it expects a slice of structs.
How do I create a slice of structs that I don't know the type of?
Thanks for any advice.
package main
import (
"database/sql"
"github.com/jmoiron/sqlx"
)
var db *sql.DB
type A struct {
Name string `db:"name"`
}
type B struct {
Name string `db:"name"
}
func main() {
testA := []A{}
testB := []B{}
test(testA, "StructA")
test(testB, "StructB")
}
func test(dataStruct interface{}, name string) {
rows, err := db.Query("SELECT * FROM table WHERE name =", name)
if err != nil {
panic(err)
}
for rows.Next() {
err := sqlx.StructScan(rows, &dataStruct)
if err != nil {
panic(err)
}
}
}
Super late to the party, but ran into this question while researching another issue. For others that stumble upon it, the problem is that you're passing a pointer to dataStruct into StructScan(). dataStruct is an interface, and pointers to interfaces are almost always an error in Go (in fact, they removed the automatic dereferencing of interface pointers a few versions back). You're also passing in your destination by value.
So, you are passing a pointer to an interface that holds a copy of your destination slice, when what you want instead is to pass the interface directly, and that interface to hold a pointer to your destination slice.
Instead of:
test(testA, "StructA")
test(testB, "StructB")
// ...
err := sqlx.StructScan(rows, &dataStruct)
Use:
test(&testA, "StructA")
test(&testB, "StructB")
// ...
err := sqlx.StructScan(rows, dataStruct)
If you have no idea what the destination struct type is, use sqlx.MapScan or sqlx.SliceScan. They don't map to a struct, but both return all the columns from the query result.
See http://jmoiron.github.io/sqlx/#altScanning
I am trying to define a struct that can hold an array of any type like so:
type APIResonse struct {
length int
data []interface{}
}
I want the data property to be capable of holding an array of any type/struct so I can have a single response type, that will eventually be serialized to json. So what I want to be able to write is something like the following:
someStruct := getSomeStructArray()
res := &APIResponse{
length: len(someStruct),
data: someStruct,
}
enc, err := json.Marshal(res)
Is this possible in Go? I keep getting cannot use cs (type SomeType) as type []interface {} in assignment. Or do I have to create a different response type for every variation of data? Or maybe I am going about this wrong entirely / not Go-like. Any help would be much appreciated!
There are a couple of problems with that code.
You need to use interface{}, not []interface{}, also [] is called a slice, an array is a fixed number of elements like [10]string.
And your APIResponse fields aren't exported, so json.Marshal will not print out anything.
func main() {
d := []dummy{{100}, {200}}
res := &APIResponse{
Length: len(d),
Data: d,
}
enc, err := json.Marshal(res)
fmt.Println(string(enc), err)
}
playground