I have a table in mysql with 3 columns Profile string, userInformation JSON, badge string.
Profile
userInformation
badge
https://ps.w.org/metronet-profile-picture/assets/icon-256x256.png?rev=2464419
{"name": "Suzan Collins", "points": 10000, "countryName": "Poland"}
assets/batcherPage/gold.png
This is my struct:
type BatchersData struct {
ProfileURL string `json:"profileUrl"`
UserInformation UserInformation `json:"userInformation"`
Badge string `json:"badge"`
}
type UserInformation struct {
Points int64 `json:"points"`
Name string `json:"name"`
CountryName string `json:"countryName"`
}
What I want to do is make a select query on this table ie GET and retrieve every information..
using this code, I have accessed Profile and Badge :
func getBatchers(c *gin.Context) {
var batchers []BatchersData
rows, err := db.Query("SELECT profileUrl, badge FROM Batchers_page_db")
if err != nil {
return
}
defer rows.Close()
for rows.Next() {
var batcher BatchersData
if err := rows.Scan(&batcher.ProfileURL, &batcher.Badge); err != nil {
return
}
batchers = append(batchers, batcher)
}
if err := rows.Err(); err != nil {
return
}
c.IndentedJSON(http.StatusOK, batchers)
}
But I want to access JSON column ie UserInformation as well. I know that the query will be
rows, err := db.Query("SELECT * FROM Batchers_page_db")
But i'll have to make a change in this statement
if err := rows.Scan(&batcher.ProfileURL, &batcher.Badge);
I have tried doing this : but nothing works
rows.Scan(&batcher.ProfileURL,&batcher.UserInformation, &batcher.Badge);
You need to implement Scan interface doc to map the data to JSON. Here try this:
func (u * UserInformation) Scan(value interface{}) error {
b, ok := value.([]byte)
if !ok {
return errors.New("type assertion to []byte failed")
}
return json.Unmarshal(b, &u)
}
SELECT JSON_EXTRACT(userInformation,'$.name') as profileName,
JSON_EXTRACT(userInformation,'$.points') as userPoints from
Batchers_page_db
Not tested, but something like this will work but make sure your database field must be JSON type.
Related
I have a scenario where the JSON that has dynamic set of fields that need to get unmarshalled in to a struct.
const jsonStream = `{
"name": "john",
"age": 23,
"bvu62fu6dq": {
"status": true
}
}`
type Status struct {
Status bool
}
type Person struct {
Name string `json:"name"`
Age int `json:"age"`
Status map[string]Status `json:"status"`
}
func main() {
dec := json.NewDecoder(strings.NewReader(jsonStream))
for {
var person Person
if err := dec.Decode(&person); err == io.EOF {
break
} else if err != nil {
log.Fatal(err)
}
fmt.Println(person)
fmt.Println(person.Status["bvu62fu6dq"])
}
}
The output:
{john 23 map[]}
{false}
When it gets unmarshalled, the nested status struct is not being correctly resolved to the value in the JSON (shows false even with true value in JSON), is there any issue in the code?
Your types don't really match with the JSON you have:
type Status struct {
Status bool
}
type Person struct {
Name string `json:"name"`
Age int `json:"age"`
Status map[string]Status `json:"status"`
}
Maps to JSON that looks something like this:
{
"name": "foo",
"age": 12,
"status": {
"some-string": {
"Status": true
}
}
}
The easiest way to unmarshal data with a mix of known/unknown fields in a go type is to have something like this:
type Person struct {
Name string `json:"name"`
Age int `json:"age"`
Random map[string]interface{} `json:"-"` // skip this key
}
Then, first unmarshal the known data:
var p Person
if err := json.Unmarshal([]byte(jsonStream), &p); err != nil {
panic(err)
}
// then unmarshal the rest of the data
if err := json.Unmarshal([]byte(jsonStream), &p.Random); err != nil {
panic(err)
}
Now the Random map will contain every and all data, including the name and age fields. Seeing as you've got those tagged on the struct, these keys are known, so you can easily delete them from the map:
delete(p.Random, "name")
delete(p.Random, "age")
Now p.Random will contain all the unknown keys and their respective values. These values apparently will be an object with a field status, which is expected to be a boolean. You can set about using type assertions and convert them all over to a more sensible type, or you can take a shortcut and marshal/unmarshal the values. Update your Person type like so:
type Person struct {
Name string `json:"name"`
Age int `json:"age"`
Random map[string]interface{} `json:"-"`
Statuses map[string]Status `json:"-"`
}
Now take the clean Random value, marshal it and unmarshal it back into the Statuses field:
b, err := json.Marshal(p.Random)
if err != nil {
panic(err)
}
if err := json.Unmarshal(b, &p.Statuses); err != nil {
panic(err)
}
// remove Random map
p.Random = nil
The result is Person.Statuses["bvu62fu6dq"].Status is set to true
Demo
Cleaning this all up, and marshalling the data back
Now because our Random and Statuses fields are tagged to be ignored for JSON marshalling (json:"-"), marshalling this Person type won't play nice when you want to output the original JSON from these types. It's best to wrap this logic up in a custom JSON (un)-Marshaller interface. You can either use some intermediary types in your MarshalJSON and UnmarshalJSON methods on the Person type, or just create a map and set the keys you need:
func (p Person) MarshalJSON() ([]byte, error) {
data := make(map[string]interface{}, len(p.Statuses) + 2) // 2 being the extra fields
// copy status fields
for k, v := range p.Statuses {
data[k] = v
}
// add known keys
data["name"] = p.Name
data["age"] = p.Age
return json.Marshal(data) // return the marshalled map
}
Similarly, you can do the same thing for UnmarshalJSON, but you'll need to create a version of the Person type that doesn't have the custom handling:
type intermediaryPerson struct {
Name string `json:"name"`
Age int `json:"age"`
Random map[string]interface{} `json:"-"`
}
// no need for the tags and helper fields anymore
type Person struct {
Name string
Age int
Statuses map[string]Status // Status type doesn't change
}
func (p *Person) UnmarshalJSON(data []byte) error {
i := intermediaryPerson{}
if err := json.Unmarshal(data, &i); err != nil {
return err
}
if err := json.Unmarshal(data, &i.Random); err != nil {
return err
}
delete(i.Random, "name")
delete(i.Random, "age")
stat, err := json.Marshal(i.Random)
if err != nil {
return err
}
// copy known fields
p.Name = i.Name
p.Age = i.Age
return json.Unmarshal(stat, &p.Statuses) // set status fields
}
In cases like this, it's common to create a type that handles the known fields and embed that, though:
type BasePerson struct {
Name string `json:"name"`
Age int `json:"age"`
}
and embed that in both the intermediary and the "main"/exported type:
type interPerson struct {
BasePerson
Random map[string]interface{} `json:"-"`
}
type Person struct {
BasePerson
Statuses map[string]Status
}
That way, you can just unmarshal the known fields directly into the BasePerson type, assign it, and then handle the map:
func (p *Person) UnmarshalJSON(data []byte) error {
base := BasePerson{}
if err := json.Unmarshal(data, &base); err != nil {
return err
}
p.BasePerson = base // takes care of all known fields
unknown := map[string]interface{}{}
if err := json.Unmarshal(data, unknown); err != nil {
return err
}
// handle status stuff same as before
delete(unknown, "name") // remove known fields
// marshal unknown key map, then unmarshal into p.Statuses
}
Demo 2
This is how I'd go about it. It allows for calls to json.Marshal and json.Unmarshal to look just like any other type, it centralises the handling of unknown fields in a single place (the implementation of the marshaller/unmarshaller interface), and leaves you with a single Person type where every field contains the required data, in a usable format. It's a tad inefficient in that it relies on unmarshalling/marshalling/unmarshalling the unknown keys. You could do away with that, like I said, using type assertions and iterating over the unknown map instead, faffing around with something like this:
for k, v := range unknown {
m, ok := v.(map[string]interface{})
if !ok {
continue // not {"status": bool}
}
s, ok := m["status"]
if !ok {
continue // status key did not exist, ignore
}
if sb, ok := s.(bool); ok {
// ok, we have a status bool value
p.Statuses[k] = Status{
Status: sb,
}
}
}
But truth be told, the performance difference won't be that great (it's micro optimisation IMO), and the code is a tad too verbose to my liking. Be lazy, optimise when needed, not whenever
Type doesn't meet with your json value.
const jsonStream = `{
"name": "john",
"age": 23,
"bvu62fu6dq": {
"status": true
}
}`
For above json your code should look like below snnipet to work (some modifications in your existing code).
package main
import (
"encoding/json"
"fmt"
"io"
"log"
"strings"
)
const jsonStream = `{
"name": "john",
"age": 23,
"bvu62fu6dq": {
"status": true
}
}`
type bvu62fu6dq struct {
Status bool
}
type Person struct {
Name string `json:"name"`
Age int `json:"age"`
Status bvu62fu6dq `json:"bvu62fu6dq"`
}
func main() {
dec := json.NewDecoder(strings.NewReader(jsonStream))
for {
var person Person
if err := dec.Decode(&person); err == io.EOF {
break
} else if err != nil {
log.Fatal(err)
}
fmt.Println(person)
fmt.Println(person.Status)
}
}
Based on your json data you have to map with type fields.
Run code snippet
I have a situation where my API return []byte {"name":"Windows"} on Windows and {"myName":"Linux"} on Linux, both are reporting OS Name but the key (name/MyName) is changed based on OS. How to Marshal/Unmarshal it to a JSON struct. Where I have single ember that can hold OS Name.
type OsName struct {
Name string `json:"name"` //I want to map Myname as well to this member.
}
Note: above question is not about how to get OS name, its about how to map 2 different keys to a single Json member.
Probably not the best solution, but you can write an adapter that typecasts the struct in base to the OS:
type OsName struct {
Name string `json:"name"`
}
type LinuxOsName struct {
Name string `json:"myName"`
}
func adapt(o OsName) interface{} {
if o.Name == "Linux" {
l := LinuxOsName(o)
return &l
}
return &o
}
func print(o OsName) {
b, err := json.Marshal(adapt(o))
if err != nil {
fmt.Printf("Error: %s\n", err)
}
fmt.Println(string(b))
}
https://go.dev/play/p/_M1orrGIP9p.go
You should probably Unmarshall to a map[string]string{} and then go from there.
dataFromAPI := "{\"name\": \"Windows\"}"
m := map[string]string{}
err := json.Unmarshal([]byte(dataFromAPI), &m)
if err != nil {
panic(err)
}
if _, ok := m["name"]; ok {
fmt.Println(m["name"]) // or m["myName"]
}
I need to parse this json
{
"version": "1.1.29-snapshot",
"linux-amd64": {
"url": "https://origin/path",
"size": 7794688,
"sha256": "14b3c3ad05e3a98d30ee7e774646aec7ffa8825a1f6f4d9c01e08bf2d8a08646"
},
"windows-amd64": {
"url": "https://origin/path",
"size": 8102400,
"sha256": "01b8b927388f774bdda4b5394e381beb592d8ef0ceed69324d1d42f6605ab56d"
}
}
Keys like linux-amd64 are dynamic and theirs amount is arbitrary. I tried something like that to describe it and unmarshal. Obviously it doesn't work. Items is always empty.
type FileInfo struct {
Url string `json:"url"`
Size int64 `json:"size"`
Sha256 string `json:"sha256"`
}
type UpdateInfo struct {
Version string `json:"version"`
Items map[string]FileInfo
}
It's similar to this use case, but has no parent key items. I suppose I can use 3rd party library or map[string]interface{} approach, but I'm interested in knowing how to achieve this with explicitly declared types.
The rest of the parsing code is:
func parseUpdateJson(jsonStr []byte) (UpdateInfo, error) {
var allInfo = UpdateInfo{Items: make(map[string]FileInfo)}
var err = json.Unmarshal(jsonStr, &allInfo)
return allInfo, err
}
Look at the link I attached and you will realize that is not that simple as you think. Also I pointed that I interested in typed approach. Ok, how to declare this map[string]FileInfo to get parsed?
You can create a json.Unmarshaller to decode the json into a map, then apply those values to your struct: https://play.golang.org/p/j1JXMpc4Q9u
type FileInfo struct {
Url string `json:"url"`
Size int64 `json:"size"`
Sha256 string `json:"sha256"`
}
type UpdateInfo struct {
Version string `json:"version"`
Items map[string]FileInfo
}
func (i *UpdateInfo) UnmarshalJSON(d []byte) error {
tmp := map[string]json.RawMessage{}
err := json.Unmarshal(d, &tmp)
if err != nil {
return err
}
err = json.Unmarshal(tmp["version"], &i.Version)
if err != nil {
return err
}
delete(tmp, "version")
i.Items = map[string]FileInfo{}
for k, v := range tmp {
var item FileInfo
err := json.Unmarshal(v, &item)
if err != nil {
return err
}
i.Items[k] = item
}
return nil
}
This answer is adapted from this recipe in my YouTube video on advanced JSON handling in Go.
func (u *UpdateInfo) UnmarshalJSON(d []byte) error {
var x struct {
UpdateInfo
UnmarshalJSON struct{}
}
if err := json.Unmarshal(d, &x); err != nil {
return err
}
var y map[string]json.RawMessage{}
if err := json.Unsmarshal(d, &y); err != nil {
return err
}
delete(y, "version"_ // We don't need this in the map
*u = x.UpdateInfo
u.Items = make(map[string]FileInfo, len(y))
for k, v := range y {
var info FileInfo
if err := json.Unmarshal(v, &info); err != nil {
return err
}
u.Items[k] = info
}
return nil
}
It:
Unmarshals the JSON into the struct directly, to get the struct fields.
It re-unmarshals into a map of map[string]json.RawMessage to get the arbitrary keys. This is necessary since the value of version is not of type FileInfo, and trying to unmarshal directly into map[string]FileInfo will thus error.
It deletes the keys we know we already got in the struct fields.
It then iterates through the map of string to json.RawMessage, and finally unmarshals each value into the FileInfo type, and stores it in the final object.
If you really don't want to unmarshal multiple times, your next best option is to iterate over the JSON tokens in your input by using the json.Decoder type. I've done this in a couple of performance-sensitive bits of code, but it makes your code INCREDIBLY hard to read, and in almost all cases is not worth the effort.
How can I do something Like this?
I am trying to pass a struct as a parameter to function in Go.
func handleEntityProperties(w http.ResponseWriter, r *http.Request) {
const sliceSize = 100
var entityProperties struct {
Instance string `json:"instance"`
Entities []struct {
Id string `json:"id"`
Properties map[string]string `json:"properties"`
Type string `json:"type"`
} `json:"entities"`
}
body, err := ioutil.ReadAll(r.Body)
if err != nil {
panic(err)
}
if !json.Valid([]byte(body)) {
fmt.Fprintf(w, "invalid json")
return
}
err = json.Unmarshal(body, &entityProperties)
sendData(entityProperties.Entities[0:100])
return
}
func sendData(entities struct) {
log.Println("Doing things with entities ", entities)
}
as you can see in code I am trying to send first 100 elements of entityProperties.Entities struct to a sendData. I know this is syntactically wrong.
Just declare your type outside of the functions:
type entity struct {
Id string `json:"id"`
Properties map[string]string `json:"properties"`
Type string `json:"type"`
}
And reuse it in handleEntityProperties() and in the signature of sendData():
func handleEntityProperties(w http.ResponseWriter, r *http.Request) {
const sliceSize = 100
var entityProperties struct {
Instance string `json:"instance"`
Entities []entity `json:"entities"`
}
body, err := ioutil.ReadAll(r.Body)
if err != nil {
panic(err)
}
if !json.Valid([]byte(body)) {
fmt.Fprintf(w, "invalid json")
return
}
err = json.Unmarshal(body, &entityProperties)
sendData(entityProperties.Entities[0:sliceSize])
return
}
func sendData(entities []entity) {
log.Println("Doing things with entities ", entities)
}
Also note that there is no guarantee that the client will send at least 100 entities, so you should check that else the slicing expression might result in a runtime panic:
max := 100
if len(entityProperties.Entities) < max {
max = len(entityProperties.Entities)
}
sendData(entityProperties.Entities[:max])
Also that check for invalid JSON is unnecessary: if the JSON is invalid, json.Unmarshal() will report a (non-nil) error and you'll know it.
Going further, you don't even have to read the complete body into memory (into a byte slice), you may use json.Decoder to read from it directly (without the intermediate memory buffer) like this:
dec := json.NewDecoder(r.Body)
if err := dec.Decode(&entityProperties); err != nil {
// handle error
}
And the final return statement is also unnecessary.
So an improved version may look like this:
func handleEntityProperties(w http.ResponseWriter, r *http.Request) {
var entityProperties struct {
Instance string `json:"instance"`
Entities []entity `json:"entities"`
}
dec := json.NewDecoder(r.Body)
if err := dec.Decode(&entityProperties); err != nil {
// handle error
http.Error(w, "invalid json", http.StatusBadRequest)
return
}
max := 100
if len(entityProperties.Entities) < max {
max = len(entityProperties.Entities)
}
sendData(entityProperties.Entities[:max])
}
Firestore returns map[string]interface{} while getting data. How can I render "details" values?
user:{
fname:"john",
lname:"con",
detail:{
address:"Delhi, India",
mob:"0000000009"
}
}
sn := snap.Data()
var bt []byte
for _, val := range sn {
for _, v := range val {
log.Println("value ", v)
}
}
Use json.Unmarshal to convert your JSON content to a map.
jsonString := `{"user":true,"lname":"con","detail":{"address":"Delhi, India","mob":"0000000009"}}`
aMap := make(map[string]interface{})
err := json.Unmarshal([]byte(jsonString), &aMap)
if err != nil {
fmt.Println("error:", err)
}
fmt.Printf("%+v\n", aMap)
fmt.Printf("Address := %s\n",aMap["detail"].(map[string]interface{})["address"])
https://play.golang.org/p/3133C_sKDf4
First things first. Your JSON appears to be invalid. It looks like a few
quotes are missing.
Try to validate your sample data here and you'll see what's wrong with it.
Alternatively to decode your JSON to a map, you can also unmarshal it into a struct as long as you know its structure in advance.
type User struct {
FirstName string `json:"fname"`
LastName string `json:"lname"`
Detail Detail `json:"detail"`
}
type Detail struct {
Address string `json:"address"`
Mobile string `json:"mob"`
}
if err := json.NewDecoder(strings.NewReader(out)).Decode(&u); err != nil {
log.Fatal(err)
}
Full working example