I need to create dynamic json i.e whose key value varies, below mentioned is the json
[{"email":"xxx#gmail.com","location":{"set":"Redmond"},"fname":{"set":"xxxxx"},"clicked_time":{"set":"zz"},"domain":{"add":"ttt"}},{"email":"zzz#gmail.com","location":{"set":"Greece"},"fname":{"set":"zzzzz"},"clicked_time":{"set":"zzz"},"domain":{"add":"zxxxx"}}]
I tried using below code:
rows := []map[string]string{}
if i > 0 {
row := make(map[string]string)
for j:=0;j<len(record);j++ {
key := header[j]
value := record[j]
row[key] = value
}
rows = append(rows, row)
}
How may I add set to location and add to domain to create a nested structure as map can have only one type string or nested structure?
Perhaps I have missed the point a little here, but I am not seeing why this is so dynamic in a way that can't be handled by a struct and the json unmarshal method.
Please see the following for an example
https://play.golang.org/p/8nrO36HQGhy
package main
import (
"encoding/json"
"fmt"
)
type (
Details struct {
Email string `json:"email"`
Location Entry `json:"location"`
FName Entry `json:"fname"`
ClickedTime Entry `json:"clicked_time"`
Domain Entry `json:"domain"`
}
Entry struct {
Set string `json:"set"`
Add string `json:"add"`
}
)
func main() {
d := []byte(`[{
"email": "xxx#gmail.com",
"location": {
"set": "Redmond"
},
"fname": {
"set": "xxxxx"
},
"clicked_time": {
"set": "zz"
},
"domain": {
"add": "ttt"
}
}, {
"email": "zzz#gmail.com",
"location": {
"set": "Greece"
},
"fname": {
"set": "zzzzz"
},
"clicked_time": {
"set": "zzz"
},
"domain": {
"add": "zxxxx"
}
}]`)
x := []Details{}
_ = json.Unmarshal(d, &x)
fmt.Printf("%+v\n", x)
}
Related
I am trying to create a JSON object on the fly based on the data received from the API.
Sample Data received: Unmarshalled the data into CiItems struct given below
{
"class_name": "test",
"configuration_items": [
{
"id": "ea09a24f-01ef-42ad-ab19-e0369341d9b3",
"ci_name": "makk",
"comments": null,
"created_by": "mike",
"updated_by": "sam",
"created": "2019-08-02T21:16:35.656Z",
"updated": "2019-08-02T21:21:08.073Z",
"ci_state_id": "randomid",
"super_ci_id": null,
"ci_attributes": [
{
"attribute_id": "c995c693-b97c-4863-a61b-81a5d904c967",
"df_attribute_value": "xsmall",
"attribute_name": "tname",
"data_type": "string"
},
{
"attribute_id": "58845f48-7d2a-4c8c-8591-eaf59a23d84d",
"df_attribute_value": "vmware",
"attribute_name": "provider",
"data_type": "string"
}
]}]}
Below are the structs created:
type Attribute struct {
AttributeID string `json:"attribute_id "`
DfAttributeValue string `json:"df_attribute_value"`
AttName string `json:"attribute_name"`
DataType string `json:"data_type"`
}
// Attributes - array of Attribute
type Attributes []Attribute
// CiItem - confiuraion item of a VM
type CiItem struct {
ID string `json:"ci_id"`
Created string `json:"created"`
Updated string `json:"updated"`
CreatedBY string `json:"created_by"`
UpdatedBY string `json:"updated_by"`
Atts Attributes `json:"ci_attributes"`
}
// CiItems - array of CiItem
type CiItems struct {
ClassName string `json:"class_name"`
Items []CiItem `json:"configuration_items"`
}
Code to unmarshal the data and create a extracted Json:
func (client *Client) GetList() (CiItems, error) {
var out CiItems
err := client.doJsonRequest("GET", &out)
log.Info(out)
if err != nil {
return out, err
}
var output map[string]interface{}
//parseMap(out.Items[0].(map[string]interface{}))
extractBase(out.Items[0], &output)
return out, nil
}
func extractBase(ci interface{}, output interface{}) {
fields := reflect.TypeOf(ci)
values := reflect.ValueOf(ci)
num := fields.NumField()
for i := 0; i < num; i++ {
field := fields.Field(i)
value := values.Field(i)
if string(field.Name) != "Atts" {
name := string(field.Name)
output[name] = string(value)
}
}
}
I am trying to create a JSON with key value of id, ci_name, created_by, updated_by, attribute_name as below
{
"ci_name": "makk",
"created_by": "mike",
"updated_by": "sam",
"created": "2019-08-02T21:16:35.656Z",
"updated": "2019-08-02T21:21:08.073Z",
"tname": "xsmall",
"provider": "vmware"
}
I have tried using reflect and other methods
Create a map using values from CiItem fields and return it from the function.
func extractBase(ci *CiItem) map[string]interface{} {
result := map[string]interface{}{
"ci_name": ci.Name,
"created_by": ci.CreatedBY,
"updated_by": ci.UpdatedBY,
"created": ci.Created,
"updated": ci.Updated,
}
for _, a := range ci.Atts {
result[a.AttName] = a.DfAttributeValue
}
return result
}
I have the following JSON array received as a web service output using CURL.
{
"total_rows": 4,
"offset": 0,
"rows": [
{
"id": "_design/people",
"key": "_design/people",
"value": {
"rev": "3-d707964c8a3fa0c0c71e51c600bbafb8"
}
},
{
"id": "aamir",
"key": "aamir",
"value": {
"rev": "3-4b9095435470366fb77df1a3d466bcff"
}
},
{
"id": "iyaan",
"key": "iyaan",
"value": {
"rev": "1-4fea2c459d85480bf4841c7e581180c0"
}
},
{
"id": "tahir",
"key": "tahir",
"value": {
"rev": "2-593c9237a96836a98f53c0374902964a"
}
}
]
}
I want to extract the "total_rows" object separately from it and "rows" object separately.
You just need the package encoding/json.
Defined Row struct:
type Row struct {
Id string `json:"id"`
Key string `json:"key"`
Value struct {
Rev string `json:"rev"`
} `json:"value"`
}
Defined Data Sturct:
type Data struct {
TotalRows int `json:"total_rows"`
Offset int `json:"offest"`
Rows []Row `json:"rows"`
}
And then use json.Unmarshal:
b := []byte("json string")
data := Data{}
if err := json.Unmarshal(b, &data); err != nil {
panic(err)
}
fmt.Println(data.TotalRows, data.Offset)
for _, row := range data.Rows {
fmt.Println(row.Id, row.Key)
}
As the other poster suggested, "encoding/json" will set you up with what you need. I will also recommend trying some of the third party libraries as they can maybe better fit your implementation. Initially I would suggest looking at:
https://github.com/buger/jsonparser
https://github.com/mailru/easyjson
These are just some quick suggestions and there are other libraries out there. Good luck!
I'd like to hide/show some fields of a model depending on User role.
What would be the most idiomatic way to implement it?
I don't really want to create N different types of the same model (where N is amount of User roles). Like:
UserEmployee, AdminEmployee, WhateverEmployee.
It would be perfect if there is some solution that uses the tags for it:
type Employee struct {
ID string `visibility:"admin,user"`
Name string `visibility:"admin,user"`
Salary int `visibility:"admin"`
}
jsonBytes, _ := someLib.Marshal(Employee{"1", "John", 5000}, "user")
fmt.Println(string(jsonBytes)) // {"id":"1","name":"John"}
The question is really pretty broad. I just wanted to know how you handle this situation or what is the most common way to do it in the Go community. I want clean and centralized (same for all models) solution that won't require to produce tons of duplicated code.
What have I tried before: I've just tried to use separate models for all cases and cast between them.
Create an empty struct of your type (Employee in this problem) that will hold the filtered data.
Use the reflect package to compare if the field tag contains the desired tag value (visibility role).
Copy values of base struct to our filter struct when we find a tag match and json marshal the output struct:
package main
import (
"encoding/json"
"fmt"
"reflect"
"strings"
)
type Employee struct {
ID string `visibility:"admin, hr, user" json:"id,omitempty"`
Name string `visibility:"admin, hr, user" json:"name,omitempty"`
Salary int `visibility:"admin, hr" json:"salary,omitempty"`
Password string `visibility:"admin" json:"password,omitempty"`
Rights map[string]bool `visibility:"admin" json:"rights,omitempty"`
Boss *Employee `visibility:"admin, hr" json:"boss,omitempty"`
}
func filterEmployee(emp Employee, role string) Employee {
var fEmployee Employee
ev := reflect.ValueOf(emp)
et := reflect.TypeOf(emp)
// Iterate through each field within the struct
for i := 0; i < ev.NumField(); i++ {
v := ev.Field(i)
t := et.Field(i)
roles := t.Tag.Get("visibility")
if strings.Contains(roles, role) {
switch i {
case 0: // ID
fEmployee.ID = v.String()
case 1: // Name
fEmployee.Name = v.String()
case 2: // Salary
fEmployee.Salary = int(v.Int())
case 3: // Password
fEmployee.Password = v.String()
case 4: // Rights
fEmployee.Rights = v.Interface().(map[string]bool)
case 5: // Boss
fEmployee.Boss = v.Interface().(*Employee)
}
}
}
return fEmployee
}
func main() {
e := Employee{
"1",
"Jack",
100000,
"password321",
map[string]bool{"create": false, "update": false},
&Employee{
"2",
"John",
120000,
"pwd",
map[string]bool{"create": true, "update": true},
nil,
},
}
fuser := filterEmployee(e, "user")
fhr := filterEmployee(e, "hr")
fadmin := filterEmployee(e, "admin")
buser, err := json.MarshalIndent(fuser, "", " ")
if err != nil {
fmt.Println(err)
}
fmt.Println("Filtering with role user: ")
fmt.Println(string(buser))
bhr, err := json.MarshalIndent(fhr, "", " ")
if err != nil {
fmt.Println(err)
}
fmt.Println("\nFiltering with role hr: ")
fmt.Println(string(bhr))
badmin, err := json.MarshalIndent(fadmin, "", " ")
if err != nil {
fmt.Println(err)
}
fmt.Println("\nFiltering with role admin: ")
fmt.Println(string(badmin))
}
Output:
Filtering with role user:
{
"id": "1",
"name": "Jack"
}
Filtering with role hr:
{
"id": "1",
"name": "Jack",
"salary": 100000,
"boss": {
"id": "2",
"name": "John",
"salary": 120000,
"password": "pwd",
"rights": {
"create": true,
"update": true
}
}
}
Filtering with role admin:
{
"id": "1",
"name": "Jack",
"salary": 100000,
"password": "password321",
"rights": {
"create": false,
"update": false
},
"boss": {
"id": "2",
"name": "John",
"salary": 120000,
"password": "pwd",
"rights": {
"create": true,
"update": true
}
}
}
Playground
EDIT: Updated answer for asker's request.
View the old playground for previous answer that ran into issues.
Old Playground
Use "omit empty"
type Employee struct {
ID string `json:",omitempty"`
Name string `json:",omitempty"`
Salary int `json:",omitempty"`
}
Your function can look like
func MarshallEmployee(e Employee, permission string) {
if permission == "user"{
e.Salary = 0
}
....marshall it
}
or you could also just not add the value to the struct in the first place. See the docs for more detail.
use this module: https://github.com/icoom-lab/marian/
package main
import (
"fmt"
"github.com/icoom-lab/marian"
)
type Account struct {
Id int `json:"id,omitempty" role:"admin"`
Name string `json:"name,omitempty" role:"admin,normal"`
}
func main() {
account := Account{
Id: 1,
Name: "Jhon",
}
fmt.Println(account)
// {id:1, Name:"Jhon"}
marian.CleanStruct("role", "admin", &account)
fmt.Println(account)
// {id:1, Name:"Jhon"}
marian.CleanStruct("role", "normal", &account)
fmt.Println(account)
// {Name:"Jhon"}
}
I'm trying to parse the first record in an embedded JSON array and create an object based on a subset of those properties. I have this working, but based on this question, I have to think there is a more elegant/less brittle way of doing this. For a bit more background, this is a result set from a call to the musicbrainz JSON web service, and I am treating the first artists record as the artist I am looking for.
The format of the JSON is like this:
{
"created": "2014-10-08T23:55:54.343Z",
"count": 458,
"offset": 0,
"artists": [{
"id": "83b9cbe7-9857-49e2-ab8e-b57b01038103",
"type": "Group",
"score": "100",
"name": "Pearl Jam",
"sort-name": "Pearl Jam",
"country": "US",
"area": {
"id": "489ce91b-6658-3307-9877-795b68554c98",
"name": "United States",
"sort-name": "United States"
},
"begin-area": {
"id": "10adc6b5-63bf-4b4e-993e-ed83b05c22fc",
"name": "Seattle",
"sort-name": "Seattle"
},
"life-span": {
"begin": "1990",
"ended": null
},
"aliases": [],
"tags": []
},
...
}
Here's the code I have so far. I'd like to be able to use my ArtistCollection type to get around some of the interface{} stuff, but I'm stuck as to how. I also don't want to bother with mapping all of the properties of the artist record, I'm only interested in the "name" and "id" values.
package main
import (
"fmt"
"encoding/json"
)
type Artist struct {
Id string
Name string
}
type ArtistCollection struct {
Artists []Artist
}
func main() {
raw := //json formatted byte array
var topLevel interface{}
err := json.Unmarshal(raw, &topLevel)
if err != nil {
fmt.Println("Uh oh")
} else {
m := topLevel.(map[string]interface{})
//this seems really hacky/brittle, there has to be a better way?
result := (m["artists"].([]interface{})[0]).(map[string]interface{})
artist := new(Artist)
artist.Id = result["id"].(string)
artist.Name = result["name"].(string)
fmt.Println(artist)
}
}
Requisite go playground link
Define a type that matches the structure of the JSON and unmarshal to a value of that type. I use an anonymous type below. Use an array of length one to grab the first artist record:
package main
import (
"encoding/json"
"fmt"
)
type Artist struct {
Id string
Name string
}
func main() {
raw := // JSON formatted byte array
var result struct {
Artists artist
}
err := json.Unmarshal(raw, &result)
if err != nil {
fmt.Println(err)
return
}
fmt.Printf("%#v\n", result.Artists[0])
}
playground
I'm trying to decode some json in Go but some fields don't get decoded.
See the code running in browser here:
What am I doing wrong?
I need only the MX records so I didn't define the other fields. As I understand from the godoc you don't need to define the fields you don't use/need.
// You can edit this code!
// Click here and start typing.
package main
import "fmt"
import "encoding/json"
func main() {
body := `
{"response": {
"status": "SUCCESS",
"data": {
"mxRecords": [
{
"value": "us2.mx3.mailhostbox.com.",
"ttl": 1,
"priority": 100,
"hostName": "#"
},
{
"value": "us2.mx1.mailhostbox.com.",
"ttl": 1,
"priority": 100,
"hostName": "#"
},
{
"value": "us2.mx2.mailhostbox.com.",
"ttl": 1,
"priority": 100,
"hostName": "#"
}
],
"cnameRecords": [
{
"aliasHost": "pop.a.co.uk.",
"canonicalHost": "us2.pop.mailhostbox.com."
},
{
"aliasHost": "webmail.a.co.uk.",
"canonicalHost": "us2.webmail.mailhostbox.com."
},
{
"aliasHost": "smtp.a.co.uk.",
"canonicalHost": "us2.smtp.mailhostbox.com."
},
{
"aliasHost": "imap.a.co.uk.",
"canonicalHost": "us2.imap.mailhostbox.com."
}
],
"dkimTxtRecord": {
"domainname": "20a19._domainkey.a.co.uk",
"value": "\"v=DKIM1; g=*; k=rsa; p=DkfbhO8Oyy0E1WyUWwIDAQAB\"",
"ttl": 1
},
"spfTxtRecord": {
"domainname": "a.co.uk",
"value": "\"v=spf1 redirect=_spf.mailhostbox.com\"",
"ttl": 1
},
"loginUrl": "us2.cp.mailhostbox.com"
}
}}`
type MxRecords struct {
value string
ttl int
priority int
hostName string
}
type Data struct {
mxRecords []MxRecords
}
type Response struct {
Status string `json:"status"`
Data Data `json:"data"`
}
type apiR struct {
Response Response
}
var r apiR
err := json.Unmarshal([]byte(body), &r)
if err != nil {
fmt.Printf("err was %v", err)
}
fmt.Printf("decoded is %v", r)
}
As per the go documentaiton about json.Unmarshal, you can only decode toward exported fields, the main reason being that external packages (such as encoding/json) cannot acces unexported fields.
If your json doesn't follow the go convention for names, you can use the json tag in your fields to change the matching between json key and struct field.
Exemple:
package main
import (
"fmt"
"encoding/json"
)
type T struct {
Foo string `json:"foo"`
priv string `json:"priv"`
}
func main() {
text := []byte(`{"foo":"bar", "priv":"nothing"}`)
var t T
err := json.Unmarshal(text, &t)
if err != nil {
panic(err)
}
fmt.Println(t.Foo) // prints "bar"
fmt.Println(t.priv) // prints "", priv is not exported
}
You must Uppercase struct fields:
type MxRecords struct {
Value string `json:"value"`
Ttl int `json:"ttl"`
Priority int `json:"priority"`
HostName string `json:"hostName"`
}
type Data struct {
MxRecords []MxRecords `json:"mxRecords"`
}
http://play.golang.org/p/EEyiISdoaE
The encoding/json package can only decode into exported struct fields. Your Data.mxRecords member is not exported, so it is ignored when decoding. If you rename it to use a capital letter, the JSON package will notice it.
You will need to do the same thing for all the members of your MxRecords type.