I have a large json file with multiple levels of nesting. Now I need to modify the value of each key in this file with the Go code. I know of two methods: the first is to obtain each key and then modify its value, but there is no doubt that this method is too complicated and prone to errors. The second method is to serialize the entire json file into a struct, then modify the struct field, and then deserialize it. However, this case needs to define a struct of several hundred lines, which is also very complicated.
Is there any other way?
for example my json is like this, but more bigger, 100+ lines :
{
"user": [{
"cdb_id":"",
"firstname":"Tom",
"lastname":"Bradley",
"phone":14155555555,
"email":"tom#gmail.com",
"address":[{
"street":"4343 shoemaker ave",
"city":"Brea",
"zip":"92821",
"country":"USA"
}],
"authenticators":[{
"name":"Lisa Hayden",
"phone":15625555555
},{
"name":"Pavan M",
"phone":17145555555
}],
"voice_sig":"242y5-4546-555kk54-437879ek545",
"voicesig_created_time":"2017-08-02T21:27:44+0000",
"status":"verified"
}]
}
I need modify "cdb_id"/"lastname"/"street"/"phone"/ "voice_sig".....all these keys' value, Except make a struct or get the keys' value one by one and modify, do i have any other way?
The new values for those keys will be POST request from Web Pages.
You can use json pointer:
https://godoc.org/github.com/go-openapi/jsonpointer
Or, you can read it in a map[string]interface{} and work your way through, but that gets tedious.
Now my way is use this website, switch my Json to struct, and modify one by one. But it just think this is not so good way, so i am eagering for a better way.
http://json2struct.mervine.net/
type MyJsonName struct {
User []struct {
Address []struct {
City string `json:"city"`
Country string `json:"country"`
Street string `json:"street"`
Zip string `json:"zip"`
} `json:"address"`
Authenticators []struct {
Name string `json:"name"`
Phone int `json:"phone"`
} `json:"authenticators"`
CdbID string `json:"cdb_id"`
Email string `json:"email"`
Firstname string `json:"firstname"`
Lastname string `json:"lastname"`
Phone int `json:"phone"`
Status string `json:"status"`
VoiceSig string `json:"voice_sig"`
VoicesigCreatedTime string `json:"voicesig_created_time"`
} `json:"user"`
}
Related
I am trying to parse json object in golang.
Below is the json structure :
{
"properties": {
"alertType": "VM_EICAR",
"resourceIdentifiers": [{
"azureResourceId": "/Microsoft.Compute/virtualMachines/vm1",
"type": "AzureResource"
},
{
"workspaceId": "f419f624-acad-4d89-b86d-f62fa387f019",
"workspaceSubscriptionId": "20ff7fc3-e762-44dd-bd96-b71116dcdc23",
"workspaceResourceGroup": "myRg1",
"agentId": "75724a01-f021-4aa8-9ec2-329792373e6e",
"type": "LogAnalytics"
}
],
"vendorName": "Microsoft",
"status": "New"
}
}
I have below user defined types.
type AzureResourceIdentifier struct {
AzureResourceId string `json:"azureResourceId"`
Type string `json:"type"`
}
type LogAnalyticsIdentifier struct{
AgentId string `json:"agentId"`
Type string `json:"type"`
WorkspaceId string `json:"workspaceId"`
WorkspaceResourceGroup string `json:"workspaceResourceGroup"`
WorkspaceSubscriptionId string `json:"workspaceSubscriptionId"`
}
I have a top level user defined type as properties as below.
it has resourceIdentifiers as array of two other user defined types(defined above),
AzureResourceIdentifier
LogAnalyticsIdentifier
how can I define type of resourceIdentifiers in properties ?
type Properties struct{
alertType string
resourceIdentifiers ??? `
vendorName string `
status string
}
Note: I have a existing connector and we are provisioned to input the struct of the parsing json, we cannot override or add any function.
There are a couple of options here.
[]map[string]interface{}
Using a map will allow any JSON object in the array to be parsed, but burdens you with having to safely extract information after the fact.
[]interface{}
Each is going to be a map under the hood, unless non-JSON object elements can also be in the JSON array. No reason to use it over map[string]interface{} in your case.
A slice of a new struct type which covers all possible fields: []ResourceIdentifier
type ResourceIdentifier struct {
AzureResourceId string `json:"azureResourceId"`
AgentId string `json:"agentId"`
WorkspaceId string `json:"workspaceId"`
WorkspaceResourceGroup string `json:"workspaceResourceGroup"`
WorkspaceSubscriptionId string `json:"workspaceSubscriptionId"`
Type string `json:"type"`
}
This is probably the laziest solution. It allows you to get where you want to quickly, at the cost of wasting some memory and creating a non self-explanatory data design which might cause confusion if later trying to serialize with it again.
A new struct type + custom unmarshalling implementation.
type ResourceIdentifiers struct {
AzureResourceIdentifiers []AzureResourceIdentifier
LogAnalyticsIdentifiers []LogAnalyticsIdentifier
}
Implement json.Unmarshaler to decide which type of struct to construct and in which slice to put it.
I'm working on unmarshaling some nested json data that I have already written a struct for. I've used a tool that will generate a struct based off json data, but am a bit confused how to work with accessing nested json data (and fields can sometimes be emtpy).
Here is an example of struct:
type SomeJson struct {
status string `json:"status"`
message string `json:"message"`
someMoreData []struct {
constant bool `json:"constant,omitempty"`
inputs []struct {
name string `json:"name"`
type string `json:"type"`
} `json:"inputs,omitempty"`
Name string `json:"name,omitempty"`
Outputs []struct {
Name string `json:"name"`
Type string `json:"type"`
} `json:"outputs,omitempty"`
I'm able to unmarshal the json data and access the top level fields (such as status and message), but am having trouble accessing any data under the someMoreData field. I understand this field is a (I assume an unknown) map of structs, but only have experience working with basic single level json blobs.
For reference this is the code I have to unmarshal the json and am able to access the top level fields.
someData := someJson{}
json.Unmarshal(body, &someData)
So what is exactly the best to access some nested fields such as inputs.name or outputs.name?
to iterate over your particular struct you can use:
for _, md := range someData.someMoreData {
println(md.Name)
for _, out := range md.Outputs {
println(out.Name, out.Type)
}
}
to access specific field:
someData.someMoreData[0].Outputs[0].Name
Couple of things to note:
The struct definition is syntactically incorrect. There are couple of closing braces missing.
type is a keyword.
The status and message and other fields with lower case first letter fields are unexported. So, the Json parser will not throw error, but you will get zero values as output. Not sure that's what you observed.
someMoreData is an array of structs not map.
I am using a JSON API for which I have to parse it into a struct. However, the API returns all values, even numbers, as strings and I need them to be in the format of numbers. So currently, I have a struct which has member fields which are all strings and after I have parsed the data, I loop through the entries to convert the values and add them to a new struct which has the specific entries in float or int values.
Is there any way to do the parsing and do type conversion in one go without having to use an intermediary struct representation from which to convert the values into the desired data types?
Example Code
str := []byte(`
{
"name": "Jim Burnham",
"age": "34",
"dob_day": "22",
"dob_month": "3",
"dob_year": "1984"
}
`)
Here is a sample JSON declaration of a response from an API. Notice how the age, day, month and year are returned as strings rather than integers. Now I declare a struct with the desired fields with JSON tags to map the values correctly:
type person struct {
Name string `json:"name"`
Age int `json:"age"`
DobDay int `json:"dob_day"`
DobMonth int `json:"dob_month"`
DobYear int `json:"dob_year"`
}
Then I declare an instance of the person struct and use the json package to unmarshal it into the instance of the struct:
var p person
_ = json.Unmarshal(str, &p)
fmt.Println(p)
But when I print out the person, the following output is generated:
{Jim Burnham 0 0 0 0}
As you can see, the string has been parsed correctly but the other integer fields remain at their default Golang initialized value. However, when I change the struct definition to :
type person struct {
Name string `json:"name"`
Age string `json:"age"`
DobDay string `json:"dob_day"`
DobMonth string `json:"dob_month"`
DobYear string `json:"dob_year"`
}
I get the following output:
{Jim Burnham 34 22 3 1984}
This means that currently, I have to define a raw struct which parses the information in the format of a string but then define another struct with the desired dtypes and reassign and convert the values separately, which produces untidy code as well. However, this is just one case but in my use case, there are likely thousands or even sometimes millions of such values and it seems to be inefficient, even for a compiled language. This is why I am asking for solutions for such a problem.
As explained well by #mkopriva at this link: https://play.golang.org/p/klHYlMQyb_V
I'm using this library to access CouchDB (cloudant to be specific) "github.com/mikebell-org/go-couchdb" and I've noticed a problem.
When I go to add a file to the database and pass in a struct, only the fields of the struct which started with a capital letter get added.
For example
type Person struct {
name string
Age int
}
func main() {
db, _ := couchdb.Database(host, database, username, password)
joe := Person{
name: "mike",
Age: 190,
}
m, _ := db.PostDocument(joe)
}
In this case, only the "age" field got updated and inserted into my database.
I've noticed this problem in another case also - when I'm doing something like this :
type Sample struct {
Name string
age int
}
joe := Sample{
Name: "xx",
age: 23,
}
byt, _ := json.Marshal(joe)
post_data := strings.NewReader(string(byt))
fmt.Println(post_data)
in this case, only Name would be printed out :
output : &{{"Name":"xx"} 0 -1}
Why is this? and If I would like to have a field with a lowercase and be inside the database, is that possible?
This is because only fields starting with a capital letter are exported, or in other words visible outside the curent package (and in the json package in this case).
Here is the part of the specifications refering to this: http://golang.org/ref/spec#Exported_identifiers
Still, you can unmarshall json fields that do no start with a capital letters using what is called "tags". With the json package, this is the syntax to use:
type Sample struct {
Name string `json:"name"`
Age int `json:"age"`
}
Refer to the documentation for more information about this.
json package only stringfiy fields start with capital letter.
see http://golang.org/pkg/encoding/json/
Struct values encode as JSON objects. Each exported struct field becomes a member of the object, using the field name as the object key, unless the field is omitted for one of the reasons given below.
You need define the struct like this:
type Sample struct{
Name string `json:"name"`
Age int `json:"age"`
}
json.Marshal method struct-in field-i only accepts fields that start with a capital letter
The json package only accesses the exported fields of struct types (those that begin with an uppercase letter). Therefore only the exported fields of a struct will be present in the JSON output.
type Sample struct {
Name string
Age int
}
I'm trying to pull reddit content from the json API into a custom structure for the client. the structure i've come up with in go for this is
type Subreddit struct {
offset int
num_of_posts int
subscribers: int
thumbnail string
children []post
}
type post struct {
type string
url string
thumbnail string
submitted_by string
upvotes int
downvotes int
}
unfortunately the reddit json isn't formatted even close to this and in addition i'll want to filter out url's i can't support etc.
The only way i know to do it this is to create an interface for each of the "children" in the source data, and iterate through each child manually, creating an individual "post" for each interface. and pushing them into the subreddit object's post array.
For reference the data is formatted like http://www.reddit.com/r/web_design/.json
Is this the right way to do this? Or is there a faster way. It seems like a lot of overhead for such a small task, but i'm a PHP Javascript dev, so It's just unusual for me I suppose.
Before I even start to answer the question:
Remember that your struct fields must be exported in order to be used with the encoding/json package.
Secondly I must admit I am not entirely sure what you meant with the entire create an interface for each of the "children" part. But it sounded complicated ;)
Anyway, to your answer:
If you wish to use the standard encoding/json package to unmarshal the json, you must use an intermediate structure unless you will use a similar structure as the one used by Reddit.
Below you can find an example of how parts of the Reddit structure might be mapped to Go structs. By Unmarshalling the json into an instance of RedditRoot, you can then easily iterate over the Children , remove any unwanted child, and populate your Subreddit struct:
type RedditRoot struct {
Kind string `json:"kind"`
Data RedditData `json:"data"`
}
type RedditData struct {
Children []RedditDataChild `json:"children"`
}
type RedditDataChild struct {
Kind string `json:"kind"`
Data *Post `json:"data"`
}
type Post struct {
Type string `json:"-"` // Is this equal to data.children[].data.kind?
Url string `json:"url"`
Thumbnail string `json:"thumbnail"`
Submitted_by string `json:"author"`
Upvotes int `json:"ups"`
Downvotes int `json:"downs"`
}