I am using duckling as a tool for date/time and unit extraction. When I send a request like today, I am getting (some fields have been removed for readability):
{"body":"today", "value": {"value":"2019-08-10T00:00:00.000-07:00", "grain":"day", "type":"value"}, "dim":"time"}
Thus, I created these structs:
type DucklingEntry struct {
Body string `json:"body"`
Dim string `json:"dim"`
Value DucklingValue `json:"value"`
}
type DucklingValue struct {
Value string `json:"value"`
Grain string `json:"grain"`
Type string `json:"type"`
}
When I send a request with the text value 6 euro, I am getting:
{"body":"6 euro", "value": {"value":6, "type":"value", "unit":"EUR"}, "dim":"amount-of-money"}
As you can see, the inner value field now contains an integer instead of a string. When I parse the json string using the shown struct, the inner value value will be "".
How can I account for such an inconsistent json response?
Related
I am trying to parse json object in golang.
Below is the json structure :
{
"properties": {
"alertType": "VM_EICAR",
"resourceIdentifiers": [{
"azureResourceId": "/Microsoft.Compute/virtualMachines/vm1",
"type": "AzureResource"
},
{
"workspaceId": "f419f624-acad-4d89-b86d-f62fa387f019",
"workspaceSubscriptionId": "20ff7fc3-e762-44dd-bd96-b71116dcdc23",
"workspaceResourceGroup": "myRg1",
"agentId": "75724a01-f021-4aa8-9ec2-329792373e6e",
"type": "LogAnalytics"
}
],
"vendorName": "Microsoft",
"status": "New"
}
}
I have below user defined types.
type AzureResourceIdentifier struct {
AzureResourceId string `json:"azureResourceId"`
Type string `json:"type"`
}
type LogAnalyticsIdentifier struct{
AgentId string `json:"agentId"`
Type string `json:"type"`
WorkspaceId string `json:"workspaceId"`
WorkspaceResourceGroup string `json:"workspaceResourceGroup"`
WorkspaceSubscriptionId string `json:"workspaceSubscriptionId"`
}
I have a top level user defined type as properties as below.
it has resourceIdentifiers as array of two other user defined types(defined above),
AzureResourceIdentifier
LogAnalyticsIdentifier
how can I define type of resourceIdentifiers in properties ?
type Properties struct{
alertType string
resourceIdentifiers ??? `
vendorName string `
status string
}
Note: I have a existing connector and we are provisioned to input the struct of the parsing json, we cannot override or add any function.
There are a couple of options here.
[]map[string]interface{}
Using a map will allow any JSON object in the array to be parsed, but burdens you with having to safely extract information after the fact.
[]interface{}
Each is going to be a map under the hood, unless non-JSON object elements can also be in the JSON array. No reason to use it over map[string]interface{} in your case.
A slice of a new struct type which covers all possible fields: []ResourceIdentifier
type ResourceIdentifier struct {
AzureResourceId string `json:"azureResourceId"`
AgentId string `json:"agentId"`
WorkspaceId string `json:"workspaceId"`
WorkspaceResourceGroup string `json:"workspaceResourceGroup"`
WorkspaceSubscriptionId string `json:"workspaceSubscriptionId"`
Type string `json:"type"`
}
This is probably the laziest solution. It allows you to get where you want to quickly, at the cost of wasting some memory and creating a non self-explanatory data design which might cause confusion if later trying to serialize with it again.
A new struct type + custom unmarshalling implementation.
type ResourceIdentifiers struct {
AzureResourceIdentifiers []AzureResourceIdentifier
LogAnalyticsIdentifiers []LogAnalyticsIdentifier
}
Implement json.Unmarshaler to decide which type of struct to construct and in which slice to put it.
I'm working on unmarshaling some nested json data that I have already written a struct for. I've used a tool that will generate a struct based off json data, but am a bit confused how to work with accessing nested json data (and fields can sometimes be emtpy).
Here is an example of struct:
type SomeJson struct {
status string `json:"status"`
message string `json:"message"`
someMoreData []struct {
constant bool `json:"constant,omitempty"`
inputs []struct {
name string `json:"name"`
type string `json:"type"`
} `json:"inputs,omitempty"`
Name string `json:"name,omitempty"`
Outputs []struct {
Name string `json:"name"`
Type string `json:"type"`
} `json:"outputs,omitempty"`
I'm able to unmarshal the json data and access the top level fields (such as status and message), but am having trouble accessing any data under the someMoreData field. I understand this field is a (I assume an unknown) map of structs, but only have experience working with basic single level json blobs.
For reference this is the code I have to unmarshal the json and am able to access the top level fields.
someData := someJson{}
json.Unmarshal(body, &someData)
So what is exactly the best to access some nested fields such as inputs.name or outputs.name?
to iterate over your particular struct you can use:
for _, md := range someData.someMoreData {
println(md.Name)
for _, out := range md.Outputs {
println(out.Name, out.Type)
}
}
to access specific field:
someData.someMoreData[0].Outputs[0].Name
Couple of things to note:
The struct definition is syntactically incorrect. There are couple of closing braces missing.
type is a keyword.
The status and message and other fields with lower case first letter fields are unexported. So, the Json parser will not throw error, but you will get zero values as output. Not sure that's what you observed.
someMoreData is an array of structs not map.
Suppose we want to unmarshal the JSON string {"e": "foo", "E": 1}.
Unmarshalling using the type messageUppercaseE works like expected. When using the type message though, the error json: cannot unmarshal number into Go struct field message.e of type string is returned.
Why are we not able to unmarshal the JSON, if only the "e" struct tag is present?
How would I be able to unmarshal the JSON? (I know that I am able to do this via Jeffail/gabs, but would like to stick to the type based approach.)
type message struct {
EventType string `json:"e"`
}
type messageUppercaseE struct {
EventType string `json:"e"`
UppercaseE uint64 `json:"E"`
}
Try it yourself at https://play.golang.org/p/T6KMJRLy7TN
Quoting the docs for unmarshal:
To unmarshal JSON into a struct, Unmarshal matches incoming object keys to the keys used by Marshal (either the struct field name or its tag), preferring an exact match but also accepting a case-insensitive match.
In this case, it is the case-insensitive match that causes the trouble.
I am using a JSON API for which I have to parse it into a struct. However, the API returns all values, even numbers, as strings and I need them to be in the format of numbers. So currently, I have a struct which has member fields which are all strings and after I have parsed the data, I loop through the entries to convert the values and add them to a new struct which has the specific entries in float or int values.
Is there any way to do the parsing and do type conversion in one go without having to use an intermediary struct representation from which to convert the values into the desired data types?
Example Code
str := []byte(`
{
"name": "Jim Burnham",
"age": "34",
"dob_day": "22",
"dob_month": "3",
"dob_year": "1984"
}
`)
Here is a sample JSON declaration of a response from an API. Notice how the age, day, month and year are returned as strings rather than integers. Now I declare a struct with the desired fields with JSON tags to map the values correctly:
type person struct {
Name string `json:"name"`
Age int `json:"age"`
DobDay int `json:"dob_day"`
DobMonth int `json:"dob_month"`
DobYear int `json:"dob_year"`
}
Then I declare an instance of the person struct and use the json package to unmarshal it into the instance of the struct:
var p person
_ = json.Unmarshal(str, &p)
fmt.Println(p)
But when I print out the person, the following output is generated:
{Jim Burnham 0 0 0 0}
As you can see, the string has been parsed correctly but the other integer fields remain at their default Golang initialized value. However, when I change the struct definition to :
type person struct {
Name string `json:"name"`
Age string `json:"age"`
DobDay string `json:"dob_day"`
DobMonth string `json:"dob_month"`
DobYear string `json:"dob_year"`
}
I get the following output:
{Jim Burnham 34 22 3 1984}
This means that currently, I have to define a raw struct which parses the information in the format of a string but then define another struct with the desired dtypes and reassign and convert the values separately, which produces untidy code as well. However, this is just one case but in my use case, there are likely thousands or even sometimes millions of such values and it seems to be inefficient, even for a compiled language. This is why I am asking for solutions for such a problem.
As explained well by #mkopriva at this link: https://play.golang.org/p/klHYlMQyb_V
Given the following JSON object:
{
"a": 1,
"b": [1,2,3,4]
}
And the following type:
type Thing struct {
A Int `json:"a"`
B string `json:"b"
}
I would like the Array "b" to stay as a JSON string when marshalled into go.
I currently get the following error:
panic: json: cannot unmarshal array into Go struct field Thing.b of type string
Set the field as a json.RawMessage. It'll be stored as is, without interpretation (ie. as "[1,2,3,4]"), as a slice of bytes, which can be converted to a string easily enough.
If you need a string directly, you'll have to implement the json.Unmarshaler interface on your type and do the conversion yourself.