I want to dynamically define structs in a Go project based on a JSON file.
For example, if I had a json file like so...
{
"date": "today",
"time": 12,
"era": "never",
"alive": true
}
Then I would expect a struct to be generated (that would look) like this (but not explicitly defined in the source code)...
type DynamicJSON struct {
date, era string
time int
alive bool
}
Furthermore, I want to nest JSON objects such that I could do something like this...
{
"date": "today",
"time": 12,
"era": "never",
"alive": true,
"nested": {
"date": "tomorrow",
"alive": true
}
}
...which would actually generate two different structs, like this...
type DynamicJSON1 struct {
date, era string
time int
alive bool
}
type DynamicJSON2 struct {
date string
alive bool
}
Is this something that is currently supported?
I can't garantantee final result, but easyjson does exactly what you asking.
easyjson aims to keep generated Go code simple enough so that it can
be easily optimized or fixed. Another goal is to provide users with
the ability to customize the generated code by providing options not
available with the standard encoding/json package, such as generating
"snake_case" names or enabling omitempty behavior by default.
Related
I have a sample JSON body that contains some strings. I want some of the strings to be converted into Mongo Extended JSON. For example, the JSON body passed in is the following:
{
"GuidBinary": "734cba69-4851-4869-8d0e-e870d6fb3065",
"DateTime": "12/12/2012",
"RegularString": "abcd"
}
And I want to convert this into
{
"GuidBinary": {
"$binary": {
"base64": "<payload>",
"subType": 0x03
}
},
"DateTime": {"$date": "<ISO-8601 Date/Time Format>"},
"RegularString": "abcd"
}
Is there a way to do this in Go, either through a package like mongo-go-driver or another method?
Yes, it's possible. It's part of the official mongo-go driver. Generating this extended JSON is published as the bson.MarshalExtJSON() function.
Example using it:
m := map[string]interface{}{
"GuidBinary": []byte{1, 2, 3, 4, 5},
"DateTime": time.Now(),
"RegularString": "abcd",
}
out, err := bson.MarshalExtJSON(m, false, false)
fmt.Println(string(out), err)
This will output (try it on the Go Playground):
{"DateTime":{"$date":"2009-11-10T23:00:00Z"},"RegularString":"abcd",
"GuidBinary":{"$binary":{"base64":"AQIDBAU=","subType":"00"}}} <nil>
So what you need to do is unmarshal your original JSON (using the encoding/json package), and you need to do some post processing on it: GuidBinary is a regular string in the input JSON, but it represents a UUID. You need to parse it into a UUID value (there are probably lots of libraries for this).
You also need to convert (parse) the DateTime which is also given as a JSON text in the input, but it represents a date. Use time.Parse() for this. And now you can call bson.MarshalExtJSON() to generate the extended JSON.
Im new in golang and i have a question.
I have 5 structs where i use Json , but the JSON file can have more structs than the ones i have predetermined BUT... the structures of the JSON satisfies the structures in my programing ( lets say i have 5 structs "struct1" , 6 structs "struct 2" , 1 struct "struct 3", and so on...)
My question is , i want to make a function were i take the JSON FILE , read the structs of it and have as an output the number of structs of the JSON file.
I think i could use the map[string]interface{} but i dont understand it.
I hope i have explain myself
Thank u very much!
Without example JSON or structs, the exact question you are asking is a bit hard to decipher, specifically the "output the number of structs" bit in the question, as it could be interpreted several different ways. I will do my best to answer what I think are the most probable questions you are asking.
Interfaces
First off, some basic go knowledge that might be useful, but outside JSON marshaling itself. The interface{} type appears special, but is not a hardwired keyword as it first might appear. What the interface keyword does is describe the requirements that an object must have to fulfill that interface. Because interface{} has no requirements, and because everything is automatically interfaced in go, everything satisfies the interface{} type.
Because of this implementation of interfaces, map[string]interface{} is really map[string]. This allows for the JSON un/marshal to not care about what is on the value side of the map. This exactly lines up with the format of JSON, where you have a string key on one side, and a value that could be any of the JSON datatypes on the other.
How many different objects are in the base JSON object?
let us take an example json
{
"debug": "on",
"window": {
"title": "Sample Konfabulator Widget",
"name": "main_window",
"width": 500,
"height": 500
},
"image": {
"src": "Images/Sun.png",
"name": "sun1",
"hOffset": 250,
"vOffset": 250,
"alignment": "center"
},
"text": {
"data": "Click Here",
"size": 36,
"style": "bold",
"name": "text1",
"hOffset": 250,
"vOffset": 100,
"alignment": "center",
"onMouseUp": "sun1.opacity = (sun1.opacity / 100) * 90;"
}
}
The answer to the question in this circumstance would be four. debug, window, image, and text.
The process for determining the number would then be:
Load the json into a byte array.
Marshal into an interface{}
Determine type (array vs object etc.) using type switch. see this A Tour of Go page
If you know the type, you can skip this step
Convert to desired type
Get length, or perform any other operation as desired.
package main
import (
"encoding/json"
"fmt"
)
func main() {
myJSON := `<see above>`
var outStruct *interface{}
json.Unmarshal([]byte(myJSON), &outStruct)
outMap := (*outStruct).(map[string]interface{})
fmt.Printf("Num Structs: %d", len(outMap))
}
Go Playground
How many json objects that I do not have structs for are present?
This answer has a very similar answer to the first one, and is really about manipulation of the output map and the struct
Taking almost the entire code from the first one to the second, let us assume that you have the following structs set up
type Image struct {
Name string
//etc
}
type Text struct {
Name string
//etc
}
type Window struct {
Name string
//etc
}
type Base struct {
Image Image
Window Window
Text Text
}
In this case, in addition to the previous steps, you would have to
5. Marshal the json into a base object
6. Go through the map[string]interface{}, and for each key
7. Determine if the key is one of the objects in your base struct
total := 0
for k, _ := range outMap {
if k != "image" && k != "text" && k != "window" && k != "other" {
total++
}
}
fmt.Printf("Total unknown structs: %d\n", total)
How many of my structs are empty?
This last question is also rather simple, and could be done by checking the map for a value given the input key, but for completion's sake, the example code marshals the JSON into a struct, and uses that.
Marshal JSON into base
For each of Window, Item, Text, Other in base, determine if empty.
total = 0
if (Image{}) == outBase.Image {
total++
}
if (Window{}) == outBase.Window {
total++
}
if (Text{}) == outBase.Text {
total++
}
if (Other{}) == outBase.Other {
total++
}
fmt.Printf("Total empty structs: %d\n", total)
Go Playground
See this go blog post for more information on golang JSON.
I have a struct such as this one:
type Data struct {
Id string
Value string
Custom customtype1
Special customtype2
TimeStamp Time
}
var model Data
I am reading data from a JSON object. Because the JSON is structured very differently, I can't just directly unmarshall the JSON into the struct. So I am trying to "match" the fields from the JSON objects to those of the struct one by one. I don't actually need to properly unmarshall the JSON data into the struct, all I really need is to be able to assign, for each field, the proper type to its value.
So I unmarshall the JSON to a generic interface, then convert it to a map[string]interface{} and iterate over that. For each field, I try to find a match among the field names in the model variable which I get using reflect.
Now this all works fine, but the problem arises when I try to get the right type for the values.
I can get the Type for a certain field from the model using reflect, but then I can't use that to cast the type of the value I get from the JSON because that is not a type. I can't use a switch statement either, because this is a simplified version of the situation and in reality I'm dealing with 1000+ different possible types. How can I convert the values I have for each field into their proper type ?
The only I can think of solving this would be to recreate a json string that matches the format of the struct and then unmarshall that into its proper struct, but that seems way to convoluted. Surely there must be a simpler way?
Here's a sample JSON (I can not change this structure, unless I rework it within my Go program):
{
"requestId": 101901,
"userName": "test",
"options": [1, 4],
"request": {
"timeStamp": {
"Value1": "11/02/2018",
"Value2": "11/03/2018"
},
"id": {
"Value1": "123abcd",
"Value2": "0987acd",
"Value3": "a9c003"
},
"custom": {
"Value1": "customtype1_value",
"Value2": "customtype1_value"
}
}
}
I'd advise against your current approach. You haven't provided enough context to tell us why you're choosing to unmarshall things one by one, but Go's support for JSON is good enough that I'd guess it is capable of doing what you want.
Are you aware of Marshall's support for struct tags? Those might serve the purpose you're looking for. Your struct would then look something more like:
type Data struct {
Id string `json:"id"`
Value string `json:"value"`
Custom customtype1 `json:"custom_type"`
Special customtype2 `json:"special_type"`
TimeStamp Time `json:"timestamp"`
}
If your problem is that the custom types don't know how to be unmarshalled, you can define custom unmarshalling functions for them.
This would then enable you to unmarshall an object like the following:
{
"id": "foo",
"value": "bar",
"custom_type": "2342-5234-4b24-b23a",
"special_type": "af23-af2f-rb32-ba23",
"timestamp": "2018-05-01 12:03:41"
}
I'm new to Go and working hard to follow its style and I'm not sure how to proceed.
I want to push a JSON object to a Geckoboard leaderboard, which I believe requires the following format based on the API doc and the one for leaderboards specifically:
{
"api_key": "222f66ab58130a8ece8ccd7be57f12e2",
"data": {
"item": [
{ "label": "Bob", "value": 4, "previous_value": 6 },
{ "label": "Alice", "value": 3, "previous_value": 4 }
]
}
}
My instinct is to build a struct for the API call itself and another called Contestants, which will be nested under item. In order to use json.Marshall(Contestant1), the naming convention of my variables would not meet fmt's expectations:
// Contestant structure to nest into the API call
type Contestant struct {
label string
value int8
previous_rank int8
}
This feels incorrect. How should I configure my Contestant objects and be able to marshall them into JSON without breaking convention?
To output a proper JSON object from a structure, you have to export the fields of this structure. To do it, just capitalize the first letter of the field.
Then you can add some kind of annotations, to tell your program how to name your JSON fields :
type Contestant struct {
Label string `json:"label"`
Value int8 `json:"value"`
PreviousRank int8 `json:"previous_rank"`
}
Consider that you get this JSON object:
{ id: 3, name: 'C' }
How can you tell if it's a Vitamin object or a Programming language object. Am I clear?
In typed languages, we simply understand the nature of the object (the Type of the object) from its very name. Each object has a name. How we might achieve something similar with JSON? How can we give names to JSON objects? Any established pattern?
If you have this problem it means that your serialization format is not properly defined. You should always be able to deserialize from a JSON object without any trouble.
There are two options:
Add a type field: You can't create the typed object because there are multiple types with the same set of field names. By adding a type field, there is no question about the underlying type of the object. With this option, you will have to go through the process of handling the type field when creating the real object.
Use context to decide what type the object is: If the query you submitted implies that only one type of object should be returned, then there is no problem. Alternately, if multiple types can be returned, the response format should group the objects by type.
{
"states": [ { id: 3, name: 'New York' } ],
"cities": [ { id: 4, name: 'New York' } ]
}
You have to add a field describing the type of the object. That way you can never have doubts about the type. Look for example at Google's Calendar API. Google's calendar resources all have a field "kind" describing the type.
A Calendar Event looks like:
{
"kind": "calendar#event",
"etag": etag,
"id": string,
"created": datetime,
"updated": datetime,
"summary": string,
...
}
A Calendar List Entry like:
{
"kind": "calendar#calendarListEntry",
"etag": etag,
"id": string,
"summary": string,
"description": string,
"location": string,
...
}
etc.
There's no way to do it for JSON fetched from somewhere else.
If you have control over the JSON, then you can do this:
Add a "type" field to each object.
Tailor make a JSON function to handle this. This can be done in two ways, one secure, one insecure.
Secure method
Create a function stringifyJSONType(). This one stringifies as usual, but adds a type parameter on-the-fly.
function stringifyJSONType(o){
o.type=o.constructor.name;
var s=JSON.stringify(o);
delete o.type; //To be clean and not modify the object.
return s;
}
Now, in the "secure" method, we have to create a switch-case for every type we expect for parsing. This only allows certain types (those which have been kept in the switch-case).
function parseJSONType(s){
var o=JSON.parse(s);
switch(o.type){
case "String":
o.__proto__=String;
break;
case "Date":
o.__proto__=Date;
break;
case "City": //Your custom object
o.__proto__=City;
break;
case "State": //Your custom object
o.__proto__=State;
break;
case "Country": //Your custom object
o.__proto__=Country;
break;
/*.... more stuff... */
case "Object":
default:
o.__proto__=Object;
break;
}
delete o.type;
return o;
}
Now, use these two methods just like JSON.parse() and JSON.stringify(), and it'll work. But for every new type you want to support, you'll have to add an extra case.
Insecure method
Not too insecure, just that it uses the nefarious eval() method. Which isn't too good.. As long as nobody else has the ability to add a custom type parameter to your JSON, it's OK, though.
Here, you use the same stringifyJSONType() as above, but use a different parse method.
function stringifyJSONType(o){
o.type=o.constructor.name;
var s=JSON.stringify(o);
delete o.type; //To be clean and not modify the object.
return s;
}
function parseJSONType(s){
var o=JSON.parse(s);
o.__proto__=eval(o.type);
delete o.type;
return o;
}
This has the advantage of not requiring switch-case and being easily extended to new types (no code changes required).