Golang Gin Validation and SQLC - json

I am using SQLC to generate structures based on my schema and Gin for the web API.
After viewing this topic here: Golang how to use Validator with SqlC, I noticed the answer specifically says:
I wasn't able to change the Param structs but I guess using the table
main struct use the validation.
This is exactly the behavior I am after. I noticed that if I have a query that uses a subset of a given table, e.g.:
-- name: CreateUserBasic :one
INSERT INTO users (
username, email, phone
) VALUES (
$1, $2, $3
) RETURNING *;
It yields a structure as follows:
type CreateUserBasicParams struct {
Username string `json:"username"`
Email string `json:"email"`
Phone pgtype.Text `json:"phone"`
}
In my case, I even included the overrides for adding the "required" tag:
version: 2
sql:
- engine: "postgresql"
schema: [
...
]
queries: [
...
]
strict_function_checks: true
gen:
go:
emit_json_tags: true
sql_package: "pgx/v5"
package: "hmdb"
out: "hmdb"
overrides:
- db_type: users.username
go_struct_tag: validate:"required"
But this only modifies the table struct, not any of the param structs:
type User struct {
ID pgtype.UUID `json:"id"`
Email string `json:"email"`
Phone pgtype.Text `json:"phone"`
Username string `json:"username"`
Password pgtype.Text `json:"password"`
MFAToken pgtype.Text `json:"mfatoken"`
ActivationID pgtype.UUID `json:"activation_id"`
ActivationCode string `json:"activation_code"`
CreatedAt pgtype.Timestamptz `json:"created_at"`
ActivatedAt pgtype.Timestamptz `json:"activated_at"`
Active bool `json:"active"`
}
Using Gin, I want to invoke the CreateUserBasic with the received data and JSON can be directly Unmashall-ed into the CreateUserBasicParams structure, but unfortunately without the decorators on the params structure, missing fields are permitted during parsing and it seems I will need to re-create a new structure with a nearly identical layout to the CreateUserBasicParams struct, only with the proper tags:
func (c *Controller) Register(ctx *gin.Context) {
var params hmdb.CreateUserBasicParams
if err := ctx.ShouldBindWith(&params, binding.JSON); err != nil {
ctx.AbortWithError(http.StatusBadRequest, err)
}
log.Printf("Parameters:\n%+v", params)
q := hmdb.New(c.DBPool)
u, err := q.CreateUserBasic(ctx, params)
if err != nil {
ctx.AbortWithError(http.StatusBadRequest, err)
}
log.Printf("New User:\n%+v", u)
ctx.JSON(http.StatusOK, gin.H{
"message": "Good!",
})
}
How can I best apply arbitrary validation to the SQLC-generated "Params" structures without re-creating the structure or modifying the tags?

As you are using per-column overrides, the indentation level for "overrides" should match the version line in the yaml file. For example:
version: "1"
packages:
- name: "foo"
path: "bar"
.
.
.
overrides:
- column: "users.username"
go_struct_tag: validate:"required"
Documented here - https://docs.sqlc.dev/en/stable/reference/config.html?highlight=override#per-column-type-overrides

Related

Write specific JSON fields to file

I've just started studying Golang and don't understand how to write only specific JSON fields to an output file.
For example I have this struct:
type example struct {
Ifindex int `json:"ifindex"`
HostID int `json:"host_id"`
Hostname string `json:"hostname"`
Name string `json:"name"`
}
My output file should be in the following format:
[{"Ifindex": int, "Hostname": string}, {...}]
How can I do it?
If I understood correctly, you'd like to omit some of the fields when marshalling to JSON. Then use json:"-" as a field tag.
Per the json.Marshal(...) documentation:
As a special case, if the field tag is "-", the field is always omitted.
So you just need to use the tag "-" for any public field that you do not want serialized, for example (Go Playground):
type Example struct {
Ifindex int `json:"ifindex"`
HostID int `json:"-"`
Hostname string `json:"hostname"`
Name string `json:"-"`
}
func main() {
eg := Example{Ifindex: 1, HostID: 2, Hostname: "foo", Name: "bar"}
bs, err := json.Marshal(&eg)
if err != nil {
panic(err)
}
fmt.Println(string(bs))
// {"ifindex":1,"hostname":"foo"}
}

How to process JSON

I'm building a web server with Go and I don't know how to process JSON with Go.
Saying that I have a struct as below:
type User struct{
Id int
Name string
Password string
Status int
}
and now I have had an object of the struct User:
user := User{1, "Test", "password", 1}
Now I need to convert user to a JSON object. Here is what I've found:
b, err := json.Marshal(user)
fmt.Println(string(b))
It works well.
Now I want to do two things:
1) remove the Password from the JSON object
2) add a new filed: "code": 200 into the JSON object
What should I do?
If you want to keep the Password property accessible to outer packages, you can set a tag: json:"-" on it. As specified in the docs:
The encoding of each struct field can be customized by the format
string stored under the "json" key in the struct field's tag. The
format string gives the name of the field, possibly followed by a
comma-separated list of options. The name may be empty in order to
specify options without overriding the default field name.
The "omitempty" option specifies that the field should be omitted from
the encoding if the field has an empty value, defined as false, 0, a
nil pointer, a nil interface value, and any empty array, slice, map,
or string.
As a special case, if the field tag is "-", the field is always
omitted. Note that a field with name "-" can still be generated using
the tag "-,".
type User struct {
Id int
Name string
Password string `json:"-"`
Status int
Code int `json:"code"`
}
make Password lower case (and add Code int to your struct):
Try this:
package main
import (
"encoding/json"
"fmt"
)
func main() {
user := User{1, "Test", "password", 1, 200}
b, err := json.Marshal(user)
if err != nil {
panic(err)
}
fmt.Println(string(b))
}
type User struct {
Id int
Name string
password string
Status int
Code int
}
output:
{"Id":1,"Name":"Test","Status":1,"Code":200}

Merging two JSON strings in golang

I have a struct which I convert to JSON in the old fashioned way:
type Output struct {
Name string `json:"name"`
Command string `json:"command"`
Status int `json:"status"`
Output string `json:"output"`
Ttl int `json:"ttl,omitempty"`
Source string `json:"source,omitempty"`
Handlers []string `json:"handlers,omitempty"`
}
sensu_values := &Output{
Name: name,
Command: command,
Status: status,
Output: output,
Ttl: ttl,
Source: source,
Handlers: [handlers],
}
I want to read an arbitrary JSON file from the filesystem, which can be defined as anything by the user, and then add it to the existing JSON string, taking the duplicates from the original.
How can I do this?
Input JSON :
{
"environment": "production",
"runbook": "http://url",
"message": "there is a problem"
}
It's better to unmarshal the input JSON and combine the two structures before marshaling Output struct.
Sample Code
inputJSON := `{"environment": "production", "runbook":"http://url","message":"there is a problem"}`
out := map[string]interface{}{}
json.Unmarshal([]byte(inputJSON), &out)
out["name"] = sensu_values.Name
out["command"] = sensu_values.Command
out["status"] = sensu_values.Status
outputJSON, _ := json.Marshal(out)
Play Link

Unmarshalling neo4j results to a nested struct in Golang

I'm using the neoism library (https://github.com/jmcvetta/neois) to talk to a local neo4j database - I'm fairly new to go and am new to neo4j so the gap in my understanding could be on either side of the problem.
I have a simple database, a single "page" node which is related to a single "template" node. I was hoping to be able to have struct representing each node and nest them inside one another but I'm strugging to get this to work.
Creating the simple DB:
template, err := ioutil.ReadFile(viewPath + "templates/default.mustache")
if err != nil{
panic(err)
}
defaultTemplate, _ := db.CreateNode(neoism.Props{
"name": "default",
"content": string(template),
})
defaultTemplate.AddLabel("Template")
n0, _ := db.CreateNode(neoism.Props{
"name": "Home",
"slug": "home",
"title": "Home Page",
"content" : "here I am",
})
n0.AddLabel("Page")
n0.Relate("TEMPLATE", template.Id(), neoism.Props{})
Now to the business of trying to get the data back out...
Here is my query which works just fine:
type PageStruct struct{
Name string `json:"p.name"`
Slug string `json:"p.slug"`
Title string `json:"p.title"`
Content string `json:"p.content"`
TemplateName string `json:"t.name"`
TemplateContent string `json:"t.content"`
}
res := []PageStruct{}
cq := neoism.CypherQuery{
Statement: `
MATCH (p:Page)-[r:TEMPLATE]->(t:Template)
WHERE p.slug = {slug}
RETURN p.name, p.slug, p.title, p.content, t.name, t.content
`,
Parameters: neoism.Props{"slug": pageSlug},
Result: &res,
}
db.Cypher(&cq)
page := res[0]
But ideally what I want is to unmarshall into a nested struct, something like this:
type PageStruct struct{
Name string `json:"p.name"`
Slug string `json:"p.slug"`
Title string `json:"p.title"`
Content string `json:"p.content"`
Template struct {
Name string `json:"t.name"`
Content string `json:"t.content"`
} `json:"t"`
}
I've been trying various things with no success, could anyone give me any advice on how to achieve this abitious feat of computer engineering...
Additionally I'm slightly unclear as to how to handle relationships in neo4j in the sense of having no enforcement (that I'm aware of) as to the type of relationship (e.g one-to-one, one-to-many) so how to handle this - do we always just assume an array of data?
Any help/advice is greatly appreciated.
Use embedding.
type Page struct {
Name string `json:"p.name"`
// ...
Template
}
type Template struct {
Name string `json:"t.name"`
// ...
}
Playground: http://play.golang.org/p/B3ro3wgsGS.

When serializing to JSON, how to omit certain fields based on a run-time condition?

In a web service implemented in Go, I want to be able to restrict fields returned in a JSON response based on a user's role.
For example I may have a currently logged in user who has a role of guest and another with the role of admin
For an admin I want json to have all the keys eg
{
id: 1,
name: "John",
role: "admin"
}
and for a guest to not have the role key eg
{
id: 1,
name: "John"
}
I can currently marshal the json and it returns all fields. I need to be able to restrict it.
You can go by the suggestion #Volker made and clear struct fields for which the user has no permissions. This is probably the easiest to implement.
A second option in a similar vein is to create a custom JSON encoder. One which encodes fields only if a role struct tag matches the current user's role. Here is some pseudo code to illustrate:
type T struct {
currentRole Role `json:"-"`
FieldA string `json:"field_a,omitempty", role:"guest"`
FieldB string `json:"field_b,omitempty", role:"guest"`
FieldC int `json:"field_c,omitempty", role:"admin"`
}
// Have T implement the encoding/json.Marshaler interface.
func (t *T) MarshalJSON() ([]byte, error) {
var buf bytes.Buffer
// Use some reflection magic to iterate over struct fields.
for _, field := range getStructFields(t) {
// More reflection magic to extract field tag data.
role := getFieldTag(field, "role")
// If the field tag's role matches our current role,
// we are good to go. otherwise, skip this field.
if !matchingRole(role, t.currentRole) {
continue // skip this field
}
data, err := json.Marshal(fieldValue(field))
...
_, err = buf.Write(data)
...
}
return buf.Bytes(), nil
}
This is going to be a pain to maintain if you need new roles though. So this would not be something I would lightly consider doing.
Security concerns
I am not entirely sure that what you are looking for is the right solution to your problem. This depends on the context in which you use your code, which is not clear from your question. But if this concerns a website where a user's abilities on the website are defined solely by the value of the role JSON field, then you are looking at a security hole. They can simply go into a browser debugger and change the value of this JSON object to include the "role: "admin" field. And presto! Instant administrative powers. Whether or not to render certain parts of a page, based on user role, should really be handled by the server, during template processing. Just like any and all data posted to the server should be checked and checked again to ensure it came from a trusted source.
If none of this is applicable to you, then by all means, disregard this paragraph.
This question seems old, but I recently wanted to do the same thing. Maybe this will help someone in the future. Here is another method: you can define your own Marshal interface and use anonymous structs.
//User holds all variables
//even private ones
type User struct {
ID int64
Name string
Role string
}
//MarshalJSON gives back json user
//but only the public fields!
func (u *User) MarshalJSON() ([]byte, error) {
return json.Marshal(struct {
ID string `json:"id"`
Name string `json:"name"`
}{u.ID, u.Name})
}
it would be pretty easy to put an if u.Role == "admin" statement in the block to decide whether to marshal the rest.
Another option that also works to define the set of fields in the output for a list of struct that comes from an appengine datastore query.
// Setting different JSON output field for the same struct, using anonymous
// fields (inspired by inspired by http://choly.ca/post/go-json-marshalling/)
// This alternative could be used to load a resultset from an appengine datastore
// query and returned a custom field combination for the list items.
package main
import (
"encoding/json"
"fmt"
)
type User struct {
ID string `json:"id"`
Name string `json:"name"`
Role string `json:"-"`
LaunchCode string `json:"-"`
}
type AdminOutputUser User
func (user *AdminOutputUser) MarshalJSON() ([]byte, error) {
type Alias AdminOutputUser
return json.Marshal(&struct {
*Alias
Role string `json:"role"`
}{
(*Alias)(user),
user.Role,
})
}
type SuperadminOutputUser User
func (user *SuperadminOutputUser) MarshalJSON() ([]byte, error) {
type Alias SuperadminOutputUser
return json.Marshal(&struct {
*Alias
Role string `json:"role"`
LaunchCode string `json:"code"`
}{
(*Alias)(user),
user.Role,
user.LaunchCode,
})
}
func main() {
user := User{"007", "James Bond", "admin", "12345678"}
adminOutput := AdminOutputUser(user)
superadminOutput := SuperadminOutputUser(user)
b, _ := json.Marshal(&user)
fmt.Printf("%s\n\n", string(b))
// {"id":"007","name":"James Bond"}
b, _ = json.Marshal(&adminOutput)
fmt.Printf("%s\n\n", string(b))
// {"id":"007","name":"James Bond","role":"admin"}
b, _ = json.Marshal(&superadminOutput)
fmt.Printf("%s\n\n", string(b))
// {"id":"007","name":"James Bond","role":"admin","code":"12345678"}
}
// for appengine could do something like
// ...
// var users []AdminOutputUser // or User or SuperadminOutputUser
// q := datastore.NewQuery("User")
// keys, err := q.GetAll(ctx, &users)
// ...
https://play.golang.org/p/ignIz0hP0z
You might just define your struct like this
type User struct {
ID int64 `json:"id"`
Name string `json:"name"`
Role string `json:"role,omitempty"`
}
And then set them like this
normalUser := User{ID: "boring", Name: "Rubber"}
adminUser := User{ID: "powers", Name: "Ruler", Role: "admin"}
Then json.Marshal() or json.NewEncoder().Encode() as usual
Found in How To Use Struct Tags in Go
Note: I know that omitempty was mentioned in a comment and is even
part of #jimt's code example and mentioned as first option, albeit without a simple example. To me being still pretty new to
Go wasn't clear that that would just work as expected. So I figured it might help others as well 🤓