How to marshal JSON - json

I am trying to marshal JSON in a specific format using Go. I am looping over the JSON and printing the individual object response. What I want is to store all the objects according to a format. Right now I am stuck and marshaling JSON results like this:
{
"Address":null,
"Email":"abc#hotmail.com",
"HashedPassword":"4233137d1c510f2e55ba5cb220b864b11033f156",
"DeHashedPassword":"123456",
"ID":"Gd0YhYEJdE6oejsjBm7xLTQ4lWIaRecbS-k=",
"IPAddress":null,
"Name":null,
"ObtainedFrom":"LinkedIn",
"Password":null,
"Phone":null,
"Username":null,
"Vin":null,
"Success":true
}{
"Address":"",
"Email":"abc#hotmail.com",
"HashedPassword":"",
"DeHashedPassword":"123456",
"ID":"Jge4Mm6M-5-yJedG2ql48M9H2p7qP83aggM=",
"IPAddress":"",
"Name":"",
"ObtainedFrom":"DailyMotion.com",
"Password":"dm_51978c5a67a88",
"Phone":"",
"Username":"",
"Vin":"",
"Success":true
}{
"Address":"",
"Email":"abc#hotmail.com",
"HashedPassword":"",
"DeHashedPassword":"123456",
"ID":"9k8llNeinyrmxhL7yg3zZ50rQiQk_BmzZS8=",
"IPAddress":"",
"Name":"",
"ObtainedFrom":"BreachCompilation",
"Password":"hello123",
"Phone":"",
"Username":"",
"Vin":"",
"Success":true
}
What I want to get is to marshal json like this
{
"entries": [
{
"id": "CHzLLBdoJiwd7WaySw8QBOoxkj2lmKFhJK8=",
"email": "abc#hotmail.com",
"username": null,
"password": null,
"hashed_password": "4233137d1c510f2e55ba5cb220b864b11033f156",
"name": null,
"vin": null,
"address": null,
"ip_address": null,
"phone": null,
"obtained_from": "LinkedIn"
},
{
"id": "O6W3lxVMo_faf7MWoGGgkMb_CGcjo5vinFQ=",
"email": "abc#hotmail.com",
"username": "",
"password": "dm_51978c5a67a88",
"hashed_password": "",
"name": "",
"vin": "",
"address": "",
"ip_address": "",
"phone": "",
"obtained_from": "DailyMotion.com"
}
],
"success": true
}
Code is here in Go:
for i := 0; i < len(img.Entries); i++ {
address := img.Entries[i].Address
email1 := img.Entries[i].Email
hashedPassword := img.Entries[i].HashedPassword
deHashedPassword := "12233"
id := img.Entries[i].ID
iPAddress := img.Entries[i].IPAddress
name := img.Entries[i].Name
obtainedFrom := img.Entries[i].ObtainedFrom
password := img.Entries[i].Password
phone := img.Entries[i].Phone
username := img.Entries[i].Username
vin := img.Entries[i].Vin
success := img.Success
group := ColorGroup{
Address: address,
Email: email1,
HashedPassword: hashedPassword,
DeHashedPassword: deHashedPassword,
ID: id,
IPAddress: iPAddress,
Name: name,
ObtainedFrom: obtainedFrom,
Password: password,
Phone: phone,
Username: username,
Vin: vin,
Success: success,
}
b, err := json.Marshal(group)
if err != nil {
fmt.Println("error:", err)
}
ab := string(b)
fmt.Println("New json", ab)
}

What you need is adding all the results to a slice, then marshal a map or struct with the key "entries" pointing to that slice.
Your code should look like this
groups := make([]ColorGroup, 0)
for i := 0; i < len(img.Entries); i++ {
address := img.Entries[i].Address
email1 := img.Entries[i].Email
hashedPassword := img.Entries[i].HashedPassword
deHashedPassword := "12233"
id := img.Entries[i].ID
iPAddress := img.Entries[i].IPAddress
name := img.Entries[i].Name
obtainedFrom := img.Entries[i].ObtainedFrom
password := img.Entries[i].Password
phone := img.Entries[i].Phone
username := img.Entries[i].Username
vin := img.Entries[i].Vin
success := img.Success
group := ColorGroup{
Address: address,
Email: email1,
HashedPassword: hashedPassword,
DeHashedPassword: deHashedPassword,
ID: id,
IPAddress: iPAddress,
Name: name,
ObtainedFrom: obtainedFrom,
Password: password,
Phone: phone,
Username: username,
Vin: vin,
Success: success,
}
groups = append(groups, group)
}
b, err := json.Marshal(map[string]interface{}{
"entries": groups,
})
if err != nil {
fmt.Println("error:", err)
}
fmt.Println("New JSON\n", string(b))
Also to change the naming of the Marshaled fields, don't forget to name the fields with json tag like so
type ColorGroup struct {
ID int `json:"id"`
Address string `json:"address"`
Email string `json:"email"`
HashedPassword string `json:"hashed_password"`
DeHashedPassword string `json:"de_hashed_password"`
IPAddress string `json:"ip_address"`
Name string `json:"name"`
ObtainedFrom string `json:"obtained_from"`
Password string `json:"password"`
Phone string `json:"phone"`
Username string `json:"username"`
Vin string `json:"vin"`
Success bool `json:"success"`
}

Related

Access data json in golang

I have the following JSON document:
{
id: int
transaction_id: string
total: string
line_items: [
{
id: int
name: string
}
]
}
This is my code
type Order struct {
ID int `json:"id"`
TransactionId string `json:"transaction_id"`
Total string `json:"total"`
LineItems []interface{} `json:"line_items"`
}
...
var order Order
json.Unmarshal([]byte(sbody), &order)
for index, a := range order.LineItems {
fmt.Println(a["name"])
}
I got the error:
invalid operation: cannot index a (variable of type interface{})
Should I create an Item struct?
Modify the LineItems field type to []map[string]interface{}.
package main
import (
"encoding/json"
"fmt"
)
func main() {
type Order struct {
ID int `json:"id"`
TransactionId string `json:"transaction_id"`
Total string `json:"total"`
LineItems []map[string]interface{} `json:"line_items"`
}
var order Order
err := json.Unmarshal([]byte(`{
"id": 1,
"transaction_id": "2",
"total": "3",
"line_items": [
{
"id": 2,
"name": "444"
}
]
}`), &order)
if err != nil {
panic(err)
}
for _, a := range order.LineItems {
fmt.Println(a["name"])
}
}

How to catch nulls and give defaults to an entire array(Transform function in Dataweave)?

I used an elseif function to prevent my API from failing due to null values.
"4435555555"
else if ( payload.ship_address.phone as String != "")
payload.ship_address.phone
else
"4435555555"
var cust_name = if ( payload.ship_address.name == null)
"Client Customer"
else if (payload.ship_address.name as String == "")
"Client Customer"
else
payload.ship_address.name
I know I could do this for every value by doing a ton of functions but is there a way to do this for the entire array in one function?
var telephoneNumber = if ( payload.ship_address.phone == null)
"4435555555"
else if ( payload.ship_address.phone as String != "")
payload.ship_address.phone
else
"4435555555"
var cust_name = if ( payload.ship_address.name == null)
"Client Customer"
else if (payload.ship_address.name as String == "")
"Client Customer"
else
payload.ship_address.name
---
{
order: {
client_identifier: payload.po,
brand: "CLIENT_BRAND",
shipping_method: shippingMethod default "Standard",
price_cents: floor (payload.total as Number * 100),
channel: "rts_dropship",
tax_cents: 0,
shipping_cost_cents: 0,
shipping_tax_cents: 0,
billing_address: {
name: cust_name default "Client Customer",
telephone_number: telephoneNumber,
street: payload.ship_address.line1,
city: payload.ship_address.city,
state: payload.ship_address.state,
zip_code: payload.ship_address.postal_code,
country: payload.ship_address.country,
street2: payload.ship_address.line2,
company: payload.ship_address.company
},
shipping_address: {
name: cust_name default "Client Customer",
telephone_number: telephoneNumber,
street: payload.ship_address.line1,
city: payload.ship_address.city,
state: payload.ship_address.state,
zip_code: payload.ship_address.postal_code,
country: payload.ship_address.country,
street2: payload.ship_address.line2,
company: payload.ship_address.company
},
order_items: payload.items map ( item , indexOfItem ) -> {
client_identifier: item.id,
sku: item.quants.vpc[0],
condition: "N",
quantity: item.quants.quantity[0] as Number,
price_cents: floor ((item.unit_cost as Number default 0) * 100),
tax_cents: 0,
shipping_cost_cents: 0,
shipping_tax_cents: 0,
brand: item.brand_name,
comments: payload.vendor_notes default ""
},
packing_slip_identifier: payload.items.quants[0].ositems[0].order_channel_refnum[0]
}
}
You can create a Helper function using the isEmpty function and use it like you use default keyword.
%dw 2.0
output application/json
fun ifEmptyThen(string, defaultValue) =
if(isEmpty(string)) defaultValue
else string
---
{
nonEmptyValue: "nonEmpty" ifEmptyThen "defaultForNonEmpty",
blankString: "" ifEmptyThen "defaultForBlankString",
nullValue: null ifEmptyThen "defaultForNull"
}
The output of this will look like:
{
"nonEmptyValue": "nonEmpty",
"blankString": "defaultForBlankString",
"nullValue": "defaultForNull"
}

How to query the int64 filed with between and in Go/gorm?

I am new to Go, is there any one who knows how to query the int64 filed with between and in Go/gorm?
I would like to use below sql select * from porject where created_at between ? and ?;, but I don't know how to do it? thanks so much for any advice.
type Project struct {
Id int64 `json:"id" gorm:"primary_key;AUTO_INCREMENT;comment:'id'"`
Email string `json:"email" gorm:"type:varchar(255);comment:'email'"`
CreatedAt int64 `json:"created_at" gorm:"type:bigint(20);comment:'created_time'"`
}
func (p *Project) ProjectList(pageInfo cts.PageInfo) ([]Project, int, error) {
if p.Email != "" {
db = db.Where("email like ?", "%"+p.Email+"%")
}
if p.CreatedAt != 0 {
db = db.Where("created_at <> ?", p.CreatedAt)
}
err1 := db.Offset((pageInfo.Page - 1) * pageInfo.PageSize).Limit(pageInfo.PageSize).Order("id desc").Find(&List).Error
db.Count(&total)
return List, total, err1
}
I need to pass front end to query data by below json
{
"page": 1,
"page_size": 20,
"email":"test#test",
"create_at_st": 1654176767,
"create_at_end": 1654896767
}
or
{
"page": 1,
"page_size": 20,
"email":"test#test",
"create_at": ['1654176767','1654896767']
}
or other json, as long as it can achieve the desired effect.
I have modified your query ,assuming you have s.CreatedAtMin and s.CreatedAtMax with you .Then you can have something like
db.Where("created_at between ? and ?", min, max)
func (p *Project) ProjectList(pageInfo cts.PageInfo) ([]Project, int, error) {
if p.Email != "" {
db = db.Where("email like ?", "%"+s.Email+"%")
}
if p.CreatedAt != 0 {
//db = db.Where("created_at <> ?", s.CreatedAt)
db.Where("created_at between ? and ?", s.CreatedAtMin, s.CreatedAtMax)
}
err1 := db.Offset((pageInfo.Page - 1) * pageInfo.PageSize).Limit(pageInfo.PageSize).Order("id desc").Find(&List).Error
db.Count(&total)
return List, total, err1
}

Prepare a json object from unmarshaled data

I have json data like this:
json: {"opt1":200,"opt3":"1","opt4":"13","opt5":null,"products":[{"product_id":1,"price":100,"variant_id":100},{"product_id":1,"price":100,"variant_id":null}]}
I have structured it using
type Products struct {
Product_id int
Price json.Number
Variant_id int
}
type Pdata struct {
Products []Products `json:"products"`
}
Then I use unmarshal
jsonb := []byte(jsonVal)
var data Pdata
err := json.Unmarshal(jsonb, &data)
if err != nil {
fmt.Println(err)
return
}
And get output like
{[{1 100 100} {2 100 0}]}
Now I need to convert that data into a json object like this
{"purchased_products": [{"product_id": 1,"price": 1200,"variation_id": 100},{"product_id": 2,"price": 100,"variation_id": null}]}
After that, I need to assign it to "json"
var d = map[string]string{
"json": jsonVal,
"created_at": time.Now().Format("2006-01-02 15:04:05"),
"updated_at": time.Now().Format("2006-01-02 15:04:05"),
}
How can I do it?
Create a type (eg : PurchasedProducts) as below.
type PurchasedProducts struct {
Products []Products `json:"purchased_products"`
}
And init a PurchasedProducts type variable and assign your unmarshaled products to Purchased products as below.
pProducts := PurchasedProducts{Products: data.Products}
jsonByte, err := json.Marshal(pProducts)
if err != nil {
fmt.Println(err)
return
}
And convert that []byte array to a string and assign it to the map like below.
var d = map[string]string{
"json": string(jsonByte),
"created_at": time.Now().Format("2006-01-02 15:04:05"),
"updated_at": time.Now().Format("2006-01-02 15:04:05"),
}
You can run and see full code here.
Simply define two more structs that model the second JSON object:
type Pdata2 struct {
PurchasedProducts []Product2
}
type Product2 struct {
Product_id int
Price json.Number
Variation_id *int // pointer to int
}
The Variation_id field is an *int type because your required output JSON shows "variation_id": null. If you declare the field as simple int, its zero value will be marshaled to 0.
Then initialize those structs using values from the previous ones:
func main() {
data2 := Pdata2{
PurchasedProducts: make([]Product2, len(data.Products)),
}
for i, p := range data.Products {
data2.PurchasedProducts[i] = Product2{
Product_id: p.Product_id,
Price: p.Price,
Variation_id: nullableInt(p.Variant_id),
}
}
b, err := json.Marshal(data2)
if err != nil {
// ... handle error
}
var d = map[string]string{
"json": string(b),
// ...
}
fmt.Println(d)
}
func nullableInt(n int) *int {
if n == 0 {
return nil
}
return &n
}
Playground: https://play.golang.org/p/xhsmHNBjRKN
For nullable fields you can use pointers, so for example if the variant_id json field can be an integer or a json null, and you want to retain that information, then you can change Variant_id int to Variant_id *int.
type Product struct {
Product_id int `json:"product_id"`
Price json.Number `json:"price"`
Variant_id *int `json:"variant_id"`
}
To change json field names between unmarshal and marshal you can declare a second Products struct with the same fields as the original but with struct tags defining the desired field names, then, if the structs are, in all other respects, equivalent you can convert between them.
type Product struct {
Product_id int `json:"product_id"`
Price json.Number `json:"price"`
Variant_id int `json:"variant_id"`
}
type PurchasedProduct struct {
Product_id int `json:"product_id"`
Price json.Number `json:"price"`
Variant_id int `json:"variation_id"` // here variant_id becomes variation_id
}
Then, if p is of type Product, you can simply convert it to PurchasedProduct like so:
pp := PurchasedProduct(p)
To offload the conversion to the marshaling process you can have the original types implement the json.Marshaler interface and do the conversion there.
func (p Product) MarshalJSON() ([]byte, error) {
type P struct {
Product_id int `json:"product_id"`
Price json.Number `json:"price"`
Variant_id *int `json:"variation_id"`
}
return json.Marshal(P(p))
}
With the above you can do the following:
func main() {
// unmarshal
var pd Pdata
err := json.Unmarshal(data, &pd)
if err != nil {
panic(err)
}
// marshal
out, err := json.MarshalIndent(pd, "", " ")
if err != nil {
panic(err)
}
fmt.Println(string(out))
}
https://play.golang.org/p/0gnrjgUslza
Here you are.
Assumptions are that:
Product is a model which could be much more complicated, thus it has dedicated structs. Thus transformation from Product to OutputProduct can be unit tested separately.
It is one time use application, not a part of the application which exposes an API. Otherwise it should be properly separated into layers and the output should be written as a structure.
package main
import (
"encoding/json"
"log"
"os"
"time"
)
type (
Product struct {
ProductID int `json:"product_id"`
VariantID int `json:"variant_id"`
Price json.Number
}
Products []Product
OutputProduct struct {
ProductID int `json:"product_id"`
VariantID int `json:"variation_id"`
Price json.Number
}
)
func (p Product) ToOutputProduct() OutputProduct {
return OutputProduct{
ProductID: p.ProductID,
VariantID: p.VariantID,
Price: p.Price,
}
}
func (p Products) ToOutputProducts() []OutputProduct {
outputProducts := make([]OutputProduct, len(p))
for i := 0; i < len(p); i++ {
outputProducts[i] = p[i].ToOutputProduct()
}
return outputProducts
}
func main() {
var inputJSON = `{"opt1":200,"opt3":"1","opt4":"13","opt5":null,"products":[{"product_id":1,"price":100,"variant_id":100},{"product_id":1,"price":100,"variant_id":null}]}`
var parsedInput struct {
Products Products
}
if err := json.Unmarshal([]byte(inputJSON), &parsedInput); err != nil {
log.Fatal(err)
}
var output = map[string]interface{}{
"json": map[string][]OutputProduct{
"purchased_products": parsedInput.Products.ToOutputProducts(),
},
"created_at": time.Now().Format("2006-01-02 15:04:05"),
"updated_at": time.Now().Format("2006-01-02 15:04:05"),
}
encoder := json.NewEncoder(os.Stdout)
encoder.SetIndent(" ", " ")
if err := encoder.Encode(output); err != nil {
log.Fatal(err)
}
}
Based on the suggestions from comments:
package main
import (
"encoding/json"
"log"
"time"
)
type Products struct {
Product_id int `json:"product_id"`
Price int `json:"price"`
Variant_id int `json:"variant_id"`
}
type ProductData struct {
Products []Products `json:"products"`
}
type Response struct {
Json json.RawMessage `json:"json"`
CreatedAt string `json:"created_at"`
UpdatedAt string `json:"updated_at"`
}
func main() {
productJson := `{
"opt1": 200,
"opt3": "1",
"opt4": "13",
"opt5": null,
"products": [
{ "product_id": 1, "price": 100, "variant_id": 100 },
{ "product_id": 1, "price": 100, "variant_id": null }
]
}`
productData := &ProductData{}
err := json.Unmarshal([]byte(productJson), &productData)
if err != nil {
panic(err)
}
b, err := json.Marshal(map[string]interface{}{"purchased_products": productData.Products})
if err != nil {
panic(err)
}
d := &Response{
Json: b,
CreatedAt: time.Now().Format("2006-01-02 15:04:05"),
UpdatedAt: time.Now().Format("2006-01-02 15:04:05"),
}
out, err := json.Marshal(d)
if err != nil {
panic(err)
}
log.Println(string(out))
}
Output:
2009/11/10 23:00:00 {"json":{"purchased_products":[{"product_id":1,"price":100,"variant_id":100},{"product_id":1,"price":100,"variant_id":0}]},"created_at":"2009-11-10 23:00:00","updated_at":"2009-11-10 23:00:00"}

sqlx missing destination name with MySQL

I have a query which returns result of mysql json_object function. But when I try to call my api I got an error
missing destination name json_object('id', vd.id, 'title', vd.title...
type Vacancies struct {
ID int `json:"id"`
Title string `json:"title"`
Logo string `json:"logo"`
Items []string `json:"items"`
}
func (r *repository) GetVacancies(ctx context.Context) []*Vacancies {
const queryString = "select json_object('id', vd.id, 'title', vd.title, 'logo', vd.logo, 'items', json_array((select GROUP_CONCAT(json_object('id', id, 'title', title, 'description', description)) from vacancies as v where department_id = vd.id order by vd.sort))) from vacancy_departments as vd order by vd.sort"
defer utils.StartRelicDatastoreSegment(
ctx, newrelic.DatastoreMySQL, "Vacancies.GetVacancies", "Select", queryString,
).End()
var result []*Vacancies
if err := mysql.Client().Slave().Select(&result, queryString); err != nil {
logger.Get().Error("err while getting vacancies", zap.Error(err))
return nil
}
return result
}
How is it possible to get valid data for golang structure?