How to create custom key in json response using golang - json

I get a following json response through golang.
[
{
"CreatedOn": "03-22-2015",
"JSONReceived": [
{
"EzpOrderID": "ezp_126",
"FirstName": "Kumar",
"LastName": "S",
"OrderDesc": "Sample"
}
],
"Status": "0",
"id": "80acbdad-8aae-4d6c-ac63-2a02a9db64b4"
},
{
"CreatedOn": "03-22-2015",
"JSONReceived": [
{
"EzpOrderID": "ezp_126",
"FirstName": "Vasanth",
"LastName": "K",
"OrderDesc": "Sample"
}
],
"Status": "0",
"id": "8f7f52a5-793a-45bd-a9b7-ed41495e0ee3"
}
]..
But i need to create with key in response. sample response as follows. How to achieve using golang programming.
{
"returnResponseData": [{
"CreatedOn": "03-22-2015",
"JSONReceived": [{
"EzpOrderID": "ezp_126",
"FirstName": "Kumar",
"LastName": "S",
"OrderDesc": "Sample"
}],
"Status": "0",
"id": "80acbdad-8aae-4d6c-ac63-2a02a9db64b4"
}, {
"CreatedOn": "03-22-2015",
"JSONReceived": [{
"EzpOrderID": "ezp_126",
"FirstName": "Vasanth",
"LastName": "K",
"OrderDesc": "Sample"
}],
"Status": "0",
"id": "8f7f52a5-793a-45bd-a9b7-ed41495e0ee3"
}]
}
Please help me to achieve this task using golang.
Whole Source code as follows:
func orderList(w http.ResponseWriter, request *http.Request) {
rows, err := r.Table("orders").Run(session)
if err != nil {
fmt.Println(err)
return
}
var resultSet []interface{}
err = rows.All(&resultSet)
if err != nil {
fmt.Printf("Error scanning database result: %s", err)
return
}
if origin := request.Header.Get("Origin"); origin != "" {
w.Header().Set("Access-Control-Allow-Origin", origin)
w.Header().Set("Access-Control-Allow-Methods", "GET")
w.Header().Set("Access-Control-Allow-Headers",
"Accept, Content-Type, Content-Length, Accept-Encoding, X-CSRF-Token, Authorization")
}
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(200)
json.NewEncoder(w).Encode(resultSet)
}

You have to define another type like this:
type Wrapper struct {
ReturnResponseData []interface{} `json:"returnResponseData"`
}
and then encode wrapper, containing your response set:
json.NewEncoder(w).Encode(&Wrapper{ReturnResponseData: resultSet})
Notice, that you have to use property tag to achieve the name "returnResponseData", starting of a small letter (because encoder doesn't encode private properties by default).

Related

How to get all objects with jsonpaths in golang

I am using jsonpath in golang but I can't get all the objects of the following json that contain in type iPhone:
{
"firstName": "John",
"lastName": "doe",
"age": 26,
"address": {
"streetAddress": "naist street",
"city": "Nara",
"postalCode": "630-0192"
},
"phoneNumbers": [
{
"type": "iPhone",
"number": "0123-4567-8888"
},
{
"type": "home",
"number": "0123-4567-8910"
},
{
"type": "iPhone",
"number": "0123-4567-8910"
}
]}
I am working with golang and I know that the following jsonpath works:
$.phoneNumbers[?(#.type == "iPhone")]
The problem I have is that it is a service in which I have input a json path like the following:
$.[*].phoneNumbers[*].type
And the value that I have to look for, I am doing it in the following way:
values, err := jsonpath.Get(jsonPath, data)
for _, value := range values {
if err != nil {
continue
}
if value.(string) == "iPhone" {
}
}
At this point I cant get the output like:
[{
"type": "iPhone",
"number": "0123-4567-8888"
},
{
"type": "iPhone",
"number": "0123-4567-8888"
}]
I cant use the [?(#.)] format it is necessary to make with if.
Any idea?
Thanks
I cooked up an example using Peter Ohler's ojg package. Here's what the implementation looks like:
package main
import (
"fmt"
"github.com/ohler55/ojg/jp"
"github.com/ohler55/ojg/oj"
)
var jsonString string = `{
// Your JSON string
}`
func main() {
obj, err := oj.ParseString(jsonString)
if err != nil {
panic(err)
}
x, err := jp.ParseString(`$.phoneNumbers[?(#.type == "iPhone")]`)
if err != nil {
panic(err)
}
ys := x.Get(obj)
for k, v := range ys {
fmt.Println(k, "=>", v)
}
}
// Output:
// 0 => map[number:0123-4567-8888 type:iPhone]
// 1 => map[number:0123-4567-8910 type:iPhone]
Go Playground

Golang iterate over nested struct

i want to iterate over a nested json struct in golang. The issue is, i do not exactly know how nested the structure will be, becuase there are multiple jsons. In this case for example the output should be:
"available": false
"type": "foo"
"name": "foo"
"street": "foo"
....
Is that possible?
{
"informations": {
"available": false,
"provide": {
"informations": {
"customer": {
"type": "foo",
"address": {
"name": "foo",
"street": "foo",
"zipcode": "123",
"city": "foo"
}
}
}
}
}
}
I think you can do this with a map[string]interface{}. Because, as you said that you don't know exactly the structure of the JSON. But, again, you need to know the JSON structure each time while extracting data. Also, you need to do type assertion for inner map[string]interface{}. You can look at the answer from here for type assertion.
Here's the example code that I wrote:
package main
import (
"encoding/json"
"fmt"
)
var j = `{
"informations": {
"available": false,
"provide": {
"informations": {
"customer": {
"type": "foo",
"address": {
"name": "foo",
"street": "foo",
"zipcode": "123",
"city": "foo"
}
}
}
}
}
}`
func main() {
var data map[string]interface{}
err := json.Unmarshal([]byte(j), &data)
if err != nil {
fmt.Println(err.Error())
return
}
fmt.Println(data["informations"])
information := data["informations"]
provide := information.(map[string]interface{})["provide"]
fmt.Println(provide)
}
Output will be:
map[available:false provide:map[informations:map[customer:map[address:map[city:foo name:foo street:foo zipcode:123] type:foo]]]]
map[informations:map[customer:map[address:map[city:foo name:foo street:foo zipcode:123] type:foo]]]
Go Playground

Protobuf custom options not showing in JSON made by protojson library

I'm trying to extract Protobuf custom options from a FileDescriptorSet generated by the protoc compiler. I'm unable to do so using protoreflect. So, I tried to do so using the protojson library.
PS : Importing the Go-generated code is not an option for my use case.
Here's the Protobuf Message I'm testing with :
syntax = "proto3";
option go_package = "./protoze";
import "google/protobuf/descriptor.proto";
extend google.protobuf.FieldOptions {
string Meta = 50000;
}
extend google.protobuf.FileOptions {
string Food = 50001;
}
option (Food) = "cheese";
message X {
int64 num = 1;
}
message P {
string Fname = 1 [json_name = "FNAME"];
string Lname = 2 [json_name = "0123", (Meta) = "Yo"];
string Designation = 3;
repeated string Email = 4;
string UserID = 5;
string EmpID = 6;
repeated X z = 7;
}
// protoc --go_out=. filename.proto
Here's how far I got :
package main
import (
"fmt"
"io/ioutil"
"os/exec"
"google.golang.org/protobuf/encoding/protojson"
"google.golang.org/protobuf/proto"
"google.golang.org/protobuf/types/descriptorpb"
)
func main() {
exec.Command("protoc", "-oBinaryFile", "1.proto").Run()
Fset := descriptorpb.FileDescriptorSet{}
byts, _ := ioutil.ReadFile("File")
proto.Unmarshal(byts, &Fset)
byts, _ = protojson.Marshal(Fset.File[0])
fmt.Println(string(byts))
}
And here's the output JSON
{
"name": "1.proto",
"dependency": [
"google/protobuf/descriptor.proto"
],
"messageType": [
{
"name": "X",
"field": [
{
"name": "num",
"number": 1,
"label": "LABEL_OPTIONAL",
"type": "TYPE_INT64",
"jsonName": "num"
}
]
},
{
"name": "P",
"field": [
{
"name": "Fname",
"number": 1,
"label": "LABEL_OPTIONAL",
"type": "TYPE_STRING",
"jsonName": "FNAME"
},
{
"name": "Lname",
"number": 2,
"label": "LABEL_OPTIONAL",
"type": "TYPE_STRING",
"jsonName": "0123",
"options": {}
},
{
"name": "Designation",
"number": 3,
"label": "LABEL_OPTIONAL",
"type": "TYPE_STRING",
"jsonName": "Designation"
},
{
"name": "Email",
"number": 4,
"label": "LABEL_REPEATED",
"type": "TYPE_STRING",
"jsonName": "Email"
},
{
"name": "UserID",
"number": 5,
"label": "LABEL_OPTIONAL",
"type": "TYPE_STRING",
"jsonName": "UserID"
},
{
"name": "EmpID",
"number": 6,
"label": "LABEL_OPTIONAL",
"type": "TYPE_STRING",
"jsonName": "EmpID"
},
{
"name": "z",
"number": 7,
"label": "LABEL_REPEATED",
"type": "TYPE_MESSAGE",
"typeName": ".X",
"jsonName": "z"
}
]
}
],
"extension": [
{
"name": "Meta",
"number": 50000,
"label": "LABEL_OPTIONAL",
"type": "TYPE_STRING",
"extendee": ".google.protobuf.FieldOptions",
"jsonName": "Meta"
},
{
"name": "Food",
"number": 50001,
"label": "LABEL_OPTIONAL",
"type": "TYPE_STRING",
"extendee": ".google.protobuf.FileOptions",
"jsonName": "Food"
}
],
"options": {
"goPackage": "./protoze"
},
"syntax": "proto3"
}
So, data about my custom options showed up in the extensions. But what I really wanted was the value of those Custom Options in the "options" as well. (Which in my case was (Food) = "Cheese" and I want Cheese)
Can someone tell me how I can extract my custom options from the FileDescriptorSet using Protoreflect or by using Protojson.
I tried a lot to try and extract it using Protoreflect but failed !
Although not specifically an answer to how to get the custom options in a generated JSON, I believe I have an answer to what sounds like your underlying question: how to access the custom options without loading the generated Go code. This is thanks to dsnet's answer to my question on the golang issues board. Needless to say all the credit for this tricky solution goes to him. The punchline is to Marshal and then Unmarshal the options using a runtime-populated protoregistry.Types that actually knows about the custom options.
I made a complete demonstration of this approach working in this repo, and the key section (all the guts of which come from dsnet's example) is here:
func main() {
protogen.Options{
}.Run(func(gen *protogen.Plugin) error {
gen.SupportedFeatures = uint64(pluginpb.CodeGeneratorResponse_FEATURE_PROTO3_OPTIONAL)
// The type information for all extensions is in the source files,
// so we need to extract them into a dynamically created protoregistry.Types.
extTypes := new(protoregistry.Types)
for _, file := range gen.Files {
if err := registerAllExtensions(extTypes, file.Desc); err != nil {
panic(err)
}
}
// run through the files again, extracting and printing the Message options
for _, sourceFile := range gen.Files {
if !sourceFile.Generate {
continue
}
// setup output file
outputfile := gen.NewGeneratedFile("./out.txt", sourceFile.GoImportPath)
for _, message := range sourceFile.Messages {
outputfile.P(fmt.Sprintf("\nMessage %s:", message.Desc.Name()))
// The MessageOptions as provided by protoc does not know about
// dynamically created extensions, so they are left as unknown fields.
// We round-trip marshal and unmarshal the options with
// a dynamically created resolver that does know about extensions at runtime.
options := message.Desc.Options().(*descriptorpb.MessageOptions)
b, err := proto.Marshal(options)
if err != nil {
panic(err)
}
options.Reset()
err = proto.UnmarshalOptions{Resolver: extTypes}.Unmarshal(b, options)
if err != nil {
panic(err)
}
// Use protobuf reflection to iterate over all the extension fields,
// looking for the ones that we are interested in.
options.ProtoReflect().Range(func(fd protoreflect.FieldDescriptor, v protoreflect.Value) bool {
if !fd.IsExtension() {
return true
}
outputfile.P(fmt.Sprintf("Value of option %s is %s",fd.Name(), v.String()))
// Make use of fd and v based on their reflective properties.
return true
})
}
}
return nil
})
}
// Recursively register all extensions into the provided protoregistry.Types,
// starting with the protoreflect.FileDescriptor and recursing into its MessageDescriptors,
// their nested MessageDescriptors, and so on.
//
// This leverages the fact that both protoreflect.FileDescriptor and protoreflect.MessageDescriptor
// have identical Messages() and Extensions() functions in order to recurse through a single function
func registerAllExtensions(extTypes *protoregistry.Types, descs interface {
Messages() protoreflect.MessageDescriptors
Extensions() protoreflect.ExtensionDescriptors
}) error {
mds := descs.Messages()
for i := 0; i < mds.Len(); i++ {
registerAllExtensions(extTypes, mds.Get(i))
}
xds := descs.Extensions()
for i := 0; i < xds.Len(); i++ {
if err := extTypes.RegisterExtension(dynamicpb.NewExtensionType(xds.Get(i))); err != nil {
return err
}
}
return nil
}

Parse JSON in Go

This is an example of JSON output when calling 'ListObjects' for AWS S3
{
"Contents": [{
"ETag": "9e2bc2894b23742b7bb688c646c6fee9",
"Key": "DSC-0237.jpg",
"LastModified": "2017-09-06 21:53:15 +0000 UTC",
"Owner": {
"DisplayName": "demo-user",
"ID": "a9e2f170a6880f1d61852df8e523e88ca2a2b7abd093476cc93f1239ab5063c6"
},
"Size": 117904,
"StorageClass": "STANDARD"
}, {
"ETag": "\"9e2bc2894b23742b7bb688c646c6fee9\"",
"Key": "DSC-0238.jpg",
"LastModified": "2017-09-06 21:52:24 +0000 UTC",
"Owner": {
"DisplayName": "demo-user",
"ID": "a9e2f170a6880f1d61852df8e523e88ca2a2b7abd093476cc93f1239ab5063c6"
},
"Size": 117904,
"StorageClass": "STANDARD"
}, {
"ETag": "\"9e2bc2894b23742b7bb688c646c6fee9\"",
"Key": "DSC-0239.jpg",
"LastModified": "2017-09-06 21:53:01 +0000 UTC",
"Owner": {
"DisplayName": "demo-user",
"ID": "a9e2f170a6880f1d61852df8e523e88ca2a2b7abd093476cc93f1239ab5063c6"
},
"Size": 117904,
"StorageClass": "STANDARD"
}],
"IsTruncated": false,
"Marker": "",
"MaxKeys": 5,
"Name": "test-bucket-x011pp3",
"Prefix": ""
}
How do I parse this in Go? Chiefly I am interested in collecting:
Bucket Name
Key
Size
Owner's DisplayName
LastModified
I am coming from Python and in Python it would be something really simple like:
json_result = json.loads(json_string)
bucket_name = json_result['Name']
for idx, obj in enumerate(json_result['Contents']):
key = obj['Key']
size = obj['Size']
lastmod = obj['LastModified']
owner= obj['Owner']['DisplayName']
Thank you for the help!
It is something like this
package main
import (
"bytes"
"encoding/json"
"fmt"
"log"
)
var jsonData = []byte(`
{
"Contents": [{
"ETag": "9e2bc2894b23742b7bb688c646c6fee9",
"Key": "DSC-0237.jpg",
"LastModified": "2017-09-06 21:53:15 +0000 UTC",
"Owner": {
"DisplayName": "demo-user",
"ID": "a9e2f170a6880f1d61852df8e523e88ca2a2b7abd093476cc93f1239ab5063c6"
},
"Size": 117904,
"StorageClass": "STANDARD"
}, {
"ETag": "\"9e2bc2894b23742b7bb688c646c6fee9\"",
"Key": "DSC-0238.jpg",
"LastModified": "2017-09-06 21:52:24 +0000 UTC",
"Owner": {
"DisplayName": "demo-user",
"ID": "a9e2f170a6880f1d61852df8e523e88ca2a2b7abd093476cc93f1239ab5063c6"
},
"Size": 117904,
"StorageClass": "STANDARD"
}, {
"ETag": "\"9e2bc2894b23742b7bb688c646c6fee9\"",
"Key": "DSC-0239.jpg",
"LastModified": "2017-09-06 21:53:01 +0000 UTC",
"Owner": {
"DisplayName": "demo-user",
"ID": "a9e2f170a6880f1d61852df8e523e88ca2a2b7abd093476cc93f1239ab5063c6"
},
"Size": 117904,
"StorageClass": "STANDARD"
}],
"IsTruncated": false,
"Marker": "",
"MaxKeys": 5,
"Name": "test-bucket-x011pp3",
"Prefix": ""
}`,
)
type Response struct {
Contents []*Content
IsTruncated bool
Marker string
MaxKeys int
Name string
Prefix string
}
type Content struct {
ETag string
Key string
LastModified string
Owner *Owner
Size int
StroageClass string
}
type Owner struct {
DisplayName string
ID string
}
func main() {
resp := &Response{}
if err := json.NewDecoder(bytes.NewBuffer(jsonData)).Decode(resp); err != nil {
log.Fatal(err)
}
fmt.Printf("%#v", resp)
}
Try it in the Go Playground
And you should read this https://golang.org/pkg/encoding/json/
See https://golang.org/pkg/encoding/json
You annotate a struct with JSON fields that should be unmarshalled from the buffer of JSON, something like this:
type AWSObject struct {
Size int `json:"Size"`
Key string `json:"key"`
Owner AWSObjectOwner `json:"Owner"`
}
type AWSObjectOwner struct {
DisplayName `json:"DisplayName"`
}
var awsObjects []AWSObject
err = json.Unmarshal(jsonBuffer, &awsObjects)
if err != nil {
fmt.Printf("Error unmarshaling objects: " + err.Error() + "\n")
...
}
Using quicktype, I generated your model, marshaling code, and usage instructions:
// To parse this JSON data, add this code to your project and do:
//
// r, err := UnmarshalListObjectsResponse(bytes)
// bytes, err = r.Marshal()
package main
import "encoding/json"
func UnmarshalListObjectsResponse(data []byte) (ListObjectsResponse, error) {
var r ListObjectsResponse
err := json.Unmarshal(data, &r)
return r, err
}
func (r *ListObjectsResponse) Marshal() ([]byte, error) {
return json.Marshal(r)
}
type ListObjectsResponse struct {
Contents []Content `json:"Contents"`
IsTruncated bool `json:"IsTruncated"`
Marker string `json:"Marker"`
MaxKeys int64 `json:"MaxKeys"`
Name string `json:"Name"`
Prefix string `json:"Prefix"`
}
type Content struct {
ETag string `json:"ETag"`
Key string `json:"Key"`
LastModified string `json:"LastModified"`
Owner Owner `json:"Owner"`
Size int64 `json:"Size"`
StorageClass string `json:"StorageClass"`
}
type Owner struct {
DisplayName string `json:"DisplayName"`
ID string `json:"ID"`
}
As others suggested, it's probably best to use the AWS SDK for Go, but this method may be useful in general the next time you need to parse arbitrary JSON.

return NULL result from N1SQL in couchbase cbq cli

in couchbase, I have a document in beer-sample bucket
it 's like following
{
"uid": "kingarthur",
"email": "kingarthur#couchbase.com",
"interests": [
"Holy Grail",
"African Swallows"
]
}
while I try to get the result from my source code application, I can get the result. here is sample code
// Use query
query := gocb.NewN1qlQuery("SELECT * FROM `beer-sample` WHERE email=?")
email := `kingarthur#couchbase.com`
params := []interface{}{
email}
rows, err := bucket.ExecuteN1qlQuery(query, params)
if err != nil {
fmt.Println(err)
return
}
var row interface{}
for rows.Next(&row) {
fmt.Printf("Row: %v", row)
}
but if I run N1SQL command from the cbq cli, I always get null result, anyone know why?
resolve,
it is caused by the character ``, we must use the single quote, or double quote
cbq> SELECT * FROM `beer-sample` WHERE email='kingarthur#couchbase.com';
{
"requestID": "17507176-519e-4b42-9808-aa3b855f4fb9",
"signature": {
"*": "*"
},
"results": [
{
"beer-sample": {
"email": "kingarthur#couchbase.com",
"interests": [
"Holy Grail",
"African Swallows"
],
"uid": "kingarthur"
}
}
],
"status": "success",
"metrics": {
"elapsedTime": "7.399438091s",
"executionTime": "7.39925411s",
"resultCount": 1,
"resultSize": 266
}
}