golang convert array of interfaces to strings - json

I read JSON data from a remote source and convert it to a map. There's some array in the data of which I want to examine the string values. After converting I think m["t"] is an array of interfaces. fmt.Print converts this to printed text on the console but I cannot figure a way to do a simple string comparison like
if val[0] == "str-c" {fmt.Println("success")}
How do I iterate through that and do string comparisons?
package main
import (
"fmt"
"encoding/json"
)
func main() {
var m map[string]interface{}
sJSON := `{"k": "v", "t":["str-a","str-b","str-c"]}`
_ = json.Unmarshal([]byte(sJSON),&m)
// find out if one of the string values of "t" is "str-b"
fmt.Println(m["t"])
}

m["t"] is of type interface{} and is the full array, if you wanted to get str-b it is at index one and you have to do some type assertion to get it as a string. Here's an example; https://play.golang.org/p/W7ZnMgicc7
If you want to check for it in the collection that would look like this;
package main
import (
"fmt"
"encoding/json"
)
func main() {
var m map[string]interface{}
sJSON := `{"k": "v", "t":["str-a","str-b","str-c"]}`
_ = json.Unmarshal([]byte(sJSON),&m)
// find out if one of the string values of "t" is "str-b"
for _, v := range m["t"].([]interface{}) {
if v.(string) == "str-b" {
fmt.Println("match found!")
}
}
//fmt.Println(m["t"].([]interface{})[1].(string))
}
https://play.golang.org/p/vo_90bKw92
If you want to avoid this 'unboxing' stuff, which I would recommend you do, you could instead define a struct to unmarshal into, itwould look like this;
type MyStruct struct {
K string `json:"k"`
T []string `json:"t"`
}
Then you can just range over T without any type assertions and do the compare, working example here; https://play.golang.org/p/ehPxOygGf5

Related

Golang equivalent to Python json.dumps and json.loads

This is a very weird situation but I need to convert a stringified json to something valid that I can unmarshall with:
"{\"hello\": \"hi\"}"
I want to be able to unmarshall this into a struct like this:
type mystruct struct {
Hello string `json:"hello,string"`
}
I know normally the unmarshall takes bytes but Im trying to convert what I currently get into something structified.
Any suggestions?
The issue is that the encoding/json package accepts well-formed JSON, in this case the initial JSON that you have has escaped quotes, first you have to unescape them, one way to do this is by using the strconv.Unquote function, here's a sample snippet:
package main
import (
"encoding/json"
"fmt"
"strconv"
)
type mystruct struct {
Hello string `json:"hello,omitempty"`
}
func main() {
var rawJSON []byte = []byte(`"{\"hello\": \"hi\"}"`)
s, _ := strconv.Unquote(string(rawJSON))
var val mystruct
if err := json.Unmarshal([]byte(s), &val); err != nil {
// handle error
}
fmt.Println(s)
fmt.Println(err)
fmt.Println(val.Hello)
}

How do I preprocess rows fetched with sqlx?

Consider the following example:
package main
import (
"fmt"
"github.com/jmoiron/sqlx"
_ "github.com/go-sql-driver/mysql"
)
type Data struct {
Stuff string
}
func main() {
db, _ := sqlx.Connect("mysql", "root:root#(localhost:3306)/data")
var datas []Data
db.Select(&datas, "select 'a,b' stuff from data limit 10")
fmt.Println(datas)
}
What I'd like to do is have Stuff as []string, where it would be split by ,. I guess I could add an extra []string field and loop over the results populating this field and removing the source data, but that seems inefficient. What's the canonical way to do this in sqlx?
While this is not supported out of the box you can create custom struct implementing Scanner interface (https://golang.org/pkg/database/sql/#Scanner):
type StringList []string
// Scan implements Scanner interface and parses
// database result to StringList struct.
func (s *StringList) Scan(src interface{}) error {
var source string
switch src.(type) {
case string:
source = src.(string)
case []byte:
source = string(src.([]byte))
default:
return errors.New("Incompatible type for StringList")
}
*s = strings.Split(source, ",")
for i := range *s {
(*s)[i] = strings.TrimSpace((*s)[i])
}
return nil
}
And use this struct instead of string:
type Data struct {
Stuff StringList
}

JSON unmarshaling with long numbers gives floating point number

I was marshaling and unmarshaling JSONs using golang and when I want to do it with number fields golang transforms it in floating point numbers instead of use long numbers, for example.
I have the following JSON:
{
"id": 12423434,
"Name": "Fernando"
}
After marshal it to a map and unmarshal again to a json string I get:
{
"id":1.2423434e+07,
"Name":"Fernando"
}
As you can see the "id" field is in floating point notation.
The code that I am using is the following:
package main
import (
"encoding/json"
"fmt"
"os"
)
func main() {
//Create the Json string
var b = []byte(`
{
"id": 12423434,
"Name": "Fernando"
}
`)
//Marshal the json to a map
var f interface{}
json.Unmarshal(b, &f)
m := f.(map[string]interface{})
//print the map
fmt.Println(m)
//unmarshal the map to json
result,_:= json.Marshal(m)
//print the json
os.Stdout.Write(result)
}
It prints:
map[id:1.2423434e+07 Name:Fernando]
{"Name":"Fernando","id":1.2423434e+07}
It appears to be that the first marshal to the map generates the FP. How can I fix it to a long?
This is the link to the program in the goland playground:
http://play.golang.org/p/RRJ6uU4Uw-
There are times when you cannot define a struct in advance but still require numbers to pass through the marshal-unmarshal process unchanged.
In that case you can use the UseNumber method on json.Decoder, which causes all numbers to unmarshal as json.Number (which is just the original string representation of the number). This can also useful for storing very big integers in JSON.
For example:
package main
import (
"strings"
"encoding/json"
"fmt"
"log"
)
var data = `{
"id": 12423434,
"Name": "Fernando"
}`
func main() {
d := json.NewDecoder(strings.NewReader(data))
d.UseNumber()
var x interface{}
if err := d.Decode(&x); err != nil {
log.Fatal(err)
}
fmt.Printf("decoded to %#v\n", x)
result, err := json.Marshal(x)
if err != nil {
log.Fatal(err)
}
fmt.Printf("encoded to %s\n", result)
}
Result:
decoded to map[string]interface {}{"id":"12423434", "Name":"Fernando"}
encoded to {"Name":"Fernando","id":12423434}
The JSON standard doesn't have longs or floats, it only has numbers. The json package will assume float64 when you haven't defined anything else (meaning, only provided Unmarshal with an interface{}).
What you should do is to create a proper struct (as Volker mentioned):
package main
import (
"encoding/json"
"fmt"
"os"
)
type Person struct {
Id int64 `json:"id"`
Name string `json:"name"`
}
func main() {
//Create the Json string
var b = []byte(`{"id": 12423434, "Name": "Fernando"}`)
//Marshal the json to a proper struct
var f Person
json.Unmarshal(b, &f)
//print the person
fmt.Println(f)
//unmarshal the struct to json
result, _ := json.Marshal(f)
//print the json
os.Stdout.Write(result)
}
Result:
{12423434 Fernando}
{"id":12423434,"name":"Fernando"}
Playground: http://play.golang.org/p/2R76DYVgMK
Edit:
In case you have a dynamic json structure and wish to use the benefits of a struct, you can solve it using json.RawMessage. A variable of type json.RawMessage will store the raw JSON string so that you later on, when you know what kind of object it contains, can unmarshal it into the proper struct. No matter what solution you use, you will in any case need some if or switch statement where you determine what type of structure it is.
It is also useful when parts of the JSON data will only be copied to the another JSON object such as with the id-value of a JSON RPC request.
Example of container struct using json.RawMessage and the corresponding JSON data:
type Container struct {
Type string `json:"type"`
Data json.RawMessage `json:"data"`
}
var b = []byte(`{"type": "person", "data":{"id": 12423434, "Name": "Fernando"}}`)
A modified version of your example on Playground: http://play.golang.org/p/85s130Sthu
Edit2:
If the structure of your JSON value is based on the name of a name/value pair, you can do the same with a:
type Container map[string]json.RawMessage

Marshall and UnMarshall JSON Content in GoLang

I have a sample json file which is structured like this
{
"method":"brute_force",
"bc":"select * from blah;",
"gc":[
"select sum(year) from blah;",
"select count(*) from table;"
]
}
I am trying to write a go program which can read this file and operate of json content.
package main
import (
"fmt"
"encoding/json"
"io/ioutil"
)
type Response2 struct {
method string
bc string
gc []string
}
func main() {
file,_ := ioutil.ReadFile("config.json")
fmt.Printf("%s",string(file))
res := &Response2{}
json.Unmarshal([]byte(string(file)), &res)
fmt.Println(res)
fmt.Println(res.method)
fmt.Println(res.gc)
}
res.method and res.gc dont print anything. I have no idea on whats going wrong.
type Response2 struct {
method string
bc string
gc []string
}
The name of the fields Must be Uppercase otherwise the Json module can't access them (they are private to your module).
You can use the json tag to specify a match between Field and name
type Response2 struct {
Method string `json:"method"`
Bc string `json:"bc"`
Gc []string `json:"gc"`
}

Dynamically create variables of certain type based on string in Go

Simple version
How can you create a variable of a certain type based upon the value of a string?
type ta struct { a int }
type tb struct { b float }
type tc struct { c string }
t := "tb"
v := MagicVarFunc(t) // Returns a new allocated var of type interface{}
v.(tb).b = 8.3
The true example
In my, surprisingly enough, working example below, I am dynamically creating variables based on a string. This is done by registering each struct type in a map with the string being the key and a nil-pointer of the type being the value.
Each type implements an interface with the method New() which returns a new variable of that specific type.
The example below is very close to what I wish to do, where each action has a set of JSON encoded data which will populate the corresponding struct. The way I've structured it is also because I wish to be able to create new stand alone actions that I register to the map.
I am not sure if am abusing the language now.
May anyone give me any pointers if I am completely out of my mind? Is there an obviously easier way?
package main
import (
"fmt"
"encoding/json"
)
// All I require of an action is that it may be executed
type ActionHandler interface {
Exec()
New() ActionHandler
}
// My list of actions
var mActions = make(map[string]ActionHandler)
// Action Exit (leaving the program)
type aExit struct {}
func (s *aExit) Exec() { fmt.Println("Good bye") }
func (s *aExit) New() ActionHandler { return new(aExit) }
func init() {
var a *aExit
mActions[`exit`] = a
}
// Action Say (say a message to someone)
type aSay struct {
To string
Msg string
}
func (s *aSay) Exec() { fmt.Println(`You say, "` + s.Msg + `" to ` + s.To) }
func (s *aSay) New() ActionHandler { return new(aSay) }
func init() {
var a *aSay
mActions[`say`] = a
}
func inHandler(action string, data []byte) {
a := mActions[action].New()
json.Unmarshal(data, &a)
a.Exec()
}
func main(){
inHandler(`say`, []byte(`{"to":"Sonia","msg":"Please help me!"}`))
inHandler(`exit`, []byte(`{}`))
}
You can use reflection to get the zero value of, or to allocate a new value (like new) of a type using reflection, if you can get the Type value at runtime. However, I don't think there is a way to get the Type from a string. You would need to have a value of that type to get the type itself.
I adopted your idea, of using a map. I map the string to the type itself, which you can get using reflect.TypeOf, which gets the type out of an interface value. Then I used reflect.Zero to get the zero value of that type (a convenient value that exists for every type). Then I got the value out as an interface.
package main
import "reflect"
type ta struct { a int }
type tb struct { b float64 }
type tc struct { c string }
var mActions map[string]reflect.Type = make(map[string]reflect.Type)
func init() {
var a ta
mActions[`ta`] = reflect.TypeOf(a)
var b tb
mActions[`tb`] = reflect.TypeOf(b)
var c ta
mActions[`tc`] = reflect.TypeOf(c)
}
func MagicVarFunc(action string) interface{} {
return reflect.Zero(mActions[action]).Interface()
}
func main() {
t := "tb"
v := MagicVarFunc(t) // Returns a new allocated var of type interface{}
x := v.(tb)
x.b = 8.3
}
jorelli's answer is very good. I'm just going to show a few options. Your "true example" looks essentially like command dispatch, with command parameters specified with JSON. To start with simple code that does this,
package main
import (
"encoding/json"
"fmt"
)
func inHandler(action string, data []byte) {
arg := make(map[string]interface{})
json.Unmarshal(data, &arg)
switch action {
case "say":
fmt.Printf("You say, %q to %s\n", arg["msg"], arg["to"])
case "exit":
fmt.Println("Good bye")
}
}
func main() {
inHandler(`say`, []byte(`{"to":"Sonia","msg":"Please help me!"}`))
inHandler(`exit`, []byte(`{}`))
}
Your register new commands by adding cases to the switch statement. Yeah, didn't think you'd like that. So, adding your map and init() idea,
package main
import (
"encoding/json"
"fmt"
)
type jmap map[string]interface{}
var mActions = map[string]func(jmap){}
func init() {
mActions["say"] = func(arg jmap) {
fmt.Printf("You say, %q to %s\n", arg["msg"], arg["to"])
}
}
func init() {
mActions["exit"] = func(jmap) { fmt.Println("Good bye") }
}
func inHandler(action string, data []byte) {
args := make(jmap)
json.Unmarshal(data, &args)
mActions[action](args)
}
func main() {
inHandler(`say`, []byte(`{"to":"Sonia","msg":"Please help me!"}`))
inHandler(`exit`, []byte(`{}`))
}
Now if you wanted, you could put each of those init functions in a separate source file and new commands could be registered by creating a new source file with a new init function.
The rest of the program is simplified with some assumptions that the commands have flat argument lists that the JSON will always encode as an object. This allows you to dispense with separate Go struct definitions for each command. inHandler just creates the same type of object (a map) for all commands, unmarshals into it, and passes it to the command. If you wanted to handle a little more arbitrary JSON, you could unmarshal into an empty interface, and the functions would have to do some extra work to dig out the arguments. If that was too much work and you really wanted to unmarshal directly into a struct, then you arrive near jorelli's solution of making each command function unmarshal its own JSON.
start by defining a function type that does the thing you want:
type Producer func([]byte) interface{}
make a few of them:
func FooProducer(raw []byte) interface{} {
foo := new(Foo)
... // do something to foo
return foo
}
func BarProducter(raw []byte) interface{} {
bar := new(Bar)
... // do something to bar
return bar
}
stick them in a map:
likeThis := map[string]Producer{
"foo": FooProducer,
"bar": BarProducer,
}
and then just do one of these:
myVal := likeThis[someString](raw)
but you probably want to define some interface and make your producer something more like:
type Producer func([]byte) MyAwesomeInterface
since there's probably some common stuff you want to do with those things you're decoding. You also probably want to handle the case of a bad string input, like-a-this:
f, ok := likeThis[someString]
if !ok {
// return, break, panic... something, just get the hell away from here.
}
myVal := f(raw)
The whole notion of inspecting types is kinda cumbersome in Go. It's generally less work to just add new types than it is to try to do reflection gymnastics with the type system.