I'd like to make a json out of a hash received from redis using redigo:
func showHashtags(c *gin.Context) {
hashMap, err := redis.StringMap(conn.Do("HGETALL", MyDict))
if err != nil {
fmt.Println(err)
}
fmt.Println(hashMap) //works fine and shows the map
m := make(map[string]string)
for k, v := range hashMap {
m[k] = v
}
jmap, _ := json.Marshal(m)
c.JSON(200, jmap)
}
However the result in browser is gibberish like:
"eyIgIjoiMiIsIjExX9iq24zYsSAiOiIxIiwiQWxsNFJhbWluICI6IjEiLCJCSUhFICI6IjMiLCJCVFNBUk1ZICI6IjIiLCJDTUJZTiAiOiIxI....
What is wrong here? How can I fix it?
The variable jmap is type []byte. The call to JSON encoder in c.JSON() marshals []byte as a base64 encoded string as you see in the output.
To fix the problem, use one level of JSON encoding by passing the map directly to c.JSON:
hashMap, err := redis.StringMap(conn.Do("HGETALL", MyDict))
if err != nil {
// handle error
}
m := make(map[string]string)
for k, v := range hashMap {
m[k] = v
}
c.JSON(200, m)
Because hashMap is a map[string]string, you can use it directly:
hashMap, err := redis.StringMap(conn.Do("HGETALL", MyDict))
if err != nil {
// handle error
}
c.JSON(200, hashMap)
Related
The usual approach for unmarshalling is like this:
atmosphereMap := make(map[string]interface{})
err := json.Unmarshal(bytes, &atmosphereMap)
But how to unmarshal json data to custom interface:
type CustomInterface interface {
G() float64
}
atmosphereMap := make(map[string]CustomInterface)
err := json.Unmarshal(bytes, &atmosphereMap)
The second way gives me an error:
panic: json: cannot unmarshal object into Go value of type main.CustomInterface
How to do it correctly?
To unmarshal into a set of types, which all implement a common interface, you can implement the json.Unmarshaler interface on the parent type, the map[string]CustomInterface in your case:
type CustomInterfaceMap map[string]CustomInterface
func (m CustomInterfaceMap) UnmarshalJSON(b []byte) error {
data := make(map[string]json.RawMessage)
if err := json.Unmarshal(b, &data); err != nil {
return err
}
for k, v := range data {
var dst CustomInterface
// populate dst with an instance of the actual type you want to unmarshal into
if _, err := strconv.Atoi(string(v)); err == nil {
dst = &CustomImplementationInt{} // notice the dereference
} else {
dst = &CustomImplementationFloat{}
}
if err := json.Unmarshal(v, dst); err != nil {
return err
}
m[k] = dst
}
return nil
}
For a full example, see this playground. Make sure you unmarshal into a CustomInterfaceMap, not map[string]CustomInterface, otherwise the custom UnmarshalJSON method will not be called.
json.RawMessage is a useful type, which is just a raw encoded JSON value, meaning it is a simple []byte, into which the JSON is stored in unparsed form.
I have a data encryption function that returns a [] byte. Of course, what has been encrypted must be decrypted (through another function) in another micro-service.
The problem is created when I send the []byte via JSON: the []byte is transformed into a string and then when I go to read the JSON through the call, the result is no longer the same.
I have to be able to pass the original []byte, created by the encryption function, through JSON or otherwise pass the []byte through a call like the one you can see below. Another possibility is to change the decryption function, but I have not succeeded.
caller function
func Dati_mono(c *gin.Context) {
id := c.Param("id")
oracle, err := http.Get("http://XXXX/"+id)
if err != nil {
panic(err)
}
defer oracle.Body.Close()
oJSON, err := ioutil.ReadAll(oracle.Body)
if err != nil {
panic(err)
}
oracleJSON := security.Decrypt(oJSON, keyEn)
c.JSON(http.StatusOK, string(oJSON))
}
function that is called with the url
func Dati(c *gin.Context) {
var (
person Person
result mapstring.Dati_Plus
mmap []map[string]interface{}
)
rows, err := db.DBConor.Query("SELECT COD_DIPENDENTE, MATRICOLA, COGNOME FROM ANDIP021_K")
if err != nil {
fmt.Print(err.Error())
}
for rows.Next() {
err = rows.Scan(&person.COD_DIPENDENTE, &person.MATRICOLA, &person.COGNOME)
ciao := structs.Map(&person)
mmap = append(mmap, ciao)
}
defer rows.Close()
result = mapstring.Dati_Plus{
len(mmap),
mmap,
}
jsonEn := []byte(mapstring.Dati_PlustoStr(result))
keyEn := []byte(key)
cipherjson, err := security.Encrypt(jsonEn, keyEn)
if err != nil {
log.Fatal(err)
}
c.JSON(http.StatusOK, cipherjson)
}
encryption and decryption functions
func Encrypt(json []byte, key []byte) (string, error) {
k, err := aes.NewCipher(key)
if err != nil {
return "nil", err
}
gcm, err := cipher.NewGCM(k)
if err != nil {
return "nil", err
}
nonce := make([]byte, gcm.NonceSize())
if _, err = io.ReadFull(rand.Reader, nonce); err != nil {
return "nil", err
}
return gcm.Seal(nonce, nonce, json, nil), nil
}
func Decrypt(cipherjson []byte, key []byte) ([]byte, error) {
k, err := aes.NewCipher(key)
if err != nil {
return nil, err
}
gcm, err := cipher.NewGCM(k)
if err != nil {
return nil, err
}
nonceSize := gcm.NonceSize()
if len(cipherjson) < nonceSize {
return nil, errors.New("cipherjson too short")
}
nonce, cipherjson := cipherjson[:nonceSize], cipherjson[nonceSize:]
return gcm.Open(nil, nonce, cipherjson, nil)
}
Everything works, the problem is created when I print cipherjson in c.JSON (): the []byte is translated into a string.
At the time it is taken and read by the calling function it is read as string and ioutil.ReadAll () creates the [] byte of the read string.
Instead I must be able to pass to the Decryot function the return of the Encrypt function used in the called function.
I hope I was clear, thanks in advance
You are not decoding the response before decrypting. In other words, you are handing the JSON encoding of the ciphertext to Decrypt. That is obviously not going to do what you want. To recover the plaintext you have to precisely undo all of the operations of the encryption and encoding in reverse order.
Either decode before decrypting, or don't JSON encode on the server. For instance:
oJSON, err := ioutil.ReadAll(oracle.Body)
if err != nil {
panic(err)
}
var ciphertext string
if err := json.Unmarshal(oJSON, &ciphertext); err != nil {
// TODO: handle error
}
oracleJSON := security.Decrypt(ciphertext, keyEn)
Although it is unclear why you even go through the trouble of JSON encoding in the first place. You might as well just write the ciphertext directly. If you really want to encode the ciphertext, you should not convert it to a string. The ciphertext is just a bunch of random bytes, not remotely resembling a UTF-8 encoded string, so don't treat it like one. encoding/json uses the base64 encoding for byte slices automatically, which is a much cleaner (and probably shorter) representation of the ciphertext than tons of unicode escape sequences.
Independent of the encoding you choose (if any), your Encrypt function is broken.
// The plaintext and dst must overlap exactly or not at all. To reuse
// plaintext's storage for the encrypted output, use plaintext[:0] as dst.
Seal(dst, nonce, plaintext, additionalData []byte) []byte
The first argument is the destination for the encryption. If you don't need to retain the plaintext, pass json[:0]; otherwise pass nil.
Also, Decrypt expects the ciphertext to be prefixed by the nonce, but Encrypt doesn't prepend it.
I have a function that returns an interface{}. How can I serialize this into a JSON Array without "hardcoding" the fields in a struct.
I am using https://github.com/jmoiron/jsonq to return the interface.
json.Unmarshal(resp.Bytes(), &response)
data := map[string]interface{}{}
dec := json.NewDecoder(strings.NewReader(resp.String()))
dec.Decode(&data)
jq := jsonq.NewQuery(data)
results, err := jq.Array("results")
if err != nil {
log.Fatalln("Unable to get results: ", err)
}
if len(results) == 0 {
return nil
}
return results // this is returning an interface{}
a json string can always be unmarshalled to map[string]interface{}. That is what you need to work with then.
I should have checked the type I was dealing with. I found out by:
fmt.Println(reflect.TypeOf(results))
which returned: []interface {}
I was then able to iterate over it by using:
for _, event:= range results {
v, err := json.MarshalIndent(event, "", " ")
if err != nil {
fmt.Println("error:", err)
}
fmt.Println(string(v))
}
I build a client and a server in golang both are using this functions to encrypt/decrypt
func encrypt(text []byte) ([]byte, error) {
block, err := aes.NewCipher(key)
if err != nil {
return nil, err
}
b := base64.StdEncoding.EncodeToString(text)
ciphertext := make([]byte, aes.BlockSize+len(b))
iv := ciphertext[:aes.BlockSize]
if _, err := io.ReadFull(rand.Reader, iv); err != nil {
return nil, err
}
cfb := cipher.NewCFBEncrypter(block, iv)
cfb.XORKeyStream(ciphertext[aes.BlockSize:], []byte(b))
return ciphertext, nil
}
func decrypt(text []byte) ([]byte, error) {
block, err := aes.NewCipher(key)
if err != nil {
return nil, err
}
if len(text) < aes.BlockSize {
return nil, errors.New("ciphertext too short")
}
iv := text[:aes.BlockSize]
text = text[aes.BlockSize:]
cfb := cipher.NewCFBDecrypter(block, iv)
cfb.XORKeyStream(text, text)
data, err := base64.StdEncoding.DecodeString(string(text))
if err != nil {
return nil, err
}
return data, nil
}
so yeah I make a normal post request
url := "https://"+configuration.Server+configuration.Port+"/get"
// TODO maybe bugs rest here
ciphertext, err := encrypt([]byte(*getUrl))
if err != nil {
fmt.Println("Error: " + err.Error())
}
fmt.Println(string(ciphertext))
values := map[string]interface{}{"url": *getUrl, "urlCrypted": ciphertext}
jsonValue, _ := json.Marshal(values)
jsonStr := bytes.NewBuffer(jsonValue)
req, err := http.NewRequest("POST", url, jsonStr)
and the servercode is as following
requestContent := getRequestContentFromRequest(req)
url := requestContent["url"].(string)
undecryptedUrl := requestContent["urlCrypted"].(string)
decryptedurl, err := decrypt([]byte(undecryptedUrl))
if err != nil {
fmt.Println("Error: " + err.Error())
}
fmt.Println(decryptedurl)
where getRequestContentFromRequest is as following
func getRequestContentFromRequest(req *http.Request)
map[string]interface{} {
buf := new(bytes.Buffer)
buf.ReadFrom(req.Body)
data := buf.Bytes()
var requestContent map[string]interface{}
err := json.Unmarshal(data, &requestContent)
if err != nil {
fmt.Println(err)
}
return requestContent
}
Now to the problem.
If I encrypt my string in the client and decrypt it direct after that everything is fine.
But, when I send the encrypted string to the server and try to decrypt it with literrally the same function as in the client, the decrypt function throws an error.
Error: illegal base64 data at input byte 0
I think the Problem is the unmarshalling of the JSON.
Thanks for help.
P.S.
Repos are
github.com/BelphegorPrime/goSafeClient and github.com/BelphegorPrime/goSafe
UPDATE
Example JSON
{"url":"facebook2.com","urlCrypted":"/}\ufffd\ufffd\ufffdgP\ufffdN뼞\ufffd\u0016\ufffd)\ufffd\ufffd\ufffdy\u001c\u000f\ufffd\ufffd\ufffdep\ufffd\rY\ufffd\ufffd$\ufffd\ufffd"}
UPDATE2
I made a playground here
The problem is that you encode in base64 twice. The first time in the encrypt function and the second time during the JSON marshalling. byte slices are automatically converted into base64 strings by the encoding/json marshaller.
The solution is to decode the base64 string before calling decrypt.
Example on the Go PlayGround
EDIT
Working solution here
I have some valid JSON files and some which are not (without the surrounding brackets)
Currently I have a method for each case: one uses json.Unmarshal for the valid ones and the other uses json.NewDecoder for the bracketless ones.
How can I merge it into one function what can handle both cases?
EDIT:
Here is the code of the two cases:
func getDrivers() []Drivers {
raw, err := ioutil.ReadFile("/home/ubuntu/drivers.json")
if err != nil {
fmt.Println(err.Error())
os.Exit(1)
}
var d []Drivers
json.Unmarshal(raw, &d)
return d
}
func getMetrics() []Metrics {
file, err := os.Open("/home/ubuntu/metrics.json")
if err != nil {
fmt.Println("bad err!")
}
r := bufio.NewReader(file)
dec := json.NewDecoder(r)
// while the array contains values
var metrics []Metrics
for dec.More() {
var m Metrics
err := dec.Decode(&m)
if err != nil {
log.Fatal(err)
}
metrics = append(metrics, m)
}
return metrics
}
Thank you