JSON Marshal uint or int as integer - json

Question

I'm looking for information about the json marshal with Go. I'll explain the situation first.
I'm developing an app for a IoT device. The app sends a JSON inside a MQTT Packet to our broker. How the device is using a SIM for data connection I need to reduce to minimum the bytes of the packet.
Right now, The JSON has this structure
{
"d": 1524036831
"p": "important message"
}
The field d is a timestamp and p is the payload.
When the app sends this JSON it has 40 bytes. But if d is 1000, pe, the JSON will be 34 bytes. So the marshal is converting the field d as uint32 to ASCII representation of the number and then sends the string.
What I want is to send this field as a true int or uint. I want to say, 1524036831 is a int32, 4 bytes, the same as 1000. So with this change I could reduce the packet size some bytes and the number is be able to grow to 32 bits.
I read the docs for json.Marshal and I did not find anything about this.
I found a "solution" but I guest it is not pretty but does the work. I want another opinions.
Ugly solution (for me)
package main
import (
"encoding/binary"
"encoding/json"
"fmt"
)
type test struct {
Data uint32 `json:"d"`
Payload string `json:"p"`
}
type testB struct {
Data []byte `json:"d"`
Payload string `json:"p"`
}
func main() {
fmt.Println("TEST with uin32")
d := []test{test{Data: 5, Payload: "Important Message"}, test{Data: 10, Payload: "Important Message"}, test{Data: 1000, Payload: "Important Message"}, test{Data: 1524036831, Payload: "Important Message"}}
for _, i := range d {
j, _ := json.Marshal(i)
fmt.Println(string(j))
fmt.Println("All:", len(j))
fmt.Println("-----------")
}
fmt.Println("\nTEST with []Byte")
d1 := []testB{testB{Data: make([]byte, 4), Payload: "Important Message"}, testB{Data: make([]byte, 4), Payload: "Important Message"}, testB{Data: make([]byte, 4), Payload: "Important Message"}, testB{Data: make([]byte, 4), Payload: "Important Message"}}
binary.BigEndian.PutUint32(d1[0].Data, 5)
binary.BigEndian.PutUint32(d1[1].Data, 20)
binary.BigEndian.PutUint32(d1[2].Data, 1000)
binary.BigEndian.PutUint32(d1[3].Data, 1524036831)
for _, i := range d1 {
j, _ := json.Marshal(i)
fmt.Println(string(j))
fmt.Println(len(j))
fmt.Println("-----------")
}
}
Play

Answer 1

To re-interate my comment: JSON is a text format, and text format are not designed to produce small messages. In particular there is no representation for numbers other than decimal strings in JSON.
Encoding numbers in a base larger than 10 will reduce the message size for large enough numbers.
You can reduce the message size your "ugly" code produces by removing leading zero bytes and encoding with base64.RawStdEncoding (which omits the padding characters). Doing this pays of for numbers >= 1e6.
If you put this all in a custom type it becomes much nicer to use:
package main
import (
"bytes"
"encoding/base64"
"encoding/binary"
"encoding/json"
"fmt"
)
type IntB64 uint32
func (n IntB64) MarshalJSON() ([]byte, error) {
b := make([]byte, 4)
binary.BigEndian.PutUint32(b, uint32(n))
b = bytes.TrimLeft(b, string(0))
// All characters in the base64 alphabet need not be escaped, so we don't
// have to call json.Marshal here.
l := base64.RawStdEncoding.EncodedLen(len(b)) + 2
j := make([]byte, l)
base64.RawStdEncoding.Encode(j[1:], b)
j[0], j[l-1] = '"', '"'
return j, nil
}
func main() {
enc(1) // "AQ"
enc(1000) // "A+g"
enc(1e6 - 1) // "D0I/"
enc(1e6) // "D0JA"
enc(1524036831) // "Wtb03w"
}
func enc(n int64) {
b, _ := json.Marshal(IntB64(n))
fmt.Printf("%10d %s\n", n, string(b))
}
Updated playground: https://play.golang.org/p/7Z03VE9roqN