Go json.Unmarshall() works with single entity but not with slice - json

I'm using Go for a simple http client. Here's the entity I'm unmarshalling:
type Message struct {
Id int64
Timestamp int64
Text string
Author User
LastEdited int64
}
type User struct {
Id int64
Name string
}
A single entity looks like this in JSON:
{
"text": "hello, can you hear me?",
"timestamp": 1512964818565,
"author": {
"name": "andrea",
"id": 3
},
"lastEdited": null,
"id": 8
}
Go/json has no problem unmarshalling the single entity:
var m Message
err = json.Unmarshal(body, &m)
if err != nil {
printerr(err.Error())
}
println(m.Text)
However, if the return of the endpoint is multiple entities:
[
{
"text": "hello, can you hear me?",
"timestamp": 1512964800981,
"author": {
"name": "eleven",
"id": 4
},
"lastEdited": null,
"id": 7
}
]
And I change my corresponding Unmarshall to work on a slice of structs, Go throws an error:
var m []Message
err = json.Unmarshal(body, &m)
if err != nil {
printerr(err.Error()) // unexpected end of JSON input
}
for i := 0; i < len(m); i++ {
println(m[i].Text)
}
What gives?

Works fine for me (try it on playground), where are you getting the payload data from? sounds like that's truncating it.
package main
import (
"encoding/json"
"fmt"
)
type Message struct {
Id int64
Timestamp int64
Text string
Author User
LastEdited int64
}
type User struct {
Id int64
Name string
}
func main() {
body := []byte(`[
{
"text": "hello, can you hear me?",
"timestamp": 1512964800981,
"author": {
"name": "eleven",
"id": 4
},
"lastEdited": null,
"id": 7
}
]`)
var m []Message
err := json.Unmarshal(body, &m)
if err != nil {
fmt.Printf("error: %v") // unexpected end of JSON input
}
for i := 0; i < len(m); i++ {
fmt.Println(m[i].Text)
}
}
running it gives this output
hello, can you hear me?

Related

How to connect JSON that is unmarshaled to an HTML page using golang?

I've reached the point where I have unmarshaled my JSON and I can print the data directly from the registry.go file. What I don't understand (and I'm new to golang and backend dev) is how to connect that data to my html page. I would very much appreciate if someone could explain to me how to do this.
I have seen some examples, but they are all in one file, I want to keep the JSON, HTML, and Go pages separate. The JSON is being pulled directly from an API, and there will be multiple HTML pages using that data.
Here is an example, very close to the JSON I'm using, it is being pulled from an API:
{
"count": 4,
"next": null,
"previous": null,
"results": [
{
"id": 1,
"description": "some text",
"displayName": "some more text",
"status": {
"status": "UP",
"lastUpdate": "2020-07-21T12:42:24.968647Z"
},
"type": "test"
},
{
"id": 2,
"description": "some text",
"displayName": "some more text",
"status": {
"status": "UP",
"lastUpdate": "2020-07-21T12:42:24.968647Z"
},
"type": "test"
},
{
"id": 3,
"description": "some text",
"displayName": "some more text",
"status": {
"status": "UP",
"lastUpdate": "2020-07-21T12:42:24.968647Z"
},
"type": "test"
},
{
"id": 4,
"description": "some text",
"displayName": "some more text",
"status": {
"status": "UP",
"lastUpdate": "2020-07-21T12:42:24.968647Z"
},
"type": "test"
}
]
}
Here is the registry.go page I am using. I have a separate main.go page as I have other pages that have been created. I am trying to keep this piece separate, but if I cannot please tell me.
package main
import (
"encoding/json"
"fmt"
"io/ioutil"
"os"
)
// Results struct which contains
// an array of results
type Results struct {
Count int `json:"count"`
Results []Result `json:"results"`
}
// Results struct which contains the ID and description
type Result struct {
ID int `json:"id"`
Description string `json:"description"`
DisplayName string `json:"displayName"`
Status Status `json:"status"`
Type string `json:"type"`
}
// Status struct
type Result struct {
Status string `json:"status"`
LastUpdated string `json:"lastUpdated"`
}
func main() {
// Open the jsonFile
jsonFile, err := os.Open("test.json")
// if os.Open returns an error then handle it
if err != nil {
fmt.Println(err)
}
fmt.Println("Successfully Opened test.json")
// defer the closing of the jsonFile in order to parse it later on
defer jsonFile.Close()
// read the opened xmlFile as a byte array.
byteValue, _ := ioutil.ReadAll(jsonFile)
// initialize the results array
var results Results
// unmarshal the byteArray
json.Unmarshal(byteValue, &results)
// Print the count
fmt.Println(results.count)
// iterate through the results and print
for i := 0; i < len(users.Users); i++ {
fmt.Println("ID: " + results.Results[i].ID)
fmt.Println("Desctiption: " + results.Results[i].Description)
}
}
And here is the html page I have created (test.html), just showing the body piece:
<body>
{{range .}}
<div>
<h3>{{.DisplayName}}</h3>
</div>
<div>{{.Id}}</div>
<div>{{.Description}}</div>
<div>{{.Type}}</div>
{{end}}
</body>
In Main.go I have the following references:
//handler to display the page
http.HandleFunc("/test", hndlerTest)
//hndlerTest - the Testing Page
func hndlerTest(w http.ResponseWriter, req *http.Request){
renderTemplate(w,"test.html", nil)
}
You can use the html template package to parse and render html filed and write the executed template to your http response. You can refer to the following example to start with.
package main
import (
"encoding/json"
"html/template"
"log"
"net/http"
)
var htmlFile = `
<body>
{{range .Foos}}
<span>Foo: {{.Foo}}</span>
{{end}}
</body>
`
var jsonData = `[{"foo":1},{"foo":2},{"foo":3}]`
type foo struct {
Foo int `json:"foo"`
}
type Data struct {
Foos []foo
}
func main() {
var d Data
if err := json.Unmarshal([]byte(jsonData), &d.Foos); err != nil {
panic(err)
}
http.HandleFunc("/", func(rw http.ResponseWriter, r *http.Request) {
// Use template.New("index").ParseFiles("index.html") for files
templ, err := template.New("index").Parse(htmlFile)
if err != nil {
panic(err)
}
if err := templ.Execute(rw, d); err != nil {
panic(err)
}
})
log.Panic(http.ListenAndServe(":3030", nil))
}

Zero date while unmarshaling JSON [duplicate]

This question already has answers here:
Unmarshal incorrectly formatted datetime
(2 answers)
Closed 3 years ago.
I'm trying to unmarshal JSON with time value. I have this JSON structure.
{
"Nick": "cub",
"Email": "cub875#rambler.ru",
"Created_at": "2017-10-09",
"Subscribers": [
{
"Email": "rat1011#rambler.ru",
"Created_at": "2017-11-30"
},
{
"Email": "hound939#rambler.ru",
"Created_at": "2016-07-15"
},
{
"Email": "worm542#rambler.ru",
"Created_at": "2017-02-16"
},
{
"Email": "molly1122#rambler.ru",
"Created_at": "2016-11-30"
},
{
"Email": "goat1900#yandex.ru",
"Created_at": "2018-07-10"
},
{
"Email": "duck1146#rambler.ru",
"Created_at": "2017-09-04"
},
{
"Email": "eagle1550#mail.ru",
"Created_at": "2018-01-03"
},
{
"Email": "prawn1755#rambler.ru",
"Created_at": "2018-04-20"
},
{
"Email": "platypus64#yandex.ru",
"Created_at": "2018-02-17"
}
]
}
And a function that implements reading from JSON file to struct User. Everything if fine but when I set CreatedAt field in User struct to time.Time type I get 0001-01-01 00:00:00 +0000 UTC value for each field with type format in JSON file.
type Subscriber struct {
Email string `json: "Email"`
CreatedAt time.Time `json: "Created_at"`
}
type User struct {
Nick string `json: "Nick"`
Email string `json: "Email"`
CreatedAt string `json: "Created_at"`
Subscribers []Subscriber `json: "Subscribers"`
}
func main() {
// Slice where objects from users.json will be pushed
var jsonData []User
// ReadJSON and push User struct to jsonData slice
readJSON("users.json", &jsonData)
for _, user := range jsonData {
fmt.Println(user.Nick, user.Email, user.CreatedAt) // 0001-01-01 00:00:00 +0000 UTC
fmt.Println(user.Subscribers)
}
}
func readJSON(fileName string, userSlice *[]User) {
file, err := os.Open(fileName)
if err != nil {
log.Fatal(err)
}
bytes, err := ioutil.ReadAll(file)
if err != nil {
log.Fatal(err)
}
err1 := json.Unmarshal(bytes, &userSlice)
if err1 != nil {
log.Fatal(err1)
}
}
What is an appropriate way to read time from JSON file to User in RFC3339 format?
You can use a custom time type that implements the json.Unmarshaler interface.
You can start with this struct:
type CustomTime struct {
time.Time // Embed time.Time to allow calling of normal time.Time methods
}
Then add the required UnmarshalJSON([]byte) error function. It could look like this:
func (c *CustomTime) UnmarshalJSON(b []byte) error {
if len(b) < 3 {
// Empty string: ""
return fmt.Errorf("Empty time value")
}
t, err := time.Parse("2006-01-02", string(b[1:len(b)-1])) // b[1:len(b)-1] removes the first and last character, as they are quotes
if err != nil {
return err
}
c.Time = t
return nil
}
You can try the example on the Go Playground

Golang json unmarshal according to key value pair

I have json as following
"data": [
{
"id": "recent_search",
"items": [],
"name": ""
},
{
"id": "popular_search",
"items": [],
"name": ""
},
{
"id": "digital",
"items": [],
"name": "DIGITAL"
}
]
and the structs are as follows:
type universeTypeData struct {
Recent universeSearchInfo
Popular universeSearchInfo
Digital universeSearchInfo
}
type universeSearchInfo struct {
ID string `json:"id"`
Name string `json:"name"`
Items []universeSearchItem `json:"items"`
}
I want to unmarshal my json as "id" with value "recent_search" map to Recent, "id" with value "popular_search" map to Popular. Is there any way of doing this in golang?
My approach of doing it is
for _, v := range result.Data {
if v.ID == "in_category" {
finalResult.Universe.InCategory.ID = v.ID
finalResult.Universe.InCategory.Name = v.Name
for _, abc := range v.Items {
finalResult.Universe.InCategory.Items = append(finalResult.Universe.InCategory.Items, abc)
}
}
if v.ID == "recent_search" {
finalResult.Universe.Recent.ID = v.ID
finalResult.Universe.Recent.Name = v.Name
for _, abc := range v.Items {
finalResult.Universe.Recent.Items = append(finalResult.Universe.Recent.Items, abc)
}
}
if v.ID == "popular_search" {
finalResult.Universe.Popular.ID = v.ID
finalResult.Universe.Popular.Name = v.Name
for _, abc := range v.Items {
finalResult.Universe.Popular.Items = append(finalResult.Universe.Popular.Items, abc)
}
}
Is there any better way of doing it?
You want to unmurshal JSON array into Go struct which is not natural mapping. Any way, you most likely should be first unmurshal in slice and then parse this slice. Some workaround is to use json.Decoder
dec := json.NewDecoder(JSONdataReader)
var res universeTypeData
// read open bracket
dec.Token()
// while the array contains values
for dec.More() {
var m universeSearchInfo
// decode an array value
dec.Decode(&m)
switch m.ID {
case "recent_search":
res.Recent = m
case "popular_search":
res.Popular = m
case "digital":
res.Digital = m
}
}
// read closing bracket
dec.Token()
which allow you to decode on the fly, in one pass, without consuming intermediate slice representation. Working example
Implement Unmarshaler interface:
Unmarshaler is the interface implemented by types that can unmarshal a
JSON description of themselves. The input can be assumed to be a valid
encoding of a JSON value. UnmarshalJSON must copy the JSON data if it
wishes to retain the data after returning.
json unmarshaler interface assign the value from json to struct after parsing the result and applying conditions to fetch the value.
package main
import (
"encoding/json"
"fmt"
)
type Details struct {
Data []universeSearchInfo `json:"data"`
}
type universeTypeData struct {
Recent universeSearchInfo
Popular universeSearchInfo
Digital universeSearchInfo
}
type universeSearchInfo struct {
ID string `json:"id"`
Name string `json:"name"`
Items []string `json:"items"`
}
func main() {
var result universeTypeData
jsonBytes := []byte(`{"data": [
{
"id": "recent_search",
"items": [],
"name": ""
},
{
"id": "popular_search",
"items": [],
"name": ""
},
{
"id": "digital",
"items": [],
"name": "DIGITAL"
}
]}`)
if err := json.Unmarshal(jsonBytes, &result); err != nil {
fmt.Println(err)
}
fmt.Println(result)
}
func (universeData *universeTypeData) UnmarshalJSON(data []byte) error {
var result Details
if err := json.Unmarshal(data, &result); err != nil {
return err
}
for _,value := range result.Data{
switch value.ID {
case "recent_search":
universeData.Recent = value
}
}
return nil
}
Working code on Go Playground

Unmarshalling json arrays as json objects

I have to unmarshal a series of Json objects, but one of the objects contain a json array which is not really structured in a good way.
"labels": [
{
"key": "owner",
"value": "harry"
},
{
"key": "group",
"value": "student"
}
]
I am unmarshalling it using this struct -
type StudentDetails struct {
Id string `json:"id"`
Name string `json:"name"`
Labels []Label `json:"labels,omitempty"`
}
type Label struct {
Key string `json:"key"`
Value string `json:"value"`
}
And I have to access it using x.Labels[0].key == "owner" inside a for loop which is very annoying.
I want to be able to do x.Labels.Owner == "harry" instead. How do I go about achieving this? The rest of JSON is unmarshalled fine using the default unmarshal function, so I don't think writing custom function will be good option.
With the constraints you have here, this is about as close as you will get (run in playground):
package main
import (
"encoding/json"
"fmt"
)
func main() {
j := `
{
"id": "42",
"name": "Marvin",
"labels": [
{
"key": "owner",
"value": "harry"
},
{
"key": "group",
"value": "student"
}
]
}`
d := StudentDetails{}
err := json.Unmarshal([]byte(j), &d)
if err != nil {
panic(err)
}
fmt.Println(d.Labels["owner"])
fmt.Println(d.Labels["group"])
}
type StudentDetails struct {
Id string `json:"id"`
Name string `json:"name"`
Labels Labels `json:"labels"`
}
type Labels map[string]string
func (l *Labels) UnmarshalJSON(b []byte) error {
a := []map[string]string{}
err := json.Unmarshal(b, &a)
if err != nil {
return err
}
t := map[string]string{}
for _, m := range a {
t[m["key"]] = m["value"]
}
*l = t
return nil
}
How about to define custom []Label type and add function on it.
For instance
type Labels []Label
func (l Labels) Owner() string {
if len(l) > 1 {
return l[0].Value
}
return ""
}

How to unmarshal JSON in to array of interface and use

I am having difficulty understanding how to correctly unmarshal some JSON data that goes in to an array of type inteface and then use it. I tried to make this example code as simple as possible to illustrate the problem I am having. The code can be found in the playground here: https://play.golang.org/p/U85J_lBJ7Zr
The output looks like:
[map[ObjectType:chair ID:1234 Brand:Blue Inc.] map[ID:5678
Location:Kitchen ObjectType:table]] { } false { } false
Code
package main
import (
"fmt"
"encoding/json"
)
type Chair struct {
ObjectType string
ID string
Brand string
}
type Table struct {
ObjectType string
ID string
Location string
}
type House struct {
Name string
Objects []interface{}
}
func main() {
var h House
data := returnJSONBlob()
err := json.Unmarshal(data, &h)
if err != nil {
fmt.Println(err)
}
fmt.Println(h.Objects)
s1, ok := h.Objects[0].(Table)
fmt.Println(s1, ok)
s2, ok := h.Objects[0].(Chair)
fmt.Println(s2, ok)
}
func returnJSONBlob() []byte {
s := []byte(`
{
"Name": "house1",
"Objects": [
{
"ObjectType": "chair",
"ID": "1234",
"Brand": "Blue Inc."
},
{
"ObjectType": "table",
"ID": "5678",
"Location": "Kitchen"
}
]
}
`)
return s
}
I'm not sure if this is practical, since this is a simplified version of your scenario. However, one way to do this is combine the two object types to a new one, Object, and then unmarshal them directly to Object instead of using interface{}:
package main
import (
"encoding/json"
"fmt"
)
type Object struct {
ObjectType string
ID string
Brand string
Location string
}
type House struct {
Name string
Objects []Object
}
func returnJSONBlob() []byte {
s := []byte(`
{
"Name": "house1",
"Objects": [
{
"ObjectType": "chair",
"ID": "1234",
"Brand": "Blue Inc."
},
{
"ObjectType": "table",
"ID": "5678",
"Location": "Kitchen"
}
]
}
`)
return s
}
func main() {
var h House
data := returnJSONBlob()
err := json.Unmarshal(data, &h)
if err != nil {
fmt.Println(err)
}
fmt.Println(h.Objects[0].Brand)
fmt.Println(h.Objects[1].Location)
}
Prints:
Blue Inc.
Kitchen
Example here: https://play.golang.org/p/91F4UrQlSjJ