Zero date while unmarshaling JSON [duplicate] - json

This question already has answers here:
Unmarshal incorrectly formatted datetime
(2 answers)
Closed 3 years ago.
I'm trying to unmarshal JSON with time value. I have this JSON structure.
{
"Nick": "cub",
"Email": "cub875#rambler.ru",
"Created_at": "2017-10-09",
"Subscribers": [
{
"Email": "rat1011#rambler.ru",
"Created_at": "2017-11-30"
},
{
"Email": "hound939#rambler.ru",
"Created_at": "2016-07-15"
},
{
"Email": "worm542#rambler.ru",
"Created_at": "2017-02-16"
},
{
"Email": "molly1122#rambler.ru",
"Created_at": "2016-11-30"
},
{
"Email": "goat1900#yandex.ru",
"Created_at": "2018-07-10"
},
{
"Email": "duck1146#rambler.ru",
"Created_at": "2017-09-04"
},
{
"Email": "eagle1550#mail.ru",
"Created_at": "2018-01-03"
},
{
"Email": "prawn1755#rambler.ru",
"Created_at": "2018-04-20"
},
{
"Email": "platypus64#yandex.ru",
"Created_at": "2018-02-17"
}
]
}
And a function that implements reading from JSON file to struct User. Everything if fine but when I set CreatedAt field in User struct to time.Time type I get 0001-01-01 00:00:00 +0000 UTC value for each field with type format in JSON file.
type Subscriber struct {
Email string `json: "Email"`
CreatedAt time.Time `json: "Created_at"`
}
type User struct {
Nick string `json: "Nick"`
Email string `json: "Email"`
CreatedAt string `json: "Created_at"`
Subscribers []Subscriber `json: "Subscribers"`
}
func main() {
// Slice where objects from users.json will be pushed
var jsonData []User
// ReadJSON and push User struct to jsonData slice
readJSON("users.json", &jsonData)
for _, user := range jsonData {
fmt.Println(user.Nick, user.Email, user.CreatedAt) // 0001-01-01 00:00:00 +0000 UTC
fmt.Println(user.Subscribers)
}
}
func readJSON(fileName string, userSlice *[]User) {
file, err := os.Open(fileName)
if err != nil {
log.Fatal(err)
}
bytes, err := ioutil.ReadAll(file)
if err != nil {
log.Fatal(err)
}
err1 := json.Unmarshal(bytes, &userSlice)
if err1 != nil {
log.Fatal(err1)
}
}
What is an appropriate way to read time from JSON file to User in RFC3339 format?

You can use a custom time type that implements the json.Unmarshaler interface.
You can start with this struct:
type CustomTime struct {
time.Time // Embed time.Time to allow calling of normal time.Time methods
}
Then add the required UnmarshalJSON([]byte) error function. It could look like this:
func (c *CustomTime) UnmarshalJSON(b []byte) error {
if len(b) < 3 {
// Empty string: ""
return fmt.Errorf("Empty time value")
}
t, err := time.Parse("2006-01-02", string(b[1:len(b)-1])) // b[1:len(b)-1] removes the first and last character, as they are quotes
if err != nil {
return err
}
c.Time = t
return nil
}
You can try the example on the Go Playground

Related

Parse nested json array from a file

I have a json file sample.json containing a json array as follows:
[
{
"time": "2021-01-04T00:11:32.362Z",
"extra_data": {
"id": "123"
},
"info": "event123"
},
{
"time": "2021-01-05T00:11:32.362Z",
"extra_data": {
"id": "456"
},
"info": "event456"
},
{
"time": "2021-01-06T00:11:32.362Z",
"extra_data": {
"id": "789"
},
"info": "event789"
}
]
I am trying to unmarshal this json array so that for each json object (i.e. event), I can access the id and info values. This is what I have so far:
func main() {
file, err := ioutil.ReadFile("/Users/janedoe/Downloads/sample.json")
var events Event
json.Unmarshal([]byte(file), &events)
fmt.Println(reflect.TypeOf(events))
// Reading each value by its key for each event
fmt.Println("Event123_Time :", events.Timestamp,
"\nEvent123_ExtraData_Id :", events.ExtraData.Id,
"\nEvent123_Info :", events.Info)
}
type Event struct {
Time string `json:"time"`
ExtraData ExtraData `json:"extra_data"`
Info string `json:"info"`
}
type ExtraData struct {
Id string `json:"id"`
}
The output that I am getting is:
main.Event
Event123_Time :
Event123_ExtraData_Id :
Event123_Info :
I am not getting any values, implying that the marshalling is not happening as expected. How can I fix this?
Since your json is an array, I think you want:
var events []Event
not
var events Event
Here's a version with that change:
func main() {
file, err := ioutil.ReadFile("/Users/janedoe/Downloads/sample.json")
if err != nil {
panic(err)
}
var events []Event
if err := json.Unmarshal([]byte(file), &events); err != nil {
panic(err)
}
fmt.Println(reflect.TypeOf(events))
for i, event := range events {
// Reading each value by its key for each event
fmt.Println(i, "Event123_Time :", event.Time,
"\nEvent123_ExtraData_Id :", event.ExtraData.Id,
"\nEvent123_Info :", event.Info)
}
}
type Event struct {
Time string `json:"time"`
ExtraData ExtraData `json:"extra_data"`
Info string `json:"info"`
}
type ExtraData struct {
Id string `json:"id"`
}

Unmarshal nested JSON with different structure but same keys

I'm trying to unmarshal some nested JSON which looks like:
{
"id": "aRandomId",
"type": "aRandomType",
"aRandomField": {
"type": "someType",
"createdAt": "2020-07-07T15:50:02",
"object": "anObject",
"modifiedAt": "2020-07-07T15:50:02"
},
"aParameter": {
"type": "Property",
"createdAt": "2020-07-07T15:50:02",
"value": "myValue",
"modifiedAt": "2020-07-07T15:50:02"
},
"location": {
"type": "GeoProperty",
"value": {
"type": "Point",
"coordinates": [
7.0054,
40.9999
]
}
},
... other things with type, value ...
"createdAt": "2020-07-07T15:50:02",
"modifiedAt": "2020-07-07T15:50:02",
}
I would like to get all keys and values which are: type, createdAt, Value (also if they are nested)
Actually, I have 2 structs:
type Attribute struct {
Type string `json:"type"`
CreatedAt string `json:"createdAt"`
Value string `json:"value"`
ModifiedAt string `json:"modifiedAt"`
}
type Entity struct {
Id string `json:"id"`
Type string `json:"type"`
CreatedAt string `json:"createdAt"`
Attribute Attribute
}
in := []byte(buf.String())
var entity Entity
err := json.Unmarshal(in, &entity)
if err != nil {
panic(err)
}
frame.Fields = append(frame.Fields,
data.NewField("key", nil, []string{"type : ", "createdAt : ", "name : "}),
)
frame.Fields = append(frame.Fields,
data.NewField("value", nil, []string{entity.Type, entity.CreatedAt, entity.Attribute.Value}),
)
The problem is there can be several different Attribute struct andIi can't provide them all.
I would like to display all key (only type, createdAt and Value) in one frame and all their value in another.
Maybe have something like that?
type Entity struct {
attribute List<Attribute>
}
type Attribute struct{
Type string
CreatedAt string
Value string
ModifiedAt string
}
The problem is there can be several different Attribute struct andIi can't provide them all
It looks like your JSON data can have a set of keys with similar values (Attribute) and you can't know how many of them might be in the data.
For this case, you can use a map[string]json.RawMessage as the starting entity to unmarshal to
var e map[string]json.RawMessage
if err := json.Unmarshal([]byte(jsonData), &e); err != nil {
panic(err)
}
You can then range over the values to see if you can unmarshal them into Attribute type
for k, v := range e {
var a Attribute
if err := json.Unmarshal(v, &a); err == nil {
log.Printf("Got attribute %s: %s", k, string(v))
}
}
Run it on playground

Trying to parse the JSON and create an extracted JSON

I am trying to create a JSON object on the fly based on the data received from the API.
Sample Data received: Unmarshalled the data into CiItems struct given below
{
"class_name": "test",
"configuration_items": [
{
"id": "ea09a24f-01ef-42ad-ab19-e0369341d9b3",
"ci_name": "makk",
"comments": null,
"created_by": "mike",
"updated_by": "sam",
"created": "2019-08-02T21:16:35.656Z",
"updated": "2019-08-02T21:21:08.073Z",
"ci_state_id": "randomid",
"super_ci_id": null,
"ci_attributes": [
{
"attribute_id": "c995c693-b97c-4863-a61b-81a5d904c967",
"df_attribute_value": "xsmall",
"attribute_name": "tname",
"data_type": "string"
},
{
"attribute_id": "58845f48-7d2a-4c8c-8591-eaf59a23d84d",
"df_attribute_value": "vmware",
"attribute_name": "provider",
"data_type": "string"
}
]}]}
Below are the structs created:
type Attribute struct {
AttributeID string `json:"attribute_id "`
DfAttributeValue string `json:"df_attribute_value"`
AttName string `json:"attribute_name"`
DataType string `json:"data_type"`
}
// Attributes - array of Attribute
type Attributes []Attribute
// CiItem - confiuraion item of a VM
type CiItem struct {
ID string `json:"ci_id"`
Created string `json:"created"`
Updated string `json:"updated"`
CreatedBY string `json:"created_by"`
UpdatedBY string `json:"updated_by"`
Atts Attributes `json:"ci_attributes"`
}
// CiItems - array of CiItem
type CiItems struct {
ClassName string `json:"class_name"`
Items []CiItem `json:"configuration_items"`
}
Code to unmarshal the data and create a extracted Json:
func (client *Client) GetList() (CiItems, error) {
var out CiItems
err := client.doJsonRequest("GET", &out)
log.Info(out)
if err != nil {
return out, err
}
var output map[string]interface{}
//parseMap(out.Items[0].(map[string]interface{}))
extractBase(out.Items[0], &output)
return out, nil
}
func extractBase(ci interface{}, output interface{}) {
fields := reflect.TypeOf(ci)
values := reflect.ValueOf(ci)
num := fields.NumField()
for i := 0; i < num; i++ {
field := fields.Field(i)
value := values.Field(i)
if string(field.Name) != "Atts" {
name := string(field.Name)
output[name] = string(value)
}
}
}
I am trying to create a JSON with key value of id, ci_name, created_by, updated_by, attribute_name as below
{
"ci_name": "makk",
"created_by": "mike",
"updated_by": "sam",
"created": "2019-08-02T21:16:35.656Z",
"updated": "2019-08-02T21:21:08.073Z",
"tname": "xsmall",
"provider": "vmware"
}
I have tried using reflect and other methods
Create a map using values from CiItem fields and return it from the function.
func extractBase(ci *CiItem) map[string]interface{} {
result := map[string]interface{}{
"ci_name": ci.Name,
"created_by": ci.CreatedBY,
"updated_by": ci.UpdatedBY,
"created": ci.Created,
"updated": ci.Updated,
}
for _, a := range ci.Atts {
result[a.AttName] = a.DfAttributeValue
}
return result
}

Issue parsing JSON file with Go

I make a GET request, and receive a JSON file, that I cannot parse.
Here is the data I have to parse
{
"codeConv": "ACC00000321",
"start": "2019-07-01T00:00:00Z",
"end": "2019-08-21T00:00:00Z",
"details": [
{
"idPrm": "30000000123456",
"keys": [
{
"timestamp": "2019-07-01T00:00:00Z",
"value": 0
},
{
"timestamp": "2019-07-01T00:30:00Z",
"value": 0
},
...
]
},
{
"idPrm": "30000000123457",
"keys": [
{
"timestamp": "2019-07-01T00:00:00Z",
"value": 1
},
{
"timestamp": "2019-07-01T00:30:00Z",
"value": 2
},
...
]
}
]
}
Here are my objects:
type APIMain struct {
CodeConv string `json:"codeConv"`
Start string `json:"start"`
End []Keys `json:"end"`
Details []APIData `json:"details"`
}
//APIData match the data we receive from api
type APIData struct {
Prm string `json:"idPrm"`
Keys []Keys `json:"keys"`
}
type Keys struct {
Timestamp string `json:"timestamp"`
Value string `json:"value"`
}
and here is the method to get data with basic auth:
tr := &http.Transport{
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
}
client := &http.Client{Transport: tr}
req, err := http.NewRequest("GET", url, nil)
if err != nil {
return nil, err
}
if login != "" && password != "" {
req.SetBasicAuth(login, password)
}
response, err := client.Do(req)
//defer response.Body.Close()
if err != nil {
return nil, err
}
if response.StatusCode != 200 {
fmt.Println(response.Body)
panic(response.Status)
}
err = json.NewDecoder(response.Body).Decode(&result)
fmt.Println("result", result) // result is empty array
How can I see if the problem is in a request, or if the problem is in parsing ?
I have get a response.Body object, but it needs to be decoded.
I fixed it using: https://mholt.github.io/json-to-go/
which generated this structure:
type AutoGenerated struct {
CodeConv string `json:"codeConv"`
Start time.Time `json:"start"`
End time.Time `json:"end"`
Details []struct {
IDPrm string `json:"idPrm"`
Keys []struct {
Timestamp time.Time `json:"timestamp"`
Value float64 `json:"value"`
} `json:"keys"`
} `json:"details"`
}
Thanks for your comments
Great time saver !

Go json.Unmarshall() works with single entity but not with slice

I'm using Go for a simple http client. Here's the entity I'm unmarshalling:
type Message struct {
Id int64
Timestamp int64
Text string
Author User
LastEdited int64
}
type User struct {
Id int64
Name string
}
A single entity looks like this in JSON:
{
"text": "hello, can you hear me?",
"timestamp": 1512964818565,
"author": {
"name": "andrea",
"id": 3
},
"lastEdited": null,
"id": 8
}
Go/json has no problem unmarshalling the single entity:
var m Message
err = json.Unmarshal(body, &m)
if err != nil {
printerr(err.Error())
}
println(m.Text)
However, if the return of the endpoint is multiple entities:
[
{
"text": "hello, can you hear me?",
"timestamp": 1512964800981,
"author": {
"name": "eleven",
"id": 4
},
"lastEdited": null,
"id": 7
}
]
And I change my corresponding Unmarshall to work on a slice of structs, Go throws an error:
var m []Message
err = json.Unmarshal(body, &m)
if err != nil {
printerr(err.Error()) // unexpected end of JSON input
}
for i := 0; i < len(m); i++ {
println(m[i].Text)
}
What gives?
Works fine for me (try it on playground), where are you getting the payload data from? sounds like that's truncating it.
package main
import (
"encoding/json"
"fmt"
)
type Message struct {
Id int64
Timestamp int64
Text string
Author User
LastEdited int64
}
type User struct {
Id int64
Name string
}
func main() {
body := []byte(`[
{
"text": "hello, can you hear me?",
"timestamp": 1512964800981,
"author": {
"name": "eleven",
"id": 4
},
"lastEdited": null,
"id": 7
}
]`)
var m []Message
err := json.Unmarshal(body, &m)
if err != nil {
fmt.Printf("error: %v") // unexpected end of JSON input
}
for i := 0; i < len(m); i++ {
fmt.Println(m[i].Text)
}
}
running it gives this output
hello, can you hear me?