I'm working on a middleware using golang.
I'm consuming a REST-API which returns a Date in form of a LocalDateTime (e.g. "created": "2022-01-09T00:00:00",) and it should get mapped into an protoc message with the data class google/protobuf/timestamp.proto as we don't want to do the converting in the frontend. But apparently the timestamp.proto only supports DateTime with a timezone (so like that "created": "2022-01-09T00:00:00Z...") but as its an external API I'm consuming I cant change their datatype to DateTime. Does anyone know how to find and elegant solution without doing the complete mapping/unmashalling process manually?
That's the protoc message:
google.protobuf.Timestamp created = 7 [json_name = "created"];
That's the unmashaller we're using:
err = protojson.Unmarshal(body, protoMessageClass)
That's the error I'm getting:
ERROR: proto: (line ...): invalid google.protobuf.Timestamp value "2021-12-07T00:00:00""
First convert the time you receive as a string to a time.Time object:
t, err := time.Parse("2006-01-02T15:04:05", val)
Here "2006-01-02T15:04:05" represents the layout you expect.
Then you can use timestamppb built-in function to create a protobuf timestamp from a time.Time object:
tpb := timestamppb.New(t)
If you need the opposite, you can use AsTime from timestampb.Timestamp type and then format from time.Time object to make a string.
Related
I am sending this json:
{"origin":{"lat":23.589367061768648,"lng":58.42860314995051},"destination":null,"vehicle":"WATER_TANKER_600GL","scheduled_time":"2022-07-27T14:16:00Z04:00"}
Golang Gin handler function:
func (config *Config) CreateOrder(c *gin.Context) {
userID := auth.GetToken(c).ID()
data := &struct {
Origin models.Origin `json:"origin" binding:"required"`
Vehicle VehicleType `json:"vehicle" binding:"required"`
Destination *models.Destination `json:"destination" `
ScheduledTime *time.Time `json:"scheduled_time"`
}{}
if err := c.ShouldBindWith(data, binding.JSON); err != nil {
fmt.Println(err) // -> prints the below error
res.UnprocessableEntity(c, fmt.Sprintf("%s", err))
return
}
// rest of the code
}
I get this error:
parsing time "\"2022-07-27T14:16:00Z04:00\"" as "\"2006-01-02T15:04:05Z07:00\"": cannot parse "04:00\"" as "\""
Note that JSON does not define a date or timestamp data type. Transferring a timestamp as a JSON string is possible, but how that timestamp string is interpreted–or should be interpreted–is not written in stone. The standard lib (the encoding/json package) supports unmarshaling time.Time values from JSON strings, but it does so by requiring them to adhere the RFC 3339 standard, else the unmarshaling will fail.
time.Time implements json.Unmarshaler, and Time.UnmarshalJSON() states that the JSON string must be in RFC 3339 layout which is:
RFC3339 = "2006-01-02T15:04:05Z07:00"
The Z in the layout means:
Numeric time zone offsets format as follows:
"-0700" ±hhmm
"-07:00" ±hh:mm
"-07" ±hh
Replacing the sign in the format with a Z triggers the ISO 8601 behavior of printing Z instead of an offset for the UTC zone. Thus:
"Z0700" Z or ±hhmm
"Z07:00" Z or ±hh:mm
"Z07" Z or ±hh
Which means if the time has UTC offset, it should be
"2022-07-27T14:16:00Z"
If the time has +04:00 zone, it should be
"2022-07-27T14:16:00+04:00"
The input "2022-07-27T14:16:00Z04:00" is invalid for RFC 3339. Either change your source to provide RFC 3339 compliant timestamp strings, or if you can't or don't want to, you may create your custom type that implements json.Unmarshaler, and you handle this non-RFC 3339 format yourself.
i have to index documents to elasticsearch to an index which has a date field mapping and i'm trying to build a json with this date value, but ballerina says this seems not possible.
I thought about storing this date value into an xml and after that to convert it to a json but xml has the same problem (i thought this might be a trick...).
I tried to store it into a string and after that to extract the json payload from that string but it gives me this error:
error: {ballerina/io}GenericError message=unrecognized token 'date=time=1591128342000'
I thought about dealing with this string to date conversion from inside elasticsearch but i would like to keep this scenario as the last one. I don't like it, beacause i have to do some queries based on timestamp after and storing date as a string would give me additional problems
So is there any way to trick ballerina in order to achive this json containing a date value ?
-----here is snapshot of the code giving me the error-----
It says:
incompatible types: expected 'json', found 'ballerina/time:Time'
JSON is a text format that is completely language independent (see e.g. json.org).
time:Time is a Ballerina language specific type JSON knows nothing about. Because there is no implicit conversion (for a good reason) one have to provide the conversion.
In this case you most likely want to convert time:Time to a ISO 8601 string presentation with time:toString.
The following code (Ballerina 1.2):
import ballerina/io;
import ballerina/time;
public function main() {
var btime = time:currentTime();
var j = <json> {
time: time:toString(btime)
};
io:println(j.toJsonString());
}
Correctly prints:
{"time":"2020-06-03T08:39:07.897+03:00"}
Maryam Ziyad has written a good introduction to Ballerina's JSON support.
The following code is updated for Ballerina Swan Lake Update 1 (2201.1.0) to show how to convert a Ballerina UTC time (time:Utc) to JSON representation. Note that it's also possible to use localized time (time:Civil) but that is no different from time to JSON conversion point of view.
One can read more about Ballerina time handling from the documentation of time module.
import ballerina/io;
import ballerina/time;
public function main() {
time:Utc now = time:utcNow(3);
json j = {
time: time:utcToString(now)
};
io:println(j.toJsonString());
}
That correctly prints:
{"time":"2022-07-20T06:03:46.078Z"}
I have a working (in production) web app (material + angular 5 (5.2.11)). Also I've an API written in .dot core 2 (C#) using Nancy FX and newtonsoft json.
Problem:
DB (mariaDB running on Ubuntu Server): I have this value: 2018-05-16 20:42:36 on a record.
Calling the endpoint yields the correct JSON:
{"timestamp":"2018-05-16T20:42:36Z"}
(the other fields were removed for sanity)
On Angular app I use:
... return this._http.get(this.endpoint + '/' + uuid, { headers:
this._getHeaders }).catch(this.handleError);
Where <T> represents a model that includes timedate: Date; as a property.
Using the service:
this._dataService.getByUuid(uuid).subscribe(result => {
console.log(result);
});
gives:
Object { timedate: "2018-05-16 08:05:36" }
So, the time lacks of AMPM format and I can't display it correctly. {{element.timedate | date: 'dd/MM/yyyy HH:mm' }} does nothing since timedate is just that, a bare string.
What have I tried:
Implementing a different format in JSON output (in NancFx API)
Adding a HTTP INTERCEPTOR
Reading this
Declaring the properties as Date, String
Problem is with any datetime field. The JSON is always on point and so the database.
Any help is appreciate
JSON doesn't have a Date type (only arrays, numbers, string, object, null and undefined), so the converter from JSON to TypeScript cannot know whether it's a date or a plain string.
You need to parse (Date.Parse(yourString) or new Date(yourString)) the Date property everytime your object is deserialized.
** Date.Parse and the Date constructor can take in a Date as well as a string so you don't really have to type check the value before using them.*
I currently having the following problem:
I get a []byte / string via websocket which looks like
eventname {"JSON": "data", "In": "different formats"}
I split the string by the whitespace between the eventname and the JSON data and, depending on the eventname, I want to json.Unmarshal() the JSON data into a specific type or a var of a specific type, to make sure its all type safe.
So I would probably have a map which holds all the possible eventnames and the corresponding type for the JSON data, but im not sure how I would save a type, maybe by reference or by the stringified name?
type EventTypeList map[string]*interface{}
or
type EventTypeList map[string]string
So I can lookup if the event is in the EventTypeList and then let the Unmarshal func parse the data into the type from the map.
So basicly instead of the specific type "SpecificData":
type SpecificData struct {
JSON string
In string
}
socketData := SpecificData{}
err := json.Unmarshal(jsonData, &socketData)
I want to dynamically create the socketData var by the type from the EventTypeList
socketData := [dynamically determine this Type from EventTypeList]{}
After I parsed the data into that type, I want to call all listeners for that event and give them the socketData with the right type, so they can work with the expected data.
But im really not sure how I would accomplish this, if this is even possible or if this is even the right way...
Thanks for any help!
Ok, after reasearching, I think what I wanted to achive is not possible, because the type needs to be know at compile time, as it has been mentioned here:
Golang: cast an interface to a typed variable dynamically
But I think I can just pass the raw JSON string to the listener callback and Unmarshal the data in that function where the type of the data is known.
Thanks for all contributions!
I have a JSON file stored on the local machine. I need to read it in a variable and loop through it to fetch the JSON object values. If I use the Marshal command after reading the file using the ioutil.Readfile method, it gives some numbers as an output. These are my few failed attempts,
Attempt 1:
plan, _ := ioutil.ReadFile(filename) // filename is the JSON file to read
var data interface{}
err := json.Unmarshal(plan, data)
if err != nil {
log.Error("Cannot unmarshal the json ", err)
}
fmt.Println(data)
It gave me following error,
time="2016-12-13T22:13:05-08:00" level=error msg="Cannot unmarshal the json json: Unmarshal(nil)"
<nil>
Attempt 2: I tried to store the JSON values in a struct and then using MarshalIndent
generatePlan, _ := json.MarshalIndent(plan, "", " ") // plan is a pointer to a struct
fmt.Println(string(generatePlan))
It give me the output as string. But if I cast the output to string then I won't be able to loop it as JSON object.
How can we read a JSON file as JSON object in golang? Is it possible to do that?
Any help is appreciated. Thanks in advance!
The value to be populated by json.Unmarshal needs to be a pointer.
From GoDoc :
Unmarshal parses the JSON-encoded data and stores the result in the value pointed to by v.
So you need to do the following :
plan, _ := ioutil.ReadFile(filename)
var data interface{}
err := json.Unmarshal(plan, &data)
Your error (Unmarshal(nil)) indicates that there was some problem in reading the file , please check the error returned by ioutil.ReadFile
Also please note that when using an empty interface in unmarshal, you would need to use type assertion to get the underlying values as go primitive types.
To unmarshal JSON into an interface value, Unmarshal stores one of
these in the interface value:
bool, for JSON booleans
float64, for JSON numbers
string, for JSON strings
[]interface{}, for JSON arrays
map[string]interface{}, for JSON objects
nil for JSON null
It is always a much better approach to use a concrete structure to populate your json using Unmarshal.
If you're looking at this in 2022, the ioutil package has been deprecated. You can still use it but you'll get annoying errors.
Instead you can use the os package.
someStruct := SomeStruct{} // Read errors caught by unmarshal
fileBytes, _ := os.ReadFile(filename)
err := json.Unmarshal(fileBytes, spec)
Note, I'm specifically ignoring the error from os.ReadFile since it will also cause an error in json.Unmarshal for the sake of the example.