I have start working in golang, I am trying to parse JSON string to struct but its not working.
JSON String:
dailies":[{"userAccessToken":"acessToken","uploadStartTimeInSeconds":1499744832,"uploadEndTimeInSeconds":1499744832,"callbackURL":"callbackurl"}]}
type pingDataFormat struct {
userAccessToken string
uploadStartTimeInSeconds int
uploadEndTimeInSeconds int
callbackURL string
}
Below is code which convert JSON String to structs
pingJSON := make(map[string][]pingDataFormat)
err := json.Unmarshal([]byte(pingData), &pingJSON)
if err != nil {
panic(err)
}
fmt.Printf("\n\n json object:::: %v", pingJSON)
output is:
json object:::: map[dailies:[{ 0 0 }]]
Here "dailies" dont have data, So do i following right approach to parse and how to get data after parse?
The fields in pingDataFormat need to be exported (start with a capital letter)
type pingDataFormat struct {
UserAccessToken string `json:"userAccessToken"`
UploadStartTimeInSeconds int `json:"uploadStartTimeInSeconds"`
UploadEndTimeInSeconds int `json:"uploadEndTimeInSeconds"`
CallbackURL string `json:"callbackURL"`
}
That should solve your problem
https://play.golang.org/p/Z3yszFl01L
Related
Here I want to take DefaultResponse as String but later validate if it's a valid json before mapping to any json struct.
Complete code with validation can be found here :
https://go.dev/play/p/knGNMj1QG3l
type AddEndpointRequest struct {
Method string `json:"method"`
ContentType int `json:"contentType"`
DefaultStatusCode int `json:"defaultStatusCode"`
DefaultResponse string `json:"defaultResponse"`
}
I tried different option but none of them are working
if I pass this : “defaultResponse”:{“title”:“Buy cheese and bread for breakfast.”}
Getting error : json: cannot unmarshal object into Go struct field AddEndpointRequest.defaultResponse of type string
body := {"method":"GET","contentType":1,"defaultWaitTimeInMillis":100,"defaultStatusCode":200,"defaultResponse":"[{"p":"k"}]"}
Error : invalid character 'p' after object key:value pair
3)
body := {"method":"GET","contentType":1,"defaultWaitTimeInMillis":100,"defaultStatusCode":200,"defaultResponse":"{"p":"k"}"}
./prog.go:21:117: syntax error: unexpected { at end of statement
body := {"method":"GET","contentType":1,"defaultWaitTimeInMillis":100,"defaultStatusCode":200,"defaultResponse":"{/"p/":/"k/"}"} Error : ./prog.go:21:130: syntax error: unexpected literal "} at end of statement
And many more
Option A:
Declare the field as a string. Use a valid JSON string as the field value in the document. Note the escaping of the quotes in the string.
type AddEndpointRequest struct {
Method string `json:"method"`
ContentType int `json:"contentType"`
DefaultStatusCode int `json:"defaultStatusCode"`
DefaultResponse string `json:"defaultResponse"`
}
…
body := `{"method":"GET","contentType":1,"defaultWaitTimeInMillis":100,"defaultStatusCode":200,"defaultResponse":"{\"p\":\"k\"}"}`
Convert the field to a []byte to unmarshal:
err := json.Unmarshal([]byte(request.DefaultResponse), &data)
https://go.dev/play/p/ailgQQ3eQBH
Option B:
Declare the field as json.RawMessage. Use any valid JSON in the document.
type AddEndpointRequest struct {
Method string `json:"method"`
ContentType int `json:"contentType"`
DefaultStatusCode int `json:"defaultStatusCode"`
DefaultResponse json.RawMessage `json:"defaultResponse"`
}
…
body := `{"method":"GET","contentType":1,"defaultWaitTimeInMillis":100,"defaultStatusCode":200,"defaultResponse":{"p":"k"}}`
The call to json.Unmarshal([]byte(body), &request) validates the json.RawMessage field. If the call to json.Unmarshal does not return an error, then the application is assured that AddEndpointRequest.DefaultResponse contains valid JSON.
Unmarshal the field like this:
err := json.Unmarshal(request.DefaultResponse, &data)
https://go.dev/play/p/Xd_gWzJmvC_K
If you want to set a struct in defaultResponse, I think doing it like this should work.
package main
import (
"encoding/json"
"errors"
"fmt"
)
type AddEndpointRequest struct {
Method string `json:"method"`
ContentType int `json:"contentType"`
DefaultStatusCode int `json:"defaultStatusCode"`
DefaultResponse string `json:"defaultResponse"`
}
func main() {
body := `{"method":"GET","contentType":1,"defaultWaitTimeInMillis":100,"defaultStatusCode":200,"defaultResponse":{"title":"Buy cheese and bread for breakfast."}}`
fmt.Println("Hello GO")
err := validateBody(body)
if err != nil {
fmt.Println("erro102")
fmt.Println(err)
}
}
func validateBody(body string) error {
var data map[string]interface{}
err := json.Unmarshal([]byte(body), &data)
if err != nil {
return errors.New("invalid json body provided for the request")
}
fmt.Println("data===>", data)
fmt.Println("defaultResponse===>", data["defaultResponse"].(map[string]interface{})["title"])
return nil
}
I'm reading a file.json into memory. It's an array of objects, sample:
[
{"id":123123,"language":"ja-JP","location":"Osaka"}
,{"id":33332,"language":"ja-JP","location":"Tokyo"}
,{"id":31231313,"language":"ja-JP","location":"Kobe"}
]
I want to manipulate certain keys in this JSON file, so that they start with uppercase. Meaning
"language" becomes "Language" each time it's found. What I've done so far is to make a struct representing each object, as such:
type sampleStruct struct {
ID int `json:"id"`
Language string `json:"Language"`
Location string `json:"Location"`
}
Here, I define the capitalization. Meaning, id shouldn't be capitalized, but location and language should.
Rest of the code is as such:
func main() {
if len(os.Args) < 2 {
fmt.Println("Missing filename parameter.")
return
}
translationfile, err := ioutil.ReadFile(os.Args[1])
fileIsValid := isValidJSON(string(translationfile))
if !fileIsValid {
fmt.Println("Invalid JSON format for: ", os.Args[1])
return
}
if err != nil {
fmt.Println("Can't read file: ", os.Args[1])
panic(err)
}
}
func isValidJSON(str string) bool {
var js json.RawMessage
return json.Unmarshal([]byte(str), &js) == nil
}
// I'm unsure how to iterate through the JSON objects and only uppercase the objects matched in my struct here.
func upperCaseSpecificKeys()
// ...
Desired output, assuming the struct represents the whole data object, transform each key as desired:
[
{"id":123123,"Language":"ja-JP","Location":"Osaka"}
,{"id":33332,"Language":"ja-JP","Location":"Tokyo"}
,{"id":31231313,"Language":"ja-JP","Location":"Kobe"}
]
The documentation on json.Unmarshal says (with added emphasis):
To unmarshal JSON into a struct, Unmarshal matches incoming object
keys to the keys used by Marshal (either the struct field name or its
tag), preferring an exact match but also accepting a case-insensitive
match
See example here: https://play.golang.org/p/1vv8PaQUOfg
One way is to implement custom marshal method, although not very flexible:
type upStruct struct {
ID int `json:"id"`
Language string
Location string
}
type myStruct struct {
ID int `json:"id"`
Language string `json:"language"`
Location string `json:"location"`
}
func (m myStruct) MarshalJSON() ([]byte, error) {
return json.Marshal(upStruct(m))
}
....
func main() {
var mySArr []myStruct
// 1. Unmarshal the input
err := json.Unmarshal([]byte(myJson), &mySArr)
if err != nil {
log.Fatal(err)
}
fmt.Printf("Input: \n%+v\n", mySArr)
// 2. Then, marshal it using our custom marshal method
val, err := json.Marshal(mySArr)
if err != nil {
log.Fatal(err)
}
fmt.Printf("Output: \n%v\n", string(val))
}
Link to working code: https://play.golang.org/p/T4twqPc34k0
Thanks to mkopriva
Hey guys I am getting websocket information from external Api and it's give me json response in this way:
`{"name":"message","args":["{\"method\":\"chatMsg\",\"params\":{\"channel\":\"channel\",\"name\":\"name\",\"nameColor\":\"B5B11E\",\"text\":\"https://play.spotify.com/browse\",\"time\":1455397119}}"]}`
I am putting it into this struc
type main struct {
Name string `json:"name"`
Args []arg `json:"args"`
}
type arg struct {
Method string`json:"method"`
Params par `json:"params"`
}
type par struct {
Channel string `json:"channel,omitempty"`
Name string `json:"name,omitempty"`
NameColor string `json:"nameColor,omitempty"`
Text string `json:"text,omitempty"`
Time int64 `json:"time,omitempty"`
}
and decode it with code
sReplace := strings.NewReplacer(`"{`, "{", `"]`, "]", "\\", ``)
strN := sReplace.Replace(str)
r := strings.NewReader(strN)
d := json.NewDecoder(r)
m := main{}
I am getting error
invalid character 'h' after object key:value pair
I know that the error is result of text field value. Is there any good way to clean it up or tell the decoder to ignore content of text field?
The application is parsing data containing the substring "text":"<a href="https. This is not valid JSON. The error message is complaining about the h in href.
Because the JSON value includes encoded JSON values, the application must decode in two steps:
type main struct {
Name string `json:"name"`
Args []string `json:"args"`
}
type arg struct {
Method string `json:"method"`
Params par `json:"params"`
}
type par struct {
Channel string `json:"channel,omitempty"`
Name string `json:"name,omitempty"`
NameColor string `json:"nameColor,omitempty"`
Text string `json:"text,omitempty"`
Time int64 `json:"time,omitempty"`
}
str := `{"name":"message","args":["{\"method\":\"chatMsg\",\"params\":{\"channel\":\"channel\",\"name\":\"name\",\"nameColor\":\"B5B11E\",\"text\":\"https://play.spotify.com/browse\",\"time\":1455397119}}"]}`
var m main
if err := json.Unmarshal([]byte(str), &m); err != nil {
log.Fatal(err)
}
var args arg
if err := json.Unmarshal([]byte(m.Args[0]), &args); err != nil {
log.Fatal(err)
}
playground example
i'm trying to decode 2 JSON items into the same struct, because the second JSON complet the first one, but it doesn't work (do nothing) have you got some ideas ?
func getUserClip(this *LibraryController, id string) (*Clip){
//Test Api
//Send Request to azure search
Data := Clip{}
if req := GetClipById("b373400a-bd7e-452a-af68-36992b0323a5"); req == nil {
return nil
} else {
str, err := req.String()
if err != nil {
beego.Debug("Error Json req.String: ", err)
}
//Uncode Json to string
if err := json.Unmarshal([]byte(str), &Data); err != nil {
beego.Debug("Error json", err)
}
for i := range Data.Value {
if req = GetCliRedis(Data.Value[i].Id); err != nil {
return nil
} else {
str, err := req.String()
beego.Debug("JSON REDIS DEBUG: ", str)
if err != nil {
beego.Debug("Error Json req.String: ", err)
}
if err := json.Unmarshal([]byte(str), &Data); err != nil {
beego.Debug("Error json", err)
}
}
i++
}
}
return &Data
}
and the struct
type Clip struct {
Value []InfoClip `json:value`
}
type InfoClip struct {
Id string `json:id`
CreatedAt time.Time `json:createdAt`
StartTimeCode int `json:startTimeCode`
EndTimeCode int `json:endTimeCode`
Metas metas `json:metas`
Tags []string `json:tags`
Categories []string `json:categories`
UserId string `json:userId`
SourceId string `json:sourceId`
ProviderName string `json:providerName`
ProviderReference string `json:providerReference`
PublicationStatus string `json:publicationStatus`
Name string `json:name`
FacebookPage string `json:facebookPage`
TwitterHandle string `json:twitterHandle`
PermaLinkUrl string `json:permalinkUrl`
Logo string `json:logo`
Link string `json:link`
Views int `json:views`
}
type metas struct {
Title string `json:title`
Tags []string `json:tags`
Categories []string `json:categories`
PermaLink string `json:permalink`
}
The JSON I receive is:
{
"clipId":"9b2ea9bb-e54b-4291-ba16-9211fa3c755f",
"streamUrl":"https://<edited out>/asset-32e43a5d-1500-80c3-cc6b-f1e4fe2b5c44\/6c53fbf5-dbe9-4617-9692-78e8d76a7b6e_H264_500kbps_AAC_und_ch2_128kbps.mp4?sv=2012-02-12&sr=c&si=17ed71e8-5176-4432-8092-ee64928a55f6&sig=KHyToRlqvwQxWZXVvRYOkBOBOF0SuBLVmKiGp4joBpw%3D&st=2015-05-18T13%3A32%3A41Z&se=2057-05-07T13%3A32%3A41Z",
"startTimecode":"6",
"endTimecode":"16",
"createdAt":"2015-05-19 13:31:32",
"metas":"{\"title\":\"Zapping : Obama, Marine Le Pen et Michael Jackson\",\"tags\":[\"actualite\"],\"categories\":[\"actualite\"],\"permalink\":\"http:\/\/videos.lexpress.fr\/actualite\/zapping-obama-marine-le-pen-et-michael-jackson_910357.html\"}",
"sourceId":"6c53fbf5-dbe9-4617-9692-78e8d76a7b6e",
"providerName":"dailymotion",
"providerReference":"x1xmnxq",
"publicationStatus":"1",
"userId":"b373400a-bd7e-452a-af68-36992b0323a5",
"name":"LEXPRESS.fr",
"facebookPage":"https:\/\/www.facebook.com\/LExpress",
"twitterHandle":"https:\/\/twitter.com\/lexpress",
"permalinkBaseURL":"https:\/\/tym.net\/fr\/{CLIP_ID}",
"logo":"lexpress-120px.png",
"link":"http:\/\/videos.lexpress.fr\/"
}
The Redis complet the azure search missing information :
here the struct :
type Clip struct {
Value []SearchClip `json:value`
}
type SearchClip struct {
Id string `json:id`
CreatedAt string`json:createdAt`
Tags []string `json:tags`
Categories []string `json:categories`
UserId string `json:userId`
SourceId string `json:sourceId`
Views int `json:views`
}
this is the basic information and redis complet this
I don't want to merge 2 struct into a third one i think it's not the better process, i will do it if it's the last solution.
For lack of activity I'm just gonna post the embedding option as a solution. It's probably the simplest way to do what you want.
type ClipInfoAndMeta struct {
Metas
InfoClip
}
Note I upper cased the name on metas not sure it's necessary but I believe it will be. The language feature being used here is called 'embedding' and it works a lot like composition except that the fields/methods for embedded types are more or less 'hoisted' to the containing types scope. IE with an instance of ClipInfoAndMeta you can directly access any exported field that is defined on InfoClip.
One oddity of your set up is that you'll have collisions on field names between the two types. Not sure how that would play out. With all this being said, it would be helpful to see the json string you're trying to Unmarshal from. As I've been writing this I realized that metas is just a subset of InfoClip. Which has confused me about what you are actually trying to do? I mean, if the data coming back is all in one object, it would mean InfoClip is sufficient for storing all of it. If that is the case you have no reason for the other object... And if you want to trim down the fields which get passed to the display layer of your app you should just define a method on the InfoClip type like func (i *InfoClip) GetMetas() Metas { return &Metas{ ... } } then you can just deal with the one type everywhere and hand off the Metas to the display layer when it's needed.
After a lot of trial and error, I present to you this fully functional solution:
package main
import (
"encoding/json"
"fmt"
"log"
"time"
)
type Clip struct {
Value []InfoClip `json:value`
}
type customTime struct {
time.Time
}
const ctLayout = "2006-01-02 15:04:05"
func (ct *customTime) UnmarshalJSON(b []byte) (err error) {
if b[0] == '"' && b[len(b)-1] == '"' {
b = b[1 : len(b)-1]
}
ct.Time, err = time.Parse(ctLayout, string(b))
return
}
type InfoClip struct {
Id string `json:"clipId"`
CreatedAt customTime `json:"createdAt"`
StartTimeCode string `json:"startTimeCode"` //if you want ints here, you'll have to decode manually, or fix the json beforehand
EndTimeCode string `json:"endTimeCode"` //same for this one
Metas metas `json:"-"`
MetasString string `json:"metas"`
Tags []string `json:"tags"`
Categories []string `json:"categories"`
UserId string `json:"userId"`
SourceId string `json:"sourceId"`
ProviderName string `json:"providerName"`
ProviderReference string `json:"providerReference"`
PublicationStatus string `json:"publicationStatus"`
Name string `json:"name"`
FacebookPage string `json:"facebookPage"`
TwitterHandle string `json:"twitterHandle"`
PermaLinkUrl string `json:"permalinkBaseURL"`
Logo string `json:"logo"`
Link string `json:"link"`
Views int `json:"views"`
}
type metas struct {
Title string `json:"title"`
Tags []string `json:"tags"`
Categories []string `json:"categories"`
PermaLink string `json:"permalink"`
}
var jsonString = `{
"clipId":"9b2ea9bb-e54b-4291-ba16-9211fa3c755f",
"streamUrl":"https://<edited out>/asset-32e43a5d-1500-80c3-cc6b-f1e4fe2b5c44\/6c53fbf5-dbe9-4617-9692-78e8d76a7b6e_H264_500kbps_AAC_und_ch2_128kbps.mp4?sv=2012-02-12&sr=c&si=17ed71e8-5176-4432-8092-ee64928a55f6&sig=KHyToRlqvwQxWZXVvRYOkBOBOF0SuBLVmKiGp4joBpw%3D&st=2015-05-18T13%3A32%3A41Z&se=2057-05-07T13%3A32%3A41Z",
"startTimecode":"6",
"endTimecode":"16",
"createdAt":"2015-05-19 13:31:32",
"metas":"{\"title\":\"Zapping : Obama, Marine Le Pen et Michael Jackson\",\"tags\":[\"actualite\"],\"categories\":[\"actualite\"],\"permalink\":\"http:\/\/videos.lexpress.fr\/actualite\/zapping-obama-marine-le-pen-et-michael-jackson_910357.html\"}",
"sourceId":"6c53fbf5-dbe9-4617-9692-78e8d76a7b6e",
"providerName":"dailymotion",
"providerReference":"x1xmnxq",
"publicationStatus":"1",
"userId":"b373400a-bd7e-452a-af68-36992b0323a5",
"name":"LEXPRESS.fr",
"facebookPage":"https:\/\/www.facebook.com\/LExpress",
"twitterHandle":"https:\/\/twitter.com\/lexpress",
"permalinkBaseURL":"https:\/\/tym.net\/fr\/{CLIP_ID}",
"logo":"lexpress-120px.png",
"link":"http:\/\/videos.lexpress.fr\/"
}`
func main() {
res := parseJson(jsonString)
fmt.Printf("%+v\n",res)
}
func parseJson(theJson string) InfoClip {
toParseInto := struct {
InfoClip
MetasString string `json:"metas"`
}{
InfoClip: InfoClip{},
MetasString: ""}
err := json.Unmarshal([]byte(jsonString), &toParseInto)
if err != nil {
log.Panic(err)
}
err = json.Unmarshal([]byte(toParseInto.MetasString), &toParseInto.InfoClip.Metas)
if err != nil {
log.Panic(err)
}
return toParseInto.InfoClip
}
What are we doing in the parseJson function?
We create a new struct and assign that to the toParseInto variable. We design the struct in a way that it contains all of the fields from InfoClip via embedding, and We add a field to temporarily hold the JSON string metas.
We then unmarshal into that struct, which, after fixing the issues listed below, works fine.
After that, we unmarshal that inner JSON into the correct field in the embedded InfoClip.
We can now easily return that embedded InfoClip to get what we really wanted.
Now, all the issues I have identified in your original solution:
The time format in your JSON is not the standard time format to be used in JSON. That is defined in some RFC, but anyways: because of that, we have to use our own type customTime to parse that. It handles just like a normal time.Time, because that is embedded within.
All your json tags were wrong. All of them had missing quotes, and some were just not even correct.
startTimeCode and endTimeCode are strings in the JSON, not ints
Left to you to improve:
Error handling: Don't just panic in the parseJson function, but rather return the error somehow
If you want startTimecode and endTimecode to be available as ints, parse them manually. You can employ a "hack" similar to the one I used to parse the inner JSON.
One final note, not related to this answer but rather to your question: If you had provided both your code and the JSON with your original question, you would have had an answer in probably less than an hour. Please, please don't make this harder than it needs to be.
EDIT: I forgot to provide my sources, I used this question to parse your time format.
I have struct
type tySurvey struct {
Id int64 `json:"id,omitempty"`
Name string `json:"name,omitempty"`
}
I do json.Marshal write JSON bytes in HTML page. jQuery modifies name field in object and encodes object using jQueries JSON.stringify and jQuery posts string to Go handler.
id field encoded as string.
Sent: {"id":1} Received: {"id":"1"}
Problem is that json.Unmarshal fails to unmarshal that JSON because id is not integer anymore.
json: cannot unmarshal string into Go value of type int64
What is best way to handle such data? I do not wish to manually convert every field. I wish to write compact, bug free code.
Quotes is not too bad. JavaScript does not work well with int64.
I would like to learn the easy way to unmarshal json with string values in int64 values.
This is handled by adding ,string to your tag as follows:
type tySurvey struct {
Id int64 `json:"id,string,omitempty"`
Name string `json:"name,omitempty"`
}
This can be found about halfway through the documentation for Marshal.
Please note that you cannot decode the empty string by specifying omitempty as it is only used when encoding.
use json.Number
type tySurvey struct {
Id json.Number `json:"id,omitempty"`
Name string `json:"name,omitempty"`
}
You could also create a type alias for int or int64 and create a custom json unmarshaler
Sample code:
Reference
// StringInt create a type alias for type int
type StringInt int
// UnmarshalJSON create a custom unmarshal for the StringInt
/// this helps us check the type of our value before unmarshalling it
func (st *StringInt) UnmarshalJSON(b []byte) error {
//convert the bytes into an interface
//this will help us check the type of our value
//if it is a string that can be converted into a int we convert it
///otherwise we return an error
var item interface{}
if err := json.Unmarshal(b, &item); err != nil {
return err
}
switch v := item.(type) {
case int:
*st = StringInt(v)
case float64:
*st = StringInt(int(v))
case string:
///here convert the string into
///an integer
i, err := strconv.Atoi(v)
if err != nil {
///the string might not be of integer type
///so return an error
return err
}
*st = StringInt(i)
}
return nil
}
func main() {
type Item struct {
Name string `json:"name"`
ItemId StringInt `json:"item_id"`
}
jsonData := []byte(`{"name":"item 1","item_id":"30"}`)
var item Item
err := json.Unmarshal(jsonData, &item)
if err != nil {
log.Fatal(err)
}
fmt.Printf("%+v\n", item)
}