i try to put json data to web, i use json.Marshal to create json data.
Flowing picture is fmt.Println(string(jsonOut)) result
i use template.HTMLEscape(w, []byte(jsonOut)) to show in web, it will show like following picture.
the " become ".
why it will show " and how can i do for show "?
If you just want show json in the http response
w.Write(jsonOut)
If you want to show json in html
t, _ := template.New("foo").Parse(`<head></head><body>{{$.data}}</body>`)
_ = t.Execute(w, map[string]string{
"data": string(jsonOut),
})
template.HTMLEscape will escape special character.
use following code can post json data to web
w.Header().Set("Content-Type", "application/json")
w.Write(jsonOut)
reference
https://www.alexedwards.net/blog/golang-response-snippets#json
Related
so I am trying to POST a csv file in JSON format to a website in Golang. I have been successful in POSTing a singular JSON file. However, that is not what I want the purpose of my program to be. Basically, I'm trying to create an account generator for a site. I want to be able to generate multiple accounts at once. I feel the best way to do this is with a csv file.
I've tried using encoding/csv to read the csv file then marshal to JSON. Also, ioutil.ReadFile(). However, the response from the site is, 'first name is mandatory field, last name is a mandatory field' etc etc. So, this obviously means the csv data is not going into JSON format. I'll show my code with the ioutil.ReadFile() below.
func main() {
file, _ := ioutil.ReadFile("accounts.csv")
jsonConv, _ := json.Marshal(file)
client := http.Client{}
req, err := http.NewRequest("POST", "websiteUrlHere", bytes.NewBuffer(jsonConv))
req.Header.Add("cookie", `"really long cookie goes here"`)
req.Header.Set("user-agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36")
req.Header.Set("content-type", "application/json")
resp, err := client.Do(req)
if err != nil {
fmt.Print(err)
}
defer resp.Body.Close()
}
(^ This is just a snippet of the code).
I'm still pretty new to all of this so please understand if the question lacks anything. I've also looked for similar questions but all I find is the use of a struct. I feel this wouldn't be applicable for this as the goal is to create unlimited accounts at once.
Hope the above is sufficient. Thank you.
The issue with your code is actually that you're trying to convert a file to bytes with:
file, _ := ioutil.ReadFile("accounts.csv")
...and then you are AGAIN trying to convert that slice of bytes to JSON bytes with:
jsonConv, _ := json.Marshal(file)
Where the text contents of the file are stored as a slice of bytes in the variable file, and then that slice of bytes (the file contents in bytes form) is then being converted to a JSON array of bytes. So you are basically sending a JSON array of numbers...not good.
The normal flow here would be to take the file bytes and then create a Go struct(s) out of it. Once your Go objects are in place, THEN you would marshal to JSON. That converts the Go objects to a slice of bytes AFTER it has been converted to JSON text form.
So what you are missing is the Go structure middle step but you also should keep in mind that converting a Go struct to JSON bytes with json.Marshal() will only show fields that are exported. Also, usually you should use struct tags to customize exactly how the fields will show up.
Your best bet is just to stick with JSON, forget about the CSV. In your own code example, you are taking a CSV and then trying to convert it to JSON...so, then, just use JSON.
If you want to send multiple accounts, just make your Go structure a slice, which will marshal into a JSON array, which is basically what you are trying to do. The end result will be a JSON array of accounts. Here's a simple example:
package main
import (
"fmt"
"encoding/json"
)
type Account struct {
Username string `json:"username"`
Email string `json:"email"`
}
type AccountsRequest struct {
Accounts []Account `json:"accounts"`
}
func main() {
//gather account info
acct1 := Account{Username: "tom", Email: "tom#example.com"}
acct2 := Account{Username: "dick", Email: "dick#example.com"}
acct3 := Account{Username: "harry", Email: "harry#example.com"}
//create request
acctsReq := AccountsRequest{Accounts: []Account{acct1, acct2, acct3}}
//convert to JSON bytes/data
//jsonData, _ := json.Marshal(acctsReq)
//debug/output
jsonDataPretty, _ := json.MarshalIndent(acctsReq, "", " ")
fmt.Println(string(jsonDataPretty))
//send request with data
//...
}
Runnable here in playground.
The key is that the structs are set up and ready to go and the struct tags determine what the JSON field names will be (i.e. username & email for each account and accounts for the overall array of accounts).
Hope that helps. Drop a comment if you need more specific help.
You need to parse the CSV file first and convert it into the list that you want:
package main
func main() {
file, err := os.Open("file.csv")
if err != nil {
log.Fatal("failed opening file because: %s", err.Error())
}
r := csv.NewReader(file)
records, err := r.ReadAll()
if err != nil {
log.Fatal(err)
}
fmt.Print(records)
}
The above code is parsing the list into a [][]string array. you will now need to iterate over that array and turn it into the json object that the page needs. Then you can send it. You can read more about the csv package here : https://golang.org/pkg/encoding/csv/
A word of advise: never ignore errors, they might give you useful information.
My objective is to add fields to json on user request.
Everything is great, but when displaying the fields with
fmt.Printf("%s: %s\n", content.Date, content.Description)
an error occurs:
invalid character '{' after top-level value
And that is because after adding new fields the file looks like this:
{"Date":"2017-03-20 10:46:48","Description":"new"}
{"Date":"2017-03-20 10:46:51","Description":"new .go"}
The biggest problem is with the writting to file
reminder := &Name{dateString[:19], text} //text - input string
newReminder, _ := json.Marshal(&reminder)
I dont really know how to do this properly
My question is how should I wrap all member fields into one object?
And what is the best way to iterate through member fields?
The code is available here: https://play.golang.org/p/NunV_B6sud
You should store the reminders into an array inside the json file, as mentioned by #Gerben Jacobs, and then, every time you want to add a new reminder to the array you need to read the full contents of rem.json, append the new reminder in Go, truncate the file, and write the new slice into the file. Here's a quick implentation https://play.golang.org/p/UKR91maQF2.
If you have lots of reminders and the process of reading, decoding, encoding, and writing the whole content becomes a pain you could open the file, implement a way to truncate only the last ] from the file contents, and then write only , + new reminder + ].
So after some research, people in the go-nuts group helped me and suggested me to use a streaming json parser that parses items individually.
So I needed to change my reminder listing function:
func listReminders() error {
f, err := os.Open("rem.json")
if err != nil {
return err
}
dec := json.NewDecoder(f)
for {
var content Name
switch dec.Decode(&content) {
case nil:
fmt.Printf("%#v\n", content)
case io.EOF:
return nil
default:
return err
}
}
}
Now everything works the way I wanted.
I am trying to do the following:
render [assignedSchol:assignedSchol,scholType:scholType] as JSON
assignedSchol is an object, and scholType is just a value. I get a "No map entry allowed at this place error". any help?
When you use one liner like you did you must put the conversion in braces like this:
render ( [assignedSchol:assignedSchol,scholType:scholType] as JSON )
But I think the above code returns json data as plain text format so I usually prefer doing it this way:
render(contentType: "text/json") {
[assignedSchol:assignedSchol,scholType:scholType]
}
Which gives me json data with response type json.
This should work:
render(contentType: "application/json") {[assignedSchol:assignedSchol,scholType:scholType]}
I have a io.Reader which I get from http.Request.Body that reads a JSON byte slice from a server.
I would like to stream this to json.NewDecoder. However I would also like to intercept the JSON before it hits json.NewDecoder and substitute certain parts of it. For example, the JSON string contains empty hashes "{}" which I would like to remove due to a bug in the server's JSON output.
I am currently achieving my goal using json.Unmarshal but not using the JSON streaming parser:
data, _ := ioutil.ReadAll(r.Body)
data = bytes.Replace(data, []byte("{}"), "", -1)
json.Unmarshal(data, [my struct])
How can I achieve the same thing as above but using json.NewDecoder so I can save the many times the above code has to parse through r.Body's data? Here's some code using a pseudo function ReplaceStream(r io.Reader, old, new []byte):
reader := ReplaceStream(r.Body, []byte("{}"), "")
dec := json.NewDecoder(reader)
dec.Decode([my struct])
I know ReplaceStream might be fairly trivial to make, but is there anything in the standard library to do this that I am unaware of?
My advice is to just treat that kind of message as a special case and avoid the extra parsing / substituting for all the other requests
data, _ := ioutil.ReadAll(r.Body)
// FIXME: overcome bug #12312 of json server
if data == `{"list": [{}]}` {
return []
}
// Normal datastruct ..
I apologise in advance for a very long question. Hopefully you'll bear with me.
I'm working with the goweb library, and experimenting with the example web app.
I've been trying to modify the RESTful example code, which defines a Thing as:
type Thing struct {
Id string
Text string
}
A Thing is created by sending an HTTP Post request, with an appropriate JSON body, to http://localhost:9090/things. This is handled in the example code in the Create function, specifically the lines:
dataMap := data.(map[string]interface{})
thing := new(Thing)
thing.Id = dataMap["Id"].(string)
thing.Text = dataMap["Text"].(string)
This is all well and good, and I can run the example server (which listens on http://localhost:9090/) and the server operates as expected.
For example:
curl -X POST -H "Content-Type: application/json" -d '{"Id":"TestId","Text":"TestText"}' http://localhost:9090/things
returns with no error, and then I GET that Thing with
curl http://localhost:9090/things/TestId
and it returns
{"d":{"Id":"TestId","Text":"TestText"},"s":200}
So far, so good.
Now, I would like to modify the Thing type, and add a custom ThingText type, like so:
type ThingText struct {
Title string
Body string
}
type Thing struct {
Id string
Text ThingText
}
This in itself isn't an issue, and I can modify the Create function like so:
thing := new(Thing)
thing.Id = dataMap["Id"].(string)
thing.Text.Title = dataMap["Title"].(string)
thing.Text.Body = dataMap["Body"].(string)
and the run the previous curl POST request with the JSON set to:
{"Id":"TestId","Title":"TestTitle","Title":"TestBody"}
and it returns with no error.
Once again I can GET the Thing URL and it returns:
{"d":{"Id":"TestId","Text":{"Title":"TestTitle","Body":"TestBody"}},"s":200}
Again, so far, so good.
Now, my question:
how do I modify the Create function to allow me to POST complex JSON to it?
For example, that last returned JSON string above includes {"Id":"TestId","Text":{"Title":"TestTitle","Body":"TestBody"}}. I'd like to be able to POST that exact JSON to the endpoint and have the Thing created.
I've followed the code back, and it seems that the data variable is of type Context.RequestData() from https://github.com/stretchr/goweb/context, and the internal Map seems to be of type Object.Map from https://github.com/stretchr/stew/, described as "a map[string]interface{} with additional helpful functionality." in particular, I noticed "Supports dot syntax to set deep values."
I can't work out how I can set up the thing.Text.Title = dataMap... statement so that the correct JSON field is parsed into it. I can't seem to use anything other than string types in the dataMap, and if I try that JSON it gives an error similar to:
http: panic serving 127.0.0.1:59113: interface conversion: interface is nil, not string
Once again, sorry for the ridiculously long question. I really appreciate you reading, and any help you may have to offer. Thanks!
As the JSON package documentation and JSON and Go introduction describe, JSON data can be parsed either generically through interface{} / string maps or unmarshalling directly to the struct types.
The example code you linked to and you based your changes on seems to use the generic string-map approach, dataMap := data.(map[string]interface{}).
As your desired JSON data is an object in an object, it’s simply a map within a map.
So you should be able to
dataMap := data.(map[string]interface{})
subthingMap := dataMap["Text"].(map[string]interface{})
thing.Text.Title = subthingMap["Title"].(string)
thing.Text.Body = subthingMap["Body"].(string)
I’m not sure why that code uses casts and generic types over type-safe unmarshalling directly from JSON to struct types (abstraction I guess). Using the json packages unmarshalling to struct types would go something like
type ThingText struct {
Title string
Body string
}
type Thing struct {
Id string
Text ThingText
}
…
decoder := json.NewDecoder(body)
var thingobj Thing
for {
if err := decoder.Decode(&thingobj); err == io.EOF {
break
} else if err != nil {
log.Fatal(err)
}
fmt.Println(thingobj)
}
where body is a io.Reader - in simple/most cases from http.Response.Body.