Submit variable in payload of golang http.NewRequest - json

I'm learning Golang so please excuse what may seem to be a basic question. I have searched for a couple of hours for clues as to how I might achieve sending variable data in my JSON formatted API POST from my golang app, but not found a clue or solution yet. I know the answer will be my lack of syntax knowledge.
So the problem is with the 'lastcontact' field I'm trying to POST. I want to use my 'dt' variable that contains the current datetime.
package main
import (
"fmt"
"strings"
"net/http"
"io/ioutil"
"time"
)
func main() {
dt := time.Now()
url := "https://fakeapi.io/API/apiActions/update/"
method := "POST"
payload := strings.NewReader(`{
"name" : "Dumpty",
"saveconfig" : "true",
"lastcontact" : {dt}
}`)
client := &http.Client {
}
req, err := http.NewRequest(method, url, payload)
if err != nil {
fmt.Println(err)
return
}
req.Header.Add("api-key", "gjhgjhgjhg")
req.Header.Add("api-secret", "jhgjhgjhg")
req.Header.Add("Content-Type", "application/json; charset=UTF-8")
res, err := client.Do(req)
if err != nil {
fmt.Println(err)
return
}
defer res.Body.Close()
body, err := ioutil.ReadAll(res.Body)
if err != nil {
fmt.Println(err)
return
}
fmt.Println(string(body))
}

You can use a struct to store your response type if you know the fields beforehand. If you don't, you could use a map[string]interface{} to store arbitrary data.
Then using json.Marshal to convert it to a correctly formatted JSON response.
type myStruct struct {
Name string `json:"name"`
SaveConfig string `json:"saveconfig"`
LastContact time.Time `json:"lastcontact"`
}
dt := time.Now()
myData := myStruct{
Name: "Dumpty",
SaveConfig: "true",
LastContact: dt,
}
myBytes, err := json.Marshal(myData)
// bytes.NewBuffer returns *bytes.Buffer
// which implements the io.Reader interface
// that you need for your http.NewRequest call
payload := bytes.NewBuffer(myBytes)
Full playground example https://play.golang.org/p/Mc9UXT32Wi1

Related

Read extern JSON file

I am trying to read the following JSON file:
{
"a":1,
"b":2,
"c":3
}
I have tried this but I found that I had to write each field of the JSON file into a struct but I really don't want to have all my JSON file in my Go code.
import (
"fmt"
"encoding/json"
"io/ioutil"
)
type Data struct {
A string `json:"a"`
B string `json:"b"`
C string `json:"c"`
}
func main() {
file, _ := ioutil.ReadFile("/path/to/file.json")
data := Data{}
if err := json.Unmarshal(file ,&data); err != nil {
panic(err)
}
for _, letter := range data.Letter {
fmt.Println(letter)
}
}
Is there a way to bypass this thing with something like json.load(file) in Python?
If you only want to support integer values, you could unmarshal your data into a map[string]int. Note that the order of a map is not defined, so the below program's output is non-deterministic for the input.
package main
import (
"fmt"
"encoding/json"
"io/ioutil"
)
func main() {
file, _ := ioutil.ReadFile("/path/to/file.json")
var data map[string]int
if err := json.Unmarshal(file ,&data); err != nil {
panic(err)
}
for letter := range data {
fmt.Println(letter)
}
}
You can unmarshal any JSON data in this way:
var data interface{}
if err := json.Unmarshal(..., &data); err != nil {
// handle error
}
Though, in this way you should handle all the reflection-related stuffs
since you don't know what type the root data is, and its fields.
Even worse, your data might not be map at all.
It can be any valid JSON data type like array, string, integer, etc.
Here's a playground link: https://play.golang.org/p/DiceOv4sATO
It's impossible to do anything as simple as in Python, because Go is strictly typed, so it's necessary to pass your target into the unmarshal function.
What you've written could otherwise be shortened, slightly, to something like this:
func UnmarshalJSONFile(path string, i interface{}) error {
f, err := os.Open(path)
if err != nil {
return err
}
defer f.Close()
return json.NewDecoder(f).Decode(i)
}
But then to use it, you would do this:
func main() {
data := Data{}
if err := UnmarshalJSONFile("/path/to/file.json", &data); err != nil {
panic(err)
}
}
But you can see that the UnmarshalJSONFile is so simple, it hardly warrants a standard library function.

json unmarshal not working but decode does

I have a hard time understanding why the code below, which uses the unmarshal method does not work, but then almost the same I write with NewDecoder and it works fine.
package conf
import (
"os"
"io/ioutil"
"encoding/json"
)
type Configuration struct {
Agents []Agent `json:"agents"`
IbmWmqFolder string `json:"ibmWmqFolder"`
}
type Agent struct {
AgentName string `json:"agentName"`
Folders []string `json:"folders"`
}
func LoadConfiguration() (configuration Configuration) {
jsonFile, err := os.Open("config.json")
if err != nil {
panic(err)
}
defer jsonFile.Close()
byteValue, _ := ioutil.ReadAll(jsonFile)
json.Unmarshal(byteValue, configuration)
return
}
but if I do all the same but instead of the two last lines with the byteValue and the unmarshal itself, but use the decoder, it works,
jsonParser := json.NewDecoder(jsonFile)
jsonParser.Decode(&configuration)
return
Thanks!
I would guess that you need to pass a pointer to the configuration, like so:
json.Unmarshal(byteValue, &configuration)
You should also check the error value returned by Unmarshal, e.g.:
err = json.Unmarshal(byteValue, &configuration)
if err != nil {
panic(err)
}
See the the docs.

How do I send a JSON string in a POST request in Go

I tried working with Apiary and made a universal template to send JSON to mock server and have this code:
package main
import (
"encoding/json"
"fmt"
"github.com/jmcvetta/napping"
"log"
"net/http"
)
func main() {
url := "http://restapi3.apiary.io/notes"
fmt.Println("URL:>", url)
s := napping.Session{}
h := &http.Header{}
h.Set("X-Custom-Header", "myvalue")
s.Header = h
var jsonStr = []byte(`
{
"title": "Buy cheese and bread for breakfast."
}`)
var data map[string]json.RawMessage
err := json.Unmarshal(jsonStr, &data)
if err != nil {
fmt.Println(err)
}
resp, err := s.Post(url, &data, nil, nil)
if err != nil {
log.Fatal(err)
}
fmt.Println("response Status:", resp.Status())
fmt.Println("response Headers:", resp.HttpResponse().Header)
fmt.Println("response Body:", resp.RawText())
}
This code doesn't send JSON properly, but I don't know why. The JSON string can be different in every call. I can't use Struct for this.
I'm not familiar with napping, but using Golang's net/http package works fine (playground):
func main() {
url := "http://restapi3.apiary.io/notes"
fmt.Println("URL:>", url)
var jsonStr = []byte(`{"title":"Buy cheese and bread for breakfast."}`)
req, err := http.NewRequest("POST", url, bytes.NewBuffer(jsonStr))
req.Header.Set("X-Custom-Header", "myvalue")
req.Header.Set("Content-Type", "application/json")
client := &http.Client{}
resp, err := client.Do(req)
if err != nil {
panic(err)
}
defer resp.Body.Close()
fmt.Println("response Status:", resp.Status)
fmt.Println("response Headers:", resp.Header)
body, _ := ioutil.ReadAll(resp.Body)
fmt.Println("response Body:", string(body))
}
you can just use post to post your json.
values := map[string]string{"username": username, "password": password}
jsonValue, _ := json.Marshal(values)
resp, err := http.Post(authAuthenticatorUrl, "application/json", bytes.NewBuffer(jsonValue))
If you already have a struct.
import (
"bytes"
"encoding/json"
"io"
"net/http"
"os"
)
// .....
type Student struct {
Name string `json:"name"`
Address string `json:"address"`
}
// .....
body := &Student{
Name: "abc",
Address: "xyz",
}
payloadBuf := new(bytes.Buffer)
json.NewEncoder(payloadBuf).Encode(body)
req, _ := http.NewRequest("POST", url, payloadBuf)
client := &http.Client{}
res, e := client.Do(req)
if e != nil {
return e
}
defer res.Body.Close()
fmt.Println("response Status:", res.Status)
// Print the body to the stdout
io.Copy(os.Stdout, res.Body)
Full gist.
In addition to standard net/http package, you can consider using my GoRequest which wraps around net/http and make your life easier without thinking too much about json or struct. But you can also mix and match both of them in one request! (you can see more details about it in gorequest github page)
So, in the end your code will become like follow:
func main() {
url := "http://restapi3.apiary.io/notes"
fmt.Println("URL:>", url)
request := gorequest.New()
titleList := []string{"title1", "title2", "title3"}
for _, title := range titleList {
resp, body, errs := request.Post(url).
Set("X-Custom-Header", "myvalue").
Send(`{"title":"` + title + `"}`).
End()
if errs != nil {
fmt.Println(errs)
os.Exit(1)
}
fmt.Println("response Status:", resp.Status)
fmt.Println("response Headers:", resp.Header)
fmt.Println("response Body:", body)
}
}
This depends on how you want to achieve. I made this library because I have the same problem with you and I want code that is shorter, easy to use with json, and more maintainable in my codebase and production system.
Example post request for http or https
//Encode the data
postBody, _ := json.Marshal(map[string]string{
"name": "Test",
"email": "Test#Test.com",
})
responseBody := bytes.NewBuffer(postBody)
//Leverage Go's HTTP Post function to make request
resp, err := http.Post("https://postman-echo.com/post", "application/json", responseBody)
//Handle Error
if err != nil {
log.Fatalf("An Error Occured %v", err)
}
defer resp.Body.Close()
//Read the response body
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
log.Fatalln(err)
}
sb := string(body)
log.Printf(sb)
Use io.Pipe for large request bodies as mentioned in another answer. This approach avoids building the entire request body in memory by streaming the data from the JSON encoder to the network.
This answer builds on the other answer by showing how to handle errors. Always handle errors!
Use the pipe's CloseWithError function to propagate encoding errors back to error returned from http.Post.
Handle the error returned from http.Post
Close the response body.
Here's the code:
r, w := io.Pipe()
go func() {
w.CloseWithError(json.NewEncoder(w).Encode(data))
}()
// Ensure that read side of pipe is closed. This
// unblocks goroutine in scenario where http.Post
// errors out before reading the entire request body.
defer r.Close()
resp, err := http.Post(url, r)
if err != nil {
// Adjust error handling here to meet application requrirements.
log.Fatal(err)
}
defer resp.Body.Close()
// Use the response here.
If you have a lot of data to send, you can use a pipe:
package main
import (
"encoding/json"
"io"
"net/http"
)
func main() {
m := map[string]int{"SNG_ID": 75498415}
r, w := io.Pipe()
go func() {
json.NewEncoder(w).Encode(m)
w.Close()
}()
http.Post("https://stackoverflow.com", "application/json", r)
}
https://golang.org/pkg/io#Pipe
if you want to do it like that, you need to use this map for unmarshalling json string.
var data map[string]interface{}
but if you need to change the json each time and to make initialization of your requst body more convenient, you can use this map for creating json body.
var bodyJsonMap map[string]interface{}{
"key1": val1,
"key2": val2,
...
}
Then marshal it to a json-string.

Golang Converting JSON

map[key:2073933158088]
I need to grab the key out of this data structure as a string, but I can't seem to figure out how!
Help with this overly simple question very much appreciated.
The value above is encapsulated in the variable named data.
I have tried: data.key, data[key], data["key"], data[0] and none of these seem to be appropriate calls.
To define data I sent up a JSON packet to a queue on IronMQ. I then pulled the message from the queue and manipulated it like this:
payloadIndex := 0
for index, arg := range(os.Args) {
if arg == "-payload" {
payloadIndex = index + 1
}
}
if payloadIndex >= len(os.Args) {
panic("No payload value.")
}
payload := os.Args[payloadIndex]
var data interface{}
raw, err := ioutil.ReadFile(payload)
if err != nil {
panic(err.Error())
}
err = json.Unmarshal(raw, &data)
Design your data type to match json structure. This is how can you achieve this:
package main
import (
"fmt"
"encoding/json"
)
type Data struct {
Key string `json:"key"`
}
func main() {
data := new(Data)
text := `{ "key": "2073933158088" }`
raw := []byte(text)
err := json.Unmarshal(raw, data)
if err != nil {
panic(err.Error())
}
fmt.Println(data.Key)
}
Since the number in the json is unquoted, it's not a string, Go will panic if you try to just handle it as a string (playground: http://play.golang.org/p/i-NUwchJc1).
Here's a working alternative:
package main
import (
"fmt"
"encoding/json"
"strconv"
)
type Data struct {
Key string `json:"key"`
}
func (d *Data) UnmarshalJSON(content []byte) error {
var m map[string]interface{}
err := json.Unmarshal(content, &m)
if err != nil {
return err
}
d.Key = strconv.FormatFloat(m["key"].(float64), 'f', -1, 64)
return nil
}
func main() {
data := new(Data)
text := `{"key":2073933158088}`
raw := []byte(text)
err := json.Unmarshal(raw, data)
if err != nil {
panic(err.Error())
}
fmt.Println(data.Key)
}
You can see the result in the playground: http://play.golang.org/p/5hU3hdV3kK

How to get JSON object by calling a url in Go Language?

I'm starting to learn Golang and I would like to know how to get a json response by calling an url, if you could give me an example it would be great in order to guide myself.
Here's a simple example to get you started. Instead of a map[string]interface{} you should consider making a struct to hold the result of your request.
package main
import (
"encoding/json"
"fmt"
"log"
"net/http"
)
func main() {
resp, err := http.Get("http://api.geonames.org/citiesJSON?north=44.1&south=-9.9&east=-22.4&west=55.2&lang=de&username=demo")
if err != nil {
log.Fatal(err)
}
var generic map[string]interface{}
err = json.NewDecoder(resp.Body).Decode(&generic)
if err != nil {
log.Fatal(err)
}
fmt.Println(generic)
}
I'd write a little helper function to do it:
// getJSON fetches the contents of the given URL
// and decodes it as JSON into the given result,
// which should be a pointer to the expected data.
func getJSON(url string, result interface{}) error {
resp, err := http.Get(url)
if err != nil {
return fmt.Errorf("cannot fetch URL %q: %v", url, err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return fmt.Errorf("unexpected http GET status: %s", resp.Status)
}
// We could check the resulting content type
// here if desired.
err := json.NewDecoder(resp.Body).Decode(result)
if err != nil {
return fmt.Errorf("cannot decode JSON: %v", err)
}
return nil
}
A full working example can be found here: http://play.golang.org/p/b1WJb7MbQV
Note that it is important to check the status code as well as the Get error, and the response body must be closed explicitly (see the documentation here: http://golang.org/pkg/net/http/#Get)