I wanna crawl a site (bbs.archusers.ir) for making a new post or topic notification for users.
I try to write below code for a reading name, last change date, and URL every section and save into struct and save struct into JSON object. but link part in my struct won't get any value and either appear in the console for me, this is my code
package main
import (
"fmt"
"github.com/anaskhan96/soup"
"log"
"os"
"encoding/json"
)
type Message struct {
Name string
Date string
link string
}
type myJSON struct {
Array []Message
}
var input []Message
func main() {
resp, err := soup.Get("https://bbs.archusers.ir")
if err != nil {
os.Exit(1)
}
doc := soup.HTMLParse(resp)
links := doc.Find("div", "id", "brdmain").FindAll("a")
for i := 0; i < len(links); i += 2 {
link := Message{links[i].Text(), links[i+1].Text(), links[i].Attrs()["href"] }
input = append(input, link)
}
if err != nil {
log.Fatal(err)
}
jsondat := &myJSON{input}
encjson, _ := json.Marshal(jsondat)
fmt.Println(string(encjson))
}
which part I did wrong?
There is a problem in your Message struct
it must name 'link' with capital k ---< Link
type Message struct {
Name string
Date string
Link string
}
and output will be correct
{"Name":"مانجارو Manjaro","Date":"2018-10-18 10:55:09","Link":"viewforum.php?id=22"}
Related
Problem
I'm new to Go and I'm trying to store json data in a struct from the Gov.uk public holidays API, so I can use this later on in my frontend.
If I run
var sb = string(body)
fmt.Println(sb)
I can see the data that's being returned in my terminal. I know that the response body is made up of bytes and the above converts it to a string.
I would like to iterate through the response body and store the data in a slice of structs called holidays, each struct will contain the data for a single public holiday. For some reason, the holidays variable returns an empty slice: [].
I guess my two questions are:
What's the best way to transform json data into a slice of structs to be used later on?
Why does the holidays variable return an empty slice?
Thanks!
Here's my code below:
package main
import (
"encoding/json"
"fmt"
"io/ioutil"
"log"
"net/http"
)
type Response struct {
Data []Data
}
type Data struct {
Division string
Bankholiday Bankholiday
}
type Bankholiday struct {
Title string
Date string
}
func main() {
resp, err := http.Get("https://www.gov.uk/bank-holidays.json")
if err != nil {
log.Fatal(err)
}
if resp.Body != nil {
defer resp.Body.Close()
}
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
log.Fatalln(err)
}
var response Response
json.Unmarshal(body, &response)
var holidays = []Bankholiday{}
for _, date := range response.Data {
holidays = append(holidays, Bankholiday{
Title: date.Bankholiday.Title,
Date: date.Bankholiday.Date,
})
}
fmt.Println("holidays: ", holidays)
}
I had to adjust the Response struct to handle correct unmarshaling of data. Find below the working code:
package main
import (
"encoding/json"
"fmt"
"io"
"log"
"net/http"
)
type Response map[string]Data
type Data struct {
Division string `json:"division"`
Events []struct {
Title string `json:"title"`
Date string `json:"date"`
Notes string `json:"notes"`
Bunting bool `json:"bunting"`
} `json:"events"`
}
func main() {
resp, err := http.Get("https://www.gov.uk/bank-holidays.json")
if err != nil {
log.Fatal(err)
}
if resp.Body != nil {
defer resp.Body.Close()
}
body, err := io.ReadAll(resp.Body)
if err != nil {
log.Fatalln(err)
}
var response Response
if err = json.Unmarshal(body, &response); err != nil {
log.Fatalln(err)
}
for div, _ := range response {
for _, event := range response[div].Events {
fmt.Printf("Division=%s, Holiday=%s, Date=%s\n", div, event.Title, event.Date)
}
}
}
Because your json fields must match your structs.
type Response map[string]Data
type Data struct {
Division string `json:"division"`
Events []Event `json:"events"`
}
type Event struct {
Title string `json:"title"`
Date string `json:"date"`
Notes string `json:"notes"`
Bunting bool `json:"bunting"`
}
I'm trying to read a json file with golang but i'm getting this error.
I've checked almost every question about it but still couldnt get it.
Here's the example json file:
https://jsonplaceholder.typicode.com/posts
And my code:
package main
import (
"net/http"
"log"
"fmt"
"io/ioutil"
"encoding/json"
)
type Posts struct {
Post []struct{
UserId int `json:"userId"`
ID int `json:"id"`
Title string `json:"title"`
Body string `json:"body"`
}
}
func main (){
resp, err := http.Get("https://jsonplaceholder.typicode.com/posts")
if err != nil {
log.Fatal(err)
}
content, _ := ioutil.ReadAll(resp.Body)
var posts Posts
parsed := json.Unmarshal([]byte(content), &posts)
//fmt.Println(string(content))
fmt.Println(parsed)
}
Posts is an array of Post struct but you defined Post as array it is your first mistake, also Unmarshal doesn't returns result it returns only error and fills given parameter.
package main
import (
"net/http"
"log"
"fmt"
"io/ioutil"
"encoding/json"
)
type Post struct {
UserId int `json:"userId"`
ID int `json:"id"`
Title string `json:"title"`
Body string `json:"body"`
}
type Posts []Post
func main (){
resp, err := http.Get("https://jsonplaceholder.typicode.com/posts")
if err != nil {
log.Fatal(err)
}
content, _ := ioutil.ReadAll(resp.Body)
var posts Posts
err = json.Unmarshal(content, &posts)
if err != nil {
log.Fatal(err)
}
fmt.Println(posts[0].Body)
}
That JSON is, at its root, an array. You're trying to unmarshal it into an object, which contains, as a field, an array - hence the error that you passed an object when the JSON is an array. You want to pass an array (or slice, really), as in:
type Post struct {
UserId int `json:"userId"`
ID int `json:"id"`
Title string `json:"title"`
Body string `json:"body"`
}
//...
var posts []Post
err := json.Unmarshal([]byte(content), &posts)
// Check err, do stuff with posts
I'm trying to do an API request to get some information from steams public API (this is mainly for learning Go and just learning how to deal with Json / API requests) I have gotten this code so far:
package main
import (
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
"strconv"
)
type SteamAPI struct {
APIKey string
}
type GetAppNews struct {
AppNews struct {
AppId int `json:"appid"`
NewsItems []struct {
Gid int `json:"gid"`
Title string `json:"title"`
Url string `json:"url"`
IsExternalUrl bool `json:"is_external_url"`
Author string `json:"author"`
Contents string `json:"contents"`
Feedlabel string `json:"feedlabel"`
Date int `json:"date"`
} `json:"newsitems"`
} `json:"appnews"`
}
type JsonResponse map[string]GetAppNews
func (s SteamAPI) GetNewsForApp(appid, count, maxlength int) error {
sAppid := strconv.Itoa(appid)
sCount := strconv.Itoa(count)
sMaxlength := strconv.Itoa(maxlength)
resp, err := http.Get("http://api.steampowered.com/ISteamNews/GetNewsForApp/v0002/?appid=" + sAppid + "&count=" + sCount + "&maxlength=" + sMaxlength + "&format=json")
if err != nil {
return err
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
return err
}
var jsonReturn JsonResponse
json.Unmarshal(body, &jsonReturn)
fmt.Println(jsonReturn)
return nil
}
func main() {
Tester := SteamAPI{""}
Tester.GetNewsForApp(440, 3, 300)
}
Things seem to work, alright, I guess but its not formatting it the way I would expect it to Unmarshal. It prints out like this:
map[appnews:{{0 []}}]
You can click here to see exactly what the format of the JSON response looks like, if anybody could tell me what I have done wrong with my struct, in the end I expect to be able to go something like:
fmt.Println(blah["appnews"]["appid"]) and it would return 440.
Thats all I really got to go off of, if you need anymore specific information let me know! Thanks for the help!
The data fits the struct just fine, no need for map[string]GetAppNews.
type JsonResponse map[string]GetAppNews should just be GetAppNews.
playground
For example:
{"id":
{"12345678901234":
{"Account":"asdf",
"Password":"qwerty"
"LastSeen":"1397621470",
}
}
}
A program I've been trying to make needs to get the id as a string and then later use it to check the time in LastSeen.
I've tried using simplejson and jsonq,but still cant figure out how to do that.
You can use RawMessage and make it much simpiler (play with it) :
package main
import (
"encoding/json"
"fmt"
)
var data []byte = []byte(`{"id": {"12345678901234": {"Account":"asdf", "Password":"qwerty", "LastSeen":"1397621470"}}}`)
type Message struct {
Id string
Info struct {
Account string
Password string
LastSeen string
}
}
func main() {
var (
tmpmsg struct {
Data map[string]json.RawMessage `json:"id"`
}
msg Message
)
if err := json.Unmarshal(data, &tmpmsg); err != nil {
panic(err) //you probably wanna use or something instead
}
for id, raw := range tmpmsg.Data {
msg.Id = id
if err := json.Unmarshal(raw, &msg.Info); err != nil {
panic(err)
}
}
fmt.Printf("%+v\n", msg)
}
Looking at the Golang blog post on JSON here it can be done using the encoding/json package. I created a small program to do this as follows:
package main
import (
"encoding/json"
"fmt"
)
var data []byte = []byte(`{"id": {"12345678901234": {"Account":"asdf", "Password":"qwerty", "LastSeen":"1397621470"}}}`)
type Message struct {
id string
LastSeen int64
}
var m Message
func main() {
var i interface {}
err := json.Unmarshal(data, &i)
if err != nil {
println("Error decoding data")
fmt.Printf("%s", err.Error())
return
}
m := i.(map[string]interface{})
for k, v := range m {
println(k)
im := v.(map[string]interface{})
for ik, iv := range im {
println("\t", ik)
jm := iv.(map[string]interface{})
for jk, jv := range jm {
println("\t\t", jk, ": ", jv.(string))
}
}
}
}
I apologise if this is poor in terms of Go best practices and such, I am new to the language. And I know that some elements of this aren't entirely necessary like the Message type definition but this works, at least on your data.
I am trying to convert a Go struct to JSON using the json package but all I get is {}. I am certain it is something totally obvious but I don't see it.
package main
import (
"fmt"
"encoding/json"
)
type User struct {
name string
}
func main() {
user := &User{name:"Frank"}
b, err := json.Marshal(user)
if err != nil {
fmt.Printf("Error: %s", err)
return;
}
fmt.Println(string(b))
}
Then when I try to run it I get this:
$ 6g test.go && 6l -o test test.6 && ./test
{}
You need to export the User.name field so that the json package can see it. Rename the name field to Name.
package main
import (
"fmt"
"encoding/json"
)
type User struct {
Name string
}
func main() {
user := &User{Name: "Frank"}
b, err := json.Marshal(user)
if err != nil {
fmt.Println(err)
return
}
fmt.Println(string(b))
}
Output:
{"Name":"Frank"}
Related issue:
I was having trouble converting struct to JSON, sending it as response from Golang, then, later catch the same in JavaScript via Ajax.
Wasted a lot of time, so posting solution here.
In Go:
// web server
type Foo struct {
Number int `json:"number"`
Title string `json:"title"`
}
foo_marshalled, err := json.Marshal(Foo{Number: 1, Title: "test"})
fmt.Fprint(w, string(foo_marshalled)) // write response to ResponseWriter (w)
In JavaScript:
// web call & receive in "data", thru Ajax/ other
var Foo = JSON.parse(data);
console.log("number: " + Foo.number);
console.log("title: " + Foo.title);
This is an interesting question, it is very easy using the new go versions. You should do this:
package main
import (
"fmt"
"encoding/json"
)
type User struct {
Name string `json:"name"`
}
func main() {
user := &User{name:"Frank"}
b, err := json.Marshal(user)
if err != nil {
fmt.Printf("Error: %s", err)
return;
}
fmt.Println(string(b))
}
Change this name to Name.
You can define your own custom MarshalJSON and UnmarshalJSON methods and intentionally control what should be included, ex:
package main
import (
"fmt"
"encoding/json"
)
type User struct {
name string
}
func (u *User) MarshalJSON() ([]byte, error) {
return json.Marshal(&struct {
Name string `json:"name"`
}{
Name: "customized" + u.name,
})
}
func main() {
user := &User{name: "Frank"}
b, err := json.Marshal(user)
if err != nil {
fmt.Println(err)
return
}
fmt.Println(string(b))
}
Struct values encode as JSON objects. Each exported struct field becomes
a member of the object unless:
the field's tag is "-", or
the field is empty and its tag specifies the "omitempty" option.
The empty values are false, 0, any nil pointer or interface value, and any array, slice, map, or string of length zero. The object's default key string is the struct field name but can be specified in the struct field's tag value. The "json" key in the struct field's tag value is the key name, followed by an optional comma and options.