Convert Go map to json - json

I tried to convert my Go map to a json string with encoding/json Marshal, but it resulted in a empty string.
Here's my code :
package main
import (
"encoding/json"
"fmt"
)
type Foo struct {
Number int `json:"number"`
Title string `json:"title"`
}
func main() {
datas := make(map[int]Foo)
for i := 0; i < 10; i++ {
datas[i] = Foo{Number: 1, Title: "test"}
}
jsonString, _ := json.Marshal(datas)
fmt.Println(datas)
fmt.Println(jsonString)
}
My output is :
map[9:{1 test} 2:{1 test} 7:{1 test} 3:{1 test} 4:{1 test} 5:{1 test} 6:{1 test} 8:{1 test} 0:{1 test} 1:{1 test}]
[]
I really don't know where I'm wrong. Thank you for your help.

If you had caught the error, you would have seen this:
jsonString, err := json.Marshal(datas)
fmt.Println(err)
// [] json: unsupported type: map[int]main.Foo
The thing is you cannot use integers as keys in JSON; it is forbidden. Instead, you can convert these values to strings beforehand, for instance using strconv.Itoa.
See this post for more details: https://stackoverflow.com/a/24284721/2679935

It actually tells you what's wrong, but you ignored it because you didn't check the error returned from json.Marshal.
json: unsupported type: map[int]main.Foo
JSON spec doesn't support anything except strings for object keys, while javascript won't be fussy about it, it's still illegal.
You have two options:
1 Use map[string]Foo and convert the index to string (using fmt.Sprint for example):
datas := make(map[string]Foo, N)
for i := 0; i < 10; i++ {
datas[fmt.Sprint(i)] = Foo{Number: 1, Title: "test"}
}
j, err := json.Marshal(datas)
fmt.Println(string(j), err)
2 Simply just use a slice (javascript array):
datas2 := make([]Foo, N)
for i := 0; i < 10; i++ {
datas2[i] = Foo{Number: 1, Title: "test"}
}
j, err = json.Marshal(datas2)
fmt.Println(string(j), err)
playground

This behaviour has been changed over time. I am using go v1.16.5 and I can happily pass int type in as JSON key. I have tried the same problem now and I can see the below result. Type conversion of a non-string key has been added by textMarshaler and textUnmarshaler interfaces. For more info, you can visit this https://golang.org/pkg/encoding/json/#Marshal
type Foo struct {
Number int `json:"number"`
Title string `json:"title"`
}
datas := make(map[int]Foo)
for i := 0; i < 5; i++ {
datas[i] = Foo{Number: 1, Title: "test"}
}
jsonString, _ := json.Marshal(datas)
fmt.Println("Datasets Result : ", datas)
fmt.Println("Marshal Datasets Result : ", string(jsonString), err)
m := make(map[int]Foo)
err = json.Unmarshal(jsonString, &m)
if err != nil {
panic(err)
}
fmt.Println("Unmarshal JSON Result : ", m)
Output:
Datasets Result map : [0:{1 test} 1:{1 test} 2:{1 test} 3:{1 test} 4:{1 test}]
Marshal Datasets Result : {"0":{"number":1,"title":"test"},"1":{"number":1,"title":"test"},"2":{"number":1,"title":"test"},"3":{"number":1,"title":"test"},"4":{"number":1,"title":"test"}} <nil>
Unmarshal JSON Result : map[0:{1 test} 1:{1 test} 2:{1 test} 3:{1 test} 4:{1 test}]

data := map[string]interface{}
jsonByte, err := json.Marshal(data)
if err != nil{
log.Fatal(err)
}
jsonString := string(jsonByte)
fmt.Println(jsonString) // will be printing data map in json formate.

Since this question was asked/last answered, support for non string key types for maps for json Marshal/UnMarshal has been added through the use of TextMarshaler and TextUnmarshaler interfaces here. You could just implement these interfaces for your key types and then json.Marshal would work as expected.
package main
import (
"encoding/json"
"fmt"
"strconv"
)
// Num wraps the int value so that we can implement the TextMarshaler and TextUnmarshaler
type Num int
func (n *Num) UnmarshalText(text []byte) error {
i, err := strconv.Atoi(string(text))
if err != nil {
return err
}
*n = Num(i)
return nil
}
func (n Num) MarshalText() (text []byte, err error) {
return []byte(strconv.Itoa(int(n))), nil
}
type Foo struct {
Number Num `json:"number"`
Title string `json:"title"`
}
func main() {
datas := make(map[Num]Foo)
for i := 0; i < 10; i++ {
datas[Num(i)] = Foo{Number: 1, Title: "test"}
}
jsonString, err := json.Marshal(datas)
if err != nil {
panic(err)
}
fmt.Println(datas)
fmt.Println(jsonString)
m := make(map[Num]Foo)
err = json.Unmarshal(jsonString, &m)
if err != nil {
panic(err)
}
fmt.Println(m)
}
Output:
map[1:{1 test} 2:{1 test} 4:{1 test} 7:{1 test} 8:{1 test} 9:{1 test} 0:{1 test} 3:{1 test} 5:{1 test} 6:{1 test}]
[123 34 48 34 58 123 34 110 117 109 98 101 114 34 58 34 49 34 44 34 116 105 116 108 101 34 58 34 116 101 115 116 34 125 44 34 49 34 58 123 34 110 117 109 98 101 114 34 58 34 49 34 44 34 116 105 116 108 101 34 58 34 116 101 115 116 34 125 44 34 50 34 58 123 34 110 117 109 98 101 114 34 58 34 49 34 44 34 116 105 116 108 101 34 58 34 116 101 115 116 34 125 44 34 51 34 58 123 34 110 117 109 98 101 114 34 58 34 49 34 44 34 116 105 116 108 101 34 58 34 116 101 115 116 34 125 44 34 52 34 58 123 34 110 117 109 98 101 114 34 58 34 49 34 44 34 116 105 116 108 101 34 58 34 116 101 115 116 34 125 44 34 53 34 58 123 34 110 117 109 98 101 114 34 58 34 49 34 44 34 116 105 116 108 101 34 58 34 116 101 115 116 34 125 44 34 54 34 58 123 34 110 117 109 98 101 114 34 58 34 49 34 44 34 116 105 116 108 101 34 58 34 116 101 115 116 34 125 44 34 55 34 58 123 34 110 117 109 98 101 114 34 58 34 49 34 44 34 116 105 116 108 101 34 58 34 116 101 115 116 34 125 44 34 56 34 58 123 34 110 117 109 98 101 114 34 58 34 49 34 44 34 116 105 116 108 101 34 58 34 116 101 115 116 34 125 44 34 57 34 58 123 34 110 117 109 98 101 114 34 58 34 49 34 44 34 116 105 116 108 101 34 58 34 116 101 115 116 34 125 125]
map[4:{1 test} 5:{1 test} 6:{1 test} 7:{1 test} 0:{1 test} 2:{1 test} 3:{1 test} 1:{1 test} 8:{1 test} 9:{1 test}]

Related

How to Unmarshal value into json struct of type map[string][int] [closed]

Closed. This question is not reproducible or was caused by typos. It is not currently accepting answers.
This question was caused by a typo or a problem that can no longer be reproduced. While similar questions may be on-topic here, this one was resolved in a way less likely to help future readers.
Closed 1 year ago.
Improve this question
After doing ioutil.ReadAll on a resp.body of some API ,I'm getting:-
[91 34 123 92 34 78 111 79 102 86 105 101 119 115 92 34 58 123 92 34 48 92 34 58 48 125 44 92 34 78 111 79 102 76 105 107 101 115 92 34 58 123 92 34 48 92 34 58 48 125 44 92 34 78 111 79 102 67 111 109 109 101 110 116 115 92 34 58 123 92 34 48 92 34 58 48 125 44 92 34 78 111 79 102 83 104 97 114 101 115 92 34 58 123 92 34 48 92 34 58 48 125 125 34 44 34 123 125 34 93 10]
Now I've to unmarshal this to a struct json of type =
type ActUser struct {
NoOfViews map[string]int `json:"NoOfViews,omitempty"`
NoOfLikes map[string]int `json:"NoOfLikes,omitempty"`
NoOfComments map[string]int `json:"NoOfComments,omitempty"`
NoOfShares map[string]int `json:"NoOfShares,omitempty"`
}
But when I do
var try []ActUser
err = json.Unmarshal(bodyBytes, &try)
I'm getting error := cannot unmarshal string into Go value of type model.ActUser
I tried converting,but nothing seems to work.
Your example JSON data [91 34 123 ... corresponds to ["{.
This indicates the JSON you are receiving is probably invalid -- it's an array of strings, not an array of objects. It looks like your object is probably getting quoted when it is marshalled.
It can be unmarshalled into []string, not []ActUser. This is almost certainly undesirable, and a mistake when the source data was encoded. The best approach would be to fix the bug which causes the JSON object to be quoted as a string.
Alternatively, if you must extract data from the buggy JSON, you could:
var strs []string
if err := json.Unmarshal(bodyBytes, &strs); err != nil {
log.Fatal(err)
}
if len(strs) == 0 {
log.Fatal("missing ActUser object")
}
var user ActUser
if err := json.Unmarshal([]byte(strs[0]), &user); err != nil {
log.Fatal(err)
}
Separately, I'd recommend using fmt.Printf("%s\n", bodyBytes) to display your raw JSON data for debugging (much easier than a list of ASCII codes).

haskell tls file to list of list

I have a tls file such as:
1224 926 1380 688 845 109 118 88 1275 1306 91 796 102 1361 27 995
1928 2097 138 1824 198 117 1532 2000 1478 539 1982 125 1856 139 475 1338
848 202 1116 791 1114 236 183 186 150 1016 1258 84 952 1202 988 866
946 155 210 980 896 875 925 613 209 746 147 170 577 942 475 850
1500 322 43 95 74 210 1817 1631 1762 128 181 716 171 1740 145 1123
3074 827 117 2509 161 206 2739 253 2884 248 3307 2760 2239 1676 1137 3055
183 85 143 197 243 72 291 279 99 189 30 101 211 209 77 198
175 149 259 372 140 250 168 142 146 284 273 74 162 112 78 29
169 578 97 589 473 317 123 102 445 217 144 398 510 464 247 109
3291 216 185 1214 167 495 1859 194 1030 3456 2021 1622 3511 222 3534 1580
2066 2418 2324 93 1073 82 102 538 1552 962 91 836 1628 2154 2144 1378
149 963 1242 849 726 1158 164 1134 658 161 1148 336 826 1303 811 178
3421 1404 2360 2643 3186 3352 1112 171 168 177 146 1945 319 185 2927 2289
543 462 111 459 107 353 2006 116 2528 56 2436 1539 1770 125 2697 2432
1356 208 5013 4231 193 169 3152 2543 4430 4070 4031 145 4433 4187 4394 1754
5278 113 4427 569 5167 175 192 3903 155 1051 4121 5140 2328 203 5653 3233
how can I read it in a list of list of int in haskell?
I have tried few options but I could not manage to do it. I am very new to haskell so please be patience.
First break your input into lines using lines:
let test = "1 2 3 4\n 5 6 7 \n 4 2 5"
let rows = lines test --literally "lines test"! Beautiful, eh?
Result:
["1 2 3 4"," 5 6 7 "," 4 2 5"] :: [[Char]]
Then, extract individual numbers as strings using words:
let nums_as_strings = map words rows
Result:
[["1","2","3","4"],["5","6","7"],["4","2","5"]] :: :: [[[Char]]]
The last thing to do is convert these strings to integers with read:
let numbers = map (map read) nums_as_strings :: [[Int]]
Result:
[[1,2,3,4],[5,6,7],[4,2,5]] :: [[Int]]
Or, squashed into one line:
let numbers = map (map read) (map words $ lines test) :: [[Int]]
Example with your data:
Prelude> let test = "1224 926 1380 688 845 109 118 88 1275 1306 91 796 102 1361 27 995\n1928 2097 138 1824 198 117 1532 2000 1478 539 1982 125 1856 139 475 1338"
Prelude> map (map read) (map words $ lines test) :: [[Int]]
[[1224,926,1380,688,845,109,118,88,1275,1306,91,796,102,1361,27,995],[1928,2097,138,1824,198,117,1532,2000,1478,539,1982,125,1856,139,475,1338]]
You may need to take care of empty lines, but that's really simple.
import System.IO
readListOfLists :: Handle -> IO [[Int]]
readListOfLists handle = do
contents <- hGetContents handle
let ls :: [String]
ls = lines contents
ws :: [[String]]
ws= map words ls
res :: [[Int]]
res = map (map read) ws
return res;
or you can write the same code in one line:
readListOfLists :: Handle -> IO [[Int]]
readListOfLists = fmap (map (map read . words) . lines) . hGetContents
To use it:
do
handle <- openFile fileName ReadMode
table <- readListOfLists handle
hClose handle
print table

Elasticsearch return raw JSON with Golang Beats

I'm working on a beat that interrogate our elastic search, the goal is to get elements in elastics to modify them to avoid duplication.
I've written a search and it works:
client,err :=elasticsearch.NewClient(elasticsearch.ClientSettings{URL:host,Index:indexSel,Username: username,Password: password,Timeout: 60 * time.Second,}, nil)
if err != nil {
log.Fatal(err)
}
params := map[string]string{
"q": "_id:"+maref,
}
_, resp, err := client.SearchURI(index, "", params)
if err != nil {
log.Fatal(err)
return
} else {
fmt.Println(string(resp))
}
And here is the function from the elastic api.go:
func (es *Connection) SearchURI(index string, docType string, params map[string]string) (int, *SearchResults, error) {
status, resp, err := es.apiCall("GET", index, docType, "_search", "", params, nil)
if err != nil {
return status, nil, err
}
result, err := readSearchResult(resp)
return status, result, err
}
func readSearchResult(obj []byte) (*SearchResults, error) {
var result SearchResults
if obj == nil {
return nil, nil
}
err := json.Unmarshal(obj, &result)
if err != nil {
return nil, err
}
return &result, err
}
With the following types:
type SearchResults struct {
Took int `json:"took"`
Shards json.RawMessage `json:"_shards"`
Hits Hits `json:"hits"`
Aggs map[string]json.RawMessage `json:"aggregations"`
}
type Hits struct {
Total int
Hits []json.RawMessage `json:"hits"`
}
For now the answer is a raw JSON:
&{11 [123 34 116 111 116 97 108 34 58 53 44 34 115 117 99 99 101 115 115 102 117 108 34 58 53 44 34 102 97 105 108 101 100 34 58 48 125] {1 [[123 34 95 105 110 100 101 120 34 58 34 99 111 112 105 108 98 101 97 116 45 50 48 49 54 46 49 48 46 50 53 34 44 34 95 116 121 112 101 34 58 34 73 110 99 105 100 101 110 116 34 44 34 95 105 100 34 58 34 65 86 102 54 55 69 103 109 74 66 45 119 85 116 103 82 99 90 113 97 34 44 34 95 115 99 111 114 101 34 58 49 46 48 44 34 95 115 111 117 114 99 101 34 58 123 34 64 116 105 109 101 115 116 97 109 112 34 58 34 50 48 49 54 45 49 48 45 50 53 84 48 56 58 49 54 58 53 55 46 53 53 57 90 34 44 34 97 103 101 110 116 95 105 100 34 58 49 53 44 34 98 101 97 116 34 58 123 34 104 111 115 116 110 97 109 101 34 58 34 53 55 99 53 99 49 57 49 101 48 100 57 34 44 34 110 97 109 101 34 58 34 53 55 99 53 99 49 57 49 101 48 100 57 34 125 44 34 100 101 115 99 114 105 112 116 105 111 110 34 58 34 101 108 97 115 116 105 99 34 44 34 102 105 110 97 108 99 108 97 115 115 34 58 34 34 44 34 105 100 34 58 34 73 45 48 48 53 56 50 54 34 44 34 111 114 103 95 105 100 34 58 49 55 44 34 111 114 103 95 110 97 109 101 34 58 34 83 104 97 109 34 44 34 112 114 105 111 114 105 116 121 34 58 52 44 34 114 101 102 101 114 101 110 99 101 34 58 34 73 45 48 48 53 56 50 54 34 44 34 115 116 97 114 116 95 100 97 116 101 34 58 34 50 48 49 54 45 49 48 45 50 53 84 49 48 58 49 54 58 53 54 46 56 56 49 55 55 52 49 43 48 50 58 48 48 34 44 34 115 116 97 116 117 115 34 58 34 97 115 115 105 103 110 101 100 34 44 34 116 116 111 34 58 48 44 34 116 116 111 95 49 48 48 95 116 114 105 103 103 101 114 101 100 34 58 48 44 34 116 116 114 34 58 48 44 34 116 116 114 95 49 48 48 95 116 114 105 103 103 101 114 101 100 34 58 48 44 34 116 121 112 101 34 58 34 73 110 99 105 100 101 110 116 34 125 125]]} map[]}
So my question is:
How can I convert the result to ascii?
I've read it's probably due to the json.RawMessage type that doesn't convert JSON but I can't find how to convert it.
Thanks by advance,
Plosxh.
EDIT : Solution:
Thx to icza, here's my solution I change my "fmt.Println(resp)" into:
for _, v := range resp.Hits.Hits {
fmt.Println(string(v))
}
It works like a charm!
Edit 2 :
I had to re-create a whole type en json.Unmarshal "resp.Hits.Hits" in order to use it like a regular json.
json.RawMessage is a byte slice:
type RawMessage []byte
You can convert a byte slice to a string with a simple type conversion:
var result SearchResults
// ...
for _, v := range result.Hits {
fmt.Println("Total:", v.Total, "Hits:", string(v.Hits))
}
Or:
fmt.Printf("Total: %d, Hits: %s\n", v.Total, v.Hits)
See this example:
var raw json.RawMessage
raw = json.RawMessage(`something raw`)
fmt.Println(raw)
fmt.Println(string(raw))
fmt.Printf("%s\n", raw)
Output (try it on the Go Playground):
[115 111 109 101 116 104 105 110 103 32 114 97 119]
something raw
something raw

AWS Cognito Decode JWKS.json

I am working on verifying the access token on AWS Cognito. One thing I have to do is to convert the key from jwks.json (of the userpool) into a rsa Public Key structure (depending on the used JWT API).
So the values for 'n' and 'e' must be big int (long) and int.
What would be the proper value of those two variables, for example:
raw_n := "rdTmzrh7t0i_YN0MDLejnS0jXIFoSzRfFEbqf-bwGuRLnhLI4T3zGAk9HGZeAG6B5gg1D40Jsz1upo4E70VS0raGfSBPYPO7ZAJ2VCUUeblr9X_aWK4f294v4Cf3n8jZyFcGK9qhgcqy3DlHqqDANtjamWVtEhTRTFc-qoz1ScvHmPupsXlj1FsAEFEbVhP4705ez5gW3uQOoidrm38sPFwCN7g7xhA9CyzF04Zsjky55OfMCyWlIt7nljLx7ZRG3dVRD3vdEBI99qtxf43qMCWSPUk7Whn11Wf_u0xDrWhtGR9k599rKBBRWuqcujYYnFuOT0BeQIL25cePPK8lxw"
raw_e := "AQAB"
I suppose this is Base64 URL encoded.
I am using Go and when decoding them, I am having those values:
Value of N = 21944212446918148307583266513211511961176501179660110972882270757464120247554839864039423096862533136364974693915136668416630333929475539217929135693935014796040215031370110392351836608149288005498214604075814317642291632460369313969964278103135047317799644939426174469533574133378199425168189176269507635563270873057483981163804984573367938044175828995131343800369166239708527999544583873649523245709447765091038652758632917341001616695912404965678061555823269517387737693508193881028187912513285002546706797506255288735805912213209305939523528571103281920754204216834697552752238593850648051945624431639572960454087
Value of E = 65537
And the final values are:
TokenSignature= GYZQKv7o8_o9E4ktVKZngYD4BS5QluOMwE-MRcJB432CmNimQm6JbvT3H48ECThe4f3sZ1KyVbgDJbyUnlkaAwMEBjMnlV7AUaZb-ifveM7kHM30BS5LCV_SCiCk-PvmWjeIHu9bR3EwG8azJCceD5A7gDLmhAtPN94gRy-opXJPAnaCba00AwKBd_pN3UH7LYu4u4EQ29eIfn4k4RCLuR31jr7ad3dvvjhhy658dQSHzSuPZGcN1-CRVSlrd0nk0Ba2t8W33LtjxM6wzPThWgh0fpy2XEDosGU_9FiXdEjUKisE3VHxroygQ8ekVWKHssa2eujXCx8OthWzaGag0w1
Signing String= eyJhbGciOiJSUzI1NiIsImtpZCI6ImpBNlFvakp0RkI0TmNIR1BmcS85ZWgzSHI2YnVXWEI0VzkxRTd5bWNjSk09In0.eyJleHAiOjE0NzAyNzgzMTEsInRva2VuX3VzZSI6ImFjY2VzcyIsImlzcyI6Imh0dHBzOi8vY29nbml0by1pZHAudXMtZWFzdC0xLmFtYXpvbmF3cy5jb20vdXMtZWFzdC0xX0d1OFlhVGg3MiIsImNsaWVudF9pZCI6IjRwNzNuYjhra3NsbHJrbTlzMzdzYXZsNzEzIiwidXNlcm5hbWUiOiJtYXRlbyIsImtpZCI6IiIsImFsZyI6IiIsImp0aSI6IjhmNTBiZmU4LWVlNGUtNGFkZi04MDQxLWU5MGM4YWJkZDExZCIsImlhdCI6MTQ3MDI3NDcxMSwic3ViIjoiYjFjMDZhMTktYjE5Yy00NGMwLTgzZjctODY2NTZjYzRmMjMxIn0
rsa.PublicKey{N:21944212446918148307583266513211511961176501179660110972882270757464120247554839864039423096862533136364974693915136668416630333929475539217929135693935014796040215031370110392351836608149288005498214604075814317642291632460369313969964278103135047317799644939426174469533574133378199425168189176269507635563270873057483981163804984573367938044175828995131343800369166239708527999544583873649523245709447765091038652758632917341001616695912404965678061555823269517387737693508193881028187912513285002546706797506255288735805912213209305939523528571103281920754204216834697552752238593850648051945624431639572960454087, E:65537}
However, with those revisiter values, I am still getting crypto/rsa: verification error. How can I determine if the problem is coming from the JWT Library or the values?
Thank you very much
You should be able to verify the signature with the rsa.PublicKey generated by the program below or at https://play.golang.org/p/VZqD5m057b. It is based on code from https://github.com/mendsley/gojwk (please see the playground link or https://github.com/mendsley/gojwk/blob/master/LICENSE for the Copyright attribution) - I would recommend to clone that repository and use it as an example and model.
package main
import (
"crypto/rsa"
"encoding/base64"
"encoding/binary"
"fmt"
"math/big"
)
func main() {
rawN := "rdTmzrh7t0i_YN0MDLejnS0jXIFoSzRfFEbqf-bwGuRLnhLI4T3zGAk9HGZeAG6B5gg1D40Jsz1upo4E70VS0raGfSBPYPO7ZAJ2VCUUeblr9X_aWK4f294v4Cf3n8jZyFcGK9qhgcqy3DlHqqDANtjamWVtEhTRTFc-qoz1ScvHmPupsXlj1FsAEFEbVhP4705ez5gW3uQOoidrm38sPFwCN7g7xhA9CyzF04Zsjky55OfMCyWlIt7nljLx7ZRG3dVRD3vdEBI99qtxf43qMCWSPUk7Whn11Wf_u0xDrWhtGR9k599rKBBRWuqcujYYnFuOT0BeQIL25cePPK8lxw"
rawE := "AQAB"
decodedE, err := base64.RawURLEncoding.DecodeString(rawE)
if err != nil {
panic(err)
}
// make sure that the E field is at least 4 bytes, pad if necessary
if len(decodedE) < 4 {
ndata := make([]byte, 4)
copy(ndata[4-len(decodedE):], decodedE)
decodedE = ndata
}
pubKey := &rsa.PublicKey{
N: &big.Int{},
E: int(binary.BigEndian.Uint32(decodedE[:])),
}
decodedN, err := base64.RawURLEncoding.DecodeString(rawN)
if err != nil {
panic(err)
}
pubKey.N.SetBytes(decodedN)
fmt.Println(decodedN)
fmt.Println(decodedE)
fmt.Printf("%#v\n", *pubKey)
}
Output:
[173 212 230 206 184 123 183 72 191 96 221 12 12 183 163 157 45 35 92 129 104 75 52 95 20 70 234 127 230 240 26 228 75 158 18 200 225 61 243 24 9 61 28 102 94 0 110 129 230 8 53 15 141 9 179 61 110 166 142 4 239 69 82 210 182 134 125 32 79 96 243 187 100 2 118 84 37 20 121 185 107 245 127 218 88 174 31 219 222 47 224 39 247 159 200 217 200 87 6 43 218 161 129 202 178 220 57 71 170 160 192 54 216 218 153 101 109 18 20 209 76 87 62 170 140 245 73 203 199 152 251 169 177 121 99 212 91 0 16 81 27 86 19 248 239 78 94 207 152 22 222 228 14 162 39 107 155 127 44 60 92 2 55 184 59 198 16 61 11 44 197 211 134 108 142 76 185 228 231 204 11 37 165 34 222 231 150 50 241 237 148 70 221 213 81 15 123 221 16 18 61 246 171 113 127 141 234 48 37 146 61 73 59 90 25 245 213 103 255 187 76 67 173 104 109 25 31 100 231 223 107 40 16 81 90 234 156 186 54 24 156 91 142 79 64 94 64 130 246 229 199 143 60 175 37 199]
[0 1 0 1]
rsa.PublicKey{N:21944212446918148307583266513211511961176501179660110972882270757464120247554839864039423096862533136364974693915136668416630333929475539217929135693935014796040215031370110392351836608149288005498214604075814317642291632460369313969964278103135047317799644939426174469533574133378199425168189176269507635563270873057483981163804984573367938044175828995131343800369166239708527999544583873649523245709447765091038652758632917341001616695912404965678061555823269517387737693508193881028187912513285002546706797506255288735805912213209305939523528571103281920754204216834697552752238593850648051945624431639572960454087, E:65537}
I have used npm module jwk-to-pem in my nodeJS app. As seen in the documentation:
var jwkToPem = require('jwk-to-pem'),
jwt = require('jsonwebtoken');
var jwk = { kty: 'EC', crv: 'P-256', x: '...', y: '...' },
pem = jwkToPem(jwk);
jwt.verify(token, pem);
You can export pem to a file and use it in your code.
Decoding a JWK into a *rsa.PublicKey can be done by unmarshing the JSON into the following data structure and using the method below.
A link to this data structure:
// JSONKey represents a raw key inside a JWKs.
type JSONKey struct {
Curve string `json:"crv"`
Exponent string `json:"e"`
ID string `json:"kid"`
Modulus string `json:"n"`
X string `json:"x"`
Y string `json:"y"`
precomputed interface{}
}
A link to this method:
// RSA parses a JSONKey and turns it into an RSA public key.
func (j *JSONKey) RSA() (publicKey *rsa.PublicKey, err error) {
// Check if the key has already been computed.
if j.precomputed != nil {
var ok bool
if publicKey, ok = j.precomputed.(*rsa.PublicKey); ok {
return publicKey, nil
}
}
// Confirm everything needed is present.
if j.Exponent == "" || j.Modulus == "" {
return nil, fmt.Errorf("%w: rsa", ErrMissingAssets)
}
// Decode the exponent from Base64.
//
// According to RFC 7518, this is a Base64 URL unsigned integer.
// https://tools.ietf.org/html/rfc7518#section-6.3
var exponent []byte
if exponent, err = base64.RawURLEncoding.DecodeString(j.Exponent); err != nil {
return nil, err
}
// Decode the modulus from Base64.
var modulus []byte
if modulus, err = base64.RawURLEncoding.DecodeString(j.Modulus); err != nil {
return nil, err
}
// Create the RSA public key.
publicKey = &rsa.PublicKey{}
// Turn the exponent into an integer.
//
// According to RFC 7517, these numbers are in big-endian format.
// https://tools.ietf.org/html/rfc7517#appendix-A.1
publicKey.E = int(big.NewInt(0).SetBytes(exponent).Uint64())
// Turn the modulus into a *big.Int.
publicKey.N = big.NewInt(0).SetBytes(modulus)
// Keep the public key so it won't have to be computed every time.
j.precomputed = publicKey
return publicKey, nil
}
If you're looking to authenticate JWTs from a JWKs resource like AWS Cognito, I took these examples from a project that does just that: github.com/MicahParks/keyfunc

Got error "invalid character 'ï' looking for beginning of value” from json.Unmarshal

I use a Golang HTTP request to get json output as follow.
The web service I am trying to access is Micrsoft Translator https://msdn.microsoft.com/en-us/library/dn876735.aspx
//Data struct of TransformTextResponse
type TransformTextResponse struct {
ErrorCondition int `json:"ec"` // A positive number representing an error condition
ErrorDescriptive string `json:"em"` // A descriptive error message
Sentence string `json:"sentence"` // transformed text
}
//some code ....
body, err := ioutil.ReadAll(response.Body)
defer response.Body.Close()
if err != nil {
return "", tracerr.Wrap(err)
}
transTransform = TransformTextResponse{}
err = json.Unmarshal(body, &transTransform)
if err != nil {
return "", tracerr.Wrap(err)
}
I got an error from invalid character 'ï' looking for beginning of value
So, I try to print the body as string fmt.Println(string(body)), it show:
{"ec":0,"em":"OK","sentence":"This is too strange i just want to go home soon"}
It seems the data doesn't have any problem, so I tried to create the same value by jason.Marshal
transTransform := TransformTextResponse{}
transTransform.ErrorCondition = 0
transTransform.ErrorDescriptive = "OK"
transTransform.Sentence = "This is too strange i just want to go home soon"
jbody, _ := json.Marshal(transTransform)
I found the original data might have problem, so I try to compare two data in []byte format.
Data from response.Body:
[239 187 191 123 34 101 99 34 58 48 44 34 101 109 34 58 34 79 75 34 44 34 115 101 110 116 101 110 99 101 34 58 34 84 104 105 115 32 105 115 32 116 111 111 32 115 116 114 97 110 103 101 32 105 32 106 117 115 116 32 119 97 110 116 32 116 111 32 103 111 32 104 111 109 101 32 115 111 111 110 34 125]
Data from json.Marshal
[123 34 101 99 34 58 48 44 34 101 109 34 58 34 79 75 34 44 34 115 101 110 116 101 110 99 101 34 58 34 84 104 105 115 32 105 115 32 116 111 111 32 115 116 114 97 110 103 101 32 105 32 106 117 115 116 32 119 97 110 116 32 116 111 32 103 111 32 104 111 109 101 32 115 111 111 110 34 125]
Any idea how I parse this response.Body and Unmarshal it into data structure?
The server is sending you a UTF-8 text string with a Byte Order Mark (BOM). The BOM identifies that the text is UTF-8 encoded, but it should be removed before decoding.
This can be done with the following line (using package "bytes"):
body = bytes.TrimPrefix(body, []byte("\xef\xbb\xbf")) // Or []byte{239, 187, 191}
PS. The error referring to ï is because the UTF-8 BOM interpreted as an ISO-8859-1 string will produce the characters .