Best way to group two structs into a third one on Go - json

I have the following structs:
type A1 struct {
IdA1 string
NameA1 string
key string
}
type B1 struct {
IdB1 string
NameB1 string
Size string
key string
}
type C1 struct {
IdA1 string
Name string
B1Set B1
}
and I need to create a []struct of type C1, that holds a B1Set of B1, B1 is huge more than 2K entries and A1 is just 10 entries, a really slow and inefficient implementation would be to loop over A1, and ask if B1-key is equal to A1-key and store in a map the result, but.. is there any better way to implement this?
thanks in advance,
Add more info:
They are two different JSON files:
Json1:
[
{
"id": "device1",
"name": "dev1",
"pool": "pool1"
},
{
"id": "device2",
"name": "dev2",
"pool": "pool2"
}
...
]
And the other Json:
[
{
"name": "port1",
"size": 10,
"pool": "pool1",
"status": "active"
},
{
"name": "port2",
"size": 60,
"pool": "pool1",
"status": "active"
},
{
"name": "port3",
"size": 20,
"pool": "pool2",
"status": "down"
},
{
"name": "port8",
"size": 100,
"pool": "pool2",
"status": "active"
},
{
"name": "port10",
"size": 8000,
"pool": "pool1",
"status": "active"
},
...
]
So I need to create a new JSON file from those two with the following:
[
{
"id": "device1",
"name": "dev1",
"pool": "pool1",
"ports": [
{
"name": "port1",
"size": 10,
"pool": "pool1",
"status": "active"
},
{
"name": "port2",
"size": 60,
"pool": "pool1",
"status": "active"
},
{
"name": "port10",
"size": 8000,
"pool": "pool1",
"status": "active"
}
]
},
{
"id": "device2",
"name": "dev2",
"pool": "pool2",
"ports": [
{
"name": "port3",
"size": 20,
"pool": "pool2",
"status": "active"
},
{
"name": "port8",
"size": 100,
"pool": "pool2",
"status": "active"
}
]
}
]
taking into account that Json1 is not more than 10-12 entries and Json2 has more than 800-1000 entries.
So far:
read Json1 (devices), and Json2 (ports) and pass them into two []structs. Then:
results := make(map[string]*models.PortDevices}
portDevs := []models.PortDevices{}
for i := range devices {
results[devices[i].Pool] = &models.PortDevices{}
m := results[devices[i].Pool]
m.Id = devices[i].Id
m.Name = devices[i].Name
m.Pool = devices[i].Pool
m.Status = devices[i].Status
for p := range ports {
if val, ok := results[ports[p].Pool]; ok {
m := val
m.Ports = ports
}
}
portDevs = append(portDevs, *m)
}
devports := []models.PortDevices{}
for _, value := range results {
devports = append(devports, *value)
}

Is this what you are looking to do? https://play.golang.org/p/AZNzQAwRhN0
What this does is build a map grouping all of the ports by pool. Then it loops through what I labeled clusters and assigns the slice of Port to the matching Cluster by grabbing the matching slice by Pool value.
package main
import (
"encoding/json"
"fmt"
)
type Cluster struct {
ID string `json:"id"`
Name string `json:"name"`
Pool string `json:"pool"`
Ports []Port `json:"ports"`
}
type Port struct {
Name string `json:"name"`
Size int `json:"size"`
Pool string `json:"pool"`
Status string `json:"status"`
}
func main() {
var resources []Port
err := json.Unmarshal([]byte(resourceJSON), &resources)
if err != nil {
panic(err)
}
resourcesByPool := make(map[string][]Port)
for _, resource := range resources {
if _, ok := resourcesByPool[resource.Pool]; !ok {
resourcesByPool[resource.Pool] = []Port{}
}
resourcesByPool[resource.Pool] = append(resourcesByPool[resource.Pool], resource)
}
var clusters []Cluster
err = json.Unmarshal([]byte(clusterJSON), &clusters)
if err != nil {
panic(err)
}
for i := 0; i < len(clusters); i++ {
clusters[i].Ports = resourcesByPool[clusters[i].Pool]
}
out, err := json.MarshalIndent(clusters, "", " ")
if err != nil {
panic(err)
}
fmt.Println(string(out))
}
var (
clusterJSON = `[
{
"id": "device1",
"name": "dev1",
"pool": "pool1"
},
{
"id": "device2",
"name": "dev2",
"pool": "pool2"
}
]`
resourceJSON = `[
{
"name": "port1",
"size": 10,
"pool": "pool1",
"status": "active"
},
{
"name": "port2",
"size": 60,
"pool": "pool1",
"status": "active"
},
{
"name": "port3",
"size": 20,
"pool": "pool2",
"status": "down"
},
{
"name": "port8",
"size": 100,
"pool": "pool2",
"status": "active"
},
{
"name": "port10",
"size": 8000,
"pool": "pool1",
"status": "active"
}]`
)

Related

Flutter - How to read json object with two lists?

I have a json like this:
{
"data": [
{
"id": "99412",
"type": "post_list_item",
"attributes": {
"message": "#:user/3 \n#:user/77 \n#:user/52 #:user/3921 #:user/267 #:user/5 \n\nWhich sector will outperform in 2023 ???",
"visibility": "public_visible",
"created_at": "2023-01-07T14:38:00.691+05:30",
"updated_at": "2023-01-07T14:38:00.779+05:30",
"meta_info": null,
"type": "Post",
"likes_count": 8,
"replies_count": 2,
"widgets_data": [],
"is_liked": false,
"is_bookmarked": false,
"is_deleted": false,
"share_count": null
},
"relationships": {
"user": {
"data": {
"id": "86806",
"type": "user"
}
},
"poll": {
"data": {
"id": "483",
"type": "poll"
}
},
"tagged_users": {
"data": [
{
"id": "3",
"type": "user"
},
{
"id": "5",
"type": "user"
},
{
"id": "52",
"type": "user"
},
{
"id": "77",
"type": "user"
},
{
"id": "267",
"type": "user"
},
{
"id": "3921",
"type": "user"
}
]
},
"tagged_companies": {
"data": []
},
"tagged_topics": {
"data": []
},
"tagged_instruments": {
"data": []
},
"stories": {
"data": []
},
"attachments": {
"data": []
},
"media_files": {
"data": []
}
}
},
{
"id": "99548",
"type": "post_list_item",
"attributes": {
"message": "๐“๐จ๐ฉ ๐ง๐ž๐ฐ๐ฌ ๐ญ๐จ๐๐š๐ฒ๐ŸŽŒ \n\n๐Ÿ“SGX Nifty up by 132 pts indicating a strong open
today. FIIs sold Rs 29 Bn while DIIs bought Rs 10.8 Bn of stocks on Friday.\n\n๐Ÿ“India surpasses Japan to become world's 3rd largest auto market in 2022: SIAM report\n\n๐Ÿ“Dr Lal PathLabs, leader in North India, expands in West India. Subsidiary opens apex lab in Mumbai - all nearby samples to be tested here instead of Delhi (positive).\n\n๐Ÿ“IL&FS pays 5 PSU Banks 87% of their โ‚น1,500 crore dues. Canara, Union Bank, Central, Punjab & Sind Bank.\n\n๐Ÿ“EV scooter co. Ather Energy to achieve $1 billion in revenue in 2023 and turn profitable in few years. Beneficiary: Hero Motors\n\n๐Ÿ“Outstanding dues by power distributors (discoms) to producers (gencos) has nearly halved to Rs 62,681 cr in last 1 year. Benficiary: PFC, REC (but priced in).\n\n๐Ÿ“Solar panel manufacters boost production as costs fall for polysilicon and wafer (RM for solar cells). Positive for Sterling Wilson Renewables, Borosil Renewables.\n\n1/2",
"visibility": "public_visible",
"created_at": "2023-01-09T09:05:41.226+05:30",
"updated_at": "2023-01-09T09:05:41.277+05:30",
"meta_info": null,
"type": "Post",
"likes_count": 4,
"replies_count": 1,
"widgets_data": [],
"is_liked": false,
"is_bookmarked": false,
"is_deleted": false,
"share_count": null
},
"relationships": {
"user": {
"data": {
"id": "83681",
"type": "user"
}
},
"poll": {
"data": null
},
"tagged_users": {
"data": []
},
"tagged_companies": {
"data": []
},
"tagged_topics": {
"data": []
},
"tagged_instruments": {
"data": []
},
"stories": {
"data": []
},
"attachments": {
"data": []
},
"media_files": {
"data": []
}
}
}, ],
"included": [
{
"id": "86806",
"type": "user",
"attributes": {
"display_name": "Ravi Gupta",
"username": "ravigupta132",
"dp_thumbnail": "/rails/active_storage/representations/proxy/eyJfcmFpbHMiOnsibWVzc2FnZSI6IkJBaHBBZ2E4IiwiZXhwIjpudWxsLCJwdXIiOiJibG9iX2lkIn19--763b2ccc4372862bb99de6620aa9fe9661693603/eyJfcmFpbHMiOnsibWVzc2FnZSI6IkJBaDdCem9MWm05eWJXRjBTU0lJYW5CbkJqb0dSVlE2RkhKbGMybDZaVjkwYjE5c2FXMXBkRnNIYVFHQWFRR0EiLCJleHAiOm51bGwsInB1ciI6InZhcmlhdGlvbiJ9fQ==--7b4a499a9ad66d05dd5deec71f9adfd36a29feb5/image_cropper_1672468958147.jpg",
"market_view": "neutral"
},
"relationships": {}
},
{
"id": "483",
"type": "poll",
"attributes": {
"question": "#:user/3 \n#:user/77 \n#:user/52 #:user/3921 #:user/267 #:user/5 \n\nWhich sector will outperform in 2023 ???",
"options": [
"Metal",
"PSU banks",
"IT",
"Infrastructure"
],
"vote_count": 52,
"grouped_count": null,
"ends_at": "2023-01-14T14:38:01.760+05:30",
"my_vote": null,
"is_expired": false
},
"relationships": {
"base_post": {
"data": {
"id": "99412",
"type": "post"
}
}
}
}, ]
}
Basically, 2 lists in an object.
I am unable to deserialize this data. How do I de-serialize this data to get 2 lists? I need to get data based on a common id.
This is what I have currently (WIP):
Future<List<Datum>> getPostData() async {
try {
Response response = await _dio.get(url);
var jsonData = response.data;
//Map<String, dynamic> postMap = response.data;
final someMappedObjectList = <Datum>[];
for (var map in jsonData["data"]) {
Datum someObject = Datum(
id: map["id"],
type: map["type"],
attributes: map["attributes"],
relationships: map["relationships"]);
//final someObject = Datum.fromJson(map);
someMappedObjectList.add(someObject);
}
//List<dynamic> listDatum = postMap["data"].toList();
//List<Datum> listDatum = Datum.fromJson(jsonData["data"]) as List<Datum>;
return someMappedObjectList;
} //
catch (e) {
print(e);
return [];
}
}
Future<List> getPostIncluded() async {
try {
Response response = await _dio.get(url);
var jsonData = response.data;
Map<String, dynamic> postMap = response.data;
List<dynamic> listIncluded = postMap["included"].toList();
return listIncluded;
} //
catch (e) {
print(e);
return [];
}
}
Especially for something this complex, I strongly suggest using the json_serializable package. It will save you quite a bit of trouble in the long term.

Parse unstructured json in golang

Is there any solution to parse an unstructured json(text) data?
below is a sample response of a web requst that i want to parse and access data (the inner list)
res,err := http.Get("url_of_server")
[[
{
"id": "1",
"text": "sample text",
"user": {
"user_id": "1",
"username": "user1"
},
"created_at_utc": "2022-12-20T16:38:06+00:00",
"status": "Active"
},
{
"id": "2",
"text": "sample text",
"user": {
"user_id": "2",
"username": "user2"
},
"created_at_utc": "2022-12-01T10:15:00+00:00",
"status": "Active"
}
],
"{"code": "hsdvnkvuahudvhafdlfv",
"is_updated": true}",
null
]
what i want to get is:
[
{
"id": "1",
"text": "sample text",
"user": {
"user_id": "1",
"username": "user1"
},
"created_at_utc": "2022-12-20T16:38:06+00:00",
"status": "Active"
},
{
"id": "2",
"text": "sample text",
"user": {
"user_id": "2",
"username": "user2"
},
"created_at_utc": "2022-12-01T10:15:00+00:00",
"status": "Active"
}
]
in python it is possible by easily using res.json()[0]
I have tried using json.Unmarshal() to a map and also struct but does not work,
i don't know how to get rid of this part of response:
"{"code": "hsdvnkvuahudvhafdlfv",
"is_updated": true}",
null
Declare a type for the items:
type Item struct {
ID string `json:"id"`
Text string `json:"text"`
User struct {
UserID string `json:"user_id"`
Username string `json:"username"`
} `json:"user"`
CreatedAtUtc time.Time `json:"created_at_utc"`
Status string `json:"status"`
}
Declare a slice of the items:
var items []Item
Declare a slice representing the entire JSON thing. The first element is the items.
var v = []any{&items}
Unmarshal to v. The items slice will have the values that you are looking for. The second and third elements of v will contain the values you want to ignore.
err := json.Unmarshal(data, &v)
Run the code in the GoLang PlayGround.
Go's standard JSON library is not as flexible as others when it comes to dealing with unexpected or uncontrolled input.
A great alternative is tidwall's gjson.
Example code with gjson:
package main
import (
"fmt"
"github.com/tidwall/gjson"
)
const textInput = `[[
{
"id": "1",
"text": "sample text",
"user": {
"user_id": "1",
"username": "user1"
},
"created_at_utc": "2022-12-20T16:38:06+00:00",
"status": "Active"
},
{
"id": "2",
"text": "sample text",
"user": {
"user_id": "2",
"username": "user2"
},
"created_at_utc": "2022-12-01T10:15:00+00:00",
"status": "Active"
}
],
"{"code": "hsdvnkvuahudvhafdlfv",
"is_updated": true}",
null
]`
func main() {
jsonBody := gjson.Parse(textInput)
fmt.Println(jsonBody.Get("0"))
}

Gorm Has Many relationship can't append more than one model instance during association

Summary
Main question: Can I append more than one model instance in a hasMany relationship in gorm or should I use a ManyToMany relationship? E.g. A Product model would have more than one Image model instance.
Here are the relevant models:
type Publication struct {
ID string `gorm:"not null"`
CreatedAt time.Time `sql:"DEFAULT:CURRENT_TIMESTAMP" json:"created_at"`
UpdatedAt time.Time `sql:"DEFAULT:CURRENT_TIMESTAMP" json:"updated_at"`
Views uint64 `gorm:"default:0" json:"views"`
Product Product `gorm:"foreignkey:ID" json:"product"`
}
type Product struct {
ID string `gorm:"not null"`
Title string
Price uint64
Images []ProductImage `gorm:"foreignkey:ID" json:"images"`
}
type ProductImage struct {
ID string
Name string
}
During Server Initialization, I create, associate and load some data to the MySQL DB in the following way:
err = db.Debug().Model(&models.Product{}).Create(&mk_list_products[i]).Error
if err != nil {
log.Fatal(err)
}
err = db.Debug().Model(&models.Publication{}).Create(&mk_list_publications[i]).Error
if err != nil {
log.Fatal(err)
}
err = db.Debug().Model(&mk_list_products[i]).Association("Images").Append(&product_images)
if err != nil {
log.Fatal(err)
}
I'm querying Publication model with Product Association as follow:
pbs := []models.Publication{}
err = db.
Debug().
Preload("Product.Images").
Model(&models.Publication{}).
Limit(3).
Where("created_at <= ?", time.Now()).
Find(&pbs).Error
if err != nil {
console.Pretty(err)
return
}
For Publication with ID=3 (similar with the rest) I get
{
"ID": "3",
"created_at": "2022-01-06T21:26:41.585-06:00",
"updated_at": "2022-01-07T21:26:42.154-06:00",
"views": 0,
"product": {
"ID": "3",
"Title": "Product Title A",
"Price": 200,
"category": {
"ID": "",
"Parent": "",
"Image": "",
"Order": 0,
"Name": "",
"Description": "",
"Slug": ""
},
"State": "",
"Description": "Some Description",
"Availability": "",
"Brand": "Some Brand",
"labels": null,
"SKU": "Some SKU",
"images": [
{
"ID": "3",
"Name": "/path/cannabis_product_1.png"
}
]
}
Expected result
{
"ID": "3",
"created_at": "2022-01-06T21:26:41.585-06:00",
"updated_at": "2022-01-07T21:26:42.154-06:00",
"views": 0,
"product": {
"ID": "3",
"Title": "Product Title A",
"Price": 200,
"category": {
"ID": "",
"Parent": "",
"Image": "",
"Order": 0,
"Name": "",
"Description": "",
"Slug": ""
},
"State": "",
"Description": "Some Description",
"Availability": "",
"Brand": "Some Brand",
"labels": null,
"SKU": "Some SKU",
"images": [
{
"ID": "2",
"Name": "/path/cannabis_product_2.png"
}
{
"ID": "3",
"Name": "Name": "/path/cannabis_product_3.png""
}
]
}
Solution
I still don't understand why declaring the ID of ImageProduct as the foreign on the Product model gives unexpected result.
To make this work, I had to defined and add a different foreignKey other than ID on product Imagesfield
type Product struct {
ID string `gorm:"not null"`
Title string
Price uint64
...
Images []ProductImage `gorm:"foreignkey:ProductId" json:"images"`
}
Then, on the ProductImage model I added an additional field ProductId
type ProductImage struct {
ID string
ProductId string
Name string
}
After appending two test ProductImage instances to Publication with ID=3 I get the expected result.
err = db.Debug().Model(&mk_list_products[2]).Association("Images").Append(&product_images[2], &product_images[3])
if err != nil {
log.Fatal(err)
}
...
{
"ID": "3",
"created_at": "2022-01-07T11:20:32.298-06:00",
"updated_at": "2022-01-08T11:20:32.867-06:00",
"views": 0,
"product": {
"ID": "3",
"Title": "Product Title 3",
"Price": 200,
"category": {
"ID": "",
"Parent": "",
"Image": "",
"Order": 0,
"Name": "",
"Description": "",
"Slug": ""
},
"State": "",
"Description": "Some Description",
"Availability": "",
"Brand": "Some Brand",
"labels": null,
"SKU": "Some SKU",
"images": [
{
"ID": "3",
"ProductId": "3",
"Name": "/path/cannabis_product_3.png"
},
{
"ID": "4",
"ProductId": "3",
"Name": "/path/cannabis_product_4.png"
}
]
}
}

parse complex json from http response

I'm doing a http.PostForm against an API and the result is a json.
The Json structure is like this:
{
"sites": [
{
"site_id": 456,
"status": "pending-dns-changes",
"domain": "blabla",
"account_id": 123,
"acceleration_level": "advanced",
"site_creation_date": 1515455285000,
"ips": [
"1.2.3.4"
],
"dns": [
{
"dns_record_name": "something.bla.com",
"set_type_to": "CNAME",
"set_data_to": [
"something.x.incapdns.net"
]
}
],
"original_dns": [
{
"dns_record_name": "blabla2.com",
"set_type_to": "A",
"set_data_to": [
""
]
},
{
"dns_record_name": "blabla3.something.com",
"set_type_to": "A",
"set_data_to": [
"1.2.4.5"
]
},
],
"warnings": [],
"active": "bypass",
"support_all_tls_versions": false,
"use_wildcard_san_instead_of_full_domain_san": true,
"add_naked_domain_san": true,
"additionalErrors": [],
"display_name": "something.blablabla2.com",
"security": {
"waf": {
"rules": [
{
"action": "api.threats.action.alert",
"action_text": "Alert Only",
"id": "api.threats.sql_injection",
"name": "SQL Injection"
},
{
"action": "api.threats.action.alert",
"action_text": "Alert Only",
"id": "api.threats.cross_site_scripting",
"name": "Cross Site Scripting"
},
{
"action": "api.threats.action.alert",
"action_text": "Alert Only",
"id": "api.threats.illegal_resource_access",
"name": "Illegal Resource Access"
},
{
"block_bad_bots": true,
"challenge_suspected_bots": false,
"id": "api.threats.bot_access_control",
"name": "Bot Access Control"
},
{
"activation_mode": "api.threats.ddos.activation_mode.auto",
"activation_mode_text": "Auto",
"ddos_traffic_threshold": 1000,
"id": "api.threats.ddos",
"name": "DDoS"
},
{
"action": "api.threats.action.quarantine_url",
"action_text": "Auto-Quarantine",
"exceptions": [
{
"values": [
{
"urls": [
{
"value": "/neverbogus.pvr",
"pattern": "EQUALS"
}
],
"id": "api.rule_exception_type.url",
"name": "URL"
}
],
"id": 1618746719
},
{
"values": [
{
"urls": [
{
"value": "/doubleneverbogus.pvr",
"pattern": "EQUALS"
},
{
"value": "/ddoubleneverbogus.pvr",
"pattern": "EQUALS"
}
],
"id": "api.rule_exception_type.url",
"name": "URL"
}
],
"id": 856301271
},
{
"values": [
{
"client_apps": [
"537"
],
"id": "api.rule_exception_type.client_app_id",
"name": "Client app ID"
}
],
"id": 58318563
},
{
"values": [
{
"ips": [
"192.168.66.66"
],
"id": "api.rule_exception_type.client_ip",
"name": "IP"
}
],
"id": 707083378
},
{
"values": [
{
"geo": {
"countries": [
"BF"
]
},
"id": "api.rule_exception_type.country",
"name": "Country"
}
],
"id": 1432086237
},
{
"values": [
{
"user_agents": [
"gasdfafafdfasdfadsfadsffads"
],
"id": "api.rule_exception_type.user_agent",
"name": "User agent"
}
],
"id": 1876871261
},
{
"values": [
{
"parameters": [
"bogusparamnamehere234"
],
"id": "api.rule_exception_type.http_parameter",
"name": "Http parameter"
}
],
"id": 1338747790
}
],
"id": "api.threats.backdoor",
"name": "Backdoor Protect"
},
{
"action": "api.threats.action.alert",
"action_text": "Alert Only",
"id": "api.threats.remote_file_inclusion",
"name": "Remote File Inclusion"
},
{
"action": "api.threats.action.disabled",
"action_text": "Ignore",
"id": "api.threats.customRule",
"name": "IncapRules"
}
]
}
},
"sealLocation": {
"id": "api.seal_location.none",
"name": "No seal "
},
"ssl": {
"origin_server": {
"detected": true,
"detectionStatus": "ok"
},
"custom_certificate": {
"active": false
},
"generated_certificate": {
"ca": "GS",
"validation_method": "dns",
"validation_data": [
{
"dns_record_name": "blablablasomething2",
"set_type_to": "TXT",
"set_data_to": [
"somethingtexty"
]
}
],
"san": [
"*.blabla2.com"
],
"validation_status": "done"
}
},
"siteDualFactorSettings": {
"enabled": false,
"customAreas": [],
"customAreasExceptions": [],
"allowAllUsers": true,
"shouldSuggestApplicatons": true,
"allowedMedia": [
"ga",
"sms"
],
"shouldSendLoginNotifications": true,
"version": 0
},
"login_protect": {
"enabled": false,
"specific_users_list": [],
"send_lp_notifications": true,
"allow_all_users": true,
"authentication_methods": [
"ga",
"sms"
],
"urls": [],
"url_patterns": []
},
"performance_configuration": {
"advanced_caching_rules": {
"never_cache_resources": [],
"always_cache_resources": []
},
"acceleration_level": "advanced",
"async_validation": true,
"minify_javascript": true,
"minify_css": true,
"minify_static_html": true,
"compress_jpeg": true,
"compress_jepg": true,
"progressive_image_rendering": false,
"aggressive_compression": false,
"compress_png": true,
"on_the_fly_compression": true,
"tcp_pre_pooling": true,
"comply_no_cache": false,
"comply_vary": false,
"use_shortest_caching": false,
"perfer_last_modified": false,
"prefer_last_modified": false,
"disable_client_side_caching": false,
"cache300x": false,
"cache_headers": []
},
"extended_ddos": 1000000,
"log_level": "security",
"incap_rules": [
{
"id": 51589,
"name": "Wordpress",
"action": "api.rule_action_type.rule_action_alert",
"rule": "(URL contains \"/xmlrpc.php$\" | URL contains \"/wp-login.php$\") & (User-Agent == \"Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36\" | User-Agent == \"Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101 Firefox/40.1\")",
"creation_date": 1518810206000
},
{
"id": 52179,
"name": "No Browser",
"action": "api.rule_action_type.rule_action_alert",
"rule": "(ClientType != Browser & ClientType != SpamBot & ClientType != DDoSBot & ClientType != SiteHelper) & (User-Agent != \"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/534+ (KHTML, like Gecko) BingPreview/1.0b\" & User-Agent != \"Pingdom.com_bot_version_1.4_(http://www.pingdom.com/)\")",
"creation_date": 1519432068000
}
],
"res": 0,
"res_message": "OK",
"debug_info": {
"id-info": "13019"
}
},
API Docs: https://docs.imperva.com/bundle/cloud-application-security/page/api/sites-api.htm#List
This seems to be very painful for me to understand in go as I'm just starting..
From what I've been reading I should just define an object of type struct with my expected format.
Any way I can tackle this a lot easier? I'm interested in extracted for each individual site a few items like site_id / ips / domain.
Unsure how I should tackle this. I've been trying to get my head around https://github.com/buger/jsonparser but I'm failing to actually understand how to use it.
So far what I've been doing is:
func main() {
fmt.Println("Starting this..")
formData := url.Values{
"api_id": {keyid},
"api_key": {apikey},
"page_size": {"100"},
"page_num": {"0"},
}
response, err := http.PostForm("https://my.imperva.com/api/prov/v1/sites/list", formData)
if nil != err {
fmt.Println("Ooops..", err)
}
log.Println(response.Status)
bodyBytes, err := ioutil.ReadAll(response.Body)
if err != nil {
log.Fatal(err)
}
var results map[string]interface{}
fmt.Println(string(bodyBytes))
json.Unmarshal([]byte(bodyBytes), &results)
defer response.Body.Close()
}
If you only need to extract a couple of fields, you don't really need to define an elaborate struct type to describe the whole structure. You can use the stdlib to unmarshal the JSON into a map[string]interface{} and then use something like this function to look for interesting keys:
// findNested looks for a key named s in map m. If values in m map to other
// maps, findNested looks into them recursively. Returns true if found, and
// the value found.
func findNested(m map[string]interface{}, s string) (bool, interface{}) {
// Try to find key s at this level
for k, v := range m {
if k == s {
return true, v
}
}
// Not found on this level, so try to find it nested
for _, v := range m {
nm, ok := v.(map[string]interface{})
if ok {
found, val := findNested(nm, s)
if found {
return found, val
}
}
}
// Not found recursively
return false, nil
}
This code deals only with nested maps. You can adjust it slightly to see through slices, etc.
You can define a single, complex go struct that reflects the entire json object structure, and annotate all of the elements with the json object/element names.
You can define simple structs for each of the nested objects, and annotate these structs with the json object/element names; and then build the complex struct by composing these structs.
Either of these are a ton of effort. You do not need the entire struct, you only need a few elements.
You can either define only the parts of the struct that you need to extract, and annotate those, or you can use the map[string]interface approach you have used. Both approaches work well.
package main
import (
"encoding/json"
"fmt"
)
func main() {
// your http query/response here
// Declare an empty interface
var results map[string]interface{}
body, err := ioutil.ReadAll(response.Body)
if err != nil {
log.Fatal(err)
}
fmt.Println(string(body))
// Unmarshal or Decode the JSON to the interface.
json.Unmarshal([]byte(body), &results)
defer response.Body.Close()
json.Unmarshal([]byte(empJson), &result)
//Reading each value by its key
fmt.Println("site_id:", result["site_id"],
"\ndomain:", result["domain"],
"\nip:", result["ips"][0],
// etc
)
}
See also:
How to unmarshal JSON in to array of interface and use

Grab an ID from particular array index

I have some JSON which returns the following:
"data": [
{
"created_time": "2010-09-03T16:07:14+0000",
"name": "Profile Pictures",
"id": "125287297520173"
},
{
"created_time": "2010-12-03T00:05:31+0000",
"name": "Mobile Uploads",
"id": "146617845387118"
},
{
"created_time": "2013-07-27T11:34:50+0000",
"name": "Timeline Photos",
"id": "546011742114391"
},
{
"created_time": "2017-01-04T19:02:40+0000",
"name": "Untitled Album",
"id": "1178578645524361"
},
{
"created_time": "2016-09-10T18:26:25+0000",
"name": "Untitled Album",
"id": "1076646985717528"
},
{
"created_time": "2016-07-06T18:27:09+0000",
"name": "OS X Photos",
"id": "1033031426745751"
},
{
"created_time": "2013-06-22T07:32:01+0000",
"name": "iOS Photos",
"id": "530462737002625"
},
{
"created_time": "2012-05-22T19:01:42+0000",
"name": "Cover Photos",
"id": "370987619616805"
},
{
"created_time": "2015-08-27T18:59:56+0000",
"name": "Untitled Album",
"id": "879780692070826"
},
{
"created_time": "2014-12-06T16:13:01+0000",
"name": "DRMC 2005 Batch",
"id": "761469943901902"
},
{
"created_time": "2013-06-16T09:01:17+0000",
"name": "Instagram Photos",
"id": "528368577212041"
},
{
"created_time": "2012-09-09T17:37:55+0000",
"name": "Liverpool Exclusive",
"id": "416230538425846"
},
{
"created_time": "2012-09-10T16:31:52+0000",
"name": "LIVERPOOL FC TOUR",
"id": "416540875061479"
},
{
"created_time": "2010-06-11T19:37:20+0000",
"name": "cars",
"id": "104577376257832"
},
{
"created_time": "2011-03-29T23:50:18+0000",
"name": "Camera+ Photos",
"id": "174268382622064"
}
]
I would like to grab the id of the field "name": "Profile Pictures" . I am using SwiftyJSON to cast the types and so far I managed to do this :
let dictionary = JSON(result)
// print("albums **************\(dictionary)")
if let data = dictionary["data"].array {
print("data of profilePicture ******* \(data)")
let index = data.index{ $0["name"] == "Profile Pictures" }
}
}
I could detech the name of profilePicture as you can see but now I want to grab the id of that json object. Please help.
Use filter to get the dictionary you're interested inโ€ฆ
let dictionary = JSON(result)
// print("albums **************\(dictionary)")
if let data = dictionary["data"].array {
print("data of profilePicture ******* \(data)")
if let dict = data.filter{ $0["name"] == "Profile Pictures" }.first as? [String: String] {
let id = dict["id"]
}
}
let dictionary = JSON(result)
// print("albums ID are **************\(dictionary)")
if let data = dictionary["data"].array {
print("data of profilePicture ******* \(data)")
if let dict = data.first(where: { ($0["name"].string ) == "Profile Pictures" }) {
let id = dict["id"]
print("my desired id : ********* \(id)")
}
}