Parse unstructured json in golang - json

Is there any solution to parse an unstructured json(text) data?
below is a sample response of a web requst that i want to parse and access data (the inner list)
res,err := http.Get("url_of_server")
[[
{
"id": "1",
"text": "sample text",
"user": {
"user_id": "1",
"username": "user1"
},
"created_at_utc": "2022-12-20T16:38:06+00:00",
"status": "Active"
},
{
"id": "2",
"text": "sample text",
"user": {
"user_id": "2",
"username": "user2"
},
"created_at_utc": "2022-12-01T10:15:00+00:00",
"status": "Active"
}
],
"{"code": "hsdvnkvuahudvhafdlfv",
"is_updated": true}",
null
]
what i want to get is:
[
{
"id": "1",
"text": "sample text",
"user": {
"user_id": "1",
"username": "user1"
},
"created_at_utc": "2022-12-20T16:38:06+00:00",
"status": "Active"
},
{
"id": "2",
"text": "sample text",
"user": {
"user_id": "2",
"username": "user2"
},
"created_at_utc": "2022-12-01T10:15:00+00:00",
"status": "Active"
}
]
in python it is possible by easily using res.json()[0]
I have tried using json.Unmarshal() to a map and also struct but does not work,
i don't know how to get rid of this part of response:
"{"code": "hsdvnkvuahudvhafdlfv",
"is_updated": true}",
null

Declare a type for the items:
type Item struct {
ID string `json:"id"`
Text string `json:"text"`
User struct {
UserID string `json:"user_id"`
Username string `json:"username"`
} `json:"user"`
CreatedAtUtc time.Time `json:"created_at_utc"`
Status string `json:"status"`
}
Declare a slice of the items:
var items []Item
Declare a slice representing the entire JSON thing. The first element is the items.
var v = []any{&items}
Unmarshal to v. The items slice will have the values that you are looking for. The second and third elements of v will contain the values you want to ignore.
err := json.Unmarshal(data, &v)
Run the code in the GoLang PlayGround.

Go's standard JSON library is not as flexible as others when it comes to dealing with unexpected or uncontrolled input.
A great alternative is tidwall's gjson.
Example code with gjson:
package main
import (
"fmt"
"github.com/tidwall/gjson"
)
const textInput = `[[
{
"id": "1",
"text": "sample text",
"user": {
"user_id": "1",
"username": "user1"
},
"created_at_utc": "2022-12-20T16:38:06+00:00",
"status": "Active"
},
{
"id": "2",
"text": "sample text",
"user": {
"user_id": "2",
"username": "user2"
},
"created_at_utc": "2022-12-01T10:15:00+00:00",
"status": "Active"
}
],
"{"code": "hsdvnkvuahudvhafdlfv",
"is_updated": true}",
null
]`
func main() {
jsonBody := gjson.Parse(textInput)
fmt.Println(jsonBody.Get("0"))
}

Related

How to parameterize a part of string in a field of the response in karate

I am trying to print a response based on the particular parameters.
For that, I have the response from an API as below:
{
"ABC": {
"code": "ABC",
"isActive": "true",
"lastUpdatedBy": "username",
"execution": {
"status": "0",
},
"priority": "1"
},
"DEF": {
"code": "DEF",
"isActive": "true",
"lastUpdatedBy": "username",
"execution": {
"status": "1",
},
"priority": "1"
},
"GHI": {
"code": "GHI",
"isActive": "true",
"lastUpdatedBy": "username",
"execution": {
"status": "2",
},
"priority": "1"
},
"JKL": {
"code": "JKL",
"isActive": "true",
"lastUpdatedBy": "username",
"execution": {
"status": "0",
},
"priority": "1"
},
}
Here is the feature file that I am using to print that particular value:
Feature: Value extraction
Background:
* def all = [ABC,DEF,GHI,JKL]
Scenario: Extract response value
Given url
When method get
Then status 200
And def value = []
And eval for(var i=0;i<all.length;i++) {value.add(response['#(all[i])']["execution"]["status"]) }
And print value
I want to extract the value of all parameters whose status is "0".
print response["ABC"]["execution"]["status"]
The above line gives the result but I want to parameterise the ["ABC"] part
Any help on this? Is there any other way I can achieve this or am I doing something wrong to achieve this particular edge case?
I'm providing a sample below that answers multiple questions:
* def response =
"""
{
"ABC": {
"code": "ABC",
"isActive": "true",
"lastUpdatedBy": "username",
"execution": {
"status": "0",
},
"priority": "1"
},
"DEF": {
"code": "DEF",
"isActive": "true",
"lastUpdatedBy": "username",
"execution": {
"status": "1",
},
"priority": "1"
}
}
"""
# get all values in root json
* def items = $.*
* def code = 'ABC'
* def found = items.filter(x => x.code == code && x.execution.status == '0')
* assert found.length == 1
# the expression on the right below is pure JS
* match found[0] == response[code]
Note:
how to use json-path to simplify JSON, also refer: https://stackoverflow.com/a/68811696/143475
how to use the filter() API to "find" data using complex expressions, that can be parameterized, also see: https://github.com/karatelabs/karate#json-transforms
how to get the value of a key that can be dynamic and parameterized

How would you reduce a json value

I have a Graphql returning values that look like this
"{\"date\":\"2020-05-21\",\"time\":null,\"changed_at\":\"2020-05-25T16:16:33.201Z\"}"
How would you target just the date to return "2020-05-21"?
Here is a larger picture of the retuned.
"column_values": [
{
"value": null,
"id": "check8",
"title": "Approved?"
},
{
"value": "{\"date\":\"2020-05-21\",\"time\":null,\"changed_at\":\"2020-05-25T16:16:33.201Z\"}",
"id": "due_date2",
"title": "Due Date"
},
{
"value": null,
"id": "qtr",
"title": "QTR"
},
{
"value": null,
"id": "status5",
"title": "Priority"
},
{
"value": "{\"index\":2,\"post_id\":null,\"changed_at\":\"2020-05-22T17:56:59.936Z\"}",
"id": "status",
"title": "Status"
},
{
"value": null,
"id": "progress",
"title": "Progress"
},
{
"value": null,
"id": "link",
"title": "Video Link"
},
{
"value": null,
"id": "formula8",
"title": "DATE_API"
}
]
}
Do you have to parse the JSON or Stringify it first? perhaps Graphql isn't returning usable json. Thank you for any help or bump in the correct direction.
JSON.stringify() is the method used to generate a JSON string. If you apply it to something that's already a JSON string then you'll get a double-encoded JSON string.
What you need to use is the JSON.parse() method:
const jsonObject= '{"foo": "bar"}';
const decoded = JSON.parse(jsonObject);
console.log(decode, typeof decoded);
Result:
{ foo: 'bar' } 'object'

unencode json in Zapier

I'm trying to make a Zapier zap to get a JSON from Olark.
This is the sample JSON Olark sends...
{
"kind": "Conversation",
"id": "EV695BI2930A6XMO32886MPT899443414",
"tags": ["olark", "customer"],
"items": [{
"kind": "MessageToVisitor",
"nickname": "John",
"timestamp": "1307116657.1",
"body": "Hi there. Need any help?",
"operatorId": "1234"
},
{
"kind": "MessageToOperator",
"nickname": "Bob",
"timestamp": "1307116661.25",
"body": "Yes, please help me with billing."
}],
"visitor": {
"kind": "Visitor",
"id": "9QRF9YWM5XW3ZSU7P9CGWRU89944341",
"fullName": "Bob Doe",
"emailAddress": "bob#example.com",
"phoneNumber": "(555) 555-5555",
"city": "Palo Alto",
"region": "CA",
"country": "United State",
"countryCode": "US",
"organization": "Widgets Inc.",
"ip": "123.4.56.78",
"browser": "Chrome 12.1",
"operatingSystem": "Windows",
"conversationBeginPage": "http://www.example.com/path",
"customFields": {
"myInternalCustomerId": "12341234",
"favoriteColor": "blue"
},
"chat_feedback": {
"comments": "Very helpful, thanks",
"friendliness": 5,
"knowledge": 5,
"overall_chat": 5,
"responsiveness": 5
}
},
"operators": {
"1234": {
"kind": "Operator",
"id": "1234",
"username": "jdoe",
"nickname": "John",
"emailAddress": "john#example.com"
}
},
"groups": [{
"name": "My Sales Group",
"id": "0123456789abcdef",
"kind": "Group"
}]
}
I can get what I want -- the email -- like this...
var obj = {};
var data = {};
data = JSON.parse(inputData.data);
obj.value = data.visitor["emailAddress"];
return obj;
However, the live data that comes from Olark is an encoded version of the JSON. It looks like this...
raw_body
data=%7B%22kind%22%3A+%22Conversation%22%2C+%22id%22%3A+%224pkhSGlkBYHz0gw83L6TF0UBa6rA39Bo%22%2C+%22manuallySubmitted%22%3A+false%2C+%22items%22%3A+%5B%7B%22kind%22%3A+%22MessageToOperator%22%2C+%22nickname%22%3A+%22juliachevron%40gmail.com%22%2C+%22timestamp%22%3A+%221588965389.388434%22%2C+%22body%22%3A+%22Hi.+I+have+a+stimulator+and+it+is+no+longer+working+at+the+higher+levels%22%2C+%22visitor_nickname%22%3A+%22juliachevron%40gmail.com%22%7D%2C+%7B%22kind%22%3A+%22MessageToVisitor%22%2C+%22nickname%22%3A+%22Christian%22%2C+%22operatorId%22%3A+%22663710%22%2C+%22timestamp%22%3A+%221588965397.901964%22%2C+%22body%22%3A+%22Hi+there%22%7D%2C+%7B%22kind%22%3A+%22MessageToVisitor%22%2C+%22nickname%22%3A+%22Christian%22%2C+%22operatorId%22%3A+%22663710%22%2C+%22timestamp%22%3A+%221588965408.398821%22%2C+%22body%22%3A+%22Can+you+explain+more%3F%22%7D%2C+%7B%22kind%22%3A+%22MessageToOperator%22%2C+%22nickname%22%3A+%22juliachevron%40gmail.com%22%2C+%22timestamp%22%3A+%221588965445.279921%22%2C+%22body%22%3A+%22it+is+only+flashing+up+to+the+2+and+sometimes+3%22%2C+%22visitor_nickname%22%3A+%22juliachevron%40gmail.com%22%7D%2C+%7B%22kind%22%3A+%22MessageToVisitor%22%2C+%22nickname%22%3A+%22Christian%22%2C+%22operatorId%22%3A+%22663710%22%2C+%22timestamp%22%3A+%221588965482.985280%22%2C+%22body%22%3A+%22Try+our+%5C%22oreo%5C%22+troubleshooting+test+from+page+12+of+the+User+Guide%22%7D%2C+%7B%22kind%22%3A+%22MessageToVisitor%22%2C+%22nickname%22%3A+%22Christian%22%2C+%22operatorId%22%3A+%22663710%22%2C+%22timestamp%22%3A+%221588965488.741729%22%2C+%22body%22%3A+%22It+involves+you+pressing+the+wet+sponges+into+each+other+as+you+turn+the+device+to+the+max+setting%22%7D%2C+%7B%22kind%22%3A+%22MessageToOperator%22%2C+%22nickname%22%3A+%22juliachevron%40gmail.com%22%2C+%22timestamp%22%3A+%221588965539.836595%22%2C+%22body%22%3A+%22thank+you%21%22%2C+%22visitor_nickname%22%3A+%22juliachevron%40gmail.com%22%7D%2C+%7B%22kind%22%3A+%22MessageToVisitor%22%2C+%22nickname%22%3A+%22Christian%22%2C+%22operatorId%22%3A+%22663710%22%2C+%22timestamp%22%3A+%221588965583.359044%22%2C+%22body%22%3A+%22This+test+is+to+make+sure+the+device+is+functioning+correctly%22%7D%2C+%7B%22kind%22%3A+%22MessageToVisitor%22%2C+%22nickname%22%3A+%22Christian%22%2C+%22operatorId%22%3A+%22663710%22%2C+%22timestamp%22%3A+%221588965619.075057%22%2C+%22body%22%3A+%22If+you+are+not+getting+to+the+higher+levels+when+the+sponges+are+on+your+temples%2C+it+may+mean+that+you+need+to+use+more+water%2C+tighten+the+headband+a+little+more%2C+or+that+you+need+to+replace+the+sponges%22%7D%2C+%7B%22kind%22%3A+%22MessageToOperator%22%2C+%22nickname%22%3A+%22juliachevron%40gmail.com%22%2C+%22timestamp%22%3A+%221588965698.050212%22%2C+%22body%22%3A+%22it+worked.+thanks%22%2C+%22visitor_nickname%22%3A+%22juliachevron%40gmail.com%22%7D%2C+%7B%22kind%22%3A+%22MessageToVisitor%22%2C+%22nickname%22%3A+%22Christian%22%2C+%22operatorId%22%3A+%22663710%22%2C+%22timestamp%22%3A+%221588965702.713077%22%2C+%22body%22%3A+%22You%27re+very+welcome.%22%7D%5D%2C+%22tags%22%3A+%5B%5D%2C+%22visitor%22%3A+%7B%22kind%22%3A+%22Visitor%22%2C+%22id%22%3A+%22YN7k5dd6nGH52Lvd3L6TF0VBb03B3roI%22%2C+%22fullName%22%3A+%22juliachevron%40gmail.com%22%2C+%22emailAddress%22%3A+%22juliachevron%40gmail.com%22%2C+%22ip%22%3A+%22%22%2C+%22city%22%3A+%22Oak+Park%22%2C+%22region%22%3A+%22IL%22%2C+%22country%22%3A+%22United+States%22%2C+%22countryCode%22%3A+%22US%22%2C+%22organization%22%3A+%22Comcast+Cable%22%2C+%22browser%22%3A+%22Chrome+80.0.3987.149%22%2C+%22operatingSystem%22%3A+%22Macintosh%22%2C+%22referrer%22%3A+%22https%3A%2F%2Fwww.google.com%2F%22%2C+%22conversationBeginPage%22%3A+%22https%3A%2F%2Fwww.fisherwallace.com%2F%22%2C+%22chat_feedback%22%3A+%7B%7D%7D%2C+%22operators%22%3A+%7B%22663710%22%3A+%7B%22kind%22%3A+%22Operator%22%2C+%22id%22%3A+%22663710%22%2C+%22nickname%22%3A+%22Christian%22%2C+%22emailAddress%22%3A+%22christian%40fisherwallace.com%22%2C+%22username%22%3A+%22fisherwallace%22%7D%7D%7D
And I get "SyntaxError: Unexpected token d in JSON at position 0" when I try to parse it with the code that works on the sample JSON.
Is there a way in Zapier to unencode the JSON?
There sure is! What you're seeing is percent encoding. Node.js can decode that, but to get valid json, you'll also need to replace the + characters with spaces.
Try this:
const inputWithReplacedSpaces = inputData.data.replace(/\+/g, '%20') // '%20' is a space
const jsonStr = decodeURIComponent(inputWithReplacedSpaces)
const data = JSON.parse(jsonStr)
return { email: data.visitor.emailAddress }
There were two issues. See screenshot.
I needed to set the Import Data to "data" in the Custom Run Javascript step. And also remove the "data=" from the string at the beginning of the Raw Body.

lodash sort an array of objects by a property which has an array of objects

I have a an object. I am able to sort the items by using lodash's _.orderBy().
However, in one of the scenario I have to sort by subject, which is an array of objects. Items inside the subject array are already sorted based on the name.
As subject is an array of the objects, I need to consider the first item for sorting.
[
{
"id": "1",
"name": "peter",
"subject": [
{
"id": "1",
"name": "maths"
},
{
"id": "2",
"name": "social"
}
]
},
{
"id": "2",
"name": "david",
"subject": [
{
"id": "2",
"name": "physics"
},
{
"id": "3",
"name": "science"
}
]
},
{
"id": "3",
"name": "Justin",
"subject": [
]
}
]
You can use _.get() to extract the name (or id) of the 1st item in subjects. If no item exists, _.get() will return undefined, which can be replaced with a default value. In this case, we don't want to use an empty string as a default value, since the order would change. Instead I'm checking if the value is a string, if it is I use lower case on it, if not I return it as is.
const arr = [{"id":"1","name":"peter","subject":[{"id":"1","name":"maths"},{"id":"2","name":"social"}]},{"id":"2","name":"david","subject":[{"id":"2","name":"physics"},{"id":"3","name":"science"}]},{"id":"3","name":"Justin","subject":[]}]
const result = _.orderBy(arr, o => {
const name = _.get(o, 'subject[0].name')
return _.isString(name) ? name.toLowerCase() : name
})
console.log(result)
<script src="https://cdnjs.cloudflare.com/ajax/libs/lodash.js/4.17.11/lodash.js"></script>
Use _.sortBy with a comparison/sorting function argument. Your function itself can look into the receiving arguments subject key (I think its the subject you want to compare?)
Since you have the question also tagged with ES6 here is an JS only solution via Array.sort:
let arr = [ { "id": "1", "name": "peter", "subject": [ { "id": "1", "name": "maths" }, { "id": "2", "name": "social" } ] }, { "id": "2", "name": "david", "subject": [ { "id": "2", "name": "physics" }, { "id": "3", "name": "science" } ] }, { "id": "3", "name": "Justin", "subject": [] }, ]
const result = arr.sort((a,b) =>
a.subject.length && b.subject.length
? a.subject[0].name.localeCompare(b.subject[0].name)
: a.subject.length ? -1 : 1)
console.log(result)

How do I parse empty dictionary JSON in Swift?

I am parsing JSON using swift. I came to a dictionary which returns empty. How would I go about parsing this into a custom object?
** I am looking at Image**
},
"name": "Al bake",
"image": {},
"website": "",
"category": {
"id": "d59d2b1c-ca37-4c76-bb93-1c0ba967ee84",
"name": "Test Category",
"slug": "test-category",
"is_active": true
},
If you need the value for image, which should be a dictionary (empty or containing keys & values), you need to know the type that contains it.
For instance, let's assume that the following code is contained in a dictionary:
"name": "Al bake",
"image": {},
"website": "",
"category": {
"id": "d59d2b1c-ca37-4c76-bb93-1c0ba967ee84",
"name": "Test Category",
"slug": "test-category",
"is_active": true
You would do something like this:
if let image = dictionary["image"] as? Dictionary<String, Any> {
print(image.description)
} else {
print("Image does not contain a value")
}