Related
I am practising Scala and Akka Streams on Stripe-payment API. I am looking to extract the transaction date: created from the Json response.
This is the response (showing 2 transactions):
{
"object": "list",
"data": [
{
"id": "txn_1Fqdyl2eZvKYlo2CXfUAnz1z",
"object": "balance_transaction",
"amount": 10000,
"available_on": 1577145600,
"created": 1576581159,
"currency": "usd",
"description": null,
"exchange_rate": null,
"fee": 320,
"fee_details": [
{
"amount": 320,
"application": null,
"currency": "usd",
"description": "Stripe processing fees",
"type": "stripe_fee"
}
],
"net": 9680,
"source": "ch_1Fqdyk2eZvKYlo2CwjSNI1vO",
"status": "available",
"type": "charge"
},
{
"id": "txn_1Fqdyk2eZvKYlo2C7MuBhLpe",
"object": "balance_transaction",
"amount": 2000,
"available_on": 1577145600,
"created": 1576581158,
"currency": "usd",
"description": "テスト支払い",
"exchange_rate": null,
"fee": 88,
"fee_details": [
{
"amount": 88,
"application": null,
"currency": "usd",
"description": "Stripe processing fees",
"type": "stripe_fee"
}
],
"net": 1912,
"source": "ch_1Fqdyk2eZvKYlo2Ccg96i1QQ",
"status": "available",
"type": "charge"
}
],
"has_more": true,
"url": "/v1/balance_transactions"
}
It can vary from 1 to 100 transactions. So far I have been able to extract the value I need when there is only 1 transaction. I need to be able to extract values when there are multiple results. At the moment the value is hardcoded as: case Some(JArray(List(JInt(int)))).
My code:
def epochToDate(epochMillis: BigInt): String = {
val convertedToLong = epochMillis * 1000L
val df:SimpleDateFormat = new SimpleDateFormat("dd-MM-yyyy HH:mm:ss")
df.format(convertedToLong)
}
val stripe = new Stream("txn_1Fqdyl2eZvKYlo2Cn0OSmwaY", "starting_after", hasMoreFromJson, 1, idFromJson)
stripe.asSource().runForeach(id => {
val rawValue = (parse(id) \ "data" \ "created").toOption
rawValue match {
case Some(JArray(List(JInt(int)))) => println("Date: " + epochToDate(int))
case _ => println("Value not present in JSON")
}
})
Try
val rawValue = (parse(id) \ "data" \\ "created").toOption
rawValue match {
case createdObjOpt: Option[JObject] =>
createdObjOpt match {
case Some(createdObj) =>
createdObj.obj.foreach { created =>
println(created)
}
case None => println("oh no!")
}
case _ => println("Value not present in JSON")
}
I have captured two sets of values by using JSON extractor in JMeter which I want to concatenate. Let me give you an example below for the format which I want to use.
The following are the two sets of captured values:
Set 1: [V2520 V2522 V2521 V2500 V2500]
Set 2: [PL PL PL NP NP]
So from the above sets, I am looking for the something like the following value, because the body which I have to send in a subsequent call contains the combination of these 2 values:
Answer: ["V2520PL", "V2522PL", "V2521PL", "V2500NP", "V2500NP"]
Can you please help me how to solve this in JMeter using Groovy?
This is the JSON I have:
{ "body": {
"responseObject": [
{
"benefitInfo": [
{
"procedureCode": "V2520",
"modifier": "PL",
"usage": "Dress",
"authorizationID": null,
"description": "ContactLensDisposable",
"id": "96",
"coPayAmount": "25"
},
{
"procedureCode": "V2522",
"modifier": "PL",
"usage": "Dress",
"authorizationID": null,
"description": "ContactLensDisposableBifocal",
"id": "98",
"coPayAmount": "25"
},
{
"procedureCode": "V2521",
"modifier": "PL",
"usage": "Dress",
"authorizationID": null,
"description": "ContactLensDisposableToric",
"id": "97",
"coPayAmount": "25"
},
{
"procedureCode": "V2500",
"modifier": "NP",
"usage": "Dress",
"authorizationID": null,
"description": "ContactLens (Non Plan)",
"id": "89",
"coPayAmount": "0"
},
{
"procedureCode": "V2500",
"modifier": "NP",
"usage": "Dress",
"authorizationID": null,
"description": "ContactLensConventional (Non Plan)",
"id": "157",
"coPayAmount": "0"
}
]
}
]}}
An easy way to do this is to combine them as you collect the values from the JSON when you parse it.
def json = new groovy.json.JsonSlurper().parseText(text)
def answer = json.body.responseObject[0].benefitInfo.collect { it.procedureCode + it.modifier }
assert answer == ["V2520PL", "V2522PL", "V2521PL", "V2500NP", "V2500NP"]
Another method would be to use transpose() and join():
def r = new groovy.json.JsonSlurper().parseText(text).body.responseObject.benefitInfo[0]
def answer = [r.procedureCode, r.modifier].transpose()*.join()
assert answer == ["V2520PL", "V2522PL", "V2521PL", "V2500NP", "V2500NP"]
Add JSR223 PostProcessor as a child of the request which returns the above JSON
Put the following code into "Script" area:
def answer = []
def benefitInfos = com.jayway.jsonpath.JsonPath.read(prev.getResponseDataAsString(), '$..benefitInfo')
benefitInfos.each { benefitInfo ->
benefitInfo.each { entry ->
answer.add(entry.get('procedureCode') + entry.get('modifier'))
}
}
vars.put('answer', new groovy.json.JsonBuilder(answer).toPrettyString())
That's it, you will be able to access generated value as ${answer} where required:
References:
Jayway JsonPath
Groovy: Parsing and producing JSON
Apache Groovy - Why and How You Should Use It
In the below json file I want to access "934934507945312256", "934934503604174848",.... and then the keys inside them.
But after using UNWIND clause I am unable to access the data of these keys(quote_count,reply_count,etc.) as these keys("934934507945312256" ,"934934503604174848",...) are randomly generated.
{
"934934507945312256": {
"quote_count": 0,
"reply_count": 0,
"hashtags": null,
"datetime": "2017-11-26 23:58:51",
"date": "2017-11-26",
"like_count": 0,
"verified": "False",
"sentiment": 0,
"author": "Judy💯The Resistance",
"location": "Hollywood, California USA🇺🇸",
"tid": "934934507945312256",
"retweet_count": 0,
"type": "retweet",
"media_list": null,
"quoted_source_id": null,
"url_list": null,
"tweet_text": "RT #kylegriffin1: Reminder: The Senate Judiciary Committee gave Jared Kushner a November 27 deadline to turn over the missing records… ",
"author_profile_image": "https://pbs.twimg.com/profi...",
"author_screen_name": "jgirl66",
"author_id": "23737528",
"lang": "en",
"keywords_processed_list": [
"reminder",
"senate judiciary committee",
"kushner november",
"deadline"
],
"retweet_source_id": "934872065471115264",
"mentions": [
"kylegriffin1"
],
"replyto_source_id": null
},
"934934503604174848": {
"quote_count": 0,
"reply_count": 2,
"hashtags": [
"MissUniverse",
"Thailand"
],
"datetime": "2017-11-26 23:58:50",
"date": "2017-11-26",
"like_count": 38,
"verified": "False",
"sentiment": 0,
"author": "P'Hmee7.5",
"location": "Bangkok, Thailand",
"tid": "934934503604174848",
"retweet_count": 105,
"type": "Tweet",
"media_list": null,
"quoted_source_id": null,
"url_list": null,
"tweet_text": "รอโหวต มรญ #MissUniverse #Thailand",
"author_profile_image": "
Thumbnail
",
"author_screen_name": "Peehmee75",
"author_id": "700720806972624897",
"lang": "th",
"keywords_processed_list": null,
"retweet_source_id": null,
"mentions": null,
"replyto_source_id": null
},
"934934336381636608": {
"quote_count": 0,
"reply_count": 0,
"hashtags": null,
"datetime": "2017-11-26 23:58:10",
"date": "2017-11-26",
"like_count": 0,
"verified": "False",
"sentiment": 0,
"author": "selfresqingprncess",
"location": "Maine, USA",
"tid": "934934336381636608",
"retweet_count": 0,
"type": "retweet",
"media_list": null,
"quoted_source_id": null,
"url_list": null,
"tweet_text": "RT #kylegriffin1: Reminder: The Senate Judiciary Committee gave Jared Kushner a November 27 deadline to turn over the missing records… ",
"author_profile_image": "https://pbs.twimg.com/profi...",
"author_screen_name": "slfresqngprncss",
"author_id": "100536014",
"lang": "en",
"keywords_processed_list": [
"reminder",
"keywords_processed_list": [
"reminder",
"senate judiciary committee",
"kushner november",
"deadline"
],
"retweet_source_id": "934872065471115264",
"mentions": [
"kylegriffin1"
],
"replyto_source_id": null
}
}
There's I have tried :-
query = """
with {json} as data UNWIND data as doc FOREACH( l in doc| MERGE (label1:Label1 {author:l.author}))
"""
But I am getting error:- Cannot merge node using null property value for author.
Your JSON file is malformed.
A JSON string cannot contain control characters (like carriage returns or linefeeds). And your JSON contains an extraneous section (that is probably a copy/paste error):
"keywords_processed_list": [
"reminder",
Furthermore, UNWIND can only be used with lists. You cannot use it to get the properties from a map. Once you have fixed your JSON errors, try the following query (I assume that json is a parameter whose value is your JSON data):
UNWIND KEYS($json) AS k
MERGE(label1:Label1 {author: $json[k].author})
I'm making an API GET call using Ruby - the call is made to a Learning Management System and returns the following JSON:
{
"id": 12345,
"body": null,
"url": null,
"grade": "75",
"score": 75,
"submitted_at": "2020-05-02T11:30:53Z",
"assignment_id": 9876,
"user_id": 1111,
"submission_type": "online_upload",
"workflow_state": "graded",
"grade_matches_current_submission": true,
"graded_at": "2017-06-05T08:47:49Z",
"grader_id": 2222,
"attempt": 1,
"cached_due_date": "2020-05-03T15:00:00Z",
"excused": false,
"late_policy_status": null,
"points_deducted": null,
"grading_period_id": null,
"late": false,
"missing": false,
"seconds_late": 0,
"entered_grade": "75",
"entered_score": 75,
"preview_url": "https://etcetc",
"turnitin_data": {
"attachment_33333": {
"status": "scored",
"object_id": "44444444",
"similarity_score": 0,
"web_overlap": 0,
"publication_overlap": 0,
"student_overlap": 0,
"state": "none"
}
},
"attachments": [
{
"id": 33333,
"uuid": "kjsdkjhsdfkhsfd",
"folder_id": 55555,
"display_name": "Submission.pdf",
"filename": "Submission.pdf",
"content-type": "application/pdf",
"url": "https://etcetc",
"size": 2668226,
"created_at": "2020-05-02T11:30:51Z",
"updated_at": "2020-06-06T15:01:46Z",
"unlock_at": null,
"locked": false,
"hidden": false,
"lock_at": null,
"hidden_for_user": false,
"thumbnail_url": null,
"modified_at": "2020-05-02T11:30:51Z",
"mime_class": "pdf",
"media_entry_id": null,
"locked_for_user": false,
"preview_url": "api/etcetc"
}
],
"submission_comments": [
{
"id": 99999,
"comment": "here’s a comment",
"author_id": 1,
"author_name": "Mickey Mouse",
"created_at": "2020-05-15T12:54:08Z",
"edited_at": null,
"avatar_path": "/images/users/1",
"author": {
"id": 1,
"display_name": " Mickey Mouse ",
"avatar_image_url": "https://etcetc",
"html_url": "https://etcetc"
}
},
{
"id": 223344,
"comment": "another comment",
"author_id": 2,
"author_name": "Donald Duck",
"created_at": "2020-06-05T10:48:51Z",
"edited_at": null,
"avatar_path": "/images/users/2",
"author": {
"id": 2,
"display_name": "Donald Duck",
"avatar_image_url": "https://etcetc",
"html_url": "https://etcetc"
}
}
]
}
I need to be able to retrieve specific values from "submission_comments", namely the values for "comment", "author_id" and "author_name". At the moment the best I can do is retrieve "submission_comments" as one big entity. Here's how I'm getting that far:
require 'typhoeus'
require 'link_header'
require 'json'
require 'csv'
the_url = 'https://etctetc'
token = 'mytoken'
api_endpoint = '/api/etc'
output_csv = 'C:\Users\me\Desktop\Ruby Canvas course\assignment_comments.csv'
CSV.open(output_csv, 'wb') do |csv|
csv << ["user_id", "TII", "marker"]
end
request_url = "#{the_url}#{api_endpoint}"
count = 0
more_data = true
while more_data
get_comments = Typhoeus::Request.new(
request_url,
method: :get,
headers: { authorization: "Bearer #{token}" }
)
get_comments.on_complete do |response|
#get next link
links = LinkHeader.parse(response.headers['link']).links
next_link = links.find { |link| link['rel'] == 'next' }
request_url = next_link.href if next_link
if next_link && "#{response.body}" != "[]"
more_data = true
else
more_data = false
end
if response.code == 200
data = JSON.parse(response.body)
data.each do |comments|
CSV.open(output_csv, 'a') do |csv|
csv << [comments['id'], comments['turnitin_data'], comments['submission_comments']]
end
end
else
puts "Something went wrong! Response code was #{response.code}"
end
end
get_comments.run
end
puts "Script done running"
I'm new to this (the ruby code is based on an exercise so I may not fully understand it)- any help/advice would be really appreciated!
EDIT: I should also note that this isn't the total JSON response I'm dealing with - this is just one of ten items that are returned
"submission_comments": [
{
"id": 99999,
}
]
the [] means it is array. {} means it is an object.
So you probably need to do something like this:
json["submission_comments"].first["id"]
or better iterate through it:
ids = json["submission_comments"].map{|comment| comment["id"]}
I'm able to get the variables you need if you can read the JSON file in as text, then use Ruby's JSON.parse(...) method on it. I think the main problem is that JSON uses null but Ruby hashes use nil. You could do a string replace or try something like this (I did not modify your JSON, only put it into a single quoted string):
json_text = '{
"id": 12345,
"body": null,
"url": null,
"grade": "75",
"score": 75,
"submitted_at": "2020-05-02T11:30:53Z",
"assignment_id": 9876,
"user_id": 1111,
"submission_type": "online_upload",
"workflow_state": "graded",
"grade_matches_current_submission": true,
"graded_at": "2017-06-05T08:47:49Z",
"grader_id": 2222,
"attempt": 1,
"cached_due_date": "2020-05-03T15:00:00Z",
"excused": false,
"late_policy_status": null,
"points_deducted": null,
"grading_period_id": null,
"late": false,
"missing": false,
"seconds_late": 0,
"entered_grade": "75",
"entered_score": 75,
"preview_url": "https://etcetc",
"turnitin_data": {
"attachment_33333": {
"status": "scored",
"object_id": "44444444",
"similarity_score": 0,
"web_overlap": 0,
"publication_overlap": 0,
"student_overlap": 0,
"state": "none"
}
},
"attachments": [
{
"id": 33333,
"uuid": "kjsdkjhsdfkhsfd",
"folder_id": 55555,
"display_name": "Submission.pdf",
"filename": "Submission.pdf",
"content-type": "application/pdf",
"url": "https://etcetc",
"size": 2668226,
"created_at": "2020-05-02T11:30:51Z",
"updated_at": "2020-06-06T15:01:46Z",
"unlock_at": null,
"locked": false,
"hidden": false,
"lock_at": null,
"hidden_for_user": false,
"thumbnail_url": null,
"modified_at": "2020-05-02T11:30:51Z",
"mime_class": "pdf",
"media_entry_id": null,
"locked_for_user": false,
"preview_url": "api/etcetc"
}
],
"submission_comments": [
{
"id": 99999,
"comment": "here’s a comment",
"author_id": 1,
"author_name": "Mickey Mouse",
"created_at": "2020-05-15T12:54:08Z",
"edited_at": null,
"avatar_path": "/images/users/1",
"author": {
"id": 1,
"display_name": " Mickey Mouse ",
"avatar_image_url": "https://etcetc",
"html_url": "https://etcetc"
}
},
{
"id": 223344,
"comment": "another comment",
"author_id": 2,
"author_name": "Donald Duck",
"created_at": "2020-06-05T10:48:51Z",
"edited_at": null,
"avatar_path": "/images/users/2",
"author": {
"id": 2,
"display_name": "Donald Duck",
"avatar_image_url": "https://etcetc",
"html_url": "https://etcetc"
}
}
]
}'
Part I added:
ruby_hash = JSON.parse(json_text)
submission_comments = ruby_hash["submission_comments"]
submission_comments.each do |submission_comment|
comment = submission_comment["comment"]
author_id = submission_comment["author_id"]
author_name = submission_comment["author_name"]
puts "Comment: #{comment}, Author ID: #{author_id}, Author Name: #{author_name}\n\n"
end
Terminal Result:
=> Comment: here’s a comment, Author ID: 1, Author Name: Mickey Mouse
=> Comment: another comment, Author ID: 2, Author Name: Donald Duck
Edit: I added a jenky af one-liner version just for fun (presuming the json_text variable above is already initialized)
JSON.parse(json_text)["submission_comments"]
.map{|txt| puts(["comment","author_id","author_name"]
.map{|k| k.instance_eval{"#{upcase}: #{txt[to_s]}"}}.join(', '))}
COMMENT: here’s a comment, AUTHOR_ID: 1, AUTHOR_NAME: Mickey Mouse
COMMENT: another comment, AUTHOR_ID: 2, AUTHOR_NAME: Donald Duck
I've used pymongo to connect to mongo replica set and print the status of replica set using json dump. I want to parse this output and display "name" and "stateStr" into a list or array for the user to be able to pick a particular host.Here is my json dump output:
{
{
"replSetGetStatus": {
"date": "2016-10-07T14:21:25",
"members": [
{
"_id": 0,
"health": 1.0,
"name": "xxxxxxxxxxx:27017",
"optime": null,
"optimeDate": "2016-10-07T13:50:11",
"self": true,
"state": 1,
"stateStr": "PRIMARY",
"uptime": 32521
},
{
"_id": 1,
"health": 1.0,
"lastHeartbeat": "2016-10-07T14:21:24",
"lastHeartbeatRecv": "2016-10-07T14:21:24",
"name": "xxxxxxxxxxxx:27017",
"optime": null,
"optimeDate": "2016-10-07T13:50:11",
"pingMs": 0,
"state": 2,
"stateStr": "SECONDARY",
"syncingTo": "xxxxxxxxxxxx:27017",
"uptime": 27297
},
{
"_id": 2,
"health": 1.0,
"lastHeartbeat": "2016-10-07T14:21:24",
"lastHeartbeatRecv": "2016-10-07T14:21:24",
"name": "xxxxxxxxxxxxx:27020",
"pingMs": 0,
"state": 7,
"stateStr": "ARBITER",
"uptime": 32517
}
],
"myState": 1,
"ok": 1.0,
"set": "replica1"
}
}
Please try below Javascript code. It worked for me.
use admin;
var result = rs.status();
var length = result.members.length;
for (var i=0;i<length;i++){
print ("Server Name-" +result.members[i].name);
print ("Server State-" +result.members[i].stateStr);
}