Ruby API call to get data from complex json - json

Question

I'm making an API GET call using Ruby - the call is made to a Learning Management System and returns the following JSON:
{
"id": 12345,
"body": null,
"url": null,
"grade": "75",
"score": 75,
"submitted_at": "2020-05-02T11:30:53Z",
"assignment_id": 9876,
"user_id": 1111,
"submission_type": "online_upload",
"workflow_state": "graded",
"grade_matches_current_submission": true,
"graded_at": "2017-06-05T08:47:49Z",
"grader_id": 2222,
"attempt": 1,
"cached_due_date": "2020-05-03T15:00:00Z",
"excused": false,
"late_policy_status": null,
"points_deducted": null,
"grading_period_id": null,
"late": false,
"missing": false,
"seconds_late": 0,
"entered_grade": "75",
"entered_score": 75,
"preview_url": "https://etcetc",
"turnitin_data": {
"attachment_33333": {
"status": "scored",
"object_id": "44444444",
"similarity_score": 0,
"web_overlap": 0,
"publication_overlap": 0,
"student_overlap": 0,
"state": "none"
}
},
"attachments": [
{
"id": 33333,
"uuid": "kjsdkjhsdfkhsfd",
"folder_id": 55555,
"display_name": "Submission.pdf",
"filename": "Submission.pdf",
"content-type": "application/pdf",
"url": "https://etcetc",
"size": 2668226,
"created_at": "2020-05-02T11:30:51Z",
"updated_at": "2020-06-06T15:01:46Z",
"unlock_at": null,
"locked": false,
"hidden": false,
"lock_at": null,
"hidden_for_user": false,
"thumbnail_url": null,
"modified_at": "2020-05-02T11:30:51Z",
"mime_class": "pdf",
"media_entry_id": null,
"locked_for_user": false,
"preview_url": "api/etcetc"
}
],
"submission_comments": [
{
"id": 99999,
"comment": "here’s a comment",
"author_id": 1,
"author_name": "Mickey Mouse",
"created_at": "2020-05-15T12:54:08Z",
"edited_at": null,
"avatar_path": "/images/users/1",
"author": {
"id": 1,
"display_name": " Mickey Mouse ",
"avatar_image_url": "https://etcetc",
"html_url": "https://etcetc"
}
},
{
"id": 223344,
"comment": "another comment",
"author_id": 2,
"author_name": "Donald Duck",
"created_at": "2020-06-05T10:48:51Z",
"edited_at": null,
"avatar_path": "/images/users/2",
"author": {
"id": 2,
"display_name": "Donald Duck",
"avatar_image_url": "https://etcetc",
"html_url": "https://etcetc"
}
}
]
}
I need to be able to retrieve specific values from "submission_comments", namely the values for "comment", "author_id" and "author_name". At the moment the best I can do is retrieve "submission_comments" as one big entity. Here's how I'm getting that far:
require 'typhoeus'
require 'link_header'
require 'json'
require 'csv'
the_url = 'https://etctetc'
token = 'mytoken'
api_endpoint = '/api/etc'
output_csv = 'C:\Users\me\Desktop\Ruby Canvas course\assignment_comments.csv'
CSV.open(output_csv, 'wb') do |csv|
csv << ["user_id", "TII", "marker"]
end
request_url = "#{the_url}#{api_endpoint}"
count = 0
more_data = true
while more_data
get_comments = Typhoeus::Request.new(
request_url,
method: :get,
headers: { authorization: "Bearer #{token}" }
)
get_comments.on_complete do |response|
#get next link
links = LinkHeader.parse(response.headers['link']).links
next_link = links.find { |link| link['rel'] == 'next' }
request_url = next_link.href if next_link
if next_link && "#{response.body}" != "[]"
more_data = true
else
more_data = false
end
if response.code == 200
data = JSON.parse(response.body)
data.each do |comments|
CSV.open(output_csv, 'a') do |csv|
csv << [comments['id'], comments['turnitin_data'], comments['submission_comments']]
end
end
else
puts "Something went wrong! Response code was #{response.code}"
end
end
get_comments.run
end
puts "Script done running"
I'm new to this (the ruby code is based on an exercise so I may not fully understand it)- any help/advice would be really appreciated!
EDIT: I should also note that this isn't the total JSON response I'm dealing with - this is just one of ten items that are returned

Answer 1

"submission_comments": [
{
"id": 99999,
}
]
the [] means it is array. {} means it is an object.
So you probably need to do something like this:
json["submission_comments"].first["id"]
or better iterate through it:
ids = json["submission_comments"].map{|comment| comment["id"]}

Answer 2

I'm able to get the variables you need if you can read the JSON file in as text, then use Ruby's JSON.parse(...) method on it. I think the main problem is that JSON uses null but Ruby hashes use nil. You could do a string replace or try something like this (I did not modify your JSON, only put it into a single quoted string):
json_text = '{
"id": 12345,
"body": null,
"url": null,
"grade": "75",
"score": 75,
"submitted_at": "2020-05-02T11:30:53Z",
"assignment_id": 9876,
"user_id": 1111,
"submission_type": "online_upload",
"workflow_state": "graded",
"grade_matches_current_submission": true,
"graded_at": "2017-06-05T08:47:49Z",
"grader_id": 2222,
"attempt": 1,
"cached_due_date": "2020-05-03T15:00:00Z",
"excused": false,
"late_policy_status": null,
"points_deducted": null,
"grading_period_id": null,
"late": false,
"missing": false,
"seconds_late": 0,
"entered_grade": "75",
"entered_score": 75,
"preview_url": "https://etcetc",
"turnitin_data": {
"attachment_33333": {
"status": "scored",
"object_id": "44444444",
"similarity_score": 0,
"web_overlap": 0,
"publication_overlap": 0,
"student_overlap": 0,
"state": "none"
}
},
"attachments": [
{
"id": 33333,
"uuid": "kjsdkjhsdfkhsfd",
"folder_id": 55555,
"display_name": "Submission.pdf",
"filename": "Submission.pdf",
"content-type": "application/pdf",
"url": "https://etcetc",
"size": 2668226,
"created_at": "2020-05-02T11:30:51Z",
"updated_at": "2020-06-06T15:01:46Z",
"unlock_at": null,
"locked": false,
"hidden": false,
"lock_at": null,
"hidden_for_user": false,
"thumbnail_url": null,
"modified_at": "2020-05-02T11:30:51Z",
"mime_class": "pdf",
"media_entry_id": null,
"locked_for_user": false,
"preview_url": "api/etcetc"
}
],
"submission_comments": [
{
"id": 99999,
"comment": "here’s a comment",
"author_id": 1,
"author_name": "Mickey Mouse",
"created_at": "2020-05-15T12:54:08Z",
"edited_at": null,
"avatar_path": "/images/users/1",
"author": {
"id": 1,
"display_name": " Mickey Mouse ",
"avatar_image_url": "https://etcetc",
"html_url": "https://etcetc"
}
},
{
"id": 223344,
"comment": "another comment",
"author_id": 2,
"author_name": "Donald Duck",
"created_at": "2020-06-05T10:48:51Z",
"edited_at": null,
"avatar_path": "/images/users/2",
"author": {
"id": 2,
"display_name": "Donald Duck",
"avatar_image_url": "https://etcetc",
"html_url": "https://etcetc"
}
}
]
}'
Part I added:
ruby_hash = JSON.parse(json_text)
submission_comments = ruby_hash["submission_comments"]
submission_comments.each do |submission_comment|
comment = submission_comment["comment"]
author_id = submission_comment["author_id"]
author_name = submission_comment["author_name"]
puts "Comment: #{comment}, Author ID: #{author_id}, Author Name: #{author_name}\n\n"
end
Terminal Result:
=> Comment: here’s a comment, Author ID: 1, Author Name: Mickey Mouse
=> Comment: another comment, Author ID: 2, Author Name: Donald Duck
Edit: I added a jenky af one-liner version just for fun (presuming the json_text variable above is already initialized)
JSON.parse(json_text)["submission_comments"]
.map{|txt| puts(["comment","author_id","author_name"]
.map{|k| k.instance_eval{"#{upcase}: #{txt[to_s]}"}}.join(', '))}
COMMENT: here’s a comment, AUTHOR_ID: 1, AUTHOR_NAME: Mickey Mouse
COMMENT: another comment, AUTHOR_ID: 2, AUTHOR_NAME: Donald Duck