How to join two values to print in one line using jq - json

I have this list that prints out using the gh api command:
[
{
"login": "anthonyd-cd",
"id": 35614549,
"node_id": "MDQ6VXNlcjM1NjE0NTQ5",
"avatar_url": "https://avatars.githubusercontent.com/u/35614549?v=4",
"gravatar_id": "",
"url": "https://api.github.com/users/anthonyd-cd",
"html_url": "https://github.com/anthonyd-cd",
"followers_url": "https://api.github.com/users/anthonyd-cd/followers",
"following_url": "https://api.github.com/users/anthonyd-cd/following{/other_user}",
"gists_url": "https://api.github.com/users/anthonyd-cd/gists{/gist_id}",
"starred_url": "https://api.github.com/users/anthonyd-cd/starred{/owner}{/repo}",
"subscriptions_url": "https://api.github.com/users/anthonyd-cd/subscriptions",
"organizations_url": "https://api.github.com/users/anthonyd-cd/orgs",
"repos_url": "https://api.github.com/users/anthonyd-cd/repos",
"events_url": "https://api.github.com/users/anthonyd-cd/events{/privacy}",
"received_events_url": "https://api.github.com/users/anthonyd-cd/received_events",
"type": "User",
"site_admin": false,
"contributions": 5
},
{
"login": "elJoeyJojo",
"id": 17091968,
"node_id": "MDQ6VXNlcjE3MDkxOTY4",
"avatar_url": "https://avatars.githubusercontent.com/u/17091968?v=4",
"gravatar_id": "",
"url": "https://api.github.com/users/elJoeyJojo",
"html_url": "https://github.com/elJoeyJojo",
"followers_url": "https://api.github.com/users/elJoeyJojo/followers",
"following_url": "https://api.github.com/users/elJoeyJojo/following{/other_user}",
"gists_url": "https://api.github.com/users/elJoeyJojo/gists{/gist_id}",
"starred_url": "https://api.github.com/users/elJoeyJojo/starred{/owner}{/repo}",
"subscriptions_url": "https://api.github.com/users/elJoeyJojo/subscriptions",
"organizations_url": "https://api.github.com/users/elJoeyJojo/orgs",
"repos_url": "https://api.github.com/users/elJoeyJojo/repos",
"events_url": "https://api.github.com/users/elJoeyJojo/events{/privacy}",
"received_events_url": "https://api.github.com/users/elJoeyJojo/received_events",
"type": "User",
"site_admin": false,
"contributions": 2
}
]
I am trying to use jq to print out only the login and contributions field but into one line like so:
gh api repos/eljoeyjojo/diner-app/contributors | jq -r '.[].login, .[].contributions'
Expected Result
anthony-cd 5
eljoeyjojo 2
Actual Result
anthony-cd
eljoeyjojo
5
2
Any idea how I could print/merge these two fields? Would I have to use a for loop in this case?
Thanks

You are using the iterator .[] twice, so each field is extracted from a different iteration. Pull the iterator up front and extract from the context of an iteration step. Then you can use various methods to combine those, one being string interpolation "\(...)".
jq -r '.[] | "\(.login) \(.contributions)"'
anthonyd-cd 5
elJoeyJojo 2
Demo

Related

How can I get the total build time of a GitHub action workflow?

Is there any way to get the total build time of a GitHub action workflow? I did not find anything related to this in GitHub API and GraphQL.
What you are looking for appears to be in the github documentation here
Here's an example from that documentation
curl \
-H "Accept: application/vnd.github.v3+json" \
https://api.github.com/repos/octocat/hello-world/actions/runs/42/timing
Note that the run number is not the sequentially generated number but what appears at the end of the url when you got to the instance of a run
You can use the Get a workflow run service (Github API) to extract all datas from a specific workflow based on its workflow_id.
There is also a service to get all workflow runs from a specific repository.
Both will return a similar output (one on a list, and the other with a workflow object) similar to this one:
{
"id": 30433642,
"name": "Build",
"node_id": "MDEyOldvcmtmbG93IFJ1bjI2OTI4OQ==",
"check_suite_id": 42,
"check_suite_node_id": "MDEwOkNoZWNrU3VpdGU0Mg==",
"head_branch": "master",
"head_sha": "acb5820ced9479c074f688cc328bf03f341a511d",
"run_number": 562,
"event": "push",
"status": "queued",
"conclusion": null,
"workflow_id": 159038,
"url": "https://api.github.com/repos/octo-org/octo-repo/actions/runs/30433642",
"html_url": "https://github.com/octo-org/octo-repo/actions/runs/30433642",
"pull_requests": [],
"created_at": "2020-01-22T19:33:08Z",
"updated_at": "2020-01-22T19:33:08Z",
"jobs_url": "https://api.github.com/repos/octo-org/octo-repo/actions/runs/30433642/jobs",
"logs_url": "https://api.github.com/repos/octo-org/octo-repo/actions/runs/30433642/logs",
"check_suite_url": "https://api.github.com/repos/octo-org/octo-repo/check-suites/414944374",
"artifacts_url": "https://api.github.com/repos/octo-org/octo-repo/actions/runs/30433642/artifacts",
"cancel_url": "https://api.github.com/repos/octo-org/octo-repo/actions/runs/30433642/cancel",
"rerun_url": "https://api.github.com/repos/octo-org/octo-repo/actions/runs/30433642/rerun",
"workflow_url": "https://api.github.com/repos/octo-org/octo-repo/actions/workflows/159038",
"head_commit": {
"id": "acb5820ced9479c074f688cc328bf03f341a511d",
"tree_id": "d23f6eedb1e1b9610bbc754ddb5197bfe7271223",
"message": "Create linter.yaml",
"timestamp": "2020-01-22T19:33:05Z",
"author": {
"name": "Octo Cat",
"email": "octocat#github.com"
},
"committer": {
"name": "GitHub",
"email": "noreply#github.com"
}
},
"repository": {
"id": 1296269,
"node_id": "MDEwOlJlcG9zaXRvcnkxMjk2MjY5",
"name": "Hello-World",
"full_name": "octocat/Hello-World",
"owner": {
"login": "octocat",
"id": 1,
"node_id": "MDQ6VXNlcjE=",
"avatar_url": "https://github.com/images/error/octocat_happy.gif",
"gravatar_id": "",
"url": "https://api.github.com/users/octocat",
"html_url": "https://github.com/octocat",
"followers_url": "https://api.github.com/users/octocat/followers",
"following_url": "https://api.github.com/users/octocat/following{/other_user}",
"gists_url": "https://api.github.com/users/octocat/gists{/gist_id}",
"starred_url": "https://api.github.com/users/octocat/starred{/owner}{/repo}",
"subscriptions_url": "https://api.github.com/users/octocat/subscriptions",
"organizations_url": "https://api.github.com/users/octocat/orgs",
"repos_url": "https://api.github.com/users/octocat/repos",
"events_url": "https://api.github.com/users/octocat/events{/privacy}",
"received_events_url": "https://api.github.com/users/octocat/received_events",
"type": "User",
"site_admin": false
},
"private": false,
"html_url": "https://github.com/octocat/Hello-World",
"description": "This your first repo!",
"fork": false,
"url": "https://api.github.com/repos/octocat/Hello-World",
"archive_url": "https://api.github.com/repos/octocat/Hello-World/{archive_format}{/ref}",
"assignees_url": "https://api.github.com/repos/octocat/Hello-World/assignees{/user}",
"blobs_url": "https://api.github.com/repos/octocat/Hello-World/git/blobs{/sha}",
"branches_url": "https://api.github.com/repos/octocat/Hello-World/branches{/branch}",
"collaborators_url": "https://api.github.com/repos/octocat/Hello-World/collaborators{/collaborator}",
"comments_url": "https://api.github.com/repos/octocat/Hello-World/comments{/number}",
"commits_url": "https://api.github.com/repos/octocat/Hello-World/commits{/sha}",
"compare_url": "https://api.github.com/repos/octocat/Hello-World/compare/{base}...{head}",
"contents_url": "https://api.github.com/repos/octocat/Hello-World/contents/{+path}",
"contributors_url": "https://api.github.com/repos/octocat/Hello-World/contributors",
"deployments_url": "https://api.github.com/repos/octocat/Hello-World/deployments",
"downloads_url": "https://api.github.com/repos/octocat/Hello-World/downloads",
"events_url": "https://api.github.com/repos/octocat/Hello-World/events",
"forks_url": "https://api.github.com/repos/octocat/Hello-World/forks",
"git_commits_url": "https://api.github.com/repos/octocat/Hello-World/git/commits{/sha}",
"git_refs_url": "https://api.github.com/repos/octocat/Hello-World/git/refs{/sha}",
"git_tags_url": "https://api.github.com/repos/octocat/Hello-World/git/tags{/sha}",
"git_url": "git:github.com/octocat/Hello-World.git",
"issue_comment_url": "https://api.github.com/repos/octocat/Hello-World/issues/comments{/number}",
"issue_events_url": "https://api.github.com/repos/octocat/Hello-World/issues/events{/number}",
"issues_url": "https://api.github.com/repos/octocat/Hello-World/issues{/number}",
"keys_url": "https://api.github.com/repos/octocat/Hello-World/keys{/key_id}",
"labels_url": "https://api.github.com/repos/octocat/Hello-World/labels{/name}",
"languages_url": "https://api.github.com/repos/octocat/Hello-World/languages",
"merges_url": "https://api.github.com/repos/octocat/Hello-World/merges",
"milestones_url": "https://api.github.com/repos/octocat/Hello-World/milestones{/number}",
"notifications_url": "https://api.github.com/repos/octocat/Hello-World/notifications{?since,all,participating}",
"pulls_url": "https://api.github.com/repos/octocat/Hello-World/pulls{/number}",
"releases_url": "https://api.github.com/repos/octocat/Hello-World/releases{/id}",
"ssh_url": "git#github.com:octocat/Hello-World.git",
"stargazers_url": "https://api.github.com/repos/octocat/Hello-World/stargazers",
"statuses_url": "https://api.github.com/repos/octocat/Hello-World/statuses/{sha}",
"subscribers_url": "https://api.github.com/repos/octocat/Hello-World/subscribers",
"subscription_url": "https://api.github.com/repos/octocat/Hello-World/subscription",
"tags_url": "https://api.github.com/repos/octocat/Hello-World/tags",
"teams_url": "https://api.github.com/repos/octocat/Hello-World/teams",
"trees_url": "https://api.github.com/repos/octocat/Hello-World/git/trees{/sha}",
"hooks_url": "http://api.github.com/repos/octocat/Hello-World/hooks"
},
"head_repository": {
"id": 217723378,
"node_id": "MDEwOlJlcG9zaXRvcnkyMTc3MjMzNzg=",
"name": "octo-repo",
"full_name": "octo-org/octo-repo",
"private": true,
"owner": {
"login": "octocat",
"id": 1,
"node_id": "MDQ6VXNlcjE=",
"avatar_url": "https://github.com/images/error/octocat_happy.gif",
"gravatar_id": "",
"url": "https://api.github.com/users/octocat",
"html_url": "https://github.com/octocat",
"followers_url": "https://api.github.com/users/octocat/followers",
"following_url": "https://api.github.com/users/octocat/following{/other_user}",
"gists_url": "https://api.github.com/users/octocat/gists{/gist_id}",
"starred_url": "https://api.github.com/users/octocat/starred{/owner}{/repo}",
"subscriptions_url": "https://api.github.com/users/octocat/subscriptions",
"organizations_url": "https://api.github.com/users/octocat/orgs",
"repos_url": "https://api.github.com/users/octocat/repos",
"events_url": "https://api.github.com/users/octocat/events{/privacy}",
"received_events_url": "https://api.github.com/users/octocat/received_events",
"type": "User",
"site_admin": false
},
"html_url": "https://github.com/octo-org/octo-repo",
"description": null,
"fork": false,
"url": "https://api.github.com/repos/octo-org/octo-repo",
"forks_url": "https://api.github.com/repos/octo-org/octo-repo/forks",
"keys_url": "https://api.github.com/repos/octo-org/octo-repo/keys{/key_id}",
"collaborators_url": "https://api.github.com/repos/octo-org/octo-repo/collaborators{/collaborator}",
"teams_url": "https://api.github.com/repos/octo-org/octo-repo/teams",
"hooks_url": "https://api.github.com/repos/octo-org/octo-repo/hooks",
"issue_events_url": "https://api.github.com/repos/octo-org/octo-repo/issues/events{/number}",
"events_url": "https://api.github.com/repos/octo-org/octo-repo/events",
"assignees_url": "https://api.github.com/repos/octo-org/octo-repo/assignees{/user}",
"branches_url": "https://api.github.com/repos/octo-org/octo-repo/branches{/branch}",
"tags_url": "https://api.github.com/repos/octo-org/octo-repo/tags",
"blobs_url": "https://api.github.com/repos/octo-org/octo-repo/git/blobs{/sha}",
"git_tags_url": "https://api.github.com/repos/octo-org/octo-repo/git/tags{/sha}",
"git_refs_url": "https://api.github.com/repos/octo-org/octo-repo/git/refs{/sha}",
"trees_url": "https://api.github.com/repos/octo-org/octo-repo/git/trees{/sha}",
"statuses_url": "https://api.github.com/repos/octo-org/octo-repo/statuses/{sha}",
"languages_url": "https://api.github.com/repos/octo-org/octo-repo/languages",
"stargazers_url": "https://api.github.com/repos/octo-org/octo-repo/stargazers",
"contributors_url": "https://api.github.com/repos/octo-org/octo-repo/contributors",
"subscribers_url": "https://api.github.com/repos/octo-org/octo-repo/subscribers",
"subscription_url": "https://api.github.com/repos/octo-org/octo-repo/subscription",
"commits_url": "https://api.github.com/repos/octo-org/octo-repo/commits{/sha}",
"git_commits_url": "https://api.github.com/repos/octo-org/octo-repo/git/commits{/sha}",
"comments_url": "https://api.github.com/repos/octo-org/octo-repo/comments{/number}",
"issue_comment_url": "https://api.github.com/repos/octo-org/octo-repo/issues/comments{/number}",
"contents_url": "https://api.github.com/repos/octo-org/octo-repo/contents/{+path}",
"compare_url": "https://api.github.com/repos/octo-org/octo-repo/compare/{base}...{head}",
"merges_url": "https://api.github.com/repos/octo-org/octo-repo/merges",
"archive_url": "https://api.github.com/repos/octo-org/octo-repo/{archive_format}{/ref}",
"downloads_url": "https://api.github.com/repos/octo-org/octo-repo/downloads",
"issues_url": "https://api.github.com/repos/octo-org/octo-repo/issues{/number}",
"pulls_url": "https://api.github.com/repos/octo-org/octo-repo/pulls{/number}",
"milestones_url": "https://api.github.com/repos/octo-org/octo-repo/milestones{/number}",
"notifications_url": "https://api.github.com/repos/octo-org/octo-repo/notifications{?since,all,participating}",
"labels_url": "https://api.github.com/repos/octo-org/octo-repo/labels{/name}",
"releases_url": "https://api.github.com/repos/octo-org/octo-repo/releases{/id}",
"deployments_url": "https://api.github.com/repos/octo-org/octo-repo/deployments"
}
}
In this response, there are 2 fields - created_at and updated_at - that can be used to obtain the duration of the workflow execution.
As you'll see on the documentation, others services are available to get interesting datas about a workflow as well, such as logs, review history, run usage (duration for each runner) ...

jq - looping through json object and only displaying certain values

I am currently learning how to usage jq. I have a json object and I am able to loop and read values with jq as such cat test.json | jq -r '.test'. However, I am running into some complexity when I only want to display on values that have outdated or deprecated = true but ommit from result if relase = never-update-app or never-upgrade-app. I am trying cat test.json | jq '.test | select(.[].outdated==true)' but this is not returning the desired results. Is this something possible through jq and have it display in the below desired format?
My desired output is shown below:
Release Name Installed Latest Old Deprecated
test-app 1.0.0 2.0.0 true false
json:
{
"test": [{
"release": "myapp1",
"Installed": {
"version": "0.3.0",
"appVersion": "v1.2.6"
},
"Latest": {
"version": "",
"appVersion": ""
},
"outdated": false,
"deprecated": false
}, {
"release": "myapp2",
"Installed": {
"version": "6.5.13",
"appVersion": "1.9.1"
},
"Latest": {
"version": "",
"appVersion": ""
},
"outdated": false,
"deprecated": false
}, {
"release": "test-app",
"Installed": {
"version": "1.0.0",
"appVersion": ""
},
"Latest": {
"version": "2.0.0",
"appVersion": ""
},
"outdated": true,
"deprecated": false
}, {
"release": "never-update-app",
"Installed": {
"version": "1.0.0",
"appVersion": ""
},
"Latest": {
"version": "3.0.0",
"appVersion": ""
},
"outdated": true,
"deprecated": false
}, {
"release": "never-upgrade-app",
"Installed": {
"version": "2.0.0",
"appVersion": ""
},
"Latest": {
"version": "2.0.0",
"appVersion": ""
},
"outdated": false,
"deprecated": true
}]
}
How to act on each element in an array
You have an array in .test. If you want to extract the individual elements of that array to use in the rest of your filter, use .test[]. If you want to process them but keep the results in an array, use .test|map(...) where ... is the filter you want to apply to the elements of the array.
How to filter objects with multiple conditions
As described in this question you can use the select() filter with multiple conditions. So, for example:
jq '.test[]|
select(
(.outdated or .deprecated) and
.release != "never-update-app" and
.release != "never-upgrade-app"
)'
How to combine non-trivial filters
Use parentheses! As an example, suppose you want to replace the two "never-" tests with a startswith test. By surrounding the pipes with parentheses you isolated the calculation and just get the true/false answer to combine in a boolean expression.
jq '.test[]|
select(
(.outdated or .deprecated) and
(.release|startswith("never-")|not)
)'
How to format results in a table
I think this answer on formatting results in a table is probably sufficient. If you have more specific needs I think you might want to spin this off into a separate question.
Filter
(["ReleaseName","Installed","Latest","Old","Deprecated"] | #tsv), (.test[] | select( (.outdated==true or .deprecated==true) and ((.release=="never-update-app" or .release=="never-upgrade-app") | not) ) | "\(.release) \(.Installed.version) \(.Latest.version) \(.outdated) \(.deprecated)" / " " | #tsv)
Output
ReleaseName Installed Latest Old Deprecated
test-app 1.0.0 2.0.0 true false
Demo
https://jqplay.org/s/B4a5cxlL28

How can I parse nested JSON in PowerShell?

I'm trying to parse the results of a cURL command and the information I need is in a structure.
I tried getting to the data unsuccessfully and tried converting to PS Object but not sure how to access the structure as I'm new to PS.
Below is a sample of our cURL response.
I have a git commit hash ('c64a568399a572e82c223d55cb650b87ea1c22b8' matches latestCommit in fromRef for entry id 1101) and I need to find the corresponding displayId ('develop' in toRef)
I've done this in Linux using jq but need to replicate this in PS.
jq '.values | map(select(.fromRef.latestCommit=="'"$HASH"'")) | .[0].toRef.displayId'
I'm having 2 issues.
I can get to fromRef but it looks like #{id=refs/heads/feature/add-support; displayId=feature/add-support; latestCommit=c64a568399a572e82c223d55cb650b87ea1c22b8; repository=} and I cannot figure out how to parse
I'm not sure how to get the id so I can find the correct corresponding toRef
Any help would be greatly appreciated.
{
"size": 15,
"limit": 20,
"isLastPage": true,
"values": [
{
"id": 1101,
"version": 0,
"title": "Added header",
"description": "Added notes in header",
"state": "OPEN",
"open": true,
"closed": false,
"createdDate": 1595161367863,
"updatedDate": 1595161367863,
"fromRef": "#{id=refs/heads/feature/add-support; displayId=feature/add-support; latestCommit=c64a568399a572e82c223d55cb650b87ea1c22b8; repository=}",
"toRef": "#{id=refs/heads/develop; displayId=develop; latestCommit=58b3e3482bb35f3a735048849c2474cc676fbd9b; repository=}",
"locked": false,
"author": "#{user=; role=AUTHOR; approved=False; status=UNAPPROVED}",
"reviewers": " ",
"participants": "",
"properties": "#{mergeResult=; resolvedTaskCount=0; openTaskCount=0}",
"links": "#{self=System.Object[]}"
},
{
"id": 1053,
"version": 4,
"title": "Help with checking,",
"description": "fixed up code.",
"state": "OPEN",
"open": true,
"closed": false,
"createdDate": 1591826401310,
"updatedDate": 1595018917357,
"fromRef": "#{id=refs/heads/bugfix/checking-2.7; displayId=bugfix/checking-2.7; latestCommit=cf7d8860262c6a46b0b65ef5b6d66ae8cd698b75; repository=}",
"toRef": "#{id=refs/heads/hotfix/2.7_Improvements; displayId=hotfix/2.7_Improvements; latestCommit=01f1100c559ba41ec317421399c3bfb9a0aea91f; repository=}",
"locked": false,
"author": "#{user=; role=AUTHOR; approved=False; status=UNAPPROVED}",
"reviewers": " ",
"participants": "",
"properties": "#{mergeResult=; resolvedTaskCount=0; commentCount=4; openTaskCount=0}",
"links": "#{self=System.Object[]}"
}
],
"start": 0
}
Once you have converted the result with ConvertTo-Json and the correct -Depth parameter, you can get the values of the returned object quite easily in PowerShell.
Let's say you have used something like $json = $curlResult | ConvertTo-Json -Depth 100, then finding the displayId from the corresponding toRef can be done like this:
# this is the known hashvalue of the `fromRef` value to look for
$latestCommitHash = "c64a568399a572e82c223d55cb650b87ea1c22b8"
# get the value item. from here you can get all other properties belonging to that item
$valueItem = $json.values | Where-Object { $_.fromRef.latestCommit -eq $latestCommitHash }
# get the displayId value of the corresponding 'toRef' element:
$displayId = $valueItem.toRef.displayId
Returns
develop

JQ if then statement scope

I'd like to use JQ to grab only the sub-records that match an if-then statement. When I use
jq 'if .services[].banner == "FQMDAAICCg==" then .services[].port else empty end
it grabs all of the ports for the record. (there are multiple services under each record and I want to restrict my then statement to only the services scope where I actually found the if condition).
How do I just get the port, banner, etc. for the specific service underneath the record which hit my condition?
example:
{
"services": [
{
"tls_detected": false,
"banner_is_raw": true,
"transport_protocol": "tcp",
"banner": "PCFET0NUWVBFIEhU",
"certificate": null,
"timestamp": "2020-03-22T00:38:01.074Z",
"protocol": null,
"port": 4444
},
{
"tls_detected": false,
"banner_is_raw": true,
"transport_protocol": "tcp",
"banner": "SFRUUC8xLjEgMzA",
"certificate": null,
"timestamp": "2020-03-19T01:39:45.288Z",
"protocol": null,
"port": 8080
},
{
"tls_detected": false,
"banner_is_raw": true,
"transport_protocol": "tcp",
"banner": "FQMDAAICCg==",
"certificate": null,
"timestamp": "2020-03-19T01:39:45.288Z",
"protocol": null,
"port": 8085
},
{
"tls_detected": false,
"banner_is_raw": false,
"transport_protocol": "tcp",
"banner": "Q2FjaGUtQ29ud",
"certificate": null,
"timestamp": "2020-03-20T04:25:24Z",
"protocol": "http",
"port": 8080
}
],
"ip": "103.238.62.68",
"autonomous_system": {
"description": "CHAPTECH-AS-AP Chaptech Pty Ltd",
"asn": 133493,
"routed_prefix": "103.238.62.0/24",
"country_code": "AU",
"name": "CHAPTECH-AS-AP Chaptech Pty Ltd",
"path": [
11164,
3491,
63956,
7594,
7594,
7594,
7594,
133493
]
},
"location": {
"country_code": "AU",
"registered_country": "Australia",
"registered_country_code": "AU",
"continent": "Oceania",
"timezone": "Australia/Sydney",
"latitude": -33.494,
"longitude": 143.2104,
"country": "Australia"
}
}
Update:
Thanks to peak but I couldn't get the additional goals bit working below. I ended up using
jq 'select(.services[].banner == "FQMDAAICCg==") | {port: .services[].port, banner: .services[].banner, ip: .ip}' censys.json | jq 'if .banner == "FQMDAAICCg==" then .ip,.port else empty end'
which is ugly but did the trick and still allowed me to stream the data to the first filter.
Original question
How do I just get the port, banner, etc. for the specific service underneath the record which hit my condition?
To get just the "port" for the service matching the condition, you could modify your query:
.services[]
| if .banner == "FQMDAAICCg==" then .port else empty end
Equivalently:
.services[]
| select(.banner == "FQMDAAICCg==")
| .port
Additional goal
I want to end up in this example with '8085' + '103.238.62.68'
If you really want the two values in that format, you could write something along the following lines, invoking jq with the -r option:
.ip as $ip
| (.services[] | select(.banner == "FQMDAAICCg==") | .port) as $port
| "'\($port)' + '\($ip)'"
or more briefly but less readably:
"'\(.services[] | select(.banner == "FQMDAAICCg==") | .port)' + '\(.ip)'"

Unable to extract specific value of json file using sed in shell script

I am using sed command for extracting a value which I get from the output after doing curl. Below is my shell script.
#!/bin/bash
release=$(curl -H "Authorization:token xxxxxxxxxxxxxxxxxxxxxx" -H "Content-Type: application/json" https://api.github.com/repos/octocat/Hello-World/releases/tags/v1.0.0)
id=$(echo "$release" |sed -n -e 's/"id":\\ \\([0-9]\\+\\),/\\1/p' | head -n 1 | sed 's/[[:blank:]]//g')
echo $id
The output I got after curl is a json file. Below is the output
{
"url": "https://api.github.com/repos/octocat/Hello-World/releases/1",
"html_url": "https://github.com/octocat/Hello-World/releases/v1.0.0",
"assets_url": "https://api.github.com/repos/octocat/Hello-
World/releases/1/assets",
"upload_url": "https://uploads.github.com/repos/octocat/Hello-
World/releases/1/assets{?name,label}",
"tarball_url": "https://api.github.com/repos/octocat/Hello-
World/tarball/v1.0.0",
"zipball_url": "https://api.github.com/repos/octocat/Hello-
World/zipball/v1.0.0",
"id": 1234332,
"node_id": "MDc6UmVsZWFzZTE=",
"tag_name": "v1.0.0",
"target_commitish": "master",
"name": "v1.0.0",
"body": "Description of the release",
"draft": false,
"prerelease": false,
"created_at": "2013-02-27T19:35:32Z",
"published_at": "2013-02-27T19:35:32Z",
"author": {
"login": "octocat",
"id": 1353322,
"node_id": "MDQ6VXNlcjE=",
"avatar_url": "https://github.com/images/error/octocat_happy.gif",
"gravatar_id": "",
"url": "https://api.github.com/users/octocat",
"html_url": "https://github.com/octocat",
"followers_url": "https://api.github.com/users/octocat/followers",
"following_url":
"https://api.github.com/users/octocat/following{/other_user}",
"gists_url": "https://api.github.com/users/octocat/gists{/gist_id}",
"starred_url": "https://api.github.com/users/octocat/starred{/owner}
{/repo}",
"subscriptions_url":
"https://api.github.com/users/octocat/subscriptions",
"organizations_url": "https://api.github.com/users/octocat/orgs",
"repos_url": "https://api.github.com/users/octocat/repos",
"events_url": "https://api.github.com/users/octocat/events{/privacy}",
"received_events_url":
"https://api.github.com/users/octocat/received_events",
"type": "User",
"site_admin": false
},
"assets": [
{
"url": "https://api.github.com/repos/octocat/Hello-
World/releases/assets/1",
"browser_download_url": "https://github.com/octocat/Hello-
World/releases/download/v1.0.0/example.zip",
"id": 56432211,
"node_id": "MDEyOlJlbGVhc2VBc3NldDE=",
"name": "example.zip",
"label": "short description",
"state": "uploaded",
"content_type": "application/zip",
"size": 1024,
"download_count": 42,
"created_at": "2013-02-27T19:35:32Z",
"updated_at": "2013-02-27T19:35:32Z",
"uploader": {
"login": "octocat",
"id": 5663322,
"node_id": "MDQ6VXNlcjE=",
"avatar_url": "https://github.com/images/error/octocat_happy.gif",
"gravatar_id": "",
"url": "https://api.github.com/users/octocat",
"html_url": "https://github.com/octocat",
"followers_url": "https://api.github.com/users/octocat/followers",
"following_url":
"https://api.github.com/users/octocat/following{/other_user}",
"gists_url": "https://api.github.com/users/octocat/gists{/gist_id}",
"starred_url": "https://api.github.com/users/octocat/starred{/owner}
{/repo}",
"subscriptions_url":
"https://api.github.com/users/octocat/subscriptions",
"organizations_url": "https://api.github.com/users/octocat/orgs",
"repos_url": "https://api.github.com/users/octocat/repos",
"events_url":
"https://api.github.com/users/octocat/events{/privacy}",
"received_events_url":
"https://api.github.com/users/octocat/received_events",
"type": "User",
"site_admin": false
}
}
]
}
The sed command I posted above must work as it was working for another curl commands.
There are multiple occurrence of the id value but I want the value of the first ID to be stored in the id variable which is "id": 1234332
But as output I am getting nothing.
Need help on this.
I corrected the sed command and it worked.
The double backslashes \\ where escaping the command so I changed it to single
backslash \
Below is the modified shell script.
release=$(curl -H "Authorization:token xxxxxxxxxxxxxxxxxxxxxx" -H "Content-Type: application/json" https://api.github.com/repos/octocat/Hello-World/releases/tags/v1.0.0)
id=$(echo "$release" | sed -n -e 's/"id":\ \([0-9]\+\),/\1/p' | head -n 1 | sed 's/[[:blank:]]//g')
echo $id
Don't use double quotes instead of single ones unless you want anything in the quoted string to be expanded,
Use jq for parsing JSON values, sed is a line-oriented tool and can't parse JSON syntax-wise.
E.g:
id=$(curl -H 'Authorization:token xxxxxxxxxxxxxxxxxxxxxx' -H 'Content-Type: application/json' 'https://api.github.com/repos/octocat/Hello-World/releases/tags/v1.0.0' |
jq -r '.id')
echo "$id"
The double backslashes match a literal backslash, that is why your code did not work.
You may try this grep:
id=$(grep -o -m 1 '"id": [0-9]\{1,\},' <<< "$release" | sed 's/"id": //' | sed 's/,//')
-m 1 will get the first occurrence only (check Grep only the first match and stop), -o will get the regex match text - "id": + 1 or mor digits and a comma, and sed 's/"id": //' | sed 's/,//' will remove "id": and the comma from the return text.