I ran this awscli command and and im trying to get the value of the 'State' key using --query:
ws emr describe-step --profile dev --cluster-id j-12345678 --step-id s-12345678abc
output:
{
"Step": {
"Status": {
"Timeline": {
"EndDateTime": 123456.632,
"CreationDateTime": 1234566.779,
"StartDateTime": 1623122991201544.588
},
"State": "COMPLETED",
"StateChangeReason": {}
},
"Config": {
"Args": [
"s3-dist-cp",
"--s3Endpoint",
"s3.amazonaws.com",
"--src",
"s3://test-bucket/",
"--dest",
"s3://test-bucket-2"
],
"Jar": "command-runner.jar",
"Properties": {}
},
"Id": "s-123456789",
"ActionOnFailure": "CONTINUE",
"Name": "Command Runner"
}
}
From the output I would like to extract the the value from the 'State' Key. If thats possible can jq be used instead?
I don't have EMR to test against but using your example this should work:
aws emr describe-step --profile dev --cluster-id j-12345678 --step-id s-12345678abc --query 'Step.Status.State'
This is an example using ec2 where I know --query works:
aws ec2 describe-instances --query 'Reservations[].Instances[].InstanceId'
To use jq you can do something like this:
aws emr describe-step --profile dev --cluster-id j-12345678 --step-id s-12345678abc |jq -r '.Step.Status.State'
References
CLI Usage Filter
When I try to push a custom metric to the Oracle Cloud Monitoring service using the Oracle Cloud CLI, I receive the following error:
ServiceError:
{
"code": "NotAuthorizedOrNotFound",
"message": "Authorization failed or requested resource not found.",
"opc-request-id": "request id",
"status": 404
}
This occurs when using the Administrator account and when using an instance principal which has monitoring permission.
Here is the JSON that I am pushing to the Monitoring service:
[
{
"namespace": "myFirstNamespace",
"compartmentId": "tenant id",
"resourceGroup": "myFirstResourceGroup",
"name": "successRate",
"dimensions": {
"resourceId": "ocid1.exampleresource.region1.phx.exampleuniqueID",
"appName": "myAppA"
},
"metadata": {
"unit": "percent",
"displayName": "MyAppA Success Rate"
},
"datapoints": [
{
"timestamp": "2021-06-01T22:19:20Z",
"value": 83.0
}
]
}
]
The CLI command that I am using is:
oci monitoring metric-data post --metric-data file://metric-data.json
The OCI CLI command should be:
oci monitoring metric-data post --metric-data file://metric-data.json --endpoint https://telemetry-ingestion.{{ region }}.oraclecloud.com
replacing {{ region }} with your region.
The --endpoint https://telemetry-ingestion.{{ region }}.oraclecloud.com parameter needs to be added.
Looks like some authorization issue. Please cross check if the instance principle has all the required permission assigned. Please review this document Publishing Custom Metrics and Overview of Monitoring
I'm currently working on an AWS course, and have been using the Windows CLI in order to setup demo infrastructure in my AWS VPC. I was attempting to create a role called "bastion-role" via using the following command:
aws iam create-role --role-name bastion-role --assume-role-policy-document file://role-policy.json
but it yields this error:
Error parsing parameter --assume-role-policy-document: Unable to
load paramfile file://role-policy.json: [Errno 2] No such file or
directory: 'role-policy.json'
I'm assuming it's having a problem referencing the local directory that I'm currently in. I've attempted the following troubleshooting measures:
Not using the file:// part and just the name 'role-policy.json'. This didn't work
Trying to reference the local directory: file://c:\role-policy-json
Tried a relative path: file://../role-policy.json
Tried using 3 forward slashes: file:///role-policy.json
Tried using 3 forward slashes and local directory: file:///c:\role-policy.json
Unfortunately, none of these combinations have worked. I'm assuming I'm missing something, but I can't figure it out. If someone could shed some light on this problem, I'd appreciate it.
Probably its too late to provide this answer, but the right way to execute this command is like this:
aws iam --region us-east-2 create-role --role-name <YOUR_ROLE_NAME> --assume-role-policy-document file://C:\Users\<USERNAME>\FULL\PATH\TO\file_containing_policy.json
After successful execution you should see something like this:
{
"Role": {
"Path": "/",
"RoleName": "YOUR_ROLE_NAME",
"RoleId": "AROXXXXXXXXXXXXXXXXXX",
"Arn": "arn:aws:iam::XXXXXXXXXXXX:role/YOUR_ROLE_NAME",
"CreateDate": "2020-11-17T04:53:35+00:00",
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"Service": "<SERVICE>.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
}
}
I am attempting to trigger a Lambda function when a CloudWatch event detects an EC2 Server that has shutdown. I need this function only to trigger when the EC2 instance contains a certain tag doing this via CloudWatch Events Rules. In this case, the tag I am using is called nameOfTheTag.
The below JSON is valid for CloudWatch but is not triggering for some reason
{
"source": [
"aws.ec2"
],
"detail-type": [
"EC2 Instance State-change Notification"
],
"detail": {
"state": [
"stopped"
],
"tags:nameOfTheTag": [
"example"
]
}
}
I don't think so this would work as you can't get EC2 with specific tags you only get EC2 Ids
I am trying to move an S3 bucket from one account (A) to another (B).
I have succeeded with that operation and remove the bucket from account A.
I am trying to move the new bucket from account B to another bucket on account B, but learning that beside the bucket itself I have no access to the files.
After much fighting with s3 cli and its permissions I checked s3api commands and found out that the files (surprise surprise) still holds the old ownership.
I am trying now to change it, but came to a stand still with the put-bucket-acl, the JSON file isn't working for s3api command.
I tried running the command in debug , but didn't make too much out of it.
Anybody knows what to do ?
Maybe a better way to solve this issue ?
what I did so far:
the command:
aws s3api put-bucket-acl --bucket my-bucket --cli-input-json file://1.json
(Same with put-object-acl)
1.json file:
"Grantee": {
"DisplayName": "account_B",
"EmailAddress": "user#mail.com",
"ID": "111111hughalphnumericnumber22222",
"Type": "CanonicalUser",
"Permission": "FULL_CONTROL"
}
The errors I get :
Unknown parameter in input: "Grantee", must be one of: ACL,
AccessControlPolicy, Bucket, ContentMD5, GrantFullControl, GrantRead,
GrantReadACP, GrantWrite, GrantWriteACP Unknown parameter in input:
"Permission", must be one of: ACL, AccessControlPolicy, Bucket,
ContentMD5, GrantFullControl, GrantRead, GrantReadACP, GrantWrite,
GrantWriteACP
UPDATE:
AssumeRole between the 2 accounts doesn't work in my case.
cli (s3cmd,s3api) GUI (MCSTools,bucketexplorer), ACL using headers,body (Postman) did not help as well..
I'm connecting AWS support and hoping for the best.
I'll update when I have a solution.
So, AWS support came to the rescue... I'm leaving this for others to see, so they won't have to waste 2 days like I did trying to figure what the hell went wrong...
aws s3api get-object-acl --bucket <bucket_on_B> --key <Key_on_B_Owned_by_A> --profile IAM_User_A > A_to_B.json
apply the outcome of:
aws s3api get-bucket-acl --bucket <Bucket_on_B> --profile IAM_User_B
onto the json file that was created, and then run
aws s3api put-object-acl --bucket <Bucket_on_B> --key <Key_on_B_Owned_by_A> --access-control-policy file://A_to_B.json --profile IAM_User_A
Your JSON is wrong. According to the documentation for the put-bucket-acl option you can generate valid JSON template ('skeleton') using --generate-cli-skeleton. For example:
aws s3api put-bucket-acl --bucket BUCKETNAME --generate-cli-skeleton
And here is the output:
{
"ACL": "",
"AccessControlPolicy": {
"Grants": [
{
"Grantee": {
"DisplayName": "",
"EmailAddress": "",
"ID": "",
"Type": "",
"URI": ""
},
"Permission": ""
}
],
"Owner": {
"DisplayName": "",
"ID": ""
}
},
"Bucket": "",
"ContentMD5": "",
"GrantFullControl": "",
"GrantRead": "",
"GrantReadACP": "",
"GrantWrite": "",
"GrantWriteACP": ""
}
For anyone who's still looking to do this - OP probably looked at the right aws doc but overlooked the right command. I'm just glad I got to right command because of this stackoverflow page :)
https://docs.aws.amazon.com/cli/latest/reference/s3api/put-bucket-acl.html
^^ The json syntax with example is present there and instead of --cli-input-json , use --access-control-policy
{
"Grants": [
{
"Grantee": {
"DisplayName": "string",
"EmailAddress": "string",
"ID": "string",
"Type": "CanonicalUser"|"AmazonCustomerByEmail"|"Group",
"URI": "string"
},
"Permission": "FULL_CONTROL"|"WRITE"|"WRITE_ACP"|"READ"|"READ_ACP"
}
...
],
"Owner": {
"DisplayName": "string",
"ID": "string"
}
}
I had the policy as a json file and used this command it worked just fine.
aws s3api put-bucket-acl --bucket bucketname --access-control-policy file://yourJson.json
Also one more thing to note is that I wasn't able to add permissions along with existing ones, old acl was being overwritten. So any permission you want to add needs to be in json policy file along with existing policy. It will be easier when you use some command to describe all the ACLs first.
The syntax is the following (with example):
aws s3api put-bucket-acl --bucket bucket_name --access-control-policy file://grant.json
grant.json file:
{
"Grants": [
{
"Grantee": {
"ID": "CANONICAL_ID_TO_GRANT",
"Type": "CanonicalUser"
},
"Permission": "WRITE"
},
{
"Grantee": {
"ID": "CANONICAL_ID_TO_GRANT",
"Type": "CanonicalUser"
},
"Permission": "READ"
}
],
"Owner": {
"DisplayName": "example_owner",
"ID": "CANONICAL_ID_OWNER"
}
}