Pass variable into Step Function start_execution() input parameter - json

I am using boto3 with Python 3.6 to start a Step Function execution. The Step Function is designed to share my base AMI across all my accounts. I have 4 variables I need to pass to the input parameter to kick off the execution. These are the AMI ID, the account list of accounts I own, the source account, and the KMS key. The AMI ID and the account list are constructed in my code and are the variables that need to get passed dynamically. According to the documentation, input is a string that contains the JSON input data for the execution and gives the following example: "input": "{\"ami_id\" : \"ami_id\"}". My question is how do I pass the variables in question to this parameter as the value? My code is below with the traceback:
CODE:
import boto3
import json
# Get an STS token to assume roles into AWS accounts
def get_sts_token(**kwargs):
role_arn = kwargs['RoleArn']
region_name = kwargs['RegionName']
sts = boto3.client(
'sts',
region_name=region_name,
)
token = sts.assume_role(
RoleArn=role_arn,
RoleSessionName='GetInstances',
DurationSeconds=900,
)
return token["Credentials"]
def get_accounts():
role_arn = "arn:aws:iam::xxxxxxxxxx:role/list-accounts-role"
region_name = "us-east-1"
token = get_sts_token(RoleArn=role_arn, RegionName=region_name)
access_key = token['AccessKeyId']
secret_key = token['SecretAccessKey']
session_token = token['SessionToken']
client = boto3.client('organizations',
aws_access_key_id=access_key,
aws_secret_access_key=secret_key,
aws_session_token=session_token)
moreAccounts=True
nexttoken=''
global accountList
accountList =[]
while moreAccounts:
if (len(nexttoken)>0):
accounts=client.list_accounts(NextToken=nexttoken)
else:
accounts=client.list_accounts()
if 'NextToken' in accounts:
nexttoken=accounts['NextToken']
else:
moreAccounts=False
for account in accounts['Accounts']:
if account['Status'] != 'SUSPENDED' and account['Status'] != 'CLOSED' :
account_id = account['Id']
accountList.append(account_id)
print(accountList)
def trigger_sfn():
ssm = boto3.client('ssm')
role_arn = "arn:aws:iam::xxxxxxxx:role/execute-sfn"
region_name = "us-east-1"
ami_id = ssm.get_parameter(Name='/BaseAMI/newest')['Parameter']['Value']
print(ami_id)
token = get_sts_token(RoleArn=role_arn, RegionName=region_name)
print(token)
access_key = token['AccessKeyId']
secret_key = token['SecretAccessKey']
session_token = token['SessionToken']
sfn = boto3.client('stepfunctions',
aws_access_key_id=access_key,
aws_secret_access_key=secret_key,
aws_session_token=session_token)
response = sfn.start_execution(
stateMachineArn='arn:aws:states:us-east-1:xxxxxxxx:stateMachine:ami-share',
input="{\"ami_id\": ami_id, \"source_account_id\": \"112233445566\", \"accountList\": accountList, \"kms_key_arn\": \"alias/aws/ebs\"}"
)
print(response)
TRACE:
An error occurred (InvalidExecutionInput) when calling the
StartExecution operation: Invalid State Machine Execution Input:
'com.fasterxml.jackson.core.JsonParseException: Unrecognized token
'ami_id': was expecting ('true', 'false' or 'null')
at [Source: (String)"{"ami_id": ami_id, "source_account_id":
"112233445566", "accountList": accountList, "kms_key_arn":
"alias/aws/ebs"}"; line: 1, column: 18]': InvalidExecutionInput
Traceback (most recent call last):
File "/var/task/lambda_function.py", line 6, in lambda_handler
trigger_sfn()
File "/var/task/lambda_function.py", line 96, in trigger_sfn
input="{\"ami_id\": ami_id, \"source_account_id\": \"112233445566\",
\"accountList\": accountList, \"kms_key_arn\": \"alias/aws/ebs\"}"
File "/var/runtime/botocore/client.py", line 314, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/var/runtime/botocore/client.py", line 612, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.errorfactory.InvalidExecutionInput: An error occurred
(InvalidExecutionInput) when calling the StartExecution operation:
Invalid State Machine Execution Input:
'com.fasterxml.jackson.core.JsonParseException: Unrecognized token
'ami_id': was expecting ('true', 'false' or 'null')
at [Source: (String)"{"ami_id": ami_id, "source_account_id":
"112233445566", "accountList": accountList, "kms_key_arn":
"alias/aws/ebs"}"; line: 1, column: 18]'

You can add the value of the variable to the string like this:
input="{\"ami_id\": \"" + ami_id + "\", \"source_account_id\": \"112233445566\", \"accountList\": accountList, \"kms_key_arn\": \"alias/aws/ebs\"}"
Your code currently sends the literal string "ami_id", instead of the variable value of ami_id

Related

809: unexpected token at '' (JSON parser error ruby3.0.4 mac os catalina

I have just started learning ruby and trying to learn extracting web data. I am using samplewebapp code given in a sites API documentation
On executing the test.rb file as given below it generates a token.
The token.json file contains the following
{"access_token":"eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJhdWQiOiIzY2U1NjFhMC01ZmE5LTQ5NjUtYTQ2Mi01NmI2MzEzNjRlZGMiLCJqdGkiOiJmMjc0ZTgwM2FmZjVlMTFiMTAzNzIwMzAwNzRlNDRhZDE3ODg3ZjAzOGQ2ODk0OWIwNzVkMmQ5N2E2MjAwYzc5ZWRhNzU4MmQ3MTg1MTc1YiIsImlhdCI6MTY1NDU2OTc0MS41MjgxNTUsIm5iZiI6MTY1NDU2OTc0MS41MjgxNTcsImV4cCI6MTY1NDU3MzM0MS41MjgwMzksInN1YiI6IjE5NGFlNTU5LTU5OWMtNDQ2ZC1hOTRhLWM2MTIyYjZkMTA2ZiIsInNjb3BlcyI6W10sImNyZWRpdHNfcmVtYWluaW5nIjo0NzAwLCJyYXRlX2xpbWl0cyI6W3sicmF0ZSI6NSwiaW50ZXJ2YWwiOjYwfV19.XoN5A-H8Z7d8p_0e2rCcyV4yWO9MAF3TZlHk5VP5NjUEqtTXTVYKfKuzidYDs7K8ZzAtWzyt3aR5VSUG0_nw-uPd7Z3_Y75alQMqa2b5rHGn5JFWH7nuAriskr26WSCqAdj4cNjccPORryRr5sYKwpE4Y4kTG0IX_8frvwYxwp-8wFpLLj98K3axwN7CCpWdnPDSzuMqmH6tSpF0XhdYxB5LTVH0AyH7lN0S0_Lftq0b3sOLIvEaTfNkuRGNqwLkBYFkHFuPqwrKd8RJhC2W1QZhrmUw3eYnh-0iQABGk2V0skIqDlb6BbQ5GFX6MXgiXAc-h5Ndda7pZ5N5UCQU3g","expires_at":1654573341}
However it also generates a JSON parser error as under
/Users/mm/.rbenv/versions/3.0.4/lib/ruby/3.0.0/json/common.rb:216:in `parse': 809: unexpected token at '' (JSON::ParserError)
from /Users/mm/.rbenv/versions/3.0.4/lib/ruby/3.0.0/json/common.rb:216:in `parse'
from /Users/mm/Desktop/prokerala/client.rb:21:in `parseResponse'
from /Users/mm/Desktop/prokerala/client.rb:99:in `get'
I googled around for a while and some of the responses talk about the issues between double and single quotes.. I tried changing those also but it doesn't work.
I have also tried running it with system ruby - which gives a different error
Would appreciate very much if someone could explain what the error is / logic behind this.. Also I am not clear about the error message itself. The message talks about parse error 809:"unexpected token" on column no 809 of the token there is nothing.. Am i reading it wrong?
code used in test.rb file
client = ApiClient.new('YOUR CLIENT_ID', 'YOUR CLIENT_SECRET');
result = client.get('v2/astrology/thirumana-porutham/advanced', {
:girl_nakshatra => 4,
:girl_nakshatra_pada => 2,
:boy_nakshatra => 26,
:boy_nakshatra_pada => 3
})
puts JSON.pretty_generate(result)
code used in client.rb file
require 'net/http'
require 'json'
class ApiError < StandardError
end
class ApiClient
BASE_URL = "https://api.prokerala.com/"
# Make sure that the following file path is set to a location that is not publicly accessible
TOKEN_FILE = "./token.json"
def initialize(clientId, clientSecret)
# Instance variables
#clientId = clientId
#clientSecret = clientSecret
end
def parseResponse(response)
content = response.body
res = JSON.parse(content)
if res.key?('access_token')
return res
end
if res['status'] == "error"
raise ApiError, res['errors'].map {|e| e['detail']}.join("\n")
end
if res['status'] != "ok"
raise "HTTP request failed"
end
return res
end
def saveToken(token)
# Cache the token until it expires.
File.open(ApiClient::TOKEN_FILE,"w") do |f|
token = {
:access_token => token['access_token'],
:expires_at => Time.now.to_i + token['expires_in']
}
f.write(token.to_json)
end
end
def getTokenFromCache
if not File.file?(ApiClient::TOKEN_FILE)
return nil
end
begin
# Fetch the cached token, and return if not expired
text = File.read(ApiClient::TOKEN_FILE)
token = JSON.parse(text)
if token['expires_at'] < Time.now.to_i
return nil
end
return token['access_token']
rescue JSON::ParserError
return nil
end
end
def fetchNewToken
params = {
:grant_type => 'client_credentials',
:client_id => #clientId,
:client_secret => #clientSecret
}
res = Net::HTTP.post_form(URI(ApiClient::BASE_URL + 'token'), params)
token = parseResponse(res)
saveToken(token)
return token['access_token']
end
def get(endpoint, params)
# Try to fetch the access token from cache
token = getTokenFromCache
# If failed, request new token
token ||= fetchNewToken
uri = URI(ApiClient::BASE_URL + endpoint)
uri.query = URI.encode_www_form(params)
req = Net::HTTP::Get.new(uri.to_s, {'Authorization' => 'Bearer ' + token})
res = Net::HTTP.start(uri.hostname) do |http|
http.request(req)
end
return parseResponse(res)
end
end
running the test file using ruby 3.0.4 gives the a JSON parse error. Full error reproduced as below
/Users/mm/.rbenv/versions/3.0.4/lib/ruby/3.0.0/json/common.rb:216:in `parse': 809: unexpected token at '' (JSON::ParserError)
from /Users/mm/.rbenv/versions/3.0.4/lib/ruby/3.0.0/json/common.rb:216:in `parse'
from /Users/mm/Desktop/prokerala/client.rb:21:in `parseResponse'
from /Users/mm/Desktop/prokerala/client.rb:99:in `get'
from test.rb:11:in `<main>'
I executed the above code and from this, we are getting
#<Net::HTTPMovedPermanently 301 Moved Permanently readbody=true> as response
and response body as empty string '', which results in error in parsing the body.
Since, API endpoint is secure with https, we need set use_ssl: true before starting the session. Try below code and it will fix the issue.
res = Net::HTTP.start(uri.hostname, use_ssl: true) do |http|
http.request(req)
end

AWS SAM CLI DEPLOYMENT - Issues Parsing Out of the Box secret = get_secret_value_response['SecretString']

I am very green when it comes to Python - so bare with me.
Using the out of the Box Python3 Sample Code provided by AWS to return the 'SecretString' from the AWS: Secrets Manager Service.
No issues there .. I get the returned object (note I have blanked out some details)
{"username":"postgres","password":"XXXXXXXXX","engine":"postgres","host":"srdataset.XXXXXXXXX.ap-southeast-2.rds.amazonaws.com","port":5432,"dbInstanceIdentifier":"srdataset"}
detail are all correct.
I am then using json.loads() to parse the above into my next function so I can extract the details like so
# request details
login_details = get_secret("pg_srdataset_login_details")
# load json
y = json.loads(login_details)
# extract result is a Python dictionary:
print(y["username"])
This again all works fine in my IDE (PyCharm). I can run the code, build in a Docker container .. and I then use the PyCharm AWS SAM CLI to deploy the code to the cloud .. no issues.
However when I test the function in AWS the code bugs out on the line y = json.loads(login_details) step.
the error being ..
{
"errorMessage": "Expecting value: line 1 column 1 (char 0)",
"errorType": "JSONDecodeError",
"stackTrace": [
" File \"/var/task/update_sp_changes.py\", line 229, in lambda_handler\n y = json.loads(login_details)\n",
" File \"/var/lang/lib/python3.8/json/__init__.py\", line 357, in loads\n return _default_decoder.decode(s)\n",
" File \"/var/lang/lib/python3.8/json/decoder.py\", line 337, in decode\n obj, end = self.raw_decode(s, idx=_w(s, 0).end())\n",
" File \"/var/lang/lib/python3.8/json/decoder.py\", line 355, in raw_decode\n raise JSONDecodeError(\"Expecting value\", s, err.value) from None\n"
]
}
To test this I also copied the JSON 'SecretString' returned from AWS, hard coded it as a variable, then passed this variable directly into the y = json.loads(login_details) step. Tested again and works a treat.
What am I doing wrong - how can I work around this issue.
For reference, I'd share Lambda print debug code.
Preparation:
from the Lambda-py38 initial state, SecretsManagerReadWrite policy is attached to Lamda
set SecretId and VersionId
Output:
json.loads() works for plaintext
print() output almost the same for plain_text:str and secret_dict:dict. (Difference is quotation:single/double. Which are treated as the same string when copy-pasted.)
import json
import boto3
def lambda_handler(event, context):
client = boto3.client('secretsmanager')
response = client.get_secret_value(
SecretId='arn:aws:secretsmanager:xxxxxxxxxxxxxxxxxxxxxxxxxxxxx',
VersionId='xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx',
VersionStage='AWSCURRENT'
)
print("type of response: ", type(response))
print("response: ", response)
plaintext = response['SecretString']
print("type of plaintext: ", type(plaintext))
print("plaintext: ", plaintext)
secret_dict = json.loads(plaintext)
print("type of secret_dict: ", type(secret_dict))
print("secret_dict: ", secret_dict)
return 200

Passing arguments as json object

I am trying to link my django web app to Azure ML API. I do have Django form with all the required inputs for my Azure API.
def post(self,request):
form = CommentForm(request.POST)
url = 'https://ussouthcentral.services.azureml.net/workspaces/7061a4b24ea64942a19f74ed36e4b438/services/ae2c257d6e164dca8d433ad1a1f9feb4/execute?api-version=2.0&format=swagger'
api_key = # Replace this with the API key for the web service
headers = {'Content-Type':'application/json', 'Authorization':('Bearer '+ api_key)}
if form.is_valid():
age = form.cleaned_data['age']
bmi = form.cleaned_data['bmi']
args = {"age":age,"bmi":bmi}
json_data = str.encode(json.dumps(args))
print(type(json_data))
r= urllib.request.Request(url,json_data,headers)
try:
response = urllib.request.urlopen(r)
result = response.read()
print(result)
except urllib.request.HTTPError as error:
print("The request failed with status code: " + str(error.code))
print(json_data)
# Print the headers - they include the requert ID and the timestamp, which are useful for debugging the failure
print(error.info())
print(json.loads(error.read()))
return render(request,self.template_name)
When i try to submit the form i am getting type error -
TypeError('POST data should be bytes, an iterable of bytes, or a file object. It cannot be of type str.',)
Getting status code - 400 and below error
{'error': {'code': 'BadArgument', 'message': 'Invalid argument provided.', 'details': [{'code': 'RequestBodyInvalid', 'message': 'No request body provided or error in deserializing the request body.'}]}}
Arguments are using print(json_data) -
b'{"age": 0, "bmi": 22.0}'
Can someone help me on this?
Try to use JsonResponse:
https://docs.djangoproject.com/en/2.1/ref/request-response/#jsonresponse-objects
Also, I don't think you need a template for API response.
return JsonResponse({'foo': 'bar'})
Thanks all. I found the error it was the data which was not in proper order.

Requests for Instagram Access token

I am working on Instagram API in Django(python)
I am getting code from
'https://api.instagram.com/oauth/authorize/?client_id=%s&response_type=code&redirect_uri=%s' % (INSTAGRAM_APP_CONSUMER_KEY, redirect_uri)
but when i am exchanging code for access token code is failing
# All arguments are valid
def execute(self, code, redirect_uri, app_id, app_secret):
exchange_url = 'https://api.instagram.com/oauth/access_token'
try:
#Update : get request to post
r = requests.post(exchange_url, params={
'client_id': app_id,
'redirect_uri': redirect_uri,
'client_secret': app_secret,
'code': code,
'grant_type': 'authorization_code'
})
#print(r)
#print(json.loads(r))
print(r.json())
return r.json()
except Exception as e:
print(e)
r.json() gives simplejson.scanner.JSONDecodeError: Expecting value: line 1 column 1
Update 1 : r.json() works after request changed from get to post but error
message comming 'You must provide a client_id'
Please let me know what i am doing wrong
I think it requires data in post data, not in query params.
Try this :
your_post_data = {'client_id': '', ... }
r = requests.post('your_url', data=your_post_data)
Ref: Python Requests Docs

error in json in corona lua

hi i have found a tutorial on how to use post json in lua.
here is the code :
http = require("socket.http")
crypto = require("crypto")
ltn12 = require("ltn12")
url = require("socket.url")
local json = require("json")
local commands_json =
{
["message"] = "Hello",
}
print (commands_json)
local json = {}
json.api_key = "6_192116334"
json.ver = 1
json.commands_json = json.encode(commands_json)
json.commands_hash = crypto.digest(crypto.md5, json.commands_json .. 'hkjhkjhkjh')
local post = "api=" .. url.escape(Json.Encode(json))
local response = {}
local r, c, h = http.request {
url = "http://127.0.0.1/?page=api",
method = "POST",
headers = {
["content-length"] = #post,
["Content-Type"] = "application/x-www-form-urlencoded"
},
source = ltn12.source.string(post),
sink = ltn12.sink.table(response)
}
local path = system.pathForFile("r.txt", system.DocumentsDirectory)
local file = io.open (path, "w")
file:write (Json.Encode(json) .. "\n")
file:write (post .. "\n")
file:write (response[1] .. "\n")
io.close (file)
json = Json.Decode(table.concat(response,''))
native.showAlert("hey", json.commands[1].tot_nbr_rows)
now i got these error:
Windows simulator build date: Dec 9 2011 # 14:01:29
Copyright (C) 2009-2011 A n s c a , I n c .
Version: 2.0.0
Build: 2011.704
table: 0346D6D0
Runtime error
...nistrator\my documents\corona projects\json\main.lua:17: attempt to c
all field 'encode' (a nil value)
stack traceback:
[C]: in function 'encode'
...nistrator\my documents\corona projects\json\main.lua:17: in main chun
k
Runtime error: ...nistrator\my documents\corona projects\json\main.lua:17: attem
pt to call field 'encode' (a nil value)
stack traceback:
[C]: in function 'encode'
...nistrator\my documents\corona projects\json\main.lua:17: in main chun
k
i don't know why i got the error from encode.
can anyone can help me about my case?
thanks in advance ...
This includes the Json code provided externally, likely with an encode function:
local json = require("json")
This throws away your old json variable and replaces it with an empty table:
local json = {}
And this tries to call json.encode which is now undefined since you redefined json as an empty table above:
json.commands_json = json.encode(commands_json)
The solution is to pick a different variable name.