CosmosDb with DocumentDB API through Azure API Management - azure-api-management
I am attempting to query my new CosmosDB collection through API management. Once proved out, this will be a front-end for user access to logged data. For that reason, I have the data partitioned by subscription ID. In Azure Portal for the Logs collection of my WebApi DB I see partition key as /api_subscription_key. I have the data going in from API Mgt. -> Event Hub -> Stream Analytics -> Cosmos.
Using the query explorer in Azure portal, I can try a query like:
SELECT * FROM c WHERE c.api_subscription_key = '573a1c65bceb52192c140131'
this brings back expected documents I have been successfully writing to CosmosDB for many days
[
{
"eventenqueuedutctimesecond": "2017-07-27T15:09:02Z",
"business_unit_key": null,
"user_key": null,
"api_message_id": "1718ea66-d225-45ec-b3fc-5daff4c7f426",
"api_identifier": "21926e9d-9206-42b0-b4b1-7e7f1eb4e7dd",
"api_id": "58d94cc622be39392343d4b6",
"api_operation_id": "58e682bde055cd0ba4215d4b",
"api_adapter_id": "573a1c64bceb520aac127ee5",
"api_subscription_id": "573a1c65bceb52192c140131",
"api_policy_id": "64BC4270-54AC-42DA-835C-E285F35BCA81",
"basic_username": "",
"message_version": "10",
"claim_business_unit_key": null,
"claim_user_key": null,
...
"lasterrorsource": null,
"lasterrorreason": null,
"lasterrorscope": null,
"lasterrorsection": null,
"lasterrorpolicyid": null,
"id": "7/27/2017 3:09:02 PM",
"_rid": "9Fc0ANW4fwAoAAAAAAAADA==",
"_self": "dbs/9Fc0AA==/colls/9Fc0ANW4fwA=/docs/9Fc0ANW4fwAoAAAAAAAADA==/",
"_etag": "\"0700d90c-0000-0000-0000-597a020e0000\"",
"_attachments": "attachments/",
"_ts": 1501168140
}...
My CosmosDB instance is plexconnectcosmos. Thru API management and its policies I am POSTing to
https://plexconnectcosmos.documents.azure.com/dbs/WebApi/colls/Logs/docs
with these headers (many vestigial and hopefully having no effect):
[
{
name: "Postman-Token",
value: "756c2c21-ef23-4e5a-a63a-ae6aed961d35"
},
{
name: "Ocp-Apim-Subscription-Key",
value: "a2a05eff128943bc89f62b81a63aa368"
},
{
name: "Accept-Charset",
value: "UTF-8"
},
{
name: "Cache-Control",
value: "no-cache"
},
{
name: "Content-Type",
value: "application/query+json"
},
{
name: "Accept",
value: "application/json;odata=nometadata"
},
{
name: "Accept-Encoding",
value: "gzip,deflate"
},
{
name: "Cookie",
value: "x-ms-gateway-slice=008; stsservicecookie=ests; BIGipServerpmc_rest_webservices_http_prod=1242575370.20480.0000"
},
{
name: "User-Agent",
value: "PostmanRuntime/6.2.5"
},
{
name: "x-ms-date",
value: "Wed, 09 Aug 2017 20:10:09 GMT"
},
{
name: "x-ms-version",
value: "2017-02-22"
},
{
name: "MaxDataServiceVersion",
value: "3.0"
},
{
name: "DataServiceVersion",
value: "1.0;NetFx"
},
{
name: "Api-Message-Id",
value: "12427ae7-7704-44cb-b4af-d7e622898b99"
},
{
name: "Api-Identifier",
value: "461f0c19-8df3-4272-9ac7-c64bb776dd56"
},
{
name: "Api-Id",
value: "58987927bceb5204c4e59168"
},
{
name: "Api-Operation-Id",
value: "598b3c72e055cd14fc3abdd1"
},
{
name: "Api-Adapter-Id",
value: "573a1c64bceb520aac127ee5"
},
{
name: "Api-Subscription-Id",
value: "573a1c65bceb52192c140131"
},
{
name: "Api-Policy-Id",
value: "64BC4270-54AC-42DA-835C-E285F35BCA81"
},
{
name: "X-Basic-Username",
value: ""
},
{
name: "x-ms-documentdb-isquery",
value: "True"
},
{
name: "x-ms-documentdb-query-enablecrosspartition",
value: "False"
},
{
name: "x-ms-max-item-count",
value: "1000"
},
{
name: "x-ms-documentdb-partitionkey",
value: "573a1c65bceb52192c140131"
},
{
name: "x-ms-partition-key",
value: "573a1c65bceb52192c140131"
},
{
name: "Authorization",
value: "type=master&ver=1.0&sig=Ke...Q="
},
{
name: "X-Forwarded-For",
value: "75.39.38.67"
}
]
The response I get back is either
{
"code": "BadRequest",
"message": "Partition key 573a1c65bceb52192c140131 is invalid.\r\nActivityId: 61836599-fe4b-4232-b55b-2c568eecc767"
}
or
{
"code": "Unauthorized",
"message": "The input authorization token can't serve the request. Please check that the expected payload is built as per the protocol, and check the key being used. Server used the following payload to sign: 'post\ndocs\ndbs/WebApi/colls/Logs\nwed, 09 aug 2017 20:35:41 gmt\n\n'\r\nActivityId: 429....2e2"
}
These seems to give me two problems to solve. In the first place how do I go about troubleshooting this partition? It seems to be from my analysis, it is a valid partition, validating by queries in the portal and the headers "x-ms-documentdb-partitionkey" and "x-ms-partition-key". (I have seen both header names in MS documentation, so I am covering my bases with both.)
The "The input authorization token can't serve the request." message suggests to me something variably wrong in my query. I suspect maybe the data value? My policy is little different from one I use for Azure Table Storage REST API and I never have that problem. I am using my read-only primary key taken from Azure portal and stored in API Management's Named Values:
<policies>
<inbound>
<base />
<set-variable name="Content-Type" value="application/query+json" />
<set-variable name="x-ms-documentdb-isquery" value="True" />
<set-variable name="x-ms-documentdb-query-enablecrosspartition" value="False" />
<set-variable name="x-ms-max-item-count" value="1000" />
<set-variable name="x-ms-version" value="2017-02-22" />
<set-header name="Content-Type" exists-action="override">
<value>#((string)context.Variables["Content-Type"])</value>
</set-header>
<set-header name="x-ms-documentdb-isquery" exists-action="override">
<value>#((string)context.Variables["x-ms-documentdb-isquery"])</value>
</set-header>
<set-header name="x-ms-documentdb-query-enablecrosspartition" exists-action="override">
<value>#((string)context.Variables["x-ms-documentdb-query-enablecrosspartition"])</value>
</set-header>
<set-header name="x-ms-max-item-count" exists-action="override">
<value>#((string)context.Variables["x-ms-max-item-count"])</value>
</set-header>
<set-header name="x-ms-version" exists-action="override">
<value>#((string)context.Variables["x-ms-version"])</value>
</set-header>
<!-- MS docs may conflict here. Possibly "x-ms-documentdb-partitionkey" req'd and "x-ms-partition-key" not supported -->
<set-header name="x-ms-documentdb-partitionkey" exists-action="override">
<value>#(context.Subscription.Id)</value>
</set-header>
<set-header name="x-ms-partition-key" exists-action="override">
<value>#(context.Subscription.Id)</value>
</set-header>
<set-variable name="StringToSign" value="#(string.Format("post\ndocs\ndbs/WebApi/colls/Logs\n{0}\n\n", ((string)context.Variables["x-ms-date"]).ToLowerInvariant()))" />
<set-variable name="cosmosreadonlykey" value="{{CosmosReadOnlyKey}}" />
<set-variable name="SharedKey" value="#{
// https://learn.microsoft.com/en-us/rest/api/documentdb/access-control-on-documentdb-resources#constructkeytoken
System.Security.Cryptography.HMACSHA256 hasher = new System.Security.Cryptography.HMACSHA256(Convert.FromBase64String((string)context.Variables["cosmosreadonlykey"]));
return Convert.ToBase64String(hasher.ComputeHash(System.Text.Encoding.UTF8.GetBytes((string)context.Variables["StringToSign"])));
}" />
<set-variable name="Authorization" value="#(string.Format("type=master&ver=1.0&sig={0}", (string)context.Variables["SharedKey"]))" />
<set-header name="Authorization" exists-action="override">
<value>#((string)context.Variables["Authorization"])</value>
</set-header>
<set-backend-service base-url="https://plexconnectcosmos.documents.azure.com" />
<rewrite-uri template="/dbs/WebApi/colls/Logs/docs" />
</inbound>
Some thins I wonder: Could the returned ActivityId help me get any more details, some how? Even without it, is there some logging in Azure I haven't found that would reveal more details.
If anything is obviously wrong here that I am doing, please someone let me know.
I got it running with some minor adjustments.
<policies>
<inbound>
<base />
<set-variable name="Content-Type" value="application/query+json" />
<set-variable name="x-ms-documentdb-isquery" value="True" />
<set-variable name="x-ms-documentdb-query-enablecrosspartition" value="False" />
<set-variable name="x-ms-max-item-count" value="1000" />
<set-variable name="x-ms-version" value="2017-02-22" />
<set-variable name="x-ms-date" value="#( DateTime.UtcNow.ToString("R") )" />
<set-header name="Content-Type" exists-action="override">
<value>#((string)context.Variables["Content-Type"])</value>
</set-header>
<set-header name="x-ms-documentdb-isquery" exists-action="override">
<value>#((string)context.Variables["x-ms-documentdb-isquery"])</value>
</set-header>
<set-header name="x-ms-documentdb-query-enablecrosspartition" exists-action="override">
<value>#((string)context.Variables["x-ms-documentdb-query-enablecrosspartition"])</value>
</set-header>
<set-header name="x-ms-max-item-count" exists-action="override">
<value>#((string)context.Variables["x-ms-max-item-count"])</value>
</set-header>
<set-header name="x-ms-version" exists-action="override">
<value>#((string)context.Variables["x-ms-version"])</value>
</set-header>
<set-header name="x-ms-documentdb-partitionkey" exists-action="override">
<value>#("[\""+context.Subscription.Id+"\"]")</value>
</set-header>
<set-header name="x-ms-date" exists-action="override">
<value>#( (string)context.Variables["x-ms-date"] )</value>
</set-header>
<set-variable name="StringToSign" value="#(string.Format("post\ndocs\ndbs/WebApi/colls/Logs\n{0}\n\n", ((string)context.Variables["x-ms-date"]).ToLowerInvariant()))" />
<set-variable name="cosmosreadonlykey" value="{{CosmosReadOnlyKey}}" />
<set-variable name="SharedKey" value="#{
// https://learn.microsoft.com/en-us/rest/api/documentdb/access-control-on-documentdb-resources#constructkeytoken
System.Security.Cryptography.HMACSHA256 hasher = new System.Security.Cryptography.HMACSHA256(Convert.FromBase64String((string)context.Variables["cosmosreadonlykey"]));
return Convert.ToBase64String(hasher.ComputeHash(System.Text.Encoding.UTF8.GetBytes((string)context.Variables["StringToSign"])));
}" />
<set-variable name="Authorization" value="#(string.Format("type=master&ver=1.0&sig={0}", ((string)context.Variables["SharedKey"]).Replace("&","%26").Replace("+","%2B").Replace("=","%3D")))" />
<set-header name="Authorization" exists-action="override">
<value>#((string)context.Variables["Authorization"])</value>
</set-header>
<set-backend-service base-url="https://mycosmosdb.documents.azure.com" />
<rewrite-uri template="/dbs/WebApi/colls/Logs/docs" />
</inbound>
</policies>
the partition key needs to be formatted as an array
date is put into the header and the StringToSign based on the same value
did some hacky URL hex encoding - could be improved with proper hex encoding
Related
Azure APIM How to assign Header response to a variable
I have the following policy fragment that connects to an API we are using for authorization.
<fragment>
<send-request mode="new" response-variable-name="bearerToken" timeout="20" ignore-error="true">
<set-url>{{url}}</set-url>
<set-method>POST</set-method>
<set-header name="Content-Type" exists-action="override">
<value>application/x-www-form-urlencoded</value>
</set-header>
<set-body>#{ return "{{target-url}}"; }</set-body>
</send-request>
<set-header name="Authorization" exists-action="override">
<value>#("Bearer " + (String)((IResponse)context.Variables["bearerToken"]).Body.As<JObject>()["access_token"])</value>
</set-header>
</fragment>
This works fine with no issues, however, I would like to capture the information pushed to the Authorization Header and push it to a variable for later use in my APIs.
When I do something similar to the following or it's variants, I only receive errors and no information as to what has happened.
<set-variable name="BearerToken" value="#("Bearer " + (String)((IResponse)context.Variables["bearerToken"]).Body.As<JObject>()["access_token"]" />
I'm not sure if this is an order of execution or something else I might have missed?
Thank you - Greg.
how to check value passed in header is Guid or not in azure APIM policy if not throw bad request
APIM policy:
<set-variable name="xcid" value="#(context.Request.Headers.GetValueOrDefault("x-correlation-id", Guid.NewGuid().ToString()))" />
<choose>
<!--Request will be rejected if vaild x-correlation-id is not passed -->
<when condition="#((bool)Guid.TryParse(context.Variables["xcid"], out newGuid)== false)">
<return-response>
<set-status code="400" reason="Invalid x-correlation-id is passed please pass valid guid x-correlation-id" />
</return-response>
</when>
</choose>
Your solution was almost correct.
I assume you want to achieve the following.
if header x-correlation-id is missing, create a new guid and proceed
if header x-correlation-id is present and the value is a valid guid, then proceed
otherwise, reject the request with a status code 400
I had to fix only this line:
<when condition="#(Guid.TryParse(context.Variables.GetValueOrDefault<string>("xcid"), out Guid newGuid) == false)">
The variable xcid has to be string.
The datatype Guid his to be used for the out parameter newGuid.
Policy:
<set-variable name="xcid" value="#(context.Request.Headers.GetValueOrDefault("x-correlation-id", Guid.NewGuid().ToString()))" />
<choose>
<!--Request will be rejected if vaild x-correlation-id is not passed -->
<when condition="#(Guid.TryParse(context.Variables.GetValueOrDefault<string>("xcid"), out Guid newGuid) == false)">
<return-response>
<set-status code="400" reason="Invalid x-correlation-id is passed please pass valid guid x-correlation-id" />
</return-response>
</when>
</choose>
Azure APIM calls function even though there was an error
I have an Azure APIM which checks the request bearer JWT token:
<inbound>
<base />
<set-variable name="TenantId" value="
#{
var tenantId = context.Request.Headers.GetValueOrDefault("TenantId","");
if (tenantId != null) {
return tenantId;
}
return null;
}" />
<limit-concurrency key="#((string)context.Variables["TenantId"])" max-count="1">
<set-header name="Ocp-Apim-Subscription-Key" exists-action="delete" />
<set-header name="subscription-key" exists-action="delete" />
<set-variable name="JWTIssuer" value="
#{
var authStr = context.Request.Headers.GetValueOrDefault("Authorization","");
if (authStr.StartsWith("Bearer")) {
var jwt = authStr.Substring("Bearer ".Length).AsJwt();
if (jwt != null) {
return jwt.Issuer;
}
}
return null;
}" />
<choose>
<when condition="#(context.Variables.GetValueOrDefault("JWTIssuer", "") == "<URL of issuer>")">
<!-- Authorisation using HPCA token -->
<validate-jwt header-name="Authorization" failed-validation-httpcode="401" failed-validation-error-message="Unauthorized" require-expiration-time="true" require-scheme="Bearer" require-signed-tokens="true">
<openid-config url="<Open Id Url>" />
<required-claims>
<claim name="MyClaim" match="any" separator=",">
<value>X</value>
</claim>
</required-claims>
</validate-jwt>
</when>
<!-- HPCA Error handling -->
<otherwise>
<return-response>
<set-status code="401" reason="Unauthorized" />
<set-header name="WWW-Authenticate" exists-action="override">
<value>Bearer error "Invalid HPCA token"</value>
</set-header>
</return-response>
</otherwise>
</choose>
</limit-concurrency>
</inbound>
<on-error>
<base />
<return-response>
<set-status code="#(context.Response.StatusCode)" reason="#(context.Response.StatusReason)" />
</return-response>
</on-error>
This all works as expected: if either the issuer is incorrect or the JWT token is invalid, then the OnError handler is invoked and a 401 error status is returned to the caller. The problem is, if i add a call to a function after the Choose policy:
</otherwise>
</choose>
<send-request mode="copy" response-variable-name="Response" timeout="10" ignore-error="false">
<set-url>https://myfuncApp.azurewebsites.net/api/Function1</set-url>
<set-method>POST</set-method>
</send-request>
<return-response response-variable-name="Response">
<set-status code="#(context.Response.StatusCode)" reason="#(context.Response.StatusReason)" />
</return-response>
</limit-concurrency>
</inbound>
Then even with a invalid bearer token and the error handler being invoked, the APIM still proceeds to call the function: the return response within the error handler is seemingly ignored - why ? I can fudge it by wrapping the send-request within another conditional (i.e. check the context.Response.StatusCode isn't 401) but this doesn't seem right to me.
For your question, I think the result of your test is normal. You added the <send-request> to request the function in <inbound> area. The <validate-jwt> can just determine if the APIM to continue request the backend url but can not determine if it continues to request function url in <inbound> area.
How Do I Throw An Error In Azure API Management Policy?
In my Azure API Management Policy I am checking for some headers and do certain actions depending on what is found. How do I throw an error when none of the conditions are matched (i.e. in the otherwise block) <policies> <inbound> <choose> <when condition=""> </when> <when condition=""> </when> <otherwise> </otherwise> </choose> <base/> </inbound> <backend> <base/> </backend> <outbound> <base/> </outbound> <on-error> <base/> </on-error> </policies> I probably want to return a 401 since I am checking groups in the headers.
You can use a <choose> policy to detect and report failure, return a 401 response. <otherwise> <return-response > <set-status code="401" reason="Unauthorized" /> <set-header name="WWW-Authenticate" exists-action="override"> <value>Bearer error="invalid_token"</value> </set-header> </return-response> </otherwise> Here is also a similar SO thread you could refer to.
Work with URL template parameter values in policy templates
In API Management, how do you access URL template parameters through a policy?
In this instance, my operation is called test, the HTML verb is GET, and the URL template is as below -
/test/{variable_name1}/{variable_name2}
I was under the impression that accessing the value of a parameter was as simple as {variable_name1}. However, the example below does not set the variable "rowkey" as expected. Rather it has a value of {variable_name1}-{variable_name2}.
What am I doing wrong here?
<policies>
<inbound>
<set-variable name="rowkey" value="{variable_name1}-{variable_name2}" />
</inbound>
<backend>
<base />
</backend>
<outbound>
<set-header name="Row-Key" exists-action="override">
<value>#((string)context.Variables["rowkey"])</value>
</set-header>
</outbound>
</policies>
You'd have to use expressions to achieve what you want, like:
<set-variable
name="rowkey"
value="#(context.Request.MatchedParameters["variable_name1"] + "-" + context.Request.MatchedParameters["variable_name2"])" />
or use string interpolation:
<set-variable
name="rowkey"
value="#($"{context.Request.MatchedParameters["variable_name1"]}-{context.Request.MatchedParameters["variable_name2"]}")" />