Azure API Management: cache-store not honoring duration - azure-api-management

In Azure APIM, I have an API that queries a SQL table. I created a cache policy for the API to expire after 300 sec. I then created a script to invoke the API every 10 min, and traced the API response and SQL Server db.
For almost an hour, all calls to the API were cache hits - only two cache misses were recorded in App Insights (the first call and last call at the +50 min mark), and only two sql calls were recorded in my sql profiler trace (the first and last call).
I was expecting to see a cache-miss almost every time, but did not. Am I misunderstanding cache-store?
Here is the policy definition:
<policies>
<inbound>
<base />
<cache-lookup vary-by-developer="false" vary-by-developer-groups="false" must-revalidate="true" downstream-caching-type="none" caching-type="internal">
<vary-by-query-parameter>xxxxx</vary-by-query-parameter>
</cache-lookup>
</inbound>
<backend>
<base />
</backend>
<outbound>
<cache-store duration="300" />
<base />
</outbound>
<on-error>
<base />
</on-error>
</policies>

Can you please try the modified outbound policy:
<cache-store duration="10" cache-response="true" />
Store to cache
cache-response:
Set to true to cache the current HTTP response. If the attribute is omitted or set to false, only HTTP responses with the status code 200 OK are cached.
This does also mean, you have to check your current response codes 200/400/404/500/...
Maybe there was most of the time a 404 response, which was not cached.

Related

API Manager: Use re-written URI in Send-Request method

I currently have two resource groups: A and B. Resource group A contain my APIM and one LogicApp. However, I also need to call upon a LogicApp in resource group B.
To do this, I'm trying to manually send a HTTP call to the LogicApp in the other resource group through the APIM Policy.
I've been stuck on this the entire day thus far.
My current policy code looks like this:
<policies>
<inbound>
<base />
<!-- Load query parameter -->
<set-query-parameter name="id" exists-action="override">
<value>#(context.Request.MatchedParameters["id"])</value>
</set-query-parameter>
<!-- Insert the ID into the URL by rewriting it -->
<rewrite-uri template="https://<logicAppAddress>/triggers/manual/paths/invoke/{id}?<otherPartofUrl>" copy-unmatched-params="true" />
<send-request ignore-error="false" timeout="20" response-variable-name="MasterKeyResponse" mode="new">
<!-- How do I get the re-written URL here? -->
<set-url>#( context.Request.Url.Path )</set-url>
<set-method>GET</set-method>
</send-request>
</inbound>
<backend>
<base />
</backend>
<outbound>
<base />
</outbound>
<on-error>
<base />
</on-error>
The "id" variable gets put into the address correctly, which is great, seeing as the LogicApp calls upon a CosmosDB to retrieve data and then manipulates it.
However, for some reason, I can't seem to get this rewritten URL into the <set-url> section of the <send-request> function.
I really apologize if this is a stupid question, I'm currently interning and this is the first time working with Azure APIM, so it's all relatively new to me. I couldn't find an answer to this on Stackoverflow.
Or is there a better way to do this? Perhaps I'm overlooking something.
Thank you very much for your help in advance and have a nice day.
Solved! For anyone struggling with this later on, this post helped me solve it:
Azure APIM pass through request querystring to backend completely

Sending events to event hub from APIM result in unauthorized when using managed identity

I am attempting to setup an APIM endpoint that sends messages to an event hub. I also want to use managed identities in order to authorize the APIM with the event hub. Note that all resources lie in the same subscription. The setup is as follows:
I have an APIM instance with a system assigned identity. This identity has been giving the contributor role on a subscription level.
I have an event hub namespace and event hub, which is setup to receive the events.
I have created an API + operation, that generates events, based on the payload and sends them to the event hub. The example below just sends some hardcoded body, I want to get it working before working on the payload.
The policy for the operation looks like this:
<policies>
<inbound>
<base />
<authentication-managed-identity resource="https://eventhubs.azure.net" output-token-variable-name="msi-access-token" ignore-error="false" />
<set-header name="Authorization" exists-action="override">
<value>#(String.Concat("Bearer ",(string)context.Variables["msi-access-token"]))</value>
</set-header>
<set-body>{ "Event":"apim-using -aad token", "TrustedService":"AAD" }</set-body>
<set-backend-service base-url="https://[someeventhub].servicebus.windows.net" />
<rewrite-uri template="/input/messages?api-version=2014-01" />
</inbound>
<backend>
<base />
</backend>
<outbound>
<base />
</outbound>
<on-error>
<base />
</on-error>
</policies>
Whenever I try to test the operation though, I get a 401 Unauthorized.
HTTP/1.1 401 SubCode=40100: Unauthorized : Unauthorized access for 'Send' operation on endpoint 'sb://[someeventhub].servicebus.windows.net/input/messages?api-version=2014-01'. Tracking Id: [X]
Looking at the trace, the authentication requests looks like it went through okay:
authentication-managed-identity (0.365 ms)
{
"message": "Obtaining managed identity token using clientId:[X] AAD Authority:https://login.windows.net/[A] for https://eventhubs.azure.net audience succeeded.",
"errorResponse": null
}
Am I missing something here? It seems to me that there might be something with an app registration? I don't understand why though - the app already has contributor rights for the subscription. Does it need anything else?
As a final note, the forward request looks like this:
forward-request (0.129 ms)
{
"message": "Request is being forwarded to the backend service. Timeout set to 300 seconds",
"request": {
"method": "POST",
"url": "https://[someeventhub].servicebus.windows.net/input/messages?api-version=2014-01",
"headers": [
{
// A bunch of headers
},
{
"name": "Authorization",
"value": "Bearer [A VALID JWT TOKEN]"
}
]
}
}
the app already has contributor rights for the subscription. Does it need anything else?
Yes; The "Contributor" role gives the app access to the Azure resource management plane for operations like creating a new Event Hub but does not grant access for the data plane.
The app will need to have either "Event Hubs Data sender" or "Event Hubs Data owner" role in order to publish events. (see: Authorize access to Event Hubs resources using Azure Active Directory for more context)

Azure API Management Restrict multiple caller IP Address API in API endpoint level

I wanted to restrict some IP's in Azure APIM policy level.
I went thro below links;
https://learn.microsoft.com/en-us/azure/api-management/api-management-access-restriction-policies#RestrictCallerIPs
Azure API Management Restrict multiple caller IP Address
But not sure how can I do this to API end-point level using policy scope
I have below code in the policy.xml:
<policies>
<inbound>
<base />
<!-- statements to be applied to the request go here -->
<authentication-certificate thumbprint="#((string)context.Variables["ClientCertificateThumbprint"])" />
<rate-limit-by-key calls="100" renewal-period="60" counter-key="#(context.Request.Headers.GetValueOrDefault("Ocp-Apim-Subscription-Key"))" />
<cors>
<allowed-origins>
<origin>*</origin>
</allowed-origins>
<allowed-methods preflight-result-max-age="600">
<method>*</method>
</allowed-methods>
<allowed-headers>
<header>*</header>
</allowed-headers>
<expose-headers>
<header>*</header>
</expose-headers>
</cors>
<ip-filter action="allow">
<address>55.11.187.20</address>
<address-range from="186.168.95.0" to="186.168.95.20" />
</ip-filter>
</inbound>
<backend>
<base />
<!-- statements to be applied before the request is forwarded to
the backend service go here -->
</backend>
<outbound>
<base />
<!-- statements to be applied to the response go here -->
</outbound>
<on-error>
<base />
<!-- statements to be applied if there is an error condition go here -->
</on-error>
</policies
>
Using control flow in Advanced policies you can change the scope to API endpoint level (operation) to restrict IP addresses as below
<choose>
<when condition="#(context.Operation.Id.Equals("StatusGet"))">
<ip-filter action="allow">
<address>55.11.187.20</address>
<address-range from="186.168.95.0" to="186.168.95.20" />
</ip-filter>
</when>
</choose>
</inbound>
Refer: https://learn.microsoft.com/en-us/azure/api-management/api-management-advanced-policies
Navigate to Azure portal, your APIM service, APIs.
Click API you want to apply IP filter to
In the "Inbound processing" section click "Add policy" and select IP filter.

Is Limit call rate by subscription supported in Consumption tier?

I am trying to add a rate-limit policy for an API management (per API Management access restriction policies) and I am not sure if I do something wrong or if documentation is not correct.
According to the documentation, the
Set usage quota by key
Limit call rate by key
are not available in consumption tier, but Limit call rate by subscription has no such notice, so it should work (plus Set usage quota by subscription can be set).
I have an Azure API Management, I have created a sample API and a product and in the policy of the product is:
<policies>
<inbound>
<base />
<quota calls="100" renewal-period="86400" />
<rate-limit calls="20" renewal-period="90" />
</inbound>
<backend>
<base />
</backend>
<outbound>
<base />
</outbound>
<on-error>
<base />
</on-error>
</policies>
When I click on Save, I get
One or more fields contain incorrect values:
Error in element 'rate-limit' on line 16, column 10: Policy is not allowed in this sku 'Consumption'
Is there a way to set the rate-limit (=am I missing something) for consumption tier or is the documentation not correct?
You can now use the rate-limit and quota tag in policies at API level.
<policies>
<inbound>
<base />
<rate-limit calls="5" renewal-period="60" remaining-calls-variable-name="remainingCallsPerSubscription" />
</inbound>
<backend>
<base />
</backend>
<outbound>
<base />
</outbound>
<on-error>
<base />
</on-error>
</policies>
Using quotas:
<policies>
<inbound>
<base />
<quota calls="10000" renewal-period="3600" />
</inbound>
<outbound>
<base />
</outbound>
</policies>
Edit: on the consumption tier you can only apply these policies on API level. (So they apply to everyone that uses the API)
There is an ongoing update, once complete (a few days at most) "rate-limit" will be enabled for Consumption SKU, "rate-limit-by-key" disabled.

Azure API Management: Doing a global policy for set-backend-service results in scope error

So globally on my API service, I always wish to set the backend service URL based on certain calling regions.
According to this MSDN library article, using the set-backend-service policy sounds perfect for this, and it's a global policy according to its policy scope at the bottom.
However, even posting their exact example...
<policies>
<inbound>
<choose>
<when condition="#(context.Request.Url.Query.GetValueOrDefault("version") == "2013-05")">
<set-backend-service base-url="http://contoso.com/api/8.2/" />
</when>
<when condition="#(context.Request.Url.Query.GetValueOrDefault("version") == "2014-03")">
<set-backend-service base-url="http://contoso.com/api/9.1/" />
</when>
</choose>
</inbound>
....results in an error:
"Error in element 'set-backend-service' on line 0, column 0: Policy is
not allowed in the specified scope"
I can't figure out how to make it any simpler of a situation to troubleshoot. I even removed the conditional statement and just left the policy alone and it still shows the scope error.
I know this global scope works okay, since I was able to put in an xml-to-json policy as a temporary test and save successfully.
I would figure someone ran into this issue already, as this must be a common use case for this policy. Otherwise, I think the MSDN article is out of date, unless anyone here can see any issues.
Here's my policy scope for global:
Change the quotes to single quotes where "version" is. I just ran the following no problem:
<policies>
<inbound>
<choose>
<when condition="#(context.Request.Url.Query.GetValueOrDefault('version') == '2013-05')">
<set-backend-service base-url="http://contoso.com/api/8.2/" />
</when>
<when condition="#(context.Request.Url.Query.GetValueOrDefault('version') == '2014-03')">
<set-backend-service base-url="http://contoso.com/api/9.1/" />
</when>
</choose>
<base />
</inbound>
<outbound>
<base />
</outbound>
</policies>