Azure APIM validate-content is not validating as expected for JSON - azure-api-management

I had an issue with validate-content policy our API, so for sharing in public
I created a simple Open API Spec yaml file with a schema to create an API and then added validate-content policy. It's validating as expected when a JSON when the right JSON is sent or if I miss any required fields it's preventing with right validation error. However when I send a completely unrelated JSON it's going through fine, expectation is to prevent otherwise it will break the policy expressions.
As suggested in one of the other threads, I also tried adding request representation and specified message schema but behaviour is same.
I also verified it on https://www.jsonschemavalidator.net/, where validation is as expected
I am not sure it's ignoring as a fact that JSON validation ignores extra elements
Contents of the YAML used
openapi: "3.0.0"
info:
title: address-schema-validation
description: This is a loop back API to test schema validation
version: '1.0'
license:
name: MIT
paths:
/validate:
post:
summary: Submit a request for validation
operationId: validate
requestBody:
content:
application/json:
schema:
$ref: "#/components/schemas/AddressBody"
responses:
'200':
description: valid payload
content:
application/json:
schema:
$ref: "#/components/schemas/AddressBody"
components:
schemas:
AddressBody:
type: object
properties:
address:
type: string
example: "01 Auckland"
name:
type: object
required:
- fistName
- lastName
properties:
fistName:
type: string
example: Fist Name
lastName:
type: string
example: Last Name
Policy
<policies>
<inbound>
<base />
<validate-content unspecified-content-type-action="prevent" max-size="102400" size-exceeded-action="prevent" errors-variable-name="requestBodyValidation">
<content type="application/json" validate-as="json" action="prevent" />
</validate-content>
<return-response>
<set-status code="200" reason="OK" />
<set-header name="Content-Type" exists-action="override">
<value>application/json</value>
</set-header>
<set-body>#(context.Request.Body.As<String>())</set-body>
</return-response>
</inbound>
<backend>
<base />
</backend>
<outbound>
<base />
</outbound>
<on-error>
<base />
</on-error>
</policies>
JSON that picks up the validation error
{
"address": "01 Auckland",
"name": {
"fistName1": "Fist Name",
"lastName": "Last Name"
}
}
with error message
{
"statusCode": 400,
"message": "Body of the request does not conform to the definition which is associated with the content type application/json. Required properties are missing from object: fistName. Line: 6, Position: 5"
}
The following JSON is expected to be prevented but it's not
{"prevent":"me"}

In fact validation is being done as expected, to prevent unexpected JSON, the schema should have a required root element, if the root element is optional then when completely unrelated JSON is posted the validation policy will simply ignore the unexpected elements.

Related

Schema object properties validation

According to OpenAPI specification [https://swagger.io/specification/] I can define the minimum and maximum values of the field in the schema object properties.
But actually Google endpoint does not perform any validation
My Swagger file fragment:
parameters:
- name: message
in: body
description: sug
schema:
required:
- Message
properties:
Message:
type: integer
minimum: 1
maximum: 1
MessageId:
When I send a wrong request:
curl https://xxxxxxxxxxxxxxxxxxxxd"Message=10&MessageId=456789123456&TimeStamp=20190611101010212&OperatorId=15&GlobalAccountId=81165751216851320000&Reason=3"
The Endpoint does not reject the request, but invokes the cloud function.
You are correct. Google Cloud Endpoints doesn't do any OpenAPI object property validation.

How to update Azure API Management named values in a batch?

We have existing production services which are based on this policy sample which acquires a bearer token via client grant flow. However we want to switch the client application settings to a new app, which requires updating 4 named values. How can this be done with high throughput traffic volumes?
We have updated named value pairs as documented and we've looked at the REST API. There is only 1-by-1 update of values. We applied this on lower volume environments with no problems. However when we applied this on production environments we have outages with the expression unable to parse the value, which results in a null and caused an outage.
This was the policy, copied from the portal.
<choose>
<when condition="#(!context.Variables.ContainsKey("access_token"))">
<send-request ignore-error="true" timeout="20" response-variable-name="response" mode="new">
<set-url>{{authorizationServer}}</set-url>
<set-method id="apim-generated-policy">POST</set-method>
<set-header name="Content-Type" exists-action="override">
<value>application/x-www-form-urlencoded</value>
</set-header>
<set-body>#{return "client_id={{clientid}}&resource={{scope}}&client_secret={{clientsecret}}&grant_type=client_credentials";}</set-body>
</send-request>
<set-variable name="responseJson" value="#(((IResponse)context.Variables["response"]).Body.As<JObject>())" />
<set-variable name="access_token" value="#("Bearer " + (String)((JObject)context.Variables["responseJson"])["access_token"])" />
<set-variable name="expires_in" value="#((int)((JObject)context.Variables["responseJson"])["expires_in"] - 20)" />
<cache-store-value key="access_token" value="#((string)context.Variables["access_token"])" duration="#(((int)context.Variables["expires_in"]))" />
</when>
</choose>
<set-header name="Authorization" exists-action="override">
<value>#((string)context.Variables["access_token"])</value>
</set-header>
"Elapsed": 109,
"Source": "set-variable[3]",
"Reason": null,
"Message": "Expression evaluation failed. Value cannot be null.\r\nParameter name: value\r\n at Newtonsoft.Json.Linq.JToken.EnsureValue(JToken value)\r\n at Newtonsoft.Json.Linq.JToken.op_Explicit(JToken value)",
"Scope": "product",
"Section": "inbound",
"Path": "choose\\when[1]",
"PolicyId": "",
"TransportErrorCode": 0,
"HttpErrorCode": 0
}
I suspected that the issue was related to one of these variables being null but our logs were unclear as to which variable.
There is no API to anatomically update multiple named value properties. There are two approaches that come to mind. You could store all your values in a single named value encoding them into a single string and parsing out inside policy.
Or you could have two sets of named values and a fifth one that controls which one to use. This way you'll be free to update your passive set freely and only activate it later.

Issues while validating the two JSON payloads in MUNIT 2.0

I have a service which is returning the below json payload:
{
"location-details": {
"fromPostalCode": "1508XB",
"fromGeoCoordinates": "116532,496398",
"toPostalCode": "1511MA",
"toGeoCoordinates": "120427,493380"
},
"commute-distance": "5817.799",
"commute-time": "370.0152"
}
I have created a MUNIT test suite for the same where i am using the assert that utility to assert the payload. Snippet given below:
<munit:validation >
<munit-tools:assert-that doc:name="Assert That Status Code is 200" doc:id="1c2e536b-513e-4b76-958b-2ea864a64805" expression="#[attributes.statusCode]" is="#[MunitTools::equalTo(200)]" message="The HTTP Status code is not correct!" />
<munit-tools:assert-that doc:name="Assert That - Payload is Expected" doc:id="f4b811fa-ff11-4746-93cc-f87576504808" expression="#[payload]" is="#[MunitTools::getResourceAsString('SuccessResponse.json')]" message="The response payload is not correct!"/>
</munit:validation>
But this is throwing an below error and its not working:
...38 more
Caused by: org.mule.runtime.api.el.ExpressionExecutionException: Unable to convert '{
"location-details": {
"fromPostalCode": "1508XB",
"fromGeoCoordinates": "116532,496398",
"toPostalCode": "1511MA",
"toGeoCoordinates": "120427,493380"
},
"commute-distance": "5817.799",
"commute-time": "370.0152"
}' with class 'java.lang.String' to class 'Matcher', while writing Java at org.mule.munit.tools.util.GetResourceFunctions.getResourceAsString.
at org.mule.weave.v2.el.WeaveExpressionLanguage.doEvaluate(WeaveExpressionLanguage.scala:139)
at org.mule.weave.v2.el.WeaveExpressionLanguage.evaluate(WeaveExpressionLanguage.scala:236)
You are missing the matcher component in your expression, please change it as below and try again:
<munit-tools:assert-that doc:name="Assert Payload is Correct" doc:id="5b45470f-05e8-406e-9d44-877ff1506220" expression='#[output application/json --- write(payload, "application/json")]' is="#[output application/json --- MunitTools::equalTo(MunitTools::getResourceAsString('json\SuccessResponse.json'))]" message="The message is not correct"/>
This will work for sure !!
I had to use readUrl instead of MunitTools::getResourceAsString to compare two JSON objects in MUNIT 2.1
<munit-tools:assert-that doc:name="Assert That - JSON compare" doc:id="2fab5bf0-a710-4619-a7a9-262a867e0ad9" expression="#[payload]" is="#[MunitTools::equalTo(readUrl('classpath://sample_data/test_payload.json', 'application/json'))]" message="payload is not correct" />

Convert POST to GET Azure API Management Rest API

I am using Azure API Management REST API's to import a WSDL and convert it to a REST endpoint calling SOAP backend.
However when you import the WSDL all the methods are imported as POST (which makes sense since you need to send the soap envelope). Now I want to convert the operation from POST to GET via the REST API (which I can do through portal).
Has anyone tried that before, and if yes which API's should I call?
For this case I used a public soap service:
https://www.dataaccess.com/webservicesserver/numberconversion.wso?op=NumberToDollars
I imported this SOAP service to API Management:
Request-Body:
<?xml version="1.0" encoding="utf-8"?>
<Envelope xmlns="http://www.w3.org/2003/05/soap-envelope">
<Body>
<NumberToDollars xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.dataaccess.com/webservicesserver/">
<dNum>1</dNum>
</NumberToDollars>
</Body>
</Envelope>
The NumberToDollars operation has to read the XML, transform it into JSON, and pass the data to a GET API.
For testing purposes I created an mocked Operation which returns 200:
https://rfqapiservicey27itmeb4cf7q.azure-api.net/echo/200/test
<policies>
<inbound>
<base />
<set-variable name="num" value="#{
string xml = context.Request.Body.As<string>(preserveContent: true);
xml = Regex.Unescape(xml);
// Remove the double quotes
xml = xml.Remove(0,1);
xml = xml.Remove(xml.Length-1,1);
XmlDocument doc = new XmlDocument();
doc.LoadXml(xml);
var data = JObject.Parse(JsonConvert.SerializeObject(doc));
JObject envelope = data["Envelope"] as JObject;
JObject body = envelope["Body"] as JObject;
JObject numberToDollars = body["NumberToDollars"] as JObject;
return numberToDollars["dNum"].Value<string>();
}" />
<set-method>GET</set-method>
<set-backend-service base-url="https://rfqapiservicey27itmeb4cf7q.azure-api.net/echo/200/" />
<rewrite-uri template="#("/test?q=" + context.Variables.GetValueOrDefault<string>("num"))" copy-unmatched-params="false" />
</inbound>
<backend>
<base />
</backend>
<outbound>
<base />
</outbound>
<on-error>
<base />
</on-error>
</policies>
Testing the operation:
The trace log:
The JSON document generated from XML:
{
"?xml": {
"#version": "1.0",
"#encoding": "utf-8"
},
"Envelope": {
"#xmlns": "http://www.w3.org/2003/05/soap-envelope",
"Body": {
"NumberToDollars": {
"#xmlns:xsi": "http://www.w3.org/2001/XMLSchema-instance",
"#xmlns": "http://www.dataaccess.com/webservicesserver/",
"dNum": "1"
}
}
}
}

API Managment unable to cast response body as string

In Azure API Management, when the response going back to the client is a 500, I wish to check the body of the response to see if it matches "Some text". I need to do this so that I may change the body of the response to contain some more helpful text in this particular scenario.
The following <outbound> section of my policy is accepted by the API Management console, but when I test and get a 500, API Management generates an error -
Expression evaluation failed. Unable to cast object of type 'Microsoft.WindowsAzure.ApiManagement.Proxy.Gateway.MessageBody' to type 'System.String'.
I'm guessing this is my fault, but does anybody know how I can amend the ploicy so that it does not generate an error? To clarify, the error is being generated by this line - ((string)(object)context.Response.Body == "Some text").
<outbound>
<choose>
<when condition="#((context.Response.StatusCode == 500) && ((string)(object)context.Response.Body == "Some text"))">
<set-status code="500" reason="Internal Server Error" />
<set-header name="Content-Type" exists-action="override">
<value>application/json</value>
</set-header>
<set-body>
{
"statusCode": "500",
"Message": "Some different, more helpful text."
}
</set-body>
</when>
</choose>
</outbound>
Update
I've discovered that context.Response.Body is of type IMessageBody. There seems to be woefully little documentation around this type, and the only reference I can find comes under <set-body> in the Transformation Policies API management documentation.
The troube is, the example that MS havd documented produces an exception when I try and save my policy -
<set-body>
#{
JObject inBody = context.Request.Body.As<JObject>();
if (inBody.attribute == <tag>) {
inBody[0] = 'm';
}
return inBody.ToString();
}
</set-body>
Property or indexer 'string.this[int]' cannot be assigned to -- it is read only
Try context.Request.Body.As<string>(). Method As currently supports following types as generic argument value:
byte[]
string
JToken
JObject
JArray
XNode
XElement
XDocument
Mind that if you try to call .As<JObject> over response that does not contain valid JSON you would get an exception, same applies to other types as well.