I have an APIM policy using choose that is going down an unexpected logic path.
How Do I Access Active Directory Users/Groups In An Azure APIM Policy?
How do I debug this?
Is there a verbose mode that adds extra info to some logs?
Is it possible to dump the contents of the context or context.User or context.User.Groups to a log file that I can read?
How would I do that and where would the log file be?
Currently you cannot really debug.
You may add <trace source="yourtraceidentifier">#(...your expression to trace goes here...)</trace> policy expressions which you would see
in the developer portal trace
on the URL emitted by the response HTTP header Ocp-Apim-Trace-Location: https://apimstoxnurcnsqhqwudvs35.blob.core.windows.net/apiinspectorcont... when you turn on the HTTP header Ocp-Apim-Trace: true
in the future this trace information should also be emitted to Application Insights telemetry as trace elements
correction August 2021
With the Visual Studio extension for API Management it is possible to debug policies : https://learn.microsoft.com/en-us/azure/api-management/api-management-debug-policies
Related
We are experiencing an issue with running TestCafe where it will consistently stall on an API call. The application outside of TestCafe will consistently run without issue. From what I can trace, it seems to be within the TestCafe Hammerhead proxy request-pipeline.
I can see the request is (pending) but we don't see the request actually reaching the API server logs. There is an auth token in the request and I can see that Hammerhead will manage the auth flow from the header prefix:
~~~TestCafe added this prefix to control the authorization flow~~~
We have other API calls with non-standard auth headers (x-ns-authorization). TestCafe/Hammerhead will not stall on these calls and there is no added prefix. With the request that stalls, we cannot modify the auth header.
These are the versions we're running:
Node 14.20.1
└─┬ testcafe#2.1.0
├── testcafe-hammerhead#28.1.0
Hammerhead becomes a black box (aside from source code) as I don't see a way to debug the proxy. Is there a way to debug the proxy server? Or is it possible to prevent TestCafe from controlling the auth flow?
Also to note: the full tests will run successfully about <10% of the time. So it's a bit random when the tests don't stall.
I'm working on an IoT project using AWS IoT and Greengrass v2 and I'm trying to integrate the ShadowManager component to use local shadows but when I deploy it on my device, it return a fatal exception during the synchronization step
{greengrass-root}/logs/greengrass.log
2021-09-15T09:54:29.044Z [INFO] (pool-2-thread-33) com.aws.greengrass.shadowmanager.sync.SyncHandler: sync. Executing sync request. {Type=LocalUpdateSyncRequest, thing name=mydevice, shadow name=}
2021-09-15T09:54:29.082Z [WARN] (pool-2-thread-33) com.aws.greengrass.shadowmanager.sync.SyncHandler: sync. Received conflict when processing request. Retrying as a full sync. {thing name=mydevice, shadow name=}
software.amazon.awssdk.aws.greengrass.model.ConflictError: Missed update(s) from the cloud
at com.aws.greengrass.shadowmanager.sync.model.LocalUpdateSyncRequest.execute(LocalUpdateSyncRequest.java:142)
at com.aws.greengrass.shadowmanager.sync.SyncHandler.lambda$new$0(SyncHandler.java:136)
at com.aws.greengrass.util.RetryUtils.runWithRetry(RetryUtils.java:49)
at com.aws.greengrass.shadowmanager.sync.SyncHandler.lambda$new$1(SyncHandler.java:134)
at com.aws.greengrass.shadowmanager.sync.SyncHandler.syncLoop(SyncHandler.java:270)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2021-09-15T09:54:29.083Z [INFO] (pool-2-thread-33) com.aws.greengrass.shadowmanager.sync.SyncHandler: sync. Executing sync request. {Type=FullShadowSyncRequest, thing name=mydevice, shadow name=}
2021-09-15T09:54:29.357Z [ERROR] (pool-2-thread-33) com.aws.greengrass.shadowmanager.sync.model.FullShadowSyncRequest: Could not execute cloud shadow get request. {thing name=mydevice, shadow name=}
2021-09-15T09:54:29.358Z [ERROR] (pool-2-thread-33) com.aws.greengrass.shadowmanager.sync.SyncHandler: sync. Skipping sync request. {thing name=mydevice, shadow name=}
com.aws.greengrass.shadowmanager.exception.SkipSyncRequestException: software.amazon.awssdk.services.iotdataplane.model.IotDataPlaneException: null (Service: IotDataPlane, Status Code: 403, Request ID: 84d49520-0162-7416-61a4-9973ecd32dad, Extended Request ID: null)
at com.aws.greengrass.shadowmanager.sync.model.FullShadowSyncRequest.getCloudShadowDocument(FullShadowSyncRequest.java:479)
at com.aws.greengrass.shadowmanager.sync.model.FullShadowSyncRequest.execute(FullShadowSyncRequest.java:93)
at com.aws.greengrass.shadowmanager.sync.SyncHandler.lambda$new$0(SyncHandler.java:136)
at com.aws.greengrass.util.RetryUtils.runWithRetry(RetryUtils.java:49)
at com.aws.greengrass.shadowmanager.sync.SyncHandler.lambda$new$1(SyncHandler.java:134)
at com.aws.greengrass.shadowmanager.sync.SyncHandler.syncLoop(SyncHandler.java:270)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: software.amazon.awssdk.services.iotdataplane.model.IotDataPlaneException: null (Service: IotDataPlane, Status Code: 403, Request ID: 84d49520-0162-7416-61a4-9973ecd32dad, Extended Request ID: null)
at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handleErrorResponse(CombinedResponseHandler.java:123)
at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handleResponse(CombinedResponseHandler.java:79)
at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handle(CombinedResponseHandler.java:59)
at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handle(CombinedResponseHandler.java:40)
at software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:40)
at software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:30)
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:73)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:42)
at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:77)
at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:39)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:50)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:36)
at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:64)
at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:34)
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:56)
at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:36)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.executeWithTimer(ApiCallTimeoutTrackingStage.java:80)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:60)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:42)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:48)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:31)
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:37)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:26)
at software.amazon.awssdk.core.internal.http.AmazonSyncHttpClient$RequestExecutionBuilderImpl.execute(AmazonSyncHttpClient.java:193)
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.invoke(BaseSyncClientHandler.java:133)
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.doExecute(BaseSyncClientHandler.java:159)
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.lambda$execute$1(BaseSyncClientHandler.java:112)
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.measureApiCallSuccess(BaseSyncClientHandler.java:167)
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:94)
at software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:45)
at software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.execute(AwsSyncClientHandler.java:55)
at software.amazon.awssdk.services.iotdataplane.DefaultIotDataPlaneClient.getThingShadow(DefaultIotDataPlaneClient.java:221)
at com.aws.greengrass.shadowmanager.sync.IotDataPlaneClientWrapper.getThingShadow(IotDataPlaneClientWrapper.java:89)
at com.aws.greengrass.shadowmanager.sync.model.FullShadowSyncRequest.getCloudShadowDocument(FullShadowSyncRequest.java:458)
... 10 more
It seems like the ShadowManager has not the good access rights but I checked multiple time and I have well added the good policies:
iot:GetThingShadow
iot:UpdateThingShadow
iot:DeleteThingShadow
I've tested with classic shadow and named shadow but same result.
The configuration of my ShadowManager is:
{
"synchronize": {
"coreThing": {
"classic": true,
"namedShadow": ["mydevice"]
}
}
}
It's also important to know that it worked the first time I've deployed it but after several minutes and some update messages published it failed (without changing anything).
Someone could help me on this?
Thank you
I was having a similar issue but with a client device.
This device was previously added to an old ggc v1. I migrated it to the ggc v2 but it seems that some config remained on the old server and it was overwriting my shadow.
So while testing with the help of the MQTT test I would create a shadow, ggc v2 would detect a conflict and then it will do a full sync deleting the shadow I had sent.
Using a completely new device avoided this error. I am still not able to sync sending the data from the client but at least from the MQTT test I can.
It looks like you are getting a 403 during a full sync operation when it tries to get the cloud shadow from IoT Device Shadow service. This 403 indicates that the device doesn't have permission to get device shadows. These permissions are required on the core device's AWS IoT policy. To confirm the correct permissions, please see the documentation about the minimal AWS IoT policy for core devices.
I was having the same issue and going crazy triple-checking configuration and IAM policies. Michael's answer will lead you to the right place, but the important thing to realize is that operations through the ShadowManager use AWS IoT Policies rather than IAM policies through the TokenExchangeRole.
From Device authentication and authorization for AWS IoT Greengrass
AWS IoT policies define the set of operations allowed for AWS IoT devices. Specifically, they allow and deny access to AWS IoT Core and AWS IoT Greengrass data plane operations, such as publishing MQTT messages and retrieving device shadows.
So, interactions with Device Shadows through the ShadowManager are IoT data plane operations which are checked against IoT Policies associated with the certificate identifying your Greengrass Core device. The TokenExchangeRole is not used for operations over the IoT data plane, meaning you don't need an IAM policy with permissions for shadow operations.
The previously linked document explains how to update an IoT Policy.
I am setting up automated deployment pipeline for my website ,as part of it i have to automate Api import using VSTS RM . I have achieved this using custom PS scripts in VSTS tasks. I have used swagger url to import
i.e
Import-AzureRmApiManagementApi –Context $apimContext –SpecificationFormat 'swagger' –SpecificationUrl 'http://mywebapp.com/swagger/docs/v1' –Path 'apis'
To Improve the security we have implemented to redirect the http request https which is secured by client certificate. Here comes the problem.
Now we are not able to use above command to import which is returning 403 forbidden error as API manager don't have option to bypass certificate validation. what can be done to solve this ?
Even i have tried to invoke-webrequest the url with specific cert and to import the API which worked fine in my local machine.
$swaggerurl="https://mywebapp.org/swagger/docs/1"
$cert=New-Object System.Security.Cryptography.X509Certificates.X509Certificate2("xyz.cer")
$test=Invoke-WebRequest -Uri "$swaggerurl" -Certificate $cert
$test.statuscode
Import-AzureRmApiManagementApi –Context $apimContext –SpecificationFormat 'swagger' –SpecificationUrl 'http://mywebapp.com/swagger/docs/v1' –Path 'apis'
But not in VSTS Inline power shell. It is not accepting certificate value and returning 403 forbidden error.
Please let me know how to resolve this ?
Issue :
1.Unable to import to API manager due to https client certificate validation on swagger url.
VSTS inline power shell not accepting certificate parameter details even though i use the right cert. Same case works in local machine. is there any limitation in VSTS inline power shell.
Thanks in advance.
Using Azure PowerShell step/task instead. (Include in Deploy category)
For this to work, API Management service needs to make the WebRequest on your behalf to the url, with a ClientCertificate in the Request.
We currently don't provide that option to call Import-AzureRmApiManagementApi with a Client Certificate.
Only available option is to make WebRequest using Powershell (Invoke-RestMethod), download the swagger to a local file and use the -SpecificationPath parameter in the cmdlet
This issue has been resolved by changing the private agent to run as admin account. Now Everything works as expected :)
Import of API using VSTS private agent
I am trying to enable push notifications on my website using VAPID keys.
When i include the gcm_sender_id and remove the applicationServerKey from the pushManager.subscribe method, it runs fine.
Only when i enable VAPID keys and remove the gcm_sender_id from manifest.json file. i get the foloowing error.
DOMException: Registration failed - push service error
I am using Chrome browser.
I encountered this error in Brave browser. By default, Google Services for push messaging are disabled in Brave. To enable this, open the following URL in brave:
brave://settings/privacy
After this, enable the flag "Use Google services for push messaging":
Source:
https://github.com/firebase/firebase-js-sdk/issues/3195#issuecomment-848036637
The applicationServerKey that i was using in the pushManager.subscribe method was somehow incorrect.
It worked when i regenerated the keys in node using the following module.
const webpush = require('web-push');
const vapidKeys = webpush.generateVAPIDKeys()
In my case,I was trying to run firebase messaging on a flutter web.
My Browser was BRAVE.
It always failed with an exception of firebase fcm registration push servic error.
I followed #Nicodemuz answer, but it didn't solve the issue. I get the same error.
The only solution was setting Google chrome as my executable.
Anyhow the issue is not with firebase or flutter, it's with the brave browser itself.
We are having problems with the exporting of the CEP definition file from the authoring tool to an (external) repository.
In the response preview from the developer tools of the browser we get the following error message:
"HTTP Status 500 - A javax.ws.rs.ext.MessageBodyReader implementation was not found for class org.apache.wink.json4j.JSONArray type and text/html;charset=utf-8 media type. Verify that all entity providers are correctly registered. Add a custom javax.ws.rs.ext.MessageBodyReader provider to handle the type and media type if a JAX-RS entity provider does not currently exist."
How can we make sure we are able to export to an external repository?
Your problem is that the external repository is not available.
Since you didn't mention this, my guess is that you are using the default external repository which is http://localhost:8080/ProtonOnWebServerAdmin/resources/definitions
but don't have a running instance of ProtonOnWebServerAdmin. You have got to have ProtonOnWebServerAdmin running on a Tomcat server on your local machine for it to actually process the request.
If you're using anything else - make sure that repository knows how to handle the request.