I have a Debezium connector correctly listening to an outbox table and providing a JSON object as the Kafka message value.
I have read the documentation and tried various things but can't find the second part of what I need.
If I have a json object as my message can the connector filter the message such that only a specific attribute or set of attributes are included.
For example
{
"FirstObject": {
"name": "an object name"
},
"SecondObject": {
"legs": 12
}
}
Currently I receive the complete json, is there a way to say only publish SecondObject in the kafka message?
I Am using Azure pipelines to automate the Log query based alerts. I am passing runtime parameter values to Azure variable first and then replacing the parameter.json file by the query by using the replacing token task in the pipeline. When I am passing the queries which are not ending with double quotes", the ResourceGroup deployment task getting succeeded. But when I am passing a query which is ending already with Double quotes getting failed.
Eg:
This is my base query.
"ApiManagementGatewayLogs
| where ApiId == ""my-api""
| where ResponseCode == 429
| where _SubscriptionId==""xxxxxxxxxxxxxxxxxxxxxxx"""
since my runtime parameter is type of "string", passing this as single line as below
ApiManagementGatewayLogs| where ApiId == ""my-api""| where ResponseCode == 429| where _SubscriptionId==""xxxxxxxxxxxxxxxxxxxxxxx""
But the deployments getting failed with below error
Template deployment validation was completed successfully.
Starting Deployment.
Deployment name is digitalAlerts
There were errors in your deployment. Error code: DeploymentFailed.
##[error]At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.
##[error]Details:
##[error]BadRequest: {
"error": {
"message": "The request had some invalid properties",
"code": "BadArgumentError",
"correlationId": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
"innererror": {
"code": "SyntaxError",
"message": "Request is invalid and cannot be processed: Syntax error: SYN0001: I could not parse that, sorry. [line:position=0:0]. Query: 'let ['ApiManagementGatewayLogs'] = view () { datatable(['TenantId']:string,['TimeGenerated']:datetime,['OperationName']:string,['CorrelationId']:string,['Region']:string,['IsRequestSuccess']:bool,['Category']:string,['TotalTime']:long,['CallerIpAddress']:string,['Method']:string,['Url']:string,['ClientProtocol']:string,['ResponseCode']:int,['BackendMethod']:string,['BackendUrl']:string,['BackendResponseCode']:int,['BackendProtocol']:string,['RequestSize']:int,['ResponseSize']:int,['Cache']:string,['CacheTime']:long,['BackendTime']:long,['ClientTime']:long,['ApiId']:string,['OperationId']:string,['ProductId']:string,['UserId']:string,['ApimSubscriptionId']:string,['BackendId']:string,['LastErrorElapsed']:long,['LastErrorSource']:string,['LastErrorScope']:string,['LastErrorSection']:string,['LastErrorReason']:string,['LastErrorMessage']:string,['ApiRevision']:string,['ClientTlsVersion']:string,['RequestHeaders']:dynamic,['ResponseHeaders']:dynamic,['BackendRequestHeaders']:dynamic,['BackendResponseHeaders']:dynamic,['RequestBody']:string,['ResponseBody']:string,['BackendRequestBody']:string,['BackendResponseBody']:string,['Errors']:dynamic,['TraceRecords']:dynamic,['SourceSystem']:string,['Type']:string,['_ResourceId']:string,['_SubscriptionId']:string)[] };restrict access to (*);\r\nApiManagementGatewayLogs\\n| where ApiId == \\\"my-api\\\"\\n| where ResponseCode == 429\\n| where _SubscriptionId==\\\"xxxxxxxxxxxxxxxxxxxxxxxx\\\"\\n\\n'"
}
}
}
##[error]Check out the troubleshooting guide to see if your issue is addressed: https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/deploy/azure-resource-group-deployment?view=azure-devops#troubleshooting
##[error]Task failed while creating or updating the template deployment.
Looking for 2 solutions:
From Pipeline side How Can I pass the queries in the same format as my Base query where users can easily pass their queries , which will replace my parameters.json in proper format
Secondly, how can avoid the above issue by passing the queries with double quotes ending.
Note already tried by modifying the query by replacing the " with / but didn't resolve the issue
It's not completely clear from the question but it sounds like you're deploying something like a scheduled query rule using an ARM template via an Azure DevOps pipeline.
As the ARM template is a json document any strings will need to be wrapped in double quotes. However the log query you are passing into is written in Kusto and as per the docs strings in Kusto queries can be wrapped in single or double quotes.
If you rewrite your query as:
ApiManagementGatewayLogs | where ApiId == 'my-api'| where ResponseCode == 429 | where _SubscriptionId=='xxxxxxxxxxxxxxxxxxxxxxx'
it should succeed. In your parameters.json file this would look something like:
{
"$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"logQuery": {
"value": "ApiManagementGatewayLogs | where ApiId == 'my-api'| where ResponseCode == 429 | where _SubscriptionId=='xxxxxxxxxxxxxxxxxxxxxxx'"
}
}
}
I'm trying to create an alert policy with Cloud Functions in python. I have the following very simple alert policy:
alert_policy = {
'combiner': 'OR',
'conditions': [
{
'conditionAbsent': {
'duration': '3900s',
'filter': 'resource.type = "l7_lb_rule" AND metric.type = "logging.googleapis.com/user/name_stuff_here"'
}
}
]
}
When running the function I have the following error: Protocol message Condition has no "conditionAbsent" field error
How should I write this alert Policy? I also had some error because of Displayname field which I've removed entirely. Is there an alert policy builder or something like that where I can validate alerts json?
Looking more closely at the documentation I found that conditionAbsent should be passed as condition_absent
I'm using postman to validate the schema of json data returned from an api.
I have a test that runs through basic http validation, then ends with:
if (tv4.error){
console.log("Validation failed: ", tv4.error);
}
The error I get back is difficult to fathom.
Validation failed: 12:22:41.316
Object:{}
message:"Invalid type:
number (expected string)"
name:"ValidationError"
type:"Error"
But I need to know which field the validation failed on. How can I get this info? The npm page for tv4 suggests that the error message should be more descriptive.
According to the documentation of tv4, you can print the path of the error location using console.log(tv4.error.dataPath), I have no idea why this attribute is not logged in the console.
Documentation is here.
The relevant section in the documentation is:
If validation returns false, then an explanation of why validation failed can be found in tv4.error.
The error object will look something like:
{
"code": 0,
"message": "Invalid type: string",
"dataPath": "/intKey",
"schemaPath": "/properties/intKey/type"
}
I am sending to stackdriver the following json trying to annotate an aws instance:
{ "message": "instance impaired", "annotated_by": "Ops User", "level": "WARN", "instance_id": "i-xxxxxxxxxxxxxxxx", "event_epoch": 1484903331 }
It returns the error:
HTTP 400: Bad Request (CustomMetric data payloads must either be a list or a dict. You sent: <type 'NoneType'>)
What is wrong with my json? What does a "dict" mean in stackdriver language. I couldn't find anything in the web about that error.
It turned out I was using a wrong endpoint. Now it is working as expected.