Defining Azure Stream Analytics iot-hub input source through Powershell - json

I'm trying to write a powershell script that creates a new streamAnalytics job in my azure portal account, with input source as iot-hub and output source as blob storage account.
To do so, I'm using AzureRM command new-streamAnalyticsJob, and json files.
my problem is: I have not seen any documentation or example for json file where the inputs source is iot-hub. only event-hub.
what are the parameters I need to give in the json file? can anyone display an example for json file with input source to streamAnalytics job as Iot-hub?

I got the answer eventually: the required field I had to add to the inputs Oliver posted earlier here is:
"endpoint":"messages/events"
I added it under Datasource Properties section, and it works fine!
Thanks Oliver

To come back on the error message you are seeing, to add to Olivier's sample you need a Property named endpoint which corresponds to the endpoint in IoT Hub. If you are looking for Telemetry messages this will be:
"endpoint": "messages/events"
This can be found in the schema for Azure ARM: https://github.com/Azure/azure-rest-api-specs/blob/current/specification/streamanalytics/resource-manager/Microsoft.StreamAnalytics/2016-03-01/examples/Input_Create_Stream_IoTHub_Avro.json
So to complete Olivier's example, when using API version '':
"Inputs": [
{
"Name": "Hub",
"Properties": {
"DataSource": {
"Properties": {
"consumerGroupName": "[variables('asaConsumerGroup')]",
"iotHubNamespace": "[parameters('iotHubName')]",
"sharedAccessPolicyKey": "[listkeys(variables('iotHubKeyResource'), variables('iotHubVersion')).primaryKey]",
"sharedAccessPolicyName": "[variables('iotHubKeyName')]",
"endpoint": "messages/events"
},
"Type": "Microsoft.Devices/IotHubs"
},
"Serialization": {
"Properties": {
"Encoding": "UTF8"
},
"Type": "Json"
},
"Type": "Stream"
}
}
],

That'd look like the following for the inputs part of the ASA resource:
"Inputs": [
{
"Name": "IoTHubStream",
"Properties": {
"DataSource": {
"Properties": {
"consumerGroupName": "[variables('CGName')]",
"iotHubNamespace": "[variables('iotHubName')]",
"sharedAccessPolicyKey": "[listkeys(variables('iotHubKeyResource'), variables('iotHubVersion')).primaryKey]",
"sharedAccessPolicyName": "[variables('iotHubKeyName')]"
},
"Type": "Microsoft.Devices/IotHubs"
},
"Serialization": {
"Properties": {
"Encoding": "UTF8"
},
"Type": "Json"
},
"Type": "Stream"
}
}
]

Related

JSON Schema Validation Error in MarkLogic - XDMP-VALIDATEERRORS

Using MarkLogic version 10.0-4.2, I am trying to validate a simple JSON record against a simple JSON schema.
JSON Schema:
{
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"properties": {
"SourceSystemName": {
"type": "string"
},
"BatchDtTm": {
"type": "string"
},
"SubjectArea": {
"type": "string"
},
"DocumentType": {
"type": "string"
},
"LastUpdatedDt": {
"type": "string"
},
"required": [
"SourceSystemName",
"BatchDtTm",
"SubjectArea",
"DocumentType",
"LastUpdatedDt",
]
}
}
Code being run in Query Console:
let jsonRecord = {"SourceSystemName":"ODH","BatchDtTm":"09/17/21 08:51:48:472723","SubjectArea":"Customer","DocumentType":"Preference","LastUpdatedDt":"09/17/21 03:59:53:629707"};
xdmp.jsonValidate(jsonRecord, cts.doc('/schemas/NewSchema.json').toString());
When I run the above code, I get error
XDMP-JSVALIDATEBADSCHEMA: Invalid schema "": ""
I'm not really sure what is 'invalid' about my schema. Can someone offer some insight into what MarkLogic is viewing as 'invalid'?
The second parameter for $schema is supposed to be the URI of the schema document.
$schema URI of the JSON schema to use for validation.
You are attempting to pass in the stringified content.
Try:
xdmp.jsonValidate(jsonRecord, '/schemas/NewSchema.json');
And ensure that the schema document is inserted into the Schemas database, not the content database.

How to create new alerts in azure for all webapps at same time

I want to create new alerts(High CPU,RAM) for all AppServicePlans in a given subscription. I could not find Powershell commands to create new alerts. Is there a way we can create these alerts for all appserviceplans with a single script? May be using ARM template?
Sorry for pasting a direct answer, but I cannot yet comment. You get the error because in the documentation for Add-AzMetricAlertruleV2 it states: "$act is the output of New-AzActionGroup cmdlet". Meaning you need to use for example:
$act = New-AzActionGroup -ActionGroupId "testActionGroup"
After that you need to add it in the parameter -ActionGroup $act for it to work.
If you look at Resource Explorer and navigate to a manually created (near-realtime) alert, you should see the "critera" object defined like below. Here is a full example of a resource that seems to be working. Create some variables for each of your values:
{
"type": "Microsoft.Insights/metricAlerts",
"apiVersion": "2018-03-01",
"name": "[variables('alertName')]",
"location": "global",
"dependsOn": [],
"tags": {
"[concat('hidden-link:', variables('applicationInsightsResourceId'))]": "Resource",
"[concat('hidden-link:', variables('webtestResourceId'))]": "Resource"
},
"properties": {
"description": "[concat('Alert for ', parameters('availibilityTestName'))]",
"severity": 4,
"enabled": true,
"scopes": [
"[variables('webtestResourceId')]",
"[variables('applicationInsightsResourceId')]"
],
"evaluationFrequency": "PT5M",
"windowSize": "PT15M",
"criteria": {
"odata.type": "Microsoft.Azure.Monitor.WebtestLocationAvailabilityCriteria",
"webTestId": "[variables('webtestResourceId')]",
"componentId": "[variables('applicationInsightsResourceId')]",
"failedLocationCount": 3
},
"actions": [
{
"actionGroupId": "[resourceId('microsoft.insights/actiongroups', 'webhook')]",
"webHookProperties": {
// Some properties to send to webhook
}
}
]
}
}

Multiple schema JSON validation in Golang

I need to validate several JSON files against a schema in Golang.
I have been able to achieve it by using gojsonschema, that is really a straight forward library.
However, the problem I'm facing right now is that I have been given with schemas that have dependencies to another schemas and haven't found the way to load all the schemas that I need. Therefore, my validations always fail.
This is my main schema:
{
"$schema": "http://json-schema.org/draft-04/schema#",
"$ref": "#/definitions/List",
"definitions": {
"List": {
"type": "array",
"items": {
"$ref": "#/definitions/Item"
}
},
"Item": {
"description": "An item ....",
"type": "object",
"additionalProperties": false,
"properties": {
"property01": {
"description": "The property01 code.",
"$ref": "./CommonTypes.json#/definitions/Type01Definition"
}
},
"required": [
"property01"
]
}
}
}
And, I have another one with common types:
{
"$schema": "http://json-schema.org/draft-04/schema#",
"definitions": {
"Type01Definition": {
"description": "The definition for the type 01",
"type": "string",
"pattern": "^[A-Z0-9]{3}$"
}
}
}
Is there a way to load several schemas using that library? Or is there any other Golang library that allows to achieve that?
The way to refer to a file using $ref is to specify the absolute path of the file using a URL scheme. If you change the $ref to look like "$ref" : "file:///home/user/directory/CommonTypes.json#/definitions/Type01Definition, your example will work as expected.
If you need a bit more flexibility you can either try gojsonschema's NewReferenceLoaderFilesystem or switch to a different Golang library https://github.com/santhosh-tekuri/jsonschema. That library allows you to add custom resources so you can load several schemas at once.

How to create an internal comment on a jira issue using the JIRA Cloud REST API

It's really hard to find a clear answer about how to create a comment on an issue which is internal only.
The JIRA Cloud REST API Documentation specifies the following schema for setting properties on comments when creating or updating a comment on an incident
https://docs.atlassian.com/jira/REST/cloud/#api/2/issue/{issueIdOrKey}/comment-addComment
"properties": {
"type": "array",
"items": {
"title": "Entity Property",
"type": "object",
"properties": {
"key": {
"type": "string"
},
"value": {}
},
"additionalProperties": false
}
}
To make a comment on an issue internal (meaning only service desk agents can see the comment) you need to set the sd.public.comment key to have the value { "internal": true }
Which can be achieved by passing the following JSON in the body of the create or update API request.
{
"properties": {
"key": "sd.public.comment",
"value": {
"internal": true
}
}
}
You will also need to set the Content-Type header on the request.
Content-Type: application/json
The following is an example of a creating an internal comment using a Groovy script - the scripting language used by ScriptRunner (a popular JIRA plugin)
post("/rest/api/2/issue/${issue.id}/comment")
.header("Content-Type", "application/json")
.body([
body: "This is the text which will appear in the comment",
properties: [
[key: "sd.public.comment", value: [ "internal": true ]]
]
]).asString()
Note that Object / JSON mapping will differ depending on which scripting language or HTTP Request framework you are using.

ARM Error: The Template Resource is not found using resource(), copyIndex()

I'm trying to conditionally provide resource property values through translation of runtime resource properties within a copyIndex loop..
Upon deploying the following ARM template, I receive the error:
Unable to process template language expressions for resource '/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/Microsoft.Compute/virtualMachines/{vm-name}/extensions/Microsoft.EnterpriseCloud.Monitoring' at line '30' and column '10'. 'The template resource '/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/Microsoft.Compute/virtualMachines/{vm-name}' is not found.' (Code: InvalidTemplate)
"type": "[variables('extensionType')[reference(concat('Microsoft.Compute/virtualMachines/', parameters('virtualMachines')[copyIndex()].name)).storageProfile.osDisk.osType]]",
However, the VM exists with the ID it provides, so it doesn't make sense that the engine cannot find it. If I hard-code the Extension Type, there are no errors and the Extension is installed on the VM with the same ID.
Unfortunately, I don't know if this is a bug within ARM or if I'm just doing something wrong..
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"workspaceResourceId": { "type": "string" },
"virtualMachines": { "type": "array" }
},
"variables": {
"extensionType": {
"Windows": "MicrosoftMonitoringAgent",
"Linux": "OmsAgentForLinux"
}
},
"resources": [
{
"copy": {
"name": "VMMonitoringExtensionsCopy",
"count": "[length(parameters('virtualMachines'))]"
},
"type": "Microsoft.Compute/virtualMachines/extensions",
"apiVersion": "2015-05-01-preview",
"location": "[parameters('virtualMachines')[copyIndex()].location]",
"name": "[concat(parameters('virtualMachines')[copyIndex()].name, '/Microsoft.EnterpriseCloud.Monitoring')]",
"properties": {
"publisher": "Microsoft.EnterpriseCloud.Monitoring",
"type": "[variables('extensionType')[reference(concat('Microsoft.Compute/virtualMachines/', parameters('virtualMachines')[copyIndex()].name)).storageProfile.osDisk.osType]]",
"typeHandlerVersion": "1.0",
"autoUpgradeMinorVersion": true,
"settings": {
"workspaceId": "[reference(parameters('workspaceResourceId'), '2015-11-01-preview').customerId]"
},
"protectedSettings": {
"workspaceKey": "[listKeys(parameters('workspaceResourceId'), '2015-11-01-preview').primarySharedKey]"
}
}
}
]
}
The object array being passed in for virtualMachines looks like this:
[
{ "name": "vm-name", "location": "azure-region" }
]
A couple things you can try:
1) Assuming the VM is not defined in the same template try using the "full" resourceId in the reference function. See the last example in this doc:
https://azure.microsoft.com/en-us/documentation/articles/resource-group-template-functions/#reference - it seems like the error already knows the full resourceId, but it's worth trying
2) the other thought is that the reference function is evaluated at runtime and the resource provider doesn't like the expression but that's a swag.
I will do some more poking and see if we can't nail this down.