The goal is to use the az iot edge deployment update command to change a module in an azure iot hub/edge deployment. The attempt to do this uses the property-path within the deployment configuration json to replace the image path. The problem is that there is a dot in a json property properties.desired and attempts of escaping it have been futile. The file is a default azure deployment configuration file.
Command format
az iot edge deployment update --deployment-id <name-of-deployment> --hub-name <name-of-iot-hub> --set <json-path>=<new-value>
First part of the deployment configuration (json)
The goal is to change the value of image
{
"content": {
"modulesContent": {
"$edgeAgent": {
"properties.desired": {
"modules": {
"demoimage1-latest": {
"settings": {
"image": "demoworkspac2478a907.azurecr.io/demoimage1:6",
The most obvious attempt
az iot edge deployment update --deployment-id demoimage1-6 --hub-name iot-hubski --set content.modulesContent.'$edgeAgent'.'properties.desired'.modules.'demoimage1-latest'.settings.image=demoworkspac2478a907.azurecr.io/demoimage1:5
Gives
Couldn't find 'properties' in 'content.modulesContent.$edgeAgent.properties.desired.modules.demoimage1-latest'. Available options: ['properties.desired']
Status
Many things have been tried using both bash (ubuntu LTS vm) and powershell (win10)
[properties.desired]
'[properties.desired]'
['properties.desired']
properties\.desired
properties.desired`
properties.desired
'..."properties.desired"...'
'...\"properties.desired\"...'
'$edgeAgent'[properties.desired]
'$edgeAgent'['properties.desired']
^[properties.desired^]
^^[properties.desired^^]
``[properties.desired]
```[properties.desired``]`
you need to manually strinigying the $edgeHub JSON.
az iot edge deployment update --deployment-id testedge --hub-name microwaves --set content.modulesContent.'$edgeHub'="{'properties.desired': {'routes': {'route': 'FROM /messages/* INTO $upstream'},'schemaVersion': '1.0','storeAndForwardConfiguration': {'timeToLiveSecs': 7201}}}"
However it doesn't do anything because of content being immutable. Items that can be updated by az iot edge deployment update command: labels, metrics, priority and targetCondition. labels and metrics do not allow values with ‘.’ in the name.
Related
I'm developing an Azure Function which has to consume JSON as input and then trigger a hybrid CI/CD pipeline split between on-prem and Azure DevOps. To split configuration from code I intend to use an Azure App Configuration store to retrieve configuration settings that the Function will use to trigger the correct pipeline depending on JSON input. I'm completely new to App Config but have tried to investigate how to properly use it. However, I have stumbled into a perplexing issue and can't find an explanation for it. I apologize if I have missed something obvious out there.
For the purpose of this question I have abstracted away any business-related terminology.
Imagine I have a JSON object stored in a file TestStructure.json that looks like this:
{
"TestStructure": {
"Repository1": {
"RepositoryName": "Repository1",
"RepositoryUrl": "https://url.repository1.com/"
},
"Repository2": {
"RepositoryName": "Repository2",
"RepositoryUrl": "https://url.repository2.com/"
},
"Repository3": {
"RepositoryName": "Repository3",
"RepositoryUrl": "https://url.repository3.com/"
}
}
}
I store this in my App Config using the Azure CLI with the following command:
az appconfig kv import -n <myAppConfigName> -s file --format json --path "C:\workspace\TestStructure.json" --content-type "application/json" --separator . --depth 2
The command yields the following key-value pairings:
---------------- Key Values Preview ----------------
Adding:
{"key": "TestStructure.Repository1", "value": "{\"RepositoryName\": \"Repository1\", \"RepositoryUrl\": \"https://url.repository1.com/\"}"}
{"key": "TestStructure.Repository2", "value": "{\"RepositoryName\": \"Repository2\", \"RepositoryUrl\": \"https://url.repository2.com/\"}"}
{"key": "TestStructure.Repository3", "value": "{\"RepositoryName\": \"Repository3\", \"RepositoryUrl\": \"https://url.repository3.com/\"}"}
These keys are what I expect to find in my App Config store.
Going to the App Config in the Azure Portal I find that the JSON object has been stored correctly, i.e. the keys are TestStructure.Repository1, TestStructure.Repository2 and so forth, all with their corresponding values as the Azure CLI command reported back. This screenshot verifies it:
Now, to the actual problem. When I try to fetch a key from my App Config I get some weird behavior.
I have put together a simple Console App in .NET 6 to test how to read from the App Config:
1 using Microsoft.Extensions.Configuration;
2
3 var config = new ConfigurationBuilder()
4 .AddAzureAppConfiguration("MyConnectionString")
5 .Build();
6
7 var repository = config["TestStructure.Repository1"] // Returns null
It doesn't make sense to me why line 7 returns null, so I attached a debugger to inspect the ConfigurationRoot object a bit further and found the following:
What is going on here? Inspecting the config object reveals that the actual keys to index with are stored as TestStructure.Repository1:RepositoryName and not TestStructure.Repository1 and then the corresponding values.
Thank you for taking your time to read my question. I hope I have expressed clearly what I am trying to achieve and what my problem is.
Is there a way to reference a VM name that is deployed in VM Scale Set? I'm using custom script extension that runs install script when each VM in scale set is deployed and for one of the parameters of the script I want to use a name of the VM. For single instance it was easy:
"commandToExecute": "[concat('sh ap-cluster-setup.sh -h=',parameters('virtualMachineName'),' -s=',parameters('subnetAddressPrefix'),'')]"
but since VM name in scale set is created dynamically when it is deployed I can't think of a way to reference it in JSON. The VM naming convention is vmssTemplate_0, vmssTemplate_1, etc. where vmssTemplate is parameters('virtualMachineScaleSets_name') in VMSS template.
There's no way to do this in the template itself since the same parameters get passed to each VM on script execution. However, You should be able to figure out the VM name from within the script. For instance, on a Linux machine you could use the hostname command, which will return the VM name. Does this meet your needs?
where vmssTemplate is parameters('virtualMachineScaleSets_name') in
VMSS template.
We can find where vmss template define the Virtual machine's name.
computernameprefix:
"osProfile": {
"computerNamePrefix": "[variables('namingInfix')]",
"adminUsername": "[parameters('adminUsername')]",
"adminPassword": "[parameters('adminPassword')]"
},
and the 'namingInfix' define here:
"namingInfix": "[toLower(substring(concat(parameters('vmssName'), uniqueString(resourceGroup().id)), 1, 9))]",
By default the VMSS with autoscale settings, and the vm's name should be create automatically.
If you want to get the VMs' name, we can use powershell to get it:
PS C:\windows> Get-AzureRmVmssVM -ResourceGroupName YOUR_RESOURCE_NAME -VMScaleSetName YOUR_VMSS_NAME
Also you can add some script (to get the name of your VMs) in your script, then upload the script to custom script extension.
I want to migrate an existing job to Jenkins Pipeline and this process I try to migrate the different plugins attached to this job to use the new syntax.
However, not all plugins provide a corresponding wrapper syntax.
In the current case, I want to allocate three separate ports (embedded DB, container, and process engine) so that the build can run independently from other builds on the same machine. In the classic Jenkins job, we could use the Port Allocator Plugin, but it's not (yet) available via Pipeline Syntax.
My idea is to trigger a classic build that uses the port allocator plugin and returns the free ports that I can then use in a later build to start up the required services.
node {
stage("Allocate Ports") {
build job: "allocate-port", parameters: [ // <- the classic build
string(name: "name1", value: "PORT_DB"),
string(name: "name2", value: "PORT_CONTAINER"),
string(name: "name3", value: "PORT_ENGINE")
]
}
stage("Integration Tests") {
sh """
run-test \
-db=${PORT_DB} \
-container=${PORT_CONTAINER} \
-engine=${PORT_ENGINE}
"""
}
}
Is there a good way to store the results from the allocate-port build and return it to the enclosing pipeline?
I have installed node-push-server. The configuration is loaded from a json like this:
{
"webPort": 8000,
"mongodbUrl": "mongodb://username:password#localhost/database",
"gcm": {
"apiKey": "YOUR_API_KEY_HERE"
},
"apn": {
"connection": {
"gateway": "gateway.sandbox.push.apple.com",
"cert": "/path/to/cert.pem",
"key": "/path/to/key.pem"
},
"feedback": {
"address": "feedback.sandbox.push.apple.com",
"cert": "/path/to/cert.pem",
"key": "/path/to/key.pem",
"interval": 43200,
"batchFeedback": true
}
}
}
How can I set the enviroment variables for my application in this json file?
I don't think it's possible. You should be able to change all these settings in the code though. For example in node you can do: process.env.OPENSHIFT_VARIABLENAME to read an environment variable.
Example for MongoDB connection string from docs:
//provide a sensible default for local development
mongodb_connection_string = 'mongodb://127.0.0.1:27017/' + db_name;
//take advantage of openshift env vars when available:
if(process.env.OPENSHIFT_MONGODB_DB_URL){
mongodb_connection_string = process.env.OPENSHIFT_MONGODB_DB_URL + db_name;
}
As an alternative, there is a quick and easy deployable gear called AeroGear Push that might serve your needs.
Config files can be awkward because including them in your source repo isn't always a good move.
OpenShift deployments are mostly git push-driven, so there are several options for helping you correctly resolve your configs on the server.
Configuring your service using ENV vars is the most common approach, but since this one requires a flat file, you'll need to find a way to update the file with the correct values.
If you know what keys and values are needed, you should be able to write a script that updates the example json, or merges two json objects to produce a flat config file including the strings node-pushserver will expect.
It looks like mongodbUrl, webPort, (and domain?) would need to be populated with OpenShift-provided values (when available). config-multipaas might be able to help with that.
I would probably implement the config bootstrapping / merging work as a build step, allowing you to prep the config file and start node-pushserver in it's usual way
System : Running VMPlayer on Windows Server. One VM is playing image of IBM Infosphere Streams (3.2) QuickStart Edition and Other MessageSight(1.1) Virtual Appliance.
When using MQTTSource Operator I get following compilation error
1. "make: *** No rule to make target `/home/streamsadmin/sdk/clients/c/include/MQTTAsync.h', needed by `build/operator/mqttStream.o'. Stop.
2. CDISP0141E ERROR: The compilation of the generated code failed."
The sdk path is the SDK of IBM MessageSight. I am trying to connetc streams with MessageSight using MQTT operator. Is the problem with SDK or with my code? Please find the code below.
composite MQTTtestApp {
graph
(stream<blob demoData> mqttStream;stream<rstring errorMessage> myErrorStream) = MQTTSource()
{
param
serverURI : "192.168.206.130:1883";
topics : ["DemoMessagingPolicy"];
format: block;
output
myErrorStream : errorMessage = getError();
}
stream<rstring dataSchema> ParsedMsg = Parse(mqttStream)
{
param
format :csv;
}
() as myMessageSink = FileSink(ParsedMsg)
{
param
file : "data.csv";
format : csv;
}
In this version of the operator, you will need to do the following to get the code to compile:
1) Download the MQTT client and have it installed on the VM. See this link for details: http://www-01.ibm.com/support/knowledgecenter/SSCRJU_3.2.0/com.ibm.swg.im.infosphere.streams.messaging-toolkit.doc/doc/msgtoolkit-reqs.html?lang=en
2) Set he STREAMS_MESSAGING_MQTT_HOME environment variable. This environment should point to the install location of the MQTT client
There is a newer version of the MQTT operators that is easier to set up. In the new version, the MQTT client is included as part of the package. Therefore, you will no longer need to install the client separately, or set the environment variable.
See this project for details.
http://ibmstreams.github.io/streamsx.messaging/