Securely pass credentials to DSC Extension from ARM Template - json

According to https://learn.microsoft.com/en-gb/azure/virtual-machines/windows/extensions-dsc-template, the latest method for passing credentials from an ARM template to a DSC extension is by placing the whole credential within the configurationArguments of the protectedSettings section, as shown below:
"properties": {
"publisher": "Microsoft.Powershell",
"type": "DSC",
"typeHandlerVersion": "2.24",
"autoUpgradeMinorVersion": true,
"settings": {
"wmfVersion": "latest",
"configuration": {
"url": "[concat(parameters('_artifactsLocation'), '/', variables('artifactsProjectFolder'), '/', variables('dscArchiveFolder'), '/', variables('dscSitecoreInstallArchiveFileName'))]",
"script": "[variables('dscSitecoreInstallScriptName')]",
"function": "SitecoreInstall"
},
"configurationArguments": {
"nodeName": "[parameters('CMCD VMName')]",
"sitecorePackageUrl": "[concat(parameters('sitecorePackageLocation'), '/', parameters('sitecoreRelease'), '/', parameters('sitecorePackageFilename'))]",
"sitecorePackageUrlSasToken": "[parameters('sitecorePackageLocationSasToken')]",
"sitecoreLicense": "[concat(parameters('sitecorePackageLocation'), '/', parameters('sitecoreLicenseFilename'))]",
"domainName": "[parameters('domainName')]",
"joinOU": "[parameters('domainOrgUnit')]"
},
"configurationData": {
"url": "[concat(parameters('_artifactsLocation'), '/', variables('artifactsProjectFolder'), '/', variables('dscArchiveFolder'), '/', variables('dscSitecoreInstallConfigurationName'))]"
}
},
"protectedSettings": {
"configurationUrlSasToken": "[parameters('_artifactsLocationSasToken')]",
"configurationDataUrlSasToken": "[parameters('_artifactsLocationSasToken')]",
"configurationArguments": {
"domainJoinCredential": {
"userName": "[parameters('domainJoinUsername')]",
"password": "[parameters('domainJoinPassword')]"
}
}
}
}
Azure DSC is supposed to handle the encrypting/decrypting of the protectedSettings for me. This does appear to work, as I can see that the protectedSettings are encrypted within the settings file on the VM, however the operation ultimately fails with:
VM has reported a failure when processing extension 'dsc-sitecore-de
v-install'. Error message: "The DSC Extension received an incorrect input: Comp
ilation errors occurred while processing configuration 'SitecoreInstall'. Pleas
e review the errors reported in error stream and modify your configuration code
appropriately. System.InvalidOperationException error processing property 'Cre
dential' OF TYPE 'xComputer': Converting and storing encrypted passwords as pla
in text is not recommended. For more information on securing credentials in MOF
file, please refer to MSDN blog: http://go.microsoft.com/fwlink/?LinkId=393729
At C:\Packages\Plugins\Microsoft.Powershell.DSC\2.24.0.0\DSCWork\dsc-sitecore-d
ev-install.0\dsc-sitecore-dev-install.ps1:103 char:3
+ xComputer Converting and storing encrypted passwords as plain text is not r
ecommended. For more information on securing credentials in MOF file, please re
fer to MSDN blog: http://go.microsoft.com/fwlink/?LinkId=393729 Cannot find pat
h 'HKLM:\SOFTWARE\Microsoft\PowerShell\3\DSC' because it does not exist. Cannot
find path 'HKLM:\SOFTWARE\Microsoft\PowerShell\3\DSC' because it does not exis
t.
Another common error is to specify parameters of type PSCredential without an e
xplicit type. Please be sure to use a typed parameter in DSC Configuration, for
example:
configuration Example {
param([PSCredential] $UserAccount)
...
}.
Please correct the input and retry executing the extension.".
The only way that I can make it work is to add PsDscAllowPlainTextPassword = $true to my configurationData, but I thought I was using the protectedSettings section to avoid using plain text passwords...
Am I doing something wrong, or is it simply that my understanding is wrong?

Proper way of doing this:
"settings": {
"configuration": {
"url": "xxx",
"script": "xxx",
"function": "xx"
},
"configurationArguments": {
"param1": xxx,
"param2": xxx
etc...
}
},
"protectedSettings": {
"configurationArguments": {
"NameOfTheCredentialsParameter": {
"userName": "USERNAME",
"password": "PASSWORD!1"
}
}
}
this way you don't need PsDSCAllowPlainTextPassword = $true
Then you can receive the parameters in your Configuration with
Configuration MyConf
param (
[PSCredential] $NameOfTheCredentialsParameter
)
An use it in your resource
Registry DoNotOpenServerManagerAtLogon {
Ensure = "Present"
Key = "HKEY_CURRENT_USER\SOFTWARE\Microsoft\ServerManager"
ValueName = "DoNotOpenServerManagerAtLogon"
ValueData = 1
ValueType = REG_DWORD"
PsDscRunAsCredential = $NameOfTheCredentialsParameter
}

The fact that you still need to use the PsDSCAllowPlainTextPassword = $true is documented
Here is the quoted section:
However, currently you must tell PowerShell DSC it is okay for credentials to be outputted in plain text during node configuration MOF generation, because PowerShell DSC doesn’t know that Azure Automation will be encrypting the entire MOF file after its generation via a compilation job.
Based on the above, it seems that it is an order of operations issue. The MOF is generated and THEN encrypted.

Related

Does config.GetSection not work in Azure Functions? And what is the recommended alternative?

Azure Function with a complex (List of objects) configuration type is working locally (with that complex type in local.settings.json) but fails to read / create list of objects in Azure (with that complex type in Azure Function configuration settings). I'm looking for the recommended / optimal way to support that across both platforms / methods of access.
This works great in my local.settings.json where I use the configuration builder and pull data out like
var myList = config.GetSection("ConfigurationList").Get<List<MyType>>();
however this doesn't seem to work in Azure Functions?? Now I think that is because in local.settings.json it is a json file and looks like
"ConfigurationList" : [ { "Name": "A", "Value": 2 }, { "Name": "B", "Value": 3 }]
while in Azure Functions it is a setting "ConfigurationList" with the value
[ { "Name": "A", "Value": 2 }, { "Name": "B", "Value": 3 }]
(so there isn't really a "section" in Azure Functions?)
It seems like the "easy" solution to this is to just change the .json to be a quoted string and deserialize the string (and then it would work the same in both places); but that doesn't seem like it would be the "best" (or "recommended" solution)
i.e. something like
"ConfigurationList" : "[ { \"Name\": \"A\", \"Value\": 2 }, { \"Name\": \"B\", \"Value\": 3 }]"
var myList = (List<MyType>)JsonConvert.DeserializeObject(config["ConfigurationList"], typeof(List<MyType>));
Which isn't the worst; but makes the json a bit "not as nice" and doesn't "flow" across the two platforms ... if it is what I have to do, fine; but hoping for a more standard approach / recommendation
As I metioned in the comment, on local you can process local.settings.json as a json file, but when on azure, the value in configuration settings is environment variable. There is no section, it just string.
Please notice that only string values are allowed, and that anything nested will break. Learn how to use nest settings on azure web app(azure functon is based on azure app service sandbox, so it is the same.):
https://learn.microsoft.com/en-us/archive/blogs/waws/asp-net-core-settings-for-azure-app-service
For example, if this is the json structure:
{
"Parent": {
"ChildOne": "C1 from secrets.json",
"ChildTwo": "C2 from secrets.json"
}
}
Then in web app, you should save it like this:
(source: windows.net)
Not sure if you are looking something like this , it seems a list but if it is a simple JObject like
"ConfigurationList" : {
"Name": "A",
"Value": 2
}
Then you can declare ConfigurationList:Name , ConfigurationList:Value in the configuration settings of function app

ASP.NET Core 3 - Serilog how to configure Serilog.Sinks.Map in appsettings.json file?

I came across the Serilog.Sinks.Map addon today which will solve my challenge with routing specific log events to a specific sink interface. In my environment, I am writing to a log file as well as using the SQL interface. I only want certain logs to be written to the SQL Server though.
Reading the instructions on GitHub by the author, I can only see an example for implementing the LoggerConfiguration through C# in the Program.CS, but I am using the appsettings.json file and unsure what to change from the provided example to the required json format.
Example given by Serilog on GitHub:
Log.Logger = new LoggerConfiguration()
.WriteTo.Map("Name", "Other", (name, wt) => wt.File($"./logs/log-{name}.txt"))
.CreateLogger();
My current configuration: Note I haven't implemented the Sinks.Map in my code yet.
Program.CS File:
public static void Main(string[] args)
{
// Build a configuration system with the route of the app settings.json file.
// this is becuase we dont yet have dependancy injection available, that comes later.
var configuration = new ConfigurationBuilder()
.AddJsonFile("appsettings.json")
.Build();
Log.Logger = new LoggerConfiguration()
.ReadFrom.Configuration(configuration)
.CreateLogger();
var host = CreateHostBuilder(args).Build();
}
And here is my appsettings.json file. I want to be able configure sink name 'MSSqlServer' as the special route, then use the standard file appender sink for all the other general logging.
"AllowedHosts": "*",
"Serilog": {
"Using": [],
"MinumumLevel": {
"Default": "Information",
"Override": {
"Microsoft": "Warning",
"System": "Warning"
}
},
"Enrich": [ "FromLogContext", "WithMachineName", "WithProcessId", "WithThreadId" ],
"WriteTo": [
{ "Name": "Console" },
{
"Name": "File",
"Args": {
//"path": "C:\\NetCoreLogs\\log.txt", // Example path to Windows Drive.
"path": ".\\Logs\\logs.txt",
//"rollingInterval": "Day", // Not currently in use.
"rollOnFileSizeLimit": true,
//"retainedFileCountLimit": null, // Not currently in use.
"fileSizeLimitBytes": 10000000,
"outputTemplate": "{Timestamp:dd-MM-yyyy HH:mm:ss.fff G} {Message}{NewLine:1}{Exception:1}"
// *Template Notes*
// Timestamp 'G' means UTC Time
}
},
{
"Name": "MSSqlServer",
"Args": {
"connectionString": "DefaultConnection",
"schemaName": "EventLogging",
"tableName": "Logs",
"autoCreateSqlTable": true,
"restrictedToMinimumLevel": "Information",
"batchPostingLimit": 1000,
"period": "0.00:00:30"
}
}
//{
// "Name": "File",
// "Args": {
// "path": "C:\\NetCoreLogs\\log.json",
// "formatter": "Serilog.Formatting.Json.JsonFormatter, Serilog"
// }
//}
]
}
Lastly if i could squeeze in another quick question on the topic, when using the SQL sink interface, how do manage the automatic purging/deletion of the oldest events i.e. DB should only store max 1,000,000 events then automatically write over the oldest event first, thanks in advance
I believe it is currently impossible to configure the standard Map call in json, since it relies on a few types that have no serialization support right now, like Action<T1, T2>. I created an issue to discuss this in the repository itself:
Unable to configure default Map call in json? #22
However, there is a way to still get some functionality out of it in Json, by creating a custom extension method. In your particular case, it would be something like this:
public static class SerilogSinkConfigurationExtensions
{
public static LoggerConfiguration MapToFile(
this LoggerSinkConfiguration loggerSinkConfiguration,
string keyPropertyName,
string pathFormat,
string defaultKey)
{
return loggerSinkConfiguration.Map(
keyPropertyName,
defaultKey,
(key, config) => config.File(string.Format(pathFormat, key));
}
}
Then, on your json file, add a section like this:
"WriteTo": [
...
{
"Name": "MapToFile",
"Args": {
"KeyPropertyName": "Name",
"DefaultKey": "Other",
"PathFormat": "./logs/log-{0}.txt"
}
}
]
To have these customizations work properly, Serilog needs to understand that your assembly has these kinds of extensions, to load them during the parsing stage. As per the documentation, you either need to have these extensions on a *.Serilog.* assembly, or add the Using clause on the json:
// Assuming the extension method is inside the "Company.Domain.MyProject" dll
"Using": [ "Company.Domain.MyProject" ]
More information on these constraints here:
https://github.com/serilog/serilog-settings-configuration#using-section-and-auto-discovery-of-configuration-assemblies

How to create html table from blob json in Azure Logic App

Hi I would like to create html tabe in Azure Logic App.
I have data loaded from blob via Blob connector (Get blob content using path)
I used Compose connector based on answer in this post
But I get error -
Unable to process template language expressions in action 'Create_HTML_table' inputs at line '1' and column '1747': 'The template language function 'json' parameter is not valid. The provided value '[{"ServiceName":"routingsf","SubServiceName":"roadinfo/supportedmaps","ErrorType":"System.AggregateException","ErrorMessage":"One or more errors occurred. (Object reference not set to an instance of an object.)","Count":4} ]' cannot be parsed: 'Unexpected character encountered while parsing value: . Path '', line 0, position 0.'. Please see https://aka.ms/logicexpressions#json for usage details.'.
Code of Compose and Create HTML table connector is like :
"Compose": {
"inputs": "#base64ToString(body('Get_blob_content_using_path').$content)",
"runAfter": {
"Get_blob_content_using_path_2": [
"Succeeded"
]
},
"type": "Compose"
},
"Create_HTML_table": {
"inputs": {
"format": "HTML",
"from": "#json(outputs('Compose'))"
},
"runAfter": {
"Compose": [
"Succeeded"
]
},
"type": "Table"
},
Can you help?
Maybe you could try my way to create table. After get the content, use the Parse JSON , the content input json(body('Get_blob_content_using_path')), the Scheme choose Use sample payload to generate schema and just copy and paste your json file content.
Then create HTML table , the From choose the expression array(body('Parse_JSON')).
Here is the Logic flow and my result.
Hope this could help you, if you still have other questions, please let me know.
Update: I copy your json content tomy json file and test again. And It works.
So please make sure you logic app flow is right. Or you could share you flow.
Make sure these two steps are right:

How to use custom_data parameter in ARM template in Terraform?

I have an Azure ARM template that successfully bootstraps a VM from a file directory within an Azure Storage Account. I would like to get this working in Terraform, but I am really struggling getting it to work correctly.
Here is a working Azure ARM template that creates the VM and bootstraps it with files in an Azure storage account. The bootstrapping occurs by using the customData parameter.
"variables": {
"uniqueId": "[uniqueString(resourceGroup().id)]",
"customData": "[concat('storage-account=', parameters('STORAGE_ACCOUNT'), ',access-key=', parameters('ACCESS_KEY'), ',file-share=', parameters('FILE_SHARE'), ',share-directory=', parameters('SHARE_DIRECTORY'))]"
},
"resources": [
{
"apiVersion": "2016-04-30-preview",
"type": "Microsoft.Compute/virtualMachines",
"name": "MY-VM",
"location": "[resourceGroup().location]",
"properties": {
"hardwareProfile": {
"vmSize": "Standard_DS3_v2"
},
"osProfile": {
"computerName": "My-Computer-Name",
"adminUsername": "[parameters('Username')]",
"adminPassword": "[parameters('Password')]",
"customData": "[base64(variables('customData'))]"
}
}
}
Here is my non-working Terraform script that does not work when I try to do the same type of Bootstrapping.
resource "azurerm_virtual_machine" "MY-VM" {
name = "${var.vm_name}"
location = "${var.location}"
resource_group_name = "${azurerm_resource_group.rg.name}"
vm_size = "${var.vm_size}"
primary_network_interface_id = "${azurerm_network_interface.nic0.id}"
os_profile {
computer_name = "${var.vm_name}"
admin_username = "${var.adminuser}"
admin_password = "${var.adminuserpassword}"
custom_data = "${base64encode(join("", list("storage-account=", var.STORAGE_ACCOUNT, ",access-key=", var.ACCESS_KEY, ",file-share=", var.FILE_SHARE, ",share-directory=None")))}"
}
}
This is the error that I receive when I run it. If I do not use the custom_data field, the machine launches fine, but is not bootstrapped. I am out of ideas here..
azurerm_virtual_machine.MY-VM:
compute.VirtualMachinesClient#CreateOrUpdate: Failure sending
request: StatusCode=0 -- Original Error: autorest/azure: Service
returned an error. Status=400 Code="InvalidRequestFormat"
Message="Cannot parse the request." Details=[]
i dont think join works for strings? for your case you can just do
"storage-account=${var.STORAGE_ACCCOUNT},access-key=${var.ACCESS_KEY},file-share=${var.FILE_SHARE},share-directory=None"

$filter on schema extensions microsoft graph doesn't support 'contains'

I added a schema extension to users in my org, to keep track of training a user has taken. Since lists are not supported I am trying to store this as a comma separated string, as follows:
{
"id": "voctestextension",
"description": "voc test extension",
"targetTypes": ["User"],
"properties": [
{
"name": "trainings",
"type": "String"
}
]
}
Now, while trying to fetch the users who have taken training 'X' I am making the below call:
https://graph.microsoft.com/v1.0/users?$filter=contains(extrw7rtbc9_voctestextension/trainings, 'Azure'), $select=extrw7rtbc9_voctestextension,displayName
This doesn't give the correct response, but throws this error:
{
"error": {
"code": "Request_UnsupportedQuery",
"message": "Unsupported Query.",
"innerError": {
"request-id": "dc3fda19-6464-43d9-95ce-54a0567bf5a9",
"date": "2018-03-15T09:14:30"
}
}
}
From different forum answers, I understand that contains is not supported. Can you suggest a better way to track this info in the user's profile?
Contains is not supported. You need to use startswith or add multiple properties like training1, training2, training3.. and then use filter with ORs and EQs.
https://graph.microsoft.com/v1.0/users?$filter(extrw7rtbc9_voctestextension/trainign1 eq 'Azure' or extrw7rtbc9_voctestextension/trainign2 eq 'Azure')
Kepaas solution is working (almost) perfectly, but the "=" is missing after "$filter":
https://graph.microsoft.com/v1.0/users?$filter=(extrw7rtbc9_voctestextension/trainign1 eq 'Azure' or extrw7rtbc9_voctestextension/trainign2 eq 'Azure')